Lower extremity non-contact soft tissue (LE-ST) injuries are prevalent in elite futsal. The purpose of this study was to develop robust screening models based on pre-season measures obtained from questionnaires and field-based tests to prospectively predict LE-ST injuries after having applied a range of supervised Machine Learning techniques. One hundred and thirty-nine elite futsal players underwent a pre-season screening evaluation that included individual characteristics; measures related to sleep quality, athlete burnout, psychological characteristics related to sport performance and self-reported perception of chronic ankle instability. A number of neuromuscular performance measures obtained through three field-based tests  were also recorded. Injury incidence was monitored over one competitive season. There were 25 LE-ST injuries. Only those groups of measures from two of the field-based tests , as independent data sets, were able to build robust models [area under the receiver operating characteristic curve (AUC) score ≥0.7] to identify elite futsal players at risk of sustaining a LE-ST injury. Unlike the measures obtained from the five questionnaires selected, the neuromuscular performance measures did build robust prediction models (AUC score ≥0.7). The inclusion in the same data set of the measures recorded from all the questionnaires and field-based tests did not result in models with significantly higher performance scores. The model generated by the UnderBagging technique with a cost-sensitive SMO as the base classifier and using only four ROM measures reported the best prediction performance scores . The models developed might help coaches, physical trainers and medical practitioners in the decision-making process for injury prevention in futsal. Lower extremity non-contact soft tissue  (LE-ST) injuries are very common events in intermittent team sports such as soccer , futsal Perhaps the lack of available valid screening models to predict LE-ST injuries could be attributed to the use of statistical techniques  that have not been specifically designed to deal with class imbalance problems, such as the LE-ST injury phenomenon, in which the number of injured players (minority class) prospectively reported is always much lower than the non-injured players (majority class) . Thus, iIn Machine Learning and Data Mining environments, some methodologies  have been specially designed to deal with complex , multifactorial and class imbalanced scenarios . These cHowever, one of the main limitations of most of these models built by the application of modern Machine Learning techniques lies in the fact that their use seems to be restricted to research settings (and not to applied environments) because sophisticated and expensive instruments , qualified technicians and time-consuming testing procedures are required to collect such data. To the authors’ knowledge, there is only one study that has built a robust screening model using Machine Learning techniques  with data from field-based tests. If Machine Learning techniques could build “user friendly” models with adequate predictive properties and exclusively using data obtained from questionnaires and/or cost-effective, technically undemanding, and time-efficient field-based tests, then injury prediction would not be a waste of time and resource in applied settings. In case these techniques provided a trustworthy positive response, coaches, physical trainers, and medical practitioners may know whether any of the currently available questionnaires and field-based tests to predict injuries itself works and a hierarchical rank could be developed based on their individual predictive ability of those that showed reasonably high AUC, true positive (TP), and true negative (TN) scores. Furthermore, this knowledge might be used to analyze the cost-benefit  of including measures in the screening sessions for injury prediction.Therefore, the main purpose of this study was to develop robust screening models based on pre-season measures obtained from different questionnaires and field-based tests to prospectively predict LE-ST injuries after having applied supervise Machine Learning techniques in elite male and female futsal players.To conduct this study, guidelines for reporting prediction model and validation studies in Health Research [Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (the TRIPOD statement)] were followed . The TRIA convenience sample of 139 [72  males and 67  females] elite futsal players from 12 different teams [56 players  from six club engaged in the First (top) National Spanish Futsal division and 83 players  from six clubs engaged in the Second National Futsal division] completed this study. Elite futsal players were selected in this study because a recent published meta-analysis on injury epidemiology reported that this sport present high incidence rates of injuries   and hencTo be included in this study, all players had to be free of pain at the time of the study and currently involved in futsal-related activities. Players were excluded if: (a) they reported the presence of orthopedic problems that prevented the proper execution of one or more of the neuromuscular performance tests or (b) were transferred to another club and were not available for follow up testing at the end of 9 months. Only first injuries were used for any player sustaining multiple LE-ST injuries. The study was conducted at the end of the pre-season phase in 2015 (39 players from four teams), 2016 (44 players from four teams), 2017 (30 players from three teams), and 2018 (26 players from two teams) (September). Before any participation, experimental procedures and potential risks were fully explained to the players and coaches in verbal and written form and written informed consent was obtained from players. An Institutional Research Ethics committee approved the study protocol prior to data collection (DPS.FAR.01.14) conforming to the recommendations of the Declaration of Frontera.A prospective cohort design was used to address the purpose of this study. In particular, all LE-ST injuries accounted for within the 9 months following the initial testing session (in-season phase) were prospectively collected for all players.Players underwent a pre-season evaluation of a number of personal, psychological, self-perceived chronic ankle instability and neuromuscular performance measurements, most of them considered potential sport-related injury risk factors. In each futsal team, the testing session was conducted at the end of the pre-season phase or beginning (within the first 3 weeks) of the in-season phase of the year. The testing session was divided into three different parts. The first part of the testing session was used to obtain information related to the participants’ personal or individual characteristics. The second part was designed to assess psychological measures related to sleep quality, athlete burnout and psychological characteristics related to sport performance. The subjective perception of each player regarding his/her chronic ankle joints instability was also recorded in this second part. Finally, the third part of the session was used to assess a number of neuromuscular performance measures through three field-based tests. Each of the four testers who took part in this study had more than 6 years of experience in athletes’ screening assessment.ad hoc questionnaire designed by The The Spanish version of the Karolinska Sleep Diary  was usedThe subjective perception of chronic ankle instability was measured using the Cumberland Ankle Instability Tool (CAIT). The final score was discretized into three categories of severity following the thresholds suggested by Prior to the neuromuscular risk factor assessment, all participants performed the dynamic warm-up designed by Neuromuscular capability was determined from two different performance field-based tests: (1) isometric hip abduction and adduction strength test  and (2) For a matter of space, the testing maneuvers are not described below, and the reader is to refer to their original sources. Furthermore, The order of the tests was consistent for all participants and was established with the intention of minimizing any possible negative influence among variables. A 5 min rest interval was given between consecutive testing maneuvers.For the purpose of this study, an injury was defined as any non-contact, soft tissue  injury sustained by a player during a training session or competition which resulted in a player being unable to take a full part in future football training or match play .These injuries were confirmed by team doctors. Players were considered injured until the club medical staff  allowed for full participation in training and availability for match selection. Only thigh muscle  and knee and ankle ligament injuries were considered for the analysis as these injuries are more likely to be preventable and influenced by the investigated variables.The team medical staff of each club recorded LE-ST injuries on an injury form that was sent to the study group each month. For all LE-ST injuries that satisfied the inclusion criteria, team medical staff provided the following details to investigators: thigh muscle , knee or ankle ligament, leg injured (dominant/non-dominant), injury severity based on lay-off time from futsal , date of injury, moment (training or match), whether it was a recurrence  and total time taken to resume full training and competition. At the conclusion of the 9 month follow-up period, all data from the individual clubs were collated into a central database, and discrepancies were identified and followed up at the different clubs to be resolved. Some discrepancies among medical staff teams were found to diagnose minimal LE-ST injuries and to record their total time lost. To resolve these inconsistencies in the injury surveillance process (risk of misclassification of the players), only ST-LE injuries showing a time lost of >8 days (moderate to severe) were selected for the subsequent statistical analysis.After having completed an exhaustive data cleaning process [detected anomalies or errors were removed (16 cases) and missing data (2.3%) were replaced by the mean value of the corresponding variable according to the sex  of the players] we had an imbalanced  and a high-dimensional data set comprising of 72 male and 67 female futsal players (instances) and 66 potential risk factors (features). In this study, an anomalies or error was defined as a score or value that could not be classified as real or true because of the consequence of a human error or a machine failure. An example of an error was a hip adductor PT value of 1,500 N because the measurement range of the handheld dynamometer used was from 0 to 1,335 N.Prior to analysis, continuous data were discretized as this can improve the performance of some classifiers . ContinuAfterward, eleven data sets were built. In particular, five data sets were built using the personal [data set (DS) 1—personal variables], psychological  and self-perceived (DS 5—player’s self-perceived chronic ankle joint stability) measures recorded from the questionnaires selected in this study. Likewise, three data sets were also built using the data from each of the three field-based tests carried out . Finally, three extra data sets were built, one that grouped all the measures obtained from the questionnaires , another one that included all the neuromuscular performance measures recorded from the field-based tests (DS 10—neuromuscular performance measures from field-based tests) and finally one that contained all measures recorded .The taxonomy for learning with imbalanced data sets proposed by Four classifiers based on different paradigms, namely decision trees with C4.5  and ADTrDue to the high dimensionality of the DS 10-neuromuscular measures from field-based tests (47 variables) and DS 11-Global (66 variables), before running the algorithms included in the taxonomy just described, a feature selection process was carried out. In particular, we used the metaclassifier “attribute selected classifier” (with GreedyStepwise as search technique) available in Weka’s repository to address the feature selection process.To evaluate the performance of the algorithms, the fivefold stratified cross-validation technique was used . The fivThe AUC and F-score were used as measures of a classifier’s performance . Only th0 = no differences across algorithms’ performance scores) was found (Bayesian factor [BF10] > 10), a post hoc procedure was carried out to identify the best performing model. In the cases in which either there would not be a strong evidence for rejecting H0 or a group of algorithms showed the highest F-score results [without any relevant difference (BF10 < 10) among then], the best-performing algorithm for this dataset would be the one that showed the highest F-scores.In order to compare the performance of the algorithms ran in each data set (intra data set comparisons) and whose AUC scores were >0.70, the F score was selected as criterion measure. These comparisons were conducted using separate Bayesian inference analyses . In thosFinally, the best performing algorithm of each of the data sets were compared (inter dataset comparisons) using the same statistical approach in order to know which questionnaire, field-based test, or combination showed the best ability to predict moderate LE-ST injuries in elite male and female futsal players.n = 23), whereas only eight cases were considered severe injuries (five anterior cruciate ligament injuries). Five players sustained multiple soft tissue non-contact lower extremity injuries during the observation period, so their first injury was used as the index injury in the analyses. Consequently, 25 soft-tissue injuries were finally used to develop the prediction models.There were 31  soft tissue injuries over the follow-up period, 17 (54.8%) of which corresponded to thigh muscles  injuries, eight (25.8%) to knee ligament, and six (19.3%) to ankle ligament. Injury distribution between the legs was 74.1% dominant leg and 25.9% non-dominant leg. A total of 13 injures occurred during training and 18 during competition. In terms of severity, most injures were categorized as moderate  of algorithms selected in each data set are displayed in All data sets are publicly available on As displayed in the 10 > 100 (extreme evidence for supporting H1)] among their prediction performance scores. The subsequent post hoc analysis identified a sub-group of four algorithms whose F-scores were similar among them (F-scores ranging from 0.422 to 0.450) and also statistically higher (BF10 > 10) than the rest  reported the presence of relevant differences  among their predictive ability scores. The subsequent post hoc analysis reported that a group of three algorithms showed similar F-scores among them (ranging from 0.458 to 0.474) but significantly higher than the rest. Therefore, the selection of the best performing algorithm of this DS 10 was based on the highest F-score. Thus, the algorithm CS-UBAG with SMO as base classifier was the one that showed the highest F-score (0.474 ± 0.111) and hence, it was selected for the inter data set comparisons. The feature selection process carried out in the DS 10—neuromuscular measures from field-based tests identified a subset of four ROM measures as the most relevant  on which was subsequently applied the taxonomy of learning algorithms described in the “Materials and Methods” section. Thus, a total of 66 algorithms built (using this subset of features) prediction models with AUC scores ≥0.7. The Bayesian analysis conducted with these 66 algorithms documented the existence of relevant differences , 8 [CS-UBAG (C4.5)], 10 [CS-UBAG (SMO)] and 11 [CS-UBAG (C4.5)] showed that the algorithm of the DS 8 obtained significantly lower F-scores than the other three algorithms (BFg model” . Models The main findings of this study indicate that only those groups of measures from two of the field-based tests [ROM-Sport battery (AUC = 0.751 ± 0.124) and Y-Balance (AUC = 0.701 ± 0.114)], as independent data sets, can build robust models (AUC ≥ 0.7) to identify elite futsal players at risk of sustaining a LE-ST injury. One of the possible reasons why only the lower extremity ROM and dynamic postural control measures can separately build robust prediction models may be related to the fact that they play a significant role in the hazardous lower extremity movement patterns performed by futsal players. In particular the execution of numerous weight-bearing high intensity locomotive actions  that may produce excessive dynamic valgus at the knee with limited hip and knee flexion ROMs, which have been identified as primary and modifiable LE-ST injury patterns . The facn = 627). The logistic regression-based model generated revealed that while the TP rate was moderate to high (76%) the TN rate was low (44%). This circumstance reflects one of the main limitations inherent in traditional regression techniques, that is to say, they do not deal well with imbalanced data sets [their models usually are biased toward the majority class (true negative rates) to optimize the percentage of well-classified instances]  field-based tests  . Furtherstances] .Due to their high cost (approximately 250€ per unit) currently available GPS systems may not be considered as accessible tools for most practitioners that work in applied sport settings, however, it should be noted that prediction models to identify team sport athletes (mainly soccer and rugby players) at risk of sustaining a LE-ST injury based exclusively on external training workload measures and built using learning algorithms are available . HoweverThe results of this study also reported that the combination in the same data set (DS 9) of all the measures obtained from the five questionnaires selected did not permit classification algorithms to build prediction models with acceptable performance scores (AUC scores ranged from 0.443 to 0.558). Previous studies have documented the existence of significant associations between some personal characteristics [e.g., age  and receThe prediction properties of the “model of best fit” of the current study were lower than that reported by the only other study that has used Machine Learning techniques to develop a screening model based on field-based measures  . One of n = 139) and LE-ST injuries recorded (n = 25) was large enough to build robust prediction models, the inclusion of more instances in the learning processes of the models may have improved their performance scores. Finally, out of the 88 possible combinations of measures that could have been analyzed with the data from the five questionnaires and three field-based tests, only three of them were explored, from both a time perspective and based on those that would be most interesting from a practitioner perspective. Therefore, it is unknown if other combinations of measures, different from the ones analyzed in this study, may have provided prediction models with higher AUC scores.The current study has a number of limitations that must be acknowledged. The first potential limitation of the current study is the population used. The sport background of participants was elite futsal and the generalizability to other sport modalities and level of play cannot be ascertained. Although all the measures recorded during the screening session are purported as LE-ST injury risk factors, there are a number of other measures from different questionnaires and field-based tests not included in this study (due to time constraints) which have been associated with LE-ST injury  and that may have improved the ability to predict LE-ST injuries in this cohort of athletes. Neither situational  nor movement  patterns for those futsal players who suffered a LE-ST injury were recorded for this study due to technical reasons . Although the main findings of this study may help identify futsal players at high risk of LE-ST injury, having included information regarding situational and movement injury patterns in the models might have not only increase their predictive performance scores but shed light on why and how LE-ST injuries occur in futsal players. Despite the fact that the number of both futsal players assessed  that showed moderate accuracy  for identifying elite futsal players at risk of LE-ST injury. The “model of best fit” of the current study  was comprised by only two hip (flexion with knee extended and abduction) and two ankle (dorsiflexion with knee flexed and extended) ROM measures and 10 different classifiers. Given that these ROM measures require little equipment to be recorded and can be employed quickly (approximately 5 min) and easily by trained staff in a single player, the model developed in this study should be included as an essential component of the injury management strategy in elite futsal.https://data.mendeley.com/datasets/s7fs9k3nby/2.Data is available online at: The studies involving human participants were reviewed and approved by Órgano evaluador de proyectos, Universidad Miguel Hernández de Elche (DPS.FAR.02.14). The patients/participants provided their written informed consent to participate in this study.All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


We use 36 years (1980–2015) of hourly weather data over the contiguous United States (CONUS) to assess the impact of low-cost energy storage on highly reliable electricity systems that use only variable renewable energy . Even assuming perfect transmission of wind and solar generation aggregated over CONUS, energy storage costs would need to decrease several hundred-fold from current costs (to ∼$1/kWh) in fully VRE electricity systems to yield highly reliable electricity without extensive curtailment of VRE generation. The role of energy storage changes from high-cost storage competing with curtailment to fill short-term gaps between VRE generation and hourly demand to near-free storage serving as seasonal storage for VRE resources. Energy storage faces “double penalties” in VRE/storage systems: with increasing capacity, (1) the additional storage is used less frequently and (2) hourly electricity costs would become less volatile, thus reducing price arbitrage opportunities for the additional storage.  •As storage capacity increases, hourly electricity costs become less volatile.•High-cost storage fills short-term gaps between renewable electricity generation and demand•Near-free storage provides reliable seasonal storage for variable renewable electricity•With increasing storage capacity, additional storage is used less frequently Energy Resources; Energy Policy; Energy Engineering With a goal to reduce economic damages from future climate change , energy Cost-effective grid-scale energy storage is often considered as a critical enabling technology to realize an affordable, reliable electricity system based solely on VRE generation. Capital costs of energy storage technologies have decreased rapidly in conjunction with improvements in technical performance  and deveThe technical performance and economic feasibility of energy storage in electricity systems have been evaluated using two distinct approaches. The first approach calculates the levelized cost of electricity (LCOE) in comparison with the economics of electricity generation technologies . StudiesVariable renewable electricity with ∼$100/kWh energy storage could already deliver electricity at a lower LCOE than fossil-fuel-powered generators , but thePower system modeling studies provide increased capacity to model systems with high shares of variable renewable electricity . MacDona2 limits and a range of future projected generation asset costs in two regional electricity systems , More recently, several studies investigated the role of energy storage to ensure highly reliable VRE electricity. Based on geophysical constraints, Several states and regions of the world have enacted legislation or set policy goals that specify that all electricity generation must come from zero-carbon or renewable resources before 2050 . The varThe focus of this study is to assess the effects of deep reductions in energy storage costs on highly reliable electricity systems based exclusively on VRE resources, using a transparent approach and decades-long hourly-resolution weather data. The fundamental trade-off in system design is between overbuilding VRE capacity (and thus curtailment of VRE electricity) and building and utilization of storage to mitigate the short-term and long-term variability between VRE resources and electricity demand. We used a standard yet simplified optimization model with parametrizations of a wide range of energy storage costs and considerations of VRE/storage system scenarios (in terms of technology availability and technology costs) to identify the roles of energy storage and the resulting system-level effects at different storage costs for the least-cost VRE/storage systems. Owing to abstraction, the absolute numbers in the results may be inaccurate when taken out of context, but relationships between VRE technologies and energy storage  in the least-cost systems could have practical implications for actual electricity systems and technology development and investment of energy storage.We constructed a transparent Macro-Energy Model see  to simulHistorical hourly wind and solar generation potential between 1980 and 2015 is obtained from a reanalysis dataset that used a variety of data sources to produce a best-fit description of the state of the atmosphere every hour, with ∼60 km resolution across the CONUS . We assuTo generalize the findings, we model generic energy storage characterized by capital cost, round-trip efficiency (90%), and energy loss over time see . We assuIn the main text, we report the least-cost optimization results for the baseline VRE/storage scenario  . The SupOur idealized system is a closed electricity system, and we did not consider flexibility mechanisms associated with the conversion of electricity into heat, fuel, or other types of energy services. A (best-case) perfectly efficient transmission network was assumed to connect wind turbines and solar PVs, as well as load, across the CONUS. Real-world transmission losses and regionalization of generation and demand, by states or regional interconnection boundaries, would result in increased gaps between supply and demand relative to our idealized consideration of the CONUS as a single VRE generation area and load-balancing region. We assumed that the electricity system was constructed virtually instantaneously, and therefore did not consider decreases in asset cost due to technology learning . Nor did we consider switching costs associated with a transition from a legacy electricity system, or lock-in costs associated with a path-dependent electricity system transition designed to reduce and ultimately eliminate carbon emissions.Least-cost VRE/storage systems were evaluated across four orders of magnitude, from $1,000 to $0.1/kWh, of energy storage costs. Although previous studies have analyzed the role of storage costs in power systems, the computational complexity of more detailed and realistic models have limited the number of cases analyzed. The simplifying assumptions made by our idealized model might overestimate the resource adequacy of electricity systems but thus afford a longer-term and more comprehensive analysis of those systems.Our analysis indicats that low-cost energy storage would have four critical system-level effects: (1) a decrease in total systems costs and mean electricity costs, (2) a change in the relative fractions of wind and solar electricity generation in least-cost systems, (3) a change in the roles that energy storage would play in least-cost systems, and (4) a reduction in the variability of hourly electricity costs.12 kWh per year) relative to analogous VRE/storage systems that instead used high-cost  storage.Near-free energy storage would eliminate the need to overbuild (by ∼340%) variable renewable electricity generation assets  to achieve 100% resource adequacy. In contrast, the least-cost VRE/storage system with high-cost  energy storage would require overbuilding these assets by >3-fold to generate enough renewable electricity to meet demand during periods of low wind and solar generation in a year . As showThe nonlinear relationship between the mean electricity cost of the VRE/storage systems and energy storage capital cost is a result of the different mixes of VRE resources  or near-free (∼$1/kWh) energy storage, the least-cost VRE/storage system would have a high ratio of wind electricity generation relative to solar electricity generation A, and S1In contrast, for intermediate energy storage costs , a generation mix of both wind and solar electricity would be present in the least-cost VRE/storage system A and S1.For >$150/kWh energy storage, the share of electricity discharged from storage assets in the least-cost VRE/storage systems remains small <3%) but would be essential in certain hours B and 3A.% but wouThe role of energy storage changes from high-cost storage competing with curtailment to fill short-term gaps between VRE generation and hourly demand to near-free storage serving as seasonal storage for VRE resources. In our optimized VRE/storage system, this transition is driven by deep reductions in energy storage costs. However, in systems that already have substantial VRE capacity deployment, this transition might be driven by the need to reduce curtailed electricity or price arbitrage.At high costs (>$100/kWh), energy storage is charged and discharged within a day. This could happen irregularly for wind-only or wind-dominated  systems A and 3B At high storage costs  is a widely used metric in the techno-economic assessment of energy storage technologies . As defi12 kWh per year, the expenditure on energy storage would be roughly $39 billion per year.As shown in The relationship between system expenditure on storage and cost of storage depends on the relative costs and resource availability of wind and solar energy. In a wind-dominated VRE system  per year to 31 cycles  per year, and would then decrease to 1 cycle  per year for the baseline VRE/storage system , and theIn the least-cost VRE/storage systems, energy storage roughly follows the Pareto Principle, as 20% of the storage capacity supplies 80% of the discharged energy, whereas the additional 80% of the storage capacity supplies only the remaining 20% of discharged energy . With inAs discussed above, energy storage performs distinct roles in the least-cost VRE/storage system optimized at different storage costs . In FiguLow-cost energy storage would lead to substantial system cost reductions  for the The interannual variability of VRE resources could have a substantial impact on the optimal design and operation of electricity systems based primarily on variable renewable generation assets. For the least-cost VRE/storage systems at high storage costs , the evaluated metrics .In x system cost reductions than the baseline case (with both wind and solar) for the same storage cost  comparable with a system with the baseline VRE capacity costs and $100/kWh storage ($0.095/kWh). Hence a more meaningful comparison is associated with relative cost changes between systems evaluated in this work. These relative system cost comparisons provide a tool to uncover the fundamental relationships between VRE technologies and storage for the least-cost VRE/storage systems. Reductions in generation asset costs in the future would lower system costs. However, the parameterized ratios between generation costs and energy storage costs would still apply, so decreases in future wind/solar generation costs would require proportionate reductions in energy storage costs for storage to be deployed relative to curtailment of wind and solar generation in least-cost reliable electricity systems.Previous studies  showed tAs a result, system cost reductions due to declining energy storage costs in the VRE/storage/dispatchable generation systems shrink substantially compared to the baseline VRE/storage systems (without dispatchable generation) for the same energy storage cost. At storage costs of $10–1,000/kWh, energy storage leads to only about 10% of the system cost reductions when dispatchable generation is present compared with the baseline case without dispatchable generation  and S18.The main conclusion that energy storage cost would have to be several orders of magnitude lower than today's cost for the least-cost VRE/storage systems to generate reliable electricity without overbuilding VRE capacity is still valid. Indeed, when dispatchable generation is present, storage cost would have to be even lower than the baseline case (to $0.1/kWh) for the VRE/storage system to eliminate dispatchable generation , and S7.In Geophysical constraints on the variability of wind and solar resources are a substantial driver of system costs owing to the need to oversize VRE capacities or deploy adequate storage to avoid infrequent, long-duration outages as well as compensate for seasonal resource variability. Even with the favorable assumptions that underpin the idealized VRE/storage systems modeled herein, specifically lossless transmission and ideal utilization of the solar resource simultaneously over four time zones, as well as instantaneous aggregation of the wind resource over the entire CONUS, substantial gaps can nevertheless be present between hourly electricity demand and VRE generation over a representative year  the additional storage is used less frequently and (2) hourly electricity costs would become less volatile, thus reducing price arbitrage opportunities for the additional storage. Furthermore, the role of energy storage changes from high-cost storage competing with curtailment to fill short-term gaps between VRE generation and hourly demand to near-free storage serving as seasonal storage for VRE resources.The economics of energy storage depends critically on how storage is used . For insThe general conclusion is that the needed cost reductions in energy storage in a reliable variable renewable energy (VRE) electricity system are not factors of 2–5 from present values, as might be deduced from an LCOE analysis, but are on the order of 100-fold relative to current capacity costs for grid-scale energy storage (such as batteries). This 100-fold required reduction is driven by the fundamental trade-off in system design between overbuilding VRE capacity and building and utilization of storage to mitigate short-term and long-term weather-related variability for VRE resources, even assuming perfect transmission of wind and solar generation aggregated over the CONUS to all load. This finding is robust for different VRE costs and systems with dispatchable generators . Moreover, if VRE costs were to decrease, then storage costs would have to be even lower to favor storage relative to VRE capacity.Our results are in qualitative accord with and are reinforced by recently reported work  performeThe costs of electrochemical energy storage technologies  have reduced substantially over the past several decades owing to public and private research and development (R&D) and economies of scale . Recent The steep diminishing returns for energy storage cost reductions in VRE/storage systems indicate that other technologies that could provide flexibility, such as expanded electrification and demand response in heating, industry, and transportation; low energy-cost long-duration storage technologies ; and high-voltage direct-current (HVDC) transmission lines are likely to be cost-effective components for future zero-carbon energy systems. Furthermore, as has been previously noted , the avaIn the least-cost VRE/storage systems considered herein, the amount of money spent on energy storage, for systems meeting current CONUS electricity demand, would remain mostly constant  across a wide range of energy storage costs . Hence, in these scenarios, storage innovations would provide investors with a competitive advantage, but would not substantially increase the overall market size for grid-scale storage. The total electricity system costs may decrease substantially as a result of declining storage costs, which may motivate public support for the R&D of energy storage technologies. The reductions in system costs (compared to the same electricity system but without energy storage) are almost always greater than the expenditure on energy storage for the range of storage costs studied . Moreover, the ratio of system cost reductions to storage expenditures would increase substantially if energy storage costs were to decline from $1,000/kWh to $0.1/kWh. This potential mismatch between private and public benefits suggests a need for public R&D support for energy storage technologies to account for the positive externality and promote the public good.Although we focused primarily on the impact of deep reductions in energy storage costs on highly reliable VRE/storage systems, we note that energy storage provides many other grid services such as capacity values and ancillary services . Future The Macro-Energy Model provides a transparent approach to evaluate the degree to which energy storage costs would have to decrease to ensure resource adequacy in an electricity system with high shares of VRE generation. It assesses the fundamental system trade-off associated with building and operating energy storage and overbuilding VRE assets (and curtailment of VRE electricity) to ensure resource adequacy of the VRE/storage system with explicit consideration of weather-related short-term and long-term VRE variability.The use of a macro-energy model has allowed us to evaluate 1,512 (36 years ∗ 6 storage costs ∗ 7 system designs) optimization cases to assess the impact of energy storage costs on VRE/storage systems with various levels of resource adequacy requirements in the context of short-term and long-term weather-related variability of VRE resources on a multi-decadal timescale. In comparison, it would be challenging to achieve these tasks using state-of-the-art power engineering models or energy system models that include refined geographical representations, high temporal resolutions , and sophisticated technical details. The role and utility of these emerging macro-energy systems models have been discussed recently in detail .Although batteries and all storage technologies, in general, require specification of both energy and power costs, energy storage costs are a vital quantity to assess the role of energy storage in VRE/storage systems with respect to daily, seasonal, and interannual resource variability. Storage can provide additional services to a reliable grid, including 5-min load balancing and short-term stability services, for which power costs are an essential metric, but these services are beyond the scope of this work. The focus of this work is the fundamental trade-off between overbuilding VRE capacity (and thus curtailment of VRE generation) and building and utilization of storage to achieve the requisite hourly resource adequacy for the VRE/storage systems over a long time period. To cost-effectively obtain short-term and long-term resource adequacy and reliability would likely require the utilization of several types of storage technologies synergistically, each of which has individual advantageous characteristics concerning power costs or energy costs.This work used a historical demand profile owing to the limited availability of modeled future demand data at hourly resolution . The US Actual electricity systems are spatially segregated and linked by capacity-limited transmission lines. The impacts of VRE resources are likely to be amplified by congested transmission lines and various grid conditions, which would require a more detailed electricity system model to assess in full. A more sophisticated “real-world” model that accounts for transmission losses and regionalization of generation would exacerbate the gaps between VRE resources and electricity demand and thus further emphasize the need for energy storage to effectively mitigate the variability of VRE resources relative to curtailment or other means of flexibility. Use of a shorter load balancing time in modeling, such as 5 min, would be appropriate to simulate other grid stabilization and load balancing mechanisms  but woulIn this work, we optimized the VRE/storage system on an hourly basis for 1 year. To illustrate the impact of interannual variability of VRE resources, the analysis was repeated using hourly weather data for each year from 1980 to 2015. Owing to computational (memory) limitations associated with simultaneous optimization of hourly storage capacity over 5 years, we are unable to solve the optimization for VRE/storage systems with weather data for more than 5 years at a time. Furthermore, the assumption of perfect foresight in multi-decade planning is not realistic, and the forecast data would nevertheless contain errors, both of which are fundamental challenges for long-term planning. In the additional runs, multi-year optimizations led to a slightly higher mean electricity cost and larger energy storage capacity than those obtained from single-year optimizations. These results are driven by a higher degree of mismatch between VRE resources and electricity demand over multi-year periods than in a single year. The general conclusions of this work, arising from the system design trade-off in overbuilding VRE capacity and building and utilizing energy storage to address the mismatch between VRE resources and electricity demand at different timescales, would remain robust in multi-year or multi-decade optimization cases, but the exact numerical results could vary from those specified herein.kcaldeira@carnegiescience.edu).Further information and requests for resources should be directly to and will be fulfilled by the Lead Contact, Ken Caldeira  have been deposited to Zenodo: All methods can be found in the accompanying


The novel coronavirus SARS-CoV-2 has spread across the world since 2019, causing a global pandemic. The pathogenesis of the viral infection and the associated clinical presentations depend primarily on host factors such as age and immunity, rather than the viral load or its genetic variations. A growing number of omics studies have been conducted to characterize the host immune and metabolic responses underlying the disease progression. Meta-analyses of these datasets have great potential to identify robust molecular signatures to inform clinical care and to facilitate therapeutics development. In this study, we performed a comprehensive meta-analysis of publicly available global metabolomics datasets obtained from three countries . To overcome high heterogeneity inherent in these datasets, we have (a) implemented a computational pipeline to perform consistent raw spectra processing; (b) conducted meta-analyses at pathway levels instead of individual feature levels; and (c) performed visual data mining on consistent patterns of change between disease severities for individual studies. Our analyses have yielded several key metabolic signatures characterizing disease progression and clinical outcomes. Their biological interpretations were discussed within the context of the current literature. To the best of our knowledge, this is the first comprehensive meta-analysis of global metabolomics datasets of COVID-19. COVID-19 is an unprecedented health emergency driven by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) ,4. A comMetabolism fuels all biological processes in the human body, including immune responses. Blood metabolites are the end products of many systematic processes and are informative indictors of biochemical activities or diseases’ phenotypes ,6. PowerMeta-analysis of the available datasets is a promising approach to gain a comprehensive understanding of the pathogenesis of the disease ,21, as wTo address this research gap and to gain a better understanding of the metabolic changes underlying the disease, we systematically collected the COVID-19 global metabolomics datasets that were publicly available as of 5 November 2020 and implemented a computational pipeline for spectra processing, visual exploration and meta-analysis. In this manuscript, we report our findings and discuss their implications within the context of the current understanding of the disease.https://massive.ucsd.edu/) and two directly from the authors. The remaining two datasets were annotated metabolite intensity tables obtained from the A total of 175 COVID-19 papers were identified in our initial search. After filtering these studies based on our inclusion/exclusion criteria, six studies from the USA, China and Brazil were finally included in this meta-analysis . One stup-value < 0.001). Overall, these results indicated overall significant metabolic perturbations in COVID-19 patients across all study populations. For two randomly selected datasets, we also compared the results from our spectra processing pipeline against those from two other public spectra processing tools [The five raw spectra datasets were processed using our MetaboAnalystR 3.0 pipeline for optimized peak detection, quantification and alignment . The final optimized parameters are provided in ng tools ,28 and op-values were further integrated to produce a final ranked list of perturbed pathways . Despite the ambiguities in individual compound assignments, we also attempted to extract the peaks underlying these four perturbed pathways from individual studies. The correlations between these peaks with the symptom onset days were then statistically evaluated. Nine peaks were significant , with one negative and eight positive associations  onto the KEGG global metabolic network . NetworkFour datasets contain samples from patients classified as MM and severe COVID-19. The patients with fatal outcomes were also included in the severe group for this comparison based on their clinical status. We first aimed to identify commonly perturbed metabolic pathways across the four datasets. As summarized in Three datasets  contained COVID-19 patients with mortality information. The C1 dataset was excluded because it contained only two cases to perform meaningful statistical analysis. Metabolic differences between the severe and fatal patients were evaluated with the remaining two datasets . SeveralPatients with SARS-CoV-2 infection manifest a classical respiratory virus-like clinical course with activated innate and adaptive immune responses ,30. MultSeveral common metabolic pathways were identified by comparing COVID-19 patients with healthy subjects. The most significantly perturbed pathway is Porphyrin metabolism or Heme biosynthesis, which is consistent with previous reports ,32. The The heterogeneity of COVID-19 patients shows a wide spectrum of symptoms as well as disease severity. The risk categorization of COVID-19 is difficult because of the complexity of the pathophysiological status of the patients. Therefore, understanding the molecular underpinnings of the disease severities is important to help to reduce the mortality.Patients with mild-to-moderate (MM) cases of COVID-19 typically have an optimistic prognosis and can recover very quickly. Pathway analysis between MM and severe COVID-19 showed six common perturbed pathways. Most of them were amino acids pathways. Our analysis identified propanoate as a novel pathway in the progression of COVID-19. Propanoate metabolism usually starts with the gut microbiota and enters into immune cells such as macrophages, thereby modulating the biological process . The glySARS-CoV-2 infection can not only cause pathogenic changes in the respiratory system but can also lead to systematic multi-organ damages and death . PreventThe high level of heterogeneity inherent in global metabolomics datasets poses tremendous challenges to conduct metabolomics meta-analysis at the feature (MS peaks) level. In this study, we utilized the well-established Mummichog method to first compute pathway activities from MS peaks and performed meta-analysis at the pathway level. There are however, several limitations to this analysis method. The potential bias caused by differences in the extraction procedures and analytical platforms at pathway level remains an open question. Due to the nature of putative annotations, the significant metabolites reported in this study need to be further validated using more targeted approaches. Although the potential confounding factors  were controlled within each study, they were not considered in the current meta-analysis because most meta-data are incomplete or missing from the original studies. We intend to address this issue by expanding this analysis to include multiple-cohorts-based metabolomics studies when more datasets become available in the coming year. In addition, many signatures are likely to reflect general immune and inflammatory responses. We plan to include studies on other viral infections (such as SARS-CoV and influenza) to identify unique metabolic signatures of this disease as illustrated in a recent meta-analysis based on transcriptomics .www.medrxiv.org/), and bioRxiv (www.biorxiv.org/) using the search term “(COVID-19) AND (Metabolomics)” before 5 November 2020. The inclusion criteria for further processing were as below: (1) The study should have had a matched healthy control for COVID-19 samples; (2) all raw spectra data or original/annotated peak tables should have been available publicly or upon request; (3) to ensure comparability, only LC-MS-based global metabolomics datasets were included; other metabolomics datasets generated by gas chromatography (GC)-MS or nuclear magnetic resonance (NMR) were excluded. The PRISMA 2009 Flow Diagram is provided in This meta-analysis was strictly conducted based on the PRISMA guidelines . All stuGuideline of Diagnosis and Treatment Protocol for Novel Coronavirus Pneumonia (8th) published by the National Health Commission of China [WHO R&D Blueprint novel Coronavirus COVID-19 Therapeutic Trial Synopsis [All COVID-19 patients were diagnosed separately at their original hospitals or testing centers. Their disease severities were classified according to a combined standard based on the of China , WHO R&DSynopsis , and an Synopsis .Raw LC–MS spectra were first converted and centroided from vendor format to mzML using ProteoWizard . All cenChemometrics analysis (PCA and OLS-DA) was performed based on the normalized peak tables using the corresponding functions in MetaboAnalystR 3.0. Spearman correlations between the onset days of symptoms and metabolic features were calculated using base R package (v4.0.2). The confidence interval of the significant correlation was set to 0.95.p-values were computed based on the vote counting method in the metap package in R (v4.0.2) by counting the p-value from two directions and outputting an integrated p-value based on the counting results. The enrichment ratio describes the relative percentage of the empirical compound hits to the whole empirical pathway. The enrichment ratio of the compounds from the annotated peak tables was calculated with the average of the other empirical pathway size as the denominator.The pathways analysis on the datasets from raw spectra in this present study was performed independently for every dataset using Mummichog  from theThe MS peaks from different studies were putatively annotated based on the Mummichog algorithm and mapped to the KEGG global metabolic network using the Peaks to Pathway module in MetaboAnalyst . The sizThe peak intensity tables from the individual datasets were uploaded to the Peaks to Pathway module in MetaboAnalyst 4.0 . After nThere are significant knowledge gaps in the systems biology of COVID-19. The ongoing multi-omics investigations will continue to yield valuable insights to fill this gap in the coming year. Global metabolomics can provide rich data that complement other omics layers to inform the development of diagnostics, prognostics, and treatment of COVID-19. In this study, we have systematically curated public metabolomics datasets and performed comprehensive data processing, analysis and meta-analysis to identify common as well as unique metabolic signatures underlying different clinical courses of COVID-19. Our results suggest that extensive dysregulations of amino acids metabolism, damage to the oxygen transport in red blood cells, exhaustion of endogenous immune bioactive metabolites and the suppression of multiple physiological processes are the metabolic characteristics underlying the progression of COVID-19. We will continue to improve the computational workflow and expand the scale and scope of the current meta-analysis when more metabolomics datasets become available in the coming year.


POLE exonuclease domain mutations, and are characterised by high tumour mutational burden and increased lymphocytic infiltrate. Metastatic dMMR colon cancers are highly sensitive to immune checkpoint inhibition, and recent data show POLE-mutant tumours are similarly responsive. We are conducting a phase III randomised trial to determine if the addition of the anti-PD-L1 antibody avelumab following adjuvant chemotherapy improves disease-free survival (DFS) in patients with stage III dMMR/MSI-H or POLE mutant colon cancer and is a cost-effective approach for the UK National Health Service (NHS).10%–15% of early-stage colon cancers harbour either deficient mismatch repair (dMMR), microsatellite instability high (MSI-H) or POLE exonuclease domain mutation on central testing. Eligible patients are randomised in a 1:1 ratio to standard fluoropyrimidine-based chemotherapy  or chemotherapy, followed by avelumab . Stratification is by chemotherapy received and MMR/MSI-H status. The primary endpoint is DFS. Secondary endpoints include overall survival, toxicity, quality of life and health resource use. The 3-year DFS rate in the control arm is expected to be ~75%. Avelumab is expected to improve the 3-year DFS rate by 12% . Target accrual is 402 patients, which provides 80% power to detect an HR of 0.48 for DFS at a two-sided alpha of 0.05. This national, multicentre phase III trial is sponsored by the Royal Marsden NHS Foundation Trust and it is anticipated that approximately 40 centres in the UK will participate. This study opened to recruitment in August 2018.We are recruiting patients with completely resected, stage III colon cancer confirmed to have dMMR/MSI-H, locally or NCT03827044 Fluoropyrimidine-based adjuvant chemotherapy following surgery is the current standard of care for stage III colon cancer.POLE mutant CRC (mCRC) has also been proposed as a type of mCRC, which is also responsive to immunotherapy.Immune checkpoint therapy is effective in the treatment of deficient mismatch repair (dMMR)/microsatellite instability high (MSI-H) colorectal cancer (CRC) in the advanced setting. POLE mutant stage III colon.To our knowledge, there is no mature randomised clinical data to support the use of immune checkpoint inhibitors in the curative setting such as dMMR/MSI-H or POLE mutant stage III colon cancer.The POLEM trial is an open-label, multicentre, randomised, phase III study testing the efficacy of the immune checkpoint inhibitor avelumab (anti-PD-L1) following standard adjuvant chemotherapy in dMMR/MSI-H or Eligible patients are randomly allocated to receive investigator choice chemotherapy , followed by avelumab for 24 weeks or chemotherapy alone.The recruitment aim is 402 patients and the study is currently open in the UK with potential for international collaboration.The results from this study will determine whether immune checkpoint therapy such as avelumab (anti-PD-L1) should be added to standard adjuvant chemotherapy in deficient mismatch repair/microsatellite instability high or POLE mutant stage III colon cancer.Colorectal cancer (CRC) is the third most common cancer, with a worldwide annual incidence of over 1.2 million cases and a mortality rate of approximately 50%.The DNA mismatch repair (MMR) machinery is essential for maintenance of genomic integrity. Defects in DNA MMR can occur either at the germline (Lynch syndrome) or epigenetic level.The management of metastatic dMMR/MSI-H CRC has recently been transformed by clinical data demonstrating remarkable clinical benefit of PD-1 inhibitors in this setting.Importantly, National Institute for Health and Care Excellence  guidance from 2017 recommends MMR or microsatellite instability testing should be undertaken for all patients with CRC, when first diagnosed, using immunohistochemistry or PCR for Lynch syndrome.POLE and POLD1 is critical in the maintenance of DNA replication fidelity and prevention of mutagenesis. The relevance of this to human cancer was confirmed by the demonstration that germline mutations in the POLE and POLD1 exonuclease domains cause intestinal polyposis and early-onset cancer,POLE exonuclease domain mutations in CRC by The Cancer Genome Atlas (TCGA).POLE-mutant cancers were found to have and exceptionally high number of somatic mutations (over 50 per 106 bases), in the absence of MMR defects.POLE mutant cancers also showed higher expression of T cell markers such as CD8A, PD-1 and CTLA-4 suggesting the presence of a pre-existing T cell infiltrate.The proofreading exonuclease activity intrinsic to the replicative DNA polymerases POLE exonuclease domain mutant somatic mutations are in the range of 1%–3% of patients,POLE mutations is 8%–10%. While somatic POLE mutation portends reduced risk of relapse in stage II CRC, the association with favourable prognosis appears to be lost in stage III disease.TCGA data suggest that POLE exonuclease domain mutant stage III colon cancer patients gain clinical benefit  from PD-L1 inhibition after standard fluoropyrimidine-based adjuvant chemotherapy and to examine whether this is a cost-effective approach for the National Health Service.The aim of this study is to determine whether dMMR/MSI-H and/or Avelumab is a humanised monoclonal antibody, which binds PD-L1 and blocks the interaction between PD-L1 and PD-1. This removes the suppressive effects of PD-L1 on anti-tumour CD8 +T cells, resulting in the restoration of cytotoxic T cell response.POLE exonuclease domain mutant CRCs have a highly and ultramutated genetic profile, respectively, thus leading to a high number of neoantigens with associated over expression of immune checkpoint-related proteins. This profile is expected to be highly responsive to checkpoint inhibition as suggested by data of PD-1 inhibitors in dMMR/MSI-H metastatic CRC.The rationale of giving avelumab after standard adjuvant chemotherapy to this well-defined, molecularly selected, group is based on the observation that dMMR/MSI-H and POLE exonuclease domain mutant colon cancers. Furthermore, given the availability of molecular markers for patient selection, funders of healthcare would be more likely to fund this treatment.If this study meets the primary endpoint, using avelumab in the adjuvant setting following standard chemotherapy would become the standard of care for patients with dMMR/MSI-H and/or POLE mutant CRC patients treated with the immune checkpoint inhibitor avelumab.This study also provides a unique opportunity to conduct translational research analyses on pretreatment and post-treatment tumour tissue samples and blood samples from dMMR/MSI-H or POLE exonuclease domain mutant colon cancer. Eligible patients  with standard fluoropyrimidine-based adjuvant chemotherapy alone (control arm) in patients who have undergone radical surgical resection for stage III dMMR/MSI-H or patients  will be POLE mutations of 8%–10% in patients under 50 years of age.According to the statistical design, 402 patients (201 per arm) are to be randomised. It is expected that approximately 4000 participants will need to be screened in order to recruit 402 patients to the study, assuming an incidence of dMMR/MSI-H of 10%–15% and an incidence of There are no proscriptive criteria for surgical resection of the primary tumour in this trial. However, it is expected that resection of the tumour will be undertaken in the elective setting by a colorectal specialist surgeon.POLE exonuclease domain mutations. Those who have tumours harbouring these mutations can sign the main study consent and undergo the study screening procedures. If they are found to fulfil all eligibility criteria, then they will be randomised.Tumour MMR/MSI status will be routinely tested locally as per NICE guidelinesAll eligible patients who are randomised will receive standard fluoropyrimidine-based adjuvant chemotherapy for 12 or 24 weeks depending on the decision of the local investigator. The choice of adjuvant chemotherapy  must be declared by the investigator at study entry before randomisation. The adjuvant chemotherapy regimen and MMR/MSI status will be used as stratification factors.At the end of adjuvant chemotherapy, patients who are randomised to the investigational arm will receive additional 24 weeks of treatment with avelumab .The primary endpoint is DFS at 3 years. This is defined as the proportion of subjects who are alive and disease free at 3 years. For subjects who neither progressed nor died, their DFS time will be censored at the last date of contact. The main analysis will take place once 60 events have occurred.Relapse will be based on investigator reported imaging assessments or colonoscopy. Where a patient has neither progressed nor died, their DFS time will be censored at the last date of contact .Median DFS will be reported with a 95% CIs in each group.OS at 5 and 7 years is defined as the proportion of patients who are alive at 5 and 7 years, respectively. Patients alive at the time of analysis will be censored at the date of last follow-up. Analysis methods for OS will follow those of the primary endpoint DFS (see primary endpoints above).Median OS will also be reported as DFS section above.Adverse events as assessed by National Cancer Institute Common Terminology Criteria for Adverse Events V.4.0Health-related quality of life based on EORTC QLQ C-30 and 5-level EuroQol 5 Dimensions: QLQ C-30 will be summarised over time, as per the standard scoring manual, for each of the 15 domains using mean and SD. Statistical analysis will consist of linear mixed models and the mean difference between treatments for each domain score will be presented.Quality-adjusted life years (QALYs).Health resource use: health resource use will be collected prospectively postrandomisation a healthcare services resource questionnaire designed for the trial.Resource use and associated costs will be combined with quality-of-life data to provide a cost per QALY for fluoropyrimidine-based adjuvant chemotherapy, followed by avelumab compared with standard fluoropyrimidine-based adjuvant chemotherapy alone. In terms of analysis, Kaplan-Meier plots will be used to summarise the survival estimates and report the DFS rate at 3 years along with a 95% CIs in each group. DFS rates between groups will be compared using the log-rank test. In addition, Cox proportional hazards (PH) model  will be used to compare the DFS rates overall and at 3 years adjusting for prespecified covariates. A test for non-PH will also be carried out.The primary analysis for DFS outcome will be unadjusted and will be based on the intention-to-treat (ITT) population with a sensitivity analysis in the modified ITT population and in the per-protocol population.POLE mutant patients) and second, a large effect size will also allow potential for cost-effectiveness of the experimental intervention.The 3-year DFS rate in the control arm is expected to be about 75%.Therefore, a sample size of at least 171 per group  is required  to reject the null hypothesis of:0; HR=1 vs H1: HR HR≠1 with at least 80% power assuming a two-sided 5% type I error.HAssuming further a 15% drop-out rate (lost to follow-up rate) a total sample size of 402 will be required (201 per group in a 1:1 randomisation).The incidence of stage III dMMR/MSI-H CRC is reported to be about 1753 per year in the UK (England and Scotland) (based on 41 112 CRC patients diagnosed in UK in 2013 (latest figures), 23.7% of these are stage III and 18% of them are dMMR/MSI-H). Assume some 20% of these may fail screening/eligibility criteria, the pool of patients available for recruitment is likely to be approximately 1500 per year.With a target of 402 patients, assuming recruitment over 36 months, on average, about 11 patients per month are anticipated . At least 30 sites are planned to be activated. Hence, on average, <1 patient per month is required per site. Assuming that recruitment will not be heavily competitive as the number of CRC trials in this setting are few, the target sample size is expected to be reached.POLE mutant colon cancers and has been designed to incorporate the recent IDEA collaboration trial data comparing 3 months vs 6 months fluoropyrimidine-based chemotherapy.The POLEM trial is one of two studies evaluating checkpoint inhibitor drugs in the adjuvant setting in stage III dMMR/MSI-H colon cancer; the other being the US-based ATOMIC study . POLEM is distinct from ATOMIC in that it also includes


It is increasingly recognized that deep understanding of epileptic seizures requires both localizing and characterizing the functional network of the region where they are initiated, i. e., the epileptic focus. Previous investigations of the epileptogenic focus' functional connectivity have yielded contrasting results, reporting both pathological increases and decreases during resting periods and seizures. In this study, we shifted paradigm to investigate the time course of connectivity in relation to interictal epileptiform discharges. We recruited 35 epileptic patients undergoing intracranial EEG (iEEG) investigation as part of their presurgical evaluation. For each patient, 50 interictal epileptic discharges (IEDs) were marked and iEEG signals were epoched around those markers. Signals were narrow-band filtered and time resolved phase-locking values were computed to track the dynamics of functional connectivity during IEDs. Results show that IEDs are associated with a transient decrease in global functional connectivity, time-locked to the peak of the discharge and specific to the high range of the gamma frequency band. Disruption of the long-range connectivity between the epileptic focus and other brain areas might be an important process for the generation of epileptic activity. Transient desynchronization could be a potential biomarker of the epileptogenic focus since 1) the functional connectivity involving the focus decreases significantly more than the connectivity outside the focus and 2) patients with good surgical outcome appear to have a significantly more disconnected focus than patients with bad outcomes. It has long been recognized that most anatomical subdivisions of the brain exhibit some degree of functional specialization , 2, and in vitro . Given their sharp shape on electroencephalographic recordings and the large spatial overlap of their generators with the epileptic focus , IEDs miIn the present study, we hypothesized that functional connectivity exhibits significant variations in the amplitude of synchronization time-locked to the appearance of IEDs, as recorded by intracranial macroelectrodes. In addition, we hypothesized that the epileptic focus displays smaller high-frequency synchrony during IEDs than other brain regions.For this study, we retrospectively collected data from patients with pharmacoresistant epilepsy who were admitted at the epilepsy monitoring unit of the University of Montreal Hospital Center. Thirty-five consecutive patients undergoing iEEG recordings as part of their pre-surgical evaluation were recruited for this study. In case patients had more than one epilepsy surgery, only the first invasive investigation was considered in this study. The research project was approved by the University of Montreal hospital research center ethics committee .All iEEG recordings were acquired from a combination of subdural strip, grid  and depth electrodes . iEEG signals were sampled at 2 kHz and digitized . Electrode contacts were labeled according to timing of the epileptic activity they display during seizures into 3 classes: focus (F—involved at onset), propagation (P—involved after seizure spread), and silent (S—not involved). An expert epileptologist (D.K.N.) reviewed seizure traces and assigned class labels to each electrode contact.e x Ne PLV-based synchrony matrix for each time window and frequency band, where Ne is the number of electrode contacts. For details regarding the mathematical basis of PLV, readers are referred to . To avoid sleep cycles' effect on IEDs, these were selected from awake state day recordings. An average of 51 IEDs were randomly selected per patient to provide a global picture of their interictal epileptiform activity. Data epochs were mirror band-pass filtered in consecutive frequency bands  with a finite impulse response (FIR) filter defined using a Kaiser window, as implemented in Brainstorm . The bacerred to . In briet, n) is the difference between phases of the 2 given signals at time t and trial n, and N is the number of trials.where θ on the global level of PLV-based synchrony at each time window.PLV-based synchrony values were averaged within each matrix to obtain a single time course for each IED and frequency band, representing the dynamics of global PLV-based synchrony. Since those matrices are symmetrical, only upper off-diagonal values were averaged. Time courses of PLV-based synchrony were then averaged over IEDs for each frequency band to reveal band-specific dynamics. In order to identify the time windows during which the global PLV-based synchrony level departs significantly from chance level, we designed a band-specific non-parametric statistical procedure. First, electrode signals were randomly shuffled across IEDs, such that electrode signal ED shape . DynamicTo further ensure that observed dynamic changes of PLV-based synchrony are specific to IEDs, we repeated the described analysis on randomly selected epochs. For each patient, 50 markers were set randomly on the DC component of iEEG signals to avoid any bias toward epileptic activity. Epochs  s were then exported and analyzed.We then computed the dynamics of PLV-based synchrony according to the connectivity types: the links among electrodes in the focus (F-F), among electrodes in the propagation area (P-P), among electrodes in silent areas (S-S) and across areas  were averaged separately within each connectivity matrix. The focus area (or seizure onset zone), the propagation zone, and silent electrodes were determined by an expert epileptologist based on visual interpretation of seizure recordings. The focus area was defined as the one displaying the first unequivocal intracranial EEG sign of change from the background that leads to a clear sustained rhythmic discharge, without return to background activity. The propagation zone is composed of other electrodes (areas) showing ictal activity but with a delay with regard to the onset zone. The dynamics of PLV-based synchrony of each connectivity type were then averaged across IEDs to obtain an average time course. We then quantified the change in PLV-based synchrony at the IED peak relatively to a baseline window (-1 to−0.5s relative to IED marker) by computing a z-score ([IED peak—average of baseline] / std. of baseline). The PLV-based synchrony change scores involving the focus—F-F/F-P/F-S—and those not involving the focus—P-P/P-S/S-S—were analyzed separately.t-tests: for each frequency band, we computed the t-statistic to measure the difference of average PLV-based synchrony change between patients with good and bad outcomes. This statistic was compared against 100 000 surrogate t-statistic generated by randomly shuffling patients across groups. Differences between patients with respect to surgical outcome were considered significant if the t-statistic exceeded 500 random t-statistics, which corresponds to a Bonferroni-corrected 5% significance level.We then assessed the relationship between the changes in PLV-based synchrony of each connectivity type and the surgical outcome of patients. We categorized patients as having either good  or bad (Engel III or IV) and assessed the effect of surgical outcome and frequency bands on PLV-based synchrony changes. Since data patients with good and bad outcomes had unequal variances, we conducted non-parametric In total, 35 patients  across patients, we found no statistically significant linear associations.We compared changes of PLV-based synchrony at the IED peak relative to a preceding baseline period separately for connectivity involving the focus and connectivity not involving the focus:t-tests revealed that IED-locked high-frequency  PLV-based synchrony is significantly different between patients with good and bad outcomes. However, those differences did not survive Bonferroni correction . No significant differences in low-frequency IED-locked PLV-based synchrony were found between patients with good and bad outcomes . We then averaged IED-locked PLV-based synchrony changes within low (θ+α+β) and high (γ1+γ2) frequencies and repeated the non-parametric test. We found that IED-locked high-frequency PLV-based synchrony was significantly lower in patients with good than in patients with bad outcome  while no difference was found for low-frequency PLV-based synchrony .When analyzing PLV-based synchrony involving the focus , left, np = 0.43; γ2: p = 0.48—corrected) nor low  frequency IED-locked PLV-based synchrony differed between patients with good and bad outcomes. After averaging PLV-based synchrony changes within bands, differences between patients with good and bad outcomes were not significant .When analyzing PLV-based synchrony not involving the focus , right, Patient 1 is a 35 year-old male with persistent nocturnal seizures (2–3 per day) despite eight antiepileptic drug trials. A presurgical evaluation revealed on surface EEG left fronto-temporal IEDs while ictal single-photon emission computed tomography (SPECT) and interictal positron emission tomography (PET) imaging showed, respectively hyperperfusion and hypometabolism over the left inferior frontal gyrus and anterior insula. MEG dipole imaging of IEDs showed a dense cluster of dipoles over the left superior temporal gyrus, anterior insula and left inferior frontal gyrus. IEEG recordings showed frequent interictal spiking activity in the anterior insula and broad onset of seizures, including the insula, the inferior frontal gyrus, the orbito-frontal gyrus and the superior temporal gyrus. The patient subsequently underwent an insulectomy  with an average of 51 IEDs per patient. In addition, we explored the link between changes in levels of PLV-based synchrony at the IED peak and the surgical outcome of patients.In this work, we investigated the role of PLV-based synchrony during the generation of IEDs at the macroscopic scale of brain networks using a relatively large cohort of patients (We found that the high frequency (γ1 + γ2) component of iEEG signals during interictal epileptiform discharges is associated with a statistically significant transient decrease in functional connectivity, time locked to the peak of the IEDs. Our results are in line with observations made during seizures; Mormann et al. found that seizures are consistently preceded by a state of hyposynchronization involving both the epileptic focus and distant brain areas . Burns eImportantly, our study showed that decreases in PLV-based synchrony during IEDs were restricted to the gamma 1 and gamma 2 frequency bands. By recording simultaneously unit activity in pairs of neurons and the local field potential in slice preparations of rats hippocampus, Netoff and Schiff observed that seizure initiation is marked by a decrease of synchrony both between pairs of neurons and between individual neurons and the local field potential . This trThe emergence of a hypersynchronous state, i.e., seizures, in weakly coupled networks has also been studied using numerical simulations. Nishikawa and Motter showed that any network under quantized total interactions could reach hypersynchronous states . In theiMore than fifty years ago, Cannon proposed the “Law of denervation” to account for increased neuronal sensitivity following deprivation of the nervous connections from which they routinely receive inputs . In a seWarren et al. used iEEG to compare the resting-state functional connectivity of the epileptic focus in epileptic patients to that of homologous regions in control patients suffering from facial pain injury and implanted with depth electrodes . They fop < 0.05, corrected). We were not able to reveal differences in discriminative power (among patients with good vs. bad outcomes) between γ1 and γ2 frequency bands.In line with that study, we found in our work that IED-locked high frequency (γ1+γ2) PLV-based synchrony was significantly lower in patients with good outcome than in patients with bad outcome. In other words, patients with good postsurgical outcome have a significantly more disconnected focus during IEDs than patients with bad outcome. We speculate that the epileptic focus is under the influence of resting-state networks during interictal periods and transient desynchronization is necessary to achieve transition toward the spiking and ictal states. More specifically, we found that IED-locked gamma-band PLV-based synchrony was significantly lower in patients with good outcome than in patients with bad outcome when analyzing connectivity within the focus  frequency bands. Furthermore, in this work, electrodes were labeled according to the timing of the epileptic activity they displayed during seizures as classified by an expert epileptologist. Exploring how changing electrode class can impact the observed results could be an interesting future avenue. Given the sample size of our study, we could not investigate the effect of antiseizure medications on functional connectivity patterns or postoperative seizure control.Our findings suggest band-specific changes in the time course of functional connectivity during IEDs. Although changes were only significant for the γ2 band, a preliminary distinction between frequency bands can be observed by comparing higher (>30 Hz) and lower (<30 Hz) bands, which respectively displayed decrease and increase in connectivity at the IED peak. This distinction is expected since they have distinct neural correlates and physiological roles in the healthy brain. Indeed, the low frequency range of the local field potentials reflects mainly synaptic activity and was associated with memory (theta band) , 50, aroRelative contributions of synchronized neural activity within specific frequency bands to epileptiform activity are yet poorly understood. Most of the previous studies investigating epilepsy-related changes in functional connectivity using EEG  assessed coupling in the broad band signal, which is mainly dominated by lower frequencies given the well-described 1/f shape of the power density spectrum on electrophysiological signals. In addition, we found a distinction between high (>60 Hz) and low (<60 Hz) gamma bands to connectivity changes. This distinction is in line with changes in the power spectrum density of EEG prior to seizures, which shows a bimodal distribution with two statistically independent peaks in the  gamma range .This study suggests that the emergence of interictal epileptiform activity is time-locked with a decrease in functional connectivity in the γ2 frequency range. Moreover, IED-locked gamma band PLV-based synchrony was significantly lower in patients with good than in patients with bad outcome recalling to a network configuration in which the epileptogenic focus is functionally isolated from the rest of the brain. While further studies with a larger cohort of patients are required, our results show promise for the design of quantitative methods capable of quantitatively localizing the epileptic focus.The datasets generated for this study will not be made publicly available as this is not allowed by the Ethical Research Committee. Requests to access the data should be sent to the corresponding author.The studies involving human participants were reviewed and approved by University of Montreal Hospital Research Center Ethical Committee. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.EB and YZ wrote the main manuscript text. EB formatted the manuscript and submitted it. YZ and MR retrieved the data. YZ performed the analysis. FL and PP acted as consultants for the study design, signal processing, and statistical analysis. DKN annotated the interictal epileptiform discharges and was the principal investigator of this study. All authors reviewed the manuscript. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


The exponential growth of the biomedical literature necessitates investigating strategies to reduce systematic reviewer burden while maintaining the high standards of systematic review validity and comprehensiveness.We compared the traditional systematic review screening process with (1) a review-of-reviews (ROR) screening approach and (2) a semi-automation screening approach using two publicly available tools  and different types of training sets . We evaluated performance measures of sensitivity, specificity, missed citations, and workload burdenn = 125) performed similarly to a larger training set of random citations (n = 938).The ROR approach for treatments of early-stage prostate cancer had a poor sensitivity (0.54) and studies missed by the ROR approach tended to be of head-to-head comparisons of active treatments, observational studies, and outcomes of physical harms and quality of life. Title and abstract screening incorporating semi-automation only resulted in a sensitivity of 100% at high levels of reviewer burden (review of 99% of citations). A highly curated, smaller-sized, training set (Two approaches to rapidly update SRs—review-of-reviews and semi-automation—failed to demonstrate reduced workload burden while maintaining an acceptable level of sensitivity. We suggest careful evaluation of the ROR approach through comparison of inclusion criteria and targeted searches to fill evidence gaps as well as further research of semi-automation use, including more study of highly curated training sets. Many clinical guideline and policy statements rely on a systematic evidence review (SR) that synthesizes the evidence base. SRs are labor-intensive projects due to high standards of validity and comprehensiveness. Guidance from the Institute of Medicine and the Effective Health Care (EHC) Program favor high sensitivity of literature searches and screening over specificity. All citations require dual review of abstracts and articles at each step of screening to identify and include all relevant research in the SR –3. With traditional search approach involves conducting searches in multiple databases, including searches of databases, grey literature sources, as well as a ROR: an approach intensive in labor and time. The ROR approach involves identifying and selecting SRs to either use in their entirety or to rely on for identifying relevant primary studies. Because the yield of eligible systematic reviews is smaller than comprehensive database searches, the effort requires less labor and time than a traditional approach but has the potential to miss relevant citations. Semi-automation screening software uses text-mining algorithms to find patterns in unstructured text and machine learning (ML) to train predictive classification algorithms to make inclusion and exclusion decisions or prioritize relevant citations in the title and abstract screening step of an SR [In this study, we examine strategies to reduce workload burden of title and abstract screening as an adjunct investigation in parallel to conducting a traditional update SR on treatments for early-stage prostate cancer. Early-stage prostate cancer is a common but indolent disease that may remain clinically silent for a man’s lifetime; thus, the decision to treat clinically localized prostate cancers balances the risk of common treatment side effects (such as urinary incontinence or erectile dysfunction) with the less common risk of cancer progression and death –6. In thof an SR . Semi-auof an SR . Active of an SR .In this study, we compare the traditional SR screening process with (1) a ROR screening approach and (2) a semi-automation screening approach using two publicly available tools . With respect to semi-automation approaches, we evaluate training sets composed of randomly selected citations from the traditional database search, highly curated citations identified from full-text review, and a combination of the two to examine ways reviewers can practically incorporate ML tools into the review workflow. Similarly, ML algorithms predict a probability that a citation will be included in the final SR, and we examine different thresholds for inclusion to understand the trade-off between inclusion probability thresholds and sensitivity when incorporating ML into SRs. We evaluate performance measures of sensitivity, specificity, missed citations, and workload burden to assess if either approach could maintain the sensitivity of a traditional review approach while reducing the number of full-text articles a reviewer would have to read.clinicaltrials.gov, and AHRQ and PCORI projects. Appendix 2 lists our inclusion and exclusion criteria. For studies identified through the database search, we dually and independently reviewed titles and abstracts against the inclusion and exclusion criteria. Articles flagged for inclusion by either reviewer then moved to full-text review. Each full-text article was then dually and independently reviewed. Consensus or third-party adjudication resolved conflicts. The results of the ROR (described below) were included in the final yield of the traditional review.We first conducted the traditional database search from 1 January 2014 to 27 November 2018, beginning a few months prior to the most recent reviews on the topic [For the ROR, two team members independently selected SR citations from the database search for identification of primary research studies for further title and abstract screening and potential full-text review. The same team members conducted both the traditional search and the hand search to ensure greater consistency for the traditional search and ROR. The results of the ROR approach  were compared to the traditional review approach .RobotAnalyst is publicly available, web-based software, developed by the National Centre for Text Mining, with a growing evidence base across several topic areas [AbstrackR is an open-source tool for facilitating the citation screening process that has an active learning setting designated at the start of the review. After researchers screen a certain number of randomly selected citations, the active learning algorithm assigns a score to unlabeled citations analogous to the inclusion prediction generated by RobotAnalyst, and selects the next citation for the reviewer to screen based on a priority ranking table stored in the project database. AbstrackR periodically calls on the ML library to refine the ML algorithm and re-rank the unlabeled citations based on additional human screening decisions. The AbstrackR active learning system performs citation re-prioritization asynchronously from human review due to the computational cost, which could slow down the program and impede efficient expert review [For the semi-automation approach, we used two software programs that apply machine learning for title and abstract screening and are readily available for current use. ic areas , 13. Aftt review .These two tools were chosen for their public availability. AbstrackR, specifically, is supported by the Agency for Healthcare Research and Practice, free to users, and is a familiar interface to Evidence-based Practice Centers for title-abstract screening. Our tests of each program included (1) creating training sets of dually reviewed abstracts, (2) invoking ML to predict inclusion probabilities of the unlabeled citations, and (3) comparing the semi-automated approach to a traditional review approach.n = 939). Prior studies using smaller sized training sets (< 20%) have shown poor sensitivity [For the RobotAnalyst tests Fig. , we creasitivity , 16. At n = 125, 4%), specifically inclusion after full-text review. The advantage of this training set is that it contains citations that have been dually reviewed in two stages and does not include any false positives . This training set involves greater effort in generating the curated ROR set and simulates combining an established rapid review method with semi-automation.The second training set used inclusion decisions from the ROR . This last training set builds on the ROR training set and, as with the ROR training set, there are no false-positive labels used to train the ML algorithm. This strategy is closest in workflow to traditional systematic review processes that do not employ semi-automation, while potentially reducing the number of citations to screen by over half, though does require the greatest effort to create the training set.The training sets were uploaded to RobotAnalyst, the reviewer explicitly called on RobotAnalyst to train the prediction model and update unlabeled citations with an inclusion prediction probability, and the probabilities were exported from RobotAnalyst for analysis. To compare a semi-automated approach to the traditional review approach, we chose three thresholds for the inclusion prediction probability: 30%, 40%, and 50%. The default setting in RobotAnalyst is 50%, and we examine two lower thresholds to assess the impact on sensitivity and reviewer burden.For the AbstrackR semi-automation test, a simulation program was used to estimate outcomes for multiple training sets, beginning with 500 randomly selected citations and ranging up to 3100 citations. Figure  Because Figure We present results for the ROR and semi-automation approaches compared with the traditional approach. Outcomes for the semi-automation approach can be calculated for the entire process—dual human review of the training set and incorporating ML to predict inclusion of remaining citations—or for the performance of the ML algorithm alone. We present the outcomes for the entire process as our main findings because it best represents how semi-automation would be employed in current practice.  Figure n = 125 vs. 3181, burden 4%, time savings 25.5 h), though had a low sensitivity (56%) and 26 missed citations. Table Figures n = 16); among this subset of more recent studies, 11 studies had an active treatment comparator. Missed studies were much more likely to be observational studies versus trials (88 vs. 8%). Among the 25 uniquely missed studies, 16 were cohort studies (Appendix 4). The outcome of interest in 75% of the missed cohort studies were harms—quality-of-life outcomes measuring urinary or erectile dysfunction, psychological harms such as mental health disease or decision regret, and procedure—or medication-related harms.Table The results of the semi-automation test using RobotAnalyst with a training set of random citations Table  illustraThe semi-automation test using RobotAnalyst with the ROR training set showed similar trends of increased sensitivity at lower thresholds of inclusion probability (100% sensitivity for 30% inclusion probability threshold vs. 69% sensitivity for 50% inclusion probability threshold), though higher reviewer burden and minimal time saved  (Table n = 938), the ROR training set was considerably smaller (n = 125), though demonstrated similar performance in terms of sensitivity and missed citations, as well as similar estimates of burden and time savings. Plotting sensitivity against burden shows that the semi-automation approach using a ROR training set performs slightly better than using a random dataset  compared with a training set of random citations (n = 938). . Appendix Table 2. Cochrane search string and yield (11/27/2019). Appendix Table 3. AHRQ Evidence Reports and Technology Assessments Search String and Yield (1/7/2019). Table 4. Drug Effectiveness Review Project Drug Class Reviews Search String and Yield (1/7/2019). Table 5. National Guidelines Search String and Yield (1/7/2019). Table 6. ClinicalTrials.gov Search String and Yield (1/10/2019). Table 7. HSRPproj Search String and Yield (1/10/2019). Table 8. PCORI Portfolio Search String and Yield (1/10/2019). Appendix 2: Inclusion and Exclusion Criteria. Appendix 3. Outcomes for Machine Learning Algorithm Only. Table 3A. Semi-automation test with RobotAnalyst using a Training Set of Randomly-Selected Dually-Reviewed Citations with Labels from Title and Abstract Screening: Outcome Metrics for Machine Learning Algorithm Only. Table 3B. Semi-automation test with RobotAnalyst using a Training Set of Dually-Reviewed Citations from a Review-of-Review with Labels from Full Text Screening: Outcome Metrics for Machine Learning Algorithm Only. Table 3C. Semi-automation test with RobotAnalyst using a Training Set of Dually-Reviewed Citations from a Review-of-Review and Randomly-Selected Citations with Labels from Full Text Screening: Outcome Metrics for Machine Learning Algorithm Only. Appendix 4. Cohort Studies Missed by Review of Reviews Approach. Appendix 5: Table AbstrackR Outcomes for Each Training Set.