The recent covid crisis has provided important lessons for academia and industry regarding digital reorganization. Among the fascinating lessons from these times is the huge potential of data analytics and artificial intelligence. The crisis exponentially accelerated the adoption of analytics and artificial intelligence, and this momentum is predicted to continue into the 2020s and beyond. Drug development is a costly and time-consuming business, and only a minority of approved drugs generate returns exceeding the research and development costs. As a result, there is a huge drive to make drug discovery cheaper and faster. With modern algorithms and hardware, it is not too surprising that the new technologies of artificial intelligence and other computational simulation tools can help drug developers. In only two years of covid research, many novel molecules have been designed/identified using artificial intelligence methods with astonishing results in terms of time and effectiveness. This paper reviews the most significant research on artificial intelligence in de novo drug design for COVID-19 pharmaceutical research. In December 2019, in Wuhan, China, a was patient diagnosed with atypical pneumonia. Just a few months later, on 11 March 2020, the COVID-19 pandemic was announced by the World Health Organization. Today, it is still a considerable concern for humanity ,4. Drug AI refers to machines, mainly computers, working like humans. In AI, machines execute tasks such as speech recognition, solving problems, and learning. Machines can work and act like humans if they have enough instruction and knowledge. AI systems can be divided into four groups based on machines’ ability to use past experiences to predict future decisions: (1) Machine learning refers to AI where machines are not explicitly programmed to perform tasks, but they learn and improve from training automatically. Deep learning is a subdivision of machine learning based on artificial neural networks for predictive analysis. (2) Natural language processing (NLP) is the interaction between computers and human language. (3) Automation and robotics aim to allow monotonous and repetitive tasks to be performed by machines. (4) Machine vision—machines can capture visual information and then analyze it . AI has In the field of computer-aided drug discovery (CADD), the scientific community worldwide is regularly developing new technologies and algorithms to obtain hit compounds in a short time and reduce the overall cost . NowadayAI and classical CADD tools can be used alone or combined to produce new approaches that integrate a broad range of algorithms with enhanced prediction capabilities. CADD is classically classified into two methods: structure-based drug design and ligand-based drug design. Both are faces of the same coin and massively rely on force fields, scoring functions, and algorithms to evaluate and rank the studied molecules’ energy contribution in the targeted macromolecular biological system. While computer-aided structure-based drug design  depends on the actual 3D structure of the targeted binding site of the targeted receptor protein to understand the stabilizing interactions at the molecular level between the studied ligand/receptor system, the ligand-based drug design  approach relies on the recognition of a database of already-known ligands interacting with the target receptor. Both structure- and ligand-based technologies have several success stories and play key roles in the drug modern drug discovery process ,12,13,14The SARS-CoV-2 spike protein (S protein) is the leading mediator of viral entry into cells, and thereby infection, by binding the human angiotensin-II converting enzyme (ACE2) and therefore represents an attractive target for drug therapies . RecentlSeveral molecules with good scores were selected and compared to FDA-approved drugs and the BindingDB dataset. Despite identifying compounds with great binding affinity, MTNN performance had to be improved using active learning and extending the training data to have a correlation between the SMILES string and Vina scores considering the larger domain of SMILES space. This method increased the number of molecules selected to over 300,000. Of these, 200 promising molecules were chosen using Vina scores as selection criteria.This method accelerated the exploration of the vast chemical space represented by SMILES strings to evaluate structural features and discover structural similarities between top-performing candidates.Many of the reported computational studies have selected molecules with high affinity to the spike protein. However, these studies are limited to the receptor ACE2/protein spike interface . This reTo overcome this, a new machine learning approach, namely SSnet, was used to identify new potential drugs by screening a library of approved drugs from the DrugBank and ZINC databases  targetinAfter that, to accelerate the identification of high-affinity scaffolds to test for their in vitro activity, a web interface, where molecules are grouped according to their similarity on a 2D map and colored based on binding affinity to the protein, was developed. This system allows selecting a certain point of the interface to explore the effect of singular scaffolds and functional groups on the binding score or affinity. Moreover, this approach can be applied to other therapeutical targets besides COVID-19.pro main protease  is a homodimeric cysteine protease representing an attractive target for trans-variant activity since no mutations have yet been observed in this protein.SARS-CoV-2 3CLpro inhibitors, viral protease inhibitors, or a new chemical space. This approach has the advantage of taking into consideration potential drugs that would otherwise have been omitted and gaining information into the possible mechanism of action of the selected compounds pyrano chromene (Reaxys ID 7450892), notoamide I (Reaxys ID 19384758), hemindole SB beta-mannoside (Reaxys ID 26845562), and two derivatives of bromoindole  F. Pereira et al., succeeded in predicting five new inhibitors against SARS-CoV-2 Mpro [pro chemical space of SARS-CoV-inhibitors, and formation of a classifier for the prediction of bioactivity using transfer learning. Using the improved classifier to predict the bioactivity of the 70,000 valid SMILES, the authors classified 1697 active molecules, and the Uniform manifold approximation and projection (UMAP) plot showed good to optimal overlap between the predicted results and the actual value of inhibition for Mpro inhibitors in the studied chemical space. Among the resulting molecules, 20 compounds were classified as high-confidence hits, with probabilities ranging from 0.99 to 1.0. These molecules found to be potential inhibitors were then subjected to a docking simulation using the crystalline structure of SARS-CoV-2 Mpro (PDB: 6W79). Nine compounds showed binding poses similar to experimentally validated inhibitors in X-ray crystal complexes with Mpro. These molecules include three benzotriazoles and four benzothiazolylketones, a peptidomimetic, and an N-(2-pyridyl) acetamide derivative  starting from a dataset of approximately 1.6 million drug-like small molecules from the ChEMBL database, which were collected for pre-training of a generative model . The setThe unique approach to data curation coupled with random forest (RF) analysis by K. Cooper et al., produced a specific target and validated predictive fingerprints (PFFs) that have a high predictability value on multiple targets such as plasma kallikrein, HIV protease, NSP5, NSP12, AT-1, and the JAK family. This broad applicability to different biologically relevant targets , tyrosine kinase, and a phenotypic assay) as well as to different chemotypes within a target is an important strength and differentiator of the applied methodology. The capability of this methodology allows for each target to create a binary decision tree for inactive or active compounds or a ternary decision tree for weakly active, moderately active, and highly active compounds also suggests that the models could be used for virtual screening of libraries of target-specific compounds to select the most active compounds for synthesis and/or clinical testing. Regarding the SARS-CoV-2 target, about 5600 FDA-approved drugs were examined in this study. Molecules that showed more than 75% inhibition were considered active, and among the wide range of FDA-approved drugs, 267 were identified as active. Leveraging the bioactivity data of ChEMBL and a subset of the data, it was trained on physicochemical characteristics from the set of 110 chemical properties, and a set of 868 compounds was then used to establish a binary classification model capable of predicting whether a molecule was active or inactive in the test; overall the model was able to identify the bioactivity with an accuracy of 65% .pro with micromolar activity and to a pharmacophore model. The methodology consisted of a filtering system involving screening multiple receptors and ligands in combination with a final MD simulation to confirm the binding stability of the selected compounds. The structural screening was then integrated with machine learning-based screening for compound selection using a molecular-descriptive dataset derived from known ligands and non-ligands for 3CLpro. The best compounds selected using these in silico screening methods were then experimentally evaluated to determine the subset of stable ligands and their 3CLpro inhibitory activity using the in vitro Forster-type resonance energy transfer (FRET) assay. From the various screening analyses conducted, 95 molecules were identified and tested. Seven of these compounds were confirmed active, and one of them showed the lowest IC50 value of 31 µM [E. Glaab et al., reported a combined virtual screening study, molecular dynamics (MD) simulation, machine learning, and in vitro experimental validation analysis, which led to the identification of small molecule inhibitors of 3CLof 31 µM .pro were chosen to analyze the drug design capabilities of MolAICal. In this way, the software can generate various ligands that have an ever-higher 3D structural similarity with the ligand crystallized in the active site of GCGR or SARS-CoV-2 Mpro [The deep learning (DL) model can generate 1D or 2D sequences ligands structures. However, rational drug design requires 3D ligand structures that target the crystalline structure of proteins. To solve this problem, Q. Bai and coworkers developed MolAICal software, which allows to generate 3D drugs in the 3D pocket of protein targets by combining the merits of the deep learning model and classical algorithm. The software essentially consists of two modules. In the first one, FDA-approved drug fragments are used to train the Wasserstein-based deep learning model generative adversarial networks (WGANs). The generated fragments of the deep learning model are further used to grow the 3D ligands in the protein pocket. In the second module, drug-like molecules from the ZINC database are used to train the WGAN-based deep learning model. Then the affinities between the generated molecules and the proteins are evaluated through molecular docking experiments with AutodockVina. The membrane protein glucagon receptor (GCGR) and the non-membrane target SARS-CoV-2 MV-2 Mpro .pro inhibitors from a dataset of two million commercially available compounds. The model was able to classify two hundred new chemotypes as potentially active against the viral protease [Mekni N. et al., developed a machine learning approach using support vector machine (SVM) classification, to share new knowledge for designing novel Mprotease .https://github.com/NedraMekni/COVID-19, accessed on 10 February 2022).Feature selection was made by implementing a python3 script using the Sklearn libraries. The script is available in the GitHub repository  in order to select relevant molecular descriptors [The selection of the characteristics was based on the training set, with the aim of identifying the crucial molecular descriptors able to explain the possible correlation between the activity of the Mcriptors , leadingThe five best compounds selected by consensus were then subjected to molecular dynamics to investigate the stability of the binding interactions.MD simulations at 200 ns were performed on the two best performing PDBs (6WCO and 5RGW) to verify the stability of the interactions recovered within the crystal structure.The five best compounds selected by consensus were then subjected to molecular dynamics to investigate the stability of the binding interactions. It should be noted that the compounds selected by SVM showed all the essential interactions reported in the literature.Joel Kowalewski and Anandasankar Ray, collected test data from 65 known human target proteins that could interact with SARS-CoV-2 proteins, including the ACE2 receptor . Next, tKd 259 nM) to the spike protein using microscale thermophoresis. Several other selected compounds were tested in vitro by other research groups and found to be quite active. This combined machine learning approach and in vitro testing can be expanded to virtually screen available molecules with predicted activity against SARS-CoV-2 WIV04 and other circulating variants.S. Ekins et al., implemented several machine learning methods to develop predictive models from recent in vitro SARS-CoV-2 inhibition data and used them to prioritize additional FDA-approved compounds for in vitro testing selected from our in-house library of compounds . From thAn in silico deep learning approach was proposed to predict and design a multiepitope vaccine (DeepVacPred), in combination with in silico immunoinformatics and deep neural network strategies. The DeepVacPred computing system directly predicted 26 potential vaccine subunits from the SARS-CoV-2 tip protein sequence .The DNN architecture to block 26 fragments in the SARS-CoV-2 spike protein as candidates for the vaccine subunit was the first step proposed by the authors. Subsequently, linear B cell, CTL, and HTL epitopes were used to select and construct the final vaccine.All overlapping protein fragments with a length of 30 aa were generated by the spike protein sequence 1273 aa SARS-CoV-2. DeepVacPred first tested these protein sequences and predicted 132 potential vaccine subunits. Following this prediction, DeepVacPred provided 26 potential vaccine subunits for further evaluation and construction. These subunits were those most likely to contain B cell epitopes and multiple T cell epitopes and have high antigenicity and low allergenicity.In silico methods were used to study linear B cell epitopes, cytotoxic T cell epitopes (CTL), helper T cell epitopes (HTL) in the 26 candidate subunits.The B cell epitopes, predicted on the 26 vaccine subunits, are parts of antigens that bind to immunoglobulin or antibody, capable of activating B cells to provide the immune response . Linear CTLs recognize infected cells using class I MHCs to bind to certain CTL 26 epitopes. The NetMHCpan 4.1 server  43 was uHTL helps other immune cells’ activity and recognizes infection by using MHC class II to bind with specific HTL epitopes . The NetThe 3D structure of the designed vaccine was then predicted, refined, and validated by other in silico tools. The GalaxyRefine  server wIn conclusion, this proposed artificial intelligence (AI)-based vaccine discovery facility accelerated the vaccine design process and built a 694 aa multiepitope vaccine containing 16 B cell epitopes, 82 CTL epitopes, and 89 HTL epitopes, which promise to fight SARS-CoV-2 viral infection with good antigenicity, population coverage, and good physicochemical properties and structures, providing great potential for next stage of COVID-19 vaccine design with actual clinical trials.AI was used in another study to predict the rationale for designing universal vaccines against SARS-CoV-2, which contain an extensive repertoire of T-cell epitopes capable of providing coverage and protection to the global population. To achieve these goals, the authors profiled the entire SARS-CoV-2 proteome through the 100 most frequent HLA-A, HLA-B, and HLA-DR alleles in the human population, using the presentation of the cell surface antigen infected with host and immunogenicity predictors from the NEC Immune Profiler suite of tools and generated sufficiently complete epitope maps .50 nm binding affinity data) and 13 different antigen processing predictors. The emitted AP score ranged from 0 to 1 and was used as an input to calculate immune presentation (IP) through the epitope map, penalizing those peptides that have degrees of similarity when compared to the human proteome, and rewarding peptides that are less similar. The resulting IP score represents those presented HLA peptides that can be recognized by circulating T cells.Antigen (AP) presentation was predicted by a machine learning model that integrates information from several HLA binding predictors , S2 (686–1273), and RBD (319–541) regions of the spike protein, respectively.Therefore, the authors explored host responses to the virus in both cell lines (colon carcinoma-2 and H1299) and naturally infected COVID-19 patient samples. Many proteins are involved in biological processes related to the immune system, such as regulation of immune responses, leukocyte migration, autophagy, processing, immune system development, antigen presentation, or leukocyte-mediated cytotoxicity.323 and 143 human peptides were identified in the cell line and patients, respectively. Only five  were detected in both sample sets. While MDGA1 plays an important role in inhibitory synapse formation , PIK3C2AThirty-three proteins were found and matched entries in the InnateDB database. Most of these proteins are involved in immune-related functions such as protein binding , transferase activity , the protein dimerization  and lipopeptide binding  .Once the proteins related to the immune system process were identified from the patients’ cell line and proteomes, they were used to generate a protein interaction network. The generated protein interaction network, which includes 403 nodes and 671 edges, identified higher-ranking hubs and bottlenecks.Therefore, multi-step filtering was applied to identify potential diagnostic peptides. Initially, to avoid cross-reactivity, the identified peptides (442) were filtered to exclude peptides from the human and human saliva microbiome (418) and subsequently peptides from a targeted group of pathogenic bacteria and viruses (129). Subsequently, using the results of the RNA-Seq data analysis of the infected cell lines, the expression of the selected peptides was verified to avoid the selection of poor peptides for diagnostic purposes. Then, four antigenic peptides were selected for attachment to known viral T-cell receptor (TCR), class I and II peptide major histocompatibility complex (pMHC), and paratopes sequences identified. They also tested the paratope binding affinity of SARS-CoV-1 T and B cell peptides that had previously been experimentally validated. The resulting antigenic peptides have a high potential for generating antibodies specific to SARS-CoV-2, and the peptides of the paratopes can be used directly to develop a COVID-19 diagnostic assayThe antigenic peptides found in this study have a high potential for generating antibodies specific to SARS-CoV-2, and the paratopes peptides can be used directly to develop a COVID-19 diagnostic assay. In addition, the paratope binding affinity of SARS-CoV-1 T and B cell peptides that had previously been experimentally validated was also tested. The assembled paratopes showed a greater binding affinity for SARS-CoV-2 antigens and proteins than SARS-CoV-1.In conclusion, the authors in this study explored both the cell line and proteomes of naturally infected COVID-19 patients via in silico methods and identified four SARS-CoV-2 antigens and three antigen-binding peptides that could be used to develop diagnostic assays. The proposed antigenic peptides can be used for antibody generation, and the paratopes sequences can be used directly for COVID-19 diagnostic test and vaccine development.AI technologies have shown impressive ability in COVID-19 research, from real-time tracking of the virus spread to the development of novel drugs and vaccines faster than ever before. Unfortunately, the high number of positive cases and deaths from COVID-19 infection and the absence of effective treatments and complete cover by vaccination persist in influencing global health, necessitating the discovery of novel molecules for the cure and prevention of the infection. Structural definition of targetable proteins for therapy and prevention of the infection has recently boosted the structure-based virtual identification of small molecules. The discovery of some promising compounds has driven the optimization of those scaffolds through ligand-based drug design.In this review, we reported and discussed the more cutting-edge technologies in the field of COVID-19 de novo drug design, from ligand-based AI technologies, where only the ligands’ structures are considered to train the models, to structure-based AI technologies, where the protein target is also taken into account in the process of drug design, and to vaccine design aided by AI. ML on its own and other AI-based technologies are of critical importance in responding to problems in COVID-19 research. They are proven to be efficient tools to quickly analyze large amounts of data, estimate drug repurposing against COVID-2019, identify the association of these repurposed drugs, or estimate dosage adjustments and other clinical issues such as early diagnosis, identifying people at risk, and predicting disease evolution ,76.The de novo drug design field is also part of the AI-lead research in the COVID-19 field. As reported in this review, several molecules have already been identified from impressively large databases of billions of compounds. Additionally, pairing different approaches considering all the information produced by omics sciences should lead to developing personalized strategies. Due to the mutability of this RNA virus and the emergence of drug resistance problems, it is mandatory to start considering targeting multiple targets which may be more effective and help in overcoming future drug resistance.https://clinicaltrials.gov/, accessed on 10 February 2022) [Interestingly, although none of the de novo-identified molecules has entered clinical trials, some of the repurposed molecules identified by AI technologies have entered this phase. These are mainly already used and approved antibiotics, anti-inflammatory, antivirals, anticancer, and ACE2, and some other drugs, that are already in clinical trials according to ClinicalTrials.gov (ry 2022) . In conc


A backbone-side-chain elastic network model (bsENM) is devised in this contribution to decipher the network of molecular interactions during protein dynamics. The chemical details in 5 μs all-atom molecular dynamics (MD) simulation are mapped onto the bsENM spring constants by self-consistent iterations. The elastic parameters obtained by this structure-mechanics statistical learning are then used to construct inter-residue rigidity graphs for the chemical components in protein amino acids. A key discovery is that the mechanical coupling strengths of both backbone and side chains exhibit heavy-tailed distributions and scale-free network properties. In both rat trypsin and PDZ3 proteins, the statistically prominent modes of rigidity graphs uncover the sequence-specific coupling patterns and mechanical hotspots. Based on the contributions to graphical modes, our residue rigidity scores in backbone and side chains are found to be very useful metrics for the biological significance. Most functional sites have high residue rigidity scores in side chains while the biologically important glycines are generally next to mechanical hotspots. Furthermore, prominent modes in the rigidity graphs involving side chains oftentimes coincide with the co-evolution patterns due to evolutionary restraints. The bsENM specifically devised to resolve the protein chemical character thus provides useful means for extracting functional information from all-atom MD. These functionally important features are sensitive to mutation and can trace their origin to both the polypeptide backbone that frames the structure and the side chains that define the chemical specificity Yet, beyond the backbone-framed protein structure, how to delineate the networks of molecular interactions, what are their differences in comparison to the structural network, and what are the manifestations of the side-chain sequence and dynamical motions? In particular, if the interaction network of side chains could be studied, its properties are expected to differ from that of the backbone. To address these key issues of the sequence-structure-function relationship, a graph-theoretic methodology is devised here to compute the mechanical interactions mediated by side chains and backbone from all-atom molecular dynamics (MD) simulations in an explicit solvent.The guiding principle is that given the unique structural position and chemical environment of a protein residue, its couplings with surrounding atoms would assume specific strength during dynamical motions. A mechanical-coupling dynamics perspective is thus taken to unravel the network behaviors of physical interactions. In particular, the elastic parameters in a model of backbone and side-chain nodes connected by harmonic springs, i.e., the backbone-side-chain elastic network model, bsENM, proposed in this work and illustrated in 2The elastic parameters in bsENM (2.1The X-ray structure of BPTI bound RT (PDB ID: 3TGI) is used to construct its all-atom model 2.2With the raw data of protein dynamics generated with full atomic details, the CG sites in bsENM serve to read out the statistics of inter-site elasticity. The goal is to capture the significant mechanical couplings that can survive thermal noise, and the CG sites are thus located where specific molecular interactions are typically observed in the trajectories. In particular, the backbone is represented by two coarse-grained (CG) sites at the amide nitrogen and carbonyl oxygen positions as they are the loci of hydrogen bonding, 0. To highlight the effects of chemical details within the native topology, the bsENM statistically learned from all-atom MD is compared with the bsENM0 of the same equilibrium structure and Given a set of bsENM configurations, the spring length 2.30, 0, the residue contact number of CG sites within In our graphical representation of inter-residue interaction networks, the edge weights between residue nodes 2.4A protein of 0 Laplacian, then The similarity of graph 2.5With the 5 μs production runs of RT and PDZ3, a single bsENM using the entire trajectory only provides an equilibrium harmonic approximation for structural fluctuations, omitting the potentially interesting and informative fluctuations on the interaction-network level. This limitation is overcome by dividing the trajectory into consecutive 10-ns windows to compute a series of rigidity graphs 2.6The set of prominent modes in a 3With the all-atom MD simulation, structure-mechanics statistical learning, and rigidity graph analysis for both the RT and the PDZ3 proteins, the main text primarily uses RT for introducing the rich and quantitative information made available by our new approach. The two systems illustrate the common general features of sparse mechanical coupling network, scale-free network coupling strengths, hotspots in both backbone and side-chain interaction networks, and the residue rigidity scores during protein-dynamics as a new metric for biochemical functions. In what follows, the mechanical coupling network is analyzed in detail.3.1Our structure-mechanics statistical learning with bsENM provides a way to map the chemical details during protein dynamics onto 0 from topological contacts  as the reference. For the lowest-frequency modes of Although the inter-residue Laplacian matrix  and are the least-tailed group, For the protein mechanical coupling networks of a fixed number of nodes (residues) to have scale-free behaviors, their edge weights exhibit high-strength tails as shown in In the 5 μs dynamics of PDZ3, scale-free behaviors of the BB, BS, and SS rigidity graphs are similarly observed, The results of both RT and PDZ3 show that the mechanical couplings of backbone and side chains have different network properties. To further illustrate this point, specific patterns in the 3.32+ ion in RT is also treated as an additional residue. It is thus straightforward to adapt the bsENM-graph framework for studying complex protein systems. We focus on the modes of RT residues, and those of BPTI only are not presented.Graphical analysis of the bsENM quantitatively reveals the backbone and side-chain contributions in the mechanical coupling network. The high-strength tails in the eigenvalue distributions imply that their mechanical coupling patterns are more resistant to thermal noises during protein dynamics. The averaged mean-mode contents during protein dynamics, The trypsin fold of RT containing NT and CT barrels A is a usSpectral analysis of The above results of backbone and side chains having distinct mechanical coupling networks provide molecular basis for their separate adjustability as empirically adopted in protein engineering and design 3.4The RT rigidity graphs reveal that the prominent couplings between RT and BPTI lie in the hydrogen bonding modes d K¯19SS . They arDuring the 5 μs all-atom trajectory of PDZ3, the mechanical hotspots are also captured as the significantly populated residues in the prominent modes, ipheries  rather tipheries B. While At the core of allosteric communication in proteins is the physical interactions that are persistent under thermal noise for connecting distal sites 3.5Being the strong mechanical couplings persistent through protein dynamics, the prominent modes of  see Eq. . The resIn contributing to the mechanical coupling network, From the 5 μs all-atom MD data of PDZ3, the mechanical hotspots covering most of its functional residues Although most of the characterized residues are in the 4Given a protein fold, residue contact numbers center around a value due to the packing density. But then, what is the manifestation of sequence specificities in the structure? This question is addressed here by developing a bsENM with structure-mechanics statistical learning to compute the elastic parameters from 5 μs all-atom MD simulation in explicit solvent. To analyze the network behaviors of the complicated molecular interactions in structural fluctuations, the newly devised graph-theoretic framework introduces the concept of protein rigidity graphs. A key discovery is that the chemical details during protein dynamics render scale-free network properties in the mechanical coupling strengths of both backbone and side chains. In the nano-scale network of a single protein, exhibition of small-world-like features has not been shown to the best of our knowledge. The significantly populated residues in the statistically prominent modes of rigidity graphs are thus recognized as mechanical hotspots.Furthermore, our bsENM-graph approach enables the direct comparison of backbone and side-chain mechanical couplings to accentuate their differences. Such outcomes point to an important notion that protein residues have diverse combinations of backbone and side-chain contributions to the mechanical coupling network as seen in the ot of RT  and PDZ3ot of RT . Encouraot of RT , i.e., tNixon Raj: Conceptualization, Methodology, Software, Writing - original draft. Timothy Click: Methodology, Software. Haw Yang: Conceptualization, Writing - review & editing. Jhih-Wei Chu: Conceptualization, Methodology, Software, Writing - original draft, Writing - review & editing, Supervision.The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.


In the original article, there was a mistake in Figure 1H as published. The graph incorrectly depicting higher serum HDL-cholesterol levels in control AAV mice on standard diet compared to the rest of the experimental groups. The corrected Figure 1H appears below. It depicts higher serum HDL-cholesterol levels in the mPCSK9-AAV mice on HFD compared to the rest of our experimental groups.Results, Elevated PCSK9 plasma levels and hypercholesterolemia induced by mPCSK9-AAV expression and intake of Western-type HFD, paragraph 2:A correction has been made to “Furthermore, mPCSK9-AAV injected mice on the HFD had elevated total, HDL-cholesterol and non-HDL cholesterol serum levels .”Discussion, paragraph 3:Furthermore, a correction has been made to the “Concomitantly, the non-HDL-cholesterol and HDL-cholesterol concentrations were significantly higher in mPCSK9-AAV mice on HFD compared to control AAV mice on standard diet .”The authors apologize for this error and state that this does not change the scientific conclusions of the article in any way. The original article has been updated.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


This qualitative study describes the process of building EVOLV-Rx  with a panel of clinicians, deprescribing experts, health care leaders, and other stakeholders. Can a scalable and broadly applicable metric be developed using common forms of health care data to detect low-value prescribing among older adults?In this qualitative study of 27 low-value prescribing practices, a panel of experts was convened using an online modified-Delphi approach to develop a scientifically valid and clinically useful low-value prescribing metric. This panel identified the 18 most salient low-value prescribing practices, which were included as a component of the final metric.The metric developed in this study may enhance the detection of low-value prescribing practices, reduce polypharmacy, and enable older adults to receive high-value care. Metrics that detect low-value care in common forms of health care data, such as administrative claims or electronic health records, primarily focus on tests and procedures but not on medications, representing a major gap in the ability to systematically measure low-value prescribing.To develop a scalable and broadly applicable metric that contains a set of quality indicators (EVOLV-Rx) for use in health care data to detect and reduce low-value prescribing among older adults and that is informed by diverse stakeholders’ perspectives.This qualitative study used an online modified-Delphi method to convene an expert panel of 15 physicians and pharmacists. This panel, comprising clinicians, health system leaders, and researchers, was tasked with rating and discussing candidate low-value prescribing practices that were derived from medication safety criteria; peer-reviewed literature; and qualitative studies of patient, caregiver, and physician perspectives. The RAND ExpertLens online platform was used to conduct the activities of the panel. The panelists were engaged for 3 rounds between January 1 and March 31, 2021.Panelists used a 9-point Likert scale to rate and then discuss the scientific validity and clinical usefulness of the criteria to detect low-value prescribing practices. Candidate low-value prescribing practices were rated as follows: 1 to 3, indicating low validity or usefulness; 3.5 to 6, uncertain validity or usefulness; and 6.5 to 9, high validity or usefulness. Agreement among panelists and the degree of scientific validity and clinical usefulness were assessed using the RAND/UCLA  Appropriateness Method.Of the 527 low-value prescribing recommendations identified, 27 discrete candidate low-value prescribing practices were considered for inclusion in EVOLV-Rx. After round 1, 18 candidate practices were rated by the panel as having high scientific validity and clinical usefulness (scores of ≥6.5). After round 2 panel deliberations, the criteria to detect 19 candidate practices were revised. After round 3, 18 candidate practices met the inclusion criteria, receiving final median scores of 6.5 or higher for both scientific validity and clinical usefulness. Of those practices that were not included in the final version of EVOLV-Rx, 3 received high scientific validity (scores ≥6.5) but uncertain clinical usefulness (scores <6.5) ratings, whereas 6 received uncertain scientific validity rating (scores <6.5).This study culminated in the development of EVOLV-Rx and involved a panel of experts who identified the 18 most salient low-value prescribing practices in the care of older adults. Applying EVOLV-Rx may enhance the detection of low-value prescribing practices, reduce polypharmacy, and enable older adults to receive high-value care across the full spectrum of health services. This exclusion represents a major gap in the ability to systematically detect low-value prescribing and low-value care across the full spectrum of health services.4Low-value care, which is defined as the use of health services whose harms or costs exceed their benefits, is a major factor in wasteful health care spending and has been associated with physical, psychological, and financial harms.6 In addition, 35% of Medicare Part D beneficiaries have experienced hardship in paying for their medications.7Characterizing and reducing low-value prescribing are especially important for individuals who are 65 years or older. More than 40% of older adults have been subjected to polypharmacy (prescribed ≥5 medications), and 30% to 50% have been prescribed a potentially inappropriate medication, increasing their risk for adverse drug events and hospitalizations.8 Many current recommendations, such as avoiding the use of testosterone to treat nonspecific symptoms of aging, provide only general guidance to clinicians and have not been operationalized to measure low-value prescribing using administrative claims or electronic medical record data, which would ensure the scalability and automation of these measures.9 Other tools, such as the Beers Criteria,10 may be applied to health care data but focus primarily on medication safety and do not incorporate other components of health care value, such as cost. The perspectives of patients or their caregivers were not integral to the development of these and other low-value care metrics.9There is currently no metric that consolidates and prioritizes the hundreds of recommendations from professional societies and the Choosing Wisely campaign for health systems and third-party payers to detect, quantify, and reduce low-value prescribing.In this qualitative study, we aimed to develop a scalable and broadly applicable metric that contains a set of quality indicators to detect and reduce low-value prescribing among older adults and that is informed by diverse stakeholders’ perspectives. We named this metric EVOLV-Rx .13 Second, using an online modified-Delphi approach, we convened a panel of physicians and pharmacists to codify the final components of EVOLV-Rx , peer-reviewed literature, and the results of 3 qualitative studies on low-value prescribing.EVOLV-Rx . Members14 We considered prescribing practices whose costs or harms generally outweigh their benefits for older adults (aged ≥65 years). Practices were assessed on the following 4 value domains, which were adapted from the Lown Institute approach to characterizing medication appropriateness15: (1) lack of effectiveness for a stated indication, either in general or because of inappropriately prolonged use; (2) potential for harm because of the risk of an adverse drug event, drug-drug or drug-disease interaction, or overly intensive treatment; (3) excessive cost because of the unnecessary use of a costly brand-name medication or specific preparation; or (4) use of a medication to treat the adverse effects of another medication as part of a prescribing cascade.We compiled a collection of low-value prescribing recommendations according to previously applied approaches to identifying low-value health services.8 and the most up-to-date versions of the following medication safety and appropriateness guidelines: the Beers Criteria,10 FORTA (Fit for the Aged) list,16 STOPP/START (Screening Tool of Older Persons’ Prescriptions and Screening Tool to Alert to Right Treatment),17 and STOPPFrail (Screening Tool of Older Persons’ Prescriptions in Frail Adults With Limited Life Expectancy).18 We also considered the findings from published studies and medications or medication classes that were most associated with adverse drug events requiring hospital admission.19 We integrated the findings from 3 qualitative studies (which were conducted by some of us and other members of our research team), which used focus groups with patients and caregivers as well as semistructured interviews with primary care physicians to characterize their perspectives on and specific examples of low-value prescribing . We also included medications whose frequency of use was comparable to drugs that were most frequently prescribed in Medicare Part D but may be purchased over the counter, such as aspirin or docusate sodium.All of the investigators on the research team met monthly to generate operational definitions that could be applied to health care data for each candidate low-value prescribing practice. An external advisory panel of 6 geriatricians, pharmacoepidemiologists, and medication safety experts also met every 3 months to provide general feedback and ensure that EVOLV-Rx remained focused on the prescribing practices that were most relevant to older adults.21 The goal of the sensitive criteria was to identify patients who could be subjected to potential low-value prescribing, whereas the goal of the specific criteria was to identify the subset of older patients for whom the prescribing practice would most likely be of low value, according to a priori low-value criteria we had established. For example, regarding the prolonged use of proton pump inhibitors (PPIs), the proposed sensitive criteria may identify patients who were prescribed a PPI for more than 2 months. Recognizing that the prolonged use of a PPI may at times be clinically appropriate, the specific criteria would build on the sensitive criteria to recognize situations in which PPI use would most likely be of low value, such as among patients without a guideline-concordant indication, without concurrent long-term use of NSAIDs and corticosteroids, or with a prescription for a brand-name PPI.To define each candidate low-value prescribing practice, we developed both sensitive and specific criteria, applying an approach developed by Schwartz et al.To facilitate the deliberations of the online modified-Delphi panel, we generated a 2-page peer-reviewed infographic for each candidate low-value prescribing practice. The infographic contained the proposed sensitive and specific criteria as well as an evidence-based synopsis and related utilization, cost, and qualitative data  and provided corresponding comments to explain their ratings  between the 70th and 30th percentiles of panelist scores. Next, we calculated the IPR adjusted for symmetry with the following equation: IPRAS = 2.35 + (AI × 1.5), where AI represented the asymmetry index, which is defined as the distance between the central point of the IPR and 5, the central point of the 9-point Likert scale used by panelists to rate each candidate low-value prescribing criteria. If the IPR was greater than the IPR adjusted for symmetry, then there was no agreement. If the IPR was less than the IPR adjusted for symmetry, then there was agreement. For the low-value prescribing criteria wherein agreement was found, we characterized their scientific validity and clinical usefulness according to the median scores. The final metric included only candidate low-value prescribing practices that showed panel agreement and received median scores of 6.5 or higher, which indicated both high scientific validity and high clinical usefulness.A total of 527 low-value prescribing recommendations were compiled from the Choosing Wisely campaign, other medication safety criteria, and peer-reviewed literature, along with an additional 101 medications or medication classes that were cited as being potential low-value prescribing practices by the participants of the qualitative studies we examined  were rated as having high scientific validity but uncertain clinical usefulness  are sensitive to the composition of the Delphi panel, and it is not known to what degree these results would be different with a different set of experts. Fourth, EVOLV-Rx has not yet been fully operationalized for use in administrative claims or electronic health record data. The research team is currently working to operationalize and establish the specification validity of EVOLV-Rx. Fifth, each component of EVOLV-Rx may not be applicable in all forms of health data  or in all populations, which was a necessary tradeoff in developing criteria that best characterize each form of low-value prescribing.In this qualitative study, a panel of experts identified 18 scientifically valid and clinically useful prescribing practices and related criteria for detecting low-value prescribing practices in the care of older adults, resulting in the development of the EVOLV-Rx metric. The application of EVOLV-Rx may also enhance the detection of low-value prescribing alongside other low-value tests and procedures, reduce polypharmacy, and enable older adults to receive high-value care across the full spectrum of health services in a way that aligns with their perspectives and values.