Developing new energy vehicles (NEVs) is necessary to grow the low-carbon vehicle industry. Many concentrated end-of-life (EoL) power batteries will cause large-scale environmental pollution and safety accidents when the time comes to replace the first generation of batteries if improper recycling and disposal methods are utilized. Significant negative externalities will result for the environment and other economic entities. When recycling EoL power batteries, some countries need to solve problems about lower recycling rates, unclear division of echelon utilization scenarios, and incomplete recycling systems. Therefore, this paper first analyzes representative countries’ power battery recycling policies and finds out the reasons for the low recycling rate in some countries. It is also found that echelon utilization is the critical link to EoL power battery recycling. Secondly, this paper summarizes the existing recycling models and systems to form a complete closed-loop recycling process from the two stages of consumer recycling and corporate disposal of batteries. The policies and recycling technologies are highly concerned with echelon utilization, but few studies focus on analyzing application scenarios of echelon utilization. Therefore, this paper combines cases to delineate the echelon utilization scenarios clearly. Based on this, the 4R EoL power battery recycling system is proposed, which improves the existing recycling system and can recycle EoL power batteries efficiently. Finally, this paper analyzes the existing policy problems and existing technical challenges. Based on the actual situation and future development trends, we propose development suggestions from the government, enterprises, and consumers to achieve the maximum reused of EoL power batteries. Under the dual pressure of global ecological degradation and the energy crisis, the world landscape is undergoing significant and profound changes. The global transportation sector will consume 60% of total oil consumption, contributing 26% of carbon dioxide emissions, and has become the major source of urban air pollution ,2,3,4. AA strategic emerging industry, the NEV industry has received substantial support from government . In 2021In order to reduce the negative externalities of EoL power batteries to the environment and relieve the pressure caused by insufficient raw materials, there has been widespread attention given to the issue of reuse and recycling of EoL power batteries. Existing studies have also confirmed the necessity and feasibility of the economic, environmental, and resource benefits . Neubaue(1)The recycling rate in some countries is relatively low.(2)Countries worldwide are exploring the echelon utilization of EoL power batteries. However, the specific application scenarios and implementation process need to be clearly delineated.(3)To guide remanufacturing enterprises toward a sustainable direction, policies for recycling EoL power batteries should be formulated from a systems perspective in some countries.Therefore, it is highly significant to sort through current policies, recycling methods, and systems to classify different application scenarios and propose reasonable recycling systems for EoL power batteries. These actions will help to address the challenges in EoL power battery recycling operations and improve laws and regulations related to the recycling process.This paper is different from the above-mentioned literature studies as it provides a detailed overview of the power battery recycling policies of typical countries and analyzes the characteristics of countries with high recycling rates. Second, this paper provides a comprehensive summary of the existing recycling models and systems for power batteries to fill the gap in this field. Third, this paper clearly divides the application fields of power battery gradient utilization to clarify the specific classification. Fourth, this paper proposes the 4R power battery recycling system in the context of circular economy to provide a theoretical basis for industrial development.A country’s policies and regulations play a crucial role in developing emerging industries. Some countries have begun to pay attention to recycling EoL power batteries and have introduced many regulations. By analyzing the policies of the EU, U.S., Japan, and China, we summarize the policy characteristics of different regions and propose the development trend of recycling EoL power batteries.In 1991, the EU adopted its first Battery Directive (Council Directive 91/157/EEC), which emphasized the reduction of toxic emissions . The EndEU member states have also developed regulations that are appropriate for the development of their respective industries. Germany has extended its legal framework based on EU directives. The legal system specifies the registration requirements for battery manufacturers and retailers and the recycling responsibilities of battery manufacturers, retailers, and consumers. Importers and manufacturers of batteries must register with the government before production, retailers must work with manufacturers to educate consumers on how to recycle batteries, and consumers must return used batteries to designated recycling institutions . GermanyThe U.S. was the first nation to regulate the recycling of EoL batteries with laws. As a result, the U.S. has a practical legal framework, technical specifications, and a recycling system for managing the recycling of EOL power batteries. The U.S. primarily constructs its system at the federal, state, and local levels . There aThe “Resource Conservation and Recovery Act” and “Mercury-Containing and Rechargeable Battery Management Act” regulate federal recycling laws for EoL batteries. The acts mandate that battery manufacturers design batteries by keeping disassembly and recycling in mind and controlling EoL batteries in transportation, manufacturing, and recycling . The staThe U.S. Department of Energy recognizes the critical role of EoL power batteries in the clean energy industry. It is gradually focusing on applying EoL power batteries in the field of echelon utilization in light of the increasing popularity of NEVs. The “National Lithium Development Blueprint 2021–2030”, released by the Department of Energy in 2021, proposes to maximize the use of EoL lithium batteries, establish a special fund for battery recycling, recover vital raw materials, and develop a competitive industry chain for lithium battery recycling . The DepGeneral Motors (GM) has recognized the potential advantages of recycling power batteries and has invested in ‘Lithion Recycling’, which has a recycling rate of over 95%. GM utilizes advanced recycling technology to recycle battery materials and co-design recyclable power cells. GM ensures a stable vehicle production supply chain and reduces production costs by investing .The circular economy (CE) law in Japan is one of the most inclusive in the world. The Japanese legal system consists of three levels, from highest to lowest: basic law, comprehensive law, and special law, which form a product, consumption, reuse, and based circular system. The complete law framework of the CE addresses the negative externalities of batteries and provides specific operational guidance and a legal basis for battery recycling . Japan bJapan has pioneered the commercialization of hybrid vehicles to improve fuel efficiency and reduce greenhouse gas emissions. NiMH batteries are commonly used to power hybrid vehicles, so the initial recycling policy for power batteries centered on NiMH batteries . Since NAs Japan is prone to natural disasters, emergency equipment is in high demand. Nissan, Toyota, and Mitsubishi have implemented power batteries in the step-up and reuse industries. Nissan Motor and Sumitomo Corporation have established the 4R Energy joint venture. According to the remaining capacity of EoL power batteries, 4R Energy has divided the echelon utilization scenario and applied it primarily to energy storage systems and grid energy storage. Using EoL power batteries in home emergency power and energy storage devices is a viable solution pushed by battery manufacturers and vehicle manufacturers.China needs to establish a comprehensive battery recycling model to alleviate the shortage of critical resources, such as lithium, nickel, and cobalt . The polChina has prioritized the recycling of EoL batteries since 2003. Recycling focuses on EoL rechargeable batteries and disposable button batteries. With the advancements in science and technology and the rise in popularity of electronic devices, the following policies emphasized lead–acid recycling batteries. In 2006, the National Development and Reform Commission (NDRC) proposed the first “Technical policy on recycling and utilization of automobile products” based on the system of EPR, offering the recycling of automotive product parts and components. Since 2014, the government has begun to increase its support policies for the NEV industry, resulting in its rapid development. As a result, the Chinese demand for NEV power batteries has proliferated, and the country will soon deal with an abundance of EoL batteries. However, the government should have paid more attention to recycling EoL batteries then, so no separate document existed. The policy document for the creation of NEVs merely calls for the recycling of EoL batteries.In 2016, NDRC issued the “Technical Policy on Electric Vehicle Power Battery Recycling (2015 Vision)”. The policy implemented the EPR, directed enterprises to recycle and reuse EoL power batteries, and established upstream and downstream enterprises to form a closed-loop recycling system. Since then, China has gradually unveiled its recycling policy for power batteries. Chinese approach focuses more on recycling lithium batteries and encourages manufacturers to establish product recycling channel facilities, as lithium batteries are the most common type of battery used in power batteries ,49. In 2Based on the national industrial development policies, local governments should consider factors such as urban construction planning, regional economic development, talent pools, and echelon utilization technology to formulate targeted local policies. The policies and local governments should complement each other to form “one local, one policy” to better implement the echelon utilization of EoL power batteries , such asBy analyzing the regulations of different countries, we can observe that each country has distinct characteristics. However, all of them recognize the significant role of EPR in recycling power batteries . We anal(1)Some countries face low recycling rates, with China having a lower recycling rate for end-of-life (EoL) power batteries compared to other countries. High recycling rates are typically observed in nations that implement specific characteristics, i.e., (1) the establishment of regulations that govern enterprise behavior toward recycling EoL batteries; (2) the imposition of hefty fines for any violations; (3) the regulation of recycling processing fees, taxes, and deposit payments, which serves to oversee recycling enterprises and enforce consumer recycling obligations. However, in China, the recycling of rechargeable batteries is encouraged by charging consumers a fee for recycling treatment and providing subsidies. As a result, China has an imperfect and ineffective recycling system in comparison to several other countries, and most EoL power batteries have not been recycled through formal methods . Countri(2)Currently, the policies of several countries prioritize the recycling and echelon utilization of power batteries, which are important for implementing sustainable development. The field of power battery echelon utilization is still in its early stages, and the policies introduced by various countries primarily focus on funding the R&D of enterprises in this field. The application scenarios for echelon utilization are rather vague, and there has yet to be a comprehensive and specific summary of them.EoL power batteries are associated with strong negative externalities . Two maiThe battery manufacturer recycling model involves battery manufacturers collecting EoL batteries from consumers through methods such as NEV manufacturers, NEV retailers, and EoL NEVs dismantling companies, establishing a recycling method led by battery manufacturers and based on reverse logistics . In recyJapan has established a recycling method mainly led by battery enterprises, with the idea of “reverse logistics”. Battery manufacturers use service networks such as retailers, sellers, and gas stations to recover EoL batteries from consumers free of charge and then hand them over to professional battery recycling companies for disposal. Toyota Motor Corporation offers customers cash subsidies or purchase discounts for returning EoL batteries to dealers or retailers. The Nissan Leaf once offered a battery replacement business for customers .Under the NEV enterprises recycling model, NEV enterprises will be responsible for recycling (EoL) vehicles. NEV enterprises can use their existing sales distribution network to establish recycling service stations, such as automobile sales, 4S stores, and after-sales service stations, or they can establish specialized recycling service stations. NEV enterprises recycle power batteries and transfer the recovered batteries to regular dismantling enterprises. The batteries undergo echelon utilization and resource recovery. The recycling process is shown in BYD is the first company in China to lay out the NEV market. BYD’s recycling method for EoL power batteries is mainly through authorized retailers or dealers. When a customer needs to replace the EoL power battery, the retailer or dealer will transport the EoL power battery to the BYD factory for testing. Trumpchi, FAW-Volkswagen, and Geely are establishing special recycling service stations for recycling. The three companies have established the largest number of recycling service stations in China, accounting for 26.9%. In addition, many NEV manufacturers actively sign strategic cooperation agreements with battery recycling and dismantling enterprises to jointly establish power battery recycling networks. According to data from the MIIT, as of November 2022, China has established 15,251 power battery recycling service stations nationwide.The industry alliance recycling model refers to an incentive alliance between manufacturers, sellers, recycling, and traceability technology enterprises. In the recycling process, the sales and service networks among the members combine to build a closed-loop recycling network to centralize the recycling of EoL power batteries. The EoL power batteries are transported to the professional recycling centers of the alliance members for recycling and processing. Compared with individual manufacturers or retailers, industry alliances can divide and collaborate according to their strengths, ensuring the integrity of the entire recycling industry chain. The specialized division of labor within the industry alliance can improve recycling efficiency, allowing the entire recycling system to have a more professional recycling network and processing equipment at a lower cost . The recThe industry alliance recycling model is popular in the U.S. The ‘Portables’ team in the U.S. is an incentive consortium that was formed by Everledger, HP, Fairphone, and Call2Recycle. Everledger provides technical support for the battery traceability feature, which creates a battery passport for manufactured batteries. HP and Fairphone are the manufacturers of lithium batteries, and Call2Recycle is the company responsible for recycling lithium batteries, including EoL power batteries . Call2ReUnder the third-party recycling model, manufacturers pay a service fee to third-party companies so that third-party companies can assist companies in completing the EPR. The service fee is calculated based on the quantity and type of EoL power batteries returned by the manufacturers. This model can transfer the responsibility and risk of recycling to the third-party company, so it is more suitable for small manufacturers . Third-pIn China, GEM is a third-party recycling company approved by China. As a leading company in recycling EoL power batteries, GEM built a full life cycle value chain from recycling, remanufacturing, and echelon utilization to ensure maximum reuse of EoL power batteries. Moreover, MAE built an “Internet+recycling” platform for EoL lithium batteries by cooperating with battery manufacturers, NEV manufacturers, EoL battery recycling and dismantling companies, automobile retailers, and internet e-commerce platforms. This innovative recycling model has built a nationwide recycling service network and realized the efficient recycling of metal resources refer to .A low-carbon, resource-efficient, and inclusive future requires CE systems that emphasize reducing, reusing, and recycling . CE systReducing means that society uses as few non-renewable and natural resources as possible in the process of production and consumption, reduces the amount of energy used, and improves the utilization of resources; it is also a prerequisite and basis for recycling and resourcefulness . ReducinAfter a battery is repaired (after its first use), it can be reused in the same vehicle model. This method reduces the need for new batteries to be produced for use in vehicles . The batEssentially, remanufacturing is the process of repairing, replacing, or restoring EoL products to ‘like-new’ conditions . This reRecycling companies have become the most popular method for recovering rare metals and valuable materials from power batteries ,74. By aIn conclusion, each system places much emphasis on reusing. However, the existing studies do not specify the process of reusing. In the EoL power battery recycling process, we specify that the process of echelon utilization involves reusing. From the policy analysis in the previous section, it is clear that echelon utilization is already a key area for future research and development.Echelon utilization can be defined as reusing the EoL power batteries. Echelon utilization can relieve the pressure of recovery, reduce environmental pollution, improve economic efficiency, and help the development of renewable energy. Echelon utilization aims to reduce the battery capacity to less than 80%, which is a slight scrap, and is not suitable for power batteries used in NEVs . EoL powAs the reserve energy of EoL batteries can be applied to some scenarios, it is extremely critical to pay more attention to the echelon utilization. There are three categories of application scenarios for echelon utilization: generation-side energy storage, user-side energy storage, and mobile power supply. The classification of echelon utilization scenarios presented in this paper is based on the classification of energy storage proposed by  as well In general, the division of power generation-side and user-side energy storage systems is based on the size of energy storage facilities and the standard of centralized physical distribution of energy storage systems. The scale of the generation-side energy storage is usually large and the level of battery consistency and balance management is high. ThU.S., it often needs the support of a large power grid , such asThere are two main application scenarios of energy storage cited in this paper  to ensure the stability of the power grid on the generation side.As an energy storage facility, batteries are primarily used for storing sustainable renewable energy . ApplicaUsing batteries as energy storage systems on the generation side can also cut and fill peaks and valleys. Due to the greater relative cost of energy consumption during peak periods than during low peak periods, the battery can be used as an energy storage device at a low cost to store excess energy during peak periods and then output the stored energy during low peak periods to the power grid. Generally, the application scenarios for energy storage on the power generation side have high demands on battery specifications, packaging forms, and quantity. In addition, they also have high demands on the level of selection and balance of management of batteries. For example, GM and ABB are further exploring the use of used battery packs to provide backup power for small commercial buildings when power is lost as a means of compensating for the intermittent power generation limitations of renewable energy sources, such as solar and wind power, as well as storing and using electricity during preferred periods in a way known as “peak-load shifting”.The main characteristics of the application scenarios of user-side energy storage of EoL batteries involve distributed energy storage, such as small charging stations, commercial power supply, remote areas without power grids, and scattered villages with high construction costs for power grids.In economics, the EoL battery industry is still in its infancy, and there is less risk associated with small applications. The current application of user-side energy storage has greater economic potential than power generation-side energy storage. In this area, some companies have already begun implementing plans. For example, Japanese automaker Nissan has pre-ordered a residential energy storage unit called “xstorage”, which uses 12 retired EV batteries and is connected to the grid to provide backup energy . To demoMobile power supply mainly refers to the echelon utilization of used batteries in low-speed vehicles and electric motorcycles that have less battery performance requirements than NEVs. Applications include power batteries for vehicles with low-performance requirements, backup power supplies for short trips in NEVs, and emergency start energy. Jin  demonstrEchelon utilization was first studied in Europe, the U.S., and Japan. Currently, chemical batteries, fuel batteries, and solar batteries make up the majority of power batteries on the market. Chemical batteries include lead–acid batteries and lithium-ion batteries. Lead–acid batteries are the most widely used in the automotive industry due to their low price and mature technology . There hRapid echelon utilization development has led to the significant strengthening of cooperation among leading enterprises within the foreseen scope, with the strong support of national policies . The govEoL power batteries refer to three types of situations. First, EoL batteries are created during the production process by battery manufacturers. Second, NEVs cannot guarantee regular operation due to the remaining capacity or the charge/discharge performance. Thirdly, EoL power batteries have been reused after echelon utilization. The ability to recycle power batteries effectively will directly impact national energy savings and emission reduction strategies. Integrating the battery recycling production network into the industrial ecological chain will optimize the allocation of system resources and minimize external diseconomies. However, we found that there currently needs to be clearer guidance for recycling EoL power batteries. Based on the analysis in In general, recycling EoL power batteries can be divided into two parts: recycling methods and recycling systems. Countries around the world can determine a reasonable recycling paths according to the country’s development and combine it with the four recycling methods proposed in In the early stages of developing NEV batteries, the open-loop production mode of “cradle to grave” was mainly utilized. The industrial ecological chain ends at the product’s EoL point. This paper presents a 4R recycling system for EoL batteries. In this paper, we aimed to design a 4R system for recycling EoL power batteries, addressing the industry’s original crude open-loop production structure and achieving a “cradle-to-recycle” production model ,92.In After 5–8 years of consumer use, the power battery will be retired. Consumers take their EoL power batteries to a standard recycling service station and receive a subsidy offer. The recycling service station hands the EoL power battery to the NEV manufacturer. NEV manufacturers fulfill the EPR by returning EoL power batteries to the upstream battery manufacturer. Prior to battery echelon utilization, the battery manufacturer calculates the SOH of the battery through BMS. The enterprise can assess the economy, safety, and availability of power batteries by reasonably evaluating the SOH data of vehicles and assessing the application field of the secondary reuse of power batteries ,94,95. WThe 4R recycling system for EoL power batteries improves the components’ utilization rate and reduces the industry’s ecological disorder. Power batteries can be extended in life, and a “produce-use-recycle-reuse” production model can be used and enterprises will be able to implement a “cradle-to-cradle” production model for power batteries. Meanwhile, it creates several market opportunities in the recycling system, reducing waste and negative externalities on the environment. If the government and enterprises actively implement the 4R recycling system in the future, it will be of great importance to improve the country’s economic, environmental, and social well-being.Previous studies have shown that recycling power batteries is effective, i.e., economically, environmentally, technically, and resourcefully ,25,74. HGovernment policy plays a critical role in the emerging industry, but five issues remain regarding policies. (1) The NEV battery recycling industry needs a transparent system and specific recycling and dismantling standards. NEV manufacturers consider factors such as safety, space optimization, and applicability during the design process. However, there are differences in the battery packs, modules and battery packs produced according to the requirements of different vehicle models . It willRecycling EoL power batteries may have some negative externalities on the environment . We analDue to the rising costs of raw materials for power batteries and graphite, recycling (EoL) power batteries has gained worldwide attention. In this paper, we analyze the recycling policies of representative nations and find that nations worldwide are gradually focusing on the scenario of hierarchical utilization of EoL power batteries. Based on the current recycling model and system, we propose that the 4R recycling system can effectively manage EoL power batteries in the future. Regarding the six presented development obstacles, we propose the development of recycling EoL power batteries from the perspectives of government, enterprises, and consumers.(1)According to the 4R EoL power battery recycling system, governments consider echelon utilization and clarify the responsibilities of recycling enterprises. To achieve the goals of environmental protection, resource conservation, and safe use, the government must also improve its policies and regulations, technical guidelines, evaluation standards, and industrial chain for echelon utilization.(2)The government should bolster oversight of EPR implementation by battery manufacturers and NEV manufacturers and improve the regulatory framework. In addition, the government should increase the penalties for companies that violate recycling laws.(3)The government establishes technology subsidies, focusing on subsidizing research and technological breakthroughs in critical echelon utilization applications. The government has increased support for recycling companies to promote the growth of the recycling power battery industry.(4)The government increased subsidies to consumers to incentivize consumer participation in recycling. The government and competitors should educate consumers about environmental awareness and formal recycling procedures.(5)The government rationalizes the planning of recycling infrastructures for power batteries. It may be convenient for consumers to return their power batteries at these locations and for enterprises to reduce transportation distances and reduce costs by replacing them at these locations.(1)Battery manufacturers, NEV manufacturers, NEV retailers, and hierarchical utilization enterprises must engage in collaborative design, management, and data sharing, and cooperate to address the issues of constructing battery recycling methods, building a recycling evaluation system for the entire life cycle, and establishing a service system. According to the four existing recycling methods, technology integration among enterprises will be strengthened, thus reducing investment risks and achieving win-win cooperation.(2)Battery manufacturers should increase their investments in recycling equipment, research, development, and technical personnel. They should establish a traceable power battery recycling regulatory system using blockchain technology to ensure real-time monitoring of companies’ implementation of full life cycle tracking. Enterprises should also enhance their awareness of recycling and improve their sense of corporate social responsibility.(3)NEV retailers should use advertising and education to increase consumer participation in battery recycling or promote “trade-in” activities to attract more consumers to participate in recycling. When the recycling volume reaches a particular scale, it can also increase the profit of the recycler. In addition, recycling companies should set reasonable prices to balance the interests of enterprises and consumers.As essential participants in recycling EoL power batteries, consumers should recognize the critical role of environmental protection and resource conservation. Consumers should consciously comply with their recycling obligations and take the initiative to send EoL batteries to designated recycling service stations. By consciously recycling, consumers can gain considerable benefits and effectively reduce the costs of government regulation, thus achieving the goal of maximizing social welfare.(1)This paper examines the battery recycling regulations of the EU, the U.S., Japan, and China. The government regulates countries with high recycling rates through legislation, deposit systems, high fines, and subsidies. Upon analyzing the policies, it was found that echelon utilization of EoL power batteries has become a significant concern in all nations.(2)This paper summarizes the four existing recycling “models”: battery manufacturers recycling model, NEV enterprises recycling model, the industry alliance recycling model, and the third-party recycling model. This paper summarizes the recycling systems in EoL power batteries, including reducing, reusing, remanufacturing, and recycling, by combining circular economy theory and closed-loop supply chain systems.(3)Combined with the development policy and recycling system, EoL power battery echelon utilization will be the focus of future development. However, several studies clearly delineate the application scenarios of echelon utilization. This paper presents the application scenarios of echelon utilization in terms of generation-side energy storage, user-side energy storage, and mobile power supply; we use actual cases for analysis.(4)The government must establish clear and effective recycling methods and normative recycling systems. Therefore, this paper proposes a 4R EoL power battery recycling system that accounts for echelon utilization and suggests the use of state of health (SOH) to assess the states to determine the recycling steps.(5)This paper analyzes the existing policy problems and technological challenges and proposes three aspects of development for the government, businesses, and consumers based on the above analysis.With the rapid popularity of NEVs, large-scale EoL power batteries will face retirement. There are substantial potential development opportunities and broad market prospects for recycling EoL power batteries. In order to reduce environmental damage, save resources, and protect health, it is important for the government, enterprises, and consumers to remanufacture, utilize them hierarchically, and recover resources from EoL power batteries through cooperation under a circular economy. This paper compares the current industrial development status and analyzes the problems of the existing recycling process in terms of policy and technology from a practical problem perspective. The main conclusions of this paper are summarized as follows:This paper has a limitation in that it mainly summarizes the literature studies and actual situations without using bibliometric analysis, which is commonly used by most scholars. Therefore, future studies can use this approach to analyze the literature in this field from multiple perspectives.


The Veterans Health Administration uses equity- and evidence-based principles to examine, correct, and eliminate use of potentially biased clinical equations and predictive models. We discuss the processes, successes, challenges, and next steps in four examples. We detail elimination of the race modifier for estimated kidney function and discuss steps to achieve more equitable pulmonary function testing measurement. We detail the use of equity lenses in two predictive clinical modeling tools: Stratification Tool for Opioid Risk Mitigation (STORM) and Care Assessment Need (CAN) predictive models. We conclude with consideration of ways to advance racial health equity in clinical decision support algorithms. As a proxy for structural racism in health-related research, race and ethnicity potentially identify health disparities, in turn provoking further examination of the health impacts of systemic inequalities and social determinants. When relied on as a proxy for generalizing biological differences, race proves to be imprecise for delineating genetic and phenotypic variance.3The use of race-adjusted clinical equations and algorithms spans many medical specialties and often has historical roots pointing to racism in medicine. Race adjustment has been shown as harmful in multiple clinical scenarios.4Furthermore, the conflation of ancestry and race creates clinical risks, where race is used instead of available genotype and phenotype testing to guide treatment decisions. For example, researchers found that the immunosuppressant, azathioprine—an important drug option for treating conditions such as systemic lupus erythematosus—was discontinued or the dose was reduced unnecessarily for some Black patients based on abnormal cell count laboratory values using reference ranges standardized for largely White populations.When accounting for a known genotype and the presence of the Duffy null phenotype (associated with a lower baseline absolute neutrophil count than found in laboratory result reference ranges), these patients may have safely stayed on azathioprine. The researchers proposed more widespread phenotype testing for certain conditions since the related genotype, while more common in people of African ancestry, can be found in other ancestral populations.5 Health care systems increasingly use clinical decision support tools to gauge potential health outcome risk among patients. Many of these tools use predictive modeling to assess risk. While they are potentially powerful tools to capture patients at increased risk for morbidity and mortality, they are also at risk of perpetuating health inequities by introducing unintended racial bias.Similarly, clinical decision support tools using machine learning are also at risk of introducing or exacerbating racial bias.6 The VA has begun addressing algorithmic racial bias and promoting equitable clinical equations7 and decision support tools as part of larger health equity initiatives.As the largest integrated health care system in the United States, the Veterans Health Administration (VA) serves over 9 million enrolled veterans and provides a wide range of care services across nearly 1300 health care facilities.8 As a living document, the HEAP strategically guides equity work across five aims: awareness; leadership; health outcomes; workforce cultural and linguistic competency; and data, research, and evaluation.The VA established the Office of Health Equity (OHE) in 2012 and assembled the Health Equity Coalition, which together created the Health Equity Action Plan (HEAP).9 VA patients have some of the lowest mortality disparities nationally,10 but as a learning system, VA continues to actively monitor and identify racial/ethnic and other disparities that persist.Currently, >20% of veterans come from minoritized racial/ethnic backgrounds, and the OHE works with partners to publish actionable reports that identify disparities across many demographic characteristics in quality of care and veteran patient experience.9 In addition, VA specialty care programs, such as the VA Kidney Medicine Program, track multiple health equity-related demographics in the delivery of specialty care as strategic program goals.For example, the periodically produced National Veteran Health Equity Report describes racial/ethnic, geographic, gender, and age disparities in veteran patient experience and quality of care for conditions such as hypertension, hyperlipidemia, and diabetes.1 and identified two case studies where there was knowledge of clinical leaders advocating for and pursuing changes to equations. We selected two cases focusing on decision support algorithms based on knowledge of existing work to mitigate racial bias and the widespread use and impact of these tools. We discuss approaches, successes, challenges, and next steps in addressing and eliminating racial bias both in these examples and more broadly.The work of advancing equity belongs to all VA staff, and content-specific efforts are led across VA national program offices and facilities. In this article, we present four use cases from across VA program offices that address racial bias in clinical equations or decision support algorithms. Authors from the OHE reviewed an inventory of clinical equations with known bias12Assessment of kidney function includes calculation of an estimated glomerular filtration rate (eGFR) using laboratory and demographic variables. Reductions in the eGFR may signal the presence of kidney disease and are central to the diagnosis and staging of chronic kidney disease (CKD), a condition that has affected an estimated 37 million (15%) Americans and 1 million enrolled veterans.1 As a result, these equations risk overestimating kidney function in some Black individuals, delaying the recognition of CKD and implementation of a disease-modifying therapy, as well as referral for kidney transplantation.13 Additionally, researchers and others began questioning the inclusion of race based on serum creatinine observational data, muscle mass, or other factors.14Many equations exist for calculating the eGFR, such as the widely utilized Modification of Diet in Renal Disease Study and the 2009 Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI), both of which include a race coefficient for Black individuals based on epidemiologic data.15 The NKF/ASN Task Force published its report calling for immediate implementation of the new equation.16 The VA played an active role in this process: VA clinician researchers both led and participated in the NKF/ASN Task Force that recommended against using race-based eGFR equations.17In 2020, the National Kidney Foundation (NKF) and the American Society of Nephrology (ASN) established a task force to address the inclusion of race in the diagnosis of kidney disease, and in September 2021, a new CKD-EPI eGFR equation that did not include race was released.7The VA National Kidney Medicine Program and the VA National Pathology and Laboratory Medicine Programs developed a collaborative, national strategic plan to implement the race-neutral equation. The VA Pittsburgh Medical Center was the first VA facility to implement the new equation within 30 days of electronic publication of the new race-neutral 2021 CKD-EPI eGFR equation from the task force report, and all VA laboratories were mandated to implement the race-neutral calculation by April 1, 2022.18–20 VA leadership collaborates with clinicians to incorporate all clinical data and individual patient factors where eGFR is used as an input in clinical decision-making.As part of an equity mindset and ensuring clinical excellence, VA has been at the vanguard of implementation of cystatin C testing, providing an alternative race-neutral assessment of kidney function, and continues to monitor the impact of moving to the 2021 CKD-EPI eGFR calculation.20 Such analyses permit advance preparation to reallocate resources accordingly and ensure equitable access to specialty care in the future.Impact analysis of implementation of the race-free eGFR equation has shown no difference in the estimated number of veterans impacted by CKD, but did show geographic shifts to higher CKD prevalence in regions with higher numbers of Black veterans and an increased number of Black veterans with CKD stages 3 and 4.21Identifying CKD at the earliest stage possible is essential for preventing or delaying progression to kidney failure requiring dialysis or transplantation. In July 2022, the Organ Procurement and Transplant Network (OPTN) Board implemented a policy prohibiting transplant programs from using race-based eGFR calculations in determining eligibility for transplant. Subsequently, in December 2022, the OPTN Board adopted a policy that by January 3, 2024, all kidney transplant programs have to identify Black transplant candidates whose original qualifying date was affected by the race-based eGFR calculation.Specifically, transplant centers are required to (1) determine if the use of the race-neutral 2021 CKD-EPI equation would have resulted in qualifying earlier and gaining waiting time for a transplant; (2) apply to the OPTN to modify the waiting time for eligible candidates; (3) contact and inform currently registered candidates for kidney transplant about this initiative; (4) contact candidates a second time to let them know if they qualify for a waiting time adjustment; and (5) submit attestation of compliance to the OPTN.As of the time of this article, all VA Transplant Centers have implemented the race-neutral eGFR calculation and are on track to complete the eGFR recalculation, updation of waiting times, and veteran outreach, as required by the OPTN policy.Evaluation of lung health includes pulmonary function tests (PFTs), the components of which include spirometry, measuring lung volume, maximum respiratory pressure, and diffusion capacity. Interpretations of PFT measurements are used to diagnose a wide array of lung diseases and, as a result, they impact the timeliness and use of potential treatments. Interpretation of PFTs affects treatment plans for other conditions such as bone marrow transplant and the use and dosage of chemotherapy. Additionally, PFT interpretation may directly or indirectly affect eligibility for certain types of employment , disability benefits, and insurance premiums.22–24The expected values derived from spirometry testing incorporate a patient's age, height, sex, and race/ethnicity compared with a nonsmoking, healthy, and racially/ethnically concordant representative population. The origins of including race/ethnicity in spirometry calculations are mired with racialized assumptions about innate lung function differences compared with White individuals rather than more plausible explanatory mechanisms such as exposure to childhood environmental pollution, nutrition and health disparities, and gene–environment interactions.24 However, at the same time, these racial corrections have allowed treatment options  that may have been otherwise excluded based on lung function.Race/ethnicity coefficients in spirometry calculations were not designed as, and are poor proxies for, structural racism and other causal factors that affect lung function. For some racial/ethnic groups, the use of race-specific equations may result in differential classification of the severity of lung disease, with implications for access to certain disease treatments and disability benefits.25 The ongoing use of race-specific equations continues to risk exacerbating health inequities for some individuals. Conversely, universally switching to the current race-neutral equation may widen the limit of normal function for some groups and narrow it for others, resulting in missed diagnoses in some individuals and overtesting and overdiagnosis in others.24The widely used 2019 American Thoracic (ATS)/European Respiratory Society guidelines include both race/ethnicity-specific and ethnicity-neutral  options.26In April 2023, the ATS—acknowledging ongoing concerns about using race-specific equations as well as concerns about ignoring race—recommended universal movement to a race-neutral equation and called for a broader reevaluation of how PFTs are used to make clinical, employment, and insurance decisions.In light of this recommendation, clinicians continue to navigate an imperfect state of the science and the nuanced considerations for using either race-neutral coefficients universally or continuing with race/ethnicity-specific ones. Patients at the threshold of the lower limits of normal in PFT interpretation particularly need careful consideration of all factors that go into disease diagnosis due to the ongoing imprecision of PFT equation measurements and their interpretation.The VA National Program Office for Critical Care, Pulmonary, and Lung Cancer Screening is taking a multistep approach to assess the global impact of the new recommendation, including conducting a data analysis of VA-wide data, comparing population measurements using race-specific versus race-neutral equations, and engaging veteran opinion through structured focus groups to better understand how changing to a race-neutral equation may impact multiple racially and ethnically diverse groups of veterans.The data analysis includes analyzing the possibility that the current race-neutral equation may create new disparities as unintended consequences of the change. While these steps are underway to evaluate PFT equation changes across the entire VA, some individual VA medical centers have decided at local leadership levels to move toward using the race-neutral equation universally while concomitantly collecting race-specific equation data for ongoing analysis of how this change impacts their diverse local patient populations.Data from the VA-wide analysis, insights from veteran focus groups, and equation implementation considerations will be shared with key internal VA stakeholders as part of formulating the next steps.27 As a clinical decision support tool, STORM integrates clinical practice guideline-recommended risk mitigation strategies for opioid use disorder, nonpharmacological pain treatment options, and predictive modeling. Using a wide array of demographic, medical, and psychiatric data, the predictive model generates risk scores for patients prescribed opioids.29 Although demographic factors such as gender and age are included in STORM, currently race and ethnicity are not included.The VA developed the Stratification Tool for Opioid Risk Mitigation (STORM) in 2016 and it was paired with a policy-mandated interdisciplinary case review for very high-risk patients on March 4, 2018.Recognizing the importance of equity as a foundational principle for real-world clinical decision support, the VA Office of Mental Health and Suicide Prevention's Program Evaluation Resource Center (PERC) began work on development of a framework to assess performance differences in STORM in 2019 based on gender, age, race/ethnicity, and deidentified data.The framework was developed with a group of stakeholders from VA, Food and Drug Administration's Office of Minority Health and Health Equity, and the Data Science for Social Good program. The stakeholders functioned as subject matter experts for the design of the framework and included clinicians, data scientists, and statisticians.30–32The framework for algorithmic bias assessment incorporated summary statistics and visual diagnostic tools to help make the information conveyed more intuitive to a diverse set of stakeholders and was customized to the preferences of the stakeholder group. When applied to observational data from 4,214,579 patients enrolled in VA services in 2014, to predict outcomes in 2015, the framework found evidence of performance differences by race/ethnicity.Potential algorithmic bias was detected by differences in false negatives and false positives for each true positive predicted by STORM, plus differences in false omission rates, that is, the rate that a patient predicted to be negative for the outcomes experienced the outcomes. Since this work was performed on deidentified data that lacked some variables used in the version of STORM that was deployed in clinical practice, the analysis used VA data that predated the shift in poor opioid-related outcomes to U.S. citizens of all sociodemographic strata.The PERC team encountered challenges interpreting results for subgroups of relatively small sizes  or subgroups based on more than one demographic factor . As a result, PERC decided to extend its work to more recent prediction cohorts.Building on this work, PERC is applying its performance evaluation framework to 2018 and 2019 outcomes. The framework will also be used to compare alternative techniques for mitigating algorithmic bias, for example, exploring race-specific models or defining race-specific high-risk cut points. While the STORM model does not incorporate race/ethnicity variables, the performance evaluation framework considers race to evaluate potential bias in the STORM model.Using the PERC framework to monitor performance over time could address model bias as part of continuous process improvement versus an episodic tangential activity. It could also help the STORM modeling team empirically inform when a model should be recalibrated or updated to include a new predictor variable or could benefit from a different prediction approach.Care Assessment Need (CAN) is a set of predictive models that aid VA primary care clinical teams in identifying veterans at risk of hospitalization and mortality. Currently, CAN is a set of six distinct algorithms that model three outcomes, hospitalization, death, and hospitalization or death, over two prediction periods, 90 days and 1 year. Predictor variables include demographics, vital signs, medical conditions, prior use of VA health services, laboratory results, and dispensed medications. CAN models are updated weekly and are available to clinicians in population health management tools and dashboards.33–35 Race and ethnicity demographic variables are not included in the CAN model; however, racial/ethnic disparities may emerge through other mechanisms. For example, a not yet published analysis may indicate that Black veterans are systematically underidentified as high risk for 1-year mortality outcomes compared with White veterans.36Among the general VA population, the CAN models accurately predict outcomes and have demonstrated favorable model discrimination and calibration.In an ongoing analysis of the model, age is the most influential variable in the 1-year mortality CAN model. Black veterans die earlier than White veterans and because of variation in average age of mortality, Black veterans may be potentially disadvantaged by the current model, in that it may underestimate their mortality risk, while overpredicting the risk for White veterans.36A VA-funded research team in collaboration with VA operations is exploring statistical concepts of weighting, interaction, penalization, and normalization to make age race-neutral as a predictor variable in an updated model so that CAN becomes more equitable in estimating risk for the model's outcomes.37 as reasons for younger average age at death among Black veterans, with the goal of updating the current CAN model to eliminate risk calculation disparities.In addition to exploring the impact of race/ethnicity in the model, the research team is investigating the impact of social determinants of health (SDOH) and weathering, a phenomenon related to the health consequences of experiencing racism,Researchers unintentionally discovered that age may be a proxy for weathering, which they continue to analyze. SDOH data elements derived from the electronic health record , location-based area statistics, and third-party datasets are being tested for inclusion in CAN.Both STORM and CAN do not explicitly include race/ethnicity in their predictive models. The performance evaluation framework being used to evaluate STORM includes taking race-specific evaluation approaches to see if algorithmic bias may nonetheless be present through other mechanisms, as discussed here.In 2020, the Agency for Healthcare Research and Quality issued a request for information (RFI) seeking to gather public and stakeholder perspectives about the efforts and repercussions of addressing algorithmic bias. Many themes emerged in a qualitative study based on the RFI, including the (in)appropriate use of including race and SDOH information in algorithms.when looking to identify risk groups, (1) race/ethnicity could improve an algorithm in some cases; (2) failure to include race might lead to biased care; and (3) including race/ethnicity might improve an algorithm's predictive accuracy without addressing inequities.38 Themes also included acknowledging underrepresentation of racial and ethnic minority groups in datasets, which is not easily remedied by analytic strategies to overcome the insufficient amount of diverse demographic data.A subtheme included that Determining if race/ethnicity variables in algorithms exacerbate bias, versus if, instead, they meaningfully identify and address racial inequities, remains ongoing work for VA researchers and data scientists alongside the entire health care data science community.VA has begun and seeks to advance health equity, including by examining clinical decision support algorithms, such as clinical equations and predictive models. VA helped lead the national medical society efforts to eliminate the race coefficient from the eGFR calculation and mandated the use of the race-neutral eGFR across its national integrated health care system.Accelerated by and contemporaneous with evolving PFT guidelines, VA is currently engaging veteran voices and analyzing its national data, looking at the clinical and nonclinical implications of using a variety of approaches for PFT assessment among diverse veteran groups. VA researchers identified racial bias in predictive models such as STORM and CAN, despite race/ethnicity variables being absent, and teams continue to revise modeling to mitigate bias and advance equity.We also identified challenges, limitations, and areas for improvement when considering the impact and presence of algorithmic bias. The authors may be unaware of existing VA work on other biased equations not discussed here, and an area for improvement includes continuing to systematically identify other biased equations in use, their clinical stakeholders, and any progress on addressing racial bias in them. Similarly, the authors may be unaware of other champions of change to clinical algorithms not discussed in this article, and an area for improvement includes pathways to identify those champions and empower them.One challenge for advancing the work of eliminating or mitigating algorithmic bias uniformly includes how clinical service lines are organized and their governance structure for identifying and implementing changes to clinical guidance. Not all specialty areas have centralized governance and not all areas that do can change clinical practice guidelines in the ways discussed in this article for the eGFR and PFTs, which themselves used different processes based on the nature of the algorithmic bias, implementation differences, and service line governance structures.Equation changes that do take place often require clinical champions to identify and advocate for resources for this work, that is, the VA system often relies on clinical service line champions due to governance in a complex health care organizational matrix. National medical society guidelines are crucial levers for pushing large health care organizations, such as VA, forward as well.Other stakeholders, such as the OHE, can then help advance efforts in eliminating algorithmic bias, but they themselves rely on service lines for creating policy changes. The authors continue to look for ways to identify other equations and decision support tools for potential bias and mitigation.39Another area for improvement includes exploring more deeply the impact of biased algorithms on affected veterans by adapting action- and person-centered elements in frameworks such as the Health ARC , a care delivery model for addressing health inequities.As an example, the implementation of OPTN requirements for VA kidney transplant candidates affected by the race-based eGFR equation reflects some elements of delivering care in a way that seeks to address prior harms. Across the health system, opportunities remain for creating more equitable care delivery models, including veteran voices that inform steps taken to acknowledge bias and prevent future harm.We recommend that health care systems look for and consider potential racial bias in equations and predictive models, even where race/ethnicity is not formally included in underlying variable sets. Using a mindset of both curiosity and equity, asking if diagnostic test result interpretation guidelines might be worsening disparities is essential for undoing the legacy of race-based medicine. Equity champions should also consider that corrective actions to address racial bias in equations and predictive models do not inadvertently create bias for other groups or harm patients.Finally, health care leaders can support advancing equity by empowering internal champions with resources to analyze system data for racial bias. Uncovering racial bias is the first step to advancing racial equity.


In 2020, the Health Resources and Services Administration’s HIV/AIDS Bureau funded an initiative to promote implementation of rapid antiretroviral therapy initiation in 14 HIV treatment settings across the U.S. The goal of this initiative is to accelerate uptake of this evidence-based strategy and provide an implementation blueprint for other HIV care settings to reduce the time from HIV diagnosis to entry into care, for re-engagement in care for those out of care, initiation of treatment, and viral suppression. As part of the effort, an evaluation and technical assistance provider (ETAP) was funded to study implementation of the model in the 14 implementation sites.The ETAP has used implementation science methods framed by the Dynamic Capabilities Model integrated with the Conceptual Model of Implementation Research to develop a Hybrid Type II, multi-site mixed-methods evaluation, described in this paper. The results of the evaluation will describe strategies associated with uptake, implementation outcomes, and HIV-related health outcomes for patients.This approach will allow us to understand in detail the processes that sites to implement and integrate rapid initiation of antiretroviral therapy as standard of care as a means of achieving equity in HIV care. Of more than 1.2 million people in the United States with human immunodeficiency virus (HIV), approximately 87% are aware of their HIV diagnosis .In 2020, the Health Resources and Services Administration’s (HRSA) HIV/AIDS Bureau Ryan White HIV/AIDS Program (RWHAP) Part F – Special Projects of National Significance (SPNS) funded the “Building Capacity to Implement Rapid ART Start for Improved Care Engagement in Ryan White HIV/AIDS Program” Initiative, with its Evaluation and Technical Assistance Provider (ETAP) awarded to the University of California, San Francisco (UCSF) Center for AIDS Prevention (CAPS). The ETAP’s role is to provide support to, and evaluate the implementation of, rapid ART start in 14 RWHAP settings across the U.S. with the goal of developing an implementation blueprint for integrating immediate access to ART in HIV care settings. In this manuscript, we describe the approach to evaluate whether and how strategies undertaken improve: organizational capacity, delivery of rapid ART start, and patient outcomes associated with engagement in care.In 2013, the Zuckerberg San Francisco General Hospital’s Ward 86 HIV clinic implemented services for providing ART on the same day as HIV diagnosis . BetweenART initiation is considered “rapid” when treatment begins as soon as possible after a positive HIV test  though areorganization of procedures, multidisciplinary coordination, and consolidation of patient services, including clinical evaluation, education, counseling, ascertainment of healthcare coverage and laboratory testing, all fit into a 2- to 3-hour initial clinic visit [Time is required to arrange healthcare coverage if needed, initiate work with patients for psychological and social stabilization, provide education and counseling about HIV and ART, and select an appropriate ART regimen for the patient [data collection strategies have also emerged as important tools for driving implementation, assessing success, and facilitating sustainability [Delivery of rapid start ART services requires organizational ic visit , 27. Tim patient , 28. Spenability .Barriers to rapid ART start service deployment exist on multiple levels . IdentifThe RWHAP SPNS Rapid Start ART initiative is a three-year project and funds 14 RWHAP implementation sites initiating and/or expanding rapid ART initiation. Sites differ in the context in which those services are provided , and where they are located throughout the U.S. . To support implementation, we have utilized the Institute for Healthcare Improvement’s Collaborative Model   and the Proctor Model (Implementation Strategies and Outcomes) is shown in Fig. patients .We designed a mixed-methods approach to the evaluation /electronic medical record (EMR) to generate data needed for QI. Barriers and facilitators to implementation of selected strategies are extracted on a routine basis.Document review allows us to identify We conduct open-ended assessments with up to 10 key stakeholders and implementers per site. Participants include those directly involved in implementation  and any individuals in key leadership positions. These qualitative surveys are self-administered via Qualtrics whereby open-ended responses can be entered. These assessments take place twice: at implementation mid-point reflecting over the first year (summer 2022) and toward the end of the initiative (summer 2023).provider readiness and implementation outcomes . In addition, these assessments capture information on implementation strategies. These questions ask for the staff members’ perceptions at the time of the administration as well as retrospectively.The assessments are designed to capture information related to The Organizational Assessment is orally administered by ETAP staff at baseline and prior to each Learning Session. Organizational assessments are emailed to the Project Director ahead of time to allow preparation of the information; the Project Director has the discretion to determine attendance at the administration. Answers are tracked across assessment waves to capture change over time.context , including resources needed and used , and activities  [reconfiguration capacity), using questions about updated workflows and policy decisions, as well as the structure  and Culture  subscales of the Accelerated Healthcare Innovation Capacity Scale [Information Systems Readiness assessments of the Electronic Health Record in order to understand capacity and access issues and the degree to which the system can be modified, if necessary, to track implementation and conduct QI projects. The concepts of acceptability, adoption, and feasibility are assessed using scales developed specifically for longitudinal documentation: the Acceptability of Intervention Measure , Intervention Appropriateness Measure (rapid ART start is suitable and applicable), and Feasibility of Intervention Measure (rapid ART start is easy to use and implementable) [The OA serves to capture information on agement) . It alsoty Scale . Questioentable) .all rapid start ART-eligible patients at the implementation sites, regardless of when/if they actually start ART. “Rapid-eligible patients” are as follows: (1) newly diagnosed includes any person with a new positive HIV rapid, confirmatory, or detectable viral load test result within 12 months of the site becoming aware of their positive status; (2) new to care includes any person diagnosed with HIV greater than 12 months prior who has not previously attended a HIV care medical visit or has never been on ART; and (3) out of care includes any person diagnosed with HIV with previous engagement in primary HIV care but who has not had a medical visit, ART prescription, or laboratory test result for greater than 12 months, and has agreed to return to care.Medical data are collected on The patient clinical data serve two purposes in this initiative: (1) generation of quality measures for review and reflection within the Learning Collaborative including QI projects; and (2) transformation into coded patient-level records for inclusion in the multisite evaluation. The Excel workbook into which the data are entered autogenerates reports/graphs used in coaching sessions and presentations at Learning Sessions. The same workbook autogenerates de-identified, coded data for inclusion in the multisite evaluation. Sites collect and submit data from patient medical records on a monthly basis order to capture information on all eligible patients and computed quality measures variables.retrospective baseline which includes all rapid-eligible patients from April 1, 2020, through March 31, 2021. This assesses each site on their performance prior to their substantive work within the Learning Collaborative. Sites continue to collect the same data points on all rapid-eligible patients for the prospective data collection, which includes all rapid-eligible patients seen for their initial visit between April 1, 2020, and March 31, 2023, with follow-up data collected through June 30, 2023.Service launch commenced with the first Learning Session , establishing the pre-implementation service outcomes can be determined, including provision of rapid start ART to all who could benefit (effectiveness) and demonstration that delivery does not vary by patient characteristic (equity). The amount of time elapsed between points of contact reveals any delays in treatment (timeliness). Finally, we can determine if rapid start service provision results in better health outcomes, namely viral suppression, and durable viral suppression.From this information, an overall assessment of Costing workbooks are designed by the ETAP and utilized in several previously funded RWHAP SPNS initiatives. As with the Organizational Assessment process, the Project Director is sent an advance copy of the costing template to assemble the information requested. The workbook is then submitted using an encrypted email. Costing data sheets are requested corresponding to Action Periods associated with the Learning Collaborative . The workbook documents financial and human capital resources directed toward the implementation of rapid ART start services and total monetary expenses and personnel hours dedicated to development and implementation of service models. Calculations include programmatic and structural expenses and personnel supported by RWHAP SPNS, as well as other in-kind sources. Examples of expenses included are: personnel effort to develop protocols; EHR/EMR modifications necessary for implementation; provider and staff trainings; routine performance monitoring and feedback; and personnel and care-related expenses above and beyond levels required before rapid ART start implementation.costs to implement rapid ART start, inclusive of funding sources associated with this initiative and any other source. The costs of implementation do not include expenses and personnel dedicated to the development and implementation of local or multisite RWHAP SPNS evaluations, participation in Learning Collaborative and initiative-developed QI activities, or routine costs of care independent of rapid ART start.The goal is to obtain a measure of the true In this mixed-methods study, we are using quantitative methods for understanding the implementation process and outcomes of the rapid start programs. Qualitative methods are critical to understanding team readiness and dynamics as they play out over time. Overall interpretation of our results will consist of an integration of qualitative and quantitative findings, using a modified Embedded Experimental Design .a priori domains are defined based on the research questions of interest; for example elucidating the components of implementation strategies  a web-based qualitative data analysis platform. Excerpts associated with key codes are summarized and tabled for comparison and theme identification.Analyses of all qualitative data consist of organization and examination following the principles of thematic and Framework Analysis , useful ecify It ) and howAnalyses of our quantitative data will use SAS  statistiThe clinical benefits of rapid ART start following diagnosis of HIV infection have been demonstrated in numerous studies, accelerating entry into care, shortening times to viral suppression, declining in morbidity, and preventing onward transmission. However, models for delivery of rapid ART start are not well studied or articulated in ways that provide practical guidance about how services should be organized and delivered to achieve maximum outcomes, or to reach diverse groups of patients, particularly those who are most marginalized in the health system, including sexual and racial minorities and those who are unhoused, have experience in carceral systems, use substances, or have a diagnosis of mental illness. Strategies remain untested, uncodified, and have not been disseminated to promote wide uptake of rapid ART start in all HIV care settings. Accelerating access to care and ART initiation, this initiative will help prevent the onward transmission of HIV, developing and testing models of care that will help newly diagnosed patients and those out of care quickly achieve viral suppression. This initiative’s work can reduce barriers to care for all patients, reducing disparities and increasing equity.Current studies reflect specific contexts and populations and do not account for implementation strategies within service delivery models, nor for patient population contexts , 66. The


In 2022, 13,586 candidates applied to subspecialty fellowships. Formal resources to inform candidates on subspecialty-specific fellowship application are limited. Candidates rely on residency application experience, informal advice, and online research for navigating the application process. Thus, a need exists for formal subspecialty-specific fellowship application guidance. The American Academy of Pediatrics Organization of Neonatal-Perinatal Medicine Training Program Directors (ONTPD) and Trainees and Early Career Neonatologists (TECaN) created a webinar-based curriculum to help educate trainees about the application process and recruit diverse fellowship applicants.In 2022, ONTPD and TECaN co-hosted and implemented a four-part national webinar series focused on different aspects of the Neonatal-Perinatal Medicine (NPM) fellowship application and interview processes. Webinars were advertised through list-servs and social media, conducted in two time zones, and recorded for asynchronous viewing. Registration, demographic data, and questions for webinar panelists were collected via electronic survey. Program evaluation data was collected after each webinar and following the fellowship match.In the 2022 appointment year, 310 candidates participated in the NPM fellowship match and 250 individuals registered for the webinar series. A quarter (26%) of registrants identified as underrepresented in medicine. Most registrants reported minimal or no knowledge of the fellowship application  and interview  processes. The majority of registrants  were planning on applying to fellowship in 2022, and 91% of post-webinar respondents (43/47) felt the webinars were moderately or extremely helpful, a finding that was sustained beyond the match (37/42). Almost all respondents rated the quality of the webinars as good or higher and were likely or very likely to recommend them to peers (39/42). There was considerable variability amongst respondents in the number of fellowship programs applied to, interviewed at, and ranked, and factors influencing rank list.We describe a virtual curriculum to prepare trainees for the NPM fellowship application and interview process. This webinar series provides needed education to fellowship candidates, bridges the gap between candidate knowledge and program director expectations, is generalizable to other specialties, and can be replicated with minimal resources. In 2022, 13,586 candidates applied for subspecialty fellowships, making it the largest application cycle in the history of the National Resident Matching Program (NRMP) Specialties Matching Service (SMS) ..4].Our webinar evaluations indicated these were high quality webinars and respondents were likely to recommend them to a peer. In addition, almost all of the respondents felt the webinars were helpful, demonstrating effectiveness in improving NPM fellowship applicants' comfort with the application and interview process Table . The “MaLimitations to our study include low post-webinar survey response rates and subsequently respondent bias, which hinders the ability to generalize our results. In addition, our advertisement was through email, social media, and word of mouth, therefore we may have missed some of the potential applicant population who would be interested in registering for and attending our webinars.Additional future directions of this webinar series will be based on participant feedback and program evaluation data. This webinar series will continue to be offered to NPM fellowship applicants on an annual basis for the foreseeable future. Future webinars will provide updated information as changes are made to the fellowship application by ERAS, such as geographic signaling, impact statements, and identification of significantly meaningful experiences.This webinar series serves to bridge the gap between fellowship applicants’ understanding of fellowship recruitment processes and fellowship program director expectations of applicants. The education provided was found to be helpful and of high quality. This webinar series is generalizable to other subspecialties across GME and can easily be replicated with minimal resources.


For parent-respondents only, ACE-Q scores were significantly greater for the CSD class compared to the CS and L classes. Pediatricians should attend to the needs of caregivers reporting both stress and depression, as these families may face especially high levels of adversity and low levels of protective factors.Pediatricians are well-positioned to screen for early childhood adversities, but effective responses to positive screens require an understanding of which adversities typically co-occur, and to what extent they are associated with other risk or protective factors. Among children seen at an urban academic pediatric practice, this study aimed to (1) examine the prevalence of different types of early adversity and protective experiences reported by primary caregivers, and (2) define latent classes of co-occurring adversities. Of 1434 children whose parents completed the Safe Environment for Every Kid (SEEK) at well-child visits during November 2019–January 2021, three classes of adverse experiences emerged, including those reporting low adversity , caregiver stress , and both caregiver stress and depression . Among those who also completed the Adverse Childhood Experiences Questionnaire (ACE-Q,  Childhood exposure to traumatic events has been broadly recognized as a major public health problem in the United States ,2,3. EarPediatricians are often tasked with identifying children and families who may need additional support to combat experiences of adversity ,20,21. FDespite recommendations that pediatricians screen for adverse childhood experiences (ACEs) and implement strategies to promote safe, stable, nurturing relationships within those families determined to be at the highest risk ,25,26,27Furthermore, few studies have examined rates and co-occurrence of both adversity and family strengths within the same study. Recent calls within the pediatric community have noted the importance of focusing on family strengths and resilience rather than applying a purely deficit or problem-focused perspective ,22,34. AThis study examined caregiver-reported family adversities, including those known to be associated with child maltreatment , among aParticipants were caregivers of 1434 unique pediatric patients  seen at a high-volume , urban primary care pediatric practice affiliated with an academic children’s hospital in the Midwest region of the United States. Caregivers were mostly (72.6%) parents; however, the remaining (27.4%) consisted of aunts/uncles, grandparents, and biologically unrelated primary caregivers. Slightly over half of those screened for early adversity were 0–3 years of age (55.9%), while 44.1% were 3–6 years of age.Detailed demographic data are not available for the specific sample included in this report as only child age, child biological sex, and caregiver type were on the primary screening instrument; however, they are believed to represent the general patient population at the clinic, where patients identify primarily as Black or African American (>94%) and are largely Medicaid-eligible (>85%). Over 70% of pediatric patients seen at the clinic reside in the neighborhoods directly surrounding the clinic located in Cleveland, Ohio, which make up the city’s most impoverished historically red-lined neighborhoods. About 20% of patients live in first-ring suburbs, and the remainder lives throughout the Northeast area of Ohio.As part of regular clinical care, caregivers accompanying young children ages 6 months to 6 years to well childcare visits were asked to complete several screeners addressing social determinants of health, adversity, and family protective factors. Paper screens were presented to caregivers upon check-in and rooming by clinic staff and completed on their own, either in the waiting room or the clinic exam room before the pediatric provider came in. Screeners took a maximum of 15 min to complete. Screens included a cover sheet explaining the purpose of the screeners as follows:Dear Parent/Caregiver/Guardian,At the Rainbow Center for Women & Children we care about the total health and wellness of every child and family we serve. As part of our wellness program we ask that a variety of screenings be completed so that our providers have a better understanding of how we can support our patients. We have added screenings about life stress, as well as about family strengths and supports. The forms ask some personal questions; we will keep this confidential among our healthcare team. The purpose of these screenings is to help us better support you and your child. Thank you for partnering with us during your child’s/children’s care.Caregivers gave completed screeners to their pediatric provider(s) during the visit; providers discussed responses with patients and collaborated on any needed follow-up supports. Screeners were scanned and uploaded to the patient’s electronic medical records.For the current study, responses on screeners collected between November 2019 (when these particular screeners were first implemented in the pediatric clinic) and January 2021 were utilized; data included screens collected one time per child. This study was reviewed and approved as a chart review study by the University Hospitals Cleveland Medical Center Institutional Review Board (IRB).Safe Environment for Every Kid : The ACProtective and Compensatory Experiences Scale : The PAp < 0.05) BLRT and LMR-LRT adjusted tests and lower BIC and AIC values indicate good model fit. Entropy and posterior probabilities, which are indicators of model classification quality, and interpretability of the classes were also considered in determinations of the best fitting model. All LPA analyses were completed in Mplus with MLR for missing data. Mean differences between classes in the best-fitting model and scores on the ACE-Q and PACES were analyzed in a multivariate generalized linear model (GLM) framework in SPSS with exported class membership from Mplus. Effect sizes for each model are provided.Prevalence data for adversities represented on the SEEK were described by the number and percent of caregivers who endorsed each item to characterize the sample. Heterogeneity in SEEK endorsement was analyzed using latent class analysis (LCA). LCA identifies the presence (or absence) of underlying individual differences within a sample which allows researchers to draw conclusions about meaningful, homogeneous subsets or groups of participants within a heterogeneous sample . AnalytiSample characteristics and SEEK item endorsement are reported in n = 1034, 73%), (2) caregiver stress , and (3) caregiver stress and depression   . Belongi39, 10%) .n = 916), excluding those who identified as a relative caregiver or nonbiological caregiver. In this analysis , but also health-related social determinants of health , and more general expressions of caregiver distress . Both the setting—a university-based pediatric primary care resident continuity clinic—and the families served—those residing in a low-income urban community—are similar to the settings and populations assessed in prior studies of the implementation and performance of the SEEK instrument ,47.Despite these risks and concerns, the majority of families (73%) were empirically identified as a “low adversity” class/group, among whom relatively few (14%) screened positive in any category on the SEEK. Not surprisingly, these families with fewer contextual stressors or safety concerns reported significantly fewer adversities on the ACE-Q and significantly greater protective factors on the PACEs questionnaire.The two remaining classes are notable for their shared inclusion of caregiver stress, with a greater proportion of families (30–45%) in these groups also indicating higher reports of food insecurity relative to the low class. This grouping is consistent with SEEK data generated by other primary care practices implementing this screener, where major parental stress and food insecurity were the most common risks endorsed by parents . Given tCaregiver experiences of stress, even in the absence of material hardship, have been linked with adverse outcomes for children in other ways. For example, parenting stress is higher among parents of children with emotional and behavioral challenges . ParentiIt is notable that in analyses using the full sample, no differences in total ACEs or PACES were found between caregivers with elevated stress compared to caregivers with elevated levels of stress and depression. However, in post hoc exploratory analyses including only respondents who identified as the child’s parent, the caregiver stress and depression and caregiver stress classes still had higher average scores on the ACE-Q and lower average scores on the PACES than did the low-adversity class; additionally, the caregiver stress and depression class had higher ACE-Q scores than the caregiver stress (only) class. It appears from these data that the presence of parent stress with a depressed mood may have different meanings for children when experienced by parents as compared to other caregivers in terms of the total burden of adversity. For example, it may be that non-parent caregivers are not fully aware of children’s history of adverse experiences if they have not been the child’s primary caregiver for their entire life. Additionally, results may have differed in other ways depending on the distribution of caregiver type; for example, other latent classes may have emerged if the sample was made up entirely of biological parents or non-biological parents. The absence in this dataset of further demographic details, such as caregiver age, precludes further analyses to explore the meaning of these differences. For example, we cannot determine whether non-parent caregivers are older than parents, with accordingly greater life experience or different support networks to overcome the potential impact of the caregiver’s mood on the household environment. Additionally, the absence of detailed demographic data makes it challenging to determine if family experiences are similar across other demographic subgroups. Given what we know about social determinants of health, it stands to reason that associations among adversity and protective factors may differ based on family race or ethnicity, income, and other factors that are associated with greater instances of systemic and individual discrimination ,20,22.Other limitations of this study relate to the period during which data were collected; the earliest sets of screens were collected from a subset of clinicians piloting the new screener workflow within the practice prior to the onset of the COVID-19 pandemic public health emergency, with the expansion of screening to the full practice by spring 2020. These analyses did not account for secular pressures that might have influenced what needs families reported and in what combinations, such as changes in food or housing policies, or employment changes that might have influenced which caregivers reported extreme stress. Future analyses should explore whether the classes observed here  are reproduced after the public health emergency ends. Finally, the specific profiles of adverse experiences identified through LCA in the present study may not generalize to other samples or measures of adversity. The large, representative nature of the sample does protect against this limitation to some extent; still, future work is needed to understand the experiences of caregiver stress and depression and the impact of their combination on the lives of young children. Replication of this work is also needed to explore factors that might affect class membership; we were only able to examine child sex and age in the present study due to the limited availability of demographic characteristics.Among this population of US children obtaining well-child care in an urban pediatric primary care resident continuity clinic, three empirically derived classes of adverse experiences based on the SEEK screen were identified: (1) low adversity, (2) caregiver stress, and (3) caregiver stress and depression. Overall, belonging to the low-adversity class was associated with lower ACE-Q and higher PACES compared to the other two classes, while among biological parents, the caregiver stress and depression class was associated with the highest ACE-Q scores. Findings from the current study affirm that screening for specific types of adversity (rather than solely counts of adversities) is feasible within the context of well-child visits and that profiles of adversity  can provide actionable information for clinicians and practices. In particular, experiences of stress and depression by caregivers of young children may be strong indicators of elevated risk factors and low levels of protective factors.Future work is needed to better understand the meaning of the caregiver stress and caregiver stress and depression classes and what caregivers interpret as “extreme stress.” While other measures of adversity  might geIndeed, in the US, emerging state and federal policies are incentivizing clinicians and practices to implement whole family or multigenerational interventions under an accountable healthcare framework that considers the broad set of social factors influencing physical emotional developmental, and relational health ,61. For In sum, pediatricians are well situated to survey early risk and provide guidance to caregivers and linkage to prevention programs and supports to benefit children and their families ,13. Base