Young women who sell sex (YWSS) in Zimbabwe remain at high risk of HIV infection. Effective HIV prevention strategies are needed. Through support to access a combination of evidence-based interventions, including oral pre-exposure prophylaxis (PrEP), the Determined, Resilient, Empowered, AIDS-free, Mentored and Safe (DREAMS) partnership aimed to reduce new HIV infections among adolescent girls and young women by 40% over 24 months.Non-randomised ‘plausibility’ evaluation, powered to detect a 40% HIV incidence difference between DREAMS and non-DREAMS sites. Two large cities with DREAMS funding were included, and four smaller non-DREAMS towns for comparison. In all sites, YWSS were enrolled to a cohort through peer-referral. Women were followed up for 24 months. HIV seroconversion was the primary outcome, with secondary outcomes identified through a theory of change. Outcomes were compared between YWSS recruited in DREAMS cities and non-DREAMS towns, adjusting for individual-level confounders and HIV prevalence at enrolment.From April to July 2017, 2431 women were enrolled, 1859 of whom were HIV negative at enrolment; 1019 of these women (54.8%) were followed up from March to May 2019 and included in endline analysis. Access to clinical services increased, but access to socioeconomic interventions promoted by DREAMS was limited. A total of 79 YWSS HIV seroconverted, with HIV incidence among YWSS in DREAMS cities lower (3.1/100 person-years) than in non-DREAMS towns (5.3/100 person-years). In prespecified adjusted analysis, HIV incidence was lower in DREAMS cities but with weak statistical evidence (adjusted rate ratio (RR)=0.68; 95% CI 0.40 to 1.19; p=0.18). Women in DREAMS cities were more likely to report ever and ongoing PrEP use, consistent condom use, fewer sexual partners and less intimate partner violence.It is plausible that DREAMS lowered HIV incidence among YWSS in two Zimbabwean cities, but our evaluation provides weak statistical evidence for impact and suggests any reduction in incidence was lower than the anticipated 40% decline. We identified changes to some important ‘pathways to impact’ variables, including condom use. Young women who sell sex (YWSS) in Eastern and Southern Africa are at high risk of HIV.Available evidence shows that these women are less engaged with HIV prevention services.There are few evaluations of interventions for the broader population of YWSS.We found higher engagement with clinical services among women in the places where DREAMS investments were made and differences in pre-exposure prophylaxis (PrEP) use between women recruited in the Determined, Resilient, Empowered, AIDS-free, Mentored and Safe (DREAMS) cities and non-DREAMS towns, at 24-month follow-up.After 24 months of follow-up, HIV incidence was high in both groups, and lower in DREAMS cities but with weak statistical evidence of a difference between the two groups.We found evidence for an impact on some secondary outcomes, including condom use, violence and number of partners, and some lessons for implementation.Overall, YWSS reported little uptake of social protection services.Implementation challenges, including limited experience of implementing partners working with general populations in working with YWSS and barriers faced by YWSS in accessing these general population services, need to be addressed to improve coverage of combination HIV prevention services.The available evidence indicates the continued need to implement and evaluate combination HIV prevention services for YWSS, including women who do not identify as sex workers.Further efforts are needed to identify approaches that enhance access to social protection services among YWSS combined with delivery of biomedical interventions, including PrEP.In Eastern and Southern Africa, adolescent girls and young women (AGYW) aged 15–24 accounted for 26% of all new HIV infections in 2018.In 2015, the DREAMS  partnership was launched in ten sub-Saharan African countries, including Zimbabwe. Through financial and technical support for a combination of evidence-based interventions, DREAMS aimed to reduce new HIV infections among AGYW by 40% over 24 months.We evaluated whether offering HIV testing services and increasing the availability of PrEP to YWSS aged 18–24, combined with community mobilisation and social protection interventions supported by DREAMS, reduced the number of new HIV infections by 40% over 24 months, compared with HIV incidence among YWSS residing in areas where DREAMS investments were not available.We conducted a non-randomised ‘plausibility’ evaluation of the impact of DREAMS on HIV incidence and other secondary outcomes.The study was conducted with YWSS in two large cities and four smaller towns in Zimbabwe. The cities were selected purposively as locations where DREAMS funding was being provided. The four smaller towns were selected for comparison based on their similarity with the DREAMS cities according to data from the ‘Sisters with a Voice’ (Sisters) programme,In DREAMS cities, in addition to Sisters, a package of social and clinical interventions was available to YWSS. The DREAMS package was delivered through several implementing partners in the two cities; services available included social protection, life skills, education and vocational training, gender-based violence prevention and care, and HIV prevention, including condom promotion and distribution, an offer of PrEP combined with community empowerment and adherence support for those at highest risk of HIV . CommuniWithin each city/town we recruited individuals into the study cohort through peer-referral using the same approach to respondent-driven sampling (RDS) we have used in previous surveys.We conducted community mapping to identify hotspots where young women sell sex to select ‘seed’ participants.10.1136/bmjgh-2020-003892.supp1Supplementary dataIn both groups, YWSS were interviewed and offered HIV testing using a rapid HIV test at enrolment into the study in 2017. Budgetary constraints meant that women recruited in DREAMS cities were followed up at 12 months and 24 months after the initial enrolment survey, with women in non-DREAMS towns followed up at 24 months only. In 2019, at 24-month follow-up, study participants were re-interviewed and offered HIV testing as before. Complementary strategies were implemented to retain women in the study in the DREAMS and non-DREAMS groups, including WhatsApp messaging, outreach activities and engaging YWSS who recruited women to the study to inform women about the study follow-up.At enrolment and follow-up, women were offered counselling and HIV testing services using a serial HIV testing algorithm adapted from Zimbabwe's national testing and counselling guidelines. Determine HIV-1/2 or First Response HIV-1-2 kit antibody testing was used as the first screening test. Where the result was HIV positive, this was confirmed using First Response HIV-1-2 kit or Determine HIV-1/2. Where the two test results were discordant, repeat testing was advised within 2 weeks. All women were given the results of their HIV test.The primary outcome was incident HIV infection over the 24-month study period. The outcome was calculated as the number of new HIV infections divided by the total person-years of follow-up during the 24-month study period, among YWSS who tested HIV-negative at enrolment. Selection of secondary outcomes reflected the hypothesised causal pathway for how DREAMS would reduce HIV incidence .5 We assAs part of a broader process evaluation, we collected qualitative data at three time points during the trial from implementing partner staff and participants with varying levels of engagement in different DREAMS interventions. We conducted semi-structured interviews with 14 implementing partner staff and 23 participants that explored perceptions and experiences of delivering or receiving DREAMS activities. We interviewed a further 17 participants 2–3 times to identify barriers and facilitators to engaging with the programme over time.In line with DREAMS targets, the study was powered to detect a 40% or greater reduction in HIV incidence after 24 months, assuming that HIV incidence in the comparison group would be 5.0%–8.0% per annum, as suggested by previous studies in Zimbabwe.We compared sociodemographic characteristics of enrolled YWSS by site and study group, including HIV status. We presented a flow diagram to illustrate enrolment and retention in the study at 24-month follow-up, and examined patterns of retention by age, marital status, highest level of education attained and whether women self-identified as FSW, all measured at enrolment. Using data at 24-month follow-up, we described self-reported uptake of DREAMS-related services in all sites and used logistic regression adjusted for age and site to compare uptake between groups.Reflecting the non-randomised study design, we identified potential confounding factors to adjust for to obtain the fairest comparisons between the two study groups at endline. Critically, HIV prevalence at enrolment differed between the two study groups. Using factors considered a priori as associated with HIV prevalence at enrolment or likely to be associated with HIV incidence,Primary analysis was based on the intention-to-treat principle, comparing cohorts recruited in the two DREAMS cities and four non-DREAMS towns regardless of participants’ individual-level engagement with services. We calculated HIV incidence and the prevalence of secondary outcome measures for each study site. We pooled the data across study sites and used Poisson regression with follow-up time estimated as the time between interview dates, or half of this for women who seroconverted. We first conducted an age-adjusted analysis comparing HIV incidence between the DREAMS cities and non-DREAMS towns, and subsequently conducted a fully adjusted analysis including the confounders listed above.For secondary outcomes, we used logistic regression adjusted for confounders listed above and for each respective secondary outcome measured at enrolment. In these analyses, where <5 women reported the outcome in a site, we used Fisher’s exact test.Our primary analysis excluded data collected from women followed up at 12 months post-enrolment in the DREAMS cities. In sensitivity analysis, we included this data. If a woman seroconverted by the 12-month follow-up survey, we placed the seroconversion date at the mid-point between enrolment and 12 months. If a woman seroconverted between the 12-month and 24-month follow-up surveys, we placed the seroconversion date at the mid-point between these surveys.In a second sensitivity analysis, we applied the principles of the RDS-II method, dropping seed participants from the analysis and weighting responses for each woman by the inverse of her self-reported network size at baseline .10Thirdly, assuming a uniform distribution and using 50 imputations, we randomly imputed the seroconversion date as a fraction of the interval between enrolment and 24-month survey dates, among women who seroconverted by the 24-month follow-up survey.11 12FSW work closely with the research team at the Centre for Sexual Health and HIV/AIDS Research, Zimbabwe, however, were not directly involved in the development of the research questions and design of the study. FSW helped design and implement social mapping to identify YWSS hotspots to guide survey processes and to support RDS. YWSS were integrally involved in running all aspects of the DREAMS implementation for YWSS.The Bill & Melinda Gates Foundation staff reviewed and provided non-binding comments on the protocol and study analysis plan, but had no role in data collection, data analysis, data interpretation or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.Between April and July 2017, we enrolled 2431 women (1204 in DREAMS cities and 1227 in non-DREAMS towns). Nineteen women had missing HIV test data. Among the remaining women, 76% (n=1859) tested HIV negative at enrolment  women testing HIV negative at enrolment were retained in the study When YWSS did engage in opportunities, they sometimes felt stigmatised. We received reports of returning school pupils being publicly identified as ‘sex workers,’ denied access to the library, or made to sit outside classrooms and observe teaching through open windows:Some [other children] will be saying, ‘why have you come to school? Why did you leave your work of selling sex?’ to make a scene in front of other people. … at times I felt so ashamed and I would feel it’s better not to go there. Other barriers stemmed from YWSS’ entrenched poverty, which meant they could not easily devote time to their studies, needing to earn money or look after family members. There were also additional costs such as bus fares and purchasing supplies for vocational training that YWSS could not afford. Other barriers to uptake included competing demands on YWSS’ time, that is, caring for siblings, and their inability to pay for costs associated with participation, such as transport to classes or uniforms and equipment required for vocational training. These challenges meant that YWSS had limited ability to access ‘layered’ services as intended.Overall, 79 YWSS HIV seroconverted after 24 months. HIV incidence was 3.60/100 person-years in DREAMS Site A and 2.76/100 person-years in DREAMS Site B. In non-DREAMS towns, the rate of new infections ranged from 4.28/100 person-years to 7.07/100 person-years .HIV incidence among YWSS in DREAMS cities was 3.1/100 person-years compared with 5.3/100 person-years among YWSS in non-DREAMS towns . In our primary analysis, adjusting for confounding variables, HIV incidence was lower in DREAMS cities than in non-DREAMS towns, but with weak statistical evidence of a difference (adjusted rate ratio (RR)=0.68; 95% CI 0.40 to 1.19; p=0.18) .In a sensitivity analysis, including data from women in DREAMS cities followed up at 12 months, we found a very similar result (adjusted RR=0.79, 95% CI 0.47 to 1.32; p=0.36; Compared with women in the non-DREAMS towns, YWSS in the DREAMS cities were more likely to report ever having used PrEP (28.1%, n=151/538 vs 0.6%, n=2/481; More women in the DREAMS cities reported the ability to negotiate condom use (92.6%; n=498/538 vs 81.0%, n=389/480; adjusted OR=3.39 95% CI 2.24 to 5.14; Secondary outcomes varied across sites and, for some outcomes, were not always in the direction expected . For exaAfter 24 months of follow-up, we found that HIV incidence was lower among YWSS recruited in two Zimbabwean cities implementing the DREAMS programme than in four towns where DREAMS was not implemented. After adjustment for baseline imbalance in HIV prevalence and predictors of HIV incidence, this potential effect of DREAMS on HIV incidence was not statistically significant. We identified some changes in outcomes on important ‘pathways to impact’: YWSS recruited in DREAMS cities reported fewer clients in the past month, less condom-less sex in the last month and higher levels of PrEP use at 24-month follow-up; yet we found little difference in women’s engagement with social protection services. Despite our null finding, it remains plausible that DREAMS investments may have contributed to reduced HIV incidence among YWSS through increased access to clinical HIV prevention services, including PrEP and condoms.Our study had limitations, most of which we were aware of from the start.We used data collected by the national FSW programme to identify comparison sites similar to the DREAMS cities based on, among other factors, attendance to the programme and the proportion of attendees aged 18–24.Finally, our secondary outcomes were self-reported via face-to-face-interviews and thus potentially prone to social desirability bias and the potential for an ‘intervention’ effect of more frequent contact and follow-ups among the YWSS in DREAMS cities over time.Intriguingly, in a sensitivity analysis where we dropped seed participants and applied RDS-II weighting, we found a borderline significant impact of the programme aligned with the 40% reduction hypothesised. We decided a priori not to use this weighting in the primary analysis, and consider interpretation of these findings complex, but report them for transparency. At enrolment, RDS diagnostics suggested that women recruited were representative of the network of YWSS in the majority of sites, but convergence was not realised in one of the smaller non-DREAMS towns suggesting that YWSS may not have been representative of the broader network of YWSS in this site.13Despite these limitations, our study provides invaluable and important evidence in relation to HIV prevention efforts for a critical population. Despite their vulnerability, YWSS are often underrepresented in research.Our findings of a plausible effect of DREAMS, adds an important new finding to a small overall evidence base. In Burkina Faso, a cohort study combined with modelling found a significant effect of integrated prevention and care services combined with peer-led education sessions on HIV incidence among young FSW.Although HIV incidence was lower among YWSS in DREAMS cities, HIV incidence was high in all sites. Our study reiterates that more needs to be done to strengthen implementation of evidence-based HIV prevention interventions for YWSS. YWSS, both those who do and do not self-identify as FSW, are often missed by research and programmes.PrEP formed a component of the DREAMS core package. Current use of PrEP among women participating in our study was low at 24 months, consistent with findings reported by PrEP efficacy and implementation studies with AGYW, including FSW, in sub-Saharan Africa,29To reduce HIV incidence by 40% over 24 months, DREAMS endeavoured to provide AGYW with a combination of evidence-based HIV prevention interventions to address the multiple factors influencing HIV risk.


What is the current rate of adoption of first-line systemic treatment with fluorouracil, folinic acid, oxaliplatin, and irinotecan plus bevacizumab (FOLFOXIRI-B) in daily practice?In this cross-sectional study of 282 patients in the Netherlands, FOLFOXIRI-B prescription rates marginally increased in the past 5 years. During the study period, 1 in 7 estimated, eligible patients was treated with FOLFOXIRI-B.The results of this study suggest that, despite evidence of the effectiveness of FOLFOXIRI-B therapy, use of this therapy for metastatic colorectal cancer remains low, possibly because of oncologists’ reported preference for doublet therapy. Triplet chemotherapy with fluorouracil, folinic acid, oxaliplatin, and irinotecan plus bevacizumab (FOLFOXIRI-B) is an effective first-line treatment option for patients with metastatic colorectal cancer (mCRC). However, the degree of implementation of FOLFOXIRI-B in daily practice is unknown.To evaluate the current adoption rate of FOLFOXIRI-B in patients with mCRC and investigate the perspectives of medical oncologists toward this treatment option.This 1-week, multicenter, cross-sectional study in the Netherlands used a flash mob design, which facilitates ultrafast data generation (flash) through the engagement of numerous researchers (mob). During the study week , patient data were retrieved from electronic health records of 47 hospitals on patients with mCRC who were referred to a medical oncologist between November 1, 2020, and January 31, 2021. Interviews were simultaneously conducted with 101 medical oncologists from 52 hospitals who regularly treat patients with mCRC.First-line systemic treatment as determined by the treating physician.The FOLFOXIRI-B prescription rate was the main outcome. Current practice was compared with prescription rates in 2015 to 2018. Eligibility for treatment with FOLFOXIRI-B was estimated. An exploratory outcome was medical oncologists’ reported perspectives on FOLFOXIRI-B.A total of 5948 patients in the Netherlands  were treated with first-line systemic therapy for synchronous mCRC. A total of 282 patients with mCRC underwent systemic therapy during the study period (2021). Of these 282 patients, 199 (71%) were treated with intensive first-line therapy other than FOLFOXIRI-B, of whom 184 (65%) were treated with oxaliplatin doublets with or without bevacizumab; 14 (5%) with irinotecan doublets with or without bevacizumab, panitumumab, or cetuximab; and 1 (0.4%) with irinotecan with bevacizumab. Fifty-four patients (19%) were treated with fluoropyrimidine monotherapy with or without bevacizumab, 1 patient (0.4%) with panitumumab monotherapy, and 3 (1%) with immune checkpoint inhibitors. In total, 25 patients  were treated with first-line FOLFOXIRI-B compared with 142  in 2015 to 2018. During the study period, 21 of 157 eligible patients (13.4%) in the Netherlands were treated with FOLFOXIRI-B. A total of 87 medical oncologists (86%) reported discussing FOLFOXIRI-B as a treatment option with eligible patients. A total of 47 of 85 (55%) generally communicated a preference for a chemotherapy doublet to patients. These oncologists reported a significantly lower awareness of guidelines and trial results. Toxic effects were the most reported reason to prefer an alternative regimen.The findings of this study suggest that FOLFOXIRI-B prescription rates have marginally increased in the last 5 years. Considering that most medical oncologists discuss this treatment option, the prescription rate found in this study was below expectations. Awareness of guidelines and trial data seems to contribute to the discussion of available treatment options by medical oncologists, and the findings of this study suggest a need for repeated and continuing medical education. This cross-sectional study compares current rates of prescription of fluorouracil, folinic acid, oxaliplatin, and irinotecan plus bevacizumab (FOLFOXIRI-B) with rates from 2015 to 2018 and assesses Dutch medical oncologists’ perspectives on treatment with FOLFOXIRI-B. The efficacy and safety of FOLFOXIRI-B have been assessed by several studies, including 2 phase 3 randomized clinical trials.4 In the TRIBE  trial, FOLFOXIRI-B was compared with fluorouracil, folinic acid, irinotecan, and bevacizumab (FOLFIRI-B), showing an improvement in progression-free survival of 2.6 months and, as a secondary end point, improved overall survival of 4 months.3 Since publication of the TRIBE trial in 2014, the triplet regimen plus bevacizumab has been incorporated into national and international guidelines.8In the past 2 decades, the choice of first-line treatment of patients with metastatic colorectal cancer (mCRC) has become more complex because of the increased availability of treatment options.4 which demonstrated that upfront FOLFOXIRI-B, followed by reintroduction after first disease progression, was superior to sequential administration of fluorouracil, oxaliplatin, and folinic acid plus bevacizumab (FOLFOX-B) and FOLFIRI-B . The efficacy of this regimen was confirmed in a meta-analysis9 that included all available data from randomized clinical trials. This regimen was subsequently approved in 2020 by Dutch health care authorities.10 No subgroup has been identified for which no benefit is expected,9 although efficacy may be compromised in patients previously treated with oxaliplatin-based adjuvant chemotherapy. Because FOLFOXIRI-B has not been prospectively compared to doublet chemotherapy plus an antibody to the epidermal growth factor receptor in patients with left-sided primary and RAS V600/BRAF (OMIM 164757) wild-type tumors, its benefit in this subgroup is uncertain.Additional evidence for FOLFOXIRI-B was recently provided by the TRIBE2  trial in 2020,9 Furthermore, FOLFOXIRI-B treatment involves the need for a central venous catheter and biweekly schedule. The latter is a relevant difference to the frequently used, 3-weekly capecitabine-based regimen. The published data justify FOLFOXIRI-B as a first-line treatment for eligible patients. However, the benefit-burden tradeoff presumably differs per individual.Compared with a doublet chemotherapy backbone, FOLFOXIRI-B leads to a higher incidence of grades 3 to 4 diarrhea, (febrile) neutropenia, nausea, and mucositis.11 Using this concept as a research design allows us to maximize the number of patients, physicians, and hospitals represented within a short time frame.17 Third, we investigate Dutch medical oncologists’ perspectives on treatment with FOLFOXIRI-B.In this study, we assess the adoption of FOLFOXIRI-B treatment for patients with mCRC in clinical practice in 3 parts. First, we assess the prescription of FOLFOXIRI-B from January 1, 2015, to December 31, 2018. Second, we explore the current prescription of FOLFOXIRI-B . To provide a true representation of current practice, we used a cross-sectional flash mob study design. A flash mob is defined as “a group of people summoned to a designated location at a specified time to perform an indicated action before dispersing.”STROBE) reporting guideline.18The objective of this cross-sectional study was to assess the adoption of FOLFOXIRI-B as a first-line systemic regimen for mCRC patients in daily practice. For this purpose, we made use of 3 data sources: the Netherlands Cancer Registry (NCR), patient data collected over the course of 1 week from 47 Dutch hospitals, and interviews with medical oncologists  mCRC between January 1, 2015, and December 31, 2018, and subsequently started first-line systemic therapy. Data on metachronous metastases are not included in the NCR.To assess the current adoption of FOLFOXIRI-B, we conducted a nationwide, 1-week, multicenter, cross-sectional observational study, also known as a flash mob design. All Dutch hospitals (n = 72) were invited to participate in this flash mob study through members of the Dutch Society of Medical Oncology (NVMO) and the Dutch Colorectal Cancer Group. In total, 47 of the 72 medical centers participated. Six of the 47 medical centers could not comply with timely regulatory requests. In these centers, data were collected by trained data managers from the NCR.All patients with a new diagnosis of mCRC who were referred to a medical oncologist in participating centers between November 1, 2020, and January 31, 2021, were included if first-line systemic therapy was administered before March 6, 2021. During the study week , clinicopathological data, including age, sex, Eastern Cooperative Oncology Group (ECOG) performance status (PS), tumor sidedness, molecular characteristics, time to metastasis, location of metastasis, primary tumor resection, previous exposure to oxaliplatin-based adjuvant chemotherapy, first-line clinical trial participation, and first-line systemic therapy, were collected from electronic health records by internal medicine residents, oncology nurses, and medical interns under the supervision of a medical oncologist , all of whom were local to the participating center. Patient records were anonymized and variables were entered into electronic case report forms (Castor). All data collectors were trained to collect the data according to prespecified criteria.20 We estimated which patients would have been eligible for treatment with FOLFOXIRI-B. We based our definition for estimated eligibility on the TRIBE trials’ eligibility criteria: patients 75 years or younger with an ECOG PS of 0 to 2 who were treated with oxaliplatin doublets  or FOLFOXIRI with or without bevacizumab. We chose to limit the definition to oxaliplatin-based systemic treatment to exclude patients who would not have been eligible because of oxaliplatin contraindications, such as peripheral neuropathy. Patients in clinical trials with systemic treatment were excluded. We pooled patients treated with FOLFOXIRI with and without bevacizumab. Therefore, further mention of FOLFOXIRI-B in this article also includes FOLFOXIRI  between groups. A χ2 test was used for group comparison of categorical variables. When groups were too small, we used the Fisher exact test. A 2-sided P < .05 was considered statistically significant.Data were exported from the electronic case report form to SPSS, version 27.0 (SPSS Inc). Descriptive statistics were used to present baseline characteristics. A 2-tailed, unpaired RAS variant  was present in 1514 of 2948 patients (51%) and a V600BRAF variant in 392 of 2560 patients (15%). Baseline characteristics are given in eTable 1 in From 2015 to 2018, a total of 5948 patients  were treated with first-line systemic therapy for synchronous mCRC in the Netherlands. Metastases were confined to the liver in 2277 patients (38%) of patients, and a From March 1 to March 5, 2021, clinical data were collected from 402 patients with mCRC in 47 of a total of 72 Dutch hospitals, of whom 120 patients did not meet study eligibility criteria and were excluded from analysis  and more frequently treated with adjuvant oxaliplatin-based chemotherapy  compared with estimated eligible patients not treated with FOLFOXIRI-B. Notable absolute differences were seen in ECOG PS, liver-only metastasis, and BRAF status; however, these findings were not significantly different.Since 2015 to 2018 (data source 1), FOLFOXIRI-B prescription increased from 2%  to 9%  in 2020 to 2021 (data source 2). We estimated that, currently (data source 2), 157 of 282 patients (56%) were eligible for treatment with FOLFOXIRI-B . Twenty-In total, 101 medical oncologists  working at 52 different hospitals and involved in the treatment of patients with mCRC agreed to be interviewed in the study week. Baseline characteristics of medical oncologists are presented in BRAFV600 and/or RAS variant, or with sidedness) (BRAFV600 and/or RAS variant (16 [19%]), and only patients with potentially resectable metastases (14 [16%]). A total of 47 medical oncologists (54%) reported comorbidity as a reason that they would allow oxaliplatin-containing doublets plus bevacizumab but not FOLFOXIRI-B. In all, 70 of 73 medical oncologists (96%) claimed that the COVID-19 pandemic had no effect on FOLFOXIRI-B prescription rates in the 4 months before the study week  reported that they did not discuss FOLFOXIRI-B with their patients. These medical oncologists were older  and more often male (12 of 14 [86%]) compared with medical oncologists who did discuss FOLFOXIRI-B as a treatment option  . Reportededness) , A. MediReasons why patients refrain from treatment with FOLFOXIRI-B, according to medical oncologists, are presented in the Among 85 medical oncologists, 70 (82%) agreed with the statement, “I leave the therapy choice up to the patient. For patients the survival benefit of FOLFOXIRI-B does often not outweigh the disadvantages.” A total of 47 of 85 (55%) agreed with the statement, “I often indicate my preference for treatment with chemotherapy doublets with or without a biological instead of FOLFOXIRI-B. Patients follow that recommendation.” Finally, 60 of 86 (70%) agreed with the statement, “To a young, fit, and motivated patient, I indicate my preference for first-line treatment with FOLFOXIRI-B.”3 TRIBE2,4 or meta-analysis9), 38 (38%) were familiar with the Dutch guideline recommendation on FOLFOXIRI-B (2017), and 51 (56%) were familiar with the positive recommendation of the NVMO committee that was published in December 2020.To assess scientific knowledge integration, we asked questions regarding awareness of FOLFOXIRI-B as a treatment option . A total3 TRIBE2,4 or meta-analysis9 publication (23 [30%]), conference or refresher course attendance (21 [27%]), contact with an opinion leader (16 [21%]), and the 2017 Dutch guideline recommendation and/or 2020 NVMO committee recommendation (15 [20%]).Medical oncologists reported the following events that led to incorporation of FOLFOXIRI-B into their clinical practice , D: TRIBP = .01) and the NVMO committee recommendation  but not the Dutch guideline recommendation. Medical oncologists who did not discussed FOLFOXIRI-B had significantly lower FAS scores . A low FAS score was correlated with agreement with the statement, “I often indicate my preference for treatment with chemotherapy doublets with or without a biological instead of FOLFOXIRI-B. Patients follow that recommendation” .Medical oncologists who discuss FOLFOXIRI-B compared with medical oncologists who did not discuss this treatment option were more often aware of the outcome of trials regarding FOLFOXIRI-B  of medical oncologists discuss FOLFOXIRI-B with eligible patients. Moreover, most Dutch medical oncologists do not exclude specific patients outside what is defined by the TRIBE trials’ eligibility criteria. However, reasons for the discrepancy between eligible patients and treated patients with FOLFOXIRI-B might partly lie in patient-physician communication. In our sample, 14% of interviewed medical oncologists never discuss FOLFOXIRI-B in clinical practice. Although most medical oncologists reported discussing FOLFOXIRI-B with eligible patients, 56% also report that they regularly communicate a preference for a chemotherapy doublet to these patients. In addition, oncologists reported that they were more inclined to discuss FOLFOXIRI-B with patients with potentially resectable metastases, younger patients, and patients with more aggressive disease, such as patients with V600BRAF-variant tumors is based on the post hoc subgroup analysis of the updated TRIBE trial, which was subsequently incorporated in clinical guidelines.3 However, a recent subgroup analysis9 of individual patient data, which included 1697 patients, concluded that no increased benefit is observed for patients with V600BRAF-variant tumors. This meta-analysis9 also assessed whether treatment benefit was larger for patients with liver-only metastasis. In addition, although R0 resections were more frequent (16.4% vs 11.8%), no significant interaction was found between treatment with FOLFOXIRI-B and the achievement of R0 resection in terms of overall survival.The assumption that FOLFOXIRI-B might be more efficacious for patients with When FOLFOXIRI-B is discussed as a treatment option, there are reasonable arguments as to why a patient might choose to forgo treatment with FOLFOXIRI-B. Medical oncologists reported increased toxic effects as the major reason for patients to decline FOLFOXIRI-B treatment, followed by the more frequent hospital visits from the 2-weekly schedule and the necessity of a central venous catheter. The discussion between oncologists and patients on the possible benefits and burden of a specific treatment can be challenging. When the condition of the patient allows several treatment options, personal beliefs, cultural aspects, and expectations of both physician and patient play a role. The goal should be that information provided to patients is based on objective data to allow patients to make a choice that has the best chance of meeting personal goals.4 However, it is unknown how strongly toxic effects are emphasized by medical oncologists in their discussion with patients. In this respect, 44% of medical oncologists report to have initially overestimated the toxic effects for patients compared with 6% of medical oncologists who reported an initial underestimation when prescribing FOLFOXIRI-B. This finding suggests that Dutch medical oncologists might overemphasize toxic effects in daily practice.Indeed, the adverse effects that are discussed by medical oncologists in our study are mostly in accordance with the TRIBE2 trial.26 Considering the current rate of the development of novel and effective cancer treatments, the adoption of these treatments in daily practice should be much improved. Our data indicate that medical oncologists gained knowledge on the benefit of FOLFOXIRI-B through a various number of sources, if at all. We found that medical oncologists who were aware of more literature on FOLFOXIRI-B seemed to be less inclined to communicate a preference for treatment with chemotherapy doublets to a patient, resulting in patients following that recommendation. This finding could imply that a higher level of awareness of medical oncologists leads to a more unbiased discussion of treatment options with patients.Another factor related to the relatively low adoption rate might be the transfer of scientific advancement to daily practice. Typically, knowledge is generated through clinical studies. Published results are subsequently included in the synthesized literature base and then broadly disseminated through conferences, workshops, and continuing medical education. Finally, when incorporated as standard of care, said insights may ultimately lead to improved health outcomes. Several estimates have suggested that this entire process takes up to 17 years to complete.27 In addition, real-life data should provide up-to-date information on the implementation of novel, effective treatments in daily practice and will provide more transparency on the quality of cancer care in individual hospitals. Flash mob studies, such as the current study, may be useful in achieving this goal.Continuing medical education, feedback on guideline adherence, modular updates of guidelines, and more frequent dissemination of novel, positive developments through authorized organizations should aid in this process. These types of interventions are known to correspond with guideline dissemination and implementation with significant positive changes in patient outcomes.17 Ultrafast data generation coupled with thoroughly preplanned data analysis enabled us to provide a true snapshot of current medical practice.This study is, to our knowledge, the sixth and largest flash mob study ever performed worldwide.28Although this is the largest flash mob study to date, the results are based on a subset of medical institutions that were willing to participate. The proportion of nonparticipating academic, teaching, and nonteaching hospitals was balanced, suggesting that this bias might be limited. Furthermore, the patients currently treated with FOLFOXIRI-B and the number of medical oncologists who did not discuss FOLFOXIRI-B was small, reducing statistical power for group comparison. In this study, we estimated eligibility for FOLFOXIRI-B based on the TRIBE inclusion and exclusion criteria. The true proportion of eligible patients might be slightly different considering that we did not take into account specific contraindications to irinotecan and the complex ECOG PS criteria for the 70- to 75-year age group. This study was performed during the COVID-19 pandemic. Although 96% of participating medical oncologists reported no change in prescription habits during the study period, it is possible that the pandemic has influenced the adoption rate of FOLFOXIRI-B. With respect to the interviews taken, interviewers were trained, and the questions had prespecified answer options. Nonetheless, interviewer bias could have occurred. In addition, these results should be interpreted within the geographic context of the study. The Dutch FOLFOXIRI-B guideline recommendations are similar to international recommendations. However, variation in prescription rates in other countries based on cultural differences is conceivable .This cross-sectional study found that the actual adoption rate of FOLFOXIRI-B as first-line treatment for patients with mCRC in the Netherlands remains relatively low despite the fact that most medical oncologists report discussing this treatment option with their patients. The results of this study provide insight into underlying reasons and offer guidance to accelerate the incorporation of knowledge into clinical practice for the benefit of patients.


Neurologic symptoms from leukemic infiltration of the central nervous system are an oncologic emergency, and expeditious treatment is required to preserve function. We report the case of a 44-year-old patient with relapsed acute myeloid leukemia (AML) who developed sub-acute cranial neuropathies refractory to treatment with intrathecal (IT) chemotherapy. The patient was therefore treated with an emergent course of whole-brain radiotherapy, resulting in immediate improvement and subsequent resolution of cranial neuropathies. This case illustrates that while central nervous system involvement by AML is rare, radiotherapy remains an effective modality to avoid long-term morbidity in patients failing to respond to systemic or IT chemotherapy. While central nervous system (CNS) involvement of acute myeloid leukemia (AML) is rare, the onset of neurologic symptoms can have catastrophic, irreversible consequences leading to significant morbidity and diminished quality of life -3. RapidA 44-year-old man was diagnosed with AML with a normal karyotype and FLT3-ITD and WT1 mutations. He was induced with 7+3 regimen (cytarabine and high-dose daunorubicin) and gemtuzumab ozogamicin. Post-remission therapy included two cycles of high-dose cytarabine with midostaurin followed by a myeloablative allogeneic stem cell transplant (conditioning regimen did not include RT). Approximately three months post-transplant, the patient had systemic relapse and was treated with hydroxycarbamide for cytoreduction followed by clofarabine, cytarabine, and sorafenib re-induction, and achieved molecular complete remission. Two months later, however, he presented to the emergency department with three weeks of diplopia and multiple CN palsies. At presentation, examination was notable for right eye ptosis and ophthalmoplegia signifying a right CN III, IV, and VI palsy Figure . A brainThe patient was started on IT MTX 12 mg, cytarabine 50 mg, and hydrocortisone 50 mg, with daily gilteritinib 120 mg per orem. He received two cycles of IT MTX, cytarabine, and hydrocortisone without symptomatic improvement, at which time the decision was made to proceed with RT. RT was initiated five days after his last IT MTX and three days after his last IT cytarabine to allow for washout. He received a dose of 15 Gy in five fractions using three-dimensional conformal WBRT via 6-MV opposed lateral photon beams Figures , 3B. ThiThe age-adjusted incidence of AML in the United States is approximately 4.3 per 100,000 persons, with a median age at diagnosis of 68 years . CNS invMTX is an antimetabolic drug that interferes with folate metabolism . LeukoenOur patient received two cycles of IT MTX prior to radiation oncology consultation, and the optimal timing for initiation of RT was uncertain. IT MTX is eliminated in a biphasic pattern with half-lives of 4.5 hours and 14 hours . In ordeIn contrast, the half-life of IT cytarabine is shorter than MTX (2-6 hours in the CSF) . One addIt was unclear in this patient if the ideal RT treatment field would encompass the base of skull (BOS) alone, whole brain, or entire craniospinal axis. Walker et al. published a retrospective analysis of patients with CNS leukemic involvement in which improved CNS progression-free survival was noted in patients receiving craniospinal axis/WBRT compared to BOS RT  . In lighHa et al. previously assessed the effectiveness of RT in reversing CN palsies due to leukemic involvement . ResolutParyani et al. reviewed their experience of children with ALL and non-Hodgkin’s lymphomas who developed CN palsies . Out of Due to small patient cohorts in these studies, a dose-response relationship has not been determined. Our patient was treated with a dose of 15 Gy in five fractions to minimize potential toxicities of concurrent MTX and to allow for consolidative craniospinal RT in the future. A dose of 3 Gy per fraction was used to overcome the malignancy’s prior resistance to chemotherapy. The patient had an excellent response to treatment at this dose with persistent resolution of his CN dysfunctions at his last follow-up six months after RT.In conclusion, RT can effectively relieve symptoms of leukemia with CNS involvement. The immediate symptomatic response observed in this patient after two fractions of RT indicates that the treatment effect was primarily a result of RT. In fact, in our patient with no neurologic improvement on IT chemotherapy, RT proved to be an extremely effective and efficient treatment in the setting of rapidly deteriorating neurologic function. This case highlights the role of WBRT in the treatment of symptomatic leukemic CNS involvement.


Data privacy mechanisms are essential for rapidly scaling medical training databases to capture the heterogeneity of patient data distributions toward robust and generalizable machine learning systems. In the current COVID-19 pandemic, a major focus of artificial intelligence (AI) is interpreting chest CT, which can be readily used in the assessment and management of the disease. This paper demonstrates the feasibility of a federated learning method for detecting COVID-19 related CT abnormalities with external validation on patients from a multinational study. We recruited 132 patients from seven multinational different centers, with three internal hospitals from Hong Kong for training and testing, and four external, independent datasets from Mainland China and Germany, for validating model generalizability. We also conducted case studies on longitudinal scans for automated estimation of lesion burden for hospitalized COVID-19 patients. We explore the federated learning algorithms to develop a privacy-preserving AI model for COVID-19 medical image diagnosis with good generalization capability on unseen multinational datasets. Federated learning could provide an effective mechanism during pandemics to rapidly develop clinically useful AI across institutions and countries overcoming the burden of central aggregation of large amounts of sensitive data. At the peak of the pandemic in early 2020, the clinical capacities to respond were overloaded in several countries, even with advanced healthcare systems such as present in Italy. Moreover, existing digital healthcare systems were rapidly overwhelmed and frontline clinicians challenged with an unprecedented amount of emergency workload for data analysis in a hitherto unseen disease entity2. Artificial intelligence (AI) has the potential to provide access to accurate, low-cost, and scalable solutions in combating COVID-19 through automated analysis of patient data.The COVID-19 outbreak, caused by the novel coronavirus SARS-CoV-2, has presented a public health crisis worldwide. According to data compiled by the Center for Systems Science and Engineering at Johns Hopkins University4. To this end, both the training and testing aspects should be considered with equal importance for data and model sharing. For the training phase, aggregating multiple data sources helps improve model robustness and generalizability, because scaling amounts of data with various imaging protocols and diverse patient populations could help reduce model bias5. Enabling privacy protected data sharing across clinical centers is advocated as an essential pathway to promote collaborations internationally yet underexplored so far7. For the testing phase, validation of AI models on multiple, unseen, independent external cohorts has to be a crucial criteria for assessing scalable usability toward wide model sharing9. Recent study10 has revealed the potential of federated learning models for generalizability outside federation on brain tumor application. However, there is still little evidence been reported to date on the generalization performance of decentrally developed AI models for widely collected COVID-19 cohorts, especially in the setting of multinational evaluation of heterogeneous patient cohorts.Multicenter collaborative research efforts have been expected to coordinate data sources for maximizing the potential of data-driven AI technologies13. The main findings of COVID-19 infection on CT scans are bilateral and peripheral ground-glass and consolidative pulmonary opacities14. These are currently clinically interpreted in a qualitative manner, but having a method that can quantitatively measure the disease burden and changes over time will be valuable for patient surveillance. Existing AI models to date are mostly designed for lung lesion segmentation using convolutional neural networks (CNNs)16, requiring dense pixel-wise labels through time-consuming, labor-intensive manual annotation from experts who are scarce during the crisis. We instead consider detecting lesion bounding boxes, for which annotations are easier and quicker to obtain while maintaining clinical utility of quantifying the burden of infection.A major focus of AI fighting against COVID-19 is interpreting radiological images, mainly chest CT which has been widely applied for detecting lung changes to inform patient management, assessment of severity, and monitoring of the diseaseWe aimed to demonstrate the feasibility of training a deep CNN-based AI model for automated detection of lesions from COVID-19 CT images, using a privacy-preserving method which does not require exchange of data between centers nor data to be stored centrally. Model validations were conducted using local and external datasets  with comparison to expert radiologists’ interpretations. In addition, case studies with longitudinal scans were also performed for automated estimation of the lesion progression to support monitoring hospitalized patients.This study explores the potential of federated learning methods to develop a privacy-preserving AI system for the real-world problem of automated COVID-19 image interpretation. A CNN-based model has been successfully trained on decentralized multicenter data to detect lesions from COVID-19 CT images, with wide generalizability to external patients . These attributes showed the potential of federated learning to build generalizable, low-cost, and scalable AI tools for image-based disease diagnosis and management, both for research and clinical care.In this multicenter study, the internal datasets were collected from three local hospitals in Hong Kong, i.e., Prince Wales Hospital , Princess Margaret Hospital , and Tuen Mun Hospital . A total of 75 patients  with confirmed COVID-19 infection (positive RT-PCR tests) were enrolled in this study. These retrospective CT images were collected during the time period from 24 Jan 2020 to 16 Apr 2020. The ethical approvals were obtained in accordance with all relevant laws and regulations for each recruiting hospital (see details in supplementary https://coronacases.org/) of 10 patients originally collected by Wenzhou Medical University, China, with a third-party lesion annotation released by Ma et al.17; (2) External-Set-2: a private dataset of 35 patients (collected during 02 Feb 2020 to 30 Mar 2020) from BioMedIA research group collected at the Klinikum rechts der Isar, Technical University of Munich, Germany, with lesions independently labeled locally; (3) External-Set-3: a private dataset of 10 patients (collected during 25 Jan 2020 to 04 Mar 2020) from Peking University Shenzhen Hospital, China, with lesions independently labeled locally; (4) External-Set-4: a private dataset with longitudinal studies of two patients (collected during 23 Jan 2020 to 19 Mar 2020) from Zhijiang People’s Hospital in Hubei, China, with hospitalized records acquired. All these included patients had confirmed COVID-19 infection with positive RT-PCR tests. Each participating external private center obtained individual ethical approval in accordance with respective relevant laws and regulations. The inference codes and AI models were sent to each center for independent held-out testing as external validations.To evaluate the robustness and generalizability of our AI model beyond local centers to wider data distributions of imaging protocols and patient populations, we used four datasets outside Hong Kong for external validation: (1) External-Set-1: a publicly released COVID-19 CT dataset . The network training was conducted on the training subsets. Details of annotated CT lesion datasets and the random subset splits are listed in Table An overview of our study scheme is illustrated in Fig. Individual-model-1 , Individual-model-2 , and Individual-model-3 . In addition, we established the comparison method of model ensemble of these three individual models, i.e., running each individual model on the testing cohorts and merging their prediction results. We have also added a baseline of training a single joint model with all data centralized. For all these five comparison settings, both internal and external validations were conducted following the same evaluation scheme as the federated learning model.To analyze the benefit of scaling the amount of training data through multicenter learning, we also established baseline settings to train the models for comparisons. For each internal site, we trained an individual model with standard single-site training, obtaining three independent networks, denoted with Overall, we conducted experiments representing six settings, i.e., three single-center models, their ensemble, a joint model, and a federated learning model. Besides the joint model, all other five methods were free from data exchange or centralization in the multicenter study setting, thus protecting the privacy of the patient health data.This study aimed to analyze the effectiveness of federated deep learning on the task of chest CT abnormality detection for COVID-19 with evaluating the performance of AI models trained from multicenter data and validated on both the internal and external testing datasets. The evaluation was conducted based on whole CT volumes, i.e., all axial slices were processed sequentially without any prior knowledge of whether lesions are present in a slice or not. The statistical analysis was conducted using Python 3.7 (see details in supplementary 20 for the lesion detection results (detailed computation process is referred to the “Methods” section). We also computed AUC  for each ROC curve with 95% CI computed with the DeLong approach21. To further evaluate the accuracy of bounding-box areas, we used the metric of mAP , which averages the detection precisions under different IoU  rates and is a widely-employed metric for object detectors in image processing22. The 95% CI of mAP was computed using the Clopper-Pearson method23. In addition, we also analyzed the detection sensitivity and precision at a certain false-positive rate. We chose a value of 0.1 false positive on average per slice, meaning that one false-positive bounding-box prediction is occurred in every ten slices, which is reasonable to be used in clinical practice. We computed the patient-wise variance for the metrics of sensitivity and precision, and their 95% CI of were computed using the z-table24.To evaluate the performance, we used the receiver operating characteristic (ROC) curvesp-values for detection results between the comparison methods , and the federated learning model were computed using the two-sided Student’s t-test. The statistical significance was defined as a p-value <0.05.The p-values) on the testing sets. The ROC curves with AUC (with 95% CI) are shown in Fig. We report the lesion detection results on the internal testing set  and three external sets  through comparison with radiologist interpretations. Table We first present the results of internal validation. Among three individual models, the Individual-model-2 trained with the largest single-site database (with 9398 lesions) obtained a higher AUC of 95.40% (95% CI 93.80–96.98) than the other two individual models, with the mAP of 89.98% (82.27–97.69), sensitivity of 92.03% (84.23–99.83), and precision of 82.1% (76.16–88.04). The ensemble of three individual models showed a relatively lower AUC 94.63% (93.19–96.06), compared with Individual-model-2. The joint model trained with all internal datasets performs slightly lower than model ensemble, with the AUC of 92.97% (91.71–94.21). Using multicenter data for model training, the federated learning model obtained comparable AUC of 95.27% (93.98–96.51) as Individual-model-2, with slightly higher mAP of 91.28% (84.91–97.65), sensitivity of 93.54% (87.33–99.75), and precision of 84.21% (78.61–89.81). The federated learning model also outperformed the model ensemble method consistently across all the metrics.p-value <0.05 across all external sets. Figure Next, for generalization performance on unseen external cohorts, on External-Set-1, the best performance was achieved by the federated learning model with AUC of 95.66% (95% CI 94.17–97.14) and mAP of 87.83% (84.18–91.48), among all six methods. For External-Set-2 from Germany with expectable differences in population demographics compared to the training cohorts, the model generalization generally decreased for all models compared with their performance on External-Set-1. The highest performance on this set was also achieved by the federated learning model, with AUC of 88.15% (86.38–89.91), mAP of 71.48% (68.69–74.27), sensitivity of 73.31% (70.44–76.18), and precision of 91.93% (89.48–94.38). The second best results were obtained by the ensemble model through aggregating predictions from all three individual models. External-Set-3 showed similar results, with the federated learning model attaining the highest AUC of 91.99% (89.97–93.88) among all models. Statistical analysis between our federated learning model with other five models revealed a 18 and training from scratch, with results shown in Fig. We further conducted ablation study to analyze the effect of transfer learning in our method. Specifically, we trained the federated model in two different learning strategies, i.e., training with model initialized from our previous DeepLesion modelThe potential clinical applicability of our AI model was demonstrated by estimating the lesion burden for monitoring hospitalized patients. This test was done on External-Set-4 with two case studies analyzing the longitudinal scans and clinical symptoms reports. Our detection model can provide lesion segmentation masks without additional supervision by benefiting from transfer learning (details in “Methods” section), and the predicted dense score-map for each scan of the two cases are shown in Fig. In this study, we demonstrate the feasibility of federated learning to combine COVID-19 data across participating centers in a patient privacy-protecting manner. This decentralized training strategy may be a key enabler of scalable AI-based technology during a pandemic when there is no time to set up complicated data-sharing agreements across institutions or even countries. The use of federated learning has been recently shown in other fields such as edge computing , however, the medical imaging scenario is more complicated and entails unique challenges  which exerts unexplored influences on currently used federated learning techniques. To our knowledge, this is a prior work to demonstrate the feasibility and effectiveness of federated learning for COVID-19 image analysis, where collaborative effort is especially valuable at the time of global crisis. Our experimental observations have shown that federated learning improved generalization performance over all single-site models and their ensemble, reflecting successful decentralized optimization with diverse distributions of training data.We show in a proof-of-concept that a CNN-based federated deep learning model can be used for accurately detecting chest CT abnormalities in COVID-19 patients. Importantly, the AI model trained on Hong Kong cohorts showed high performance not only on internal testing cases, but also on external, unseen, independent datasets collected from hospitals in Asia and Europe. The scanner brands, imaging protocols, and patient populations varied across these multi-national centers, and the severity of COVID-19 pneumonia differed across the patients involved in the study  achieved superior performance over the other two single-site models (trained on 958 and 660 slices) on all three external testing sites. This revealed that larger training databases (even from the same site) could improve model performance on unseen datasets. Moreover, upon merging all three internal sites for training, despite two sites contributing fewer cases, the test accuracy could be further improved. To some extent, this could reflect that, in addition to increased data scale, richer data diversity associated with imaging scanners and protocols may be equally important for reducing model bias thus improving generalizability. In this sense, collaboration across multiple clinical centers is an essential component to pave the way for developing AI systems for wide-spread deployment, especially when faced with COVID-19 pandemic where multinational efforts are crucial.This study hypothesized that multicenter training could enhance the generalizability of AI models, which has previously been demonstrated in other medical imaging scenariosAlthough this study recruited patients from seven clinical centers from different regions, the number of patients from each participating center was relatively small. The centers which were heavily involved in managing the pandemic received more cases than other hospitals, resulting in an imbalance between sites. To a certain extent, this reflected the practical situation that COVID-19 patients were generally too scarce for most single sites to train individual in-house AI models. Therefore, multicenter studies with collaborative efforts in data combination are important and valuable to handle the long-tail distribution of COVID-19 patients. Our future work will further include more patients and centers in federated learning. Nevertheless, it is worth noting that, in the current proof-of-concept study, the relatively small numbers of patients did not impede the development of a deep learning model, because the numbers of individual lesions identified from all CT volumes were fairly large .The model generalized less well on the German cohort, compared with other external cohorts. A reason may be the patient populations coming from different demographics. In addition, the lesion annotation procedures differed between different clinical sites causing what is known in machine learning as concept shift where the manual annotations in this cohort were not directly compatible with training data. For example, the training data annotated ground-glass opacification and consolidation, while the German cohort contained a few pleural effusion lesions, which are atypically seen in COVID-19, with relatively low contrast against normal lung tissue. We excluded 15 cases (with very mild lesions which could hardly be seen in the lung window), 5 cases (with severe diffuse lesions which were not suitable to be processed as a detection task predicting lesion bounding boxes), and 1 case (with no CT finding). Supplementary Fig. In conclusion, the CNN-based AI model trained using a privacy-protecting federated learning approach is effective in detecting CT abnormalities in COVID-19 patients. The wide generalizability to regional and international external cohorts, benefited from including diverse datasets, shows the promise of AI providing low-cost and scalable tools for lesion burden estimation to support clinical disease management.In this multicenter study, the internal datasets were collected from three local hospitals in Hong Kong, i.e., Prince Wales Hospital (PWH), Princess Margaret Hospital (PMH), and Tuen Mun Hospital (TMH). The ethical approvals were obtained in accordance with all relevant laws and regulations for each recruiting hospital, i.e., PWH approved by the Joint Chinese University of Hong Kong - New Territories East Cluster Clinical Research Ethics Committee, PMH approved by the Kowloon West Cluster Research Ethics Committee, TMH approved by the New Territory West Cluster Research Ethics Committee.28. The updated global model was generated by using a weighted average of the parameters from all the local models, weighted proportionally to the sample size on each node, which is provided by local node to the global server. Next, the central server distributed the updated global model to the local nodes, then each node continued to perform local optimizations based on the updated global model with its local data. After an epoch, each node sent back the updated parameters to the central server for the next federated learning iteration. This process was repeated until the global model converged.To protect data privacy during model training, we studied the feasibility of federated learning on three local hospitals, with each individual center representing a node. Data sharing across the sites was not required, while the model benefitted from the generalizability enabled by multicenter learning with the inclusion of diverse data sources. More specifically, such a decentralized scheme trained individual models on local nodes and exchanged the network parameters to update a global model stored at the central server at a certain frequency . In each iteration of the federated update, the central server first aggregated all the local models and used them to update the global model parameters using the Federated Averaging (FedAvg) algorithmK (K = 3 in our setting) hospitals for collaborative training, with nk as the number of data points in each hospital k. At the beginning of one federated training round t, the central server first sends the global model with parameters wt to all local hospitals. Each hospital k then optimizes the received model locally with its own dataset for E epochs (E = 1 in our implementation), and then sends the model update ∇wtk back to server. Once receiving the updates from all local hospitals, the server averaged these updates with weights in proportion to the sizes of local dataset to refurbish the global models as with a learning rate. Such a process repeats until the global model converges. Note that for each hospital participating federated learning, the sample size of its local dataset is given to the central server for aggregating the local parameters with weighted average to update the global model.Formally, assume that there are 19 CT lesion dataset was effective to handle the data insufficiency issue for developing the deep learning detector with COVID-19 data in this study. The dataset is currently the most comprehensive open-source data for CT lesions, that includes 32,120 axial CT slices  from 10,594 studies of 4427 unique patients. For network architecture of our model, we used our previously developed improved version of RetinaNet18, a 2D CNN with medical-domain customized design for object detection based on the original RetinaNet backbone29. The model input consisted of three adjacent CT axial slices. Our model was initially pretrained on the DeepLesion dataset (relying on our previous work18), then fine-tuned with internal COVID-19 training images. In our previous work, we utilized RECIST diameters provided in DeepLesion dataset as weak labels to generate pseudo-masks for learning an auxiliary branch for lesion segmentation . This segmentation branch with associated parameters was learned in the pre-training step, while kept frozen in the fine-tuning process without further update, as lesion segmentation annotation for our dataset was unavailable. We found that this auxiliary segmentation branch could still output lesion segmentation masks with acceptable quality at testing  for each volume before rescaling their intensities to . From experimental observations, we found that instance-level normalization to zero mean, unit variance helped improve generalizability, i.e., normalizing every single volume with its individual statistics rather than using the dataset global statistics. After normalization, three adjacent slices were combined as the input for CNN-based models.In each round of federated training, every local client optimized their model for one epoch using the individual dataset. All local clients used Adam optimizer with hyper-parameter as learning rate of 1e−4, beta1 of 0.9, beta2 of 0.999, and epsilon of 1e−7. We applied data augmentation schemes including random horizontal and vertical flipping with 50% probability; random clockwise rotation with amplitude ranging from −0.1 to 0.1; random horizontal/vertical translation with −0.1 to 0.1 of input image length/width; and random shearing and scaling with variation of −0.1 to 0.1. The data augmentations were performed with Numpy and TensorFlow libraries through processing the data arrays. A small amount of training data was held out at each node to determine the model convergence. If the global model performance on local validation data was not increased for five successive federated rounds, we considered that the training was converged and the federated learning was stopped. The deep convolutional networks were trained with one NVIDIA TitanXp GPU.30, a well-established method in image processing to retain a single entity out of overlapping entities. In our case, we adopted it to extract the bounding boxes with the highest predicted probability from a series of overlapped bounding boxes. Specifically, given a number of predicted bounding boxes from an image, we remove bounding boxes with probability lower than a threshold. With the remaining bounding boxes, we repeatedly keep the bounding box with the highest probability and discard remaining boxes with IoU higher than 0.5 with this selected one. We also applied an existing open-source lung segmentation AI model31, to remove the false-positive detections which fell outside the lung region.For the post-processing, we used non-maximum suppression32 (following the de facto setting in literature), it was identified as a true positive result. On the other hand, for each prediction, if its IoU with all of the labeled bounding boxes being smaller than 0.5, it was identified as a false-positive result. With these true positives and false positives, we computed the sensitivity and specificity pairs to obtain the ROC curves.Given an input CT volume, each axial slice was sequentially tested with the AI models. The outputs of these detection models were a set of bounding-box predictions , each of which carried a score to indicate the probability of current prediction being a lesion. We used the prediction score on which a threshold varied to calculate the ROC curves. For each proposal, if its IoU  with any of the manually labeled bounding boxes being higher than 0.5Further information on research design is available in the Supplementary InformationReporting Summary


Human T-Cell Lymphotropic Viruses (HTLV) type 1 and type 2 account for an estimated 5 to 10 million infections worldwide and are transmitted through breast feeding, sexual contacts and contaminated cellular blood components. HTLV-associated syndromes are considered as neglected diseases for which there are no vaccines or therapies available, making it particularly important to ensure the best possible diagnosis to enable proper counselling of infected persons and avoid secondary transmission. Although high quality antibody screening assays are available, currently available confirmatory tests are costly and have variable performance, with high rates of indeterminate and non-typable results reported in many regions of the world. The objective of this project was to develop and validate a new high-performance multiplex immunoassay for confirmation and discrimination of HTLV-1 and HTLV-2 strains.The multiplex platform was used first as a tool to identify suitable antigens and in a second step for assay development. With data generated on over 400 HTLV-positive blood donors sourced from USA and French blood banks, we developed and validated a high-precision interpretation algorithm. The Multi-HTLV assay demonstrated very high performance for confirmation and strain discrimination with 100% sensitivity, 98.1% specificity and 100% of typing accuracy in validation samples. The assay can be interpreted either visually or automatically with a colorimetric image reader and custom algorithm, providing highly reliable results.The newly developed Multi-HTLV is very competitive with currently used confirmatory assays and reduces considerably the number of indeterminate results. The multiparametric nature of the assay opens new avenues to study specific serological signatures of each patient, follow the evolution of infection, and explore utility for HTLV disease prognosis. Improving HTLV diagnostic testing will be critical to reduce transmission and to improve monitoring of seropositive patients. HTLV viruses are responsible for more than 10 million cases of infection worldwide. The infection is considered as a neglected disease due to lack of vaccines and treatments. Accurate diagnosis is crucial for counselling infected persons and prevention of secondary transmissions. In spite of the development of excellent serological screening assays, many cases of indeterminate and untyped results are still regularly reported and their infection status remain uncertain. To address the need of more precise diagnosis, we have developed a new cutting-edge in-vitro diagnostic confirmation test, named Multi-HTLV, which has been validated on a large panel of HTLV samples. The test is a multiplex immunoassay allowing powerful detection of antibodies against HTLV through combination of a set of selective and validated virus-specific antigens in a blood sample. The Multi-HTLV assay increases the reliability of HTLV diagnostics and strain typing thanks to a high precision mathematical algorithm. Human T-cell lymphotropic viruses (HTLV) were the first human retroviruses to be discovered. They cause blood infections through three transmission routes: mother-to-child transmission (mainly associated with prolonged breast-feeding), sexual transmission and contaminated blood products (blood recipients and drug users) . HTLVs aGag and Env proteins derived from viral lysates or recombinant proteins. Many studies have revealed shortcomings of the WB technique. Indeed, in all regions of the world and specially in high risk populations, a high number of indeterminate and non-typable results have been found [HTLV-1 is divided into four subtypes and circulates mainly in Japan, the Caribbean, West and Central Africa, South America and Australo-Melanesia . HTLV-1-en found ,21. The en found . Some coen found ,23. Hoween found . Therefoen found ,26. The en found ,27,28. Ten found ,26,29. Hen found . Furtheren found ,30. Acceen found .The aim of this study was to develop a new high performing and cost-effective multiplex test to confirm HTLV infection and discriminate the two strains, HTLV-1 and HTLV-2. We used a microplate-based multiplex platform to identify the most relevant antigens (recombinant proteins and synthetic peptides) and developed a readable and easy-to-use assay. The developed assay with updated high-performance antigens offers improved confirmation of HTLV infection and can be performed in routine laboratories with standard lab equipment.The methods section was described in accordance with the Standards for Reporting of Diagnostic Accuracy Studies (STARD) protocol .Written informed consent was obtained from all participants in the United States-based cohort HTLV Outcomes Study (HOST), including use of frozen biospecimens for future research. The study was approved by the University of California San Francisco Committee on Human Research (IRB#s H5901-14117-13D & 11–05747) and other institutional review boards. Written informed consent was obtained by EFS from blood donors after a detailed explanation of how their donation would be used for research purposes.For this study, different sample collections were used .This prospective study was conducted with samples from the United States-based cohort HTLV Outcomes Study (HOST) which includes samples from subjects recruited from five major US blood donation centers ,32. BaseFor the current study, 694 frozen serum specimens were obtained from the cohort repository. We excluded from the study three samples because of labeling errors and two samples because of mismatches, which could lead to misinterpretation. A total of 689 samples from 397 participants were therefore eligible for the analysis. Among them, 60 HTLV-positive patients were followed over time in a longitudinal study and tested to analyze the evolution of their serological profile. However, in order to have independent data for each HOST participant, we included in the study only the first sample tested of each participant, more specifically, 199 HTLV-positive samples from 199 participants (100 HTLV-1 from 100 patients and 99 HTLV-2 from 99 patients) and 198 negative samples from 198 participants .We obtained 140 positive samples (133 HTLV-1 and 7 HTLV-2) and 28 seronegative samples from different French blood donations (transfer agreement 20120224). Samples were collected between 2008 and 2013, and screening was performed by ELISA, using either Prism HTLV-I/HTLV-II or Architect rHTLV-I/II assay , and then HTLV positive samples were confirmed and typed by INNO-LIA HTLV-I/II.We obtained 65 HTLV-2 samples from a commercial source. All patients came from USA and had available data on ethnicity and ages. Samples were collected between 2013 and 2016, screened with an HTLV-1/2 ELISA (brand not specified by Trans-Hit Bio) and subsequently confirmed as HTLV-2 positive by WB (brand not specified by Trans-Hit Bio).The multiplex technology allows multiple parameters to be combined in a single well of a 96-well microplate. The printing process is based on non-contact piezo electric impulsion of a defined volume of an antigenic solution. We used the SciFLEXARRAYER printing system  to print HTLV antigens, at the bottom of each well at precise X-Y coordinates, under controlled humidity and temperature parameters. This technology was used first to make a selection of the most relevant antigens and second for the design and production of the assay, which has already been used in this manner for the development of other confirmatory test for Chagas disease . Each anenv GP21, env GP46 and gag P19; and (ii) three discriminatory antigen spots derived from type-specific immunodominant epitopes of HTLV-1 env GP46, HTLV-1 gag P19 and HTLV-2 env GP46. In contrast to other confirmatory immunoassays, we excluded the detection of anti-P24 antibodies in the Multi-HTLV test because it has been shown in WB assays that indeterminate samples frequently show an isolated P24 band attributable to false reactivity [www.uniprot.org (The Multi-HTLV assay is composed of (i) three confirmatory antigen spots for binding specific HTLV-1 and HTLV-2 antibodies and derived from common-type and type-specific immunodominant epitopes of activity . Relevanprot.org .Each antigen composition was individually optimized to obtain the best performance in terms of sensitivity and specificity, and by calculating the areas under the Receiver Operating Characteristic curve (AUC), using SPSS Statistics software . This optimization step was performed by using a panel of HTLV-infected and seronegative samples from EFS. Further, three positive control (PC) spots were printed in each well to define a precise spatial orientation pattern and to validate the correct sequential addition of all reagents . To facilitate the visual interpretation at the end of the test, two additional control “low”- and “medium”-spots–composed of anti-human-IgG antibodies–were printed and indicate the "minimal" or "intermediate" reactivity for each patient .Following the spotting process, the plates were incubated overnight to allow coating of the antigens at optimal conditions of humidity and temperature. Then, they were dried at 37°C for 2 hrs and washed with PBS-T . A blocking solution containing Bovine Serum Albumin (BSA) was added into the microplate for 2 hrs at room temperature (RT) to avoid nonspecific binding during the testing process. After the saturation step, the microplates were then washed with PBS-T. For long term storage, the blocked microplates were dried at 37°C during 2 hrs, then stored at 4°C in sealed plastic bags with desiccants.The Multi-HTLV test is performed very similarly to a standard ELISA assay enabling simultaneous detection of circulating antibodies to a set of selective and validated antigens. Hundred μL of diluted patient samples in PBS-T  and BSA (1%) (1:50) was added to each well, then microplate was incubated for 1 hr at RT. After three washing steps with PSB-T, 100 μL of anti-human-IgG secondary antibody conjugated to Horseradish Peroxidase, diluted in a defined volume of PBS-T  and BSA (1%) buffer (1:2000), was added and the microplate incubated for 1 hr at RT. To remove unbound secondary antibodies the microplate was washed three times with PSB-T. For the detection of conjugated enzyme activity, a precipitating substrate solution  was added and the microplate incubated in the dark for 20 min at RT. Upon contact of the substrate with the enzyme, it is converted to an insoluble permanent dark-blue precipitate. The reaction was stopped by removing the substrate solution, and drying the microplates at 37°C for 10 min. Each plate was read and analyzed both visually and using a specific microarray reader sciREADER CL2  which acquires high-resolution digital images and an integrated software calculates the pixel intensity for each spot. We calculated the mean value of the duplicated spots to get the net intensity of each antigen. The data were incorporated into a Microsoft Excel 2019 database and analyzed using a specific Multi-HTLV interpretation algorithm to determine the final result .Data from three cohorts  were pooled. Gold standard HTLV type assignments were available for the HOST cohort samples by either PCR or type specific ELISA and INNO-LIA results. For the two other studies, sample type assignment was only available based on INNO-LIA test results. The pooled dataset was split into a development and validation set. All data from EFS and Trans-Hit Bio were allocated to the development set and the HOST samples were allocated randomly (p = 0.5) to either the development or the validation dataset. Only HOST samples were included in the validation set because of PCR-based HTLV classifications. PCR has high confirmation power and strain discrimination, which in turn leads to more accurate validation analysis. The validation set consisted of 96 positive samples (53 HTLV-1 and 43 HTLV-2) and 103 negative samples.The addition of EFS and Trans-Hit Bio data to HOST data for the development set allowed a larger sample size, which is beneficial for statistical methods and machine learning algorithms. It comprised 308 positive samples (180 HTLV-1 and 128 HTLV-2 samples) and 123 negative samples. For both HTLV detection and typing, a logistic regression (LR) model was trained. Outliers were expected in the development set due to possible imperfect INNO-LIA labeling of EFS and Trans-Hit Bio samples. Sample deviances were used to detect outliers, which were omitted if the human interpretation of the Multi-HTLV assay did not match the INNO-LIA label. Due to the gold standard labeling of HOST samples, only samples with deviances higher than the maximum deviance of HOST samples were checked. Deviances below this threshold were expected to arise from regular variability but not necessarily from mislabeling.arm package (version 1.11–2) [Logistic regression was used as a baseline positive/negative classification method using GP21-I/II (Confirmation), P19-I/II (Confirmation), GP46-I/II (Confirmation), P19-I, GP46-I and GP46-II as independent variables (features). To prevent (quasi-) complete separation problems, Bayesian analysis with non-informative prior assumptions was applied, by use of the R  1.11–2) . A backw 1.11–2) . A RandoThen, we developed an algorithm based first on counting the number of positive antigens per sample and then applying logistic regression formulas. The algorithm is as follows: if the number of confirmatory antigens is zero, then the sample is directly considered to be negative; if the number of antigens is equal or higher than one, then a first logistic regression formula for confirmation is applied. It permits classification of samples with one reactive antigen among negative (LR probability (p)<0,5) or indeterminate categories , and samples with number of antigens higher than one in indeterminate  or positive  categories. Thereafter, only positive samples are typed as HTLV-1 and/or HTLV-2 by applying a second logistic regression formula . The IntAfter application of the interpretation algorithm, sensitivity  and specificity  with corresponding confidential interval (CI) of 95%, were used for diagnostic accuracy and performance evaluations of the Multi-HTLV assay. Indeterminate results obtained were included for the calculation of sensitivity and specificity. In this case, a PCR-negative sample that was indeterminate with Multi-HTLV, was treated as a false positive. Conversely, a PCR-positive sample that was indeterminate, it was counted as a false negative.First, we evaluated the Multi-HTLV confirmatory assay by analyzing the immunoreactivity of each antigenic marker for all tested patients. By visual analysis of images recorded with a specific image reader, we could clearly separate the majority of HTLV negative and positive results. Next, a mathematical algorithm was developed to facilitate a faster and accurate analysis of the confirmation and typing results. In addition, the algorithm enables precise determination of atypical profiles, for which visual interpretation is more complicated . The traBy analyzing the classification of the samples compared to the PCR gold standard, the specificity did not attain optimal performance as two negative samples in each set were classified in the indeterminate category .To further investigate the 4 indeterminate samples obtained, we looked at their serological profiles . In bothTyping performance showed a strain specificity of 99,03%  for the development set and 100%  for the validation set. In comparison, the INNO-LIA assay has a strain specificity of 94.17%  for the samples in the development set and 95.83%  for the samples in the validation set. Among these untyped or falsely typed samples with INNO-LIA, there was a higher proportion of HTLV-2 PCR samples (n = 7/10) .In our study, 3 HTLV-1/HCV and 12 HTLV-2/HCV co-infected samples from the HOST cohort were also included. Importantly, all these samples were correctly confirmed and typed with Multi-HTLV . We noteThen, we studied the creation of a predictive model for proviral load depending on the serological profile of HTLV positive patients. In 246 HTLV-1 and 245 HTLV-2 samples from the entire HOST cohort, linear and rank correlations between intensities of each antigen of the Multi-HTLV and proviral load were highly significant. Based on these observations, two multivariable models (one for HTLV-1 and one for HTLV-2) were created using the logarithm of the proviral load (log (PVL)) as dependent variable and the antigen intensities as independent variables. Models showed linear Pearson’s correlation coefficients of 0.56 and 0.54 respectively, between observed and predicted proviral load . AlthougFinally, we examined longitudinal patient samples from the HOST cohort using the Multi-HTLV assay. We observed in 35 of 60 patients a certain dynamic in antibody levels between the initial donation and 10–11 years of follow-up. These changes were sometimes visible on the INNO-LIA assay, but measuring the intensities of individual spots with the Multiplex reader provides more accurate values and facilitates visualization of the variations in serological signatures ; A. HoweHere we have developed an innovative and high-performance Multi-HTLV assay solving the major limitations associated with the currently used WB confirmatory test. The multiplex assay allows monitoring of different serological profiles for each patient first by visual analysis, and second through a combination of highly specific synthetic antigens as well as a custom-designed high-precision algorithm that ensures the most reliable confirmatory and typing results. The multiplex technology can be performed in any routine laboratory with simple equipment and a small sample volume.This study targets an important medical need which is to develop a cutting-edge diagnostic tool to improve counselling and reduce viral transmission of HTLV. Over the past 30 years and since the development of the INNO-LIA assay, no significant progress has been made to further improve HTLV diagnostics, thus the current strategy is to simply select the best screening method ,39. Our Despite the imperfect classification of PCR-negative samples with Multi-HTLV, it is preferable to categorize them as indeterminate rather than giving an incorrect result, as they may be subject to further investigation to better understand their status. Furthermore, it would be important to test more co-infected samples and in particular HIV/HTLV samples, which currently cause many difficulties .The Multi-HTLV assay is thus not only a useful tool to accurately diagnose HTLV infection but also a good tool to understand indeterminate samples, which represent a major challenge in HTLV diagnostics, particularly in endemic areas . Indeed,Plasmodium falciparum [gag as well as an Env antigen. The multiparametric aspect of the Multi-HTLV assay and its associated confirmation algorithm takes into account the acquisition of at least two reactivities to highly sensitive and specific Env and Gag antigens and therefore enhances the analysis and provides a more reliable diagnostic of HTLV infection.In our study, we also obtained indeterminate results for PCR-negative blood donors, due to an isolated GP21-I/II. Unfortunately, these individuals were not tested for proviral load and follow-up sampling was not performed, making it difficult to understand whether they represent seroconverting cases. For the other indeterminate results, we obtained intensities on antigens other than GP21-I/II that are very close to the minimal intensity value. It is possible that these intensities would vary when the test is repeated and that the final result may change. In a future study, it would be important to track individuals with similar profiles to better understand the role of seroconversion or cross-reactivity. It is also interesting to note, that several studies have identified two serological profiles of WB indeterminate subjects from African and Caribbean populations, called "N" and "HGIP". However, INNO-LIA and PCR tests revealed that these reactivities were probably not caused by HTLV-1 infection but were likely due to cross-reactivity with antigens of other microbial agents, in particular the parasite lciparum ,43. ThesOur study also shows that typing of HTLV strains using the Multi-HTLV assay shows promise. Only one sample in the development set was not correctly classified. The proposed method shows higher performance with regard to HTLV type assignment as compared to the INNO-LIA. This would be necessary due to the appearance of new strains in certain endemic areas and in high-risk populations. It is also interesting to note that in our study, INNO-LIA non- or falsely-typed results were related to the HTLV-2 strain. These observations are reminiscent of reported problems with WB, which lacks typing sensitivity particularly for HTLV-2 strains . For INNAccuracy in distinguishing HTLV strains remains important for prognosis of associated diseases and in cases of some co-infections . For exaenv and proviral DNA levels in HTLV-1 has been observed in patients who develop HAM/TSP compared to asymptomatic patients [Another medical need related to HTLV infection concerns the understanding of associated diseases because currently no vaccine is available and treatment options for HAM/TSP and ATLL are inadequate ,46. The patients . Howeverpatients . With thIn summary, our data demonstrate that the Multi-HTLV assay is a novel and reliable assay to detect and type HTLV-infected patients, as well as a research and discovery tool to perform serological profile studies, as recently performed for Chagas disease ,50. SincS1 Appendix(DOCX)Click here for additional data file.S2 Appendix(DOCX)Click here for additional data file.S3 AppendixIn a very few cases, sample can be declared weakly positive by visual analysis, but correctly classified as indetermined with the algorithm. In other cases, samples can be declared both HTLV-1 and HTLV-2 or untyped, but the specific serological profile of the respective type was recognised by the algorithm which assigns the correct result.(PDF)Click here for additional data file.S4 Appendix(A) Antigen reactivities obtained with Multi-HTLV and INNO-LIA assays for one HTLV-1 & two HTLV-2 patients. (B) Plots showing comparative evolution of proviral load and the sum of 6 antigens intensities measured by Multi-HTLV for one HTLV-1 and two HTLV-2 patients over 10–11 years.(PDF)Click here for additional data file.