Rare diseases are estimated to affect 150–350 million people worldwide. With advances in next generation sequencing, the number of known disease-causing genes has increased significantly, opening the door for therapy development. Rare disease research has therefore pivoted from gene discovery to the exploration of potential therapies. With impending clinical trials on the horizon, researchers are in urgent need of natural history studies to help them identify surrogate markers, validate outcome measures, define historical control patients, and design therapeutic trials.We customized a browser-accessible multi-modal  database to increase cohort sizes, identify surrogate markers, and foster international collaborations. Ninety data entry forms were developed including family, perinatal, developmental history, clinical examinations, diagnostic investigations, neurological evaluations , disability measures, parental stress, and quality of life. A customizable clinical letter generator was created to assist in continuity of patient care.Small cohorts and underpowered studies are a major challenge for rare disease research. This online, rare disease database will be accessible from all over the world, making it easier to share and disseminate data. We have outlined the methodology to become Title 21 Code of Federal Regulations Part 11 Compliant, which is a requirement to use electronic records as historical controls in clinical trials in the United States. Food and Drug Administration compliant databases will be life-changing for patients and families when historical control data is used for emerging clinical trials. Future work will leverage these tools to delineate the natural history of several rare diseases and we are confident that this database will be used on a larger scale to improve care for patients affected with rare diseases. According to the World Health Organization (WHO), the definition of a rare disease is one that affects every 1 in 2000 people or less. The global prevalence of these approximately 8000 rare genetic disorders is estimated to be between 150 and 350 million people . HistoriWith impending clinical trials on the horizon, rare disease researchers are realizing a tremendous need for natural history data , 11. TheThis diversity and quantity of data can be difficult to manage, so rare disease researchers must begin to utilize information management systems, or databases, to facilitate natural history studies. Rare disease research relies heavily on international collaboration and data sharing in order to recruit large patient populations to obtain adequate statistical power , 18. TheIf rare disease databases are going to be successful in future clinical trials, they must adhere to local and international regulations for electronic records. Title 21 Code of Federal Regulations (CFR) Part 11 published in 1997, from the U.S. Food and Drug Administration, outlines what is considered trustworthy, reliable record keeping. These regulations apply to any FDA-regulated industry, such as pharmaceutical companies, medical device manufacturers, biotechnological companies, and clinical research organizations. We chose to adhere to all general requirements that will be detailed below in the There are a variety of different databases available to aid researchers such as RedCap , Deduce An instance of LORIS was installed and configured for the MyeliNeuroGene Research Group at the Research Institute of the McGill University Health Centre. This database is easily accessible via a web browser and multi-modal, with the ability to capture genetics data, medical history, medical imaging, detailed assessments of cognition and motor function, and patient-derived outcomes, among other things.Within LORIS, data entry forms, or instruments, were created using the “Instrument Builder” module. Using the workflow found in the Detailed phenotyping including family history, perinatal history, developmental history, clinical evolution, time to event , neurological examination, neuropsychological assessment, etc. were developed in conjunction with other parent- and patient-reported outcomes such as quality of life, disability, and stress. The resulting instruments are summarized in Table One thousand patients and family members with rare diseases have been included into LORIS and assigned unique identifiers. This includes activation of enrollment, informed consent designation, external identifier logging, and family relationship mapping.In addition, a dynamic letter generator is currently in development to assist in forwarding patient information to other physicians. The tool compiles the patient’s data, entered via the phenotyping instruments, into a Clinical Examination Letter. In place of the database field names, highlighted in yellow in Fig. Most patients affected with rare diseases, from mildly to severely affected, support data sharing to promote research, healthcare, and knowledge transfer . We haveAn exportable dynamic letter generator has also been developed to save time when examining patients referred to the clinic. Patients with a rare disease who come to the Montreal Children’s Hospital undergo a battery of tests that can take up to two days to complete. These tests are performed in a standardized order at each visit (i.e. the order they appear in the database), to ensure consistency between research visits and research patients. All information is stored in the LORIS MyeliNeuroGene Database and can be exported in the form of a Clinical Examination Letter detailing all results, impressions, and plans to help treat the patients. This letter is then sent back to the referring physician for continuity of care. When this letter is written by hand it takes a few hours and introduces numerous chances for human error. Exporting the letter from quality-controlled instruments reduces this error and saves researchers’ and physicians’ time.In addition to the clinical phenotyping instruments and dynamic letter generator, we have outlined, for the first time, the methodology to become Title 21 Code of Federal Regulations Part 11 Compliant, which is a requirement to use electronic records as historical controls in clinical trials in the United States , 33. To A major obstacle in rare disease research is overcoming small cohorts. Developing an online database that international collaborators can access and contribute to from all over the world is invaluable for increasing cohort sizes, discerning surrogate markers, and improving natural history data. Using this FDA compliant natural history data to validate outcome measures will be life-changing for patients and families because it will lead to historical control data that can be used in emerging clinical trials.To adhere to Part 11 Compliance regulations, the LORIS MyeliNeuroGene Database has been customized to include additional security measures such as time stamped audit trails. We are currently implementing the electronic signatures and the 2-factor authentication. There is a gap in scientific literature detailing workflow and database development. As such, we will summarize the general requirements of Part 11 Compliance below and how they were implemented into our database.Users are required to have their credentials  verified before performing tasks within the database. Written policy must be signed holding users accountable and responsible for their electronic signatures (discussed further below). This written policy must be stored, and a hard copy sent to the Office of Regional Operations (HFC-100), 5600 Fishers Lane, Rockville, MD 20857.This is a method of verifying an individual’s identity based on a measurement of the individual’s physical features  or repeatable action that are unique to that person. In our case, we chose to use a unique pin separate from an authorized user’s password for 2-factor authentication.MyeliNeuroGene database is a closed environment, meaning that access to the system is controlled by the same people who are responsible for the content of the electronic records. This includes the researchers and principal investigator. Operational audits on the system are done on a routine basis. Time stamp audit trails are tracked for each authorized user to trace creation, modification, or deletion of any instrument, visit, or other electronic record. User access is hierarchical, meaning some users do not have full access to the database and may only have “read” or “write” access. The database also must ensure that no user has the same pin or password, and that pins and passwords are periodically checked and changed to prevent unauthorized use. If unauthorized use occurs, there are immediate system security notifications. Per Canadian predicate rules, records must be stored for 25 years after study completion. United States record retention rules require storage for a minimum of 10 years.The Processing pipelines must ensure data fits specific parameters and types. This is discussed in depth under the “This includes any combination of text, graphic, data, audio, or other information that is represented in digital form by the database. Electronic signatures must include printed names of the signers, dates and times, meanings , and an internal audit trail. These signatures are legally binding. Authority checks are completed every month to ensure only authorized users may sign, input, output, or modify records.A digital signature combines the electronic signature and its corresponding cryptographic authentification, usually a pin and/or password that is used to verify the identity of the signer. It cannot be copied or pasted to or from another document, making it inexorably linked to the signed document. To not become cumbersome, continuous signing periods only require the first to be two factors authenticated with a biometric identification and password.It is highly recommended that after database development a third-party auditor inspects the system and documentation put in place. Auditors alert parties of any gaps or shortcomings and can advise developers of what needs to be changed for full compliance with local and international regulations. This will be organized for the MyeliNeuroGene database.LORIS is a web-based data and project management software that stores demographic, clinical, behavioral, genetic, imaging, and patient-related outcomes accessible from any computer browser connected to the internet . MultiplTo properly set up our rare disease database, we first began by drafting a data dictionary in the form of an Excel sheet. This spreadsheet outlined all of the data entry forms, or instruments, that would be developed using the LORIS Instrument Builder module detailed below. After instrument creation, participant enrollment and data entry can begin, with query and dissemination details tackled later. An overview of the workflow can be found in Fig. Within LORIS are different modules to help researchers with no computer science or programming experience. The Instrument Builder module aids in the creation of demographic, clinical phenotyping, behavioral, genetic, imaging, and patient-related outcome measures. Each instrument can be customized with specific information such as a “Header”, “Label”, and “Scored Field” that give the instrument title, background information, and automatically calculated scoring respectively.Data entry can be standardized using a “Textbox”, “Text area”, “Dropdown”, “Multiselect”, “Date”, and “Numeric” question entry. Each question is assigned a variable name “Question Name”, for calculations and data querying, and “Question Text” which asks the pertinent question at hand. For Dropdown questions, instrument specific options can be added for every question.Instruments were first planned and drafted using Excel in the form of a Data Dictionary. Columns consisted of Question Names, type of question , Question Text, Question Options (available choices), and Formulas . Each row represented one question. Using the Data Dictionary and the Instrument Builder module on LORIS, each instrument was created: demographic forms, clinical phenotyping , behavioral, genetic, imaging , and patient-related outcomes . Instruments’ files were then uploaded onto the MyeliNeuroGene private repository on GitHub as Pull Requests for review.After instrument completion, a PHP scoring script was developed for instruments that required them. Automatic scoring reduces human error and dramatically decreases time spent on calculations. Scoring scripts were also uploaded onto the GitHub repository for review.After instruments and scoring scripts were developed, they were uploaded to the MyeliNeuroGene private repository on GitHub as Pull Requests for review. After revision and modification (if necessary), the Pull Requests were approved, and the instruments made available on an insulated LORIS staging server where beta testing occurred. After testing was completed, instruments were pushed to the LORIS production server for instrument pipeline completion and data entry.Before data entry could be completed, Subject Profiles had to be entered. Our group has consented more than 1000 patients and family members with different rare diseases since 2011, and patient and family recruitment is ongoing. To create a new profile, “Date of Birth”, “Sex”, “Site” (in the case of a multi-site study), and “Project” must be entered. Projects can be separated into different studies such as natural history, imaging, genetic, or even clinical trials assessing therapeutics. A new Subject Profile, or candidate, generates two identifier codes, a DCCID and a PSCID which are unique LORIS identifiers.After the creation of the Subject Profile, each candidate was activated in the study, designated for which informed consent form was signed, and mapped to any external identifier codes. Under “Participant Status”, we tracked the participant’s status in the study . Comments can be entered with both time, date, and author history tracked in the internal audit trail. “Consent Status” tracks the latest signed Research Ethics Board (REB) approved informed consent form. Finally, mapping the “External Identifier” is crucial for future correspondence with family doctors and other collaborators.“Create time point” allows for data entry of clinical, behavioral, and patient determined outcomes that were created during the Instrument Creation process. It also enables uploading of any imaging data collected. We customized our time points to correspond to the age of the patient. For instance, a participant’s birth date would be time point T000, and a follow-up appointment 6 months later would be time point T006. A prenatal examination 1 month before a T000 examination would be designated as T-001. The steps to creating a time point can be seen in Figs. Selecting time point T000 opens a page for all instruments developed to work on our database Fig. . Time poWe have further customized LORIS to include Family Relationship information. Linking de-identified individuals allows us to link a given patient’s disease characteristics to his/her parents’ reported measures such as parental stress or patient/parents/sibling’s quality of life. It also allows us to organize family genetic results when next generation sequencing (NGS) investigations are being conducted as well as any family/parent reported outcomes.


Background: Type 2 diabetes (T2D) poses a disproportionate burden on Benin, West Africa. However, no diabetes intervention has yet been developed for Benin’s contexts. This study aimed to explore specific cultural beliefs, attitudes, behaviors, and environmental factors to help adapt a diabetes self-management program to patients with T2D from Cotonou, in southern Benin. Methods: Qualitative data were collected through focus group discussions (FDGs) involving 32 patients with T2D, 16 academic partners, and 12 community partners. The FDGs were audio-recorded, transcribed verbatim from French to English, and then analyzed thematically with MAXQDA 2020. Results: Healthy food was challenging to obtain due to costs, seasonality, and distance from markets. Other issues discussed were fruits and vegetables as commodities for the poor, perceptions and stigmas surrounding the disease, and the financial burden of medical equipment and treatment. Information about local food selections and recipes as well as social support, particularly for physical activity, were identified, among other needs. When adapting the curriculum, gender dynamics and spirituality were suggested. Conclusions: The study demonstrates the need for culturally sensitive interventions and a motivation-based approach to health . It also lays the groundwork for addressing T2D contextually in Benin and similar sub-Saharan African countries. Type 2 diabetes (T2D) is now a global epidemic, with 15.9 million adults affected in sub-Saharan Africa (SSA), causing an annual economic cost of USD 3.3 billion. Furthermore, it is estimated that the burden of T2D on the African continent will increase by approximately 156% by 2045 [According to the Benin WHO STEPwise approach to surveillance survey (STEPS), 93.1% of the population consumed fewer than five servings of fruits and vegetables (F+V) daily, and only 29% participated in moderate-to-vigorous activity weekly . MoreoveDespite Benin’s growing diabetes burden and needs, there are no system-wide interventions to aggressively address this problem or coordinated care aimed at better diabetes management ,11. AmonFor this reason, we proposed to adapt Meta Salud Diabetes (MSD), an evidence-based diabetes self-management education program, to Benin’s context. MSD offers education on how to prevent disease complications through lasting behavioral changes . LifestyScholars and practitioners consistently advise considering culture and context when developing interventions for minority populations with diabetes ,17. ThusThis study was conducted between July and August 2019 in Cotonou, the economic capital of Benin, West Africa. The city is located in the Southeast of the country, between the Atlantic Ocean and Lake Nokoué. In Cotonou, French is the official language, and the population is estimated at 1.2 million . AlthougThe MSD intervention consists of two-hour participatory workshop-style sessions delivered within a support group structure over 13 consecutive weeks and is described in detail in a previous publication . During This qualitative study was conducted as part of a formative research effort to assess the contextual factors that might influence Benin’s MSD intervention delivery and inform necessary changes to the implementation strategy. In the present study, we conducted focus group discussions (FGDs) with T2D patients (>18 years of age), as well as community and academic partners to better understand the contextual factors influencing the implementation of the seven MSD components. The FGDs questions were derived from the literature and informed by the MSD curriculum  in four relevant secondary hospitals .A purposive sampling strategy was used to recruit patients with T2D from the 300 patients who participated in the survey, with the aim of achieving demographic diversity in terms of sex, age, marital status, and education. Medical profiles of 300 patients with T2D were assessed for inclusion criteria, i.e., being diagnosed with T2D for more than a year, being 18 years and above, and willing to give informed consent to participate in the study. Exclusion criteria were patients with extreme disease conditions that would restrict them from participating in the study. After being assured of their congruousness with the inclusion criteria, 40 participants meeting the preliminary inclusion criteria were randomly selected and invited to participate in the study. For the health system’s capacity assessment, key informants were identified in collaboration with the Ministry of Health. During the process, community and academic partners were purposively selected for participation in this study to include partners of different cadres and experiences.Study objectives were explained to the study participants, and those who agreed to participate gave their informed consent. A saturation of information was reached after three patient (9–11 participants per group), two community partner (6 participants per group), and two academic partner (8 participants per group) FGDs. In total, 12 community partners  and 2 traditional healers) and 16 academic partners  were identified and invited to participate. Of the 40 T2D patients formally invited by phone, 32 (80%) accepted the invitation. The FDGs were recorded with the consent of the interviewees and lasted roughly 2–2.5 h.Two experienced facilitators and the lead author conducted the focus groups. Facilitators were secondary school teachers with bachelor’s degrees in education. The training involved a daylong instructional session and covered modules, including qualitative data collection methods and research ethics. The instruments used to collect data were tested in two ways. First, two CHWs and four Beninese nutrition experts reviewed the guide to ensure that the FGD questions were relevant, context specific, and culturally appropriate. Additionally, three pilot FGDs were conducted with four T2D patients, four CHWs, and four physicians to test the flow of questions among the target populations. Feedback was incorporated into the final version of the guides used for data collection. For example, feedback was given regarding the cultural relevance and translation accuracy of the questionnaire. Most respondents interpreted the questions differently than what was intended in the original questionnaire. Consequently, the questions were reworded. Our pretest trials also revealed the length of time needed to conduct the FGDs, which may have contributed to fatigue, disengagement, or distress. Academic partners and patients suggested keeping FGDs limited in time to participate after work. As a result, the sequence of administering the questionnaire was given attention, but most frequently, the approach remained routine in moving from informed consent to the demographic survey, followed by the main FGD scripts, which were simplified.Every FGD began by introducing the MSD curriculum as a program designed to help individuals with T2D maintain their blood sugar levels within a healthy range . The FDGFGDs were audio-recorded and transcribed verbatim from French into English. A bilingual French and English interpreter reviewed the transcripts for accuracy. Data analysis was initiated in the field, with debriefing meetings held following each FGD. With the help of the RAs, the lead author reviewed the collected data, identified new areas to explore further, and discussed topics that were saturated. Detailed field notes were taken on each FDG. Formal analysis was conducted at the end of data collection using content analysis  and was In the first step, a matrix of codes based on semi-structured FGD guides was developed, covering the seven components of MSD. Next, two independent researchers reviewed each transcript individually to check whether the codes were appropriate and identify recurring themes that were not part of the initial matrix. Finally, the researchers reviewed the codes produced by the process, agreed on the final codes, and grouped and categorized the codes accordingly. Researchers coded the first 50 lines of the scripts for all the FGDs using the coding system, assessing its relevance and consistency between researchers and determining whether the coding system required any modification. Disparities identified by the researchers were resolved through consensus, and more codes were added when deemed necessary. By the end of the process, all the scripts were coded using the finalized coding system. The codes were summarized by intervention components and participant groups. The findings were then compared between the three respondent groups . The resn = 3 FGs; 32 participants), community partners over 35 years old , and academic partners . The majority of participants in all three groups were married. The academic groups, however, were dominated by men, while women dominated the community partners. Finally, most academic partners worked for governments, whereas community and patient partners were mostly non-government employees.In total, seven FGDs with 60 participants were held . ParticiSeveral sociocultural and environmental factors that may impact MSD intervention delivery were identified. The derived themes informed the cultural adaptation of each component of the MSD, as illustrated in Participants discussed multiple challenges to healthy eating, including access to healthy food, cultural norms and perceptions, and gender roles. All participants emphasized the relevance of information on locally available foods.Access to healthy food. Participants mentioned cost, seasonality, and distance to healthy food markets as limitations to accessing them. These factors tend to be intertwined, as people need to travel farther to buy more affordable and healthy food.the decision about which fruit and vegetables to buy in the market is often based on the price, not the quality.”“… (Patient)I have to go far from my house to buy cheaper fruits and vegetables.”“(Patient)because there are certain seasons when there are no fruits.”“…. (Academic partner)Cultural perception and norms. Along with the access to food, another recurring theme that was resonated among the participants was cultural perception and norms, especially concerning fruits and vegetables. Notwithstanding their nutritional benefits, vegetables and fruits have been considered commodities of the poor.People think that eating vegetables is just being poor.”“(Community partner)Even though it is fruit season, people do not take it. Plenty go to the trash. For example, when you go to the north of the country, no one looks at mangoes.”“(Community partner)Well, because fruits and vegetables are not part of our culture. They are not food that can fill us up.”“(Community partner)Gender role. Specifically, among patient FDGs, the process and act of cooking was associated with gender, suggesting the importance of engaging women in the curriculum, especially for the healthy eating component.Usually, our wives cook. My wife prepares for the house, goes to the market, and decides what to eat in the households.”“(Patient)Lack of information on local foods and recipes. The participants across the groups echoed the lack of culturally relevant information based on locally available food options.Recipes based on our local foods will be helpful.”“(Academic partner)We need to focus on local diversity. Our local diversity is rich. You have to inform people. Not everything is expensive. People do not have the information of what is available.”“(Academic partner)The responses from the participants indicated that group support and neighborhood characteristics might facilitate engagement and participation. The respondents also voiced their preferences for easy and enjoyable activities to mitigate some of the patients’ challenges and guides for physical activities.Group support and neighborhood characteristics as a potential facilitator for engagement and participation: Across all groups, many respondents commented that motivation and sustained participation in physical activities would mainly depend on within-group support. Specifically, the patient FGDs noted that neighborhood characteristics might determine participant engagement. Finally, many participants echoed that forming a group that functions as a sports club will improve engagement and participation in exercises and physical activities.If we are all in the same geographic area, it will be easier. It is said that it is unnecessary to join a club to do sport. Why don’t we form this club?”“(Patient)The support among us is better than a family who does not have diabetes and does not know about the disease.”“(Academic partner)Preferences for easy and enjoyable activities. The majority of participants emphasized that they would prefer physical activities that are easy to follow due to the factors such as age, pre-existing conditions, and fatigue from work, and would prefer activities that are enjoyable to keep them motivated.Easy activities because most diabetic patients are old or overweight.”“(Community partner)Not complicated activities because they complain that they are always tired.”“(Academic partner)Pick activities they will enjoy. Most adults need exercise to be fun, or they lose their motivation to do it over time.”“(Community partner)Lack of guides for physical activities: During the academic partner FGDs, participants noted that manual guides for physical exercise would provide examples of exercises for those who may not have adequate knowledge of which activities they could engage in.If they can have some examples of physical activities, it will help a lot, specifically for those who do not have any activity.”“(Academic partner)Monitoring the blood sugar level seems to be often obstructed due to the limited availability of personal glucose monitoring test kits in the market, financial burden, and perspectives toward self-directed glucose monitoring.Availability of medical equipment in the market. Participants, especially from the patient group, voiced their concern about the availability of medical equipment required to monitor blood sugar levels, such as strips and glucose meters.I have five blood sugar meters. When I take one, there are no strips on the market, and I have to buy another.”“(Patient)There are different brands. A pharmacy sells glucometers, and they do not even have corresponding test strips.”“(Academic partner)Financial burden. One of the barriers to monitoring blood sugar levels that recurred among respondents was financial concerns. However, participants noted that support for the medical supplies could motivate participation in future programs and events.I do not have a monitor, so I go to the hospital for my blood sugar when I have the money.”“(Patient)I will suggest not giving money directly but the devices, the glucometers that everyone needs to have at a lower cost.”“(Patient)I think that incentives such as free materials to use glucometers, blood tests, etc., can also motivate them.”“(Academic partner)Perspectives toward self-directed glucose monitoring. Participants also voiced that, in general, most people are not culturally accustomed to self-directed glucose monitoring and noted that this perspective might be a potential barrier to regular blood sugar monitoring.They are afraid of sting/puncture, and it hurts.”“(Community partner)Glucometers, they do not use. They are afraid of blood.”“(Community partner)The focus groups’ responses indicated that financial burden seems to be one of the most daunting challenges that impedes medication adherence. In addition, some participants expressed a lack of information on medications, while academic partners confirmed that they often do not have time during clinical consultations to explain and allay concerns about medications and other treatments for diabetic patients.Financial burden. Participants discussed the financial burden and the significance of financial support for treatments and medical supplies to manage diabetes.Some complaints about the burden of the cost of drugs.”“(Community partner)there are drugs for treatment. For example, Diamicron costs 10,000 CFA and depending on the prescription, you can spend more than that a month. It will be great to have help with drugs.”“… (Patient)Others are also afraid of going to the hospital because of the money. Therefore, they go to the traditional therapist.”“(Community partner)Insufficient information during doctors’ consultation. Doctors often find themselves without sufficient time to provide a detailed consultation, potentially leading to a lack of knowledge and awareness of adequate medication adherence.Although doctors advise when it comes to medications, they do not have time to address some misconceptions about medications or explain their side effects.”“(Academic partner)I am told, if they inject insulin, it is not good for your body. Therefore, the doctor told me to never go on insulin and to continue with the drugs.”“(Patient)Patients with personal monitoring kits tend to manage blood sugar levels, quickly adjusting their diets accordingly. However, insufficient information on recommended diets and glycemic index of foods and poor portion control of foods, and difficulty making healthy choices when eating out seemed to be an impediment, particularly for those without the kits.Maintaining optimal blood sugar levels enabled by access to the monitoring kits. Patients, especially those with the monitoring kits, were more likely to monitor blood sugar levels and adjust their diets regularly. They were able to set the goal for themselves to maintain sugar levels in the targeted range.I control my blood sugar almost every morning. When I have bad results, I try to see what went wrong yesterday in what I ate to find out where the problem is. I keep looking until I have the solution.”“… (Patient)For example, my rate was 2.59 yesterday. Today I lowered it to 1.35. I reduced the amount of fruit and ate more of our vegetables, and mix with ‘tchairou’.”“(Patient)Lack of information on nutrition values and glycemic index of foods. Maintaining blood sugar levels in an optimal range could be challenging, as there was a lack of information on adequate diets recommended for T2D patients.Due to the lack of information on food and cooking, I suffer from ulcers. Because of “do not eat this or that,” I do not know what to eat anymore. Sometimes I do not eat for four days, and I just drink water.”“(Patient)It looks like there are fruits rich in sugar, and some have low sugar levels. There is the information aspect, which we do not have.”“(Patient)Challenges related to portion control of foods and making healthy choices when eating out: Patients and academic partners reported the importance of considering local food options with the indication of portions when eating out in the context of Benin.The environment is not favorable. At home, in terms of food, you can do whatever you can. However, when you come to town and go to the food vendor, you have to use what is available.”“(Patients)People with diabetes need reference bowls and portions suitable for Benin. Many eat two out of three meals outside the home.”“(Academic partner)Although respondents across the groups could provide their perceptions and experiences concerning the other themes such as foods, physical activities, and monitoring blood sugar levels, little was discussed and emerged concerning this component. Part of the reason could be attributed to a lack of awareness and knowledge, as hinted in the response below.Diabetes affects many people in Cotonou. They die without knowing the reason. Most people rarely go to the hospital and many are sick all the time. Most of them have foot amputations. Also, people do not know what to do with the disease.”“(Community Partner)Negative perceptions toward diabetes, which may be a potential barrier to healthy coping with stress, were mentioned universally across the groups. The coping mechanisms that emerged included denial, religion, and focus on what is controllable.Negative perception toward the disease. Of particular interest is the perception toward the disease, especially among the patient groups. Most of them perceived diabetes as a terminal disease with its complications as fatalistic. Diabetes was often associated with negative words such as the end of life and death, as stated below. Participants across the three focus groups mentioned that people who have diabetes might suffer additional social stigma issues. Participants had convergent thoughts that patients’ social stigma makes them reluctant to have open discussions about diabetes. It implies a greater perceived burden of the disease in the context, potentially leading to increased stress.When they are told about diabetes, some parents think that we are already at the end of life and therefore that spending on drugs may be useless.”“(Patient)Even before the illness, I am told that it is the disease of the rich. These are the people who eat rich dishes, who drink champagne. That is why today, many people with diabetes do not like being known as such.”“(Patient)This statement resonated with other participants as well. Compared to other chronic diseases such as hypertension, participants reported a higher perceived severity for diabetes’ prognosis.When you say you are hypertensive, it does not bother anyone. However, when you say you have diabetes, it is as if you will die tomorrow. You no longer have the support of the family.”“(Patient)Denial. Potentially due to the negative perceptions and social stigma associated with the disease, some patients seemed reluctant to accept the status, ignoring the fact that they have the disease.I know it is a disease. However, to say that I have diabetes is like trying to give myself a name. I know I have diabetes, but I do not want to be told that I have diabetes.”“(Patient)The problem is that people do not want you to know they have diabetes. They think they are going to die anyway. Some do not care and do what they want.”“(Community partner)Roles of religion. When asked how they manage stress, religion seems to serve as a coping mechanism to drive away negative feelings or impulses.I am happy that we are all Christians. Therefore, on the psychological side, we do not need help. Because if we are called Christians, we will not say that we are tired or want to kill ourselves or isolate ourselves from the world.”“(Patient)What is depression? For me, it is a matter of faith. There is a supreme being.”“(Patient)Focusing on what is controllable. Another strategy that participants utilize to cope with stress included focusing on what is controllable. For example, rather than focusing on negative feelings, patients were actively engaged in physical activities or monitoring the blood sugar level to distress during difficult periods.What I do is to control my blood sugar almost every morning. When I have bad results, I tell myself that I have not done what I have to do. I run with friends.”“(Patient)All too often, self-management interventions are not adequately matched to the intended target audience’s characteristics and circumstances in sub-Saharan Africa (SSA) ,14,15. TSimilar to the literature , this stAll participants saw physical activity as critical to improving their diabetes outcomes. However, they noted that interventions to increase exercise would be more effective if patients received good social support, especially peer support and modules with enjoyable exercise routines. This finding is similar to Thomas et al.’s  conclusiA patient’s ability to self-manage diabetes often hinges on adherence to treatment, good coping skills, and problem solving when their blood sugar targets are not being met. As previously observed in SSA ,45, finaThis study also sheds light on negative social perceptions and stigmas associated with diabetes, which have significant implications, since they may lead to self-denial, hinder health-seeking behaviors, and negatively affect program acceptance. The findings indicate a need to sensitize the population and increase its awareness of the disease. Further attention should be given when publicly communicating the intervention to avoid unexpected adverse outcomes. Our study offers little evidence concerning reducing diabetes-related complications, reflecting a lack of awareness in the region regarding the specific complications and risk reduction . As prevThe study has several strengths and limitations. First, this study utilized qualitative comparison groups to elicit multiple perspectives involving patients, community partners, and academic partners . Second,The MSD program must incorporate a meal menu plan that complies with the Benin nutrition therapy guidelines while taking into account factors such as access, acceptance, and accessibility of local foods. Additionally, the curriculum should include resources that facilitate adherence to the menu, such as recipes, cooking tips, a list of local foods and sources, and information on low-glycemic index foods.A revised curriculum could also gradually incorporate nutritious fruit snacks into the daily diet, introduce healthy foods patiently , and ensure that portions are reasonable based on local bowls.Considering the DSME component of physical activity, findings suggest that Beninese physical activities should be based on group support and consist of easy and enjoyable activities like dancing to local music.Integrating glucose monitoring strategies within a collectivistic family framework will be essential for the MSD program to succeed. This strategy will involve the entire family in helping persons with diabetes monitor their blood sugar levels.Due to the lack of information regarding diabetes medications, clear guidance on how the medications work and are developed from traditional medicine will be imperative. Education on continuing medication will also be crucial, since Benin has a low compliance rate.Patients were reluctant to tell other family members about their diabetes symptoms for fear of burdening them. To combat these misconceptions, it will be crucial to stress the importance of being open with one’s family about diabetes symptoms for participants to ensure their long-term well-being. Early detection of symptoms and treatment can prevent significant problems and stressors in the long term.Patients with T2D also described the fatalistic belief that diabetes is God’s punishment. However, given the context of Benin, where religion plays a key role, spirituality and faith could be used to counter fatalistic beliefs.Given patients’ reticence to express their emotions, particularly outside of families, a MSD intervention must foster communication and trust  to allow for discussions about stress and how to cope with it.Moreover, in addition to identifying specific cultural aspects of each DSME element, it will be essential to consider the matriarchal culture in Benin with distinct gender roles, since women are heavily involved in food preparation.The findings of this study has implications for the success of the MSD program in Benin. Recommendations emerging from this study include the following:Ultimately, we can conclude that this study contributes in several ways to our understanding of the role of sociocultural norms, perceptions, and reality in implementing an evidence-based diabetes self-management education curriculum and provides a basis for adaptation to make it more effective. Additionally, this study fills a significant knowledge gap related to barriers and facilitators to the seven essential self-care behaviors in urban communities of Benin. Future intervention strategies should target the identified factors and emphasize the overlapping themes from our study and previous studies. Adding awareness of local foods and practices, adjusting cooking practices, and addressing stigma and gender roles associated with diabetes could be helpful when working with the surveyed communities. Additionally, as well as showing favorable views from key stakeholders about diabetes self-management programs, the findings highlight gaps in the healthcare system. We contend that this program could inform a more contextually specific self-management diabetes intervention, alleviate many of the challenges currently faced by diabetes care providers in Cotonou, and address the rising prevalence of T2D in the country.


APACHE IVa provides typically useful and accurate predictions on in-hospital mortality and length of stay for patients in critical care. However, there are factors which may preclude APACHE IVa from reaching its ceiling of predictive accuracy. Our primary aim was to determine which variables available within the first 24 h of a patient’s ICU stay may be indicative of the APACHE IVa scoring system making occasional but potentially illuminating errors in predicting in-hospital mortality. We utilized the publicly available multi-institutional ICU database, eICU, available since 2018, to identify a large observational cohort for our investigation. APACHE IVa scores are provided by eICU for each patient’s ICU stay. We used Lasso logistic regression in an aim to build parsimonious final models, using cross-validation to select the penalization parameter, separately for each of our two responses, i.e., errors, of interest, which are APACHE falsely predicting in-hospital death (Type I error), and APACHE falsely predicting in-hospital survival (Type II error). We then assessed the performance of the models with a random holdout validation sample. While the extremeness of the APACHE prediction led to dependable predictions for preventing either type of error, distinct variables were identified as being strongly associated with the two different types of errors occurring. These included a primary set of predictors consisting of mean SpO2 and worst lactate for predicting Type I errors, and worst albumin and mean heart rate for Type II. In addition, a secondary set of predictors including changes recorded in care limitations for the patient’s treatment plan, worst pH, whether cardiac arrest occurred at admission, and whether vasopressor was provided for predicting Type I error; age, whether the patient was ventilated in day 1, mean respiratory rate, worst lactate, worst blood urea nitrogen test, and mean aperiodic vitals for Type II. The two models also differed in their performance metrics in their holdout validation samples, in large part due to the lower prevalence of Type II errors compared to Type I. The eICU database was a good resource for evaluating our objective, and important recommendations are provided, particularly identifying key variables that could lead to APACHE prediction errors when APACHE scores are sufficiently low to predict in-hospital survival. Its current iteration is APACHE IVa, a relatively minor but improved update to APACHE IV, which itself was a major update to the APACHE scoring system2. APACHE scores are generated from many demographic and physiologic measures, as well as diagnosis, collected during the first 24 h of a patient’s ICU stay2.The acute physiology and and chronic health evaluation scoring system, otherwise known as APACHE, is widely known to be at least a reasonable, if not good, predictive tool for in-hospital mortality and length of stay for patients in critical care3, and India4). Even in settings where it is generally accurate, such as for predicting in-hospital mortality in United States-based ICU’s, there are dynamics which may preclude APACHE from reaching its ceiling of predictive accuracy. For example, patients with high APACHE scores can receive sufficient intervention during their stay to avoid a predicted in-hospital death , and patients with low APACHE scores may experience unexpected complications during their stay, which may result in an in-hospital death that APACHE failed to predict . In addition, before these types of dynamics can lead to an unexpected patient outcome, there may actually be features observed in the first 24 h of an ICU patient’s stay that are predictive of an inaccurate APACHE prediction.Despite its usefulness for predicting outcomes such as in-hospital mortality of ICU patients, APACHE (IVa) is not infallible. For example, it has been shown to be challenging to externally validate the promising results of predicting in-hospital mortality to different countries  might be one of the reasons for a false negative prediction. Hence, false negatives are especially important to focus upon in terms of seeing patient features that are associated with them. This is also the primary motivation to separate the analysis of the two types of APACHE prediction errors.5. In addition, we will present models that show which features collected within the first 24 h of the ICU stay are associated with each of these two types of APACHE prediction error. That is, we will identify variables early in an ICU stay that might be predictive of APACHE IVa not doing as well in its outcome prediction of in-hospital mortality as expected. We will defer to future work to describe the dynamics after the first 24 h of a patient’s ICU stay that may lead to the APACHE predictions not being correct.In this paper, we aim to quantify both the number of false positive and false negatives of APACHE IVa in a large heterogeneous multi-hospital ICU database collected from over 200 hospitals in the United StatesThe paper is organized as follows. In “eICU Collaborative Research Database, version 2.0 (PhysioNet5). There are over 200,000 de-identified ICU admissions in over 200 hospitals across the United States, during the period from 2014 to 2015. Further details on this database, including available patient measurements, diagnoses, care planning, etc., can be found in Pollard et al.5). We will discuss what aspects of the database that we will be using, as well as how we defined the final cohort used in our analyses, in the subsections below.The ICU database that we will be using in this paper is formally referred to as the 2. Patient APACHE IVa scores, and associated in-hospital mortality prediction probabilities, are both obtained directly from one of the available eICU tables, titled apachePatientResult. APACHE IVa tends to overpredict mortality risk, which, in our analysis cohort, produces an average mortality rate of 11.96% compared to the observed average mortality of 9.91%. Figure The APACHE scores provide a typically accurate estimate of ICU patients’ in-hospital mortalityFor Fig. For Fig. Type I Error, where APACHE predicted a patient to die before hospital discharge but the patient survived the hospital stay. This represents a false positive in APACHE mortality predictions; andType II Error, where APACHE predicted a patient to survive to discharge but the patient died before being discharged alive from the hospital stay. This represents a false negative in APACHE mortality predictions.In our subsequent analyses, our target responses are defined as the two types of errors that the APACHE IVa model could make: 6;Vital sign values, which includes the mean and worst values of vital signs captured in the first 24 h of an ICU stay, as suggested by Cosgriff et al.The most abnormal laboratory results;Treatments;Care plan and cure goals documented by physicians;Admission diagnoses; andPatients’ medical history.Variables from sources 1, 2, 3, and 4 were measured throughout the entire in-hospital stay, but we only considered the values observed within the first 24 h of an ICU stay to align with the data used in the calculation of APACHE (IVa) scores.We have incorporated the following six sources of variables into our analyses: 6, i.e., we only considered the mean and worst values for vital signs, and the worst laboratory test results in the first 24 h. Specifically, we extracted the minimum values for base excess, calcium, chloride, fibrinogen, hematocrit, hemoglobin, magnesium, pH, phosphate, platelets, and serum bicarbonate, and the maximum values for amylase, B-natriuretic peptide, bilirubin, blood urea nitrogen, creatinine phosphokinase, lactate, lipase, prothrombin time, serum creatinine, and troponin. For glucose and serum sodium, we selected the value that deviated the most from its normal range. For neutrophil and white blood cell counts, the minimum value was selected only if any measurements fell lower than the lower limit of its normal range; otherwise, the maximum was selected.Variable sets 1 and 2 were created in light of the definitions recommended by Cosgriff et al.6 suggested a way to group them into 21 meaningful categories, i.e., acute coronary syndrome (ACS), acute renal failure (ARF), asthma emphys, coronary artery bypass grafting (CABG), cardiac arrest, chest pain, congestive heart failure (CHF), coma, cerebrovascular accident/stroke (CVA), diabetic ketoacidosis (DKA), gastrointestinal (GI) bleed, GI obstruction, neurologic problems, overdose, pneumonia (PNA), sepsis, trauma, valve repair/replacement, other cerebrovascular problems, other respiratory problems, and other diagnoses. 20 binary features were hence created to indicate the presence of each admission diagnosis, with ACS as the baseline, i.e., the comparison group. More details are available at the authors’ GitHub site7.In dealing with the admission diagnoses, which were available only in text form, Cosgriff et al.i, let i:Another important feature that we defined was the APACHE Relative Confidence (ARC), which measures the distance from APACHE’s mortality prediction for each patient to a pre-defined cutoff, that is the cutoff determining whether or not a predicted probability is mapped to in-hospital death or survival. This distance was mapped to a range between 0 and 1, through dividing by the max possible distance, to reflect the relative confidence of APACHE in its mortality prediction probability. Formally, for a given patient We hypothesized that a larger ARC  would suggest greater confidence in APACHE making a correct mortality prediction and thus lead to a smaller likelihood of both Type I and II errors. This speculation can also provide some justification why we may consider including ARC in our two types of APACHE error prediction models; that is, if the association is strong between the ARC and the APACHE error being predicted in a given model, then a quick check on ARC may suggest whether or not an APACHE error might have a reasonable chance of occurring (ARC near 0) or not occurring (ARC near 1) for a given patient. Alternatively, if there is greater interest in identifying a larger collection of original variables as being associated with an APACHE error, then one may consider excluding ARC in the models, assuming ARC has a strong association with errors occurring, as speculated above. That is, if ARC is not included, then perhaps some of the original predictor variables that are considered in the model  may end up having explanatory value for the APACHE error occurrence and thus explicitly identified.Missing values in our final cohort mainly came from those features related to lab tests, i.e. the tests which were not performed in the first 24 h.8; (3) replace the missing values with grouped means based on a bucketing approach of the APACHE IVa scores9.If two or more measurements were observed for a certain lab test in the first 24 h, the worst value was chosen as what we defined above. If only one measurement was observed, we considered this value as the lab test’s worst value. If no measurement was observed, i.e. the lab test was not performed in the first 24 h, then this missingness was dealt with in one of the three following approaches: (1) replace the missing values with the pooled imputed values from multiple imputation, using 10 imputations; (2) replace the missing values with the median of the entire population and creating a binary flag for each lab test indicating if the test was not performed in the first 24 hDivided the patients into 10 equal-sized groups based on their APACHE IVa scores;j with the mean values in group j, Replaced any missing lab test values in group Specifically, in the third approach, the following steps were implemented to define the buckets: 10 and routine health data11, and thus we also utilized this method to address lab value missingness in our study. We also used the two easier-to-implement approaches mentioned above, i.e., fill missing values with population median or group means, respectively, as a basis of comparison.Previous studies have shown the superiority of multiple imputation (MI) in addressing missing values in different contexts, including patient registries5, “Patients will not have an APACHE IV hospital mortality prediction if they satisfy exclusion criteria for APACHE IV , or if their diagnosis is not documented within the first day of their ICU stay”. Hence, these patients either were not intended to have a calculated APACHE score or were otherwise incapable of having it calculated without additional required information on their first 24-h diagnosis. Thus, our paper’s focus is only on those patients who were both eligible and able to have an APACHE score and associated in-hospital mortality prediction calculated, and, hence, we do not impute APACHE IVa for our analyses. This decision is also consistent with Cosgriff et al.6, discussed in the next subsection. Additionally, due to the proprietary nature of APACHE IVa model, our attempts to impute the APACHE IVa scores, as well as its subsequent in-hospital mortality predictions, were deemed invalid without knowing the specific model, including variable weights for the scores and link function for the predictions, that APACHE established between the two. Attempts at direct imputation of the missing APACHE in-hospital mortality prediction did not work either, producing very unstable results, likely due, to at least some extent, to lack of knowledge of the variable weights used in their prediction model.Regarding missing APACHE values, we defer to the researchers who helped create the eICU database6. Specifically, we removed the patient stays with no APACHE IVa scores or Glasgow coma scales, admitted due to lung or heart transplant or burns, age under 16 years, re-admissions or admitted from other ICU’s, and those without periodic vital records. This resulted in a remaining cohort of 134,946 stays, consisting of 115,350 unique patients.We started by following the same cohort selection process as in the work by Cosgriff et al.Next, we imposed a further restriction to study only the last ICU stay in the final hospital stay for patients with multiple stays. However, the database did not always provide a way to allow a correct identification for the ordering of multiple hospital stays for one patient, primarily implemented due to privacy/confidentiality concerns. Consequently, we were not able to correctly identify the very last hospital stay of a subset of patients, and these patients’ records were subsequently removed from our analysis. In all, 97,547 confirmed unique final patient ICU stays remained.As we were also interested in examining if a patient’s cure goal plan, physician intervention type and frequency, admission body mass index (BMI), or their medical histories would affect APACHE’s performance, patients without those records were removed. Of the 97,547 unique stays remaining, 89,322 had these required variables. See Fig. 20% of these 89,322 unique stays were randomly split off as a holdout final validation set. All initial model building, including model parameter tuning via (10-fold) cross validation, was conducted on the remaining 80%.12 to try to account for the heterogeneity from patients across different hospitals on the given APACHE prediction error (binary) response using a random intercept term. However, the heterogeneity exhibited through the estimated random effect variance was very small, and hence a random intercept term was not required. We then used a less complex generalized linear model, specifically logistic regression, with Lasso penalization. As is typically the case, the penalization was imposed in order to find a useful model that is more parsimonious than had no penalization been imposed. In addition, Lasso penalization, in general, leads to less biased final models than do stepwise regression techniques13.We started with the idea of using a generalized linear mixed-effects model with Lasso penalization14.For all models, the variables were standardized to mean 0 and variance 1 before modeling. An adaptive synthetic sampling (ADASYN) was utilized to avoid a prediction of all 0’s since there were extremely low positive rates for Type II errorsIn our modeling approach, two parameters were required to be specified or tuned by the user: (1) the cutoff probability for mapping a dying versus surviving prediction; and (2) the penalization term  coincide with the observed mortality rate in the cohort (9.91%).When the APACHE prediction exceeds a certain pre-defined cutoff, we say the APACHE model is predicting the patient to die. We could have carefully tuned this cutoff with a great deal of computation, but we decided instead to examine the following cutoffs, with justification: 0.10, 0.50, and 0.33. We chose 0.10, as it is a cutoff sometimes used for defining patients at high risk17); or two, choose the largest penalization term within the 95% confidence interval of the one that produced the best evaluation metric. The penalization term chosen in approach two is always larger than the one chosen using approach one and thus always produces a more parsimonious model .With the goal to select the most significant features which may indicate an APACHE error, we chose a penalization value that would result in the most parsimonious model while not making a major compromise in model performance. There are two commonly used approaches in the literature for choosing the penalization term: one, choose the largest penalization term within one standard error from the one that produced the best evaluation metric , making it much more vulnerable to a single false prediction than a model with higher prevalence error rate .The results in Table mentclass2pt{minimIn Table In Table Tables Comparing Tables primary set of predictors, as they were strong enough to compete with ARC and were also the ones with strongest predicting power in the models without ARC. We then refer to the other predictors that made only the models without ARC as the secondary set of predictors.Given the difference in the predictors obtained in the models with or without ARC, we refer to the predictors that made the final model in the presence of ARC as the Among all the lab tests and all patients, 1,196,502 (39.4%) out of the 3,036,948 patientWhere there is at least one measurement for a patient1. Our goal in this paper was to use a large heterogeneous multi-hospital ICU study5 to attempt to identify some predictors of patient information collected in the first 24 h that might be predictive of when the APACHE prediction may go wrong regarding its speculation on in-hospital mortality.The APACHE scoring approach  is considered to be a good prediction tool in intensive care units for speculating on outcomes such as in-hospital mortality and length of stay based on data collected within the first 24 h of a patient’s stayAs there are two ways of APACHE leading to prediction errors, with these predictions likely leading to different approaches to patient care, we thought it was important to separate the two errors as: (i) Type I errors, or false positives , and (ii) Type II error, or false negatives .We used an out-of-sample validation approach, following a model-building step using cross-validation, to model these two errors separately with multiple logistic regression with Lasso penalization. The differential results, seen in Tables ARC variable, which represents the absolute relative difference between the APACHE prediction for a given patient and a pre-determined cutoff score used to classify a patient into either an in-hospital death or survival prediction, being very important for both types of errors. Specifically, the higher the ARC, the less likely APACHE will make an error. This suggests having intensivists pay close attention to lower ARC scores, as these are the ones that appear to be more subject to both types of errors  lactate and mean SpO2 made the final model, whereas, for Type II error, only worst (min) albumin and mean heart rate made the final model as our primary set of indicators. For a given APACHE prediction, according to our findings, the small number of predictors listed in either Tables We also presented results from models that did not include ARC as a predictor. This allows us to compare the performance of models, as well as parameter estimates, when including or excluding ARC. Unsurprisingly, the results for the non-ARC models result in a larger number of predictors. Notwithstanding any differences in predictive performance for seeing the success (or lack thereof) of APACHE’s prediction for a given patient, the choice of ARC’s inclusion may be dependent on a given researcher’s preference: if ARC is excluded, i.e., the extremeness of the APACHE (IVa) prediction of in-hospital mortality in model building is ignored, we may uncover additional variables of interest in the prediction of APACHE making incorrect classifications, but when ARC is included, the result will be a more parsimonious final model that still has relatively good predictive performance for identifying when APACHE may make a prediction error for in-hospital mortality.Specifically, in the models predicting Type I errors but with ARC removed Table , we obse20 of the models fit here in ICU’s in other countries, noting that evaluating performance of our models on the holdout validation sample amounts to a pseudo-external validation approach, and actually, within eICU, a form of external validation could be done by holding out randomly selected hospitals, as opposed to patients. This investigation can be an area of future work.We should mention some limitations of this study and areas of future work. In terms of limitations, though we utilized a large multi-hospital cohort from over 200 ICU’s, this was still a U.S.-based cohort, and the results may not necessarily replicate in other countries. It will be interesting to see true external validation21. The three distinct missing data approaches we used led to similar results, but this does not guarantee a modification of one of the current approaches  or a different missing data approach might not lead to more accurate results for these data. This could be a further area of research in dealing with the prediction of when APACHE may lead to errors. Simulation studies could be helpful for such an investigation.We also had to make a number of decisions in our cohort selection and modeling efforts, as described in “In this paper, we wanted to directly investigate variables associated with APACHE’s occasional failure for predicting in-hospital mortality. However, some may prefer an approach that directly models the prediction of in-hospital mortality, while adjusting for APACHE score, and see variables that remain important in the model after controlling for APACHE. Such differences in approach could be compared in future work, though the separation of the two types of prediction errors discussed in this paper may be better done by modeling such errors separately. The key issue for this latter approach is then determinining the appropriate APACHE prediction probability cutoff value , and this cutoff choice might need to be part of a tuning process for other datasets.22. This said, it is still important to know what variables are observed early that might lead to an APACHE prediction error, especially in the case where a low APACHE score might suggest less scrutiny of such patients is required as compared to those with higher scores. Hopefully, the work presented here is informative on its own, and that it will also lead to further research into the use, inspection, and potential improvement, of APACHE predictions of in-hospital mortality of ICU patients.Finally, we should reiterate that we only focused upon information collected in the first 24 h of an ICU’s patient stay, based on the information that is used in generating APACHE scores. But an additional area of future work is to identify what happened to the patient after the first 24 h that tends to be associated with the original APACHE prediction leading to an error. This could potentially be done in a causal inference frameworkARC, was found to be the most important predictor, with higher (closer to 1) ARC being associated with a lower likelihood of both types of APACHE prediction errors occurring. ARC could prove to be an efficient check on when APACHE prediction errors on in-hospital mortality may occur. Our analyses also suggested that a lower max lactate and higher mean oxygen saturation (SpO2) are associated with a higher chance of APACHE making a Type I error; and that lower worst albumin and higher mean heart rate are associated with a greater chance of a Type II error. However, we also generated models that did not include ARC, in order that potentially a longer list of variables could be identified as being associated with either type of APACHE prediction error; this indeed proved to be the case, including age and ventilation for Type II error, and care limitation for Type I error.In conclusion, we have determined the variables available within the first 24 h of a patient’s ICU stay that may be indicative of the APACHE IVa scoring system’s occasional prediction errors, through logistic regression modeling with Lasso penalization. The variable, Supplementary Information.


I point out fundamental mathematical errors in the recent paper published in this journal ‘Quantum correlations are weaved by the spinors of the Euclidean primitives’ by Joy Christian. In each paper, he proposes a local hidden variables model  that according to him explains those correlations and reproduces them. According to the celebrated result known as Bell’s theorem [S3: the surface of the unit ball in S7, thought of as a quaternionic three-sphere rather than a real three-sphere.Christian , publishGeometric Algebra (GA), based on the interplay between Clifford Algebra (CA), an important part of abstract algebra, and Geometry. It has been promoted as a universal language of physics. Important parts of quantum information theory have already been rewritten in the language of Geometric Algebra, though so far this did not have much impact on the field.A constant theme in the author’s papers is to exploit a very well known and very established part of mathematics, Foundations of Physics. The author had not published any of his papers in peer reviewed journals, so many of the early critiques were not even submitted to regular journals, but remained, like his, as preprints on the preprint server arXiv.org. The author did respond vigorously to all critiques in yet more preprints and has always maintained that all criticism levelled at it had been mistaken and has been adequately refuted by himself.In Gill , I publiIn my paper, I argued that the author’s work was built on a combination of ambiguous notation and elementary errors in logic, algebra and calculus. My paper included a section on an earlier version of the paper which is the topic of discussion here.IEEE Access [IEEE Access. I am grateful for the invitation by RSOS to react to [RSOS [Time did not stand still, and after 2018, the author published a pair of companion papers, Christian 2019, 2020), in the journal E Access ,6. At thE Access  on the s, 2020, ito [RSOS , who indIf physicists want to ‘rescue’ the author’s ideas because they see something in the underlying idea, it is up to them to do so. Physicists have again and again enriched mathematics by intuitively and with deep physical insight discovering patterns and abstract structures hitherto not known to mathematicians. This will surely happen again and again in the future. Debate on Bell’s theorem will also continue for many years to come. In appendix A, I will discuss proposals by two referees of this paper to salvage the author’s program. I argue that they cannot succeed. In fact, there are several other controversial and, in my opinion physically more interesting options for those who want to get around Bell’s theorem. There are plenty of fora where these controversial alternative quantum foundations are vigorously discussed, and a huge (peer-reviewed) literature. I am afraid that the author’s work has not added directly to the ongoing debate, though it did stimulate several original contributions which have made Bell’s own case even stronger than it already was, and made the task of opponents thereby harder still.That is not to say that Bell’s theorem does not have a sound mathematical core. When stated as a mathematical theorem in probability theory, there can be no doubt about its validity. My work on the subject does not challenge this mathematical core, if it is viewed as a piece of mathematics.This is however exactly what the author’s works, and in particular his RSOS paper [not do. He does challenge the core mathematics of Bell’s work, and moreover, in [challenge the metaphysical conclusions regarding locality and realism derived from that mathematical core. My work thus draws a sharp distinction between the mathematical core of Bell’s theorem and the metaphysical conclusions derived from it.But drawing that ‘essential distinction’ is exactly what he does not do. He adopts Bell’s own mathematical framing of the concepts of locality and realism and argues that one of the basic mathematical steps in Bell’s proof is wrong. But it is his own argument which is evidently wrong. He does claim that our conception of space needs revision, but in view of its defects, his own work does not provide much support for that claim.In , the autOS paper , do not over, in , he builover, in  to asser. 2mathematical claims in Christian [Hurwitz theorem  stating that Bell’s theorem, Bell [In this section, l discuss two established mathematical results which contradict hristian . These asee Baez ; and (b)em, Bell  on the iI also discuss two no-go theorems saying that certain quantum correlations cannot be simulated on a network of classical computers, where the allowed connections between computers mimic the spatial-temporal relations involved between physical subsystems of a typical Bell experiment. These can be thought of as theorems of computer science, but they are also merely repackagings of the mathematical core of Bell’s theorem. I bring them up because the author illustrates his mathematical claims by Monte Carlo computer simulations, and I use them to provide more evidence for my own claims, that his theory is badly wrong.Gull’s theorem, Gull [enhanced using martingale theory (probability) and randomization (statistics).These ‘theorems from computer science’  are (c) em, Gull , Bell’s em, Gull , which iRegarding Gull’s theorem (c), astrophysicist Stephen Gull is another powerful promotor of geometric algebra. Readers familiar with Fourier series and time-series analysis might find Gull’s proof of Bell’s theorem much easier than Bell’s. The fact that the result can be proven using a myriad different mathematical approaches is further evidence that it stands like a rock.et al. [Regarding my own contribution (d), recent ‘loophole-free experimental violations of Bell inequalities’,  and otheet al. , was usead verecundam; by authority. Perhaps the author’s work is merely blemished by details of notation or terminology. Fortunately for us, the author also presents computer simulations of his model. If that code faithfully represented his intended interpretation of his formulae, then by looking at the code one could deduce what he was trying to express in his published formulae. Moreover, one could check by inspection whether or not his code faithfully respects locality and realism. The simulations do seem to generate the results predicted by quantum mechanics. This means, in view of results (c) and/or (d), that it is not possible that the code satisfies the specifications agreed by Bell and the author.Computer simulations of Bell-type experiments have become very popular, and here too, the same statistical issues arise. They are popular both among opponents and among supporters of Bell. In the context of the author’s work, some readers might imagine that thanks to mathematical subtleties which they cannot hope to appreciate, the author’s work is still largely correct. An appeal to the Hurwitz theorem or to a mathematical version of Bell’s theorem is an argument It is indeed easy to check that the computer code effectively just draws the cosine curve built into the computer algebra package used by the author; it does not respect the constraints of local realism. It therefore weakens, rather than supports, his claims. His claims are unfounded, and the supporting evidence which he offers is actually further evidence that he has not succeeded in converting possible physical insight into hard science.. 2.1ab = ba, associativity is a(bc) = (ab)c). The useful and important thing to know here is that a division algebra has no zero divisors. There do not exist elements A and B, neither equal to zero, such that AB = 0. However, the author’s algebra has an element called the ‘pseudo-scalar’, I will denote it by M, such that M2 = 1. It follows that 0 = M2 − 1 = (M − 1)(M + 1). Taking norms, 0 = ‖M − 1‖.‖M + 1‖. Hence ‖M − 1‖ = 0 or ‖M + 1‖ = 0. Therefore M − 1 = 0 or M + 1 = 0, which implies that M = 1 or M = −1. That is a contradiction. The author’s algebra is associative, the octonions are not. It is known as The best reference for this theorem is Baez’s award-winning paper  on the osame algebra. The linear spans of those two bases are trivially the same. The multiplication operation is the same. The author’s claims regarding abstract algebra have naturally attracted the interest of algebraists. One of the founders of geometric algebra has published a paper, Lasenby [A curious elementary mathematical error is that he defines two algebras, built from two eight-dimensional real vector spaces  Lasenby , detaili. 2.2Xa and Yb are a family of random variables on a single probability space, taking values in the set {− 1, + 1}, and where a and b denote directions in ordinary three-dimensional Euclidean space, represented by unit vectors a, b. Then it is not possible that a and b.Bell , as rapia and two for b, delivering us four combinations of possible values of the pair . Since four binary random variables are supported by a discrete probability space with just 16 elementary outcomes, a proof  can be framed in absolutely elementary terms [Bell’s proof works by focussing on two choices for ry terms , which wBell used his mathematical result to argue that quantum mechanics violated the meta-physical principles of locality and realism. One can escape from this mathematical obstruction only by redefining the concepts of locality and realism. The author, however, does not take that well trodden route. He claims that the purely mathematical result is wrong. His alleged proof thereof is his explicit construction of a counterexample. I have elsewhere shown that his construction depends on simple errors in elementary algebra and calculus.He also argues that Bell’s proof contains a fundamental error in reasoning: the Bell-CHSH inequality involves correlations obtained from different sub-experiments involving measurements of non-commuting observables, and (he says) therefore cannot be combined. However, in quantum mechanics, even if two observables do not commute, a real linear combination of those observables is another observable. By the linearity encapsulated in the basic rules of quantum mechanics, expectation values of linear combinations of non-commuting observables are the same linear combination of the expectation values of each observable separately. If a local hidden variables model reproduces the statistical predictions of quantum mechanics, then it must reproduce this linearity.. 2.3Consider a computer simulation of a Bell-CHSH-type experiment. Initially, one could imagine three computers, a source computer sending information to two measurement locations, where two computers each simulate an apparatus which receives ‘stuff’ from a source, and a ‘setting’ (a measurement direction) supplied by an experimenter. The ‘measurement station computers’ each output an ‘outcome’ ±1. After a large number of trials, one collects the inputs (settings) and outputs (outcomes) together and computes the correlation  between the outcomes, for each possible pair of input settings.completely separated classical computers, which perform the following task.Now, if that could be done, one could also create a copy of the source measurement station, including all the data which is stored on it at the beginning of the computer simulation, and then merge each of the copies of the source with the two measurement stations, giving us together two N rounds, each computer is supplied an input, and each computer supplies an output. After the complete run of N ‘trials’ is completed, one collects all the data together, and correlates the outputs, for each possible pair of inputs. Gull [a · b? He outlines a proof that this is impossible using an argument from Fourier analysis. Unfortunately he never published a formal proof, so we must make do with his overhead transparencies from a conference. Inspection of Gull’s outline proof shows that he is making a particular ‘no use of memory’ assumption. Each of those computer’s outputs, in trial n, depends only on the initial data stored in the computers and their programs, and on the new setting a or b, and on the trial number n, but not on the previous n − 1 inputs. Full mathematical details can be found in Gill & Karakozak [The two computers are both loaded with data and a computer program, and then disconnected. After that, in ts. Gull  used to arakozak .if (lambda==1) q =(NA NB) else q =(NB NA). One may check the algebra embodied in the code: the program simply computes −a · b. It does not calculate the correlation between the actual ±1 valued measurement outcomes! The program does not implement the mathematical model given in the paper.The author’s paper contains computer code: just one program which simulates many times two measurement settings and the value of some hidden variables which one can imagine created by nature in the source and transmitted to two measurement stations. Measurement outcomes are computed at each measurement station from the relevant setting and from the hidden variables. Next the program goes on to compute the correlation between measurement outcomes for any pair of measurement settings. This is where things go wrong: notice the code line . 2.4nth pair of outcomes from the nth pair of settings might make use of information about the past n − 1 settings and outcomes. Using martingale theory I proved a probability bound showing that deviations from the Bell-CHSH inequality of any size had exponentially small probability, provided only that the binary setting choices at each trial were performed, outside of each measurement station, and again and again, by two fair coin tosses. The measurement station computers were even allowed to communicate with one another and with the source between each trial. This result was later refined and improved and used by all the experimenters in the four famous ‘loophole-free’ Bell experiments of 2015, starting with the Delft experiment [My paper, Gill , was my periment  and contperiment  for furt. 3S3, not of The paper Christian  is irrepClick here for additional data file.


Studies have found that long non-coding RNAs (lncRNAs) play important roles in many human biological processes, and it is critical to explore potential lncRNA–disease associations, especially cancer-associated lncRNAs. However, traditional biological experiments are costly and time-consuming, so it is of great significance to develop effective computational models. We developed a random walk algorithm with restart on multiplex and heterogeneous networks of lncRNAs and diseases to predict lncRNA–disease associations (MHRWRLDA). First, multiple disease similarity networks are constructed by using different approaches to calculate similarity scores between diseases, and multiple lncRNA similarity networks are also constructed by using different approaches to calculate similarity scores between lncRNAs. Then, a multiplex and heterogeneous network was constructed by integrating multiple disease similarity networks and multiple lncRNA similarity networks with the lncRNA–disease associations, and a random walk with restart on the multiplex and heterogeneous network was performed to predict lncRNA–disease associations. The results of Leave-One-Out cross-validation (LOOCV) showed that the value of Area under the curve (AUC) was 0.68736, which was improved compared with the classical algorithm in recent years. Finally, we confirmed a few novel predicted lncRNAs associated with specific diseases like colon cancer by literature mining. In summary, MHRWRLDA contributes to predict lncRNA–disease associations. Numerous studies have indicated that protein-coding genes accounted for less than 2% of the human genome . There aAt present, many machine learning methods have been proposed to predict the lncRNA–disease association, for example, Laplacian regulated least square method LRLSLDA; , propagaIn this study, a random walk algorithm with restart on multiplex and heterogeneous networks was developed. The downloaded known lncRNA–disease association data were used to calculate lncRNA functional similarity, lncRNA Gaussian interaction kernel similarity, disease semantic similarity, and disease Gaussian interaction kernel similarity, respectively. Then, these similarity networks and lncRNA–disease association network were constructed into multiplex and heterogeneous networks. A random walk with restart was carried out on the multiplex and heterogeneous networks, and the potential lncRNA–disease association was predicted using the final stable probability.LD to represent the lncRNA–disease association adjacency matrix. If lncRNA(i) is related to disease(j), then LD = 1; otherwise, LD = 0, that is:LncRNADisease , Lnc2CanDAGs) were used to calculate disease–disease similarity, for disease kd, let DAG, E(dk)) be its directed acyclic graph, where T(dk) are ancestor nodes of kd, and E(dk) represents the corresponding set of edges from parent node to child nodes. Semantic similarity of diseases was calculated by R package called DOSim , E(dk)), the semantic contribution of kto kd was defined as:Directed acyclic graphs  represents the ith column of LD. Next, the Gaussian Interaction Profile kernel similarity between id and jd was calculated as:In order to obtain the similarity information between diseases, the Gaussian Interaction Profile kernel similarity between disease was constructed based on the lncRNA–disease association network. First, the Interaction Profile (IP) of each disease represents a binary code in the known lncRNA–disease association network. For example, for given disease d represents the bandwidth that controls the Gaussian Interaction Profile kernel similarity, nd represents the number of diseases.Where γil and jl, Di = {kid|1 ≤ k ≤ m} and Dj = {ljd|1 ≤ l ≤ n} were disease sets associated with il and jl, respectively. The semantic similarity between disease d and disease set D was firstly defined as:Studies have shown that similar lncRNAs are usually associated with similar diseases. Therefore, lncRNA functional similarity can be roughly estimated by their similarity in related diseases . For anyil and jl was defined as:Then, the functional similarity between il and jl was:Similar to the disease Gaussian interaction profile kernel similarity. The formula for calculating the Gaussian interaction profile kernel similarity between l represents the bandwidth that controls the property similarity of Gaussian interaction kernel, nl represents the number of lncRNAs, IP(li) and IP(lj) represent the ith and jth row of the LD, respectively.Where γAn overview of MHRWRLDA is shown in ith node on the α layer. The set of disease nodes was defined as: jth node on the β layer. The adjacency matrix on each layer is:Based on disease semantic similarity network, disease Gaussian similarity network, lncRNA similarity network, and lncRNA Gaussian similarity network, we constructed a multiplex and heterogeneous network by using lncRNA–disease association. In these networks, the set of lncRNA nodes was defined as: A contains different types of jumps that the particle can follow at each step:A particle can either travel from the previous node I is the n × n identity matrix, the diagonal element of A represents the particle walking on same layer, the off-diagonal element represents the particle jumping between different layers, and the parameter δ ∈  represents the probability of the particle walking on the same layer or jumping between different layers. If δ = 0, the particles will always walk on the same layer.Where ARM(nL × nL), ADM(mK × mK) is the matrix of lncRNA similarity and disease similarity on multiplex and heterogeneous networks, respectively. n, L, m, and K are the number of lncRNAs, lncRNA similarity networks, diseases, and disease similarity networks, respectively, the adjacency matrix is: BMH = T.BMH is nL × mK, which is equivalent to replicating the adjacency matrix Bn × mL∗K times, where B = LD. Then, the adjacency matrix of the whole multiplex and heterogeneous networks is: The dimension of t ∈ ℕ, the particle is at node tv at the tth step. Then, it walks from tv to vt1. We defined a restart probability γ ∈ , and the random walk with restart can be defined as:A random walk with a restart means that a particle starts at a node and it is faced with two choices at each walk: move to a randomly selected neighbor node, or jump back to the start node. Considering the time is discrete, Pt1 and tP represent the probability distribution of tv and vt1, respectively. PRS is the initial probability distribution and PRS, where R0 and D0 represent the initial probability distribution of lncRNA similarity network and disease similarity network, respectively, and the dimensions of the vectors Pt+1, tP, and PRS are nL × mK. The parameter η ∈  controls the probability of each network restarting; if η < 0.5, the particle is more likely to be restarted in lncRNA similarity networks. HRR and HDD represent the transition probability of nodes upstream in the same layer, HRD and HDR represent the transition probability of node jump between different layers. For a given node, if dichotomous correlation exists, the particle can jump between layers or stay in the current layer with probability λ ∈ , and the closer it is to 1, the higher the probability of jumping between different networks.Where the vectors ri ∈ R. In the next step, the particle can walk to the node rj ∈ R. The transfer probability is:We suppose a particle was located at the node db ∈ D through binary correlation, and the transfer probability is:It can also jump to the node da ∈ D, then the transfer probability of the particle walking to the node db ∈ D is:Similarly, if the particle was located at the node rj ∈ R through binary correlation, then the transfer probability is:If the particle jumps to the node id, the node id will be used as the seed node in disease similarity networks. The initial probability D0 is 1 for the given node id and 0 for the remaining nodes. If there are known associations among lncRNAs r1, r2 ⋯ and disease id, then the nodes r1, r2 ⋯ are the seed nodes in lncRNA similarity networks. The initial probability R0 was assigned to seed node r1, r2 ⋯, with a probability of 1, and the remaining nodes were 0. tP converges after some iteration, that is, Pt − Pt + 1 < 10−10, and we denoted the stable probability as: When predicting lncRNAs that are potentially associated with the given disease R∞, those seed nodes r1, r2 ⋯ were removed, and the remaining lncRNAs were ranked. The higher the ranked lncRNA, the more likely it was to be associated with the given disease id. Similarly, a lncRNA can also be designated to predict diseases related to it.Based on the stabilized For a binary classification problem, the confusion matrix is shown in To evaluate the performance of MHRWRLDA, the receiver operating characteristic (ROC) curve was drawn by calculating TPR and FPR according to different thresholds. Area under the curve (AUC) is the area under the ROC curve, and this area is less than 1. Since the ROC curve cannot directly indicate which classifier has better effect in many cases, as a value, the larger the AUC is, the better the classifier has an effect.In order to evaluate the performance of MHRWRLDA for predicting lncRNA–disease association, we applied the known lncRNA–disease association data to MHRWRLDA, and used Leave-One-Out cross-validation (LOOCV) to verify. For global LOOCV, the scores of all test samples are compared with those of all candidate samples. For local LOOCV, each known lncRNA related to a particular disease is selected as the test sample, and other related lncRNAs are selected as the training samples; the scores of test samples are only compared with those of candidate samples. In this study, there are a total of three parameters, namely, γ, λ, and η, and their range is , where γ is the restart probability; λ is the jump probability, reflecting the probability of particles jumping between different networks; and η regulated the probability of each network restarting, When η = γ = 0.9 and λ = 0.9, the prediction effect is the best; at this point, AUC = 0.68736.The AUC based on global LOOCV of the KATZLDA , BPLLDA To further explore the performance of MHRWRLDA in predicting lncRNA–disease association, we selected colon cancer, hepatocellular carcinoma, and breast cancer for the case study. During the experiment, all known associations were considered as the train set, and unknown associations were regarded as the test set. According to LOOCV results, we sorted lncRNAs and selected the top 10 lncRNAs for further verification based on the LncRNADisease database and several recently published studies.Colon cancer is a malignant tumor, causing nearly 700,000 deaths each year, and has a high incidence rate record in developed countries. We applied MHRWRLDA to colon cancer experiments to predict the top 10 lncRNAs related to colon cancer . Seven oHepatocellular carcinoma is one of the most common cancers in the world. Studies have shown that hepatocellular carcinoma is the main component of primary liver cancer. We listed the top 10 lncRNAs related to hepatocellular carcinoma predicted by experiments in Breast cancer accounts for 22% of all cancers in women and is the second leading cause of cancer death in women . TraditiFinally, the network of three cases and lncRNAs predicted by MHRWRLDA is shown in In recent years, the research on the interaction between biomolecules has been growing. Due to the importance of lncRNA, the research on the associations between lncRNAs and diseases has been paid more and more attention. These associations can be characterized by complex networks, so it is urgent to develop network-based computational algorithms to explore functional associations between lncRNAs and diseases. The algorithm of constructing heterogeneous network and implementing random walk on heterogeneous network is widely used in the field of bioinformatics. However, in previous studies, most of them are single heterogeneous networks with a single information source. Therefore, we consider multiple network embedding by integrating different types of edges. Multiplex and heterogeneous networks are the combination of heterogeneous networks connected by multiple interactions; they integrate the framework of multiple information sources, and each layer is a simplex network with specific types of nodes and edges; when the data set is large, they can produce better results. Multiple heterostructures may provide a richer perspective for the study of the complex relationship between different biological components.In this study, we extend it to multi-layer heterogeneous networks so as to more effectively predict lncRNA–disease associations. We constitute a multiplex and heterogeneous network by integrating known lncRNA–disease association, lncRNA function similarity, lncRNA Gaussian similarity network, disease semantic similarity network, and disease Gaussian similarity network, and then we generate the final comprehensive predictive scores by the random walk with restart on the multiplex and heterogeneous network, so as to forecast potential lncRNA–disease associations. LOOCV experimental verification results showed that the AUC was 0.68736, which exceeded other algorithms to predict lncRNA–disease association. In novel diseases, the top 10 lncRNAs were verified and predicted by database or literature. In addition, the model can also predict diseases associated with particular lncRNAs.The network-based approach overcomes the disadvantage of machine learning methods that need to construct negative samples and not only is suitable for predicting lncRNA–disease associations, but also proved to be widely used in exploring disease-related miRNAs, drug repositioning, and prediction of disease–gene associations. Therefore, if the known lncRNA–disease association data are replaced with miRNA-disease association data, MHRWRLDA can be used to predict the potential miRNAs associated with disease; similarly, if it is replaced by drug–disease association data or gene–disease association data, it is possible to make contributions to drug repositioning and the exploration of disease-related genes, respectively. In the future, we will try to apply MHRWRLDA to the above aspects for research.However, there are some limitations. First, there are only two methods for constructing the similarity network; if the calculation method of the similarity network can be increased, the number of layers in the multi-layer heterogeneous graph can be increased to provide more possibilities for particle migration. Second, the lncRNA–disease association data contain only 117 lncRNAs and 159 diseases, of which there are only 285 pairs of correlations; a small data set may also affect the prediction results. In the future, more association data will be discovered and used to overcome the difficulties caused by the complexity and inconsistency of biological data. In addition, efforts will be made to combine multiple prediction models to achieve more accurate predictions.https://github.com/jibinbin171222/MHRWRLDA.The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: WG conceived, designed, and managed the study. YY and BJ designed the method and wrote the original manuscript. YL and LL revised the original draft. JX wrote the code. BL discussed the proposed method and gave further research. All authors read and approved the final manuscript.BJ was employed by Geneis Beijing Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The handling editor declared a past co-authorship with one of the authors JX.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.