Pitch peaks tend to be higher at the beginning of longer than shorter sentences , whereas pitch valleys at the ends of sentences are rather constant for a given speaker. These data seem to imply that speakers avoid dropping their voice pitch too low by planning the height of sentence-initial pitch peaks prior to speaking. However, the length effect on sentence-initial pitch peaks appears to vary across different types of sentences, speakers and languages. Therefore, the notion that speakers plan sentence intonation in advance due to the limitations in low voice pitch leaves part of the data unexplained. Consequently, this study suggests a complementary cognitive account of length-dependent pitch scaling. In particular, it proposes that the sentence-initial pitch raise in long sentences is related to high demands on mental resources during the early stages of sentence planning. To tap into the cognitive underpinnings of planning sentence intonation, this study adopts the methodology of recording eye movements during a picture description task, as the eye movements are the established approximation of the real-time planning processes. Measures of voice pitch  and incrementality (eye movements) are used to examine the relationship between  working memory (WM), incrementality of sentence planning and the height of sentence-initial pitch peaks. Spoken sentences are characterized by sentence intonation, i.e., modulations of voice pitch. The deviations in voice pitch or, acoustically, changes in Fundamental Frequency (F0) are intentionally controlled  and dThis study proposes and investigates the scope of  working memory (WM) in planning spontaneously spoken sentences and sentence intonation. It hypothesizes that sentence-initial pitch peaks depend on the availability of mental resources in the early planning stages. To test the role of WM, the study adopts a dual-task design by combining the picture description task with a recall task where speakers are asked to memorize a list of unrelated nouns before they describe the pictures and recall the words after they have produced the picture description. For investigating the real-time planning of sentences, the experiment records speakers’ eye movements during the picture description task. As such, the study builds on the interference effects known from the study of WM , 15 and Producing a spoken sentence that is linked to previous context and listeners’ expectations involves a number of cognitively demanding mental operations related to language planning for production. The production models posit that planning for production proceeds in three stages: conceptualization, linguistic encoding  and phonological encoding  –25. By rThe existing studies of the relationship between the language production and mental resources in terms of WM indicate that limited mental resources influence the three planning stages to varying degrees , 29–32. While some studies attempt to investigate the impact of cognitive load on formulation and phonological encoding processes, little research has investigated the relationship between WM and conceptual encoding processes. The existing studies indicate that the incrementality of formulation processes does not vary as a function of increasing cognitive load . FoPhonetic research on planning sentence intonation has focussed on the time-related changes in Fundamental Frequency (F0). Namely, the research shows that in declarative sentences, the relative height of F0 maxima  usually declines as a function of time –9. Thus,The fact that sentence intonation must be planned prior to speaking is substantiated by an observation that the lowest F0 at the end of utterances is rather consistent for a given speaker, indicating limitations in the ability to manipulate the lower register of voice pitch , 8, 10. Nevertheless, some studies show that the effect of length on sentence-initial pitch peaks may also be absent , 11, 12.Research shows that sentence intonation is sensitive to mental challenges in various metrics indexing loudness, voice quality , speech rate and sentence intonation –61. For This study aims to investigate the relationship between WM and language production by examining cognitive mechanisms underlying the length-dependent pitch raising in spontaneous speech. It suggests that the relative pitch height at the beginnings of long sentences is related to the extent of mental load involved in the speech production task.The proposal is that the length effect on sentence-initial intonation peaks indicates mental effort that occurs only when the conceptual or higher-level planning processes are weakly incremental or even non-incremental. The non-incremental conceptual processing entails that a larger network of concepts and relationships between them need to be activated and maintained in the current WM. Relatedly, the number of concepts in long sentences is greater than in short sentences, e.g., consider sentences ‘A farmer is pulling a donkey’ vs ‘A farmer is pulling a donkey and a goat’. Thus, in case of less incremental conceptual processing, the chunk of information concurrently maintained in the WM at the time of manipulating the articulatory information of the first-mentioned word is larger in longer sentences than in shorter sentences. As a result, the sentence-initial pitch peaks might become high as an unintentional (or planned) response to the information load in WM during the production of long sentences. Conversely, pitch is not raised when sentences are short and the number of concepts maintained in WM is rather small.To demonstrate the impact of WM on the length-dependent pitch raising, this study manipulates the degree of cognitive load with an aim to reduce the number of activated concepts in WM. In particular, the number of concepts is expected to decrease when the cognitive load in the speech production task is high. During the early planning of incipient sentences, the decrease of concepts corresponds to the increase in incrementality of conceptual processing. Thus, the study investigates the interaction between the increasing mental load and the incremental conceptualization processes in advance planning of sentence intonation.We devised an eye-tracking experiment, consisting of two blocks, to verify the proposed cognitive account of advance planning of sentence intonation. The aim of the first block is to establish the tonal differences between short and long sentences when no additional cognitive load is imposed on speech production. For this block, participants are asked to describe monochromatic line drawings depicting an action involving three actors . EitherThe man is pulling the donkey’ vs ‘The man and the woman are pulling the donkey’), but to be absent between the sentences containing short or long sentence-final noun phrases  [The concrete task setting ensures that the picture descriptions vary between short and long . As the length effect is sometimes reported to occur between the sentences consisting of short or long sentence-initial subject noun phrases  , the proe goat’) , 65.For the investigation of the scope of WM, the second block of the eye-tracking experiment will follow the standard dual-task design, in which speakers are first asked to memorize a list of words, then describe a picture of simple transitive actions, and finally to recall the memorized list of words , 35, 66.Contrary to earlier investigations , 15, theSecond, earlier investigations often train speakers to use certain lexical items  , 15. NatThe experiment records speech time-locked to the eye movements in a picture description task to overcome the limitations of measuring speech onset latency. Past research has established this method as reliable for assessing real-time language planning for production , 20, 69.Importantly, research shows that the proportions of eye fixations varying as a function of time in this early time window of picture processing are highly sensitive to the linguistic features of incipient utterances. For instance, the proportions of these early eye fixations appear to depend on the word order , 69, preThe eye movements in the existing studies are typically investigated as continuously changing functions of time in a fixed time window (from 0 to 400/600/1000 ms of picture presentation) , 69, 70.Namely, the time before the naming gaze indexes the time window where the conceptual frame of an incipient sentence is generated. The time after this longest gaze indicates encoding of further aspects of the incipient sentence (possibly the verb or second-mentioned actor). The highly incremental processing entails that after the picture onset, speakers start to encode the first-mentioned actors as soon as possible , 36, 69.This study investigates the impact of WM on conceptual processing and advance planning of sentence intonation based on Estonian. Estonian presents a good test bed for investigating declining intonation contours for three main reasons. First, many researchers have noted that Estonian sentence melody is mostly falling (overview in ). In theIn line with earlier studies, active sentences uttered in response to complex visual scenes are assumed to be produced less rather than more incrementally , 65, 76.As argued in the previous paragraph, sentence planning is expected to proceed weakly incrementally or even non-incrementally under low cognitive load. Accordingly, tonal variation is expected to be determined by the number of concepts maintained in the WM or by sentence length such that sentence-initial intonation peaks are higher in longer than shorter sentences. The high cognitive load and the incrementality of planning are predicted to be tightly interrelated with the advance planning of sentence intonation. First, the incrementality is expected to relate to low pitch peaks because the highly incremental conceptualization processes only foresee mentioning the first concept in subsequent utterances. Second, the cognitive load is predicted to increase the sentence-initial pitch peaks given that the mental load arising from maintaining a complex conceptual network in WM (as it may occur whilst planning long but not short sentences) is comparable to the mental load that emerges from memorizing a word list. Consequently, two different outcomes of the load manipulation on sentence-initial intonation peaks are possible see .First, regardless of sentence length, the pitch peaks could decrease to a level lower than the pitch peaks in short sentences under the low cognitive load. In this scenario, the high cognitive load affects short as well as long sentences, and the effect of high cognitive load is greater on long sentences than short sentences. Second, the effects of incrementality and the high cognitive load could accumulate such that, regardless of sentence length, the pitch peaks under the high cognitive load are about the same height as the pitch peaks in the short sentences under the low cognitive load. This scenario predicts the high cognitive load to affect the pitch peaks in long sentences but not short sentences.A picture description experiment was designed to examine the effect of cognitive load on the incrementality of conceptual planning processes, and the effects of cognitive load and the resulting incremental planning processes on sentence-initial intonation peaks. The design and analysis methods are registered prior to collecting data and the Ethics Commission at the University of Tartu approved the experiment (3271T-27).The materials consist of pictures showing transitive events and lists of three unrelated nouns and verbs for the verb probe task. The pictures depict three actors, two of whom share a thematic role. The two core roles of transitive events typically include i) an initiator or agent, and ii) a undergoer of the action or patient. The two thematic roles are distributed amongst the three pictured actors such that either two agents act upon a single patient  or a sinThe aim of this fully crossed two-factorial design is to elicit transitive sentences that differ in length, such that the similar double actor condition elicits short sentences and the different double actor condition triggers speakers to produce long sentences. The concrete task setting will ensure that the picture description will indeed be either short or long. In particular, participants are asked to describe the pictures by mentioning all three actors. Moreover, they are trained to use either plural NPs for the actors sharing the identity  and coordinated NPs for the actors differing in the identity .The complex two-by-two design is necessary for the manipulation of the length effect at the two syntactic positions. This means that either the sentential subjects of prospective sentences are short or long , or the sentential objects vary between short and long . This design allows researchers to control for the possible interactions between the prosodic phrasing and F0 declination , and obsAltogether, 64 different pictures of simple events (target items) are created for the study . Another set of 84 pictures will serve as distractor items. The distractor items will depict a variety of transitive and intransitive events comprising one to four identifiable actors. Half of the pictorial items are combined with a verb probe task separated into a second experimental block.For the verb probe task, two sets of verbs are selected based on their collocation frequencies in the Estonian National Corpus . The setThe memory lists of three nouns are constructed such that they contain a noun that refers to an animate referent and two nouns that refer to inanimate referents. Notably, the variation of animacy in to-be-memorized word lists is regarded as the conceptual aspect of intended messages. The recall task builds on the semantics of verbs. In particular, the lists of nouns  are selected such that all nouns are likely to occur with transitive verbs  but only nouns referring to animate referents are conventional with intransitive verbs . Half of the target trials in the second block will be prompted together with transitive verbs and the other half of the trials with intransitive verbs. The correct answers of the verb probe task  have been confirmed in a pre-test with a different sample of native Estonian speakers. To balance the factor of verb type, 50% of distractor trials will be presented with nouns that are incompatible with the transitive verb prompt. Conversely, the other half of the distractor trials will be prompted together with intransitive verbs that are compatible with all of the nouns on a list. Due to the limitations of time and financial resources, the lists and prompts that are combined with distractor items are selected based on the corpus statistics and will not all be pre-tested.Finally, another set of 12 pictures  is comprised of 6 pictures similar to the pictures in target trials and of another 6 pictures similar to the pictures in distractor trials. Similarly, 12 additional pairs of nouns and a verb are constructed for the practice trials.The two visual factors Double Actor and Identity, as well as the factor Cognitive Load, all comprising two levels, will be fully crossed, resulting in eight conditions in total. The two-level fixed effect Cognitive Load reflects whether the picture was described while maintaining a list of nouns in WM or not (levels low and high). The trials with no load constitute the first block of the experiment (32 target pictures) and the trials with the load (32 target pictures) constitute the second block.Four lists of target trials are created by distributing the 256 target pictures between the four conditions (Double Actor crossed with Identity) using the Latin square approach. Target pictures in lists are distributed between the same set of 84 distractor pictures such that there are one or two distractor items for every target item, resulting in 148 experimental trials per list in total. Another set of four lists is created where the leftward placement of agents is reversed, resulting in eight lists. For the verb probe task, all eight lists are reversed for a further set of eight lists, amounting to 16 lists altogether. In all 16 lists, the second half of the trials include the verb probe task.Participants are presented with all eight conditions across 64 target pictures . The within-subject design is necessary because F0 tends to be highly variable and affected by many other, also non-linguistic, factors. The tonal differences between the conditions can most reliably occur within a speaker, ideally also within the items of the same speaker. Due to this practical limitation, most of the classical intonation research typically records all conditions for a speaker .Study subjects will be recruited from the university’s students and staff through informational mailing lists at the University of Tartu in Estonia. They must be native speakers of Estonian, and between 18 and 50 years old. The upper age limit is applied because the WM capacities are known to decay in ageing speakers (see in ) and duea priori power analysis. The PANGEA analysis (variation parameters set to PANGEA defaults) indicates that, given the small to medium effect size (d = 0.45), the recommended sample size of 60 subjects is suitable for achieving the statistical power (1-β) of 0.8 in the evaluation of the three main effects and the three-way interaction between the fixed effects. Due to the spontaneous nature of the task, not all of the subjects will perform expectedly on all target trials. Therefore, the number of subjects is increased by approximately 33% of observations. Therefore, the resulting sample size is 84 subjects. The estimated sample size is at the limit of the feasibility of planned phonetic analyses and the project’s timeline.Since the experiment is designed such that samples of subjects are going to respond to samples of target trials, PANGEA web application . Otherwise, the subjects will be thanked for their interest. Together with an invitation letter, subjects will also receive a letter reminding them about their responsibility not to spread any infections, especially COVID-19, and to respect the hygiene rules in the testing rooms.Upon arriving at the recording room at the University of Tartu, subjects are asked to fill out and sign a consent form informing them about the risks of the study and their right to stop participating in the experiment at any time without any consequences. They will be introduced to the eye tracking method that is carried out with synchronous speech recording.After these introductory parts, the subjects are asked to seat themselves in front of the desktop-mounted EyeLink 1000 Plus eye tracker and lean their forehead on the head support. The chin is left unsupported because the study requires subjects to speak.Subjects will be semi-randomly assigned to the 16 lists of experimental trials such that each list will be seen five or six times. The order of the items within a block (low vs high load) is randomized. The items are subdivided into the blocks such that there is the least possibility for semantic overlap between the target and distractor items.Once the calibration procedures are completed successfully, subjects receive a thorough explanation of their task by seeing a sample of five pictures together with a possible written description in the bottom part of the screen. They will be given an opportunity to ask as many clarification questions as needed.The subjects’ task is to describe simple actions by mentioning all three depicted actors using either plural or coordinated NPs. Every picture is presented for 6500 ms . The spePictures are presented contingently to gaze. This means that a picture is only shown once the eyes move to the dot in the upper part of the screen. The gaze-contingent presentation ensures that subjects will always look at the dot in the upper part of the screen before seeing the picture. This procedure guarantees that all speakers begin processing of visual information at approximately the same time in all trials. Apart from the drift check, the experiment can proceed without any assistance or control from the experimenter and be interrupted at any point either for a break or quitting.A break is enforced half-way through the trials to stimulate concentration on the task and to introduce the second block with the trials that are accompanied with the memorization task. First, subjects are presented with 6 practice trials with a task to memorize three words in their exact order and then after 2500 ms, to orally recall the memorized words and then indicate whether all of the memorized words are conventional with the verb on the screen or not. In particular, their task is to say if all of the words fit with the verb on the screen or not by either saying ‘Kõik sobivad’  or ‘Ei sobi’ . In the target trials, a correct response to a transitive verb prompt is that all words fit with the verb on the screen. For example, a ‘tradition’ or ‘search’ from the example in the Materials section  can be ‘continued’ and an ‘investor’ can ‘continue (with)’ something. The correct response to the intransitive verbs is that not all of the words fit. Consider the verb ‘worry’ paired with the nouns ‘tradition’, ‘search’ and ‘investor’ as an example. A ‘tradition’ or ‘search’ cannot ‘worry’ but an ‘investor’ can. Indeed, it is possible to worry about a tradition or search. However, within this particular task setting, this did not occur to our sample of the norming study because the accuracy scores for this word list was 1.0 when presented with the transitive verb ‘continue’ and 0.83 when presented with the intransitive verb ‘worry’. The target items are combined with pairs of nouns and a verb with the highest accuracy scores see .The 6 memory trials are followed with 6 additional practice trials, in which memorization and recall are interposed by a task of picture description. Thereafter, the target and distractor pictures are always preceded by the list of three nouns in the middle of the screen for 2500 ms . SubjectAfter completing the experiment, subjects are debriefed about the goals of the study, related research and a short list of literature. Together with the debriefing letter, they will receive contact details for future concerns and questions. Upon signing the receipt, they are compensated with 5 euros.Any personal details collected from subjects during recruiting and experimentation will be stored encrypted for a limited time frame in a password-protected database. All data recorded during the experimentation will be anonymised and analysed separately from the personal details.First, starts and ends of utterances will be detected automatically based on the energy levels in the recorded signals. All of the starts and ends are then manually checked in the phonetic analysis software Praat , TogetheF0 of each sentence (in hertz) is extracted with the help of the autocorrelation method available in Praat in two passes. During the first pass, F0 tracks are extracted with default settings for the lowest and highest F0, the “floor” and “ceiling” . Then, the first and third quartiles of F0 (Q1 and Q3) are calculated for each speaker and recorded in a table. In the second pass, F0 contours are extracted with speaker-specific settings (0.75*Q1 for floor and 1.5*Q3 for ceiling). The F0 samples making up a contour are saved in the two-column format where the first column contains timestamps and the second column F0 measures at given times.The timing and position of eye fixations is recorded relative to the regions of depicted actors, the so-called areas of interest (AOI). The AOIs are determined on the basis of the thematic roles. This means that AOIs are defined to cover the areas of agents and patients such that the actors sharing a thematic role are enclosed into a single AOI.For linguistic scoring, all sentences will be tagged for a number of linguistic parameters including the syntactic structure, number of actors mentioned, names of the actors, correspondence to the intended actors and mentions of any additional noun phrases or sentence modifiers. In addition, the length of sentences and syllable scores of sentence constituents and actor names are extracted from the phonetic annotations contained in the TextGrids.A number of exclusion criteria are defined based on the eye movements, linguistic parameters, speech onset latencies and the accuracy of the verb probe task. These exclusions are necessary for the comparability between the sentences because the conceptual planning processes during language production are tightly related to speech timing and the syntactic structures employed. Therefore, target trials will be excluded when the first fixation to the AOI occurs later than 400 ms after picture onset, or when the time delay between the two consecutive fixations is longer than 600 ms (indication of track loss). Picture descriptions of a subject will be included only when (s)he is able to remember at least one word correctly in 90% of the load trials. The final analyses will only include fluent transitive sentences. After these exclusions, additional trials will be excluded based on speech onset latencies later than three standard deviations above the grand mean. Finally, all the data from the subject will be removed when the subject’s inclusion rate after the indicated exclusions is lower than 25%. No treatment of missing data is necessary because the planned regression modelling method (LMER) –84 is roThe dependent variables of interest are (i) speech onset latencies, (ii) duration of naming gazes, (iii) duration of naming gazes proportional to their respective speech onset latencies, (iv) the height of sentence-initial pitch peaks as indexed by F0 maxima (v) and the height of sentence-final pitch valleys as indexed by F0 minima.First, the starts of utterances (in milliseconds) index the speech onset latencies. Second, a gaze is defined as multiple consecutive fixations directed at the AOI . Gaze duF0mean is the subject’s mean F0 aggregated over all F0 contours extracted from all of the subject’s utterances. This method of converting F0 into semitones has the benefit of partly cancelling out the large-scale, gender-based differences in voice pitch . The effect of sentence length on sentence-initial pitch peaks is expected to be smaller than the gender effect. When gender variation in voice pitch is included in the data, the critical effects of sentence length and WM may remain unrevealed.Finally, F0 maxima and F0 minima are automatically extracted within the boundaries of agent and patient names and the boundaries of sentence-final words in cases where patient names do not occur at the end of a sentence  follows the patient name). F0 maxima and minima measured in hertz (Hz) will be converted into semitones relative to speaker means based on The goal of the statistical analyses is to determine whether the scaling of sentence-initial pitch peaks in long sentences is affected by an independent effect of incrementality or by the simultaneous effects of incrementality and high mental load. As the manipulations of the factors Double Actor (agent vs patient) and Identity (same vs different) elicit short and long sentences , the combinations of these are reduced into a factor Sentence Length (short vs long) for the main analysis. Thus, the main analysis examines the effect of the interaction between Cognitive Load (low vs high) and Sentence Length (short vs long) on (i) the proportional duration of naming gazes at the first-mentioned actors, (ii) duration of naming gazes, (iii) the sentence-final pitch valleys, (iv) and the sentence-initial pitch peaks.lme4 package in [lmerTest package [The four dependent variables  are modelled with separate linear mixed-effects regression analyses . Alckage in . First,  package  in R [85The first analysis tests the extent to which the proposed verb probe task tapped the conceptual retention. The proportional duration of naming gazes at the first-mentioned actors is assumed to increase under the high cognitive load. Thus, only the main effect of cognitive load is expected in this analysis. The second analysis investigates the absolute duration of naming gazes, as the interference effect may arise at the phonological planning stage (or semantico-syntactic encoding) instead. In particular, if the phonological encoding of the first-mentioned actors is disrupted due to the memorization of three words the duration of naming gazes right before the speech onsets should increase under the high cognitive load.The third analysis checks the preliminaries of advance planning of sentence intonation. Namely, the aim is to test how constant the sentence-final low F0 is in the elicited sentences. For this, the statistical evaluation should find no effect of Sentence Length. Nevertheless, the sentence-final low F0 may increase under the high cognitive load, as the existing studies have reported generally higher and flatter F0 contours for speech under mentally challenging conditions –64.Finally, the fourth analysis tests the effect of cognitive load on the scaling of sentence-initial pitch peaks. The significant interaction between Cognitive Load and Sentence Length should demonstrate that the degree of length-dependent pitch raise in sentence-initial pitch peaks depends on the cognitive load. If the cognitive load causes sentence planning processes to proceed incrementally and the incrementality of planning processes mainly bears on the scaling of the pitch peaks, then the sentence-initial pitch peaks are lower under the high load than under the low load, and the difference between the high and low load is greater in long sentences than in short sentences. If the incrementality and the high load both play a role in pitch scaling, then the tonal difference between the high and low cognitive load is expected to occur only in long sentences.The exploratory analyses inspect the three-way interaction between the factors Double Actor (agent vs patient), Identity (same vs different) and Load (low vs high) on (i) the proportional duration of naming gazes at the first-mentioned actors (ii) and the sentence-initial pitch peaks. The first exploratory analysis of naming gazes aims to determine whether the pictorial scenes combining the similar and differing event participants either in agent or patient role would induce planning increments differing and gradually increasing in size. Known from the studies relying on the grouping effect (though with moving displays), the chunk conceptualized and planned before the speech onset increases together with the more complex phrase structures , 38. TheAll data except the speech recordings will be made available on the Open Science Foundation’s data sharing platform. As the voice characteristics constitute identifying data, the speech recordings will remain unpublished. Nevertheless, the database will provide the records of eye movements (raw and processed), TextGrids with acoustic analyses, tables with F0 samples, spreadsheets with the linguistic parameters of elicited sentences and the analysis scripts.We aim to begin collecting data in July 2021 and finish in October 2021. Student assistants from Tartu (Estonia) will help to conduct the acoustic analyses. Given the amount of manual work that speech recordings require, the data analyses can be completed by January 2023 at the earliest. Following the completion of data collection, the data analyses and discussions will be finalized and documented by March 2022.S1 TableTwo experiments were necessary because the first experiment did not deliver enough suitable items for the main experiment.(TEX)Click here for additional data file.


With the outbreak of COVID-19 exerting a strong pressure on hospitals and health facilities, clinical decision support systems based on predictive models can help to effectively improve the management of the pandemic. We present a method for predicting mortality for COVID-19 patients. Starting from a large number of clinical variables, we select six of them with largest predictive power, using a feature selection method based on genetic algorithms and starting from a set of COVID-19 patients from the first wave. The algorithm is designed to reduce the impact of missing values in the set of variables measured, and consider only variables that show good accuracy on validation data. The final predictive model provides accuracy larger than 85% on test data, including a new patient cohort from the second COVID-19 wave, and on patients with imputed missing values. The selected clinical variables are confirmed to be relevant by recent literature on COVID-19. Hospitals and health facilities are under strong pressure due to limited resources compared to the patient affluence. This is particularly true during the local peaks of the pandemic, when in some regions Intensive Care Units and Emergency Departments (ED) have reached their maximum capacity. Under these conditions, clinical decision support systems based on predictive analytics can help to effectively improve the management of the emergency. In particular, early detection of COVID-19 patients who are likely to develop critical illness and death can aid in delivering suitable care and in optimizing the use of limited resources.The Coronavirus Disease 19 (COVID-19) has become a worldwide pandemic, causing by the end of December 2020 over 70 million confirmed cases and over 1.5 million deaths2, sometimes followed by a systemic response with several other complications, including thrombotic, cardiac, kidney, gastrointestinal, and neurological effects3. Numerous clinical studies, including meta-analyses4, characterise the disease on different patient cohorts5, identify comorbidities and risk factors7, and study the effect of different therapy approaches tested around the world8. The detailed mechanisms of the disease still remain to be uncovered, hence the evaluation of risk at hospitalisation time is difficult to perform, especially in patients with several risk factors. At the same time, while vaccines are currently available, it will take some time before they can be administered to enough individuals to achieve proper population coverage10. At the same time, virus variants continue to appear, while the best therapy is still to be confirmed11.COVID-19 typically manifests as a respiratory pathology, with a wide range of severity and clinical presentation. A considerable fraction of patients show mild symptoms or are asymptomatic. Many patients develop acute respiratory distress syndrome13. In particular, a large amount of work has been performed in trying to distinguish between COVID-19 and other pneumonia, or assessing severity, by employing classification with machine learning methods13. Diagnostic tools based on clinical data were also developed16. A different category of methods is using predictive models for prognostic studies based on clinical data, with a large number of models appearing in the last months, e.g.23. Recent reviews25 show that most of these models are not mature enough to be used for decision making, and require further study.Predictive models and artificial-intelligence-based technologies can be employed to assist the risk evaluation procedure and have already been employed for various COVID-19 prediction tasks. These include analysis of imaging data, where lung X-Ray or CT scans can be automatically analysed to guide and improve COVID-19 diagnostic and prognosisWe present a study that predicts clinical outcomes based on clinical data. The objective is to classify patients into two classes according to the final outcome: discharged or deceased. We start with a cohort of 265 COVID-19 patients hospitalised during the first COVID-19 wave at the Pisa University Hospital, in Italy, including 85 clinical variables measured at the time of admission to the ED. We employ a hybrid filter/wrapper feature selection algorithm to select 6 clinical predictive variables, namely: age, P/F ratio, troponin level, blood urea nitrogen (BUN) level, presence of chronic obstructive pulmonary disease (COPD) and presence of myalgia. These variables are validated with exiting literature, and employed to build predictive models based on logistic regression (LR), decision trees (DT), random forests (RF), naive bayes (NB) and support vector machines (SVM). We perform cross validation on all the patients that have those variables measured (185 patients). We also compare our method with standard filter feature selection. We then validate the selected features on a new dataset with 387 patients hospitalised in the same hospital units during the second COVID-19 wave. Further validation is performed on an extended first wave dataset, where all patients are considered, and missing data is imputed using a nearest neighbours approach. Our selected variables allow for good performance on both the first and second wave datasets: accuracy up to 90% on the first and 86% on the second wave data, and F1-score up to 0.89 on the first and 0.85 on the second wave data. Further interpretability is provided by studying the logistic regression and decision tree classification models and clustering of clinical variables, in order to validate the models from the medical perspective.Our analysis includes two datasets covering patients hospitalised in three different units of the Pisa University Hospital , during the first and second wave of COVId-19. The data was manually curated, combining both paper and electronic records from the three different units. The dataset contains over 125 clinical variables, some of which are measured several times during hospitalisation. These include: (1) comorbidities, (2) clinical symptoms, (3) physiological variables , (4) blood analysis, (5) current therapies. Available outcomes include final outcomes—dismissal at home or death—and intermediate outcomes—ICU admission, duration of hospitalisation, invasive ventilation and other complications.first wave dataset covers 313 patients admitted between the 3rd of March 2020 and the 30th of April 202026. We selected the variables measured at the moment of hospitalisation (85 variables) and the patients for which the final outcome is known, discharged or dismissed (265 patients). The rest of the patients were transferred to other units so the final outcome is not available in our data. For the variables measured several times during the hospitalisation period, if the measurement was missing in the first day of hospitalisation, we considered the measurement taken during the second day. This is an approximation but we expect that the values did not change so much from one day to another, and in this way we decreased the number of missing values for these variables. Among the 265 patients, 194 survived and were discharged. The median age across all patients considered is 69, the youngest patient is 18 and the oldest 98. A large part of the patients are male (180 out of 265).The second wave dataset includes 510 patients admitted between the 3rd of September and the 24th of December 2020. Among these, 387 patients were discharged or deceased, while the rest were transferred so the outcome is unknown. Out of 387, 296 survived and were discharged. The median age across the 387 patients considered is 71, the youngest patient is 23 and the oldest 91. Again, a large part of the patients are male . This dataset was employed for validation of selected clinical variables, hence only patients with those variables measured were extracted  and logistic regression, that we describe below in "The analysis has two closely interconnected objectives. The first objective is to select a set of clinical variables that have predictive power when it comes to final clinical outcomes, and can be used for clinical decision making. The selection of clinical variables is guided by an automatic Further validation of the selected variables and predictive models is performed by missing value imputation with a nearest-neighbour approach described in "The genetic algorithm (GA) feature selection method described in "28, and our algorithm correctly identifies it. The P/F ratio is an important variable with lower values indicating worse respiratory function. Given that the respiratory system is one of the main systems affected by the progression of COVID-19, P/F has been used in clinical decision making for external ventilation and oxygenation of patients30. Another respiratory variable we select is COPD, which flags the patients having chronic lung disease. Again, this co-morbidity has been previously found to increase the risk of a poor outcome in COVID-19 patients33.Figure 37. On the other hand, the pre-existence of cardiovascular pathology could be responsible for a transient increase of this value, as a result of a greater cardiac susceptibility in some COVID-19 patients. In our dataset, troponin is a clinical variable that is missing in 57 out of the 265 patients, and it will not be selected under some settings of the algorithm . Even so, the variable is at the top of the rankings, underlining its role in the disease.Another important aspect related to multi-organ failure and co-morbidities that our selection algorithm is able to recognise is the link between COVID-19 and the cardiovascular system, through the selection of the troponin clinical variable. The correlation between mortality and elevated troponin values could highlight acute organ damage, already suggested in the literature with acute myocarditis39.Another clinical variable selected by our automatic method is BUN. Elevated BUN levels can be linked to a pre-existing chronic kidney disease but also to a situation of hypovolemia and hypoperfusion determined by the septic state. It could therefore be interpreted as a marker of infection severity and related mechanisms. The link between BUN and mortality has been observed in COVID-19 patients also in other studies4). The identification of “protective” factors from negative outcomes of COVID infection, such as myalgia, could suggest that the disease could develop against different target organs, depending on the case. When the target organ is the lung, the disease is expressed with the most fearful respiratory manifestation. Instead, in other cases, the disease could refer to other districts, manifesting itself in a mild form.The last variable selected by our algorithm is myalgia. Presence of myalgia appears to be an advantage in the case of COVID-19, with lower incidence among the deceased patients compared to those discharged 41. In this work we use a hybrid filter/wrapper feature selection method based on Genetic Algorithms, which dynamically adjusts the patient cohort based on the clinical variables selected. The final result is a ranking of clinical variables based on their predictive power. The resulting ranking is very robust to changes in parameters of the algorithm or in the subset of patients included in the selection phase. The Supplementary Material includes a discussion of robustness, with the full variable ranking for five different patient subsets, where we observe great similarity at the top of the rankings. This gives us confidence that, even if the number of patients in our data is not large, the selected variables are generic and do not simply describe the cohort we are using. The relevance of the variables we select is reinforced by the clinical value they carry, and the amount of literature supporting them.A second challenge is that the set of measured clinical variables differs strongly from patient to patient. This could be due to different approaches of the medical personnel involved, cost restrictions, patient needs and status. A critical patient admitted directly in the intensive care unit (ICU) undergoes a different set of tests compared to a stable patient with acute symptoms. In this context, the identification of predictive variables of clinical outcomes is mandatory to ensure that they are measured for all new patients. Restricting the analysis to patients for whom all variables have been measured would make their set very limited, as would selecting variables that cover a certain fraction of patients, as we show in Table variable selection and for prediction, and must report a realistic error rate. To ensure this, the ranking of clinical variables resulting from our feature selection method is based on the performance on independent validation cohorts. The selected predictive variables are validated using cross validation on all the patients from the first wave dataset that have those variables measured (185 patients), and also on a second patient cohort from the second COVID-19 wave (72 patients). Further validation is performed on an extended first wave dataset, where all 265 patients are considered, and missing data is imputed using a nearest neighbours approach. The good performance in all validation settings support again the use of these six variables for decision making. We also demonstrate the superiority of this variable set compared to RFE.The third challenge is related to the validation of the AI predictive model: the model must generalise for 23. Here, we use two classification models: logistic regression and decision trees, both of which are so called ’white-box’ models. Logistic regression provides interpretability through the regression coefficients, while the decision tree provides a set of rules used to decide the outcome. The indications from both our models are valid from the medical perspective.A final challenge is model interpretability: the decision made by the AI method needs to be transparent. This enables both understanding of the final response and validation of the model by using medical knowledge. Examples of interpretable models for COVID-19 patient classification arepre-existing cardiac and renal conditions, and organ damage that may occur during the infection with COVID-19. Therefore, the use of troponin and BUN is stronger than simply looking at pre-existing comorbidities.We would like to underline the role of two relevant variables: troponin and BUN. They integrate information on both The study presented here concentrated on the prediction of death. A related question could be whether we can distinguish mild from severe disease, where severe disease includes cases where the outcome is either death or ICU admission. Mild cases instead are those who are discharged without any ICU admission. This prediction could be applied, for instance, to select patients for clinical trials. The method we presented can be immediately applied to this related problem, by simply changing the outcome to be predicted. To showcase this, we have retrained the predictive model on this new problem, employing the same six features optimised for predicting death on the first wave dataset. Table 42, we plan to expand the analysis on larger datasets and cohorts coming from different locations, as soon as they become available. Given that unfortunately we are now facing subsequent waves of the pandemic, we also plan to continue to apply our predictions on data from new patients from the same hospital. The variables we underlined, and the predictive models, could be used all around the world to aid clinical decision making in a time of emergency as the one we are living in.Even though we have made all efforts to follow suitable external validation practices, both during the selection of clinical variables and after, using data from two different COVID-19 waves, the fact that our study is based on data from one hospital only is still a limitation. To demonstrate the validity of our model in changing settings, which is an important aspect before including it in clinical decision makingThis study was performed according to the principles stated in the Declaration of Helsinki and it conforms to standards currently applied in our country. The patient’s informed consent was obtained in line with security protocols in place in the hospital during the emergency. The use of the data was approved by the Comitato Etico Area Vasta Nord Ovest .Due to privacy reasons the clinical data cannot be made public without restrictions. However, access can be granted to interested researchers upon motivated request to the authors of this paper, and upon subsequent approval by the interested bodies.43 for the prediction and clustering tasks.This section describes the various tools employed to achieve the various objectives of our study: selecting the clinical variables relevant for prediction, building and validating the actual predictive models, missing value imputation and clustering of clinical variables. The entire analysis was performed in python, using the library scikit-learnWe present a new hybrid filter/wrapper feature selection method based on genetic algorithms that generates a ranking of the predictive power of variables starting from the measured clinical variables. The ranking is then validated on an external dataset, and the top variables are selected to proceed with the prediction. We repeated the entire analysis five times, with five different patient cohorts, extracted from the first wave dataset, for variable selection and external validation, using a 5-fold cross-validation method. This resulted in five different rankings with large similarity especially at the top of the ranking . For the final selection of clinical variables, we opted for the top 3 variables from each ranking. This resulted in the 6 different clinical variables listed in Table N, that classifies well a training dataset. The N clinical variables are then tested on an independent validation dataset, using logistic regression, and resulting in a validation AUC value. This validation is denoted as internal validation. The procedure is repeated K times, changing every time the training and validation subsets, while maintaining their relative size . This resulted in several sets of clinical variables and their corresponding validation AUC. Among these, only those that have good AUC values, over a predefined threshold external validation data.Figure Genetic algorithm details. Genetic algorithms are a family of population-based algorithms where solutions to an optimisation problem are generated using concepts from Darwinian evolution44. An initial (random) population of solutions evolves in discrete time steps (generations) using two operators: mutation and crossover. Selection is applied at each generation so that the solutions with the best fitness are admitted to the next generation. After several generations, the population is able to explore promising areas of the search space and provide good local solutions. Genetic algorithms have been previously used for wrapper feature selection and for metaparameter optimisation in predictive models, including biomedical applications, due to their efficiency in exploring the search space in few iteration, reducing thus the number of models to be trained (e.g. the recent works of48). However, to our knowledge, a filter/wrapper hybrid similar to ours does not exist.N clinical variables that enables classification of patients into the deceased/discharged classes. Our algorithm generates initial random subsets of a fixed size N. The solutions are evaluated by training and testing a logistic regression classifier using leave-one-out cross validation. The fitness of the solutions is given by the AUC values (area under the ROC curve). Solutions are mutated at each generation by replacing one of the clinical variables with a different one. Crossover between two solutions is implemented by splitting the solutions into two parts of randomly generated sizes and swapping them among the parents. The new solutions thus obtained are then assigned a fitness value and added to the population. At each generation, selection is performed by keeping only the solutions with the largest fitness, i.e. largest AUC values.Our objective is to select a subset of Fitness evaluation embeds a flexible mechanism that adapts to data with missing values. Typically , when missing values are present, the data is cropped at the start of the analysis by removing both data points and features . However, this has the disadvantage that the size of the data is reduced, and the removal of variables is arbitrary. Here, we perform the cropping only when calculating the fitness for each solution representing a subset of variables, to maximise the number of patients that can be considered. This procedure has the risk that, when the set of variables includes many missing values, the number of patients is very reduced. We control this by a parameter of the genetic algorithm, Experimental setup. The feature selection method has several parameters: the N clinical variables that the genetic algorithm looks for; the K times the internal wrapper stage is performed; the minimum number Discussion. Our hybrid feature/wrapper feature selection method has two important characteristics. First, it is resilient to missing values in the dataset, by embedding a mechanism that adjusts the training data size based on missing values. This removes the need to crop the dataset at the start of the analysis, to remove missing values.Second, the selection of variables is based on two different independent validation procedures, ensuring that the selected variables are general enough. The first validation procedure is internal, and applies to the solutions of the genetic algorithm, that are filtered based on test AUC. The second validation procedure is external, and applies to the final ranking of clinical variables. The generality of our selected variables is also supported by the results obtained from different patient cohorts. The five rankings from different patient subgroups are very similar at the top of the ranking . In fact, by selecting the top 3 variables of each ranking, we cover 5 variables out of the top 6 of all rankings. Additionally, the performance on independent external validation is very good for all five rankings. When considering the top 6 variables from each ranking, independent validation accuracy is perfectly inline with results presented in the previous section with the final set of 6 variables. We also see good agreement for new validation data from the second COVID-19 wave, for decision trees and random forest models.41. This method ranks the clinical variables by a criterion, removes the least important variables and then reranks, repeating the procedure until a certain number of variables is obtained. Here we employ this method to obtain 10 clinical features, and we use two different ways to rank features: based on logistic regression (LR) and on decision tree (DT) feature importance values. We perform 20-fold cross validation, and combine the resulting sets of clinical variables, selecting those that appear at lest 10 times in the final RFE solutions.We compare the new approach introduced above with a standard method to perform feature selection: recursive feature elimination (RFE)patient coverage threshold under which variables are removed. Then, with the remaining variables, we eliminate all patients that have at least one missing values. Table An important step in the analysis is defining the initial dataset. As mentioned several time, our data suffers from missing values. Hence we need to select a set of variables and patients to be considered as a starting point. We adopt an approach where we first eliminate variables that have many missing values, i.e. we set a 49, both for the wrapper stage and the external validation of the five different rankings produced. During the feature selection phase clinical variables are standardised  before being fed to the logistic regression model.Our feature selection method uses internally a logistic regression classifier50, Random Forests51, Support Vector Machines52 and Naive Bayes52 classification. The supplementary material shows the predictive performance before feature selection, with different initial cropping of the dataset to remove missing values. Most of the times, logistic regression outperformed the other models, while it was sometimes outperformed by the RF model. However, RF requires a large time for training. Given the large number of models to be trained during the filter/wrapper feature selection phase, we decided to select logistic regression for this task.The logistic regression model was selected through a preliminary comparison of various predictive models, including Decision TreesOnce clinical variables were selected, prediction was studied for five different models: Logistic Regression, Decision Trees, Random Forests, Support Vector Machines and Naive Bayes. These are classical machine learning models, and we used the scikit-learn Python package for the analysis.The predictive performance was evaluated using two different metrics: the accuracy, i.e. the fraction of patients correctly classified, and the F1-score averaged over the two classes , i.e. the weighted average of precision and recall for each class. We performed leave-one-out cross validation to test the models, i.e. employing each patient to validate a model obtained from all other patients. We repeated the analysis on the datasets from both COVID-19 waves, to test the selected clinical variables.52 among the two vectors of clinical variables, after removing all variables missing in at least one of the two patients. This distance has the advantage that it does not depend on the number of dimensions of the vector. The data was again standardised before missing value imputation.We implemented a method for imputation of missing values based on a nearest neighbour approach. For each patient 52 for this purpose, with the number of clusters fixed at 27. This was chosen empirically since it allowed for four of the six selected clinical variables to belong to separate clusters, without fragmenting too much the variable space. Agglomerative clustering groups the data in a bottom up manner, by joining iteratively the two closest groups, and stopping when that number of clusters is reached. Again we used scikit-learn, with Euclidean distance and Ward linkage.After completing the missing values, we also perform clustering of all clinical variables to provide a better view over the biomarkers selected. This step also provides a means to validate the data collected and the missing value computation. We use agglomerative hierarchical clusteringSupplementary Information.


Because vaccine hesitancy is a dynamic trait, it is critical to identify and compare the contributing factors at the different stages of a pandemic. The prediction of vaccine decision making and the interpretation of the analytical relationships among variables that encompass public perceptions and attitudes towards the COVID-19 pandemic have been extensively limited to the studies conducted after the administration of the first FDA-approved vaccine in December of 2020. In order to fill the gap in the literature, we used six predictive models and identified the most important factors, via Gini importance measures, that contribute to the prediction of COVID-19 vaccine acceptors and refusers using a nationwide survey that was administered in November 2020, before the widespread use of COVID-19 vaccines. Concerns about (re)contracting COVID-19 and opinions regarding mandatory face covering were identified as the most important predictors of vaccine decision making. By investigating the vaccine acceptors and refusers before the introduction of COVID-19 vaccines, we can help public health officials design and deliver individually tailored and dynamic vaccination programs that can increase the overall vaccine uptake. Since 11 March 2020, when the World Health Organization declared the novel coronavirus disease (COVID-19) outbreak a global pandemic, the world has grappled with how to contain and minimize its devastating effects [This virus has taken many lives globally and continues to cause immense suffering in every aspect of human life. The collaborative efforts of the scientific community, in conjunction with the support of governmental organizations, led to the development of various vaccines that are highly effective against severe disease caused by the original strain of COVID-19. For instance, after Israel vaccinated almost 60 percent of its population, as of April 23, 2021, the daily number of average fatalities dropped below 10, a steep decline from the average number of deaths (70) in January [The effectiveness of the available COVID-19 vaccines can be evaluated by comparing the mortality rates among vaccine recipients and unvaccinated individuals. For instance, the recipients of the booster shots who were 50 years of age or older due to the emergence of variants had 90% lower mortality than those who did not receive a booster in Israel during the summer of 2021 . SimilarAlthough progress is being made, and cases and deaths are diminishing primarily due to the vaccination campaigns being undertaken around the world and the addition of booster shots, vaccine hesitancy threatens to undermine or even halt this progress, especially as new variants of the virus continue to emerge. Knowing who would be likely to accept or reject the vaccine has critical importance, and could drastically change the trajectory of the pandemic across the globe. However, the trajectory of COVID-19 vaccine hesitancy shows that the indecision to be vaccinated has been a fluctuating process. More than one-third of vaccine-hesitant respondents before introducing COVID-19 vaccines leaned towards the willingness to be vaccinated after the actual vaccine administration process started in early 2021 . If thisThe promising developments in the fight against this deadly virus are important milestones, but the pandemic is far from over. Addressing the global challenges, such as significant variation in governments’ capabilities, high vaccination costs, and the inability to effectively allocate and deploy vaccines , requireMuch of the existing literature focuses on identifying the predictors that would impact the willingness to accept (WTA) vaccines using survey data. These variables range from the socio-demographic characteristics of the participants to the perceptions and attitudes towards the vaccines; hence, each variable’s statistical relationship with the WTA the vaccine can be investigated. However, although the results of these studies may have important implications for understanding vaccine hesitancy by offering an interpretation of the statistical relationships, the prediction of vaccine decision making ,11 beforBased on a nationwide survey that was administered to capture the characteristics of the participants who expressed opinions about accepting or rejecting a COVID-19 vaccine, various predictive modeling techniques were performed in this study.The first COVID-19 vaccine was administered in the US in December 2020. A month earlier, in November 2020, when the average number of daily cases was increasing after reaching the declines seen in September 2020, a nationwide survey was administered. Although the news about the potential vaccine approvals was circulating in the media, there was no FDA-approved vaccine during the administration of the survey. Thus, the results of this study can serve as baseline information.A joint Institutional Review Board (IRB) application was filed and approved. QuestionPro , an online survey software company, was contracted to gather the survey data using convenience sampling. A total of 1500 responses were collected regarding the participants’ socio-demographic backgrounds, health characteristics, and experiences with the virus, and whether they were willing to accept a COVID-19 vaccine. The socio-demographic characteristics of the participants are summarized in Initially, anomalous data and outlier values were removed from the dataset. Then, the one-hot-encoding method was used to transform nominal categorical data into binary vectors. This resulted in a total of 1343 responses and 63 features that were used for the construction of our predictive models. The dataset, which consisted of 1343 cases, was split into a training set (80%) and a test set (20%). A 10-fold cross-validation method was used in the training sample for the performance evaluation of the classifier methods and hyperparameter tuning, discussed in the next section. A classifier may yield biased prediction accuracy when the dataset is imbalanced . Thereforpart algorithm in R that implements recursive partitioning, which allows for adjustable misclassification penalties.(1) Decision Tree (DT): A Decision Tree is a non-parametric supervised learning method that consists of several steps, including splitting (portioning data into subsets), pruning (reducing the size of the tree), and tree selection . It is often used in medical informatics and decision making in healthcare management . Conside(2) Random Forest Model (RFM): Random Forest classification techniques are composed of multiple decision trees which are frequently used in predicting events given their high-order interaction effects . Each in(3) Logistic Regression Classifier (LR): Logistic regression is a probabilistic binary classifier that uses logit scores to predict the target class. It has been applied to many COVID-19 related studies, such as predicting infected patients’ recovery  and vacc(4) Neural Network (NN): A Neural Network, an artificial intelligence method, is a subset of machine learning algorithms. The network consists of three layers, namely, an input layer, hidden layers, and an output player. It starts with an input layer which is fed by initial data (the features used for this study). The hidden layer is where all prediction computations are performed. The final phase, output layers, produces the results for the provided inputs. NN has been used in predicting vaccine utilization and targets in healthcare . The hyp(5) Naïve Bayes (NB): Naïve Bayes is a probabilistic predictive classifier that poses an assumption of independence among predictors, which helps in the accurate detection of classes. Naïve Bayes has higher accuracy when processing large patient data points . In this(6) Support Vector Machine (SVM): The Support Vector Machine is a versatile machine learning technique that is frequently adopted for the classification and segregation of patients’ clinical data in the healthcare sector . SVM perA confusion matrix  was creaAccuracy, precision, sensitivity, and specificity metrics are defined in terms of TP, TN, FP, and FN. Accuracy presents the number of correct predictions in the model. Precision is the ratio between the true positive and all the positive instances. Sensitivity is the rate of true positives, and specificity is defined as the ratio of true negatives.In the presence of the data balance distribution, the accuracy, sensitivity, and specificity measures were chosen to assess the performance of each selected model and class. We also report the F1 value, which provides the weighted mean of precision and sensitivity, which takes false positives and false negatives into account. In addition, the area under the receiver operating characteristics curve (AUC-ROC) was used to measure and compare the performance of the classification models at different probability thresholds.A comparison of the selected models is shown in An ROC curve was also used to compare the classification performance of the selected predictive models. A curve closest to the top left corner in an ROC plot indicates the space where the highest TP and lowest FP rates are detected. The area under the ROC curve is a measure of the model’s discriminative abilities . A variable importance analysis was performed to assess the characteristics that contribute to the overall predictions of vaccine acceptors and refusers. Variable importance was computed using the mean decrease in the Gini coefficient, which measures the mean decrease in node impurity . Higher Various socioeconomic factors, such as gender, employment status, and income level, have a significant impact on the outcome values. Gender has the most predictive power among other socioeconomic factors. The group of individuals aged between 44 and 55 years has more discriminative power than any other age group. Moreover, the income range of $0 to $19,999 is ranked as the most effective income-related factor of the vaccination intent. It is important to note that marital status held the least explanatory value among all other socioeconomic-related variables.This study aimed to (1) analyze the predictability of the vaccine acceptors and non-acceptors, and (2) establish a hierarchy of features impacting the decision to be vaccinated against SARS-CoV-2. The results offer insights into the willingness to accept or reject the COVID-19 vaccine before any of the current COVID-19 vaccines were approved and administered. This issue has been rarely studied in the literature.In pursuit of the first objective, we compared six different machine learning models to examine the decision to vaccinate. This study found that the Support Vector Machine has the highest predictive performance accuracy rate (77.40%). This simply means that the model correctly classifies 77.4% of the vaccine acceptors and refusers. However, the higher sensitivity score indicated that SVM performed better when predicting vaccine acceptors as compared to vaccine refusers. The ability to predict vaccine acceptors in a community is tremendously helpful for many reasons, including the fact that it can, when considered as a percentage of the overall community population, also provide an indication of the level of hesitancy in that same community. The “3 Cs” model of vaccine hesitancy highlights that the determinants of hesitancy fall under three main categories: complacency, convenience, and confidence . TherefoObjective two sought to determine the individual predictors that strongly influence vaccine behaviors. The two strongest predictors were (a) worrying about (re)contracting COVID-19 and (b) considerations regarding masking or face coverings. This finding is significant as it provides a foundation on which to build and implement campaigns to promote vaccine acceptance. Our findings suggest that the CDC guidelines from May 2021, in which fully vaccinated people could resume activities without wearing masks except in specific cases, may actually increase vaccine acceptance and result in more people becoming vaccinated. One of the main tenets of operant conditioning is that behavioral responses are primarily influenced by experiences with reinforcement; negative reinforcement refers to influencing behavior through the removal of something aversive. Simply put, in this context, if the consequence of a behavior is something aversive  then that behavior, in this case, vaccine hesitancy, is likely to be reduced when faced with the possibility of mask use discontinuation. Therefore, our study indicates that being able to stop wearing masks after vaccination may be one of the most effective strategies to promote vaccine acceptance. Our study also found that worrying about (re)contracting COVID-19 was the other strongest individual predictor for vaccine behavior. This is significant in our current context because, although we have not yet determined the length of time for which vaccines offer protection, we know that they protect against severe illness and reduce transmission. Our study indicates that public health messaging should focus on these two aspects to have the greatest impact on promoting vaccine acceptance, especially during the early stages of a pandemic, when vaccines are not readily available. Further, once the period of vaccine conferred immunity has been established, and the associated necessity of booster shots, widespread dissemination of this information may promote vaccine acceptance because this directly addresses the possibility of reinfection and would reduce concerns about (re)infection.The identification of two prominent features influencing WTA has practical implications for vaccination programs across the United States. If the only two messages used to influence WTA a vaccine focused on reducing the worry about (re)contracting COVID-19 and avoiding the need for mandatory face masks, a vast majority of individuals would choose to be vaccinated in the United States. These two features are identified as the most prominent in influencing WTA a COVID-19 vaccine. Vaccination program messaging across the United States warrants attention regarding the two most predictive features contributing to the decision to accept a COVID-19 vaccine, i.e., reducing the worry about (re)contracting COVID-19 and avoiding the need for mandatory face masks. However, it should be noted that these predictors may have varying degrees of importance depending on the trajectory of pandemic.Our additional findings that gender, age, education, and income level have a significant impact on vaccine behavior may also be valuable in creating strategies to promote vaccine acceptance. Specifically, strategies segmenting the population and focusing on gender, the age group 44–55, the income range of $0 to $19,999, and attainment of High School  degrees may be more effective at converting hesitancy to acceptance than more broadly focused population-wide strategies. Moreover, the perceived seriousness and threat of the pandemic was a strong indicator for distinguishing between vaccine acceptors and refusers, which supports the main premise of the Health Belief Model. This states that individual beliefs regarding susceptibility and severity of a health concern, in addition to the beliefs about the effectiveness of possible preventive action, predict the likelihood of behavior . This isOur results suggest that messages focused on (re)contracting COVID-19 would be most effective in promoting vaccine acceptance, and messages addressing the possibility of no longer wearing face masks once vaccinated would play a significant role in increasing vaccine acceptance before the approval of vaccines. However, a detailed analysis of vaccine acceptance and its determinants requires the acknowledgment that the decision or indecision to be vaccinated may not remain constant during the period of pandemic. Although the results presented in this study may serve as a baseline of vaccine hesitancy, the findings of similar studies that looked at this important issue after the widespread use of initial vaccines, and after booster shots, will help us understand how vaccine hesitancy evolved during the course of the pandemic. Only then will public health officials be able to devise optimal programs that will be aimed at increasing vaccine acceptance.This study was subject to limitations. Due to the lack of quota sampling, the data collected may not be the representative of the US population. Secondly, because the personal information needed for follow-up information was not obtained during the data collection, a follow-up study, after the administration of the initial vaccines and booster shots, was not conducted. Given the cross-sectional nature of the design, the link between cause and effect cannot be established, and all possible relevant survey responses may not have been included.Future research would benefit from a comparison of the features contributing to the decision to reject the COVID-19 vaccine before and after the introduction of COVID-19 vaccines. To predict the willingness to accept or reject the COVID-19 vaccine during the different stages of pandemics is a valuable tool for public health professionals and efforts to vaccinate the United States population. Other features to be considered in future research exploring the core of vaccine rejection include mandates, geography, religion, and spirituality. Examining the degree to which individuals practice preventative measures post-vaccination may warrant examination.


Carbon nanofibers (CNFs) have various excellent properties, such as high tensile strength, electric conductivity and current density resistance, and thus have great application potential in electrical sensor development. In this research, electrically conductive composite elements using CNFs sandwiched by thermoplastic olefin (TPO) substrates were developed by press molding. The metal mold used for press molding was processed by a femtosecond laser to generate laser-induced periodic surface structures (LIPSS) on the mold surface. The aggregate of CNFs was then flexibly fixed by the LIPSSs imprinted on the TPO substrate surface to produce a wavy conductive path of CNFs. The developed composite elements exhibited a sharp increase in electrical resistance as strain increased. A high gauge factor of over 47 was achieved, which demonstrates high sensitivity against strain when the composite element is used as a strain gauge. Scanning electron microscope observation revealed that the TPO filled the spaces in the aggregate of CNFs after press molding, and the conductive path was extended by the tensile strain. The strain-induced dynamic changes of contact states of CNFs and CNFs networks are discussed based on the electrical performance measurement and cross-sectional observation of the elements. This research provides a new approach to the production of flexible and high sensitivity strain sensors. Carbon nanofibers (CNFs) are fibrous carbon with a diameter of several tens of nanometers ,2,3. DueIn recent years, polymer composite materials containing nanocarbons such as CNFs or carbon nanotubes (CNTs) have been developed in various areas ,17,18. KIn this research, sandwich-structure composite elements containing CNFs will be fabricated by press molding. In the elements, CNFs are sandwiched by base materials with periodic wavy surface, which is formed by transferring laser-induced periodic surface structure (LIPSS) on the mold, to fix the CNFs. This method has several advantages. First, CNFs are completely sealed between the base materials and exposure of CNFs can be prevented. Second, the wavy conductive path of CNFs expands due to tensile force compared to the linear shape to maintain the length of the conductive path, so it is expected to prevent the conductive path from breaking under tension . Third, After the fabrication tests, the conductivity of the fabricated elements will be evaluated under external forces. In addition, a model for the contact states of CNFs will be established to describe the CNF networks in the composite elements affecting the resistance response, and the model will be verified by cross-sectional observation of the elements. This study will contribute to the development of low-cost superior conductive elements, which may be applied to high-flexibility and high-sensitivity strain sensors in the future.The designed structure of the composite element is shown in As the base material sandwiching CNFs, thermoplastic olefin (TPO) was used in this study. TPO is a mixture of polypropylene (PP) and ethylene-propylene-diene rubber (EPDM), and is highly elastic as compared with pure PP. In TPO, PP acts as a hard segment crosslinking EPDM which is a soft segment, and TPO can be reshaped by high temperature over the softening point of PP. By using the TPO substrates, low-cost conductive elements with high flexibility can be developed, which is useful for strain sensor applications in robotics and wearable devices, etc. TPO substrates  with 1.0 mm thickness were used, the physical properties of which are shown in The physical properties of CNFs  used in the experiments are shown in The sandwich structure containing CNFs was fabricated by second press molding as illustrated in The molding conditions are shown in In this study, three composite elements under the conditions shown in The states of CNFs in the element before and after press molding were observed and compared by using field emission scanning electron microscope (FE-SEM). In addition, changes in the state of the wavy conductive path formed by the CNF aggregate under tensile strain were also observed. Based on the observations, a model for describing the contact states of CNFs and the conductive properties of the composite elements was established for discussion.Electrical resistance of the fabricated composite element was measured by the four-terminal method using a DC power supply device  and two digital multi-meters . The voltage was set to 10 V. In this measurement, two copper wires were inserted into the composite element and connected to CNFs between the two points  to Equation (2), the electrical resistance Rs should be proportional to the strain ε as the following equation:In the experiment, the change in the electrical resistance with strains was examined. One side of the element was fixed with a vise. Following this, weights in a range of 0–500 g were attached at the opposite side to generate tensile strains in the conductive part. The strain was calculated by the following equation:K is the gauge factor, ΔR is the electrical resistance change to strain, and R0 is initial electrical resistance. In this study, the conductivity of composite elements was evaluated in terms of increased electrical resistance to strain and gauge factor. Following this, the effect of the periodic structure and pressing force in the fabrication process were investigated.In addition, the gauge factor of the fabricated composite element, which is also an important coefficient representing the sensitivity of strain gauges, was calculated. The gauge factor was calculated by the following equation:In this study, a mold on which LIPSS was formed by laser irradiation was used for the first press molding to generate nanotextures. After that, the sandwich structure of TPO with the periodic structure based on LIPSS was formed by the second press molding. The three types of fabricated composite elements are shown in Next, the CNFs on the surfaces of samples A (without LIPSS) and B (with LIPSS) were observed by SEM and the results are shown in The layer of CNFs in the cross section of sample C was observed as shown in In The cross section of a C type composite element under tensile strain observed from a side direction is shown in The effect of the sandwich structure on the behavior of CNFs under a tensile force applied to the TPO substrate is illustrated in In addition, diagrams of the microscopic behavior of CNFs in the sandwiched conductive path are illustrated in The measured electrical resistance under a varying strain is shown in The gauge factors of composite elements were calculated by Equation (4) and the result is illustrated in The high sensitivity of sample C is due to the wavy conductive path of the CNFs. As shown in In the case of tension in the Y-direction, gauge factors became smaller in both sample B and sample C. This is because the compression state for each tension direction was different. In the X-direction tension, the electrical resistance increased as the cross-sectional area of the conductive path was compressed. However, in Y-direction tension, the wavy conductive path was already formed in the cross section. Since the wavy conductive path was folded by compression, the cross-sectional area of the conductive path was not decreased ,28,29. TAs shown in previous work, carbon nanotubes and conductive polymers exhibit a nonlinear current voltage characteristic ,34. In tIt should also be mentioned that the TPO substrate thickness will affect the elastic modulus, and in turn, the performance of the strain sensor. In this study, we used a thick TPO substrate because the sensor we developed had a very high sensitivity which could detect the resistance change even for a very small bending strain. For highly flexible sensors, it may be necessary to use thinner TPO substrates which enable large bending strains. This may lead to different deformation behaviors of CNFs and the final performance of the sensor such as maximum strain. These aspects will be investigated in our future work.(1)The surface periodic structure of TPO transferred from a mold with LIPSS exhibited a strong effect of retaining CNFs and unifying the CNF distribution. As a result, the electrical resistance was decreased.(2)A wavy conductive path of CNFs was formed based on the periodic structure of TPO surface. Due to the compression of the CNF aggregate by a pressing force, the contact points among the fibers increased. The conductive path decreased the electrical resistance by approximately 70% under no tension.(3)Under tension, the contact points of CNFs decrease in the wavy conductive parts where tensile stress is generated, while in locations where compressive stress is generated, the contact points are less affected.(4)The composite element with a sandwich structure showed a gauge factor over 47 which is several times higher than that of conventional CNF-based conductive elements reported in previous studies, indicating significantly higher sensitivity as a strain sensor.TPO-CNF composite elements with nanoscale periodic interfacial structures were fabricated by press molding, and their electrical properties were evaluated. The conclusions are as follows:This study demonstrated the feasibility of fabricating highly sensitive, compact, flexible and harmless composite conductive elements from CNFs and polymers by press molding. It is expected that this kind of conductive element might be useable as low-cost and high-performance strain sensors in flexible electronic devices, robots, wearable devices, car manufacturing and so on.


Conventional magnetic resonance imaging (cMRI) in multiple sclerosis (MS) patients provides measures of focal brain damage and activity, which are fundamental for disease diagnosis, prognosis, and the evaluation of response to therapy. However, cMRI is insensitive to the damage to the microenvironment of the brain tissue and the heterogeneity of MS lesions. In contrast, the damaged tissue can be characterized by mathematical models on multishell diffusion imaging data, which measure different compartmental water diffusion. In this work, we obtained 12 diffusion measures from eight diffusion models, and we applied a deep-learning attention-based convolutional neural network (CNN) (GAMER-MRI) to select the most discriminating measures in the classification of MS lesions and the perilesional tissue by attention weights. Furthermore, we provided clinical and biological validation of the chosen metrics—and of their most discriminative combinations—by correlating their respective mean values in MS patients with the corresponding Expanded Disability Status Scale (EDSS) and the serum level of neurofilament light chain (sNfL), which are measures of disability and neuroaxonal damage. Our results show that the neurite density index from neurite orientation and dispersion density imaging (NODDI), the measures of the intra-axonal and isotropic compartments from microstructural Bayesian approach, and the measure of the intra-axonal compartment from the spherical mean technique NODDI were the most discriminating . In addition, the combination of the neurite density index from NODDI and the measures for the intra-axonal and isotropic compartments from the microstructural Bayesian approach exhibited a stronger correlation with EDSS and sNfL than the individual measures. This work demonstrates that the proposed method might be useful to select the microstructural measures that are most discriminative of focal tissue damage and that may also be combined to a unique contrast to achieve stronger correlations to clinical disability and neuroaxonal damage. Conventional magnetic resonance imaging (cMRI) in multiple sclerosis (MS) plays a major role in MS diagnosis, prognosis, and in the evaluation of patients’ therapeutic response . HoweverA microenvironment characteristic is measured differently by the measures from different mathematical models due to the different assumptions on the diffusion within the tissue. Yet, to our knowledge, the direct comparison of all considered diffusion measures on MS lesions and the possibility to combine them does not exist. Therefore, how to select the most discriminating diffusion measures for a given neurological disorder and how to combine the complementary information they might provide remain to be open questions and motivate this study.Convolutional neural network (CNN) in deep learning has proven to be promising in various applications of MR images and is able to encode spatial patterns on the images into representative hidden features . In our One hundred twenty-three MS patients  were enrolled in the study, which was approved by the local Ethics Committee of Basel University Hospital. All subjects gave written consent prior to the enrollment. MS patients underwent a multiparametric protocol on 3T whole-body MR system (Siemens MAGNETOM Prisma). The protocol included 3D SPACE-based FLAIR, 3D magnetization-prepared 2 rapid gradient echoes (MP2RAGE) , and mDW1 . The optimizer was AdamW  with theIntrinsic strong correlation between the quantitative diffusion measures can lead to instability of the obtained attention weights and the ranked order, compared to the result in To assess which selected subject-wise normalized quantitative diffusion measures, or combination of those measures, was best correlated with patients’ EDSS as well as NfL in the pure test dataset, we first averaged the diffusion measures within each lesion and then over lesions within each patient. In 31/39 patients of the test dataset, we quantified sNfL. Then, we performed Spearman’s correlation coefficient with two-sided 20,000 permutation tests. The Benjamin–Hochberg procedure  was perfIn The diffusion measures selected by using the validation datasets were the neurite density index (NDI) from NODDI, the intra-axonal and isotropic compartment from MB (Intra-MB and Iso-MB), and the intra-axonal compartment from SMT-NODDI (Intra-SMT) in p-values of the selected normalized diffusion measures, or their statistically significant combinations and EDSS, are reported in The Spearman’s correlation coefficients (ρ) and the corresponding original p-values are reported in The Spearman’s correlation coefficients (ρ) and the corresponding original Our work provided evidence that a modified version of GAMER-MRI, including a specific selection procedure for correlated measures, permits to identify the microstructural diffusion measures that are most discriminative of focal MS pathology among the ones obtained with eight open-source mathematical models of multishell diffusion data. Moreover, our data showed that some of the combinations of the selected normalized diffusion measures better correlated with patients’ disability and neuroaxonal damage than the individual measures.Diffusion-based microstructural measures quantify different compartments based on various assumptions. Nevertheless, the relative sensitivity of the different diffusion-based microstructural metrics to specific CNS pathologies is unclear. In this work, we have provided a methodological frame to discriminate the most sensitive diffusion microstructural measures to focal MS pathology in a large population of MS patients.We first aimed at identifying which measure best discriminated MS lesions from the perilesional tissue because we judged that if the neural network was able to differentiate between lesions and the immediate surrounding tissue, the learned pattern would have been most sensitive to focal MS pathology than the one we would have derived by comparing lesions to the distant normal-appearing tissue. The evaluation metrics in The core idea of the attention mechanism is to enhance important features from the data themselves relevant to the specific application . TherefoTo our knowledge so far, only few studies applied measures derived from microstructural models to study focal MS pathology (for a review, see The four selected diffusion measures include three measures for the intra-axonal compartment from three models and one measure for the isotropic compartment from one of the three models. This means that most of the discriminating information of the damaged neurons was from the loss of axonal integrity. The additional information about the inflammatory processes might be reflected by the measure for the isotropic compartment to better characterize the distinction of lesions.Besides, by combining the selected diffusion measures in the discrimination of focal pathology, it was possible to achieve a stronger correlation with patient disability than one of those metrics alone or even conventional MRI metrics, such as the lesion number and volume. These results suggest that a comprehensive description of the tissue microstructure in regions of focal damage in MS patients may well help decrease the clinical–radiological paradox . InteresMost of the combinations that best correlated to EDSS were also highly related to the sNfL levels: remarkably, the correlation coefficients between sNfL and combinations of diffusion-MRI metrics were even higher than the ones obtained between sNfL and the lesion load, which is known to be highly related to sNfL levels . The patTo perform the correlation analyses with EDSS and sNFL, we have used subject-wise normalized maps of diffusion-based microstructural measures, which were the ones encoded by GAMER-MRI. We also trained the neural network on the original images, which, however, led to worse classification performance. Because subject-wise normalized maps were used, it is challenging to determine whether the network could learn the right pattern and to generate representative attention weights. Owing to the applied normalization procedure, the interpretation of the pathological meaning of the combined metrics is particularly difficult. Another limitation of this study was that we divided the cross-validation folds based on the number of patients instead of the number of patches: this led to different distributions of lesion and perilesional patches in the validation datasets of all cross-validation folds and to the fluctuation of the validation results. On the other hand, this also had the advantage of preventing the leak of information induced by the appearance of patches from one patient in both the training and validation dataset. Based on the obtained result , the perIn summary, our work showed that the proposed attention-based neural network and the selection process based on the previous work can select important diffusion measures despite that they are highly intercorrelated. Those measures have the potential to be combined to enhance the correlation with the clinical measures. Future work will be required to directly find the best combinations without using a statistical test and tackling the multiple comparison problem. Furthermore, the use of a combination of diffusion-based microstructural measures deserves further attention and development, allowing a better interpretability of its pathological meaning.The original contributions presented in the study are included in the article/The studies involving human participants were reviewed and approved by the local Ethics Committee of Basel University Hospital. The patients/participants provided their written informed consent to participate in this study.P-JL: conceptualization, data curation, methodology, investigation, formal analysis, and writing—original draft. MB: data curation, methodology, and writing—reviewing and editing. MW: resources, data curation, and writing—reviewing and editing. RR, RG, and FL: data curation and writing—reviewing and editing. SS, MBC, and AD: resources and writing—reviewing and editing. RS: conceptualization and writing—reviewing and editing. JK and LK: writing—reviewing and editing. PC: supervision and writing—reviewing and editing. CG: supervision, conceptualization, funding acquisition, resources, and writing—reviewing and editing. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.