Here, we provide a further exploration of transposed-word effects while imposing serial reading by using rapid serial visual presentation (RSVP) in Experiments 1 (respond at the end of the sequence) and 2 (respond as soon as possible—which could be during the sequence). Crucially, in Experiment 3 we compared performance under serial RSVP conditions with parallel presentation of the same stimuli for the same total duration and with the same group of participants. We found robust transposed-word effects in the RSVP conditions tested in all experiments, but only in error rates and not in RTs. This contrasts with the effects found in both errors and RTs in our prior work using parallel presentation, as well as the parallel presentation conditions tested in Experiment 3. We provide a tentative account of why, under conditions that impose a serial word-by-word reading strategy, transposed-word effects are only seen in error rates and not in RTs.When asked to decide if an ungrammatical sequence of words is grammatically correct or not readers find it more difficult to do so (longer response times (RTs) and more errors) if the ungrammatical sequence is created by transposing two words from a correct sentence (e.g.,  The car was light blue) and ungrammatical 5-word sequences, and asked them to respond as rapidly and accurately as possible if the sequence of words they read was grammatically correct or not. The key manipulation in the Mirault et al. [The white was cat big), and control sequences  for which a grammatically correct sentence could not be formed by transposing any two words. Mirault et al. found that the transposed-word sequences were harder to classify as being ungrammatical compared with the control sequences—a transposed-word effect  compared with non-adjacent external word transpositions . Snell and Grainger argued that these findings were evidence for the noisy bottom-up assignment of word identities to their positions in a line of text as one key source of transposed-word effects.The present study provides a further investigation of transposed-word effects obtained with the grammatical decision task and first reported by Mirault et al. . Miraultt et al.  study cou et al. ; Liu et u et al. , for a ru et al.  showed tTransposed-word effects have also been reported in the psycholinguistic literature  where it has been demonstrated that given the goal to understand / interpret linguistic input, adult humans are capable of recovering from various types of error including errors in word order  procedure. All experiments tested for transposed-word effects using the same set of 5-word sequences and the grammatical decision task as in Mirault et al. . Given tOne highly relevant prior study has investigated transposed-word effects under both parallel and serial presentation conditions in the visual modality in Chinese . This sTwo hundred and eight participants  took part in the experiment. They all were native speakers of French, ranged in age from 18 to 72 years , and received monetary compensation (£9/hour). They were naïve to the purpose of the experiment and signed online an informed consent form before starting the experiment. The large number of participants was designed to provide a more robust investigation of the null effect of word transposition seen in the RT data obtained in pilot studies. The large number of participants also enabled a stronger test of the null effect in RTs by randomly sampling a sub-group (N = 50) of participants in 5 independent follow-up analyses.We applied the design of Mirault et al. , in whicThe experiment was created using LabVanced  and we lme4 package  and t- or z-values (for RTs and error rates respectively), with values beyond |1.96| deemed as significant (Baayen [10(RT)) was performed to normalize the RT data. Statistical analyses were only performed on the data obtained with the two types of ungrammatical sequences. Results for the correct sentences are provided for comparison. All the models, in this experiment and the following experiments, included random intercepts and random slopes whenever possible. We also provide an analysis of conditional accuracy functions to examine the extent to which effects in error rates vary as a function of average RT  and generalized logistic mixed-effects (GLME) models respectively, fitted in the RStudio version 3.6.1 (R Core Team ) statists et al. ). In eacs et al. ). This r (Baayen ). A loga & Dresp ; Fernánd & Dresp ). These https://osf.io/raxgj/.Prior to statistical analysis, we excluded 6 participants with accuracy lower than 75%. We also excluded trials with RTs < 100 ms and > 3,000 ms (4.93%). The remaining dataset was composed of 31,028 observations. In this and the following experiments the number of data points per condition largely exceeded the number recommended by Brysbaert and Stevens  for suffb = 1.16; SE = 0.75; t = 1.53), with practically equivalent RTs being obtained to the transposed-word sequences and the control ungrammatical sequences.Prior to analysis, we removed incorrect responses (5.21%) and trials with values beyond +/- 2.5 SD from the grand mean (2.92%). The effect of Transposition was not significant  with more errors being made to the transposed-word sequences compared with the control sequences.There was a significant effect of Transposition  and given the large sample of participants that were tested, the robustness of this pattern was put to test by randomly sampling 50 participants and re-analyzing the data associated with these 5 different random re-samplings. After randomly selecting 50 participants for each analysis , we applied the same analysis procedure as for the main analysis. Outputs of LMEs and GLMEs are reported in The results of Experiment 1 revealed significant transposed-word effects in error rates but not in RTs. The robustness of this pattern was demonstrated using a random sampling of participants, which was made possible by the large number of participants that were tested in this experiment. We found the same pattern when randomly sampling 50 participants in 5 independent follow-up analyses. Moreover, the conditional accuracy functions revealed that the effect in error rates increased as average RT increased see . HoweverOne hundred and one participants  took part in the experiment. They all were native French speakers ranging in age from 18 to 71 years , and received monetary compensation (£9/hour). They were naïve to the purpose of the experiment and signed online an informed consent form before starting the experiment.The same as Experiment 1.The procedure was the same as in Experiment 1 except that now participants could respond before the end of the 5-word sequence, and the RSVP stream stopped after participants’ response .The same as for Experiment 1.Prior to statistical analysis we excluded 6 participants with accuracy lower than 75%. We also excluded trials with RTs < 100 ms and > 3,000 ms (1.61%). The remaining dataset was composed of 14,954 observations. Condition means are shown in b = 0.89; SE = 1.20; t = 0.74) with identical mean RTs being obtained for the transposed-word sequences and the control sequences.Prior to analysis, we removed incorrect responses (5.26%) and trials with values beyond +/- 2.5 SD from the grand mean (2.86%). The effect of Transposition was not significant  with more errors in the transposed-word condition compared with the control condition.There was a significant effect of Transposition . In Experiment 2 we found the exact same pattern of effects as in Experiment 1—a transposed-word effect in error rates and no effect on RTs. Moreover, the conditional accuracy functions again revealed that the effects in error rates increased as average RT increased.In Experiment 3 we provide a direct comparison of transposed-word effects obtained with parallel presentation of words and with the RSVP presentation conditions tested in Experiment 2. Prior work investigating transposed-word effects under serial and parallel presentation conditions  used RSTwo hundred participants  participated in the experiment. They were recruited online via the Prolific platform . They rThe stimuli were the same as in the previous experiments. The design was the same as in the previous experiments except for the addition of the Procedure factor . Each participant was tested with both procedures, with half of the participants receiving the parallel presentation trials first and the other half the serial presentation trials first. The design was therefore a 2 (Transposition: Transposed vs. Control) x 2  factorial. The same stimuli were tested with the two procedures, but target repetition was avoided by using the two counterbalanced lists of stimuli of Experiments 1 and 2. So, participants were tested with both lists (as opposed to one list in the previous experiments), one with parallel presentation and one with serial presentation.Parallel presentation. A description of one experimental trial is shown in Serial presentation (RSVP). The procedure was the same as for Experiment 2 .The same as in the previous experiments, except for the addition of the Procedure factor in the analyses, and the interaction term between Transposition and Procedure. We also added Delta plots for the effects of Transposition in RTs. Correct RTs for all participants were first divided into equal-sized bins (quantiles) separately for the transposed-word and control conditions and for each procedure. The transposed-word effect (mean RT for the transposed-word condition minus mean RT for the control condition) was then calculated for each bin. Delta plots reveal how effects of a variable (here the effects of Transposition) vary as a function of average RT. These plots provide valuable information given potential differences in average RT across the parallel and serial presentation procedures.Prior to statistical analysis, we excluded 5 participants with accuracy lower than 75%. We also excluded trials with RTs < 100 ms and > 3,000 ms (0.32%). The remaining dataset was composed of 62,200 observations. Condition means are shown in b = 12.83; SE = 1.97; t = 6.49) and of Procedure . Crucially, we found an interaction between Transposition and Procedure  that revealed a greater Transposition effect with parallel presentation  compared to a non-significant effect with serial presentation .After removing the trials with incorrect responses (6.36%) we excluded 0.79% of RT values beyond 2.5 SD from the grand mean. We observed a significant effect of Transposition , a significant effect of Procedure , and a significant interaction between Transposition and Procedure . The interaction reflects the greater effect of Transposition obtained with parallel presentation  compared with serial presentation , although both effects were significant.We observed a significant effect of Transposition  conditions while matching the overall duration of stimuli in the two procedures. We replicated the findings of transposed-word effects obtained in error rates but not in RTs with serial presentation in Experiments 1 and 2, and now crucially demonstrated that the effects emerged in both RTs and error rates with parallel presentation. The conditional accuracy functions again revealed that the effect in error rates under serial presentation increased as RTs increased and showed that this was not the case for parallel presentation. Furthermore, delta plots of the transposition effects in RTs showed that these effects increased as average RT increased. Crucially, the delta plots also revealed that the greater effect obtained with parallel presentation compared with serial presentation was still present when considering the longest RTs in both procedures. Moreover, the delta plots clearly show that it is not just the longer RTs obtained with the RSVP procedure that is the cause of the absence of a transposed-word effect in RTs in this condition, since if anything, the effects should have been stronger with serial presentation.In three experiments we examined transposed-word effects under conditions where serial reading was imposed by using the RSVP procedure, and Experiment 3 provided a direct comparison of transposed-word effects under serial and parallel presentation conditions. In this critical final Experiment, the total duration of word sequences (1500 ms) was matched across serial and parallel presentation conditions, and participants made grammatical decisions to these sequences. In both procedures, participants could respond prior to stimulus offset.the white was cat big) compared with matched ungrammatical sequences that could not be transformed into a correct sentence by transposing any two words . However, contrary to the transposed-word effects observed under normal text presentation conditions , the eft et al. ). The dit et al.  in ChineDo the present findings speak to the more general question concerning the serial vs. parallel processing of words during reading? We predicted a lesser impact of the transposed-word manipulation under conditions of serial reading, principally because serial reading should provide less noisy bottom-up information about word order , hence The key question then is why do the effects only emerge in errors and not in RTs when serial reading is imposed, as in the RSVP conditions tested in the present study? Here we provide a tentative explanation for this pattern, and there are two additional pieces of information that helped forge this explanation. One concerns the fact that correct RTs on grammatical trials were faster than on ungrammatical trials with parallel presentation but slower with serial presentation see . The othA related issue that needs to be addressed is why are transposed-word effects also seen in RTs as well as error rates under parallel presentation, and more generally, why are the effects greater with parallel presentation? In sum, we examined transposed-word effects in conditions that imposed a one-word-at-a-time serial reading by using the RSVP procedure. We found significant transposed-word effects in all experiments, but only in error rates and not in RTs with serial presentation of words. Parallel presentation of words, on the other hand, revealed effects in both RTs and error rates. We conclude that serial presentation reduces uncertainty in the bottom-up process of associating word identities with their positions in a sequence of words and modifies the way in which participants perform the grammatical decision task. Moreover, even with reduced noise in bottom-up processing, top-down sentence-level constraints  continue to drive transposed-effects independently of how words are presented.


Hence, X-ray-assisted staining will be a useful tool for the development of reliable staining procedures for large samples such as entire brains of mice, monkeys, or humans.Electron microscopy of biological tissue has recently seen an unprecedented increase in imaging throughput moving the ultrastructural analysis of large tissue blocks such as whole brains into the realm of the feasible. However, homogeneous, high-quality electron microscopy staining of large biological samples is still a major challenge. To date, assessing the staining quality in electron microscopy requires running a sample through the entire staining protocol end-to-end, which can take weeks or even months for large samples, rendering protocol optimization for such samples to be inefficient. Here, we present an in situ time-lapsed X-ray-assisted staining procedure that opens the ‘black box’ of electron microscopy staining and allows observation of individual staining steps in real time. Using this novel method, we measured the accumulation of heavy metals in large tissue samples immersed in different staining solutions. We show that the measured accumulation of osmium in fixed tissue obeys empirically a quadratic dependence between the incubation time and sample size. We found that potassium ferrocyanide, a classic reducing agent for osmium tetroxide, clears the tissue after osmium staining and that the tissue expands in osmium tetroxide solution, but shrinks in potassium ferrocyanide reduced osmium solution. X-ray-assisted staining gave access to the in situ staining kinetics and allowed us to develop a diffusion-reaction-advection model that accurately simulates the measured accumulation of osmium in tissue. These are first steps towards  Mikula showed tWe developed in situ time-lapsed X-ray-assisted staining with the goal of facilitating and accelerating the optimization of staining protocols for large biological samples such as whole mouse brains . 4mm-braIn the μCT mode, projections of the sample are acquired at different rotation angles in order to reconstruct a 3D computed tomograph of the sample. Depending on the required signal to noise and resolution, this mode allowed us to acquire approximately 1–2 computed tomographs per hour.In single-projection mode, a projection of the sample is acquired in a fixed position every few seconds without rotating the sample.The advantage of the μCT mode is that arbitrary virtual reslices can be extracted to assess the staining progression in various parts of the sample. However, with our X-ray microscope, the temporal resolution in μCT mode was limited to 30–60 min per tomograph. Therefore we performed all quantitative experiments in single-projection mode where the temporal resolution for monitoring of the heavy metal diffusion and accumulation was on the order of a few seconds. Note, in this mode, the pixel intensities of the resulting X-ray projection images are accumulated through the entire sample and therefore the correspondence to the pixel intensities in thin serial-section EM images is less accurate than in μCT mode . As soon4) is one of the most commonly used contrast agents for lipid staining in electron microscopy due to its large atomic number and its ability to integrate into cellular membranes 6]) (4. (4 solution the sample height expands by about 5% (3[Fe(CN)6]+ 8 a,b), another commonly used reducing agent for OsO4.After 20 hr of incubation the staining density of osmium is homogeneous across the first 1000 μm of cortical depth . In addie(CN)6]) , which we(CN)6])  we founde(CN)6]) , wherease(CN)6]) , in part)6]) (4.  reported)6]) (4. . (Mikula)6]) (4.  hypothes)6]) 6]) (4.  reported)6]) (4. . For a c)6]) 6]) 6] without any washing step in-between. We repeated this procedure for 4 mm brain punches: First, the samples were incubated in 2% buffered OsO4 for 22 hours resulting in homogeneous staining throughout the sample. Next, we placed the sample directly without any washing step in 2.5% buffered K4[Fe(CN)6] solution. (4[Fe(CN)6] is to convert VIII-oxidized water-soluble osmium into an VI-oxidized water-soluble form which is thought to generate additional, non-polar OsO2 to be deposited in the membrane increasing the heavy metal content and the contrast in electron microscopy. However, we found that the heavy metal density in the K4[Fe(CN)6] immersed samples did not increase, but rather decrease from the sample surface towards the center as time progressed (4[Fe(CN)6] removes or “washes out” osmium from the sample (4[Fe(CN)6] readily dissolves and disintegrates the tissue (3[Fe(CN)6], but the sample height shrank by about 2%  is the density of osmium bound to the tissue, s is the density of available binding sites and m corresponds to the density of masked binding sites at time t and a radial depth x from the cortical surface. De is the effective diffusion coefficient for OsO4, kon is the reaction rate constant for the binding of osmium to the tissue and kunmask is the reaction rate constant at which masked binding sites get unmasked. In addition, the local change of the tissue geometry, e.g. the expansion as observed for OsO4, results in a velocity field v that describes the local spatio-temporal flow of the density of each component c:f is the density of freely diffusing OsOThe flow consists of an advection term 4 .We fitted the parameters of the system of reaction-diffusion-advection equations for a tissue sample with the projection geometry shown in 4  as well 4  capture en bloc staining protocols for electron microscopy. By opening the ‘black box’ of staining protocols, X-ray-assisted staining enables, for the first time, to observe directly the diffusion of heavy metals and their microscopic effects on the tissue. While the technique will be useful to develop novel staining protocols for small to intermediate sized samples with dimensions less than 1 mm, it will particularly accelerate and facilitate the development and optimization of new staining protocols for large biological samples such as whole brains 6], a reducing agent that was believed to enhance the accumulation of osmium in membranes, in contrast can wash out osmium from stained tissue if applied separately. If K4[Fe(CN)6] removes osmium from the sample, why does the protocol modification of 4[Fe(CN)6] clears the cytosol more efficiently than the cell membranes. As a consequence, the osmium content in the cytosol gets washed out more/faster, which might result in an overall electron microscopic contrast increase between the membranes and the cytosol. However, the chemical mechanisms by which K4[Fe(CN)6] interact with the tissue remain poorly understood and shining a light on those processes will require additional work.The method also revealed that KX-ray assisted staining opens the door for in silico staining protocol optimization, by enabling to measure important heavy metal staining parameters such as the effective diffusion coefficients and reaction rate constants directly in situ. We derived a diffusion-reaction-advection model that describes the accumulation of osmium in homogeneous tissue by four distinct, but coupled, processes. Osmium diffuses passively into the tissue and binds there quickly to available binding sites, for example unsaturated lipids. Thereby the tissue expands by about 5% in height. Additional binding sites are being unmasked over the course of hours, which enables the tissue to accumulate additional osmium. While the binding of osmium appears to be nearly instantaneous, the passive diffusion and unmasking reactions are slow and appear to be rate-limiting for the staining. The mechanisms and reactions underlying the unmasking of additional binding sites for osmium are not known. Unsaturated lipids, one of the prominent binding sites for osmium, can be masked by lipid-protein complexes. In the presence of osmium, these complexes could dissociate and make the previously masked unsaturated lipid available for binding osmium . A recen3 double-bonds on unsaturated lipids (see methods for the derivation). Because the dominant binding reaction for osmium is assumed to be the formation of mono- and di-ester bonds with unsaturated lipids which has been described to result in the deposition of OsO2, the density of double bonds should roughly correspond to the density of available binding sites. But the proposed model predicts 2.999 times more binding sites, even without unmasking. One interpretation would be that unsaturated lipids are not the dominant reaction partner and that other reaction partners such particular amino acids or nucleotides account for about two third of the available binding sites. Alternatively, this could indicate that each unsaturated lipid double bond gets associated with on average 3 osmium atoms, for example through Os3O5- and Os3O5H- isotopes that have previously reported to exist in osmium-stained adipose mouse tissue . Such secondary sources and tissue inhomogeneities are not considered in our model. Still, the fitted model parameters appear consistent with comparable values from the literature, although we also find some discrepancies. For example, in mouse brains we expect to find about 117.975 nmol/mme tissue . Similare tissue .We expect that X-ray-assisted staining will be very useful in establishing a more refined picture of the heterogeneity of the heavy metal diffusion processes in different tissue types and compartments as well as in different conditions. For example, we also measured and modeled the accumulation of osmium in liver tissue . It willFurthermore, X-ray-assisted staining can be used at higher resolution and in combination with computed tomography for producing a more detailed picture of the tissue mechanics involved in different staining steps. We and several other labs have noticed that brain tissue expands significantly during incubation in ‘osmium amplifiers’ such as the osmiophilic thiocarbohydrazide (TCH) . Mikula reportedAside from accelerating the development of novel staining protocols this new method will also allow for improved closed-loop quality control of the staining process. Large samples tend to be biologically more variable and more difficult to precisely fine-tune all the parameters in the staining protocol ahead of time to account for these individual differences. Hence, X-ray-assisted staining can be used to adjust the parameters in real time to compensate for these individual differences. This is particularly important for experiments in which the sample yield is low or the individual sample is precious, as for example in the case of human brains or functionally imaged brains of behaving animals that require several months of behavioral training. However, this will require calibrating what is measured with X-ray microscopy to the corresponding appearance at an electron microscopic level, because the image formation process and the resulting contrast, signal-to-noise, etc. for both imaging modalities depend not only on the staining protocol, but also on the imaging modality, energy, flux, etc. .2 at pH 7.4;  and cacodylate buffer Electron Microscopy Sciences, Hatfield, PA; CaCl2, Sigma Aldrich, St. Louis, MO). After perfusion, the animal was left for 15–60 min on ice. Subsequently, the brain was removed in order to extract tissue samples at the center of the cortical surface of each hemisphere using biopsy punches of 4 mm diameter. The liver tissue was extracted with a 4 mm biopsy punch from the center of the liver after perfusion. The samples were kept in fixative solution at 4 °C for 36 hr (4 mm biopsy punches) or for 12 hr (whole mouse brain). Next, the samples were washed 6–7 times for 30 min in 0.15 M cacodylate buffer with 2 mM CaCl2.Animal use procedures were approved by the Princeton University Institutional Animal Care and Use Committee (protocol number 2000) and carried out in accordance with National Institutes of Health standards . The experiments were carried out with C57BL/6 J mice  of both sexes, age ranging from 13 to 58 weeks (mean ± sd: 27±9 weeks). Prior to tissue sample extraction, the mice were anesthetized by isoflurane inhalation (4%) and euthanized with an intraperitoneal injection of a ketamine (100 mg/kg) and xylazine (10 mg/kg) overdose. Next, the animals were perfused transcardially with ≥50 ml fixative solution  and 2.5–2.6% paraformaldehyde (PFA) in 0.14–0.15 M cacodylate buffer with 2.0–2.1 mM CaCl2):Glass vials were prepared prior to the experiments with a 1–2 mm layer of Sylgard 184  at the bottom. The whole mouse brain was glued with a small drop of Vetbond tissue adhesive  to the bottom of the vial. For the 4 mm biopsy punch samples, a hole with a diameter of 4 mm was cut out of the Sylgard layer. Next, the samples were placed into the sylgard hole and were glued to the bottom of the glass vial with a small drop of Vetbond tissue adhesive. Subsequently, the samples were immersed in fresh cacodylate buffer solution for the transfer to the X-ray microscope. At the microscope, presets (see X-ray microscopy section below) were loaded for image acquisition. After that the washing solution was replaced with the staining solution and the sample was placed into the X-ray microscope. The acquisition of X-ray images was typically started within 5.66±2.35 min. (mean ±s.d.) after adding the staining solution. The following staining solutions have been assessed at room temperature for this study Osmium tetroxide: 1%, 2% and 3% OsOPotassium ferrocyanide: 2.5% potassium ferrocyanide  trihydrate powder, Sigma Aldrich, St. Louis, MO)4, 2.5% potassium ferrocyanideFerro-redOs: 2% OsO4, 2.5% potassium ferrocyanide, 11.25% formamide Ferro-redOs in 11.25% formamide: 2% OsO4, 2.5% potassium ferrocyanide, 5.56% formamideFerro-redOs in 5.56% formamide: 2% OsOPotassium ferricyanide: 2.5% potassium ferricyanide (EMS #13746-66-2)4, 2.5% potassium ferricyanidePotassium ferricyanide reduced osmium: 2% OsO2O 3 times for 30 min. Next, the tissue was dehydrated with iteratively increasing concentrations of ethanol  in double-distilled H2O for 30 min per step: 25% EtOH, 50% EtOH, 75% EtOH, 90% EtOH, 95% EtOH, 100% EtOH, 100% EtOH. Subsequently, the samples were transferred to 100% acetonitrile  for two 30-min washes. The tissue was embedded in hard Epon in an iterative concentration series: 1:2 Epon: acetonitrile for 12 hr, 1:1 Epon: acetonitrile for 24 hr, 2:1 Epon: acetonitrile for 2 days with fresh resin every 8–12 hr. Finally, the sample was left on the bench at room temperature for 6 days, with fresh resin every 8–12 hr, before it was placed in a 60 °C oven for curing for 5–7 days.After staining, the samples were washed with double-distilled HSerial sections of ~40 nm thickness were collected using an UC7 ultramicrotome from Leica. To assess the ultrastructural preservation of the tissue, the sections were imaged with a Zeiss Sigma SEM  with a backscattered electron detector (BSD) and the following acquisition parameters: 38 μs dwell time, 4 nm pixel size, probe current 200 nA, EHT voltage 6 kV, BSD Gain = High, all sectors of the detector ON.All X-ray microscopy experiments were performed on a Zeiss Xradia 520 Versa 3D X-ray microscope . The samples were immersed in heavy metal solution in glass containers tightly sealed with Parafilm  just before being placed in the recording chamber of the Xradia. Test images were acquired to check imaging parameters. For the biopsy punch samples, projections at a fixed view angle (rotation angle step size = 0°) were taken every 22–60 s resulting in about 1300–1350 images per sample over 22 hr of continuous acquisition. For the whole mouse brain, projections were taken every 22–60 s and after each projection the sample was rotated by 1°. All images were acquired using the image acquisition software Scout-and-Scan Control System  with the following parameters:Objective: 0.4 XTube Voltage = 100 kVOutput power = 7 WSource filter = air (no filter)Exposure time = 20 s.Effective pixel size = 2–3 μm (biopsy punches) and 13–14 μm (whole mouse brain)First, the individual images and metadata of each projection of each sample were extracted from the proprietary *.txrm and *.xrm files as saved by the Xradia image acquisition software. In addition, a time stamp of the acquisition time point was extracted from the image metadata for each projection. A custom cross-correlation based image alignment procedure  was usedFor each sample, a median image lsqcurvefit in MATLAB:The following quadratic 2d function was fitted to At each time t, the projection image was flat-field corrected using The pixel intensities of the detected absorbed X-rays were used as a proxy for the density of heavy metal accumulation in the tissue. In each sample, we quantified the pixel intensities in each projection taken at time t along a cross-section that pointed from the cortical surface radially towards the white matter . The croTo standardize Finally, 4 concentrations as shown in 4. For this, the factors This procedure ensures that the origin is at 0 for the linear relationship between the X-ray absorption intensities in the projection images and the different OsO4 was measured the same way as for the X-ray absorption in stained tissue.The X-ray absorption/pixel intensity of cacodylate-buffered solutions with 1%, 2% and 3% OsOIntensity based heuristics were used in a semi-automated procedure to detect the cortical surface in each image of each sample. The glass vial bottom was used as a reference point to measure the distance to the cortical surface 4 solution was modeled by the monomolecular, saturating growth function lsqcurvefit of MATLAB was used for parameter fitting. The fitted parameters for the average expansion in The expansion of the tissue in buffered OsOWhile we assumed that the tissue density is homogeneous across the brain punch, we took into account the variation in the tissue thickness of the 4 mm brain punch. The diffusion-reaction-advection model described by the cortical surface is in contact with an infinite source of staining agents at constant concentration,the composition of the 4 mm brain punches can be approximated by a semi-infinite, homogeneous medium,the tissue thickness is given by 4 diffusion can be described by Fick’s second law,the kinetics of OsO4 to the tissue is irreversible and can be described by first-order reaction kineticsthe binding of OsO4 binding sites is limited,the density of available OsO4the unmasking of additional binding sites is irreversible, follows first-order reaction kinetics and depends on the local concentration of free OsOthe tissue expands isotropically and can be modeled by a monomolecular, saturating growth function, which induces a local velocity field Because the projected intensity also includes any X-ray absorbing components surrounding the sample, we have to consider the change in free osmium density outside of the sample The initial conditions (t=0) for the system of coupled partial differential 4 in the sample: concentration of free OsO4: density of bound OsOdensity of available binding sites: density of masked binding sites: 4 surrounding the sample: concentration of free OsOThe boundary conditions of the system of coupled partial differential 4: concentration of free OsO4: density of bound OsOdensity of available binding sites: density of masked binding sites: 4 surrounding the sample: concentration of free OsONo flux beyond pdepe was used to solve/simulate the initial-boundary value problem described above.The MATLAB function trust-region-reflective algorithm of the nonlinear least-squares solver lsqnonlin in MATLAB to optimize the N=6 model parameters 3 (3 double bonds to which osmium tetroxide could bind through osmium mono- or di-ester formation. Our proposed model predicts 2.999 times more binding sites (353.774±27.460 nmol/mm3), even without unmasking . Introducing a compelling approach to investigate staining in situ, this work will be of interest to the wide community of scientists preparing biological samples in particular for large-volume electron microscopy. It will become a reference for the field in establishing a quantitative tool for assessing and developing staining protocols. public reviews designed to be posted alongside the preprint for the benefit of readers; (ii) feedback on the manuscript for the authors, including requests for revisions, shown below. We also include an acceptance summary that explains what the editors found interesting or important about the work.Our editorial process produces two outputs: (i) Decision letter after peer review:eLife. Your article has been reviewed by 3 peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Catherine Dulac as the Senior Editor. The reviewers have opted to remain anonymous.Thank you for submitting your article "In situ X-ray assisted electron microscopy staining for large biological samples" for consideration by The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission.Essential revisions:As you can see from the detailed reviews, all reviewers (and myself as well) felt that this work is potentially an important contribution.However, as the central claim of the work is to use CT as a benchmark for the quality of staining for EM, it will be critical to directly relate those two measures, i.e. to perform EM imaging on previously CT imaged samples and relate the two measures of contrast to each other. This will be the critical experimental dataset for revision. There are other valuable comments below. Please edit the manuscript accordingly.Reviewer #1 (Recommendations for the authors):The authors are in a very good position to disambiguate current divergences in the initial steps of osmium staining of soft tissues for electron microscopy imaging purposes.First, it would be very informative to assess the effect of a different iron salt commonly used to reduce osmium: potassium ferricyanide .Second, ref. Hua et al., 2015 implemented the osmium reduction posterior to the osmium incubation, by adding potassium ferrocyanide without intermediate washes. The authors have already explored this staining protocol in Supp Figure 5: samples being stained with OsO4 for 22h were transferred to a potassium ferrocyanide solution. It would be very informative to view those results in summary (ie averaging the 20h timepoint datasets across samples) along with the osmium and reduced osmium baselines shown in Figure 1e, and to complement the insight on sample expansion for that protocol, showing it against the same baselines, as in Figure 1f. An additional control group could be beneficial here: transferring the osmium-stained samples to a water vial, to account for decrease of staining by dilution and diffusion of unbound osmium. In this case it would not be expected to see sample degradation regardless of the incubation time.Third, after staining with osmium or reduced osmium, samples are meant to be washed with water before proceeding to the next staining step, aiming to remove any unbound osmium from the media. The goal of optimizing the first staining step would therefore be to maximize the bound osmium remaining in the sample after the water wash(es). To that end, it would be interesting to see the evolution of intensity vs depth for samples placed in staining solution for 20h and then placed in water, for at least the most typical stains explored (osmium and reduced osmium).Finally, it would be very useful if the authors released the code used for pre-processing single X-ray projections. Given the popularity of benchtop X-ray uCT devices, doing so could simplify and accelerate the adoption of this monitoring method of sample preparation protocols by the user community.These additions could slightly improve the scope and completeness of an already robust and solid study.Reviewer #2 (Recommendations for the authors):Reference, page 4: Claude and Fullam (1945) used formaldehydes + osmium fixation. Only Sabatini introduced glutaraldehyde. It might be good to slightly modify the text.Page 9: the authors refer to figure 1b to comment the staining after 20hours. Yet, fig 1b shows example at 10 hours max.Page 9: "we found that for reduced osmium the accumulation of heavy metals peaks at a depth between 300- 800 μm , whereas the tissue above or below that depth is stained less." Could this be due to the anatomy/heterogeneity of the various cortical layers?Page 9: about the shrinking/expansion of the tissue upon fixation with osmium or reduced osmium. This is very interesting as it could explain a few artifacts observed with several sample types. Yet in this experiment, the cortex biopsy is fitted inside a hole, of the same shape, made inside the sylgard support layer. It is thus possible that the tissue is constrained and can't expand in the lateral directions. Would this enhance the vertical expansion? And therefore such expansion would be less than 5%? Please comment on this.In line with this, it might be good to adapt the drawings of figure 1a. The way the biopsy is displayed, it seems as if it was just placed onto the vial bottom, when in fact is is embedded in sylgard.Page 12: about the osmium density: if I am not wrong these numbers are estimates, extrapolated from many assumptions re the density of unsaturated lipids, double bonds etc ... this should be made clear in the main text and not only in the methods. Even though the authors made their best possible efforts to describe the reactions within the tissues and to estimate the amount of binding sites, I doubt we have a solid proof of the actual concentration of bound or free osmium in this experiment.Page 12: the authors write "While we assume that the tissue density is homogeneous across the cortical thickness ..." I don't think this is a fair assumption. The cortex is organized in layers of different cellular densities.Page 14: discussion. what is not addressed in this paper is the secondary osmium blackening during dehydration. This has been described previously and attributed to unbound osmium and later reduction during the process. IT might be good to elaborate a bit on this. Of course, this leads to the comments made above, which highlight the need to check the samples in EM, i.e. to go, for some of them, through the full sample preparation.Page 16: methods.- The concentration ranges are surprising. Can you explain: "fixative solution 1.3 - 1.5% glutaraldehyde (GA) and 2.5 - 2.6% paraformaldehyde (PFA) in 0.14 - 0.15M cacodylate buffer with 2.0 - 2.1mM CaCl2"- What is the rationale behind these incubation times. Intuitively, one would expect to keep whole brains longer in the fixative? "The samples were kept in fixative solution at 4 °C for 36 hours (4-mm biopsy punches) or for 12 hours (whole mouse brain)."- About the sylgard molds: can you comment on the osmium infiltration. The sides of the biopsy might not be as well exposed as the top.Page 17: X-ray microscopy: the working temperature is not indicate, though it is a very important parameter for the osmium fixation.Reviewer #3 (Recommendations for the authors):The manuscript would be strengthened if the authors compared the actual membrane contrast observed in cross-sections from their samples to demonstrate if these intensities can be quantitatively interpreted (beyond the description of staining gradients). I suggest to cut thin sections from a reduced osmium and osmium tissue cylinder and quantify the staining intensity/contrast across the 4mm diameter to compare with the x-ray profiles.eLife. Your revised article has been evaluated by Catherine Dulac (Senior Editor) and a Reviewing Editor.Thank you for resubmitting your work entitled "In situ X-ray assisted electron microscopy staining for large biological samples" for further consideration by The manuscript has been improved but there are some remaining issues that need to be addressed by small textual adjustments, as outlined below:Please take note of the comments made by reviewer 3 regarding the strict correlation of X-ray and EM measure. Please adjust your textual description accordingly. While e.g. Mikula and Denk do provide some indication that this is the case (as does your work here), I would not want to give the impression that this is a question that has experimentally conclusively and quantitatively been settled. There is still room for direct experimental comparison for X-rays with EM under different conditions .In short, please tune down the corresponding comments and add a sentence or two in the discussion to mention the caveats and limitations discussed below and maybe point towards further such experiments. In this context it could be useful to expand a bit on the physics background and mention how X-ray contrast will depend e.g. on photon energy and that e.g. at the absorption edges the relationship of X-ray absorption and signal in the EM can become more complicated.Reviewer #2 (Recommendations for the authors):The revised version of the paper is efficiently addressing the points that I have raised and I am satisfied with the rebuttal answers as well.In my opinion, the paper is ready for publication and expect the scientific community, especially the EM community to greatly benefit from this contribution.Reviewer #3 (Recommendations for the authors):My recommendation to the authors was to correlate their observed X-ray intensities to what is actually observed by EM in thin sections. In response, they cited Figure 4b from Mikula and Denk, 2015 as evidence that the X-ray reconstructed pixel intensities are correlated with what is observed in EM sections and state "Mikula and Denk (2015) showed that the pixel intensities of serial-section EM images and the intensity of the corresponding reslice are highly correlated."I think this is an overstatement for several reasons:1) As far as I can tell, Mikula and Denk Figure 4b is a n=1 measurement2) The figure in question doesn't report a correlation coefficient, indeed their figure shows a peak SEM normalized intensity at the core of their sample and the X-ray normalized intensity has a peak at the edge of the sample3) Mikula and Denk themselves are far more cautious in discussing this: "…X-ray microcomputed tomography, which exhibits an image contrast similar to that of SEM . This allowed us to quickly test an embedded brain for defects and distortion before deciding whether to finish preparing it for electron microscopy." Mikula and Denk were clearing using X-ray reconstructions to check for gross defects/cracks, not to make quantitative comparisons.Stroh et al., do provide a comparison of X-ray vs. EM intensities  with the caveat that the X-ray intensities are measured from a projection instead of a computed slice reconstruction. Perhaps due to this caveat the intensities do not appear to be very well correlated . In light of this (and the above comments regarding the Mikula and Denk data), I remain unsure to what degree the measured X-ray intensities that form the foundation of the manuscript are informative about actual contrast in EM sections.My original point was related to the author’' Figure 1e: the figure shows that reduced osmium stains tissue with a maximum intensity of ~1.1 (a.u.) compared to osmium alone at ~0.9 (a.u.). Because these intensities are not calibrated against the appearance of the staining in EM sections, their interpretation is limited. The authors state they were not able to test whether this particular finding is consistent with the intensity/contrast of EM sections due to embedding issues of reduced osmium samples. Essential revisions:As you can see from the detailed reviews, all reviewers (and myself as well) felt that this work is potentially an important contribution.However, as the central claim of the work is to use CT as a benchmark for the quality of staining for EM, it will be critical to directly relate those two measures, i.e. to perform EM imaging on previously CT imaged samples and relate the two measures of contrast to each other. This will be the critical experimental dataset for revision. There are other valuable comments below. Please edit the manuscript accordingly.4 and Figure 1 —figure supplement 1 that shows the covariation of the EM and X-ray pixel intensities. Note, in the presented study we acquired X-ray projection images that represent the cumulative tissue and heavy metal density along the direction of projection through a 4 mm thick tissue punch. Therefore our X-ray projection pixel intensities cannot directly be compared to the EM pixel intensity of a thin section. As has been shown previously , in computed X-ray tomograms the intensity scales linearly with EM intensity, if the pixel intensity in an EM section is compared to the corresponding reslice of a computed X-ray tomogram.We added Figure 1 —figure supplement 3 showing the ultrastructural preservation after 20 hours of incubation in OsOReviewer #1 (Recommendations for the authors):The authors are in a very good position to disambiguate current divergences in the initial steps of osmium staining of soft tissues for electron microscopy imaging purposes.First, it would be very informative to assess the effect of a different iron salt commonly used to reduce osmium: potassium ferricyanide .3[Fe(CN)6]. The accumulation of heavy metals for the buffered solution with 2% OsO4 and 2.5% potassium ferricyanide is not as homogeneous as for 2% OsO4 . But in contrast to the experiments where osmium was reduced with 2.5% potassium ferrocyanide, the sample does not shrink if incubated in buffered solution of 2% osmium reduced with 2.5% potassium ferricyanide . In contrast to potassium ferrocyanide, no ‘washout’ effect was observed if the sample was placed in buffered solution of 2.5% potassium ferricyanide after 20hrs of incubation in buffered 2% OsO4 .We added Figure 1 —figure supplement 7 and 8 as well as Figure 2 —figure supplement 3 that show the spatio-temporal accumulation of osmium reduced with 2.5% potassium ferricyanide KSecond, ref. Hua et al., 2015 implemented the osmium reduction posterior to the osmium incubation, by adding potassium ferrocyanide without intermediate washes. The authors have already explored this staining protocol in Supp Figure 5: samples being stained with OsO4 for 22h were transferred to a potassium ferrocyanide solution. It would be very informative to view those results in summary (ie averaging the 20h timepoint datasets across samples) along with the osmium and reduced osmium baselines shown in Figure 1e, and to complement the insight on sample expansion for that protocol, showing it against the same baselines, as in Figure 1f.We added a corresponding Figure 1 —figure supplement 8.An additional control group could be beneficial here: transferring the osmium-stained samples to a water vial, to account for decrease of staining by dilution and diffusion of unbound osmium. In this case it would not be expected to see sample degradation regardless of the incubation time.Third, after staining with osmium or reduced osmium, samples are meant to be washed with water before proceeding to the next staining step, aiming to remove any unbound osmium from the media. The goal of optimizing the first staining step would therefore be to maximize the bound osmium remaining in the sample after the water wash(es). To that end, it would be interesting to see the evolution of intensity vs depth for samples placed in staining solution for 20h and then placed in water, for at least the most typical stains explored (osmium and reduced osmium).2O for 22 hours after 22 hrs of incubation in 2% buffered OsO4. First, the sample was quickly flushed with double distilled H2O four times to completely remove any buffered OsO4 solution. Subsequently, the sample was immersed in double distilled H2O for 22 hrs. The accumulation of heavy metals decreases by about 6% in a depth of 100-1200μm.We added Figure 2 —figure supplement 4 and 5 that show the dynamics of heavy metal washout in double distilled H3[Fe(CN)6], the reduction in heavy metal density appears to be small.Our hypothesis is that this can be explained by unbound osmium that diffuses out of the sample and by the slight expansion of the sample . But in contrast to the washout effect in KFinally, it would be very useful if the authors released the code used for pre-processing single X-ray projections. Given the popularity of benchtop X-ray uCT devices, doing so could simplify and accelerate the adoption of this monitoring method of sample preparation protocols by the user community.The python and MATLAB code for processing the X-ray data is now available on https://github.com/adwanner/XrayAssistedStaining.Reviewer #2 (Recommendations for the authors):Reference, page 4: Claude and Fullam (1945) used formaldehydes + osmium fixation. Only Sabatini introduced glutaraldehyde. It might be good to slightly modify the text.We corrected the text and citations accordingly.Page 9: the authors refer to figure 1b to comment the staining after 20hours. Yet, fig 1b shows example at 10 hours max.We corrected it accordingly.Page 9: "we found that for reduced osmium the accumulation of heavy metals peaks at a depth between 300- 800 μm , whereas the tissue above or below that depth is stained less." Could this be due to the anatomy/heterogeneity of the various cortical layers?4.’Indeed, the effects of reduced osmium accumulation could depend at least in parts of the tissue composition, i.e. the anatomy/heterogeneity of the cortical layers in the tissue block. It is, however, important to note that this effect is not present in non-reduced osmium immersed brain tissue samples that have the same composition. We added the sentences that ‘No such band is present in tissue immersed in non-reduced OsOPage 9: about the shrinking/expansion of the tissue upon fixation with osmium or reduced osmium. This is very interesting as it could explain a few artifacts observed with several sample types. Yet in this experiment, the cortex biopsy is fitted inside a hole, of the same shape, made inside the sylgard support layer. It is thus possible that the tissue is constrained and can't expand in the lateral directions. Would this enhance the vertical expansion? And therefore such expansion would be less than 5%? Please comment on this.In line with this, it might be good to adapt the drawings of figure 1a. The way the biopsy is displayed, it seems as if it was just placed onto the vial bottom, when in fact is is embedded in sylgard.The contribution to the vertical expansion from the embedding in sylgard is expected to be minimal, because the sylgard layer was typically only 2 mm thick, whereas the height of the tissue samples was typically about 6.75 mm tall. The upper end of the sylgard layer can be seen in Figure 1b at the very bottom of the image . Hence, only a small fraction of the sample sits in sylgard. We adapted Figure 1 accordingly.That the contribution from the restriction in sylgard is small can also be seen from the following considerations:4 we measured an expansion of 5%, i.e. For buffered 2% OsOPage 12: about the osmium density: if I am not wrong these numbers are estimates, extrapolated from many assumptions re the density of unsaturated lipids, double bonds etc ... this should be made clear in the main text and not only in the methods. Even though the authors made their best possible efforts to describe the reactions within the tissues and to estimate the amount of binding sites, I doubt we have a solid proof of the actual concentration of bound or free osmium in this experiment.The osmium density  was measured using the pixel intensities which were calibrated using the concentration series shown in Figure 1c. No further assumptions are being made. In the discussion section highlighted by the reviewer we compare this measured osmium density to the density of double bonds of unsaturated lipids in the brain as estimated from numbers reported in the literature. The double bonds of unsaturated lipids are thought to be the main binding site for osmium. Comparing the measured density of osmium to the estimated density of binding sites we found that the measured osmium density is 2.999 times higher than the estimated density of double bonds in the brain. We therefore hypothesize that with long osmium incubation, 3 osmium atoms get associated with 1 double bond.Page 12: the authors write "While we assume that the tissue density is homogeneous across the cortical thickness ..." I don't think this is a fair assumption. The cortex is organized in layers of different cellular densities.4 in 4mm punches of liver tissue . Interestingly, the effective diffusion coefficient appears to be >4X larger in liver tissue compared to brain tissue.We agree that the presented model and the underlying assumptions significantly simplify the actual bio-chemical environment. But despite these oversimplifications, the resulting model presented in Figure 2 captures the dynamics of osmium accumulation well. While modeling the tissue densities of different cortical layers in detail and calculating individual diffusion constants is beyond the presented study, we do agree that it would be very interesting to investigate how the diffusion constants differ in different tissue types. To illustrate this, we measured and modeled the staining kinetics of 2% buffered OsOPage 14: discussion. what is not addressed in this paper is the secondary osmium blackening during dehydration. This has been described previously and attributed to unbound osmium and later reduction during the process. IT might be good to elaborate a bit on this. Of course, this leads to the comments made above, which highlight the need to check the samples in EM, i.e. to go, for some of them, through the full sample preparation.2O washing step after the OsO4 incubation. These results of the wash-out experiments are shown in Figure 2 - Figure Supplement 4 and 5. We do find a slight decrease in heavy metal density that can likely be explained on one hand by the wash out of unbound osmium from the sample as well as by expansion of the tissue which reduces the local heavy metal density.We added Figure 1 - Figure Supplement 3that shows the ultrastructural preservation after 20 hours of osmium incubation. In addition, we performed additional experiments in which we measured how the osmium density changes during the HPage 16: methods.- The concentration ranges are surprising. Can you explain: "fixative solution 1.3 - 1.5% glutaraldehyde (GA) and 2.5 - 2.6% paraformaldehyde (PFA) in 0.14 - 0.15M cacodylate buffer with 2.0 - 2.1mM CaCl2"Some of the mice have been perfused by different experimenters that used slightly different effective fixative formulations.- What is the rationale behind these incubation times. Intuitively, one would expect to keep whole brains longer in the fixative? "The samples were kept in fixative solution at 4 °C for 36 hours (4-mm biopsy punches) or for 12 hours (whole mouse brain)."The 36 hours of post-perfusion incubation in fixative corresponds to the duration that we and other labs typically use for fixing brain tissue. The 12 hours post-perfusion incubation in fixative of the whole mouse brain might indeed be on the lower end for what might be necessary for thoroughly fixing a whole mouse brain after perfusion.- About the sylgard molds: can you comment on the osmium infiltration. The sides of the biopsy might not be as well exposed as the top.The sylgard layer was typically only 2 mm thick, whereas the height of the tissue samples was typically about 6.75 mm tall. The upper end of the sylgard layer can be seen in Figure 1b at the very bottom of the image . Hence, only a small fraction of the sample sits in sylgard and the measurements have been taken several millimeters away from the sylgard, therefore we do not expect any major impact of the sylgard onto the osmium infiltration measurements. We adapted Figure 1 accordingly.Page 17: X-ray microscopy: the working temperature is not indicate, though it is a very important parameter for the osmium fixation.All staining and washing steps have indeed been done at room temperature. We added that information accordingly in the sample preparation section of the methods.Reviewer #3 (Recommendations for the authors):The manuscript would be strengthened if the authors compared the actual membrane contrast observed in cross-sections from their samples to demonstrate if these intensities can be quantitatively interpreted (beyond the description of staining gradients). I suggest to cut thin sections from a reduced osmium and osmium tissue cylinder and quantify the staining intensity/contrast across the 4mm diameter to compare with the x-ray profiles.4 and Figure 1 —figure supplement 1 that shows the covariation of the EM and X-ray pixel intensities. Note, in the presented study we acquired X-ray projection images that represent the cumulative tissue and heavy metal density along the direction of projection through a 4 mm thick tissue punch. Therefore our X-ray projection pixel intensities cannot directly be compared to the EM pixel intensity of a thin section. As has been shown previously , in computed X-ray tomograms the intensity scales linearly with EM intensity, if the pixel intensity in an EM section is compared to the corresponding reslice of a computed X-ray tomogram.We added Figure 1 —figure supplement 3showing the ultrastructural preservation after 20 hours of incubation in OsOWe also tried to acquire EM images of tissue punches stained with reduced osmium, but unfortunately the resin could not infiltrate the samples sufficiently well to allow us to collect serial sections.The manuscript has been improved but there are some remaining issues that need to be addressed by small textual adjustments, as outlined below:Please take note of the comments made by reviewer 3 regarding the strict correlation of X-ray and EM measure. Please adjust your textual description accordingly. While e.g. Mikula and Denk do provide some indication that this is the case (as does your work here), I would not want to give the impression that this is a question that has experimentally conclusively and quantitatively been settled. There is still room for direct experimental comparison for X-rays with EM under different conditions .In short, please tune down the corresponding comments and add a sentence or two in the discussion to mention the caveats and limitations discussed below and maybe point towards further such experiments. In this context it could be useful to expand a bit on the physics background and mention how X-ray contrast will depend e.g. on photon energy and that e.g. at the absorption edges the relationship of X-ray absorption and signal in the EM can become more complicated.Reviewer #3 (Recommendations for the authors):My recommendation to the authors was to correlate their observed X-ray intensities to what is actually observed by EM in thin sections. In response, they cited Figure 4b from Mikula and Denk, 2015 as evidence that the X-ray reconstructed pixel intensities are correlated with what is observed in EM sections and state "Mikula and Denk (2015) showed that the pixel intensities of serial-section EM images and the intensity of the corresponding reslice are highly correlated."I think this is an overstatement for several reasons:1) As far as I can tell, Mikula and Denk Figure 4b is a n=1 measurement2) The figure in question doesn't report a correlation coefficient, indeed their figure shows a peak SEM normalized intensity at the core of their sample and the X-ray normalized intensity has a peak at the edge of the sample3) Mikula and Denk themselves are far more cautious in discussing this: "…X-ray microcomputed tomography, which exhibits an image contrast similar to that of SEM . This allowed us to quickly test an embedded brain for defects and distortion before deciding whether to finish preparing it for electron microscopy." Mikula and Denk were clearing using X-ray reconstructions to check for gross defects/cracks, not to make quantitative comparisons.Stroh et al., do provide a comparison of X-ray vs. EM intensities  with the caveat that the X-ray intensities are measured from a projection instead of a computed slice reconstruction. Perhaps due to this caveat the intensities do not appear to be very well correlated . In light of this (and the above comments regarding the Mikula and Denk data), I remain unsure to what degree the measured X-ray intensities that form the foundation of the manuscript are informative about actual contrast in EM sections.My original point was related to the authors' Figure 1e: the figure shows that reduced osmium stains tissue with a maximum intensity of ~1.1 (a.u.) compared to osmium alone at ~0.9 (a.u.). Because these intensities are not calibrated against the appearance of the staining in EM sections, their interpretation is limited. The authors state they were not able to test whether this particular finding is consistent with the intensity/contrast of EM sections due to embedding issues of reduced osmium samples.Thank you very much for the constructive feedback on the revised manuscript. We agree with Reviewer 3 that adding some quantitative evidence for the correlation between electron microscopy images and X-ray images would strengthen the revised manuscript. On one hand, we adjusted the wording regarding the similarity/correlation between EM and micro-CT in the introduction. On the other hand, we added a note ofcaution to the discussion that for a qualitative comparison between EM and micro-CT appearance the two imaging modalities need to be calibrated and the precise image acquisition parameters need to be considered.en bloc stained brain samples we show the correspondence between the EM and micro-CT appearance. As suggested by Reviewer 3 we included the (indeed high) correlation statistics and corresponding scatter plots.In addition, we added additional data comparing EM and micro-CT appearance similar to Mikula et al., to Figure 1 —figure supplement 1. In two different To conclude, we have tuned down the claims regarding the correspondence between EM and X-ray imaging and at the same time added more examples and quantifications supporting the claims. We believe that these changes and additional information have improved the revised manuscript.


Breast cancer is the most common cancer in women. Classification of cancer/non-cancer patients with clinical records requires high sensitivity and specificity for an acceptable diagnosis test. The state-of-the-art classification model—convolutional neural network (CNN), however, cannot be used with such kind of tabular clinical data that are represented in 1-D format. CNN has been designed to work on a set of 2-D matrices whose elements show some correlation with neighboring elements such as in image data. Conversely, the data examples represented as a set of 1-D vectors—apart from the time series data—cannot be used with CNN, but with other classification models such as Recurrent Neural Networks for tabular data or Random Forest. We have proposed three novel preprocessing methods of data wrangling that transform a 1-D data vector, to a 2-D graphical image with appropriate correlations among the fields to be processed on CNN. We tested our methods on Wisconsin Original Breast Cancer (WBC) and Wisconsin Diagnostic Breast Cancer (WDBC) datasets. To our knowledge, this work is novel on non-image tabular data to image data transformation for the non-time series data. The transformed data processed with CNN using VGGnet-16 shows competitive results for the WBC dataset and outperforms other known methods for the WDBC dataset. Breast cancer is arguably one of the deadliest forms of cancer amongst women with millions of reported cases around the world of which many cases become fatal3. Breast cancer is caused by abnormal growth of some of the breast cells in the lining of the milk glands or ducts of the breast 5. Compared to healthy cells, these cells divide more rapidly and accumulate, forming a lump or mass. At this stage, the cells become malignant and may spread through the breast to lymph nodes or other parts of the body.In recent times, there are growing interest in the development of machine learning (ML) models for medical datasets due to the advancements in digital technology and improvements in data collection methods. Increasingly, several ML-based systems have been designed as an early warning or diagnostic tool for chronic illnesses, for example diagnosing depression, diabetes and cancer7, mammogram images9 and data on symptoms and diagnosis10. Many traditional Computer-Aided Diagnosis (CADx) systems require hand-crafted feature extraction which is a challenging task12. Even conventional ML techniques require the extraction of an optimal set of features manually prior to model training. An extensive review on various feature selection and extraction techniques can be found in14. Some commonly used approaches for ML models are Principal Component Analysis (PCA)15, information gain16, GA-based feature selection17, recursive feature elimination (RFE)18, meta-heuristic methods19 and rough sets20. Feature selection and extraction, therefore, is an important consideration in the pre-processing step before applying any ML algorithm such as decision trees, Bayesian models, Support Vector Machines (SVM) and Artificial Neural Networks (ANN). The behavior of ML algorithms and their prediction accuracy is influenced by the choice of features selected22. Many times manual feature extraction or knowledge of domain experts is needed to have a good understanding on the relevance of the attributes23.The study of breast cancer has attracted considerable attention in the past decades. Improving data collection and storage technologies has resulted in various types and amounts of data collected on breast cancer from around the world. These include data on Ribonucleic Acid (RNA) signatures for cell mutations that cause breast cancer25.To address these issues surrounding the use of conventional ML algorithms has propelled the need for new approaches and methods to automatically extract features from large datasets. As a result, Deep Learning (DL) algorithms such as convolutional neural network (CNN or ConvNet) and Recurrent Neural Networks (RNNs) have emerged in recent times that can accept raw data and are automatically able to discover patterns in them28. CNN is an enhancement of a canonical Neural Networks architecture that is specifically designed for image recognition in29. Since then many variations have been added to the architecture of CNN to enhance its ability to produce remarkable solutions for deep learning problems such as AlexNet26, VGG Net27 and GoogLeNet30. CNN eliminates the need for manual feature extraction because the features are learned directly by different convolutional layers31. It does not require a separate feature extraction strategy which requires domain expert and other preprocessing techniques where complete features may still not be extracted32. Despite its huge success with image data, CNN is not designed to handle tabular non-image data in non-time series form. Note that all future referencing of non-image data are in non-time series form unless otherwise specified. Arguably, any problem that can represent the correlation of features of a given data example in a single map, maybe attempted via CNN.CNN is one of the most popular algorithms for deep learning which is mostly used for image classification, natural language processing, and time series forecasting. Its ability to extract and recognize the fine features has led to the state-of-the-art performance in various application domains such as computer vision, image recognition, speech recognition, natural and language processing33. This is attributed to the fact that the convolution technique in CNN requires data examples to have at least two dimensions. Conversely, CNN has been explored on application-specific 1-D data as well. These include gene sequencing data such as DNA sequences being treated as text data (sequence of words)34, and signals and sequences in text mining, word detection and natural language processing (NLP)36. More specifically, CNN for Time-Series Classification (TSC) has been recently explored with some new methods such as Multi-Scale CNN (MCNN)25 and an ensemble of CNN models with AlexNet on Inception-v4 architecture38. These methods have made significant improvement in the accuracy of the classifiers with the state-of-the-art ensemble methods such as Flat-COTE and HIVE-COTE40. Moreover, raw time-series data has also been used into 1-D CNN by calculating the area of the signal for convolution with better time complexity and scalability42. Nonetheless, much data still exists in a 1-D format such as clinical data of medical records, and therefore, opens challenging research questions on whether they can be effectively trained for classification using CNN. This paper is aimed at filling this gap by proposing a novel non-time series 1-D numerical data to 2-D data transformation methods and processing them with CNN. This would certainly help machine learners to train their data without being bothered about issues with feature extraction. This can also reduce a large feature vector to just a single image.CNNs have proven to work best on data that are in 2-D form, such as images and audio spectrograms10 to image data. Section “This paper is organized as follows: Section “43. The key difference between traditional ML and DL is in how features are extracted. Traditional ML approaches use handcrafted engineering features by applying several feature extraction algorithms and then apply the learning algorithms. On the other hand, in the case of DL, the features are learned automatically and are represented hierarchically at multiple levels. This is the strong point of DL against traditional machine learning approaches43.The main motivation for this paper is to realize the potential of CNN for non-image clinical data for breast cancer because it eliminates the need for manual feature extraction. The features are learned directly by CNN whereby it also produces state-of-the-art recognition resultsd-dimensional raw data has set of N, 1-D data vectors in d elements where d, N10 is a record of medical examination of patients to diagnose breast cancer, where each row is a 1-D vector representing a numerical data example. We demonstrate our method of non-image breast cancer data transformation to image data—processed in CNN—produces exceptional results for classification accuracy. Some research demonstrates the use of 1-D convolutions on 1D datasets such as data in the form of signals and time sequences44. Though this provides a possibility of using 1-D convolutions in this research, our experiments revealed their unsuitability on our experimental datasets. Having applied the data in its raw form into 1-D CNN gave highly unpredictable results.We have proposed three novel methods to transform non-image clinical tabular data of breast cancer to 2-D feature map images in mentclass2pt{minim10 for the classification of numerical data in this work.We have proposed three basic techniques of data wrangling to convert Breast Cancer numerical tabular data to image data. The converted image must reflect some patterns to depict a given class. We have used Wisconsin Original Breast Cancer (WBC) and Wisconsin Diagnostic Breast Cancer (WDBC) datasets from the UCI libraryd is total features, The bar graph represents the measurement of every feature of a given dataset. There are lots of possibilities of drawing a bar graph but we have used a simplistic approach. The dataset is first normalized to  then every feature is drawn based on its measured value. The width of the image in pixels is entclass1pt{minimaThese pictures are only useful to CNN if they depict a pattern in a convolved image. The first convolutional layer produces 6 features which are shown in Fig. 45 such as Genetic Algorithm (GA)46, Particle Swarm Optimization47 or Reincarnation Algorithm (RA)48 can be used to get the optimum order of bars based on their respective rank. Thereafter, a new set of images was created using this new order of bars. This process has been elaborated more in Section “Intuitively, the “correct” order of the bars ought to give better results. The datasets of numerical data were reorganized where the related fields were put close to each other according to the order of their similarity. Firstly, a covariance matrix on data fields was generated then each value of the matrix is converted to ‘rank’ that determines how closely one field is related to the other. This is a shortest-path problem where algorithms such as dynamic programming or any metaheuristic algorithmd represents total features of a given example. Matrix elements are the difference between two features i.e., The next method is the formation of a distance matrix which is a squared matrix of size entclass1pt{minimaApparently, the above two strategies can be combined to give a third option for generating an image from numerical data. We create a colored image of 3 layers of size The first convolutional layer in this case, is not able to produce any distinct feature but the scaled up image shows different colors with some bars in Fig. 50. In the case of a numerical dataset that is not in the form of images, first goes through the data wrangling process described in Section “CNN completes the classification process in two steps. The first step is the auto-feature extraction of the images and the second step is the classification of the same images with backpropagation neural networks10 for the experiments. The properties of these datasets are given in Table 51, Support Vector Machine (SVM)53, Decision Tree (DT)54 and Naïve Bayes (NB)55. These methods are generally supported by additional feature selection methods such as IG, Rough set or weight NB.The objective of the experiment is to provide an alternative classification method with CNN for the non-image dataset of Breast Cancer and other similar datasets without any need for manual feature selection. We have used WBC and WDBC datasets from the UCI library27 architecture with 4 convolutional blocks. Each convolutional block has 2D convolutional layer with the filter size of For CNN, we used VGG16px1 shows that the image is formed with bars of 1-pixel width only. Similarly px2 and px4 show width of 2 and 4 pixel sizes respectively for bars in an image.Initially, both datasets are divided into Additionally, it is highly desirable in medical diagnosis to have high sensitivity and specificity measures. Sensitivity is the ability of a test to correctly identify those with the disease, and specificity is correctly identifying those without the disease. Alternatively, the F1 score can be used as a derived metric that merges both sensitivity and precision measures. Tables 11 have used mammogram images of breast cancer as CNN works on images. In some cases, authors got 100% accuracy with 10-fold cross-validation for WBC dataset. Lower fold of cross-validation generally gives lower accuracy52.The comparison of our methods with other state-of-the-art methods is shown in Table px1, has been the most significant method as it carries the most information about the data in three dimensions of an image. It has outperformed other methods for the WDBC dataset by clocking 100% accuracy . It has also shown very competitive results for the WBC dataset with 99.27% accuracy and 1.0 sensitivity 0.99 specificity and 0.99 F1 score.The experimental results of data transformation from non-image tabular breast cancer datasets to image have been promising for the utilization of CNN for classification accuracy. Although the proposed methods are in the early stages, the obtained results are very significant in the development of new strategies with data wrangling for deep learning. This also provides an opportunity to derive even better alternatives for CNN in the future. It was observed that our proposed combined approach, i.e. Type-3 transformation and bar width of 1 pixel i.e. ith row with jth column of a given field. To get the “best” arrangement of fields, we minimize the total co-variance rank by using a meta-heuristic algorithm GA to solve this shortest path problem. The process of minimization for WDBC is shown in Fig. As discussed in Section “56 and parallel GPUs with enough memory can produce results in a reasonable time frame. The data wrangling process of converting non-image data to the image is not too expensive either. The every-case time complexity of the bar graph approach has the order of O(Nd) and the normalized distance matrix has the order of https://github.com/anuraganands.The only shortcoming of the CNN algorithm is its high processing cost than other methods, especially with bigger sized images. Generally, it takes 9–15 s for a MATLAB 2018 program to complete the training process on DELL XPS i7-9700 @ 3GHz machine with 8 CPUs and NVIDIA GEFORCE RTX 2060 GPU. Despite this, the experimental results demonstrate the size of data has no direct impact on the performance of CNN. Additionally, with the advent of quantum computingThe objective of this paper was to process non-image data (in a non-time series form) of Breast Cancer datasets WDBC and WBC into CNN due to its state-of-the-art performance and elimination of manual feature extraction for image recognition applications. The utilization of CNN has been confined largely to image data only except for some domain-specific data conversion techniques such as NLP and voice recognition. We have proposed three novel approaches to convert numerical non-time series data to image data. This process of conversion is very straightforward with the efficiency of the order of not more than


Last year Europe registered >365 000 excess deaths, most from preventable causes. In order to timely track deaths, the Portuguese Directorate-General of Health developed a deep neural network that codifies ICD-10 causes of death (AUTOCOD) by analyzing free text in a death certificate (DC). While the performance of AUTOCOD has been demonstrated, it was not clear if it was sustained during excess mortality periods, when text quality could be lower due to the increased pressure on health services.We performed a sensitivity analysis comparing the ICD-10 classifications of 330 098 Portuguese DC by AUTOCOD and by human-coders, from 2016 to 2019. Excess mortality was defined using the EuroMOMO methodology and a sub-analysis in periods of extreme excess (+4 and +6 SD). We compared the periods without excess mortality with the periods of excess and extreme mortality by chapter. The same analysis was performed for ICD-10 blocks, for the three most common chapters . The confusion matrixes allowed us to calculate AUTOCOD's performance metrics, like sensitivity.AUTOCOD showed high sensitivity (≥0.75) in 10 chapters, with values above 0.90 for the three most common ones. The weighted-average of sensitivity showed no difference between periods without excess mortality and periods of excess mortality, a difference of 0.01 for periods of extreme mortality (+4 SD) and a difference of 0.04 for periods of extreme mortality (+6 SD). For the block classification, performance was similar.Even in periods of excess and extreme mortality, AUTOCOD accurately predicts the classification of the cause of death. Meaning that it is not affected by a potential loss in text-quality due to pressure in health services. This allows for the use of AUTOCOD for real time mortality surveillance and it highlights the importance of Artificial Intelligence as an advisory tool for Public Health policies in emergencies.• Artificial Intelligence algorithms like AUTOCOD can predict the ICD-10 cause of death with very high sensitivity, during periods with and without excess mortality.• Artificial Intelligence algorithms like AUTOCOD can be used for real-time cause specific mortality surveillance, providing valuable information for policy making during periods of excess mortality.


In 2017 the amount of people globally affected by dementia was estimated about 50 million and is predicted to increase to 132 million by 2050. Coping with dementia is one of the most important challenges of governmental organizations’ agenda. The objective of this study is to analyse the hospital admissions trends and monitor the overall mortality in a population of older patients with dementia over the last two decades in northern Italy.This study is based on the healthcare utilization database of the Lombardy region , considering on hospital discharge records and death registry flows. Primary or secondary diagnosis at admission of dementia was coded according to the ICD9-CM. We carried out descriptive analyses of hospital admissions’ data from 1 Jan 1999 to 31 Dec 2020 of older patients aged 65 or more. We then conducted a temporal analysis of mortality rate over the study period.A total of 15,683,024 hospital admissions occurred during the study period. Over the last two decades, the prevalence of dementia among patients admitted to acute care hospitals remained around 1.1-1.3%. Considering the total of 183,268 patients with dementia over the study period, the average age at admission increased from 80.2 in 1999 to 83.3 years old in 2020, whereas annual mortality rate increased from about 1,950 to 3,230 deaths per 10,000 person-years. The mortality rate ratio of patients with versus without dementia fluctuated between 1.28 and 1.70.Our findings suggest that there is an ever-greater appropriateness of hospitalizations over the last two decades, supported by out-of-hospital care that led patients to hospitalization in increasingly late and serious phases of the disease. The present study has a great future potential as well as limitations, due to the dependence on a correct coding of cases by clinicians according to the ICD9-CM system.• We observed an increased mortality among older people with dementia admitted to hospital over the last two decades.• Our descriptive study, based on the Lombardy regional healthcare database, provides evidence of an increasing appropriateness of hospitalizations.