When the electric conductance of a nano-sized metal is measured at low temperatures, it often exhibits complex but reproducible patterns as a function of external magnetic fields called quantum fingerprints in electric conductance. Such complex patterns are due to quantum–mechanical interference of conduction electrons; when thermal disturbance is feeble and coherence of the electrons extends all over the sample, the quantum interference pattern reflects microscopic structures, such as crystalline defects and the shape of the sample, giving rise to complicated interference. Although the interference pattern carries such microscopic information, it looks so random that it has not been analysed. Here we show that machine learning allows us to decipher quantum fingerprints; fingerprint patterns in magneto-conductance are shown to be transcribed into spatial images of electron wave function intensities (WIs) in a sample by using generative machine learning. The output WIs reveal quantum interference states of conduction electrons, as well as sample shapes. The present result augments the human ability to identify quantum states, and it should allow microscopy of quantum nanostructures in materials by making use of quantum fingerprints. Scattering of electrons from defects and boundaries in mesoscopic samples is encoded in quantum interference patterns of magneto-conductance, but these patterns are difficult to interpret. Here the authors use machine learning to reconstruct electron wavefunction intensities and sample geometry from magneto-conductance data. The simplest case can be found in a ring sample where the wave propagation is completely restricted along the circumference. In such a sample, an electron circling along the ring gains a phase proportional to the magnetic field inside the ring. If the phase is an even multiple of π, constructive interference of the electrons enhances the wave nature and electric conductance. When the phase is an odd multiple of π, on the other hand, destructive interference reduces electric conductance, giving rise to a periodic oscillation of the conductance with respect to the external field, called an Aharonov-Bohm oscillation6. In a real nanometal, however, there are many scatterers for the electron waves, such as impurities and nanostructures , conductance was shown to be predicted from microscope information such as defect positions by using machine learning methods. Here we show that, by developing machine learning13 for quantum interference, fingerprint patterns in magneto-conductance can be transcribed into a real image of electron WIs and a sample shape.In metals at low temperatures, the quantum–mechanical wave nature of conduction electrons comes to the fore, which can be described in terms of their wave functions. The phase of the wave function can be modulated by a magnetic field, causing wave interference of electrons as a function of external fieldsres Fig. . As a re. 8 Fig. . These c15 (VAE)  Fig. . By traiAE) Fig. . In the To prepare a training dataset, we perform numerical calculation; Fig. G, for a sample as a function of the externally applied magnetic flux density B. Each conductance data exhibits different complex patterns. To highlight the patterns, we introduce the normalised magneto-conductance G over all the 7955 conductance data from the raw 16 with a width of 5 pixels is added to the left and right ends of each 60 × 50 pixel WI image, resulting in the final size of 60 × 60 pixels.Figure see Fig. . We also15; VAE was shown to compress its input and extract the information necessary to reproduce the input on the output of the network14. The network converts a WI image with 3600 pixels into a seven-dimensional latent-space data [(A) in Fig. The QGD is trained to generate correct sample-shape images and quantum interference patterns from (B) Fig. . To realmentclass2pt{minimThe QGD can directly generate WI images from magneto-conductance data }GB Fig. . First, Figure To check the performance of the feature extraction network [(A) + (B) in Fig. 17, which converts data points in the seven-dimensional latent space into those in a three-dimensional space while keeping their topological structure. Figure 14, VAE was found to construct well-organised data structures in the latent space  in the sample. In the network, information on the interference is compressed into the seven-dimensional latent space. Each WI image is convoluted into a Gaussian distribution in the latent space, and the set of the distribution forms a global geometric structure describing similarity among the states. We now visualise and discuss the geometric structure of the data in the latent space. To this end, we use a dimensionality reduction method called the universal manifold approximation and projection (UMAP)In order to understand the geometric structure, we performed a control experiment using geometry images without WIs  and blue (for odd x) around the red arrows in Fig. x at x values, the RMS difference for The dual-layered structure was found to be related clearly to the antidot location and interference; the upper layer consists of the data points for even In summary, we demonstrated the decipherment of quantum fingerprints in conductance by developing QGD. The QGD was found to transcribe the complicated magneto-conductance patterns into geometric information such as defect distributions and WIs in the samples. The WI patterns turned out to be encoded in the latent space of QGD as an extra dimension of the manifold representing the defect position information. It is truly worthwhile to tune the network by using data from real objects to show the versatility of the present method 20 and Leaky ReLUs21 are used as activation functions. To improve the learning performance, the batch normalisation technique22 is used. For the training and evaluation of the network, each dataset is split into training and test datasets with the ratio of 7 to 3. The loss function is the evidence lower bound14, and the optimisation algorithm is Adam23 with a learning rate of 0.0001. See Supplementary Figs. N is the number of samples in the test dataset, n-th numerical-calculated WI image input, n-th output image of the feature extraction network [(A) + (B) in Fig. The feature extraction network is based on a VAE network24 is used to improve the learning performance. The loss function is the mean squared error. For the training and evaluation of the network, the dataset is split into training and test datasets (corresponding WIs of the test data are not seen by the QGD network). See Supplementary Figs. n-th magneto-conductance input. The definition of the absolute difference image 25. In the present case, although the incident wavenumbers are limited to those on the Fermi surface, the spatial distribution of the potential is reproduced. This might be attributed to the fact that the potential shape is restricted to the antidot shape.The geometry generative network is a combination of a fully-connected network and the decoding network in the feature extraction network. The fully-connected network has 4 layers, where the dropout techniqueTo determine the appropriate dimension of the latent space, we performed control experiments. Supplementary Figure t to 2.0t at zero magnetic flux density. The data shows that the WI images can also be decoded with high fidelity when we use the Fermi energy dependence data instead of the magnetic flux density dependence.In Supplementary Figs. We quantitatively estimated the difference between Fig. In the present calculation, we chose the pixel size of the interference images so that the Fermi wavelength is equivalent to 2 pixels. In the condition, the data scattering in the latent space was found to apparently be quantised to show the most interpretable structure: the dual layers, which can be explained by the fact that a standing wave between an antidot and the sample end can be classified into two; destructive and constructive interference patterns. In fact, the dual-layer structure was found to change into a randomly scattered structure around the surface by changing the pixel size, showing that the essential feature here is the data scattering around the surface structure representing the information of the antidot positions.A and B are the original and generated WI images, respectively. As shown in Supplementary Fig. RNCC increases with the training epoch. RNCC after the training is much greater (RNCC > 0.997) than that before the training. We confirmed that the generated WI images show high fidelity in terms of the normalised cross correlation.We analysed the WI images in terms of the normalised cross correlation:Supplementary InformationPeer Review File


Human temporal bone specimens are used in experiments measuring the sound transfer of the middle ear, which is the standard method used in the development of active and passive middle ear implants. Statistical analyses of these experiments usually require that the TB samples are representative of the population of non-pathological middle ears. Specifically, this means that the specimens must be mechanically well-characterized. We present an in-depth statistical analysis of 478 data sets of middle ear transfer functions (METFs) from different laboratories. The data sets are preprocessed and various contributions to the variance of the data are evaluated. We then derive a statistical range as a reference against which individual METF measurements may be validated. The range is calculated as the two-sided 95% tolerance interval at audiological frequencies. In addition, the mean and 95% confidence interval of the mean are given as references for assessing the validity of a sample group. Finally, we provide a suggested procedure for measuring METFs using the methods described herein. System theory states that such systems are characterized in the frequency domain by their transfer function3, and the characteristic function here is commonly called the middle ear transfer function (METF). The METF can be defined as the ratio4. Actual sound pressure measurements inside the cochlea can be challenging, so other signals are often used. A common procedure, which is also the focus of this study, is to measure the displacement or velocity of the stapes footplate (SFP) with a laser Doppler vibrometer (LDV)5, and to derive the sought-after sound pressure from these signals. Note that this indirect approach may introduce additional error. For instance, single point LDV measurements may not capture higher order modes of footplate movement , which especially at higher frequencies  specimens are an important tool for the understanding of middle ear mechanics and the development of new surgical procedures, diagnostics, or devices for hearing rehabilitation. Transferring the insights gained by such studies into real-world applications, however, requires classifying the mechanical properties of the samples used in relation to the population. An important characteristic here is the sound transfer from the tympanic membrane (TM) through the ossicular chain (OC) into the inner ear, which for non-pathological human middle ears and sound pressures of up to about 124 dB SPL or below can be represented with reasonable accuracy by a one-dimensional, linear time-invariant model system8. This standard, which in accordance with general convention within the field of middle ear mechanics we shall simply call “ASTM standard”, gives a range of values for the (unaided) METF when measured using LDV, and states “Qualification Criteria” for temporal bones acceptable for further testing. These have since been widely used as validation in publications worldwide covering a range of topics in middle ear sound transfer, such as passive prostheses or reconstructive surgical techniques. However, it soon became apparent that a large number of TBs are ruled invalid by strict application of the original criteria. This was countered by modifying the criteria so as to allow for a more lenient validation process9. The cause for the discrepancy was not addressed until a paper published by Morse10, who showed that the statistical methods used in the ASTM standard are actually not well suited to derive the desired validation range for individual measurements.Around the turn of the millennium, measurements of sound transfer in temporal bone specimens using LDV were becoming more widespread, in part due to advances in implantable hearing devices. Quantifying the normalized output of such devices in ex vivo experiments was the goal of Rosowski et al., whose important work led to a standardized procedure adopted as “ASTM F2504_05” by the American Society for Testing Materials11 of a reference group of METF data as comparison12. At the time, we used a data set from our own lab only to create the reference. In order for the method to be more generally applicable, we now present a multicenter evaluation of individual METF measurements. A total of 478 frequency responses measured in intact TBs from five labs in China, Germany, USA and Switzerland were available for this study.The range given in the ASTM standard was calculated as the 95% confidence interval (CI) for the mean of means from 13 studies. This implies that it does not have a direct connection to the individual measurements that those means were calculated from. It could potentially be used as an estimate of where the mean of a new study is situated in comparison, but not for validation of individual temporal bones as has been practiced. Furthermore, the CI is proportional to the standard error of the mean and therefore to the inverse square root of the sample size Our main goal is to find a statistical range which may be used in the validation of METF measurements using sound pressure at the eardrum as the input signal and stapes footplate velocity (SFV) or displacement as the output signal, the latter measured using a single point of measurement near the center of the SFP. This range should be similar in practical application to the ASTM-range but it should be based on more representative statistics so as to be more reliable. We also aim to include all necessary materials and documentation so that the calculations described here can be extended with additional data or applied to other measurement methods.The data available for the study includes METF data of several international research groups, acquired using different methods of measurement. Therefore, we will first discuss the differences between the data from these different groups and methods . Informed consent was obtained from all subjects and/or their legal guardian(s). The TBs included in this study were either fresh and kept refrigerated in isotonic NaCl-solution or freshly frozen upon harvesting and subsequently defrosted immediately before the measurement. The following is a general description of preparation and setup pertaining to the most commonly used way of determining the METF in TB specimenslow see “”.Musculus stapedius and Musculus tensor tympani), the TM, and the inner ear intact. In the process, the middle ear is inspected by the surgeon for any abnormalities. Pathological mobility of the OC and any other issues pertaining to sound transmission (such as a damaged cochlea or TM) are noted if detected. Note that the methods for inspection may vary, commonly this includes visual and tactile inspection. Reflective foil or beads are then placed onto the SFP. LDV measurements are acquired at a single point near the center of the SFP. For accurate measurements, it is important to remove excess water droplets from the SFP to ensure a reliable LDV signal. At the same time, the TB should be kept moist for the duration of the experiment so as to avoid changes in sound transmission due to drying of tissue, which typically begin to occur after about 60 min of drying at room temperature14. For METF measurements of the intact OC, this is not an issue because these are done immediately upon commencement of the experiment and usually do not require more than an hour.Following basic preparation , access to the middle ear is accomplished via mastoidectomy and subsequent posterior tympanotomy. The cartilage and skin parts of the external ear canal are removed, leaving only the bony part of the external ear canal. The tympanic segment of the facial nerve is removed partially or completely in order to allow line-of-sight to the SFP for the LDV measurements. Care is taken to leave the whole ossicular chain, the attached ligaments and muscles , but when done correctly they can be assumed to be equivalent in outcome. This is because the METF is per definition normalized to input pressure, and the measured sound pressure in TB experiments is insensitive to exact microphone positionThe routines outlined above represent the most common procedure for determining the METF in human TBs. However, comparable results could be produced in a number of ways. Because this study includes data from an international set of laboratories, the specific conditions and methods for measurements differ. Additional descriptions of the methods used by the different research groups can be found below. Basic information on the different setups is summarized in Table The LDV used was an OLV1000 with an OLV700 sensor head . The LDV angle of incidence was not recorded but was documented to be The LDV used was either an HLV-1000 or an OFV-534/OFV-5000 . Instead of directly applying sound at the end of the bony part of the ear canal, the setup featured an ear funnel with a closed cavity cemented into the ear canal. This funnel and cavity was connected to the loudspeaker , the microphone , and an observation window. The probe microphone was inserted to about 1–2 mm in front of the TM. The stimulation signal generation and acquisition was either performed by a 16 bit Polytec data acquisition system  or a 24 bit USB-4431 data acquisition system  with a custom written software based on LabView . A multisine signal of block size 1024 was presented at a sample rate of 25.6 kHz, which comprised equal signal magnitudes at 125 Hz, 250 Hz, 500 Hz, 1 kHz, 2 kHz, 3 kHz, 4 kHz, 6 kHz, 8 kHz, and 10 kHz. Outside of these frequencies, the multisine signal contained no excitation energy. This allowed for quality checking of the measurements by calculating the signal to noise ratio (SNR) directly from the measurement file: the measured magnitude value at the frequency points above was divided by the mean value of the six adjacent points. Because of the narrow-band excitation signal, these adjacent frequency points do not contain any useful signal and were therefore deemed a good approximation of the background noise. Measurement values with an SNR greater than 12 dB were considered to be valid. Data points were commonly averaged ≥ 30 times. The LDV’s angle of incidence The group used a CLV 2534-4 LDV with a VIB-E-220 acquisition unit and corresponding VibSoft-20 acquisition software  for measuring SFV. The microphone  was inserted into the auditory canal along with the earphone  probe. The microphone probe was situated at about 1–2 mm in front of the TM. Excitation signals were single sine waves generated using a wave generator (Agilent 33210A 10 MHz) and an amplifier (B&K type 2718). LDV angle of incidence was not recorded but documented to be 15. Data from this group will be denoted as “Shanghai” and method “A”.Velocity measurements were averaged 30 times and quality-checked using SNR for each frequency, with a threshold of SNR > 10 dB below 1 kHz and SNR > 20 dB for higher frequenciesThe Zürich lab provided data in three distinct subgroups (see below). All TBs were prepared as per the general description above, but the outer ear canal was removed to about 2 mm from the tympanic annulus and replaced by an artificial ear canal of about 0.5 ml volume. This artificial ear canal also contained the earphone  and probe microphone . The tip of the probe was set to a distance of about 3–5 mm from the center of the TM. The excitation signal used was either a multisine signal or multiple single sine signals. The TBs were treated with 0.1% Cialit (sodium-2-ethylmercurimer-capto-benzoxazol-5-carbonide) as a preservative agent.The first subgroup was measured with a single point LDV as per the general description above Measurements from this subgroup were obtained using defrosted TBs only. The data is labeled as method “A”.The other two subgroups of data both used multiple (50–100) measurement points on the SFP using a scanning LDV system (Polytec OFV-3001), but differed in processing of the data. The second subgroup calculated SFV as the complex mean value of the velocities measured. Measurements from this subgroup were obtained using both fresh and defrosted TBs. The data is labeled as method “B”.16.The third group derived the piston-like component of the SFV from oval window LDV velocity scanning and X-ray microtomography (µCT). Upon fixating the reflective beads used for the SFV measurement in their respective positions, a 3D computer model of the TB was then rendered using µCT data. By combining SFV and µCT data, the 3D movement of the SFP and thus volume displacement was derived. For the sake of comparison, this derived data was then converted into an equivalent fundamental mode (piston-like) movement of the oval window . Measurements from this subgroup were obtained using fresh TBs only. The data is labeled as method “C”. Its methods are explained in more detail in the related original studysData from this group will be denoted as “Zürich” with further distinction by measurement method as previously defined.18 will be denoted as “Colorado”. Because the point of measurement is different, we label this data as method “D”.Rather than at the SFP, in this group the LDV  was pointed at the stapes capitulum (head of the stapes). The resulting measurement is therefore stapes head velocity (SHV) instead of SFV. The angle of incidence was fixed at approximately 19, using the IDE (integrated development environment) RStudio 1.2.5019. The raw data and the programming code containing the libraries and commands used are appended in the supplemental material.The processing of the data was undertaken using the data analysis software OriginLab 2019, and the subsequent analysis was done in the programming language R 3.6.1The raw data was first checked for integrity. The different groups used different methods for data quality management as described above . Frequency resolution in data acquisition varied between measurements, from multisine signals (64–1024 frequency bins at sample rates of 12–25.6 kHz) to superposition of specific frequencies and single sine wave excitement. In addition, different frequency ranges were used. We therefore selected the following frequencies to represent the data: 125 Hz, 250 Hz, 500 Hz, 1 kHz, 2 kHz, 3 kHz, 4 kHz, 6 kHz. These are commonly used frequencies in audiological measurements, and all of the data includes measurements in this frequency range. All measurements were interpolated to these frequency points as necessary. We used linear interpolation here because any higher order interpolation would introduce bias towards one of the two interpolation points. Analysis on the METF magnitudes was performed in logarithmic scale.For the analysis in R, the data is collected into data frames see Fig.  where ea20 to detect outliers is to extend the first and third quartile . We therefore use multiple Welch23 t-tests with 25. We are using the Holm method of sequentially rejective testing to avoid summation of type I errors 26 and denote the corresponding adjusted p value as The samples within groups are independent, and the groups are also independent. There is a dependence between frequency points 27 or Harrison28) for these parameters, which are divided into “fixed effects” and “random effects”. In the present case, the parameter “frequency” as a within-groups factor was declared as a fixed effect variable, assuming that all METFs follow a similar frequency dependent pattern. The typical frequency response of the METF as known from literature suggests modeling as a 3rd order polynomial. However, we are also comparing higher order models to see whether accuracy can thus be improved. The parameters “measurement method” and “research group” as between-groups factors are considered random effect variables for the purpose of this evaluation. The inter-specimen variance also needs to be taken into account here and was introduced as random effect “TB”. Random effects are modeled as random intercepts. We use a crossed random effects model because for some levels of the factor “research group” there are multiple factor levels “measurement method”, therefore the use of a nested design as explained by Krzywinski29 is not appropriate. The LMM was implemented with the R package “LME4”30.The data is to be used in constructing a general reference range for the validation of METF measurements. This requires proof that any unspecified covariables have a negligible effect. It is therefore especially important to discern how the different parameters contribute to the overall variance. A method well suited to the task is to create a multivariate linear mixed model , there are no significant differences  between research groups. However, there are some significant mean differences between some research groups at the low and high end of the spectrum see Fig. . At \docAs described above see “” even thUp to 1 kHz, Method “D” results are similar to those for method “A” but do not show the typical drop with increasing frequency  distribution see Fig. . This isEven though all data has been diligently acquired, measurement errors cannot be ruled out. In addition, there is no way of directly identifying false data, because errors present in the signals may interfere with (“hide”) one another, so that the resulting METF appears normal. However, there are some clusters of outliers which are possibly measurement errors. Using 1.5 IQR as exclusion criteria would eliminate these, but due to the negative skewness of the distribution, also a large part of the total (53 of 401). The choice of 1.75 IQR is more lenient and excludes 35 of the 401 samples, therefore it seems to offer a better balance between excluding obvious errors and including TBs which are untypical but still normal.Note that we are trying to establish a range of sound transfer in normal ears. It is possible that there are individuals in which sound conduction is very different from the rest of the population, but who still have normal hearing. However, we cannot confirm this because there is no assignment of hearing tests to temporal bones. Therefore, to be on the safe side, we have to assume that outliers are the result of undetected pathologies or measurement errors.From the Welch T-tests between groups belonging to method "A", we can see there is a lab effect . It follows that the data for method “A” from all research groups may be combined into a common data set from which to derive the desired reference range. By the same logic, this range is then also applicable to new data from any other lab, provided the measurement methodology is documented in sufficient detail and is consistent with method "A" described here.The LMM analysis gives a deeper understanding of these notions:37, which contributes to the large interspecimen variance. A parametric evaluation of the METF data as published by Gladiné38 may therefore be an interesting subject of further research.A recently published modal analysis of METF measurements shows a rather large standard deviation for the frequency distribution of the natural frequencies of the middle ear39, but sound transmission may be affected by other structural changes due to freezing/thawing43. It is as of yet unclear whether or how exactly age at death correlates with the METFs measured in TBs. Simulations suggest that there may be symptomatic changes44, and there are some studies that have examined the age dependence of sound transfer characteristics46. It has been shown that sex influences both the anatomy47 and middle ear function48. However, in all of these cases the effect on the results of this study is likely small, as evidenced by the small overall lab effect. Note that any change leading to pathological behavior (such as perilymph leakage or hypermobility due to damaged ligaments) disqualifies TBs in the preselection process8. TB state, sex and age impact on the METF may be the focus of future studies, possibly leading to separate validation ranges for fresh and defrosted TBs, for different sexes, or for different age ranges.The reasons for the detected lab effect are confounding variables that are not covered by the level of detail of the experiment definition. In other words, the lab effect collects all unexplained differences between the setup and procedure for the different research groups. Some variables that may be worth investigating further are TB state (fresh or frozen), sex, and age of the donor at time of death. It has been shown that input impedances of the middle ear are unaffected even by prolonged freezing14 because at the observed frequency range, the wavelengths of the sound waves are far larger than this. An impact on the results is therefore unlikely, because only the Zürich group potentially contained data measured at a distance of up to 5 mm. It may be advisable to narrow this down further in future studies if possible. The point of measurement on the SFP may account for a part of the variability, but this cannot be quantified on the basis of the data. A recent study by Cheng et al. indicates that nonlinear transmission effects may occur at sound pressure levels as low as 100 dB SPL, with expansive behavior dominant in the frequency range below 2 kHz and compressive behavior towards higher frequencies49. The METF measurements used here use excitement levels of about 90–100 dB SPL. Even though occasionally higher SPL may be used to acquire a better signal, the effects are likely very small at this point.METF measurements in TBs are insensitive to varying microphone positioning up to 4 mm from the umboIt can be expected that the lab effect would be smaller with a stricter definition of the experimental design for obtaining METF measurements, such as including an age range for the specimens, more exact measurements of microphone position, point of measurement on the SFP, adhering to an identical preparation method and application of the loudspeaker etc. However, it is much more practical to provide a validation range for the less restrictive setup definition as this is immediately applicable for a larger number of research teams. Having proven that the lab effect is relatively small despite the variability between setups also justifies this approach.It must be noted that at the 125 Hz sampling point, the Hannover data dominates the calculations due to the higher number of valid data points. At 6 kHz, the data from Shanghai is significantly lower than the rest. Therefore, the reference intervals are somewhat less reliable at the low and high end of the range, which is addressed in Fig. 8 which is shown in red dashed lines as a comparison. This is approximately in line with the "revised rules" of Rosowski9, which increases the range by 20% and states that deviations from the ASTM range by up to ± 6 dB may be acceptable. In contrast to this rule, which was inferred from personal experience, the TI range and mean presented here provide an objective basis. As has been pointed out10, the CI (on which the ASTM standard is based) is inversely proportional to the square root of Compared to the ASTM—where the reference range is calculated as a CI of the mean—the CI of the mean in the present study is a much tighter fit around the mean . This is important to note for any future study. Second, the differences in outcome between the chosen 95% TI and the other TIs are relatively small. This means that in practice, there would be little difference in using a proportion of 90% or 99% instead. We therefore chose the 95% TI as the proposed reference range. While we realize that this choice is arbitrary, we feel that it is a good compromise between ruling Due to the large number of available data sets, the 95% confidence interval of the mean is very narrow, implying that the study mean is highly likely to accurately represent the mean of the population as a whole. This allows one to compare a new study’s mean to the reference mean. Note that it is unlikely that the mean of a new study using a smaller sample size would fall within this range. However, comparing the study mean with the reference mean can give insight on how well the new study population matches the whole population. This comparison should take into account a possible lab effect, which by the reference data can be expected not to exceed 6 dB. A larger difference between the means might be worth investigating.Measure all METFs in a way similar to the procedure outlined above.In short, use a single point LDV near the center of the SFP to acquire the output signal and a probe microphone in front of the TM to acquire the input signal. Ensure that the LDV angle as measured from the normal on the SFP is as small as possible and note this angle along with the data.Preprocess your data.Correct for the LDV angle if possible. Check the quality of the data (e.g. using coherence or SNR). As long as data quality is high in the frequency band of 500–4000 Hz, low quality data points at the high or low end of the spectrum (lower than 500 Hz or higher than 4000 Hz) may be truncated depending on the questions at hand.Compare each METF in your study to the reference range. If an METF exceeds the range in one or more places, take note.Bear in mind that the ranges are only a tool to alert the researcher to an atypical measurement. It is up to you to determine whether this is due to a hitherto undetected pathology, a measurement error, or other causes. In some cases, the measurements from this TB may still be used, but this should be justified by the purpose of the study and discussed. In case of doubt, we suggest excluding the data set from the group statistics. Note that narrow band resonances/antiresonances may result in peaks/notches that exceed the range for an otherwise normal METF; these may be ignored but it may be advisable not to draw any conclusions from the data immediately at or around the resonance. If possible, use excitation signals with high frequency resolution  to better differentiate narrow band resonances from wider deviations.Be aware that single measurements towards the (upper or lower) edges of the range may have a large impact on your analysis.As a rule of thumb, we propose that studies using statistical analysis use Use your own judgment for case studies or special problems.The reference is intended for use in studies with a statistical analysis . If the measurements are to be examined as individual cases only, other criteria must be used to assess the validity of the measurements. The same applies if the target of your study is to examine middle ears with pathological aberration that explains the deviation from the range.If an METF deviates only at the high or low end of the frequency range, it may be truncated instead of discarded.This greatly depends on the target of the study, but any deviation with Compare the mean of your study to the reference mean.This can give an estimate of how well your study represents the whole population on average. Note that for small studies, it is highly unlikely that your mean falls inside the 95% CI of the reference mean. As a rule of thumb, if the mean of your study deviates from the reference mean by more than 6 dB, the possible implications thereof should be addressed in discussions.We have derived statistical ranges for METF magnitudes from a population of TBs with normal, non-pathological sound transfer characteristics. These intervals are intended as a reference for future studies in a variety of contexts , reconstructive surgery or basic research) using similar methods. They allow validation of individual TBs and assessment of the sample group. For such use, we suggest the following guidelines.The reference range and mean presented here are calculated from 362 METF measurements provided by 4 research groups. Its primary use is for such research where only a small number of measurements must be validated. In cases where a large number of data sets from the same experimental setup is available, it may be advisable to use the methods provided to calculate a reference range for this specific setup.The original data, as well as the calculation scripts used, are attached and summarized in the supplemental material so that it may be compared, reproduced, or updated with future data. The statistical methods used can also be adapted for use with different measurement methods, or indeed for use in other experiments where transfer functions are subject to large anatomical variations.Supplementary Information 1.Supplementary Information 2.Supplementary Information 3.Supplementary Information 4.Supplementary Information 5.Supplementary Information 6.


Machine-learning prediction algorithms such as AlphaFold and RoseTTAFold can create remarkably accurate protein models, but these models usually have some regions that are predicted with low confidence or poor accuracy. We hypothesized that by implicitly including new experimental information such as a density map, a greater portion of a model could be predicted accurately, and that this might synergistically improve parts of the model that were not fully addressed by either machine learning or experiment alone. An iterative procedure was developed in which AlphaFold models are automatically rebuilt on the basis of experimental density maps and the rebuilt models are used as templates in new AlphaFold predictions. We show that including experimental information improves prediction beyond the improvement obtained with simple rebuilding guided by the experimental data. This procedure for AlphaFold modeling with density has been incorporated into an automated procedure for interpretation of crystallographic and electron cryo-microscopy maps. This paper presents an iterative procedure where AlphaFold models are automatically rebuilt on the basis of experimental density maps and the rebuilt models are used as templates in new AlphaFold predictions. The AlphaFold1 and RoseTTAFold2 algorithms, for example, can create accurate predictions for substantial regions of a protein structure that are based on the amino acid sequence of that protein and on residue covariation information7 present in a multiple sequence alignment1. Prediction can be augmented by including experimentally determined structures of proteins with similar sequences as templates1. In many cases, the predicted models are accurate enough to allow straightforward experimental structure determination using molecular replacement in macromolecular crystallography or by docking a structure in a density map in single-particle electron cryo-microscopy (cryo-EM), without requiring that a similar structure has been previously determined8.Advanced machine-learning-based structure prediction algorithms are transforming the way that three-dimensional structures of proteins and their complexes are obtained9. In particular, machine-learning methods typically do not yield accurate predictions for all of the residues in a protein10. This is partly due to the presence of disordered segments in many proteins11, but is also due to the limited size and accuracy of multiple sequence alignments for part or all of some protein sequences, resulting in a limited amount of available information about residue covariation1. A related limitation is that parts of proteins that can adopt alternative conformations may be systematically predicted in only one of them12; this limitation may be reduced by alternative sampling of multiple sequence alignments12. Additionally, individual domains of proteins are often predicted accurately, but in the absence of extensive conserved interaction surfaces the spatial relationship between domains cannot be unambiguously predicted with current methods3. A final limitation is that as these machine-learning methods are trained on structures in the Protein Data Bank (PDB)1, predictions are likely to be biased towards these known structures even if they are not included explicitly as templates in prediction.There are limitations in using predicted models for structure determinationα13) for each residue in a protein and reports these estimates as a confidence measure, plDDT1. Validation with known structures demonstrated that these AlphaFold plDDT values are reasonably good indicators of actual accuracy ).A strength of recent machine-learning algorithms for protein structure prediction is that they can assess the accuracies of their own predictions. AlphaFold, for example, estimates the value of a commonly used measure of prediction accuracy  on a cryo-EM density map, and rebuilt to better match the map. The rebuilt model was then supplied along with the sequence to AlphaFold in a new cycle of prediction. The output of this procedure is a new AlphaFold model that has incorporated new experimental information through the use of the rebuilt template in the prediction. We applied four cycles of the iterative algorithm to the sequence of one protein chain and the full density map for each of 25 cryo-EM structures, all deposited after the training database for the version of AlphaFold we used was created (July 2020). In these tests, multiple sequence alignments were included in each stage of AlphaFold modeling. To emulate the situation where no similar structure is present in the PDB, templates from the PDB were not used. For each protein we then examined the four AlphaFold models obtained (one for each cycle of modeling), comparing them to the corresponding deposited model (used as our best estimate of the true structure) and to the corresponding deposited density map.16  entry 23914, PDB entry 7MLZ; only the spike protein is analyzed here). The five-stranded β-sheet . This rebuilt model was used as a template in AlphaFold modeling, with the goal of providing the inference procedure with some additional information about which parts of the structure are close together, and the rebuilding and modeling were repeated for a total of four iterations. The AlphaFold model obtained after iterative prediction and rebuilding is shown in green in Fig. α atoms in the deposited model were matched within 3 Å by a Cα atom in the superimposed AlphaFold model. The overall map correlation for AlphaFold model obtained after iterative prediction and rebuilding is 0.57. Note that unlike the rebuilt model, the AlphaFold predicted model shown in Fig. Figure eet Fig. , where tOverall, Fig. 17 is shown in Fig. 23883 at a resolution of 3.0 Å. The superimposed predicted model does not match the density well, and the r.m.s.d. of all matching Cα atoms from the deposited model is 3.6 Å. Figure α atoms from the deposited model of just 0.8 Å.Figure α atoms in each AlphaFold model superposing within 3 Å of a Cα atom in the corresponding deposited model . The abscissa in Fig. α atoms superposing within 3 Å from 12 to 20. This set of models is assessed on the basis of r.m.s.d. of matching Cα atoms in Fig. Figure 18 that is based on many of the same tools used here in model rebuilding, but without including AlphaFold at all (ordinate). All but one of the iterative AlphaFold models are more accurate than the corresponding models created by automatic model building alone.Figure 19  included in the CASP-14 structure prediction competition20 (target identification of T1047s2-D3). The PDB entry with the most similar sequence (PDB entry 2HM2) present at the time of AlphaFold training has a sequence identity of just 9% and has a very different structure20. Parts of the structure of this domain from the basal body are accurately predicted by AlphaFold20; however, there was a substantial difference in the arrangement of two antiparallel strands relative to the cryo-EM structure, as well as a small difference in the position of a helix  structure to test whether iterative AlphaFold prediction and model rebuilding would be effective in a case where AlphaFold was trained without any similar structures. In this test, fragments from a model automatically built from the density map were included in model rebuilding, and multiple sequence alignments were only used in the first cycle of AlphaFold modeling. These options were chosen to improve model rebuilding and to allow the conformations of the rebuilt models to guide the AlphaFold prediction. Figure 7BGL) model  and by percentile-based spread, which de-emphasizes large discrepancies21 (0.3 Å versus 2.0 Å). It is similar to, but somewhat more accurate than, the initial rebuilt model . To check that the improvement in prediction with iteration was not simply due to leaving out the multiple sequence alignment in predictions after the first, we carried out AlphaFold modeling without a multiple sequence alignment and without information from the map. This resulted in a prediction that was quite different from that of the deposited model . These observations show that the synergy in iterative AlphaFold modeling and model rebuilding using a density map can be obtained even if AlphaFold is trained in the absence of any similar structures. Three cycles of iteration were used in this example.We used the flagellar basal body , while the AlphaFold prediction in CASP-14 used a monomer, and we also used the sequence of a monomer in our analysis. It seemed possible that some of the sequence covariation present in the multiple sequence alignment would be due to inter-subunit contacts, and that if we supplied a sequence corresponding to a homo-oligomer, AlphaFold might be able to use this inter-subunit contact information to create a more accurate model of each individual chain. We carried out predictions with a trimer and a dodecamer of the 22 using the experimental density map for the SARS-CoV-2 spike protein structure illustrated in Fig. α atoms in the corresponding deposited model within 3 Å. A corresponding analysis using a cutoff of 2 Å is shown in Extended Data Fig. An immediate application of iterative prediction and model rebuilding is automatic analyses of cryo-EM or crystallographic density maps. Though tools exist for this purpose, automatic map interpretation is challenging, particularly when high-resolution maps are not available. For example, Fig. 23 or rebuilding as described above. The resulting refined or rebuilt model is an automatically generated interpretation of the corresponding part of the density map. Our procedure can therefore also be viewed as a method to automatically interpret a density map, incorporating information from a density map into AlphaFold modeling in the process.The output of the iterative AlphaFold modeling and map-based rebuilding process described above is a predicted AlphaFold model that is already positioned to match the density in a map. The predicted model may still require some adjustment indicated by the density map, and such adjustment can be carried out by automatic refinement7LSX and 7LS5, the antibody heavy and light chains in 7MJS, and the αβγδ histones in 7LV9). In these cases, a match was allowed to whichever chain matched the location the automatic docking had chosen . For each structure and density map, Fig. α atoms for the initial AlphaFold model (superimposed on the deposited chain with secondary-structure matching) and the four automatically docked and iteratively rebuilt models. The structures are arranged on the basis of the resolutions of the corresponding maps, with finer (higher) resolution on the left and coarser (lower) resolution on the right. The SARS-Cov-2 spike protein structure16 shown in Fig. 7MLZ in Fig. α atoms in the deposited model matched by the rebuilt model and improves with each cycle of rebuilding until the next-to-last cycle, where 92% are matched, and no additional improvement is obtained on the final cycle. Others that improve substantially include 7M7B (improving from 59% to 77% matched), 7LX5 (75% to 91%), and 7LCI (73% to 95%). In 18 of 25 cases a model matching at least 95% of Cα atoms in the deposited structure within 3 Å was obtained; this level of accuracy was present in only 11 of the starting AlphaFold models. Two of the cases (7LV9 and 7MSW) yielded very poor models , in others, several cycles of improvement occur and then no further improvement is found , and in others little improvement is obtained at all or the quality even decreases . Overall, the number of cycles over which improvement occurs appears to vary considerably and it may be useful to continue iteration until the changes in the model are small. It seems possible that the variability may be due to failure of our rebuilding tools to find the correct conformation of parts that need rebuilding and that improvements or new methods of rebuilding may improve the overall process considerably.We note that in some cases improvement occurs mostly in the first cycle  to be of only minimal overall utility in improving the accuracy of AlphaFold modelsThe examples described here range in resolution from 2.4 Å to 4.5 Å, and we expect that the overall approach will apply most effectively to structures at resolutions of about 4.5 Å or better because this is the range of resolutions where model rebuilding is currently most effective. As model-building tools improve, the low-resolution range where the procedure is useful may be further extended. Additionally, the low-resolution range may possibly be extended with existing tools by only carrying out refinement and not attempting to rebuild parts of the structure that do not match the density map.7MLZ and 7LX5). These differ in residues at the ends of the chain and differ also in that the SARS-Cov-2 spike protein  is bound to different antibodies in the two structures. The PDB and EMDB accession numbers for these 25 structures are listed in Extended Data Table The 25 maps and corresponding models shown in Figs 27, and for which experimental data is available in the EMDB and PDB. This structure was PDB entry 7BGL (ref. 19), EMDB entry 12183. We chose domain 3 of chain a in this structure as AlphaFold performed poorly on this target in CASP-14 (rank of 78) compared to most other targets .The choice of structure and map to test model creation using AlphaFold trained without similar sequences in the PDB was made by selecting the (one) structure in CASP-14 that was determined by cryo-EM, classified as a ‘hard’ target in CASP-14We note that the software was developed at the same time as the analysis and used some of the samples in the analysis in development. Consequently, it is possible that even though the same code and parameters are used for all the work shown here there may be choices made in parameters that improved results for these cases but that might not improve them for a completely new set of structures.28 v.1.2.5.Figures were prepared with ChimeraX7BGL structure in Fig. For the analyses shown in Figs https://colab.research.google.com/) AlphaFold2 notebook, followed by downloading the resulting model and automatically trimming, docking and rebuilding the model with the density map and the Phenix tool dock_and_rebuild. Subsequent cycles consisted of converting the rebuilt model to mmCIF format29, uploading the model to the Colab notebook, generating a new AlphaFold model using the rebuilt model as a template, and rebuilding as in the first cycle. A total of four cycles were carried out. We considered the last AlphaFold model obtained in this procedure to be the AlphaFold model created with information from a density map, and the last rebuilt model to be the overall final model produced by the procedure.The first cycle of our iterative procedure consists of creating an AlphaFold model using a Google Colab  supplied by AlphaFold for each residue in the structure. The Phenix tool process_predicted_model is used for this purpose. Residues with plDDT value less than 70 (the threshold for a ‘good’ prediction1) are removed and the remaining residues are grouped into ‘domains’  consisting of one or more parts of the chain that contain a sufficient number of residues  and form a compact unit. This grouping can be carried out based on spatial proximity (default), or on the basis of the predicted uncertainties in Cα–Cα distances. We note that in cycles after the first, a template is supplied that derives in part from the previous AlphaFold model, resulting in systematically higher plDDT values. In this work we have not quantified this effect or adjusted the threshold to account for it.AlphaFold models are automatically trimmed and split into domains on the basis of the coordinates of the AlphaFold model and on estimates of confidence  sufficiently large  the other is skipped. On the hypothesis that the transformations for different domains may often be similar, the transformations for successfully docked domains are considered as possible transformations for each additional domain. These methods typically yield a set of possible placements of each domain in the map. If symmetry is automatically detected in the map24, these placements also include all the possibilities obtained by applying this symmetry to placements found directly.The compact groups of residues (‘domains’) obtained by trimming the AlphaFold model are aligned, one at a time, to the density map. Two approaches are used. The first approach uses secondary-structure matching (SSM) to dock the domain onto the map using the Phenix tool superpose_and_morph with the setting ‘ssm_match_to_map = True’ (see below for details of this tool). The second approach consists of a direct correlation search between model-based density and the map using the Phenix tool dock_in_mapα atoms in one domain overlapping with those in another domain within 3 Å (controlled by the parameter ‘overlap_ca_ca_distance’). The score starts out at zero. If the map correlation for each domain is at least 0.15 (‘minimum_docking_cc’) the score is given large positive increases (200 units) for each of the following that occur: (1) lowest map correlation of all docked domains is greater than 0.5 (set with ‘acceptable_docking_cc’); (2) if (1) occurs, and all placements also have similar transformations , where two transformations are similar if applying them to a domain gives an r.m.s. difference in coordinates equal to the resolution of the map or less; (3) all domains are docked; (4) the fraction of residues that overlap between domains is less than 0.1 ; and (5) no domains are further apart than can be spanned by the number of residues between those domains. If any domains are further apart than can be spanned by the number of residues between those domains plus twice the resolution plus 15 Å (‘maximum_connectivity_deviation’), 200 units are subtracted from the score. The resulting score is then adjusted with the following additions and subtractions: (1) the lowest map correlation of all domains is added; (2) the average map correlation is added; (3) the fraction of transformations that are different from the first is subtracted; (4) the fraction of Cα atoms that overlap between domains is subtracted; and (5) the sum of all deviations in distances between domains, normalized to the sum of all allowed distances between domains, is subtracted. This scoring function was not optimized and does not contain weights except as described above.The final inclusion and placement of each domain is then chosen by maximizing an empirical scoring function. The function includes the fraction of domains that are placed and the map correlation for each placement. It also includes a penalty for placing two domains further apart than can be spanned by the number of residues between those domains, and a penalty function for the number of C31, creating a vector function that varies smoothly in space. The shift (distortion) applied to a particular atom in a model is the value of the shift field at the coordinates of that atom.Once a set of domains is placed to match a map, the entire AlphaFold model is morphed to superimpose on these domains as much as possible, while smoothly distorting along the chain between domains. We use a shift-field approach to morphingα atom in a docked domain. The value of the shift coordinate is the position of the corresponding Cα atom in the full AlphaFold model. The value of the shift vector is the difference between the coordinate of the Cα atom in the docked model and the corresponding Cα atom in the full AlphaFold model. The shift field at any point in space is then the weighted average of all the shift vectors, where the weights are the inverse exponential of the normalized squared distance between that point in space and the corresponding shift coordinate, and where the normalization is the square of the shift-field distance, which has a typical value of 10 Å (set with the parameter ‘shift_field_distance’ and chosen to be a compromise between maintaining the model geometry with a long shift-field distance and matching the docked domains closely with a a short one). The coordinates of a morphed AlphaFold model are then calculated from the initial coordinates and this shift field. This morphing has the property that local distortions occur on a scale of about 10 Å, the shift-field distance. The docked, morphed AlphaFold model is adjusted to match the map using the refinement tool real_space_refine23.The shift field is calculated from a set of  pairs. There is one such pair for each Cα atoms are noted, the values in the lower half are removed, and the mean and standard deviation of remaining (‘good’) density values are noted. Low map correlation is defined as more than three standard deviations below the mean (where the ratio of three is defined by the parameter ‘cc_sd_ratio’). Before applying these thresholds, the plDDT values and density values for each residue are smoothed by averaging with a window of ten residues along the chain (defined by ‘minimum_domain_length’).The parts of the docked, morphed and refined AlphaFold model that have either (1) low confidence predictions from AlphaFold , or (2) low correlation with the map, are then identified and used to specify segments of the model that require rebuilding. The threshold defining low map correlation is obtained with the following procedure. Density values in the map at positions of all C18; (4) a combination of retracing part of the loop with superimposing and splicing that part of the existing refined model that matches the remainder of the loop; (5) iterative morphing; and (6) use of an external model. The combination method addresses the situation where clear density is present in the map for the beginning and end of a loop and the remainder is unclear. In this case, the refined model for the residues that cannot be modeled from density are simply grafted onto the residues that can be modeled, using a shift-field procedure as described above to morph the refined model while superimposing three residues on each end. The iterative morphing procedure was similar to one previously used to distort a model to better match the density32, but in the current procedure morphing is carried out on six residues from each end at a time (specified by ‘n_window’), then the remainder of the model is superimposed on the 12 morphed residues, the window is shifted by one residue from either end, and the process is repeated until the loop is morphed. In cases where an externally created model has been supplied to the rebuilding procedure, another attempt to rebuild each loop consisted of selecting a matching segment from the external model, if such a segment with the expected number of residues was present and could be connected to the existing model with deviations at the ends of 3.8 Å or less (defined by the parameter ‘ca_distance’). Each attempt to rebuild a part of the refined model yields a new candidate segment of the model. All the candidate segments obtained with a particular rebuilding method  are used to replace the corresponding segments in the refined model and the resulting full model is refined on the basis of the density map. This overall process then yields several new full-length versions of the model.Then a series of attempts to improve the fit of each poorly fitting segment to the map are carried out. These attempts to improve the fit include: (1) iterative resolution refinement, in which the model is iteratively refined, initially at low resolution , then progressing in 1 Å decrements until the resolution of the map is reached; (2) rebuilding of loops, using the Phenix tool ‘fit_loops’; (3) retracing loops by finding a path through the density map that connects the ends of the loop with a chain that follows the path with the highest minimum valueThe rebuilt and refined models are then used as hypotheses for the structure to be built. In the preceding step, boundaries of regions needing or not needing rebuilding were identified. In this step, each model is broken up into the corresponding segments. Then the best version of each segment, chosen on the basis of their map correlation, is used to create a new full model. This model is refined using the density map to produce a single full-length final model.7BGL structure19 in Fig. 18 as a source of possible fragments to use in rebuilding the structure. The reason this was necessary was that without these fragments, model rebuilding with the methods described below was incomplete for this structure despite the very good resolution of 2.2 Å, possibly because the AlphaFold model was quite different from the actual structure in some places. The other exception was that in cases where multiple chains with similar sequences (and therefore presumably similar structures) were present in a structure, the secondary-structure-based docking procedure was skipped and only a direct density correlation search was used (with the Phenix tool dock_in_map). The rationale for this was that, as might be expected, docking with a correlation search was more effective than a secondary-structure search at distinguishing the correct placement from one superimposing on related but different chain.Default values were used for the parameters controlling model rebuilding, with two exceptions. One exception was that for the In the first cycle of rebuilding for each model, the corresponding AlphaFold model was supplied along with the full corresponding density map and the resolution of the structure reported in the PDB. In subsequent cycles, a new AlphaFold model was supplied as well as the rebuilt model from the previous cycle.25.Models were superimposed using the Phenix tools superpose_pdbs, superpose_and_morph and the Coot secondary-structure matching toolα atoms identified by alignment of the sequences of two models. Note that in cases where the sequences of two models are similar and the models differ largely by rigid-body movement of one domain relative to another, this procedure can lead to a superposition where neither domain superimposes closely.The superpose_pdbs tool carries out least-squares superposition of matching C33. If the option ‘ssm_match_to_map’ is used, the inputs are a model and a map. In this case the tool find_helices_strands is used to find secondary-structure elements in the map and to create a secondary-structure model containing these secondary-structure elements. Then the model to be docked is superimposed on the a secondary-structure model with a modified form of SSM. In this SSM procedure, two secondary-structure elements from the map  are paired with two matching elements from the domain to be docked , thereby defining a transformation between the domain to be docked in the map. As the precise alignment of secondary-structure elements from the map and those from the domain to be docked is not known, all possible alignments of the shorter of each pair of elements with the longer element are tested . All the Cα atoms in each element from the map are then associated with Cα atoms in the corresponding element from the domain, and a least-squares superposition is carried out. If these Cα atoms match , the resulting transformation is applied to all Cα atoms in the domain and the map-model correlation of the resulting docked domain is calculated with the Phenix tool map_model_cc. If the resulting correlation is above a minimum level , the docked model is adjusted by rigid-body refinement to maximize this correlation.The superpose_and_morph tool carries out SSM to superimpose part or all of one model on another using reduced representations of secondary-structure elements and indexing of these elements to speed up comparisons and allowing matches that are non-sequential in a procedure similar to that used inα atoms in a target model that are matched within 3 Å by any Cα atom in the matching model. Allowing any Cα atom in the matching model to superimpose effectively ignores the connectivity of the chains, but it is useful for evaluating whether a Cα atom is placed in a position where some Cα atom belongs. The distance of 3 Å is the default value and is useful for ranking pairs of models that have more than about 30% of Cα atoms matching. It is less useful for ranking pairs with lower similarity because two overlapping structures that are completely unrelated will often have 20–30% of Cα atoms matching within 3 Å.The Phenix chain_comparison tool was used to compare models that were already superimposed. This tool counts the number of C7MLZ and resulting AlphaFold and rebuilt models. The overall orientation and positions of AlphaFold models are arbitrary and the values in the atomic displacement parameter field  are plDDT values. We superimposed these models on the corresponding deposited structure before calculation of map correlations, keeping all coordinates fixed at the values obtained by direct superposition. To make a fair comparison with rebuilt and deposited models, we refined the atomic displacement parameters for all the models to match the map before calculation of map correlations. For the 7MLZ example shown in Fig. B-values increased all the map correlation values. Map correlation values for the deposited model, initial superposed AlphaFold model , initial rebuilt model and final AlphaFold model were 0.64, 0.26, 0.47 and 0.44, respectively. After B-value refinement these were 0.70, 0.41, 0.58 and 0.57, respectively.We used the Phenix tool map_model_cc to calculate correlations between experimental density maps and model-based density maps for PDB entry Further information on research design is available in the Any methods, additional references, Nature Research reporting summaries, source data, extended data, supplementary information, acknowledgements, peer review information; details of author contributions and competing interests; and statements of data and code availability are available at 10.1038/s41592-022-01645-6.Reporting Summary


The predominance of artificial intelligence (AI) will be the next Industrial Revolution. All the fields of industry will be reshaped with AI. Forensic medicine and toxicology is an important branch for investigations of crimes and this branch has tremendous scope of development with the help of AI. Any science or process cannot stay away long from newer techniques. The traditional way of doing an autopsy and framing an opinion has a lot of limitations and these limitations can be overcome with AI. Various procedures of forensic medicine like analysis of toxins, collection of the various samples of medicolegal importance from body cavity, detection of pathological changes in various organs of the body, detection of various stains on the body, detection of a weapon used in crime, time since death calculations, etc. are the areas where AI will play a key role in framing the various opinions of medicolegal importance. AI can also be integrated into existing testing and analysis processes to make the whole procedure rapid and more accurate. In the future, AI might become a key part of forensic medicine and toxicology practice. Today, the fourth Industrial Revolution is completely reshaping numerous industries. Artificial Intelligence (AI) has become a new cornerstone for all digital transformation initiatives . In the What are forensic medicine and toxicology?Forensic medicine is the application of medical knowledge to law enforcement, criminal investigation, and the legal system. Toxicology is the study of the adverse effects of chemicals on human health. The main focus of toxicology is the study of the effects of both naturally occurring and man-made chemicals on the human body .There are various sub-specialities of toxicology, like clinical toxicology, which deals with the human diseases caused by the effect of poisons on the human body. Medical toxicology deals with the treatment of poisons. Occupational toxicology deals with occupational hazards of various toxic materials on the human body. Analytical toxicology deals with the analysis and detection of poison. Toxinology is a branch of toxicology that deals with various toxins produced by living organisms that are harmful to the human body. Forensic toxicology applies this multidimensional knowledge of toxicology for the purposes of law and administration of justice .The main focus of toxicology is the study of the effects of both naturally occurring and man-made chemicals on the human body. It encompasses a wide range of substances, including prescription medications, over-the-counter medications, alcohol, narcotics, illicit drugs, environmental toxins, etc.Forensic medicine and toxicology are the foundations of forensic science. They are applied in a wide variety of scenarios and are commonly used in crime scene investigations and court cases. Forensic experts use a wide range of techniques for the purpose of identification and to determine if someone was poisoned, infected, had a certain medical condition, or died by violence. Techniques include gathering and analyzing DNA and fingerprints, examining blood and other bodily fluids, and examining hair, fibres, weapons, bones, and soil samples from a crime scene.In this article, we have discussed various key points in the field of forensic medicine, where the application of AI will be of great help to forensic medicine experts.AI in forensic medicine and toxicologyAI includes the building of smart machines that can perform tasks like human intelligence. The traditional way of doing an autopsy has many limitations, i.e., it requires experienced manpower in each and every case, some minute observations can be missed by a naked eye examination, and at some points person to person variation may occur while framing the opinions. The application of advanced technology in the field of forensic medicine is the need of the hour to perform a medicolegal autopsy in an accurate manner.While performing the medicolegal autopsy, a forensic expert has to look for various points as per the need of the case to frame a proper opinion regarding the cause of death and to answer the queries of the investigating agency. These include proving the identity of an individual, externally examining various stain marks on cloths or body, identifying and collecting samples of body fluid, examination of wounds, etc. In an internal examination of the body, the forensic expert has to find out various pathological conditions of the organs to detect the cause of death; they have to look for various minor fractures and injuries, which are usually missed by naked-eye examination, but these injuries may have a role in causing the death of the person. They also have to look for any affected or inflamed area present in the body due to the effect of poison. They also have to examine various pieces of trace evidence like blood samples, seminal stains, and fingerprint marks. In all those areas of forensic medicine, AI can play a key role in aiding forensic experts to form more accurate, quick, and uniform opinions related to forensic case examination by comparing the data from their findings with the data available from machines. Similarly, weapon examination and calculation of time since death also are areas where AI can be helpful .While reviewing the topic ‘AI in Forensic Medicine and Toxicology,’ we have gone through various pieces of literature and took references from various articles published on this matter in the standard journals. We identified some potential areas where AI can be applied by forensic experts in the field of forensic medicine, as described below. Identification is a very important aspect in the field of forensic medicine as it is required in the identification of criminals, identification of the unknown dead body, identification in mass disasters situations like earthquakes, bomb blasts, floods, etc. The traditional way of Identification involves anthropology, facial descriptions, tattoos, scars, body marks, etc. Fingerprinting and DNA analysis are newer techniques introduced nowadays. But with the advent of AI, machines can be used to establish identity. When a machine is provided with the input of various body parameters like facial features, retinal patterns, and fingerprints, the machine will store this data electronically. By using these parameters of individuals, which are already stored electronically in the machine, the machine can establish the identity of individuals by using AI ,8. FingeA machine can be fed with various parameters for establishing the identity; these include fingerprint pattern, iris pattern, facial features, gait pattern, DNA pattern, palm print, and voice pattern. Identification of the individual by this biological data is known as biometry. When the particular pattern of biometry of the individual, e.g, a fingerprint, is brought in front of the machine, it will quickly recognize that individual by matching the biometric pattern which is already stored in the machine. Thus, individual identity can be established by matching the biometric pattern put in front of a machine with the biometric data of the individual already stored at the machine. To apply this technique to a large population, the machine will require biometric data of all individuals of a particular locality. This biometric technique will be of great help in establishing the identity of an individual. However, there will be many infrastructure- and resource-related challenges while providing the biometric data of a vast population to the machine. Stains on the body or various colour changes on the body are examined externally by the naked eye. AI can have a key role here. For example, stains of seminal fluid may be present in cases of sexual assault. Sperm detection is an important factor in such cases for criminal investigation and framing the charge. A naked-eye examination can miss this important stain mark, or many times, collected samples are inadequate to detect the sperm in the sample. AI microscopy imaging technique can visualize sperm stains more accurately by using the deep convolution neural networks technique. This automated image analysis technique identifies objects and substances in images; this technique can also be used in the detection of fingerprints on a surface or blood spatter patterns on clothing .Colour changes in bruises are important for the calculation of the age of the injury. Colour interpretation by the naked eye has limitations. Computer-aided colour detection techniques can analyze the colour more accurately from the wound. The machine is provided with various graphs related to trauma time intervals and changes in wound colour pattern. By comparing the available data, the machine can interpret the time interval of injury in a very accurate manner . AI-enhanced virtual autopsy is a newer trend being used in the field of forensic medicine while performing the autopsy. Machine learning tools will take the images of the body with the help of a CT scan or MRI technique. The machine will itself identify the pathological condition of an organ from the images by comparing it with the vast amount of input data provided to the machine . The macTime since death estimation is an important factor in the field of forensic medicine and criminal investigation. There are various markers in blood which can be used to predict the time since death. These blood markers can be processed through an AI device and thus accurate opinion regarding the time since death can be obtained. Biomarkers which can be used in this technique are lactate dehydrogenase (LDH), pH of blood, triglyceride, cholesterol, sodium, potassium, etc. After death, decomposition of the body starts and the levels of biomarkers change with proportion to time duration. With the help of available statistical data, changes in the levels of biomarkers can be interpreted in the form of time since death by an AI device ,13.There are various modern methods for chemical analysis available for processing various body samples for forensic toxicological analysis. Some of the techniques used are photo spectrometer, chromatography, neutron activation analysis, high-performance liquid chromatography, etc. Though the techniques are advanced, human error can result in a wrong analysis of the sample; these analytical methods are complicated and expert staff is required for proper analysis. To avoid this, AI can play a key role. A set of algorithms is provided to a machine on the basis of which the machine can analyze the sample more accurately and the method will be less time-consuming compared to the conventional way of analysis . AI can Apart from this, AI has also potential uses in the following areas: (i) Disease Diagnosis, where AI can help with the diagnosis of diseases using medical images, such as CT, MRI, and PET scans , (ii) DiOmics data mining using AIOmics is the suffix used in various branches of biology such as genomics, proteomics, metabolomics, toxicogenomics, etc. Omics technologies involve a large amount of scientific data which can be utilized in the forensic field for calculation of postmortem interval, diagnosis of disease, and analyzing drug abuse and poisoning cases. Studies in every forensic sector are increasingly using omics data from different platforms for research purposes as well. . For exaFuture forensic research will focus on multi-omics combined analysis and AI algorithm modelling. The process and regulatory mechanism of complex diseases can be more thoroughly and methodically reflected by the joint analysis of numerous omics using AI-enabled machines. .Limitations of AI in forensic medicine and toxicologyTraining of Machine An AI machine requires a large amount of data feed for accurate interpretation. Data should be in large quantities and of high quality for the machine to be able to learn from the data and for it to be trained in various key areas of forensic medicine by forensic experts so that it can interpret the finding in post-mortem examination and give a sound opinion. Forensic medicine experts must first manually annotate images and documents with key findings, conclusions, and other details to machines. For this, initially, a lot of effort needs to be made by forensic experts. Ethics and Regulations AI is a powerful technology that can promote efficiency and accuracy. However, at the same time, its implementation raises ethical questions and challenges existing regulatory environments. Some people argue that using AI will impede the traditional practice of human judgment.Trust Forensic medicine experts need to build trust with their clients and stakeholders. They need to show the public that their work is accurate and trustworthy. Whether the opinion given by the machine will be trusted by the judiciary, investigating agency, and the common public will be a matter of great concern. Forensic experts need to explain to their clients, i.e., the judiciary and the investigation agency that the opinion derived from AI is accurate and trustworthy.Infrastructure Forensic medicine experts need to have the right equipment and infrastructure to implement AI. This includes high-performance computing infrastructure and large data storage systems. All the processes will be very costly. Whether poorer nations will be able to afford such setups is also a matter of great concern. Human Interaction AI is simply an automated tool. It cannot replace human interaction and expertise. Every data fed to the machine will require human efforts. So, initially, for training the machine a lot of effort will have to be expended by forensic experts to train the machine and also regular updating of the data is a must. Interoperability Different AI tools might not be able to interact with one another. This might lead to a situation where silos of data are created. It will lead to duplication of efforts.Medicolegal ImplicationIn a court of law, every documentary evidence needs to be testified by the expert orally. Hence, whether the AI-derived opinion will be accepted as evidence in a court of law is the most important matter of concern in the field of forensic medicine.The Scenario in Developing CountriesIn a developing country like India, it is a known fact that advanced healthcare facility is out of reach of a large section of the population. Hence, developing high-tech infrastructure in the field of forensic medicine will be very difficult for policymakers. AI promises a range of benefits for forensic medicine experts, in the field of autopsy and toxicological analysis. The main challenge to implementing AI in forensic medicine and toxicology is finding high-quality data for training the machine. So initially a lot of effort will be required from forensic medicine experts across the globe to provide highly accurate data to machines. The required data for the machines may include various autopsy findings with good supportive images and accurate opinions regarding the pattern of the injury; it will include various statistical inputs regarding biomarkers and also the analytical algorithm. Providing data will be time-consuming and time to time updation of the machine data will be required. But if the experts from the forensic field overcome these initial hurdles, the AI tool will have a potential advantage in the field of forensic medicine to frame various opinions of medicolegal importance. From a traditional morphological perspective, computer technology and AI algorithms will improve forensic investigation methods with more accuracy and promptness. With integration into existing testing and analysis processes, AI might become a key part of forensic medicine and toxicology practice. Another matter of concern regarding AI is its legal value in a court of law. The court may not consider an opinion derived from AI as conclusive proof, but this opinion can act as corroborative evidence as any machine depends on the data it is being fed. Even the judiciary needs to understand the working pattern of the machine before giving the judgement. In such a scenario, judges may take expert review for the accuracy of the machine. However, this development will occur with the advent of time and evolution in the process of AI use. Setting up high-tech infrastructure in the field of forensic medicine will be a big challenge for policymakers in developing countries like India as the healthcare facility is in a developing stage and centred only in urban areas. The current priority for developing countries is to make medicare accessible to all the population of each and every part of the country. So, in a developing country like India, considering the current scenario of the healthcare system, AI-aided forensic medicine practice can be started as a pilot project at select centres. After assessing its efficacy and usefulness, such a practice can be expanded to larger areas in a phase-wise manner.


We described the accuracy of ultrasound in determining the position of bronchial blockers (BBs) in children underwent thoracoscopic surgery.We enrolled 52 children with ASA grade I‐III who received thoracoscopic surgery with placement of BBs. Point‐of‐care ultrasound was performed according to the BLUE protocol. The ultrasound‐guided lung sliding sign and curtain sign were used to assess the position of BBs. The accuracy of ultrasound in evaluating the position of BBs, as well as the accuracy and operating time of sliding sign and curtain sign at each examination point were recorded and compared.p = 0.002). The accuracy of curtain sign at the left mid‐axillary line‐diaphragm and the right mid‐axillary line‐diaphragm was respectively 96%  and 84% , which were higher than that of sliding sign at upper blue points and lower blue points. There was no significant difference in the operating time between two ultrasound signs .The accuracy of ultrasound in evaluating the position of BBs was 88% . When using the curtain sign to assess the position of BBs, the accuracy was 90% , which was significantly higher than when using the sliding sign (65% (136/208), 95% CI 0.55–0.74) (Point‐of‐care ultrasound can effectively assess the position of BBs. The accuracy of using the curtain sign at the mid‐axillary line‐diaphragm is higher than that of using the lung sliding sign at the anterior chest wall. Point‐of‐care ultrasound can effectively assess the position of BBs. The accuracy of using the curtain sign at the mid‐axillary line‐diaphragm is higher than that of using the lung sliding sign at the anterior chest wall. As a result, ultrasound waves are reflected when they reach the pleura, and imaging only shows the pleural layer of healthy lungs. However, when the normal air‐to‐liquid ratio in the lungs is altered by a specific lung disease, especially when the pleura is involved, ultrasound can be locally reflected, creating a unique ultrasound image that helps locate lung lesions.Herein, we investigated the accuracy of using both lung sliding sign and curtain sign in determining the position of BBs in children. Children who received thoracoscopic surgery with placement of BBs were enrolled. Our findings may provide a novel method for confirming the position of BBs.22.1The study was approved by the Ethics Committee of our hospital  and registered at chictr.org.cn (ChiCTR2000032356). All methods were also performed in accordance with the relevant guidelines and regulations under the committee supervision. Informed consent was obtained from the legal guardians of each subject.2.2From May 2020 to April 2021, we enrolled children of older than 6 months, with ASA I‐III grade, and undergoing thoracoscopic surgery with BBs placement. Children with diseases that can affect lung sliding sign and curtain sign, such as pleural adhesions, pleural effusion, pulmonary edema, severe lung infection, and airway malformations, were excluded from this study.2.32 concentration of 4.0–5.7 Kpa.The tracheal intubation and BBs placement were performed under general anesthesia. Anesthesia was induced by intravenous bolus of propofol (3 mg/kg) and sufentanil (0.1 μg/kg), and the tracheal intubation and BBs were placed 2 min after the administration of cis‐atracurium (0.2 mg/kg). The sizes of ETT and BBs  were selected according to the age of the children. The placement method of BBs, that is, intraluminal placement or extraluminal placement, was determined by the anesthesiologists according to their personal experience. For the intraluminal placement, ETT was first placed under the laryngoscope, and then BBs was placed into the trachea through ETT. For extraluminal placement, BBs was first placed into the trachea under laryngoscope, and then ETT was placed into the trachea. After the placement of ETT and BBs, FOB was used to check their position to exclude endobronchial intubation and to guide BBs to be positioned in the target main bronchus. After the BBs balloon was inflated under FOB, the FOB was withdrawn. Children were mechanically ventilated to maintain breathing. The tidal volume was selected according to 6–8 ml/kg, and the respiratory frequency was adjusted to maintain the end‐tidal CO2.4Point‐of‐care ultrasound evaluation was conducted under mechanical ventilation and was performed by one anesthesiologist blinded to BBs placement. The ultrasound examinations of all children were carried out by the same anesthesiologist who had experience in ultrasound examinations on 80 cases. For ultrasound, high frequency linear array probe  was used. The ultrasound images were optimized before examination and the imaging depth was adjusted to be 3–4 times the distance between the A‐lines.2.5After ultrasound evaluation, the anesthesiologist who completed the tracheal intubation performed FOB again to verify the position of BBs, and to check whether BBs are displaced and whether the cuff is fully inflated. According to the results of FOB, the results of ultrasound examination, as well as those of lung sliding sign and curtain sign method in distinguishing the position of BBs, were evaluated.2.6χ2 test or Fisher's exact probability method, and the unidirectional ordered data was analyzed by rank sum test. The statistically significant difference was considered at p < 0.05.All data were analyzed using SPSS 24.0 software. The measurement data were tested for normality. Data conforming to the normal distribution are represented by the mean ± standard deviation, and compared with t test or ANOVA; while, those not conforming to the normal distribution are represented by the median and interquartile range and analyzed with Wilcoxon rank sum test or Kruskal–Wallis rank sum test. The count data is described by number (%). The unordered classification data was analyzed by 33.1A total of 52 children were enrolled in this study, the basic information of the children is shown in Table 3.2FOB reassessment revealed that eight cases of children with BBs placement in the right main bronchus had incomplete occlusion of the upper lobe of the right lung due to cuff displacement or insufficient cuff inflation Figure . Two casp = 0.001) (Table p = 0.002) . In the comparison of each ultrasound examination point, the accuracy of using lung sliding sign at the left upper blue point and the right upper blue point were 80%  and 61% ; and, those at the left lower blue point and the right lower blue point were 61%  and 53% , respectively (Table 3.3p = 0.065)  Table .4Point‐of‐care ultrasound is more and more widely used in anesthesia, which can provide timely diagnosis for anesthesiologists; however, its application in pediatric anesthesia is still less.Studies have shown that the lung sliding sign has high sensitivity and specificity in evaluating the position of the tracheal intubation and unilateral main bronchial intubation, and its accuracy is greater than that of auscultation.The curtain sign can be used to describe the expansion and inflation of the lungs.The right main bronchus of children is short, and the displacement of BBs can easily lead to incomplete occlusion of the upper lobe of the right lung.For the examination time, we found that the time required for the curtain sign examination was less than that of the lung sliding sign. However, the statistical difference was not significant, which may be related to the relatively small sample size in this study. We believe that finding the moving lower boundary of the lung at the costophrenic recess is the key to the curtain sign. The air movement on the unblocked side makes the lower boundary of the lung easy to identify, and the curtain sign can be recognized in a short time. On the blocked side, there is lack of lung movement, which makes the identification of the lower boundary of the lung difficult, and relatively increases the examination time. There was no difference in the examination time of the lung sliding sign at each point, but for children, the examination time at the lower blue point may take a relatively long time due to the obstruction and interference of the heart.It is worth noting that although ultrasound can effectively evaluate the position of BBs in a short time, FOB is still an important tool in lung isolation and anesthesia management. First, FOB can directly guide the placement of BBs and observe the position of BBs. Second, FOB examination is rarely affected by the body position of children. In this study, the children maintained the surgical position of contralateral decubitus after completing the placement of BBs in the supine position. Before the operation, the anesthesiologist checked whether the BBS is displaced due to the change of body position using FOB. In the contralateral decubitus position, the ultrasound curtain sign cannot be used to evaluate the ventilation of the bottom lung, which is also a defect of this method. However, this study demonstrates that point‐of‐care ultrasound is also a very helpful method in lung isolation because ultrasound is noninvasive and has reliability in evaluating the position of BBs and in preliminarily evaluating the occlusion effect. In addition, different from the intra‐airway performance of FOB, ultrasound is performed on the body surface of children, which has no effect on ventilation and reduces the incidence of adverse airway events.This study has some limitations. First, the lung pulse can be presented as intermittent fluctuations at the pleural line synchronized with the heartbeat under the M‐mode method,5In short, point‐of‐care ultrasound could be used to evaluate the position of BBs in children with high accuracy. The curtain sign at the mid‐axillary line‐diaphragm examination point, and the lung sliding sign at the upper and lower blue points on the anterior chest wall could be used to assess the position of BBs. Additionally, the accuracy of the curtain sign was higher than that of the lung sliding sign. The accuracy of ultrasound evaluation at multiple points was higher than that at single point. It is recommended to conduct comprehensive evaluation of the lungs on both sides to improve the accuracy of ultrasound evaluation, and to conduct a preliminary evaluation of the occlusion effect.This study was supported by the Comfortable Medical Fund of Shandong Medical Association.The authors report no conflict of interest.chictr.org.cn (ChiCTR2000032356). All methods were also performed in accordance with the relevant guidelines and regulations under the committee supervision.The study was approved by the Ethics Committee of Qilu Children's Hospital of Shandong University  and registered at Informed consent was obtained from the legal guardians of each subject.Video S1 The video showing the lung curtain sign.Click here for additional data file.Video S2 The conduction of pressure in the contralateral thoracic cavity during ventilation.Click here for additional data file.