In a previous work we have detailed the requirements for obtaining maximal deep learning performance benefit by implementing fully connected deep neural networks (DNN) in the form of arrays of resistive devices. Here we extend the concept of Resistive Processing Unit (RPU) devices to convolutional neural networks (CNNs). We show how to map the convolutional layers to fully connected RPU arrays such that the parallelism of the hardware can be fully utilized in all three cycles of the backpropagation algorithm. We find that the noise and bound limitations imposed by the analog nature of the computations performed on the arrays significantly affect the training accuracy of the CNNs. Noise and bound management techniques are presented that mitigate these problems without introducing any additional complexity in the analog circuits and that can be addressed by the digital circuits. In addition, we discuss digitally programmable update management and device variability reduction techniques that can be used selectively for some of the layers in a CNN. We show that a combination of all those techniques enables a successful application of the RPU concept for training CNNs. The techniques discussed here are more general and can be applied beyond CNN architectures and therefore enables applicability of the RPU approach to a large class of neural network architectures. Deep neural network (DNN)  technologies  neurons, the forward cycle involve computing a vector-matrix multiplication (y = Wx) where the vector x of length N represents the activities of the input neurons and the matrix W of size M×N stores the weight values between each pair of input and output neurons. The resulting vector y of length M is further processed by performing a non-linear activation on each of the elements and then passed to the next layer. Once the information reaches to the final output layer, the error signal is calculated and backpropagated through the network. The backward cycle on a single layer also involves a vector-matrix multiplication on the transpose of the weight matrix (z = WTδ), where the vector δ of length M represents the error calculated by the output neurons and the vector z of length N is further processed using the derivative of neuron non-linearity and then passed down to the previous layers. Finally, in the update cycle the weight matrix W is updated by performing an outer product of the two vectors that are used in the forward and the backward cycles and usually expressed as W ← W + η (δxT) where η is a global learning rate.The backpropagation algorithm is composed of three cycles—forward, backward and weight update—that are repeated many times until a convergence criterion is met. For a single fully connected layer where W can be implemented with a 2D crossbar array of two-terminal resistive devices with M rows and N columns where the stored conductance values in the crossbar array form the matrix W. In the forward cycle, input vector x is transmitted as voltage pulses through each of the columns and resulting vector y can be read as current signals from the rows  time complexity in all three cycles, independent of the array size.All three operating modes described above allow the arrays of cross-point devices that constitute the network to be active in all three cycles and hence enable a very efficient implementation of the backpropagation algorithm. Because of their local weight storage and processing capability these resistive cross-point devices are called RPU devices  due to analog nature of the computations on RPU arrays. We discuss noise and bound management techniques that mitigate these problems without introducing any additional complexity in the analog circuits, and that can be addressed by the associated digital circuitry. In addition, we discuss digitally-programmable update management and device variability reduction techniques that can be used selectively for some of the layers in a CNN. We show that a combination of these techniques enables a successful application of the RPU concept for the training of CNNs. Furthermore, a network trained with RPU devices, including imperfections, can yield a classification error indistinguishable from a network trained using conventional high-precision floating point arithmetic.n, n, d) with a width and height of n pixels and a depth of d channels corresponding to different input components  as illustrated in Figure k, k, d). During the forward cycle, each kernel slides over the input volume across the width and height and a dot product is computed between the parameters of the kernels and the input pixels at any position. Assuming no zero padding and single pixel sliding , this 2D convolution operation results in a single output plane with dimensions ((n − k + 1), (n − k + 1), 1) per kernel. Since there exists M different kernels, output becomes a volume with dimensions ((n − k + 1), (n − k + 1), M) and is passed to following layers for further processing. During the backward cycle of a convolutional layer similar operations are performed but in this case the spatially flipped kernels slide over the error signals that are backpropagated from the upper layers. The error signals form a volume with the same dimensions of the output ((n − k + 1), (n − k + 1), M). The results of this backward convolution are organized to a volume with dimensions  and are further backpropagated for error calculations in the previous layers. Finally, in the update cycle, gradient with respect to each parameter is computed by convolving the input volume with the error volume used in the forward and backward cycles, respectively. This gradient information, which has the same dimensions as the kernels, is added to the kernel parameters after scaled with a learning rate.The input to a convolutional layer can be an image or the output of the previous convolutional layer and is generally considered as a volume with dimensions of 2 has the input neuron activities with some repetition and resulting matrix Y with dimensions M × (n − k + 1)2 has all the results corresponding to the output volume. Similarly, using the transpose of the parameter matrix, the backward cycle of a convolutional layer can also be expresses as a matrix-matrix multiplication Z = KTD, where the matrix D with dimensions M × (n − k + 1)2 has the error signals corresponding to an error volume. Furthermore, in this configuration the update cycle also simplifies to a matrix multiplication where the gradient information for the whole parameter matrix K can be computed using matrices X and D, and the update rule can be written as K ← K + η(DXT).For an efficient implementation of a convolutional layer using an RPU array, all the input/output volumes as well as the kernel parameters need to be rearranged in a specific way. The convolution operation essentially performs a dot product between the kernel parameters and a local region of the input volume and hence can be formulated as a matrix-matrix multiply 2 columns in X completes all the computations required for the forward cycle. Similarly, in the backward cycle the input vector corresponding to a single column in D is serially fed to the rows of the array. The update rule shown above can be viewed as a series of updates that involves computing an outer product between two columns from X and D. This can be achieved by serially feeding the columns of X and D simultaneously to the RPU array. During the update cycle each RPU device performs a series of local multiplication and summation operations and hence calculates the product of the two matrixes.The rearrangement of the trainable parameters to a single matrix Mk2d(n − k + 1)2 and this number is independent of the method of computation. The proposed RPU mapping described above achieves this number as follows: Due to the inherent parallelism in the RPU array Mk2d operations are performed simultaneously for each vector operation performed on the array. Since there are (n − k + 1)2 vector operations performed serially on the array, the total number of computations matches the expectation. Alternatively, one can consider that there are Mk2d trainable parameters and that each parameter is used (n − k + 1)2 times due to the parameter sharing in a convolution layer. Since each RPU device in an array can perform a single computation at any given time, parameter sharing is achieved by accessing the array (n − k + 1)2 times. For fully connected layers each weight is used only once and therefore all the computations can be carried out using single vector operations on the array.We note that for a single input the total number of multiplication and summation operations that need to be computed in all three cycles for a convolutional layer is The end result of mapping a convolutional layer onto the RPU array is very similar to the mapping of a fully connected layer and therefore does not change the fundamental operations performed on the array. We also emphasize that the convolutional layer described above, with no zero padding and single pixel sliding, is only used for illustration purposes. The proposed mapping is more general and can be applied to convolutional layers with zero padding, strides larger than a single pixel, dilated convolutions or convolutions with non-square inputs or kernels. This enables the mapping of all of the trainable parameters of a conventional CNN within convolutional and fully connected layers to RPU arrays.tanh) activation functions. The first layer has 16 kernels while the second layer has 32 kernels. Each convolutional layer is followed by a subsampling layer that implements the max pooling function over non-overlapping pooling windows of size 2 × 2. The output of the second pooling layer, consisting of 512 neuron activations, feeds into a fully connected layer consisting of 128 tanh neurons, which is then connected into a 10-way softmax output layer. Training is performed repeatedly using a mini-batch size of unity for all 60,000 images in the training dataset which constitutes a single training epoch. Learning rate of η = 0.01 is used throughout the training for all 30 epochs.In order to test the validity of this method we performed DNN training simulations for the MNIST dataset using a CNN architecture similar to LeNet-5  numbers, the network achieves a classification error of 0.8% on the test data. This is the FP-baseline model that we compare against the RPU based simulations for the rest of the paper. We assume all activations and pooling layers are implemented in the digital circuits for the RPU based simulations.Following the proposed mapping above, the trainable parameters (including the biases) of this architecture are stored in 4 separate arrays with dimensions of 16 × 26 and 32 × 401 for the first two convolutional layers, and, 128 × 513 and 10 × 129 for the following two fully connected layers. We name these arrays as The influence of various RPU device properties, variations, and non-idealities on the training accuracy of a deep fully connected network are discussed in Gokmen and Vlasov . We follxi and δj) are implemented as stochastic bit streams. Each RPU device performs a stochastic multiplication . This is taken into account by using separate wij|, is assumed be 0.6 on average with a 30% device-to-device variation. We did not introduce any non-linearity in the weight update as this effect has been shown to be insignificant as long as the updates are reasonably balanced (symmetric) between up and down changes  is determined by integrating the analog current coming from the column (or row) during a measurement time (tmeas) using a capacitor (Cint) and an op-amp. This approach will have noise contributions from various sources. These noise sources are taken into account by introducing an additional Gaussian noise, with zero mean and standard deviation of σ = 0.06, to the results of vector-matrix multiplications computed on an RPU array. This noise value can be translated to an acceptable input referred voltage noise following the approach described in Gokmen and Vlasov  is transmitted as voltage pulses with a fixed amplitude and tunable durations as illustrated by Figure tmeas → 1), and all pulse durations are scaled accordingly depending on the values of xi or δj. This scheme works optimally for the forward cycle with tanh (or sigmoid) activations, as all xi in x including a bias term are between . However, this assumption becomes problematic for the backward cycle, as there are not any guarantees for the range of the error signals in δ. For instance, all δj in δ may become significantly smaller than unity (δ ≪ 1) as the training progresses and the classification error gets smaller. In this scenario the results of a vector-matrix multiplication in the backward cycle, as shown by Equation (2) below:During a vector-matrix multiplication on an RPU array, the input vector  non-linearity because the error introduced during the calculation of tanh(z) (or sigmoid(z)) due to the bound is negligible for an input value z that is otherwise much larger. However, for the output layer with softmax (or ReLU) activations the error introduced due to the bound may become significant. For instance, if there are two outputs that are above the bounded value, they would be treated equally and the classification task would choose between the two classes with equal probability, even if one of the outputs is significantly larger than the other. This results in a significant error (major information loss) in estimating the class label and hence limits the performance of the network. As with the noise, the bounded signals start to become an issue for later stages of the training as the network “starts to perform good test results” (approaches an optimum configuration) and the decision boundary between classes become more distinct. As shown by the blue curve in Figure In addition to the noise, the results of a vector-matrix multiplication will be strongly influenced by the |α| term that corresponds to a maximum allowed voltage during the integration time. The value |α| = 12 does not strongly influence the activations for hidden layers with n) the unbounded signals can be read reliably and properly rescaled in the digital domain. In this bound management scheme, the effective vector-matrix multiplication on an RPU array can be written as:In order to eliminate the error introduced due to bounded signals, we propose repeating the vector-matrix multiplication after reducing the input strength by a half when a signal saturation is detected. This would guarantee that after a few iterations  for the largest numbers that contribute most in calculation of softmax activations.with a new effective bound of 2In order to test the validity of the proposed noise management (NM) and bound management (BM) techniques, we performed simulations using the RPU-baseline model of Table max needs to be determined from δ and each element in δ (and z) value needs to be divided (and multiplied) by δmax. All of these computations require additional O(M) comparison, division and multiplication operations that are performed in the digital domain. However, given that the very same circuits need to compute O(M) error signals using the derivative of the activation functions, performing these additional operations do not change the complexity of the operations that needs to be performed by the digital circuits. Basically, the combination of all of these operations can be viewed as computing a slightly more complicated activation function. Therefore, with proper design these additional operations can be performed with only a slight overhead without causing significant slowdown on the digital circuits. Similarly, BM can be handled in the digital domain by performing O(N) computations only when a signal saturation is detected. However, BM may require an additional circuitry that detects a signal saturation that can be fed as a control signal to the digital circuits for the repeated vector-matrix multiplication.The additional computations introduced in the digital domain due to NM and BM are not significant and can be addressed with a proper digital design. For the NM technique, δwmin, wij| are completely eliminated for different layers while the average values are kept unaltered. The model that is free from device variations for all four layers achieves a test error of 1.05%. We note that most of this improvement comes from the convolutional layers as a very similar test error of 1.15% is achieved for the model that does not have device variations for K1&K2, whereas the model without any device variations for fully connected layers W3&W4 remains at 1.3% level. Among the convolutional layers, it is clear that K2 has a stronger influence than K1 as test errors of 1.2 or 1.4% are achieved respectively for models with device variations eliminated for K2 or K1. Interestingly, when we repeated similar analysis by eliminating only the device-to-device variation for the imbalance parameter The RPU-baseline model with NM and BM performs reasonable well and achieves a test error of 1.7%, however, this is still above the 0.8% value achieved with a FP-baseline model. In order to identify the remaining factors contributing to this additional classification error, we performed simulations while selectively eliminating various device imperfections from different layers. The summary of these results is shown in Figure It is clear that the reduction in device variations in some layers can further boost the network performance; however, for realistic technological implementations of the crossbar arrays variations are controlled by fabrication tolerances in a given technology. Therefore, complete or even partial elimination of any device variation is not a realistic option. Instead, in order to get better performance, the effects of the device variations can be mitigated by mapping more than one RPU device per weight, which averages out the device variations and reduces the variability  used per weight effectively reduces the device variations by a factor proportional to K2 effectively reduces the device variations by a factor of 3.6 at a cost of increase in the array dimensions to 416 × 401 (from 32 × 401) Assuming RPU arrays are fabricated with equal number of columns and rows, multi-device mapping of rectangular matrixes such as K2 does not introduce any operational (or circuit) overhead as long as the mapping fits in the physical dimensions of the array. However, if the functional array dimensions becomes larger than the physical dimensions of a single RPU array then more than one array can used to perform the same mapping. Independent of its physical implementation this method enables flexible control of the number of devices used while mapping different layers and is therefore a viable approach for mitigating the effects of device variability.To test the validity of this digitally controlled multi-device mapping approach, we performed simulations using models where the mapping of the most influential layer BL = 10 and amplification factors that are equally distributed to the columns and the rows with values wmin, BL, Cx and Cδ. Δwmin corresponds to the incremental conductance change on an RPU device due a single coincidence event; therefore the value of this parameter may be strongly restricted by the underlying RPU hardware. For instance, Δwmin may be tuned only by shaping the voltage pulses used during the update cycle and hence requires programmable analog circuits. In contrast, the control of Cx, Cδ, and BL is much easier and can be implemented in the digital domain.All RPU models presented so far use the stochastic update scheme with a bit length of Cx, Cδ, and BL on the training accuracy we performed simulations using the RPU-baseline model with the noise and bound management techniques described above. For all models, we used the same fixed learning rate η = 0.01 and Δwmin = 0.001. The summary of these results is shown in Figure BL, and both Cx and Cδ are fixed at BL to 40 did not improve the network performance, whereas reducing it to 1 boosted the performance and a test error of about 1.3% is achieved. These results may be counter intuitive as one might expect the larger BL case to be less noisy and hence would perform better. However, for BL = 40 case, the amplification factors are smaller (Cx = Cδ = 0.5) in order to satisfy the same learning rate on average. This reduces the probability of generating a pulse, but since the streams are longer during the update, the average update (or number of coincidences) and the variance do not change. In contrast, for BL = 1, the amplifications factors are larger with a value 3.16 and therefore pulse generation becomes more likely. Indeed, for cases in which the amplified values are larger than unity (Cxxi > 1 or Cδ δ j > 1) a single update pulse is always generated. This makes the updates more deterministic but with an earlier clipping for xi and δj values encoded from the periphery. Also note that for a single update cycle the weight can change at most BL×Δwmin and for BL = 1 the weight value can only move by a single Δwmin per update cycle. However, also note that the convolutional layers K1 and K2 receive 576 and 64 single bit stochastic updates per image due to weight reuse (sharing) while the fully connected layers W3 and W4 receive only one single bit stochastic update per image. The interaction of all of these terms and the tradeoffs are non-trivial and the precise mechanism by which BL = 1 performs better than BL = 10 is still unclear. However, the empirical data shows clearly there is an advantage to be had for the above CNN architecture, which favors BL = 1; whereas the DNN used in Gokmen and Vlasov . In our update management scheme, we use Cx and Cδ values such that the probability of generating pulses from columns (x) and rows (δ) are roughly the same order. This is achieved by rescaling the amplification factors with a ratio BL = 10 this method did not yield any improvement, for BL = 1 the error rate as low as 1.1% is achieved; and hence shows that the proposed update management scheme can yield better performance.In addition to x) and rows (δ) are very different; i.e., x have many elements close 1 (or −1) whereas δ may have elements very close to zero (δ ≪ 1). For this case if the same Cx and Cδ are used, the updates become row-wise correlated. Although unlikely, the generation of a pulse for δj will result in many coincidences along the row j, as there are many pulses generated by different columns since many xi values are close to unity. Our update management scheme eliminates these correlated updates by shifting the probabilities from columns to rows by simply rescaling the values used during the update. This can be viewed as using rescaled vectors (mx and δ/m) for the updates which are composed of values of roughly the same order. This update management scheme relies on a simple rescaling that is performed in the digital domain, and therefore does not change the design of the analog circuits needed for the update cycle. The additional computations introduced in the digital domain are not significant, and only require additional O(M) (or O(N)) operations, similar to the overhead associated with the noise management technique.This proposed update scheme does not alter the expected change in the weight value and therefore its benefits may not be obvious. Note that toward the end of training it is very likely that the range of values in columns (K2) brings the model's test error to 0.8%. The performance of this final RPU model is almost indistinguishable from the FP-baseline model and hence shows the successful application of RPU approach for training CNNs. We note that all these mitigation methods can be turned on selectively by simply programing the operations performed on digital circuits; and therefore can be applied to any network architecture beyond CNNs without changing design considerations for realistic technological implementations of the crossbar arrays and analog peripheral circuits.The summary of CNN training results for various RPU models that use the above management techniques is shown in Figure wmin(wij) that included a linear or a quadratic dependence on weight value. Indeed this additional non-linear weight update rule does not cause any additional error even when Δwmin is varied by a factor of about 10 within the weight range.We note that for all of the simulation results described above we do not include any non-linearity in the weight update as this effect is shown to be not important as long as the updates are symmetric in positive and negative directions  and independent of the array size, however, because of the weight sharing in convolutional layers, the RPU arrays are accessed several times, resulting in a series of vector operations performed on the array for all three cycles. These repeated vector operations introduce interesting challenges and opportunities while training CNNs on a RPU based hardware.The application of RPU device concept for training CNNs requires a rearrangement of the kernel parameters and only after this rearrangement the inherent parallelism of the RPU array can be fully utilized for convolutional layers. A single vector operation performed on the RPU array is a constant time ws) and the number of multiply and add (MAC) operations performed at different layers for a relative simple but respectable CNN architecture AlexNet .When AlexNet architecture runs on a conventional hardware , the time to process a single image is dictated by the total number of MACs; therefore, the contributions of different layers to the total workload are additive, with K1 takes the longest time among all layers because of the large weight reuse factor of ws = 3, 025, although this layer has the smallest array size and comprises only 10% of the total number of MACs. Assuming a RPU based accelerator with many RPU arrays and pipeline stages between them, the average time to process a single image can be estimated as ws×tmeas using values from layer K1, where tmeas is the measurement time corresponding to a single vector-matrix multiplication on the RPU array. First, this metric emphasizes the constant-time operation of RPU arrays as the training time is independent of the array sizes, the number of trainable parameters in the network, and the total number of MACs. This would enable practitioners to use increasing numbers of kernels, with larger dimensions, without significantly increase training times. These network configurations would be impossible to implement with conventional hardware. However, the same metric also highlights the importance of tmeas and ws for layer K1 which represents a serious bottleneck. Consequently, it is desirable to come up with strategies that reduce both parameters.In contrast to conventional hardware, when the same architecture runs on a RPU based hardware, the time to process a single image is not dictated by the total number of MACs. Rather, it is dominated by the largest weight reuse factor in the network. For the above example, the operations performed on the first convolutional tmeas we first discuss designing small RPU arrays that can operate faster. It is clear that large arrays are favored in order to achieve high degree of parallelism for the vector operations. However, the parasitic resistance and capacitance of a typical transmission line with a thickness of 360 nm and a width of 200 nm limit the practical array size to 4, 096 × 4, 096 as discussed in Gokmen and Vlasov ; and therefore, a small array has worse power and area efficiency metrics compared to a large array. However, a bimodal design consisting of large and small size arrays achieves better hardware utilization and provides speed advantage while mapping architectures with significantly varying matrix dimensions. While the large arrays are used to map fully connected layers or large convolutional layers, for a convolutional layer such as K1 using the small array would be better a solution that provides a reduction in tmeas from 80 to 10 ns.where K1, next we discuss allocating two (or more) arrays for the first convolutional layer. When more than one array is allocated for the first convolutional layer the network can be forced to learn separate features on different arrays by properly directing the upper (left) and lower (right) portions of the image to separate arrays and by computing the error signals and the updates independently. Not only this allows the network to learn independent features for separate portions of the image and does not require any weight copy or synchronization between two arrays, but also for each array the weight reuse factor is reduced by a factor of 2. This reduces the time to process a single image while making the architecture more expressive. Alternatively, one could try to synchronize the two arrays by randomly shuffling the portions of the images that are processed by different arrays. This approach would force the network to learn same features on two arrays with same reduction of 2 in the weight reuse factor. These discussed subtle changes in the network architecture do not provide any speed advantage when run on a conventional hardware; and therefore, it highlights the interesting possibilities that a RPU based architecture provides.In order to reduce the weight reuse factor on In summary, we show that the RPU concept can be applied beyond fully connect networks and the RPU based accelerators are natural fit for training CNNs as well. These accelerators promise unprecedented speed and power benefits and hardware level parallelism as the number of trainable parameters increases. Because of the constant-time operation of RPU arrays, RPU based accelerators provide interesting network architecture choices without increasing training times. However, all of the benefits of an RPU array are tied to the analog nature of the computations performed, which introduces new challenges. We show that digitally-programmable management techniques are sufficient to eliminate the noise and bound limitations imposed on the array. Furthermore, their combination with the update management and device variability reduction techniques enable a successful application of the RPU concept for training CNNs. All the management techniques discussed in this paper are addressed in the digital domain without changing the design considerations for the array or for the supporting analog peripheral circuits. These techniques make RPU approach suitable for a wide variety of networks beyond convolutional or fully connected networks.TG conceived the original idea, TG, MO, and WH developed methodology, analyzed and interpreted results, drafted and revised manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer AS and handling Editor declared their shared affiliation.


Diagnostic tests are performed in a subset of the population who are at higher risk, resulting in undiagnosed cases among those who do not receive the test. This poses a challenge for estimating the prevalence of the disease in the study population, and also for studying the risk factors for the disease.We formulate this problem as a missing data problem because the disease status is unknown for those who do not receive the test. We propose a Bayesian selection model which models the joint distribution of the disease outcome and whether testing was received. The sensitivity analysis allows us to assess how the association of the risk factors with the disease outcome as well as the disease prevalence change with the sensitivity parameter.We illustrated our model using a retrospective cohort study of children with asthma exacerbation that were evaluated for pneumonia in the emergency department. Our model found that female gender, having fever during ED or at triage, and having severe hypoxia are significantly associated with having radiographic pneumonia. In addition, simulation studies demonstrate that the Bayesian selection model works well even under circumstances when both the disease prevalence and the screening proportion is low.The Bayesian selection model is a viable tool to consider for estimating the disease prevalence and in studying risk factors of the disease, when only a subset of the target population receive the test. It is often of interest to estimate the prevalence of a certain condition in a population and this proportion is underestimated as those who were not screened for the disease were assumed negative for the condition. Under-screening of a disease is common because of resource limitation, perceived low risk which eliminates the need of screening, or lack of recommendations from guidelines. For example, fatty liver disease and metabolic syndrome among children may go undetected because relevant evaluations were not routinely recommended by pediatricians . At patiPerceived as at low risk for the condition, those who are not screened or examined are usually classified as negative, which results in an underestimated proportion or prevalence of the disease in the study population. This misclassification can also bias the association of risk factors with the disease condition –10. StanTechnically, the disease condition for those who were not screened/examined is unknown and classifying them as negative is based on a strong assumption about the missing values. Instead, we formulate the problem as a missing data problem and treat the disease status of those not tested as missing. The missing data mechanism, which concerns how the data are missing and whether the missingness is related to the underlying missing values, is critical when dealing with missingness. When the missingness depends neither on observed nor missing values, the data are missing completely at random (MCAR). When the missingness depends on the observed values but not the missing values, the missing data mechanism is called missing at random (MAR). In the case when the missingness can depend on the missing values, the missing data mechanism is called missing not at random (MNAR). When the missing data mechanism is MCAR or MAR, correct inference may be achieved based on a likelihood function which does not involve a modeling for the missingness mechanism; likelihood inference which ignores the model for missingness  includeIn this article, we propose a class of Bayesian selection model, which estimates the disease prevalence in the study population (both screened and unscreened) using a sensitivity parameter which denotes the likelihood of being screened. This model will yield estimates of the prevalence as well as the association of risk factors with the disease outcome under different values of the sensitivity parameter and therefore the big picture of the research questions.an International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis code of asthma (code 493.x) in the first 3 diagnosis positions and receipt of 1 or more doses of albuterol sulfate in the emergency department [We were interested in estimating the proportion of children with an asthma exacerbation who were diagnosed with radiographic pneumonia among those presented to the emergency department of the Cincinnati Children’s Hospital Medical Center between January 1st, 2010, and December 31, 2013. Children were identified using a validated algorithm of partment . ChildreWe investigated the risk factors for radiographic pneumonia i.e. focal opacity present on chest radiograph) . ConsequAssuming those who did not undergo chest radiography to be negative for radiographic pneumonia will underestimate the prevalence of the disease in the study population and potentially bias the association of risk factors on the disease. We formulate this problem as a missing problem and use a Bayesian selection model to jointly model the disease status and the response indicator, i.e., whether the subject received chest radiography or not.yi denote the actual binary radiographic pneumonia status, equal to 1 if the ith subject had radiographic pneumonia, and 0 if not. This outcome was observed for subjects who received chest radiography, and missing for subjects who did not receive chest radiography. We use Ri to denote whether we observed the ith subjects radiographic pneumonia status, and Ri is equal to 1 if the subject received chest radiography and equal to 0 if not. We use xi and zi to denote the covariate sets that predicts the outcome yi and the response status Ri, respectively. The covariates in xi and zi may overlap with each other. The selection model is based on the joint distribution of ,Let f and f are modeled as logistic regression aswhere βdenotes the risk for radiographic pneumonia, which is the main parameter of interest; and the parameters θand λ relate the propensity of receiving chest radiography (and hence the response indicator) to covariates zi and the actual pneumonia status yi. Note here yi is missing for subjects who did not receive radiography, which leads to identification issues for this joint model [Here the parameter nt model .λas a sensitivity parameter, taking a range of fixed values from - ∞ to∞. Whenλis 0, the propensity of a subject receiving chest radiography does not depend on this subject’s radiographic pneumonia status; this corresponds to missing at random assumption in the missing data literature. Whenλis greater than 0, the propensity of a subject receiving chest radiography is positively associated with the subject’s radiographic pneumonia status. Whenλ is less than 0, having radiographic pneumonia is associated with lower propensity of receiving a chest radiography. For this specific application, it is reasonable to assume that patients with radiographic pneumonia are more likely to receive chest radiography than patients without radiographic pneumonia, and therefore λ> 0.To address the identification issues inherent in the model, we useλ to change, we can assess how the main parameters of interest is sensitive to the perturbation of the sensitivity parameter. The Bayesian modeling setup also makes it easy to predict the overall proportion of the radiographic pneumonia in the study population for a fixedλ. For illustration purpose, we use the same set of covariates for xi and zi, which includes gender , age at visit (≥ 5 years vs. < 5 years), fever during ED stay or at triage (temperature ≥ 38 °C vs. < 38 °C), and severe hypoxia (Oxygen saturation < 90% vs. ≥ 90%). [An important feature of the model is that by allowing the sensitivity parameter ≥ 90%). yi ‘s from the corresponding posterior distributions of the parameters and the posterior predictive distributions of the unobserved yi ‘s. We use proper and non-informative prior distributions for all parameters, i.e., multivariate normal priors with mean 0 and diagonal covariance matrices with a large scale parameter of 10,000 for bothβandθ. The software package WinBUGS is used to estimate the posterior distribution of the parameters [We formulate the model in a Bayesian framework (BSM) and estimate the parameters using Markov Chain Monte Carlo (MCMC) methods. The MCMC algorithm is called “data augmentation”. The algorithm iteratively draws the next values of parameters and the unobserved rameters .Out of the 14,007 children who visited emergency department for asthma exacerbation, chest radiography was performed on 4708 children (33.6%). Radiographic pneumonia was present in 280 of the 4708 children who received chest radiography (5.9%).β1: males vs. female), age at visit (β2: ≥ 5 years vs. < 5 years), fever during ED stay or at triage (β3: temperature ≥ 38 °C vs. < 38 °C), and severe hypoxia (β4: Oxygen saturation < 90% vs. ≥ 90%), respectively. For comparison purpose, the results from the following two naïve methods were also plotted on the same plots:Complete-case analysis (CC): logistic regression only includes those had observed radiographic pneumonia status, i.e., those who received chest radiography;Negative for not tested (NNT): logistic regression with all subjects which assumes negative radiographic pneumonia for those who did not receive chest radiography.Multiple imputation (MI): multiple imputation using chained equation which assumes missing at random.Figure λ should be positive because those with pneumonia are believed to be more likely to receive chest radiography; therefore, we focus on the results whenλis positive. Whenλis greater than 0, the coefficients is negative for gender from the BSM and the 95% CI does not cover 0, implying that males had significantly higher risk of having radiographic pneumonia than females among these who visited emergency department for asthma exacerbation  from the proposed Bayesian selection model were plotted on the same plots. As the sensitivity parameter goes from −4 to 4, we see a decreasing trend of the risk of having radiographic pneumonia comparing males to females. As we mentioned before, the true λ= 0; this is not surprising because when the missingness depends on the covariates but not the outcome, the complete-case analysis for fully efficient for the regression [λis sufficiently large , the BSM methods yields results close to that of NNT. This is because when λis large, it is sufficient to assume that those who did not receive chest radiography were negative for radiographic pneumonia. MI yields estimates close to CC for gender, fever at ED, severe hypoxia but smaller effect for age.For the estimates of these risk factors Fig. , the Baygression  When λisλincreases. When λis between 0 and 4, the estimates of the prevalence range from 0.056 to 0.032.Figure The prevalence of radiographic pneumonia in the current study population is less than 6%, which is relatively low. In some other diseases such as sexually transmitted diseases and hypertension, the prevalence could be much higher. Logistic models are well-known to suffer from bias for rare events, and therefore the prevalence has an impact on the proposed BSM method . On the x1 which is a binary variable from Bernoulli distribution  and x2 from a normal distribution , and b1, b2 are the regression coefficients of x1, x2 in the main outcome model, respectively,We generate two covariates, y based on a logistic regression model,and we generate the disease status R is also generated based on a logistic regression model,and the response indicator a to be −22.2 and −18.7 so the prevalence is around 2%  and 20% , and b to be −8.5 and −7.0 so that the response rates are 30% (low screening rate) and 60% (high screening rate). The combinations result in four simulation scenarios: (1) low prevalence and low screening rate; (2) low prevalence and high screening rate; (3) high prevalence and low screening rate; (4) high prevalence and high screening rate. We simulate 10,000 subjects from the study population and plot the regression estimates vs. the sensitivity parameter for each regression coefficient in Fig. We choose ng rate;  high prex1 and x2 are both 1, and the empirical bias is less than 1% for all but one coefficient, out of all simulation scenarios for the BSM method; Only the coefficient of x2 shows more than 1% empirical bias under the scenario when both prevalence and the screening proportion are low. We see an improvement in the RMSEs  with increase in either the disease prevalence or the screening proportion. As expected, the coefficient of the binary covariate x1 has larger RMSEs than the coefficient of the continuous covariates x2. In general, the BSM method achieves good coverages for both regression coefficients. The two cases with coverage probabilities less than 90% are for the coefficient of x2 when the screening probability is low. All other methods show large bias, increased RMSE and poor confidence coverage compared to BSM method.Table x1 and x2, under the four simulation scenarios. Figure x1 under (1) low prevalence and low screening rate, (2) low prevalence and high screening rate, (3) high prevalence and low screening rate, (4) high prevalence and high screening rate, respectively, while Fig. x2 under the corresponding four simulation scenarios. We see similar trends in the coefficient estimates with the change of the sensitivity parameter. However, there is substantial improvement in precision (tighter confidence bands) with the increase of prevalence or screening proportion. The estimated prevalence under different sensitivity parameters for the four simulation scenarios were plotted in Fig. Figure ng rate,  high preUnder-screening results in missing disease status or misclassified disease status when assumptions are made for those who did not receive the screening test. The goal of our method was to demonstrate the use of the Bayesian selection model for missing outcome or misclassified outcome due to under-screening. Unlike other methods that rely on assumptions , 9 or vaFor illustration purpose, we applied the proposed BSM method to a pneumonia dataset. The results showed increased risk of pneumonia in girls, which is consistent with studies from Japan . The resThe Bayesian selection model is an important tool to consider for estimating the disease prevalence and in studying risk factors of the disease, when only a subset of the target population receive the test. For studying the association of the risk factors, i.e., the regression of outcome on risk factors, this method reduces to the complete-case analysis when the sensitivity parameter is set to zero, and approximates the NNT method when the sensitivity parameter approaches infinity. Unfortunately, there is no information available to estimate the sensitivity parameter without validation sample. The choice of the sensitivity parameter can be aided by gathering information relating the propensity of receiving the test to the actual disease status. The choice of covariates in the outcome model and the response indicator model can be aided by input from substantive experts regarding the hypothesized relationship of variables with the outcome and/or the response indicator. When validation data are available, it is possible to identify the parameters in the Bayesian selection model. In future work, we plan to study how to efficiently make use of the validation data.In the current study, we developed a Bayesian selection model that jointly modeled the binary outcome and the response indicator for the case when the binary outcome may be missing or misclassified due to under-screening. The model for the response indicator relates the propensity of receiving screening to the disease status through a sensitivity parameter. The application of the model to a pneumonia data yielded results that were consistent with previous studies. The performance of the proposed method over other methods in the simulation studies demonstrated the promise of the proposed model for modeling missing or misclassified disease outcome due to under-screening.


The case relates to a bookstore owner claiming that two DM-41 hand grenades were exploded simultaneously in his store. There were three males together at the store when the explosion occurred. One was the owner who claimed that he escaped after the explosion without any harm; the other was at the corner lying down to prevent his body from the explosion effect. He survived with very minor, almost no effects.According to the hospital report, it was stated that "cuts on the right femur with sizes of 0.5x2 and 0.5x1 cm and one cut of 0.5x2,0 cm on the left food which are curable with simple medical intervention; generalized skin erosions on body with the sizes between 0,5 to 1,0 cm"; the third male was standing and killed. He was next to the lying down male. At the autopsy report it was stated that the he was killed due to the shrapnel/fragmentation effect, breaks on humerus, radius, femur and cranium; cerebral and internal hemorrhage.The males witnessed at the court that they had survived with no vital damage on their bodies, they had seen the perpetrators and heard them talking. With the fact that the deceased male was intensively affected with the fragmentation/shrapnel due to the autopsy report, it was the court’s wonder if it is possible for the survived men to have no or very minor nonfatal fragmentation effect on their bodies even being in the same room with the deceased.It was mainly aimed to test the fragmentation effect of 2 DM-41 defence hand grenades when detonated in a closed environment (an empty room with the approximately same size of the related case). The test room was empty with no secondary fragmentation sources as window glasses etc. 3 male mannequins were used as test materials. With the post blast reconstruction of the crime scene, it was aimed to determine if the test results and the autopsy report are very coherent and the persons having the direct blast effect would be expected having maximum exposure to the fragmentation. The degree of the explosion effect on environment and body is evaluated by the explosive type used and detonation parameters. The death and woundings took place due to the direct blast or blast reflection and the primary and secondary fragmentation effects. In addition to the fragmentation/shrapnel effect, broken windows and concerete etc. are the causes for severely wounding and deaths too. The severity of blast effect on human body depends mainly on the blast parameters and the duration of blast exposed which means the bigger the effect the worse the damage is. The other main factor is the distance to the detonation center. The closer to the explosion center is the cause for being exposed to the peak pressure longer which means that the bigger damage on body exists Glover, .The blast effect of post explosion causes severe woundings and deaths at very vicinity of explosion centers. It is the scientific fact that the post blast has effects on the shrapnel and other solid formations to shatter and fly around. So, the sensitivity to the blast and fragmentation depends on the distance to the explosion center and the fragmentation , the human eardrum may rupture. With an overpressure of 100 kPa (14 PSI) almost all eardrums will be ruptured. The treshold blast value for eardrum burst is . The other main reasons for being wounded or death are the structure collapses, broken windows and debris causing fragmentation effect with enough blast and velocity. The blast value for the windows break is ≤0,5 psi .Fragmentation effect can surely be effective both explosion vicitiny and long distances. The human body tolerance is very limited to the fragmentation/shrapnel effect. The scientific studies stated that the efficient minimum velocity value for a particle to penetrate the human skin is determined as 30 m/s (100 ft./s) depending on the area of the hit on the body and the particle weight  who claimed that he escaped after the explosion without any harm; the other  was at the corner lying down to prevent his body from the explosion effect. He survived with very minor, almost no effects. According to the hospital report, it was stated that “cuts on the right femur with sizes of 0.5x2 and 0.5x1 cm and one cut of 0.5x2,0 cm on the left food which are curable with simple medical intervention; generalized skin erosions on body with the sizes between 0,5 to 1,0 cm”; the third male  was standing and killed. He was next to the lying down male  are as follows;1. The shrapnel wounds were determined; (a) Total of 20 with different sizes at head, neck, arms and hands, (b) Many with the sizes between 0,1 to 1,5 cm at right leg, hip, femoral and feet, (c) Many with the sizes between 0,1 to 1,0 cm at left leg, hip, femoral and feet. 2. Multi-piece fracture and deformities at upper right orbita, upper left first tooth, 1/3 upper right humerus, bottom right femur and bottom right radius,3. The right eye emptied from bulbus oculi.1. It was distinguished; (a) ecchymosis under head skin, (b) break at outher lamina, (c) subrocnoidal bleeding at both hemisphere (d) rough edged bone defect at inner lamina, 2. Many subpleural bleeding with sizes between 0,3 to 2,0 cm on lung and congested cuts,3. Generalized subcutan bleeding on the left neck soft tissue,4. In obdomen; free blood at hepatic lobe and full cut wound at right bottom edge liver,5. At extremities inspection; (a) four demormed metal pieces were taken out from right deltoid and right femur, (b) breaks on humerus, radius, femur and cranium, (c) cerebral and inner hemorrhage due to internal organ woundings, (d) fatal shrapnel wounds at head frontal hairy part and right subcostal area.primary fragmentation, comes from the shrapnels. The particles propelled from the explosion seat and the environment is the secondary fragmentation. The blast mostly makes the buildings collaped and windows broke. The velocity of the blast travel in the air takes the free particles on and travels far away with tremendous speed and energy creating pressure of thousands of pascals which causes fatal wounds on the body. As the distance from the blast epicenter increases, the effect of blast reduces and the effect of fragments and debris propelled by the explosive becomes more important. Conventional military explosives may create multiple fragments with supersonic initial velocities  Seyhan, These flying projectiles can produce both penetrating and blunt trauma, depending on the size of the projectile and the speed at which they travel. With these velocities, the victim does not have to be in close proximity to the explosion. Individuals far from the scene of an explosion can be struck and injured by this debris . Victims that are peppered with glass are often difficult to distinguish from victims that are peppered with glass and have penetrating injuries . Composition B (60% RDX + 40% TNT) detonates with 7900 m/s detonation velocity and applying 268 kPa  peak pressure with a fatal radius of 20 m. The fragmentation effect is provided with a grooved steel sleeve  It was positively defined at the forensic laboratory that two (b) Two DM-41 hand granades originally designed for providing fragmentation effect have total of 2000–2100 grooved steel fragments (including the base plug). The initial detonation velocity is 7900 m/s and the blast value is 268 kPa .2) in very less than a second and the blast wave started to travel inside room with an initial velocity of 7900 m/s  The shock front with supersonic velocity will move inside the room and hit the walls and be reflected. It will breake the windows and gives damages to the walls Fig. .Fig. 5BrThe particles and the broken windows will be expected to propel around dragging the free particles away all causing shrapnel effect on its way Fig. .Fig. 6Dr2) in very less than a second and the blast wave started to travel inside room with an initial velocity of 7900 m/s. (a) Two handgrenades detonated inside a room with a peak pressure of 268 kPa   2000–2100 grooved steel fragments with the dragged metals, stones and broken windows will travel along the shock wave with an initial velocity of at least 7900 m/s and peak pressure of 268 kPa As the fatal wounding threshold value is around 40 psi and the peak pressure for DM-41 hand grenade is 268 kPa , it is definite that the value is between the limits of fatal wounding. So, it can be stated that a human inside this room will suffer the initial velocity and peak pressure at the highest level if not concealed behind a conceret, steel or other likewisw concelements.. The walking/running person’s balance is inclined towards front. When a body running or walking exposed to a blast with supersonic speed, it is expected to lose the balance and fall. Even with the threshold value, a moving body can be expected to fall down with 268 kPa  pressure and 7900 m/s.(c) Propelling and severe dragging of the human body exposed to the blast effect is one of the main reasons for post blast killings and woundings. The physical injuries occur not only for the direct blast effect but also for dragging of the body.It was mainly aimed to test the fragmentation effect of 2 DM-41 defence hand grenades when detonated in a closed environment (an empty room with the approximately same size of the related case). The room dimensions for the test explosion was 4,5 m × 4,75 m lenghts and 3 m height Fig. .Fig. 8TThree male mannequins were used as test materials. Mannequins were filled with sand with an amount of normal male weight between 70 and 75 kg.Mannequin-2  was lied face down on left opposite corne of the room due to the door. Mannequin-3  was positioned as standing 150 cm right and middle left of Mannequin-1. Mannequin-1  was positioned face to the corridor door with his left side was next to the open door  experts Fig. .Fig. 10TTen secons after the detonation, we went into the room and took pictures. It was observed that the room filled with dust and the situation was choking, hard to breathe in Fig. . The expThe test room was empty with no window glasses. It was carried out to test the 2 DM-41 hand granades pimary fragmentation/shrapnel effects (grooved steel sleeve) on the body Figs.  and 13.FThe corridor where the mannequin 3  standing was affected just only from the shrapnel passed trough the open door. It was observed that mannequin 1 and 2 were exposed to the shrapnel effect at a maximum level.The primary post blast fragmentation/shrapnel effects on the mannequins were stated below;On the Mannequin-2  lied face down on left opposite corner of the room due to the door  positioned as standing  positioned faced to the corridor door . The two DM-41 hand granades were exploded in the room with aforementioned standart detonation parameters and the effects fully continued through the corridor and inside the room.(2) The blast occurred at the room is very compatible with the test results and the expected standart detonation parameters for both blast and the fragmentation/shrapnel effects.(3) Due to the test results and the the autopsy reports stated, it was understood that the persons very vicitnity of the explosion seats inside the room experienced the peak pressure very positively.(4) The test results and the autopsy reports justified that the fragments/shrapnels from the steel sleeve of two hand granades traveled inside the room with a supersonic velocity and high pressure. The test results and the autopsy report were compatible with each other.(5) The aforementioned test results justify and must be expected that the persons witnessed the explosion center would have been very well suffered the detonation as the deceased experienced. The shrapnel effect, due to the scientific findings, must be expected being compatible with the test detonation results.(6) The test results and the autopsy report are very coherent that the persons inside the room and the vicinity having the direct blast effect would be expected having maximum exposure to the fragmentation if there is not a concerete or steel etc. barrier between, which means the other two males survived with very minor, almost no effects are not scientifically satisfactory due to the reconstruction of the case.


A variety of resistance states caused by different degrees of disorder can lead to the development of multilevel memory devices, which could bring a revolution to the memory industry by significantly increasing the storage density and inspiring the neuromorphic computing. This work focuses on the study of disorder-induced carrier localization which could result in multiple resistance levels of crystalline GST. To analyze the effect of carrier localization on multiple resistant levels, the intrinsic field effect (the change in surface conductance with an applied transverse electric field) of crystalline GST was measured, in which GST films were annealed at different temperatures. The field effect measurement is an important complement to conventional transport measurement techniques. The field effect mobility was acquired and showed temperature activation, a hallmark of carrier localization. Based on the relationship between field effect mobility and annealing temperature, we demonstrate that the annealing shifts the mobility edge towards the valence-band edge, delocalizing more carriers. The insight of carrier transport in multilevel crystalline states is of fundamental relevance for the development of multilevel phase change data storage.The tunable disorder of vacancies upon annealing is an important character of crystalline phase-change material Ge Chalcogenides adopted in PCM can be switched between a high-resistive amorphous (RESET) and a low-resistive crystalline (SET) state via different current pulses3. The main challenge for commercial applications of PCM is to reduce the power consumption because the RESET process requires high current density to melt the crystalline region. The prototypical PCM material Ge2Sb2Te5 (GST) has been intensively studied in the past a few years6. A lot of work have been done for the understanding of carrier transport due to its great importance for the crystalline state of PCM7. It has been reported that metal-insulator transition (MIT) upon annealing can be observed in crystalline GeSbTe compounds, and the Anderson localization resulted from the vacancy disorder is responsible for this MIT10. The control of the degree of disorder in crystalline state is important in achieving multilevel resistance states, and thus the study of the disorder-induced localization is necessary. Disorder-induced carrier localization has been investigated by low temperature transport experiments, and its origin has been revealed by density functional theory simulations11. The vacancy ordering process was also demonstrated by high angle annular dark field scanning transmission electron microscopy (HAADF-STEM) experiments13. Previous works based on the Hall effect and conductivity measurement have limitation for that the Hall effect is not expected to yield the exact number of carriers once they are localized15. Besides, a small fraction of non-stoichiometric vacancies lead to a large concentration of carriers on the order of 1020 cm−3 17, which hampers the conductivity analysis19. Due to its close relationship with carrier transport, the analysis of field effect (the intrinsic effect of a transverse electric field on surface conductance) has been well practiced in MIT mechanisms including electron correlation (Mott transition)21 and disorder 23. Field effect studies of semiconductors have been a very active area of interest for many years, because they can probe the localized states of carriers which control the electronic properties of materials26. However, carrier transport of crystalline GST observed from field effect has not been reported before. In this work, we designed a back-gated structure to measure the field effect of GST in cubic crystalline state. We analyzed temperature dependence of the field effect mobility of crystalline GST and the delocalization of carriers as the annealing temperature (Ta) increases. This study provides observation of localized carriers in different crystalline states of GST and thus enriches the research of MIT in the GeSbTe compounds as an addition to conventional transport measurement techniques. Insights of carrier transport in GeSbTe compounds have implications for the development of low power consumption and multilevel memory devices.Phase change memory (PCM) is one of the most important candidates for future nonvolatile memory devices owing to its high scalability, high speed, and long enduranceVG) is applied on this capacitor, carriers are motivated in the semiconductor. The drain voltage (VD) is applied between source and drain electrodes, and the resulting channel current (ID) is monitored to detect the field effect. The effect of the electric field is limited to a channel thickness within nanometers, owing to the presence of Thomas-Fermi screening27. The noise of channel current is comparable to or even stronger than the field effect induced current (ID) when it is high and thus, the thickness of GST film is controlled to 10 nm so that the noise is reduced and ΔID is revealing. As shown in Fig. Figure in-situ. Starting at room temperature, the heating speed was kept at a constant rate of 5 Kmin−1. As shown in Fig. Ta reaches 140 °C, the resistance of devices drops abruptly by several orders of magnitude due to the crystallization of phase change materials. On further annealing to 250 °C, the resistance of the crystalline GST reduces again by almost two orders of magnitude. As has been demonstrated by Siegrist et al. using the samples of Ge1Sb2Te48, this annealing effect in the electrical resistance of crystalline state takes place prior to the MIT transition when GST transforms into the hexagonal phase.We preformed the resistance measurements depending on annealing temperature of GST devices by thermal annealing Ta exceeds 190 °C, the field effect of devices becomes mixed up due to the high conductance . We stabilized the samples at the target Ta for one more hour to guarantee that the samples are fully crystallized. XRD measurements were performed to obtain detailed structural information about films. The GST films measured by XRD were fabricated with the same procedure as those in back gate devices, i.e., they were annealed under the same condition as the devices. Figure Ta. The XRD results of those films show that GST has a fcc structure when the Ta is between 150 °C and 200 °C. The peaks attributed to the cubic structure have been identified and indexed. The 2θ position of the (200) peak, which is the most intensive28, shifts from 29.2° to 28.3° above the 250 °C due to the phase transformation from cubic to hexagonal phase. From the XRD measurements, we further confirm that the GST films used in field effect measurements exhibit cubic structure.Our work focuses on the rather insulating samples because the carrier localization is expected to affect significantly the electrical transport. Those devices annealed between 150 °C and 190 °C were selected to carry out further field effect measurements. When the VG. A small bias was applied between the source and drain electrodes, and the resulting change in the conductance was measured. As shown in Fig. I and VG in sample with 150 °C annealing is described as an example which is expected to have highest degree of disorder in all samples. The output characteristics are completely in the linear regime. We further performed field effect measurements on the devices with increasing Ta to analyze the effect of annealing on the carrier localization. The drain current of all the evaluated device types shows a small and approximately linear dependence on VG, which indicates that the mobility of induced carriers is independent of VG. More details of each sample type have been described in the Figure Ta between 150 °C and 190 °C have been performed. As shown in Fig. Ta.Field effect measurements are shown in Fig. 29. This structure feature inevitably leads to the vacancy disorder in the Ge/Sb sublattice11. From the step annealing process, we observed the annealing effect in the electrical resistivity of crystalline state, which is consistent with the MIT demonstrated by previous research9. Our field effect analysis focuses on the insulating part of the MIT, and the XRD measurement confirms that all GST in devices used in the field effect measurements belong to the cubic system instead of a hexagonal one.The metastable crystalline GST has a rocksalt-like structure containing the vacancy fraction of around 20% (varying with compositions in GeSbTe compounds) on the cation sublattice. Te atoms occupy one fcc sublattice, while Ge and Sb atoms as well as vacancies occupy the other sublattice randomly30. Figure 32. Figure Ta between 150 °C and 190 °C. We confirm that the Fermi levels (E = 0 eV) of GST films used in field effect measurements are all located near the VBM, consistent with previous studies on phase change materials34.We analyzed the carrier transport based on the field effect results, and find that the carriers induced in the channel are not located directly at the semiconductor-insulator interface, but are instead distributed spatially inside the bulk. This feature is probably responsible for the field effect. In field effect measurement, the energy bands near the surface is displaced relative to the Fermi level. The conductance of the sample is changed by such a displacement because the carrier concentrations in the space charge region are different from the bulk concentrations. In “depletion”, carriers are removed from the semiconductor volume, whereas in “accumulation” they are added. The “inversion” of transportation type will occur if a p-type semiconductor with a large positive charge on the surface causes an n-type surface layerVG. If the field induced carriers are restricted in a region much narrower than the insulator thickness tox then the total field induced carriers per unit area Q is given byεox is the dielectric permittivity of the insulator. The total carriers consist of two components,QF is the free carrier and QL is the localized carrier. If the distance between the source and drain electrodes is L and the width of the conducting path is W, then the change in conductance ΔG isμ0 is the mobility of the free carrier. Thus, for a constant applied source-to-drain voltage VD, the field induced current ΔI can be written asThe experiment observes the change in the mobile carrier component induced by an applied gate voltage μFE. Transport in the materials studied in the present investigation takes place via carrier motion in the extended states beyond the mobility edge, so that the field effect mobility will be related to the free carrier mobility μ0, by the ratio of free carriers to total excess induced carriers.QF is much smaller than QL. For the μFE measured in GST films is significantly small, the main contribution of induced excess carrier is the localized carriers in the band tail.The quantity in parentheses in Equation  is the eμFE from the gate modulation of drain current. The relationship between the excess carrier mobility and temperature was analyzed. As shown in Fig. μFE shows the temperature activation, a hallmark of carrier localization35. Thus, we can further conclude that the field effect induced carriers are mainly localized.We obtain the mobility Ta. The μ0 remains almost constant while the Fermi level changes little with the increase of annealing temperature36, which is proven in the XPS measurements as shown in Fig. μFE is mainly due to the proportional reduction of localized carriers in all field induced carriers. We conclude that the annealing shifts the mobility edge towards the valence band edge, delocalizing more carriers37. We find evidence for that the mobility edge is approaching Femi level as the Ta increases.As shown in Fig. 38. It is hard to distinguish the contribution from disorder due to the high carrier concentration. In the field effect measurements, the change in conductance of semiconductor due to the space charge layer in which the extra carriers are induced. The important role of localized carrier in materials can be directly observed in spite of the high concentration of carriers. Through this method, we can confirm that the annealing shifts mobility edge on the insulating side of MIT, which has not been proven in previous work, to our knowledge. Hence, through gate induced current ΔI, the carrier localization in cubic GST was observed. The direct observation of localized carriers substantiates disorder induced localization in crystalline phase change materials. The field effect measurements demonstrate a new approach to explore facets of disorder-induced localization.In phase change materials, stoichiometric vacancies could result in disorder, whereas the carrier concentration is controlled by the excess vacanciesIn summary, by field effect measurements, we investigated the disorder induced localization in cubic crystalline GST. We observed the linear area of field effect in GST back-gated devices and calculated field effect mobility. The temperature dependence of field effect mobility elucidates visually that the carriers are localized and the small activation energy was acquired. This work confirms that the annealing shifts the mobility edge towards the Fermi level, delocalizing more carriers. This method is effective in studying the disorder induced MIT in the presence of high carrier concentrations. The localized carriers are observed to determine the multilevel crystalline states of GST. These results shed light on the mechanism of the electric transport of multilevel states in crystalline GST, which provides conceptual guidance for the design of high-density memory devices.2 approximately 200 nm thick. UV-lithography was used to pattern the GST thin film channel and source/drain electrodes. The channel size was 10 μm × 100 μm. The GST film was deposited by DC magnetron sputtering from stoichiometric targets. The back ground pressure was kept smaller than 1 × 10−6 mbar and the process pressure was of the order of 5 × 10−3 mbar using argon as sputter gas. The thicknesses of the two layers were controlled via the deposition time and afterward by means of step height measuring instrument. The fabricated devices were capped with 50 nm ZnS:SiO2 (80:20) by RF sputtering, which can render the devices air stable.The devices were fabricated by thermally oxidizing a heavily n-doped crystalline silicon substrate to form a layer of SiO2Sb2Te5 by EDS.The GST films used for XRD and XPS measurements were prepared at the same time and conditions as devices. The cross sectional samples for TEM were performed by the focused ion beam system (FIB). To minimize the beam damage during the FIB sample preparation, the multiple processes with reducing voltages and beam current are used. The composition of the films was confirmed as nearly GeVG can range up to ± 40 V, and all measurements were performed in the Ohmic region. The low temperature measurement was carried out in a Quantum Design Physical Property Measurement System (PPMS).To obtain the field-effect modulation, the source was biased at 1 V and resulting drain current was monitored using Aglient B1500 semiconductor analyzer. The applied Dataset1


Many stepped wedge trials (SWTs) are analysed by using a mixed‐effect model with a random intercept and fixed effects for the intervention and time periods (referred to here as the standard model). However, it is not known whether this model is robust to misspecification.We simulated SWTs with three groups of clusters and two time periods; one group received the intervention during the first period and two groups in the second period. We simulated period and intervention effects that were either common‐to‐all or varied‐between clusters. Data were analysed with the standard model or with additional random effects for period effect or intervention effect. In a second simulation study, we explored the weight given to within‐cluster comparisons by simulating a larger intervention effect in the group of the trial that experienced both the control and intervention conditions and applying the three analysis models described previously.Across 500 simulations, we computed bias and confidence interval coverage of the estimated intervention effect.We found up to 50% bias in intervention effect estimates when period or intervention effects varied between clusters and were treated as fixed effects in the analysis. All misspecified models showed undercoverage of 95% confidence intervals, particularly the standard model. A large weight was given to within‐cluster comparisons in the standard model.Statistics in Medicine published by John Wiley & Sons Ltd.In the SWTs simulated here, mixed‐effect models were highly sensitive to departures from the model assumptions, which can be explained by the high dependence on within‐cluster comparisons. Trialists should consider including a random effect for time period in their SWT analysis model. © 2017 The Authors.  Each group begins to receive the intervention at a different time so that all clusters start the trial in the control condition, and by the end of the trial, all clusters are receiving the intervention.The control and intervention conditions can, in principle, be compared in two directions known as the vertical and horizontal comparisons In practice, most analysis methods for SWTs incorporate information from both the vertical and horizontal comparisons in the intervention effect estimate and so need some way to adjust for period effects In the context of SWTs, we are most interested in estimation of the intervention effect and how robust this effect is to misspecification of the intervention effect itself as well as misspecification of the period effect. Previous research has found that misspecifying the random effects led to biased effect estimates as well as biased precision of estimates The importance of specifying the period effect correctly will depend on how much the horizontal comparisons contribute within the model. This has not been explored in the literature. If a large weight is given to this comparison, any residual confounding of the intervention effect by the period effects could lead to a biased estimate of the intervention effect.In this paper, we will explore both issues with a simulation study comparing the standard model with other mixed‐effect models, focusing on a binary outcome with cross‐sectional measurements. We then run a second set of simulations to explore the weight given to horizontal comparisons by each analysis model. Following the simulation studies, we explore the impact of misspecifying analysis models in our motivating example.2There has been much debate in recent literature about the results of a reanalysis of a highly cited SWT that investigated the effect on school attendance of a mass deworming intervention for school children in Kenya The period effects varied between clusters. The standard model assumes that the period effects are common to all clusters. This could lead to a biased estimation of the intervention effect through biased estimation of the period effects.The intervention effect varied between clusters. The standard model assumes that the intervention effect is common to all clusters. Treating an effect that truly varies as a fixed effect has been shown to lead to biased estimation of that covariate In the reanalysis of this trial, it was found that the odds ratios (ORs) for school attendance for year 1 and year 2 were both smaller when analysed individually (OR = 1.48 and 1.23 respectively) than the OR given by the standard model when the data were pooled from both years (OR = 1.82) In this paper, we first used a simulation study based on the motivating example to explore the effect of ignoring variability between the clusters in the period effect and intervention effect in the analysis of SWTs. Second, we hypothesised that the effect of misspecification would by highly influenced by the weight given to horizontal comparisons in each analysis model and so also performed a further set of simulations to investigate this question. We then analysed the motivating example with different analysis models and compared the results in light of the findings of the simulation studies.33.1To investigate the impact of ignoring heterogeneity between clusters in the period effect and intervention effect, we compared analysis models that assumed these effects were common to all clusters (the standard model) to analysis models which allowed these effects to vary between clusters. We performed this with data in which the true underlying period effect and intervention effect were either common to all clusters or varied between clusters. A description of the scenarios we used to compare the analysis models is given, followed by the three analysis models we compared. A summary of the data scenarios simulated is given in Table We used the same trial design as our motivating example with clusters randomised into three groups and followed for two time periods. During the first period, only the first group had received the intervention, and during the second time period, the first and second groups had received the intervention. The third group never received the intervention. This trial design was chosen due to its simplicity; because there are only two time periods, the period effect is simple to model. The horizontal comparison is only possible in one group; this allowed us to explore the weight given to this comparison. To mimic the motivating example and to avoid issues with small sample size, we assigned 25 clusters to each group and the number of observations in a cluster in each time period was drawn from a log‐normal distribution , where the standard model would have a correctly specified period effect.Common period effect, low variability:The period effect was common to all clusters with between‐cluster variance = 0.25. This was the amount of between‐cluster variability observed in year 2 of the motivating example. This represents a simple scenario with a lower ICC (ICC = 0.05), where, again, the standard model would have a correctly specified period effect.Varying period effect, decreasing variability:The period effect varied between clusters with the variability between the clusters decreasing from the first period to the second period. The initial between‐cluster variance was 1.81, and the period effect variance was 1.89. The decrease in variability from period 1 to period 2 resulted from a negative covariance between the initial value and the period effect of −1.72. This complex scenario reflects the underlying trends seen in the motivating example. In this scenario, the standard model would have a misspecified period effect.Varying period effect, stable variability:The period effect varied between the clusters, but the between‐cluster variance remained the same for both periods. Here, the initial between‐cluster variability and period effect variability remained the same as in scenario (3), but the covariance was reduced to −0.94. This scenario was chosen to assess the effect of a varying period effect without the additional complication of the between‐cluster variation reducing in the second period. In this scenario, the standard model would have a misspecified period effect.We simulated two scenarios for the intervention effect; these were not based on the motivating example:An intervention effect that was common to all clusters. We simulated an intervention effect log(OR) = 0.41  for all clusters. We also simulated log(OR) = 0 to calculate the type I error rate. In these scenarios, the standard model would have a correctly specified intervention effect.An intervention effect that varied between clusters drawn from the distribution The cluster‐level distribution of the outcome in the first period and the change from period 1 to period 2 (the period effect) was based on group 3 of the motivating example. This group was chosen because it did not receive the intervention. We modelled the log‐odds in the first period and the log‐OR period effect from the motivating example as a bivariate normal distribution. This gave mean values for the log‐odds in period 1 and log‐OR period effect, together with a 2 × 2 covariance matrix. This distribution described the outcome and how it varied between the clusters in each period. The mean values were used in all the simulation scenarios, but we manipulated the covariance matrix to create four scenarios of how the outcome varied between the clusters and periods  = 1.5 in group 2 of the trial but kept an intervention effect in group 1 of log(OR) = 0.41. An unbiased intervention effect estimate from horizontal comparisons alone would have an expectation of 4Each simulated data set was analysed with three analysis models, each making different assumptions about the period effect and intervention effect.4.1i in year j for observation k, i received the intervention in year j, ui~N0σu2 is a random intercept allowing for variability in the outcome between clusters.First, we used the standard method of analysis This model assumes that the period effect and the intervention effect are common to all clusters so is a misspecified model in scenarios where either the period effect or intervention effect varied between clusters.4.2uivi~MVN00σu2σu,v2σu,v2σv2 are a random intercept and random effect for period respectively.Second, we added a random effect for period to the standard model:This model assumes that the intervention effect is common to all clusters but allows the period effect to vary between clusters. It is a misspecified model in scenarios where the intervention effect varies between the clusters.Sometimes, other literatures have used a different model to allow the period effect to vary between the clusters 4.3uizi~MVN00σu2σu,z2σu,z2σz2 are a random intercept and random effect for intervention respectively.Third, we added a random effect for the intervention to the standard model:This model assumes that the period effect is common to all clusters but allows the intervention effect to vary between clusters. The model is a misspecified model in scenarios where the period effect varies between the clusters.σu,z2) but allows this covariance to be zero, as is the case in our simulation study.Whilst the random period and random intervention models allow for variability in the period and intervention effect respectively, they can estimate a variability of close to zero if the effect is common to all clusters. The random period model is correctly specified in the scenario with common period effect, and likewise, the random intervention model is correctly specified in the scenario with common intervention effect. Similarly, the random intervention model allows for a covariance between the intervention effect and the intercept , and the IQR of the intercept, intervention effect, and period effect estimates from the 500 simulations. We calculated percentage bias asP < 0.05 against a null of the intervention effect OR = 1.We calculated the coverage of 95% CIs as the proportion of simulations with the true effect contained within the 95% CI of the estimate. We calculated the type 1 error rate as the proportion of simulations with true OR = 1 with In the set of simulations with a different intervention effect in group 2 (simulation study 2), we compared the mean of the intervention effect estimates with the horizontal intervention effect of log(OR) = 1.5.r version 3.2; the lme4 package was used for mixed‐effect models.Simulations were run in 66.1The standard model converged in all simulations for both simulation studies. When either the period effect or the intervention effect varied between clusters, the random period and random intervention models also converged in >99% of all simulations. However, when both period effect and intervention effect were common to all clusters, the random period model failed to converge in 3% to 9% of simulations and the random intervention model failed to converge in 4% to 33% of simulations. Estimates from these models were excluded from performance statistics. Further details of convergence of the models are given in 6.26.2.1Figure Where there were common period and intervention effects, all three models performed similarly, with estimation of the intervention effect in line with the true underlying effect.Where the period effect varied between the clusters, only the random period model gave unbiased estimates of the intervention effect. Depending on the scenario, the standard model had between −20% and −8% bias and the random intervention model between −51% and −8% bias. Bias was larger when the period effect varied with decreasing variability than with stable variability but was similar regardless of whether there was a common or varying intervention effect. We also observed bias in the period effect estimates and intercept estimates from the standard model and random intervention model .Where the intervention effect varied between the clusters and there was a common period effect, the random intervention model and the random period model gave unbiased estimates of the intervention effect. Only the standard model intervention effect estimates had substantial bias (−9% and −16% bias for common period effects with high and low variabilities respectively).Where either the period effect or intervention effect varied between clusters, the standard model intervention effect estimates had greater variability compared with the random period model or random intervention model. Differences were larger when the period effect varied between clusters than when the intervention effect varied between clusters. For example, the standard model intervention effect estimates were 3.6 times as variable as the random period model estimates when the period effect varied between clusters with decreasing variability with common intervention effect, whereas the standard model intervention effect estimates were 1.5 times as variable as the random intervention model estimates when the intervention effect varied between clusters with common period effect with high variability.6.2.2In scenarios with a common period and intervention effect, 95% coverage was maintained regardless of the analysis model and the estimated standard errors were similar across analysis models Figure .When period effect or intervention effect varied between clusters, the standard model gave standard errors that were markedly smaller than the random period model and random intervention model. The mean intervention effect standard error from the standard model was less than 0.33 and 0.26 times the mean standard error of the random period model and random intervention model respectively.The inappropriately small standard errors given by the standard model were in part explained by downward bias in the estimation of between‐cluster variability . For exaThe bias in estimates, standard errors, and increased variability in estimates led to undercoverage of the 95% CIs of the intervention effect estimates Figure . For theType 1 error rates followed the same patterns as coverage .6.3Figure All analysis models gave a mean estimated intervention effect close to the group 2 effect when there was a common period effect and a common intervention effect; this was the case in the high and low variability scenarios. This suggests that, in these scenarios, the intervention effect is largely estimated from horizontal within‐cluster comparisons in group 2; groups 1 and 3 appeared to contribute to estimation of the period effect but had little influence on the intervention effect estimate.The standard model estimates remained close to the group 2 intervention effect in all scenarios. The downward bias we observed in our first set of simulations suggests that at least some of the movement away from the group 2 effect is because of bias and not because of a reduction in the contribution of the horizontal comparisons. This implies that the standard model was continuing to estimate the intervention effect largely from horizontal comparisons in group 2.In contrast, when the period effect varied between clusters, the random period model gave intervention effect estimates much further from the horizontal comparison estimates. This implies that the horizontal comparisons in group 2 could not contribute as much information to the analysis because there was less certainty about separating the period effect and intervention effect in these comparisons. This was similar in the scenarios where the intervention effect varied between clusters but the period effect was common for both the random period and random intervention models, but to a smaller degree.7For our motivating example, we hypothesised that the standard model gave a larger intervention effect than either of the two time periods analysed separately because the standard model was misspecified by ignoring variability in either the period effect or the intervention effect. Our simulation study suggests that this is not the case because we would expect the standard model to underestimate the intervention effect with these types of misspecification, rather than overestimate the effect. However, we also found that the standard model gave a very large weight to the horizontal comparisons. This does help to explain the counterintuitive results seen in the motivating example We reanalysed the deworming trial by using the three analysis models investigated in the simulation study and additionally looked at years 1 and 2 separately by using a mixed‐effect model with a fixed effect for intervention and a random intercept to attain estimates for the intervention effect from vertical comparisons. In line with the published reanalysis of this study, we ignored pupil‐level clusters from multiple observations of the same pupils; this is in line with research suggesting that it is sufficient to adjust for the highest level of clustering alone, known as passing the buck The results in Table We found that the standard model combining data from both years of the study gave a larger estimate of the intervention effect than either year analysed separately, which is as was found in the reanalysis Adjusting for variation between clusters in the period effect or intervention effect (i.e. using either the random period model or random intervention model) increased the intervention effect standard error and reduced the intervention effect towards the null. Both approaches gave an intervention effect estimate between the estimated effect in year 1 and year 2. This suggests that the horizontal comparisons are contributing less to these analysis models than to the standard model; this is consistent with the findings of our second simulation study into the contribution of the horizontal comparisons.p < 0.001), and the random intervention model found strong evidence of variability in the intervention effect (p < 0.001). Because the period effect and intervention effect are confounded with one another, evidence of variability in the intervention effect could be caused by variability in the period effect or vice versa. The random period model estimated a between‐cluster covariance matrix similar to the simulation study scenario with varying period effect with decreasing variability. The random intervention model estimated lower variability between clusters in the intervention condition than in the control condition because of the reduced variability in year 2. This is a scenario that we did not consider in our simulation study where we only investigated a scenario with greater variability in the intervention condition. Inspection of the data suggests that the random period model is the most appropriate one. A mixed‐effect model with a random effect for period run on observations from group 3, which never received the intervention, finds strong evidence of variability in the period effect (p < 0.001). But a mixed‐effect model with a random effect for intervention run on observations from groups 1 and 3, where the intervention effect is not confounded with the period effect, finds no evidence of variability in the intervention effect (p = 0.34).The random period model found strong evidence of variability in the period effect (p = 0.03). The effect found by using this model is weaker, both in terms of absolute size and level of statistical significance, than the effect found by using the standard model. There are still limitations in these data and this analysis, on which further information has been published elsewhere The random period model suggests that there is some evidence that the deworming intervention increased school attendance .Given that the mixed‐effect model can be so sensitive to model assumptions, other analysis methods should be considered. This choice should be prespecified and prior knowledge used to justify the assumptions made by the chosen analysis method. We found the random period model to be the most robust of the models considered, but there was still undercoverage of CIs in some scenarios. Some have suggested using permutation tests on the standard model We have only considered a limited range of designs in this simulation study. We used a very simple SWT design to make the analyses as transparent as possible; this design only had two steps, and not all clusters received the intervention in the course of the study. Further research is needed to confirm that our findings hold for other SWT designs. In more traditional SWTs, all clusters receive both the control and intervention conditions, and so all clusters contribute horizontal comparisons. Because the problems we highlight arise from the horizontal comparisons, this might exacerbate the problems we identified. We have only considered two values for the ICC when the period effect was common to all clusters and have not assessed the effect of ICC when period effects vary between clusters. In scenarios where these effects varied between clusters, the baseline ICC was 0.20, which, in many contexts, would be considered large. Additionally, there was large variability in the period effect; the effect of a less variable period effect needs further exploration. It is not known how common it is for the period and intervention effects to vary between clusters in practice; however, we have based this simulation on real trial data. Large clusters were used in the simulation study to reflect the motivating deworming trial; however, similar results were seen with a smaller mean cluster size of 250 (data not shown). We used a large number of clusters in each group to avoid small sample issues.Whilst further research is needed to explore the potential for bias in a wider range of designs and settings, we have demonstrated that there is a potential for the standard model to give biased intervention effect estimates and undercoverage of CIs. These simulations provide clear evidence that the standard model for analysis of SWTs can be both highly sensitive to the data meeting the model assumptions and highly dependent on non‐randomised horizontal comparisons. We urge those conducting SWTs to ensure an appropriate analysis is used.Data S1: Covariance matricesData S2: Model parameterisationData S3: Table of convergence of analysis models by simulation parametersData S4: Table of mean intervention effect log odds ratio estimates from simulationsData S5a: Figure of estimated interceptsData S5b: Table of mean intercept log odds estimates from simulationsData S6a: Figure of estimated Period effectsData S6b: Table of mean period effect log odds ratio estimates from simulationsData S7: Table of mean standard error estimates from simulationsData S8: Table of coverage of 95% confidence intervalsData S9a: Estimation of intercept between‐cluster varianceData S9b: Table of mean intercept variance (between‐cluster variance) estimates from simulationsData S10a: Figure of Type 1 errorData S10b: Table of Type 1 error rate of simulationsData S11: Deworming trial data cleaningClick here for additional data file.