This study aims to analyze the effects of a novel dual-bearing shoulder prosthesis and a conventional reverse shoulder prosthesis on the deltoid and rotator cuff muscle forces for four different arm motions. The dual-bearing prosthesis is a glenoid-sparing joint replacement with a moving center of rotation. It has been developed to treat rotator cuff arthropathy, providing an increased post-operative functionality.A three-dimensional musculoskeletal OpenSim® model of an upper body, incorporating a natural gleno-humeral joint and a scapula-thoracic joint developed by Blana et al. , was used as a reference for the natural shoulder. It was modified by integrating first a novel dual-bearing prosthesis, and second, a reverse shoulder prosthesis into the shoulder joint complex. Four different arm motions, namely abduction, scaption, internal and external rotation, were simulated using an inverse kinematics approach. For each of the three models, shoulder muscle forces and joint reaction forces were calculated with a 2 kg weight in the hand.In general, the maximal shoulder muscle force and joint reaction force values were in a similar range for both prosthesis models during all four motions. The maximal deltoid muscle forces in the model with the dual-bearing prosthesis were 18% lower for abduction and 3% higher for scaption compared to the natural shoulder. The maximal rotator cuff muscle forces in the model with the dual-bearing prosthesis were 36% lower for abduction and 1% higher for scaption compared to the natural shoulder. Although the maximal deltoid muscle forces in the model with the dual-bearing prosthesis in internal and external rotation were 52% and 64% higher, respectively, compared to the natural shoulder, the maximal rotator cuff muscle forces were 27% lower in both motions.The study shows that the dual-bearing shoulder prosthesis is a feasible option for patients with rotator cuff tear and has a strong potential to be used as secondary as well as primary joint replacement. The study also demonstrates that computer simulations can help to guide the continued optimization of this particular design concept for successful clinical outcomes. Total shoulder arthroplasty is broadly classified into two categories: anatomical total shoulder arthroplasty and reverse total shoulder arthroplasty. Anatomical total shoulder arthroplasty retains the orientation of the natural joint and involves replacing the gleno-humeral joint with an anatomical shoulder prosthesis (ASP), wherein the natural humeral head is replaced by a ceramic or highly polished metal ball connected to the humerus by a stem. The ball head articulates with a concave polyethylene (PE) component, which is fixed onto the glenoid surface. Damage to the polyethylene glenoid component of the ASP and loosening of its fixation are the most common modes of failure –3.With the overall goal of reducing the high failure rates of the "conventional" ASP, P. Grammont developed in 1985 the reverse shoulder prosthesis, wherein the ball head is fixed in the glenoid cavity instead of the humerus, while the PE cup is implanted into the humerus –6. Both,The growing popularity of reverse shoulder arthroplasty notwithstanding –14, this41hemiverse AG [In an attempt to address these limitations, a new shoulder prosthesis with a novel dual-bearing design, the so-called dual-bearing shoulder prosthesis (DBSP), has recently been developed and patented by the Swiss company verse AG , 26. OneOur overall goal is to assess the biomechanical performance of this new and promising prosthesis design and, eventually, to assess its performance in vivo and in comparison with existing solutions.An ideal scientific study would be to implant the DBSP into patients and compare performance and outcome with conventional designs. However, from an ethical viewpoint, it is necessary to first make preliminary assessments to verify the claims with respect to superior  biomechanical performance compared to the existing solutions. Furthermore, measuring in vivo joint or muscle loads to assess biomechanical performance is not readily feasible, and computational musculoskeletal modeling and simulation is a valuable technique to analyze the influence of a prosthesis on the muscle forces in silico. Hence, in a first step, our study focuses on comparing the computed joint reaction forces as well as the demand on the muscles when using the novel implant with the conventional counterparts, since this is often an accepted design criterion as well as a criterion for determining placement of the implant by the surgeon during surgery.Several studies have shown the power of 3D dynamic rigid multibody simulation tools for determining movements and internal loads within the musculoskeletal system. Opensim® is one of the software suited for simulating and predicting muscle forces and also contains a repository of models developed by research groups worldwide. The software was developed by the National Centers for Biomedical Computing at Stanford University and is an established open source software . This toPeak reaction forces at the joint, as well as peak forces generated in the individual shoulder muscles during four different movements were assessed in each of the three musculoskeletal models: NS, DBSP, and RSP model Table . Of thesConsidering first, the two large ROM movements, the novel DBSP model generated the lowest joint reaction forces of all the three models during abduction Fig. A. JRF weThe shoulder muscles investigated fall into two groups: deltoid muscle group consisting of the anterior, middle and posterior deltoids, and the rotator cuff group, which comprised the supraspinatus, infraspinatus teres minor and subscapularis. Individual muscle peak forces are delineated in Table The total deltoid muscle group peak forces in the DBSP model are similar to those generated in the RSP model for abduction  after subscapularis muscle was seen in the teres minor for the DBSP model, compared to a relatively modest 12% reduction in the RSP model.Muscle activation is the ratio of actual muscle force to the maximum isometric force that a muscle can generate. Its value lies between 0 and 1 and it implies the amount of relative effort of the muscle element. For each muscle group, the sum of peak forces produced by each muscle element in that group during a particular motion was divided by the sum of maximal isometric forces of those muscle elements to calculate the peak muscle activation in order to derive the average effort of the muscle system for comparison purpose.Table Our overall goal was to assess the biomechanical performance of the dual-bearing shoulder prosthesis, which purportedly claims to address limitations in the currently available reverse shoulder prosthesis designs. In this initial study, we adopted an in silico approach, wherein we deployed a validated computational musculoskeletal model of the shoulder joint complex to estimate the gleno-humeral joint reaction forces, and the shoulder muscle forces and activations (muscle effort). We then compared these outputs between models adapted for the natural shoulder, the "conventional" reverse shoulder prosthesis and the "novel" dual-bearing shoulder prosthesis for four common shoulder movements: abduction, scaption, internal and external rotation. This approach enabled a verification of the claims with respect to superior, or, at the very least, equivalent biomechanical performance of the dual-bearing prosthesis compared to the "conventional" reverse shoulder prosthesis. In this regard, the biomechanical performance of the DBSP can be considered to benchmark against RSP, which is the currently available design.The results confirm the findings from other studies, that the RSP generates lower deltoid muscle forces by increasing the moment arm during abduction compared to the NS , 7, 8. FFor the DBSP, the peak deltoid muscle forces were lower in abduction compared to the NS model Fig. , in scapNot only did the DBSP result in lower JRF than the RSP during abduction and scaption—the two large ROM movements, but these value were either equivalent or lower than those generated in the natural shoulder. Although the forces were much larger than for the natural shoulder during internal and external rotation, they were quite similar to the joint forces developed in the RSP model. Assessing the magnitude of joint reaction forces is useful to understand the risk of implant failure due to dislocation or outright damage on the one hand, but also to assess longer term risks of wear-related complications. In this regard, given that the DBSP model generated lower or equivalent JRF to the RSP model indicate that the DBSP performed quite favorably in comparison to the RSP.The basic musculoskeletal shoulder model used in this research is a verified one originally built in SIMM , incorpoWith respect to muscle forces, peak DBSP forces in the deltoid muscle group were similar to those generated in the RSP for all the four movements investigated, again implying an equivalent biomechanical performance in this regard.The most notable outcome in this study related to the efforts (force and activation) elicited in the rotator cuff muscles for the four movements studied. The DBSP model generated significantly lower forces compared to the model representing the conventional reverse shoulder prosthesis. Furthermore, the subscapularis, which plays a substantial role during internal rotation, also developed significantly lower forces during internal rotation in the DBSP model compared to the NS model, but a much more modest, yet notable reduction compared to the RSP model. Similarly, the teres minor, which has an important role in external rotation, also developed significantly lower forces with the DBSP model compared with both, the natural shoulder and the RSP models. With regard to abduction and scaption, it is generally accepted that the deltoid muscles will generate less force in the prostheses with respect to the natural shoulder. While this certainly held true, the novel DBSP model also revealed results similar to the RSP. This indicates that, under the made assumptions, the new design is better, or, at the very least not inferior to the conventionally available RSP design when assessing demands placed on the shoulder muscles.Overall, these results indicate that the novel DBSP design can be a feasible alternative for patients suffering from severe rotator cuff arthropathy, following which rotator cuff muscle activity is severely compromised.The values of the muscle activation reported in Table In reality, the glenoid ring is just supported to the glenoid. However, the acromion, the coracoid process, and the surrounding muscles limit the motion of the ring within the shoulder joint complex.One shortcoming of this investigation is that the computer simulation predicts the behavior just after implantation and does not consider the effect of healing and muscle relaxation with the time. Furthermore, this study did not take into account patient-specific anatomic differences that may also affect muscle forces and JRF . This liAlthough preliminary tests on cadavers with inserted dual-bearing prostheses revealed good functionality of the DBSP without dislocation, further tests will need to be conducted to quantify and confirm the stability of the novel design during different movements, which could not be addressed with the computer simulations.Despite the limitations of a computational study, as mentioned above, it should be noted that the results of this initial step in the evaluation of the DBSP are promising. The results of this study can provide the basis for future investigations, including in vivo studies wherein the kinematics of the shoulder joint containing the novel prosthesis can be recorded using techniques such as motion capture or videometry. This dataset can then be further compared to the performance of the natural shoulder and reverse shoulder prostheses to help gain a better understanding of the behavior of this novel prosthesis in vivo.With the help of future validation studies and a proper understanding of how to incorporate computer simulation testing in biomechanics, new prostheses and innovative design variants in TSA can be explored and optimized before their use in clinical trials.This research presents a biomechanical comparison of generic musculoskeletal shoulder models, including a NS model and two models including a shoulder prosthesis. Furthermore, the study shows that the novel dual-bearing shoulder prosthesis can be a promising option for patients with rotator cuff arthropathy because of its lower rotator cuff muscle forces during all four arm motions compared to the reverse shoulder prosthesis. Peak values of deltoid muscle force for reverse and dual-bearing designs were in a similar range during abduction and scaption. Low peak muscle activation values in all models for anterior, middle and posterior deltoids during the four motions reveals that the muscles were not investing high effort when holding 2 kg weight in the hand. Ultimately, this study demonstrates that computer simulation can be a useful tool in the development phase to assess the effects of prosthesis design on the biomechanics of the human musculoskeletal system.An existing OpenSim® 3D musculoskeletal model of an upper body incorporating the shoulder joint complex with all involved muscles was used as the NS model in this study Fig. A and B. To develop the DBSP and RSP models, the NS model was then adapted, wherein the "natural" gleno-humeral joint was replaced by the virtual DBSP or RSP prosthetic joint respectively, as described in the next two sections. As in the original model, no friction between any gliding surfaces was considered in all the three models, reducing the computational time significantly and increasing the numerical stability (convergence of results). This assumption relies on the low coefficient of friction within the natural joint and prostheses (below 0.05 between the UHMWPE and CoCr). An analytical calculation showed that the contribution of friction to the joint reaction moment is below 4% for the DBSP and RSP, see Appendix A. Hence, the contribution of friction is negligible, especially for the present comparison purpose.The dual-bearing shoulder prosthesis concept and design is based on two patents , 26. TheBefore being implanted, the ball head (C) is mounted in the PE-bearing (B). Then, the B–C assembly is inserted in the glenoid ring (A). A snap-in system avoids the unlocking between the B and A parts. The offset adapter (D) is then connected to the part C by press-fitting and secured with a screw. Finally, the A–B–C–D assembly is fixed to the humerus stem by press-fitting and secured with a screw and is supported proximally to the following bodies: medially by the glenoid , superiorly by the acromion and anteriorly by the coracoid. Although A and C parts are made of cobalt–chromium (CoCr) alloy, part B of polyethylene, and part D and the humerus stem of titanium (Ti) alloy, all components are assumed as rigid bodies in the model. Part B has a hinge joint in respect to part A which means that part B rotates about the central axis of part A. Part C is connected to part B with restriction due to the engagement of the ball head protrusion in the groove of part B. Furthermore, part C can also rotate about the central axis of its protrusion. Thus, the connection between part C and part B represents a gimbal joint. The scapula–A, C–D and D–humerus stem connections are assumed as welded together (fixed joint) in the model. The fixed joint between scapula and part A allows the analysis of the joint reaction forces (JRF) between scapula and glenoid ring in OpenSim®. In reality, the DBSP does not require glenoid component fixation, it is placed without anchorage. The stabilization is ensured by the acromion, the coracoid process, and the muscles.The prosthesis system possesses two CORs, one of the joints between the PE-bearing (B) and the glenoid ring (A) and the other between the ball head (C) and the PE-bearing (B). Both CORs are offset from each other. The connections between the parts of the prosthesis and between the prosthesis and the skeleton in the dual-bearing model are summarized in Table The components of the implant are produced in different sizes in order to allow a best fit with the patient’s anatomy. For this comparative study, the prosthesis with a 55 mm outer diameter of the glenoid ring was chosen with a medium-sized (M) offset adapter. An ellipsoid wrapping surface with principle axes diameters of 75 mm, 75 mm and 28 mm was used in the model to avoid a penetration of the muscles into the DBSP. The wrapping surface provides a virtual deflection support for the middle line of the muscle. In order to deflect the deltoid muscle elements realistically, the diameter of the implant wrap surface was increased by half of the thickness of deltoid muscle elements (estimated to 20 mm). The glenoid ring was fixed to the glenoid fossa and tilted 10° downwards to the glenoid plane  approach was used to derive rotational joint kinematics from anatomical surface marker-based motion capture data of a single subject, named subject S4, from the shoulder movement database available on SimTK.org . The motAll three models used the same basic set of muscles. The main physical muscle parameters, like the maximum isometric force, optimal fiber length, tendon slack length and the location of the insertion and origin points of the muscles, were taken unchanged from the NS model. Actuators were added to the thorax in all the models to better follow the driven motion. Elbow and wrist were kept fixed during the simulations. The supraspinatus muscle was deactivated during simulation on RSP and DBSP models, considering that supraspinatus is usually impaired during surgery.Besides the self-weight of the body parts, a weight of 2 kg rigidly attached to the hand, simulating a held object, was defined for the simulation of the different arm motions in each of the three models. The starting position of IR and ER is the palm of the hand looking to posterior.In order to validate our model, the natural shoulder (NS) model simulation results for abduction motion were compared to a unique experimental study by Bergmann et al. Second, JRF results for abduction and scaption motion from our reverse shoulder prosthesis model simulation were also compared to results reported by Constantini et al.  for the Besides this, joint reaction forces during abduction and scaption for the Newcastle reverse shoulder model used by Costantini et al.  and the


The quality and acceptance of machine learning (ML) approaches in cardiovascular data interpretation depends strongly on model design and training and the interaction with the clinical experts. We hypothesize that a software infrastructure for the training and application of ML models can support the improvement of the model training and provide relevant information for understanding the classification-relevant data features. The presented solution supports an iterative training, evaluation, and exploration of machine-learning-based multimodal data interpretation methods considering cardiac MRI data. Correction, annotation, and exploration of clinical data and interpretation of results are supported through dedicated interactive visual analytics tools. We test the presented concept with two use cases from the ACDC and EMIDEC cardiac MRI image analysis challenges. In both applications, pre-trained 2D U-Nets are used for segmentation, and classifiers are trained for diagnostic tasks using radiomics features of the segmented anatomical structures. The solution was successfully used to identify outliers in automatic segmentation and image acquisition. The targeted curation and addition of expert annotations improved the performance of the machine learning models. Clinical experts were supported in understanding specific anatomical and functional characteristics of the assigned disease classes. In recent years publications and product developments have shown the potential of artificial intelligence systems in cardiovascular medicine –4. Especvia conventional clinical parameters such as the stenosis degree or the ejection fraction  can be used to show underlying features of the classification The FDA guidelines address the problem of the need for AI model adaptation and improvement through retraining, and suggest an efficient dynamic process for the development and quality assurance of DL/ML methods in medical image processing . The docWe hypothesize that a dedicated setup for the training and application of machine learning methods with an expert-in-the-loop approach can speed up and improve the training of the AI models for image processing and multi-modal classification. Furthermore, it can support the clinical expert in exploring and understanding the analyzed datasets.The existing infrastructure and tool solutions presented in the previous paragraphs address all aspects required to set up an environment that supports the development and application of machine learning methods for the integrated usage of cardiac MRI data in multi-modal data classification. Based on these building blocks, we present a concept for a central environment that supports dynamic machine learning with experts in the loop. This central infrastructure should manage data and the training and inference of machine learning models for multi-modal cardiac data interpretation. We envision central modules for cardiac structures segmentation, an automated pre-processing and features extraction process, and a multi-modal cardiovascular disease classification. To integrate clinical experts into the loop we suggest an interactive exploration of the extracted data and a disease hypotheses generation method. Furthermore, this module should provide an interactive data correction and data integrity check, as well as dynamic updates of machine learning models. We test the presented setup with two use-cases and publicly available data from MICCAI challenges on image-based disease classification: the ACDC challenge for the classification of cardiomyopathies  and the We propose a modular web-based software environment to support co-learning and comprehensive analysis of cardiovascular imaging data . The arcFor integration into the clinical infrastructure, DICOM network services  are usedThe data model is essential for the traceability of the origin of classification results. The image processing part of the application is developed with MeVisLab . For datThe classifications are based on features describing local and global cardiac function, and radiomics features. We apply eight-fold nested cross-validation to select relevant feature classes. Moreover, during the cross-validation, we perform a model selection of five classifiers and their respective hyperparameters and train a classifier as described in Ivantsits et al. .We perform a feature importance analysis as  proposedThe interactive visualization is designed to support the evaluation, validation, and hypothesis generation. It is provided as a web-based tool for the clinical experts .The interface provides an overview of relevant features for a given patient cohort. These features are identified by the feature importance analysis of the machine learning module. In order to be sure that features such as, e.g., gender are always considered, users can also select features to be included in the exploration view. The parallel coordinate plots (PCP) visualize the multi-dimensional data as line sets with points representing the datasets' parameters. Each y-axis represents the relevant value range of one parameter. Each line corresponds to one patient dataset. Time-resolved parameters are represented by the aggregated minimum, maximum, or mean values.Further chart types support an advanced exploration of relationships between different parameters. Scatter plots with regression lines visualize linear relationships between parameters. Histograms show the distribution of different parameters. Box plots give a standardized overview of the data set. Pie charts visualize how frequently individual values or cases of the disease class occur in the study. This can also be used to visually identify unbalanced data sets, for which appropriate measures can be taken in the case of subsequent training.The exploration tool is designed as a hierarchical tool with different interlinked views. The linking of the data is based on the data model described in Section Data Model. Cases can be selected interactively in the PCP by a technique called brushing . The selA curve diagram enables the analysis of temporal dynamics of individual parameters. As shown in via the aggregation of time-resolved data in the PCP the user can backtrack information from a specific feature to the relevant regions within the underlying image. This can further be used to identify any data integrity issues.The image viewer shows the image data of a selected case with segmentation contours and an overlay of the segmentation uncertainty. Schematic visualization of the heart shows the position of the displayed image slice with regard to the cardiac anatomy. This approach supports the identification and exploration of outliers as shown in timepoint widget users can specify which timepoints to consider . Furthermore, Figure 10E exemplifies the feature importance of the random forest classifier. We observe the patients' left myocardium sphericity to be the most crucial variable in detecting pathological cases, closely followed by the left blood pool volume and the interventricular septum thickness parameter.Cardiac cine MRI provides information on the anatomy and the function of the heart and can help to differentiate between cardiovascular diseases. In this study, we use the freely available dataset from the ACDC challenge  to demonmyocardium intensity cluster tendency, average septum thickness, and average left ventricle tortuosity. After removing these outliers, one can differentiate between HCM and DCM patients based on the left blood pool coarseness and relative septum thickness. This can also be seen in the corresponding box plots in the second row. When only selecting patients with HCM and DCM, this becomes even more prominent as shown in While outliers were excluded in this first analysis of the study cohort, it is also possible to select and analyze these individual outlier cases. In outlier cases where the segmentation had failed, it was manually corrected using our labeling interface. After correcting the outlier cases, parameters were extracted again, and the cases were classified again. mm2, the slice thickness was 8mm and the distance between slices 8–13mm. In a post-processing step, the image slices were realigned to prevent any effects of breathing motions.The EMIDEC challenge  providedAnalogously to the previous case study, the dataset from the EMIDEC challenge was integrated into our classification and exploration environment. A 2D U-Net was used to generate segmentations of the LV endocardial and epicardial border trained on data from 100 patients with myocarditis and cardiomyopathy to segment the myocardium in the LGE MRI data. As no cases with no-reflow areas were included in the patient data, we also included the 50 unlabeled cases from the challenge's test set. Initial segmentations for these cases were generated by a pre-trained model. An expert used the tools described in Section Data Correction, Data Integrity, and Dynamic Updates of Machine Learning Models to correct these segmentations. Next, the segmentation results were added to the internal cases with expert segmentation, and the final segmentation model was trained. This final model segmented the LV endocardial and epicardial contours and extracted the radiomics parameters on the 80 cases from the challenge's training set. A classifier was trained to differentiate between normal and pathological cases with 25 shape- and texture-based features. We applied a similar strategy as in the ACDC case study. An eight-fold nested CV was used for model and hyperparameters selection. Essential image-based features for the classifiers are shown in the PCP.left blood pool surface area to be the most crucial variable in detecting pathological cases, closely followed by left myocardium difference entropy and left myocardium contrast parameter.The optimal classifier identified by our grid-search on the EMIDEC challenge dataset turned out to be an extra tree classifier with 190 estimators, the Gini criterion, a maximum depth of six, a minimum of samples per leaf of six, and a minimum sample split of nine. via brushing on myocardial contrast, myocardial cluster tendency, myocardial complexity, left blood pool coarseness in the PCP analogous to the analysis of the ACDC dataset in via the PCP, the accuracy of the classifier improved from 0.75 to 0.8  and by the Deutsche Forschungsgemeinschaft  – SFB-1470 – B06.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Photon counting lidar for long-range detection faces the problem of declining ranging performance caused by background noise. Current anti-noise methods are not robust enough in the case of weak signal and strong background noise, resulting in poor ranging error. In this work, based on the characteristics of the uncertainty of echo signal and noise in photon counting lidar, an entropy-based anti-noise method is proposed to reduce the ranging error under high background noise. Firstly, the photon counting entropy, which is considered as the feature to distinguish signal from noise, is defined to quantify the uncertainty of fluctuation among photon events responding to the Geiger mode avalanche photodiode. Then, the photon counting entropy is combined with a windowing operation to enhance the difference between signal and noise, so as to mitigate the effect of background noise and estimate the time of flight of the laser pulses. Simulation and experimental analysis show that the proposed method improves the anti-noise performance well, and experimental results demonstrate that the proposed method effectively mitigates the effect of background noise to reduce ranging error despite high background noise. Benefiting from the excellent photon-efficiency sensitivity and picosecond-level temporal resolution, photon counting lidar equipped with Geiger mode avalanche photodiode (Gm-APD) has been widely used in long-range high-resolution ranging and 3D imaging ,5,6,7. RSeveral anti-noise methods have been proposed for photon counting lidar. In ranging applications, a routinely used method is the iterative sigma multiplier filter. It cuts off the photon counting histogram within a region near the signal peak with the width of a multiplier of the emitted pulse’s standard deviation ,10. Its The more effective method to deal with noise is a correlation-based method that is called matched filter, which performs a cross-correlation between the system response with the photon counting histogram to extract the signal. Aurora et al. used a matched filtering method to suppress the background noise in a highly scattering environment, and estimate the depth image of the underwater target . Markus In summary, both the histogram-based method and correlation-based method lack enough robustness under a weak signal but strong background noise cases, such as the scenes of long-range detection in daytime, resulting in poor ranging error. Therefore, a more robust anti-noise technology needs to be established. Entropy is an important tool to quantitatively evaluate the uncertainty of random events, and recently many technologies using entropy to improve detection performance have been developed in radar, ultra-wide bandwidth ranging and location, fuzzy clustering, and cypher system ,19,20,21In this paper, based on the mentioned characteristic, an entropy-based method to mitigate the effect of background noise is proposed to reduce the ranging error in photon counting lidar. Firstly, the photon counting entropy is defined as a statistical test to quantify the uncertainty of fluctuation among the PEs responded by Gm-APD. Then, a window is used to divide the sampling period into multiple subintervals, and the photon counting entropy is calculated in each subinterval as the feature that differs signal and noise. Thirdly the subinterval with the minimum photon counting entropy is searched, whose time delay is the estimation of the time of flight. The simulation and experiment show that the proposed method effectively deals with noise to reduce ranging error despite high background noise. Under a background noise rate of 9 MHz, for example, range accuracy and range precision of the proposed method are both about five times better than those of the matched filter method.The characteristic of echo signal in photon counting lidar is firstly analyzed. As shown in  daytime . The sysi-th time bin is derived after K pulses accumulation ast, in which N is the number of time bins. The detected PEs’ counts i-th bin fluctuates around Equation Y^k=∑Y^k=∑(3)Ymulation . Here a quation λ^ntλ^ntx timM is the window width. The window width is an important parameter that affects anti-noise performance. If M is too small, the stationarity of the noise is not satisfied as it must be observed over a relatively long time. If M is too large, the nonstationarity of the signal may not be distinguishable to two adjacent ranges. After a lot of trial and error, the proper width is set as The sampling period is divided into multiple subintervals by a window function. Since photon counting entropy is defined in spectral domain, the window has an effect on the spectrum of the preprocessed data. In order to avoid this, the hamming window that has the lowest sidelobe level is chosen to divide the sampling period. The hamming window refers to the window that satisfies the following functionq-th subinterval is expressed ass denotes the shifting step between two adjacent window operations, and Q is the number of truncated subintervals and q-th subinterval is obtained. With the window function shifting, the sequence of photon counting entropy isThe photon counts of the windowed essed asy^qm=wmy^Equation  and 4),,q-th subse ratio  SNR=KNs/c is the speed of light.The time of flight is estimated as the time delay of the subinterval with minimum photon counting entropy. It satisfies the condition ofl-th measurement, and L is the number of measurements. Range precision represents the random errors of system [Range accuracy and range precision are two indicators often used to quantitatively evaluate ranging error. Range accuracy represents the systematic errors . It is df system . It is dFinally, considering that incorrect range estimations may exist because of high background noise, another quantitative indicator named the correct ranging rate is defined to evaluate the robustness of range estimation. Those range estimations distributed in the region In this section, combined with Equations  and 10)10)–15),,15), thelgorithm . For theTo further reduce the ranging error under a high noise level, an effective method is to increase the number of accumulated laser pluses. The photon counting ranging system shown in To verify the anti-noise performance of the proposed method, the ranging experiments under four different background noise levels are undertaken. The target range is calculated by Equations –13). In. In13). The experimental results of the proposed method are shown in The quantitative performance of the experimental results in Then, the influence of the number of accumulated pulses on the ranging error is discussed. In conclusion, an entropy-based anti-noise method is proposed to robustly mitigate the effect of background noise in photon counting lidar, so as to reduce the ranging error. The photon counting entropy has been defined firstly to evaluate the uncertainty of the fluctuation among PEs responded by Gm-APD and used as the feature to distinguish signal from noise. Then combined with a window operation, noise was dealt with by replacing the raw photon counting data with the sequence of the photon counting entropy. Finally the time of flight is estimated by searching for the subinterval with minimum photon counting entropy. Simulation has analyzed that the proposed method has better anti-noise performance under various noise levels than the matched filter method. Experimental results have demonstrated that the proposed method effectively mitigates noise to reduce the ranging error despite a weak signal and high background noise. For example, under a background noise rate of 9 MHz, range accuracy and range precision of the proposed method are about five times better than the matched filter method. Besides, the proposed method requires less detected photons and lower SNR, i.e., the proposed method reduces the number of accumulated pulses by 0.5 times and the working SNR by 2 compared with matched filter method, at the same level of ranging performance. This method does not increase the system complexity, and may be applied to long-range and full-time detection of small targets.


Performance evaluation of source camera attribution methods typically stop at the level of analysis of hard to interpret similarity scores. Standard analytic tools include Detection Error Trade-off or Receiver Operating Characteristic curves, or other scalar performance metrics, such as Equal Error Rate or error rates at a specific decision threshold. However, the main drawback of similarity scores is their lack of probabilistic interpretation and thereby their lack of usability in forensic investigation, when assisting the trier of fact to make more sound and more informed decisions. The main objective of this work is to demonstrate a transition from the similarity scores to likelihood ratios in the scope of digital evidence evaluation, which not only have probabilistic meaning, but can be immediately incorporated into the forensic casework and combined with the rest of the case-related forensic. Likelihood ratios are calculated from the Photo Response Non-Uniformity source attribution similarity scores. The experiments conducted aim to compare different strategies applied to both digital images and videos, by considering their respective peculiarities. The results are presented in a format compatible with the guideline for validation of forensic likelihood ratio methods. Evaluation of forensic evidence relies on the concept of likelihood ratios (LRs), derived from the Bayes theorem. In fact, reporting LRs is the preferred way of presenting findings from criminal investigations across the spectrum of forensic disciplines . This isIn the vast majority of cases, the result of a comparison between a questioned sample and the reference database leads to a similarity score, which is often dimensionless, lacking any kind of probabilistic interpretation and is therefore very difficult to incorporate into the forensic work-flow, unlike the LRs. It is the case of source camera attribution based on the Sensor Pattern Noise (SPN) or Photo Response Non-Uniformity (PRNU) ,5, whereCalculation of LRs from similarity scores is described in the literature ,13,14,15The main contribution of this article is the assignment of probabilistic interpretation to the set of similarity scores obtained from PRNU comparisons in the context of source camera attribution. This aim is reached by converting similarity scores into LRs within a Bayesian interpretation framework . The perThe article is structured in the following way: in the next section we introduce the fundamentals of PRNU analysis. PRNU is a unique noise pattern that every camera sensor implants like a passive watermark into every digital image ,5 and viSeveral techniques were proposed to extract PRNU from an image but in this paper we refer to the one described in . At imagAs best practice, the PRNU associated to a given sensor is estimated by replicating the previous processing for a large enough set of flat-field images, in order to reduce the impact of the images content. The PRNU is then estimated according to the Maximum Likelihood criterion  as:(2)K^A similarity measure is needed in order to compare two PRNUs and classify whether they come from the same camera or not. Goljan et al.  proposedA straightforward solution for extracting a unique PRNU from a video is to consider video frames as images, and then to apply . This apIn order to address DMS challenge, several matching strategies have been proposed in the literature ,31,32,33Using flat-field video recording to extract key-frame sensor noise and compute PRNU camera digital fingerprint according to . Still vEmploying both flat-field images and flat-field videos  in orderThe objective of the analysis is to attribute or dissociate a questioned image or video to a specific device. As a first step, the reference PRNU needs to be extracted for the camera. In the case of the images, the process is quite straightforward: a set of flat-field images is acquired, from which the PRNU is extracted according to . In the In order to use both, video recordings and images, we briefly recall how a camera generates a video frame. The process involves three steps: acquisition of a full-frame image, cropping of an internal region with a different aspect ratio  and scaling to the final resolution. By assuming that the crop is symmetric with respect to the optical centre and posing the reference system at the centre of the frame, the relation between image and video PRNUs, namely stimated ,31,35, tU is an arbitrary neighbourhood in which the correlation peak is searched. The operation of maximization clearly impacts on the distribution of the similarity scores. Similar considerations can be made in the case of the video recordings, for those approaches that try to minimize the impact of DMS by adopting  =(a)Baseline: PRNU is obtained by cumulating the noise patterns extracted frame-by-frame according to , and the(b)Highest Frame Score (HFS): PRNU is extracted and compared frame-by-frame against the reference PRNU, and the maximum PCE is taken .(c)Cumulated Sorted Frames Score (CSFS): PRNUs, extracted from each frame and compared with the reference signal, are first sorted in a descending order according to their individual PCE values; then, they are progressively cumulated, according to ; finallyIn order to simplify our analysis, we assume that no operation aiming to maliciously modify the PRNU is applied to the data. In this setting, the similarity scores for images are computed according to . In the All the above-mentioned methods compute the PCE as described in .Finally, it is worth noting that, according to the Equation , PCE canCouple of key components are necessary in order to compute LRs from the similarity scores: the ground truth regarding the source of origin of the image/video (same source or different source), a set of forensic propositions (hypotheses set for the defence and for the prosecution), and similarity scores, which are produced by different methods described in the previous section. Unlike the traditional performance assessment, which is usually limited to the analysis of the Detection Error Trade-off (DET) and Receiver Operating Characteristic (ROC) curves, we add the probabilistic meaning and interpretation to the similarity scores by transforming them into LRs. In order to do this, we set the hypotheses at the source level:It should be noted here that it is possible, and encouraged, to set the propositions at other than the source level . Once thLR we have the probability of observing LR we have the probability of observing the same evidence S represents the left-out observation.Different ways have been described in the forensic literature to calculate the LRs from continuous similarity scores ,22. Oncestrategy , in whicPerformance assessment of the LR values under either of the propositions follows the methodology proposed in ,21,22. IGiven the limited amount of data we focus on evaluation of performance using the primary characteristics and leave the concept of validation of the LRs for forensic casework for future research. Although the full scope of the proposed “validation” framework cannot be applied, the basic concepts presented are valid and provide supplementary information, complementing the typically reported ROC/DET representations and accuracy measures at a fixed operating point.accuracy, as sum of discriminating power and calibration, represented by the Empirical Cross Entropy (ECE) plot and measured by the log LR cost (CLLR) [t (CLLR) ;discriminating power represented by the DET and  CLLRmin ;calibration represented by the Tippet and the ECE plots and measured by  CLLRcal .The following performance characteristics and corresponding graphical representations are presented in the results section:In this section, we first describe the data set we used in the experiments. Afterwards, the experimental protocol follows a logical separation, based on the type of data, namely images and video recordings. For videos, we separate the analysis in function of the type of PRNU reference and the presence or the absence of DMS, in order to perform the four basic experiments mentioned in A set of 30 randomly selected flat-field images, from which we extracted the image PRNU A set of flat-field static (labelled as still) and moving (labeled as panrot and move) videos. These videos are used to create reference PRNU A set of images with natural content that we used as query data. The set is composed of at least 200 pictures per devices.A set of non-flat query videos including still, pan-rotating and moving videos.We used the Vision Dataset   present slightly different challenges. For example, let us consider the scores distributions obtained from images analysis and shown in In some cases, for example non-stabilized video recordings against RT1 result in “perfect separation” of the mated and non-mated score distributions see a. While For some cases however, such as PRNUs obtained from the images, a significant proportion of the mated scores attains the magnitudes of the non-mated scores, thus contributing heavily to the False Rejection error rates. Again, PRNU obtained from these devices compromises the overall performance of the methods under evaluation.Our analysis into the distribution of similarity scores produced by the image test and reference samples showed that the mated samples , we proceed iteratively through the set of similarity scores, exclude one similarity score (mated or non-mated) to “play” the role of observed evidence. We use remaining similarity scores to model score distributions under either of the propositions.The Gaussian calibration with optimal risk smoothing is used for the non-stabilized videos as both, the mated . Due to the near-perfect separation of the mated and non-mated scores, the baseline method and the CSFS method are not visible in the DET plot as their EERs are close to zero.Among the methods compared by means of ECE plots , the basreliable  and the By looking at the Tippett plots , the lowLRs produced in the course of non-stabilized videos show “perfect” accuracy and calibration when compared in the scope of RT1 experiments for proposed and baseline methods. Given the relatively small dataset, these results should be further analysed and followed up by a series of experiments to show the robustness of methods to the previously unseen data and potential overfitting. Slightly better accuracy and calibration was observed for the baseline method when comparing RT2 video recordings however, lower rates of misleading evidence were observed for the proposed method. In general, the performance of baseline and proposed methods can be considered equivalent. Decisions based on the LR values observed for prior log10odds greater than 1.0 for the questioned videos in the scope of RT2 experiments should not be trusted due to the fact that the ECE curve crosses the reference line and these decisions are effectively worse than decisions based on a coin toss.Before discussing the results, we provide an analysis of the resulting LR values for the stabilized videos by means of normalized-count histograms, which perfectly suit the purpose. As shown in thod see . The CSFHigh rates of misleading evidence of the LR’s supporting the bove see .The results for the stabilized video recordings compared against reference RT1 and RT2 are summarized in In this article we addressed to our best knowledge for the first time the challenge of source camera attribution for video recordings from a perspective of a forensic evidence evaluation using likelihood ratios, and complemented previous research  on sourcReflecting on the analysis of the results of different methods and settings, particularly ECE plots prove useful as they point out regions where produced LRs provide unreliable support to forensic evidence for both still images as well as video recordings. Considering the fact that there is a lot more information present in the video recordings (sequence of images) than in a single still image, it is not surprising that the best performance in terms of accuracy, calibration and discriminating capabilities was observed for the non-stabilized video recordings. However, performance dramatically drops if digital motion stabilization is adopted. A particular attention should be paid to the analysis of images, for which apart from the image resolution the device model should be considered as a deciding parameter. The latter might affect in a positive or negative manner the overall performance of the system.Additional validation experiments accompanied by further analysis of the similarity scores will be performed in the near future. Particular attention will be given to the “perfectly separated” similarity scores and regions of high correlation, with the aim to demonstrate robustness to the lack of data, generalization and coherence —which pr


Food quality and safety issues occurred frequently in recent years, which have attracted more and more attention of social and international organizations. Considering the increased quality risk in the food supply chain, many researchers have applied various information technologies to develop real-time risk identification and traceability systems (RITSs) for preferable food safety guarantee. This paper presents an innovative approach by utilizing the deep-stacking network method for hazardous risk identification, which relies on massive multisource data monitored by the Internet of Things timely in the whole food supply chain. The aim of the proposed method is to help managers and operators in food enterprises to find accurate risk levels of food security in advance and to provide regulatory authorities and consumers with potential rules for better decision-making, thereby maintaining the safety and sustainability of food product supply. The verification experiments show that the proposed method has the best performance in terms of prediction accuracy up to 97.62%, meanwhile achieves the appropriate model parameters only up to 211.26 megabytes. Moreover, the case analysis is implemented to illustrate the outperforming performance of the proposed method in risk level identification. It can effectively enhance the RITS ability for assuring food supply chain security and attaining multiple cooperation between regulators, enterprises, and consumers. With the global intention of the food security concept shifted from fighting hunger and malnutrition to healthy living and food, the food supply chain is not only an important issue of public health and trade but also one of the most challenging sustainability issues, which significantly affects the stable development of national sovereignty, economy, and society .Nowadays, the modern food supply chains usually contain complicated transport links including planting, harvesting, handling, storage, transport, loading, packaging, and marketing, over an extended distance and different partners from producer to consumer as a result of global food trade and economy. Ensuring the quality and security of food products in the supply chain systems is the most likely and effective approach to solve many problems such as covering climate change, biodiversity loss, land degradation, and pollution . For exaNevertheless, after a series of foodborne disease outbreaks and food safety incidents occurring in the past few decades, the food supply chain has faced increased quality and safety risks. With extended haul distance and long-term storage time from the planter to the consumer across different stages and industries, various chemical and microbial contaminations, as well as foodborne viruses, have easily invaded each supply stage of the food supply chain, which have severely threatened the food safety and disrupted the stable sustainability of food supply . In partIn such a complicated and invisible supply chain, many potential factors including temperature, humidity, pasteurization, or heat treatments have increased the unpredictable risks in the food supply chain since not all participants can strictly ensure their process operations and management conform to all the standards. As a result, the unsafe vulnerabilities of food supply chains have been ulteriorly aggravated due to the inherent multitiered cascading effect . TherefoWith the development of information and intelligence technologies such as the Internet of Things (IoT), automated robots and equipment, cloud computing, and smart factory management systems, the modern food supply chains are struggling to make the most of innovative technologies to increase the interconnected efficiency of organizations and support sustainable practices in the form of products and services through various food supply linkages . As one At the same time, the system is also a valuable tool in supporting sustainable development in the food supply chain from producers to consumers, which can efficiently reduce resource waste, replace extensive operations with carbon concentration, and eventually avoid food safety incidents to mitigate climate change and other negative environmental and social impacts of the whole process . The aboTherefore, by taking advantage of deep learning techniques, the present study proposed the novel deep-stacking network (DSN) to establish an improved identification and traceability system, which aims to promote reliable food safety and sustainable assurance of the whole supply chain. This method could not only automatically process a large amount of monitoring biosignals collected by the multisource IoT sensors and instruments to evaluate food quality but also mine the interaction of multiple hazards and related risk factors to derive preferable risk grades of food supply chains.This paper is organized as follows: in Nowadays, there are several food safety traceability frameworks and construction standards, e.g., the Rapid Alert System for Food and Feed (RASFF) system approved by the European Union, Food Safety System Certification (FSSC) framework, and the GEMS/Food Contamination Monitoring and Assessment Programme (GEMS/Food) established by the World Health Organization (WHO), which aim to effectively manage accident safety risks, promote quality improvement, maximize consumer trust, support market efficiency, and achieve precision agriculture within the supply chain . SimilarAlthough lots of shared information have strengthened the communication and cooperation between each node of the food supply chain, these systems are not enough to handle the increasing challenges of current food supply chains, such as flexibility, compatibility, credibility, and comprehensive decision-making . On the Making full use of these data will support members' operating abilities in hazard analysis, enhance enterprise competitiveness in risk prewarning, and promote the effectiveness of food safety supervision and management . TherefoOn the other hand, most researchers have performed risk identification and prediction of the food supply chain from a local perspective, which means each supply chain partner in the food system can successfully find the quality and safety problems in their own production process . HoweverMany foodborne mycotoxins contaminated in those unsafe food products may spread in a fast-growing number under certain circumstances since the food supply chain naturally had the migration structure and dynamic cascading function. A classical static-method-based RITS failed to anticipate a wide variety of risk factors influencing food safety and quality throughout the whole supply chain operations and members . Thus, tk-nearest neighbour (KNN), and artificial neural network (ANN) [Listeria monocytogenes in six retail delicatessens [In order to strengthening the safety of the extended food supply chain, a variety of new approaches have been introduced to meet these challenges. Machine learning models including the random forest (RF), support vector machine (SVM), decision tree (DT), rk (ANN) –19 are aatessens .Similarly, SVM-based methods have been explored to access risk levels for making sure the supply safety of dairy production in northeastern Brazil . FurtherAlthough classical machine learning methods would offer an effective approach for food safety supervision to deal with perishable products and unpredictable supply variations to some extent, the intelligence degree of existing food systems is still too far away from meeting stringent food safety requirements, as well as assisting managers in making correct decisions dynamically for identification and mitigation of global risks . EspeciaIn recent years, as RITS management has been rapidly developed toward Industry 4.0 on the basis of artificial intelligence and IoT technologies, deep learning methods including AlexNet , GoogLeNMeanwhile, some researchers have pointed out that deep learning methods are very preferable to make full use of these valuable data to enhance enterprise competitiveness, as well as gain the trust of consumers in highly competitive markets , 35. HowTherefore, by taking advantage of deep learning techniques, the present study proposed the novel deep-stacking network (DSN) to establish an improved identification and traceability system, which aims to promote reliable food safety and sustainable assurance of the whole supply chain. This method could not only automatically process a large amount of monitoring biosignals collected by the multisource IoT sensors and instruments to evaluate food quality but also mine the interaction of multiple hazards and related risk factors to derive preferable risk grades of food supply chains. With the comprehensive evaluation of food quality, the food system could manage supply chain processes and participants effectively to avoid possible quality accidents and reduce regulatory costs. Meanwhile, the customers also obtain a rare glimpse from the inside of a hierarchical supply chain as well as the basis of a sharable and scientific risk analysis, which improve their confidence in making intelligent purchasing decisions and buying high-quality food products to meet a healthy daily diet.K-fold cross-validation method is applied to expand the utilization of limited training data in the second stage. Finally, the deep-stacking network is consolidated to extract a list of food safety prewarning features, which speed up the mining process and improve identification performance in the present study.Food safety inspection is a huge and complex Internet of Things system with various sensors. In this paper, all kinds of sensor data are processed scientifically. On this basis, we propose the deep-stacking network to forecast and estimate the food safety risks. This paper achieves the proposed method with three stages. Firstly, a multigranularity padded-scanning method is adopted to generate input vectors with adjustable dimensions in the first stage, and the As the response speed of the food system is very important, we proposed an improved risk identification and traceability based on IoT and nephanalysis technologies, which can take advantage of information sharing to coordinate the operation among nodes of the supply chain, to ensure the food safety prewarning system operates efficiently and safely. This system collected a large amount of monitoring data from various links through various equipment, including temperature and humidity sensors, oxygen concentration sensors, and carbon dioxide concentration sensors, as well as different pollutant detection equipment , ochratoxin A (OTA), zearalenone (ZON), deoxynivalenol (DON), T2 toxin, fumonisin, etc., in rice, wheat, corn, and other grains with high accuracy and sensitivity. The pesticide residue fast tester detects various organophosphorus and hydrogen-formatted pesticides in grain sales and consumption sites such as farmers' markets, supermarkets, planting bases, restaurants, and laboratories.Salmonella, Listeria, methicillin-resistant Staphylococcus aureus (MRSA), and other bacteria in grain products. All data are transmitted to the information cloud analysis and storage platform through TCP/IP and wireless approaches, which is the center for data collection, information storage, and exchanging from all relevant steps in the food supply chain. The grain product types included rice, wheat flour, corn, roughage, and other grain-processed products, which cover the most important grain structure of Chinese consumers.Additionally, the microbial and pathogen detecting instruments use immunoconcentration technology to carry out antibody capture, concentrated release, purification, separation, and automatic detection of pathogenic bacteria including coliforms (COLI), With the RITS operated in the period from March 2016 to August 2019, there are a lot of grain data covering 26 primary grain-producing areas recorded in the food system. Those provinces are the highest consumption areas because they are also the most densely populated. Due to the redundant and abnormal information recorded and collected through the IoT platform, we uniformly integrate different data from various devices and extract some important attributes to construct the retrospective database with regular formats and value ranges. The final database and detail attributes in uniformed styles are shown in Then, in order to get a reasonable hazard level, this paper consults the relevant literature, studies the combination of international standards and domestic standards, consults relevant experts in the field of food safety, and crawls public data on the internet. In order to obtain a more comprehensive, scientific, and credible risk assessment tool so proper conclusions can be made, We hope to enrich the real-time IoT datasets with information derived from hazardous, social, economic, and regulatory food safety aspects so that a retrospective database can cover more dimensional analyses and multiple participant perspectives.Therefore, these qualitative indicators such as social attention, harm degree, and accessibility of supervision are covered. And quantitative indicators are also considered, for example, the province's total annual output, food production, and consumer prices. The expanded attributes of the final database are shown in There are a lot of unstructured data recorded in the retrospective database, which were collected from different sources and stored in various formats. Moreover, with common operating errors caused by incorrect human interference, the database is filled with unauthentic and abnormal data unreflecting the real situations of grain quality security in the supply chain, which has just reduced the prewarning and traceable performance of the food system. It is necessary to add some data analyses and preprocessing to extract encoding rules and uniform multisource data format before proceeding with the sequential risk evaluation.Abnormal and unstructured data cannot be directly input to the computer for calculation, so it is necessary to digitize such data through coding. In order to encode the abnormal and unstructured data, the dictionary vectorizer (DictV) and one-hot encoder (OneH) are combined to complete the encoding process so that we could input the correct data to the grain RITS as cases which are employed to mining risk association rules and prewarning information directly. The dictionary vectorizer is an important method for extracting features and vectoring data in a dictionary. When a discrete variable is represented in an unordered string value, it needs to be encoded by this method into numeric variables. Although the dictionary vectorizer can simply transfer the original data in string type or like fashion as a numeric dataset, it might be out of standard and far from practical applications since there will be some special data, in which the category is larger than other categories, and the categories in the original features have no size relationship. It can lead to a negative effect on model building and risk evaluation.N-bit register to encode N states of the feature vector, and each state has its register bit. Once the Euclidean distance between two states is the same, the corresponding register bit is valid at any time as 1, and others are 0. In particular, if there are original data in m × l dimensionality, which content n-type features in the l column, then the one-hot encoder uses the 0/1 binary coding to represent the feature vector with different types and changes the data dimensionality into the new format m × n. This encoding process does not alter any meaning of all fundamental features and ensures the Euclidean distances among different features with types are the same, so we could analyze the risk level based on some accepted unstructured data that occurred at a supply chain node.Considering the potential problem of encoding rules for food safety, one-hot encoder is introduced to continue completing the entire encoding. When a discrete variable is processed after the dictionary vectorizer into a feature vector without numerical order, the one-hot encoder can make further improvements on the encoding quality by distinguishing different categories of feature vectors. The one-hot encoder uses an x is calculated by removing the mean and scaling to unit variance as follows:μj is the mean of the training samples and sj is the standard deviation. Relevant statistics for the samples in the training set are calculated, which can be centered and scaled independently of each feature. The mean and standard deviation are then stored to handle with subsequent data. The normalized variable values fluctuate around 0, with greater than 0 indicating above average; otherwise, it means below average. In a word, the correct and key data would be stored in the food system for the mining risk level and prewarning information after analyzing and processing unstructured abnormal data from the retrospective database.For the same feature, the values of different samples may be far apart. Some abnormally small or big data may mislead the correct training of the model. Additionally, the data with very scattered distribution will also affect the training results since the classification results will be biased toward larger values. Value standardization of the database is a common requirement of most machine learning algorithms: if individual features do not look like standard normal distribution data , they may behave badly. At this point, we can normalize the values in the features, that is, convert to a normal distribution with a mean of 0 and a variance of 1. Thus, it is important to explore the data distribution of the features and consider whether it is necessary to standardize the data before training the model. In this paper, we complete the value standardization operation in the Python programming language by calculating the mean and standard deviation of the training set as well as the test dataset in the same transform. The standard score of sample K-fold cross-validation method is applied to expand the utilization of limited training data in the second stage. Finally, the deep-stacking network is consolidated to extract a list of food safety prewarning features, which speed up the mining process and improve identification performance in the present study.Based on the preliminary analysis of IoT data, this study intends to find out inherent relationships among items in a huge database, which consists of key features with temporal and spatial distributions matching different hazard categories in grain products. However, it is still difficult to comprehensively analyze the risk grade of hazardous materials in the whole supply chain since the previous methods are not applicable to dispose a mass of heterogeneous data offered by multisource IoT sensors and instruments. Therefore, a novel deep-stacking network is proposed to accurately identify the risk level of main hazards and timely carry out risky prewarning of food products in the grain supply process. In order to improve the identification accuracy, this paper achieves the proposed method with three stages. Firstly, a multigranularity padded-scanning method is adopted to generate input vectors with adjustable dimensions in the first stage, and the M-dimensional sample, and then perform sliding sampling through a filtering window of length k and step size 1 to obtain the subsample vector of M′-dimensional feature:Because the raw database is mostly unbalanced, multigranularity scanning is to utilize many sliding windows with different sizes and functions to filter the input dataset for forming new input vectors, which obtain the enhanced representation ability of high-dimensional input data. The basic scanning process is as follows: first, input a complete P is obtained in each forest. In this way, each forest will generate a characterization vector of length M′∗P. The results of different forest layers will be spliced together to obtain the output of this layer. The underlying multigranular scan will oversample all features except the first and last dimensional features, while the first and last dimensional features will be sampled less once. However, the first dimension and the last dimension feature may be very important in practical applications; that is, reducing the two-dimensional sampling relative to other dimensions may affect the final result of the classifier.Each subsample is then used for training in partial or complete random forests, and a probability vector of length P-dimensional sample. To this end, add an extra feature with value 0 and length k − 1 to both ends of the input M-dimensional sample before the original multigranularity scanning process, as shown in k and the step size s is subjected to sliding sampling so that the k-dimensional feature of the subsample vector is obtained as M′. The formula is determined by the following equation:So, considering the limitation, this paper proposes a padding optimization to perform a zero-compensation operation on the edge of the According to the above multigranularity padded-scanning method, we could get the more effective input vectors with adjustable dimensions by scanning the feature with padding operation since most of the key information contained in the unbalanced samples is searched without losing too much raw data. Moreover, the subsequent methods can accurately determine whether raw data or feature vector should be focused on when an abnormality is detected, thereby availably avoiding the model overfitting in the risk analysis and evaluation process.d samples are randomly selected from all training data D as a training set, and the rest is used as a test set. However, in this case, the data were only used once and were not fully utilized. In order to make full use of the limited training database, we can use the K-fold cross-validation method to divide the training/test dataset into a larger number of subsets. After this operation, the model overfitting is further processed due to dataset bias and improper dataset partitioning. The training data are divided into k parts; one part is selected as the test part and the rest as the training part in each of k iterations. Then, the evaluation metric integrating different outputs of subsets is obtained as follows:k is the subset of the original sample D of equal size, i=1,2,3,…, k is the number of the ith subset, Ei is the evaluation results on each subset, and E is the overall evaluation result.Using the results of the padded scanning as inputs, a simple hold-out validation is used; that is, Pi denotes the probability score for the ith component classifier. θi= denotes the estimation parameters, which are composed of the mean vector μi and the covariance matrix σi2, respectively. And P of the risk level, which fuses the probability scores of different subclassifiers. The log-likelihood estimation of P is performed as the following expression:n is the number of subclassifiers, m indicates the number of input data, and γji denotes the hidden variable representing the mixture weight of the ith component classifier on the jth label vector yj∈γji adds up to 1, the total probability distribution of the final result is normalized by maximizing the expected value of the log-likelihood function given in multiple iterations until the parameters reach convergence.In order to promote the risk identifying capacity of the grain RITS, this paper introduces the deep-stacking network method inspired by the classic stacked learning strategy to mine the relationship among risk factors and hazards, meanwhile to assign the corresponding weights according to their different risk levels. The deep-stacking network algorithm first uses a large amount of data to train the primary learner and obtains the output of the first-layer learner through the primary learner. Next, a layer in the classic stacking algorithm is added, the original input data and the results obtained by the first layer are concatenated, and they are input into the second layer. Then, the second layer of the primary learner is trained to obtain the output of the second layer; finally, the results of the second layer of learners are concatenated, and the final result is obtained by the secondary learner. The structure diagram is shown in K-fold cross-validation is used to divide the raw database into k equal-sized subsets to train multiple different learners of the first layer; next, the output result of the layer is concatenated with the original input as an input of the next layer, and a plurality of different learners of the second layer are trained; finally, in the case of multiple trainings, the accuracy is no longer improved, the training results of the last layer of learners are merged to obtain the final results, and the model is subsequently applied for testing new data. The overall pseudo-code of the deep-stacking network framework for the decision-making process is shown in The overall algorithm steps are as follows: firstly, the training data are coded by the dictionary vectorizer and one-hot encoder, and the value is standardized according to different hazard categories in the grain; secondly, the multigranularity scanning method is improved, and all features are extracted equally by adding padding operation; meanwhile, the In order to illustrate the applicability and effectiveness of the improved RITS with the support of the deep-stacking network method, a case study has been conducted in the grain industrial chains. In order to prove the scientific and the effectiveness of this research, the grain datasets describing 34 and 170 instances were selected from the retrospective database stored in the cloud platform of the RITS to train and test the risk evaluation model. The original sample datasets contain 33 attributive indicators and the given risk level, of which 14 features are coefficient terms recorded as string-type data, and the others are recorded as numerical features. The category distribution of all instances according to eight risk levels is as follows: level I (15566), level II (3752), level III (1288), level IV (2117), level V (2575), level VI (1726), level VII (1386), and level VIII (5760).K-fold cross-validation with k = 5. Stratified random partitioning was performed to ensure the even distribution of the classes within each level subset, which affects the feasibility of training and testing the model. The 90% random split constitutes the training subset, while the remaining 10% were used as the validation subset to monitor the training process and minimize overfitting. The random splits for each fold were controlled by a random seed such that the individual split could be reproduced as required.The nature of this dataset allows for multiple risk levels to be present in each instance. Therefore, all raw data were partitioned into 90% and 10% splits of training and testing subsets for F1-score (F1) were used to compare the prediction results of each model. Among them, the ACC is the ratio of the number of correctly marked items to the total number of observations. And the PRE is the number of true positives divided by the total number of items marked as belonging to that category.When the best choice of the training process and parameters is achieved, the trained model was used to predict the unknown risk level of the 10% testing datasets. The prediction result of remaining data was then compared with the given risk level. To objectively evaluate the results of different algorithms, four performance indicators including accuracy (ACC), precision (PRE), recall (REC), and F1-score, which is the harmonic mean of the ACC and REC with better comprehensive evaluation for multicategory identification problems. The detailed definitions of indicators are as follows:Meanwhile, the REC is defined as the number of true positives divided by the total number of items belonging to that category, also known as sensitivity. Through the above three indicators, the number of results in the confusion matrix can be converted into a ratio between 0 and 1, which can be counted on the basis of true positive (TP), true negative (TN), false positive (FP), and false negative (FN). Moreover, this paper employs another high-level indicator K-nearest neighbour (KNN), support vector machine (SVM), extra trees (ET), gradient boosting (GB), random forest (RF), and decision tree (DT) and five deep learning models including AlexNet, VGG, GoogLeNet (GLNet), ResNet, and deep forests (DeepF) were selected in this section to illustrate the identification challenge in the grain supply chain, which has achieved remarkable success in other fields. All models are trained and tested on a cloud server platform with Ubuntu 16.04 system. And codes are based on the open-source framework, TensorFlow 2.3 with Python API, and run on a dual-core Intel Core i7-6800@3.6 GHz processor with four NVIDIA Tesla p40 GPUs, which have 96 G computing caches and 256 G memory. Before inputting into the model learning process, all original data were encoded and standardized to improve the robustness of the model.To verify that the proposed method could be used in the grain supply chain for recognizing risk levels, we establish some experiments to compare the performance of the deep-stacking network with that of other methods. Some classical machine learning methods including logistic regression (LR), In order to illustrate the effect of the multigranularity scanning and cross-validation operation on the accuracy, we conducted separate experiments of each model in Architecture-1  and Architecture-2 (with corresponding operations). The identification accuracy results of each model are summarized in As shown in K-fold cross-validation operation, all models achieved better average accuracies than original baselines as listed in the Architecture-2 column, which directly proves the necessity of related operations.For example, GoogLeNet, ResNet, and deep forests design the complex nonlinear multilayer architecture to exploit discriminative high-dimensional features, achieving accuracy values up to 85.18%, 87.26%, and 90.73%, respectively. Meanwhile, with the less parametric architecture, the DSN method could produce the preferable accuracy by 94.88% even without the aid of padded-scanning and cross-validation operations, which have obviously improved the result by 9.7%, 7.62%, and 4.15%, respectively, in comparison with the above three deep learning models. It illustrates the advantage of a deep-stacking strategy in improving the efficiency and characterization capability of food risk prewarning. Moreover, with the extra support of multigranularity padded-scanning and Moreover, this proposed approach does not add much computational burden to the recognition system in terms of model parameters, which achieves comparable and even outperforming results compared with other methods. In order to better evaluate the computing cost of each model, we select the model parameters as the metric, which indicates how much computer memory is occupied when the model is trained. In this paper, we count up the memory scales of conserved model weights to record the value of model parameters. As shown in F1 indicator outperforming existing methods under the same baselines and has achieved large-margin promotion in terms of F1-score up to 0.967 under Architecture-1 and 0.991 under Architecture-2. This clearly indicates that feature learning and risk forecasting capability can still be improved even though the food data are very tricky for other methods. Therefore, our method is more suitable for risk level recognition in the food supply chains. The F1-score results of different models in two architectures are shown in Besides, our DSN method achieves the best result with 97.62% accuracy in automatically and accurately identifying grain risk levels, which clearly performs over comparative methods. Similarly, the DSN method enjoys consistent better performance on the F1 results for each risk category. When the PRE of a risk level is close to 1.00, it means that 100% of the security level instances in the test set are assigned to this correct level. And REC equal to 1.00 means that each item in the certain level belongs to the correct level, but does not indicate how many other items are incorrectly marked. Additionally, the range of the F1-score is from 0 to 1, with 1 representing the best output and 0 representing the worst output of the model. Therefore, the higher the value of three indicators, the stronger the model's ability for identifying and predicting.More comparisons of prediction results in terms of different risk levels are provided in F1 values up to 0.99 for each risk level. The comparison of different risk levels revealed that level IV acquires the lowest values due to the small value divide interval between safety and risky concepts, which is still important but hard to accurately identify and assess. The results of the DSN indicate that the deep-stacking strategy has a great influence on the learning of features while performing network convergence, which further proves that the DSN model is more suitable for the recognition task of risk level recognition in grain supply chains because its stability is higher than the other methods.As shown in Moreover, the confusion matrix is used to measure the prediction performance of the DSN model for each risk level, which is the most basic, intuitive, and simplest method to assess the right and wrong distribution of the model outcome. The normalized confusion matrix of the DSN is shown in K to analyze the influence. The relation between several hyperparameters and the model's accuracy is shown as follows.Although many deep learning models have shown their efficacy for application in various fields, they may not be suitable for different kinds of datasets or problems. The main drawback restricting the expanded application of deep learning methods is that their performance will be seriously influenced by hyperparameters based on experiential knowledge or many attempts to select parameters for better results. This process may spend a lot of computing resources and time, and the hyperparameters obtained may not be optimal, resulting in the instability and limited accuracy of models. Therefore, making an effective experiment to discuss the optimal hyperparameters is an essential step to improving the model's performance. To solve the above issue, we investigate the three main hyperparameters including epochs, batch size, and K hyperparameter, the accuracy of the model increases rapidly. The difference is that the accuracy tends to be stable and even fluctuates when the value reaches 5. This means that the best value of K is 5, and there is no need to choose a higher value. It will not only increase the amount of calculation but also affect the stability of the model. Finally, it can be seen from the relationship diagram of the epoch selection that the performance is achieved when the epoch hyperparameter is 100 when other conditions are set the same. Through the above discussion, the hyperparameters of batch size, K, and epoch in this article are, respectively, set as 64, 5, and 100, in order to ensure the optimal performance of the proposed method.As shown in In the rest of this paper, we study the risk level distribution of different hazards and analyze the relative effect on each supply link in order to offer vital prewarning rules for the whole food system as well as provide a detailed explanation related to the superiority of the DSN model for food safety assurance. Through the above experiments, we can know that the risk situation of the hazardous materials is different in each supply chain link, but the risk level of various hazards may gradually accumulate and spread along with supply chain processes. When these risk levels and related features were input into the RITS for a period, we found the results are encouraging to make operators find the food safety risk immediately and enhance the food quality assurance during the whole supply chain.Moreover, with fully utilizing the IoT and cloud analysis technology, the prewarning rules could be mined by the proposed deep-stacking method from the massive unstructured data stored in the cloud platform. All generated prewarning information would be fed back into the food system and history database to allow a continuous and interactive process to achieve superior solutions and traceable schemes for the future possible problems in supply chain management. When more effective information and communication are shared among supply chain partners with the support of an IoT-based food system, the hidden food safety information would be visualized. The prewarning information created in the supply chain along with the product transactions would be analyzed by operators in different food industries and companies, and all such results are extracted and displayed in a visualization way, through which the decision-making gap between supervisors and consumers can be more clearly understood and offset.For example, we could observe the distribution discrepancy of each risk level among grain supply chains as in In addition, the food risk identification and prewarning system with the DSN method will help to analyze the relationship among various hazards, minimize the production of unsafe quality products, and reduce the economic losses by avoiding product recall activities or additional destruction operations. For instance, with the prewarning analysis of the potential relationship between the hazard category and supply links, the prevention and control rules would be more scientific, while the operators always determine the corresponding operations on the basis of personal experience before. As shown in We could find that the production and sales links suffered a greater threat of various pollutants. Nearly 70% of the heavy metal hazards are detected in the production link originating from the metal residue of polluted soil and irrigation water. Heavy metals exceeding the standard are also frequently detected in other links, which means that heavy metals easily continue to migrate and spread in the grain supply chain. The excessive situation of pesticides has a similar proportional distribution except for the sale link, indicating that there are lots of excessive additives and unqualified operations in the subsequent process of sales and consumption, which need to be focused on and prevented in grain supply chain management. The typical issue is that the illegal addition of benzoyl peroxide (BP) which has been confirmed to damage the human body and forbidden in the Chinese national standard has increased in wheat flour and related products.Meanwhile, most microorganisms are concentrated in the early production links with 90.79%, which reveals that the grain safety risks of microorganisms are mainly caused by the production industries and processing enterprises since most microorganisms are sensitive to many environmental factors of the global grain supply chain including transit time, temperature, humidity, conveyance, and product type. In contrast, the mycotoxins are affected by more complicated factors in all links of the grain supply chain, where there are various intrusion sources. The distribution ratio also reveals that mycotoxins can exist for a long time with biodegradability degradation resistance, which are easy to migrate and enrich in continuous links of the grain supply chain. Thereby, with the support of the risk identifying and prewarning system, the performance improvement of risk identification and prewarning enhances the safety and quality level of food supply chains, which even promotes the management efficiency of the food crisis and prevents the interruption of sustainable food supply and the food recall cost, and waste would also be reduced.For example, with the geographical analysis of unqualified grain products in different provinces as shown in All in all, the developed DSN method has provided an effective approach for the food risk identifying and prewarning system to strengthen the quality, security, and sustainability of food supply chains. With the support of IoT, cloud analysis, artificial intelligence, and other technologies, it becomes easier to allow various food enterprises to effectively monitor, identify, and predict risk levels in real time. Accurately identifying and sharing of risky factors can reduce recall costs and unnecessary waste while promoting the consumers' satisfaction and security for high-quality food productions. The system also provides more prewarning information for policymakers to develop targeted regulatory policies and provide low-risk financial assistance with a comprehensive decision-making perspective to ensure the grain supply chains' quality sustainability. Experimental results and analyzing discussion of the practical cases validate the feasibility of adopting the proposed DSN model embedded into the IoT-based RITS, which would avoid food safety incidents to a significant extent.Overall, this study has proposed a new deep-stacking network for the IoT-based risk identification and traceability system to identify risk levels and mine traceability rules in food supply chains. With the support of IoT technology and cloud analysis platform, the system would monitor massive multisource biosignals about food supply safety through various sensors and instruments in real time. Then, a deep-stacking network optimized with data mining is proposed to enhance the efficiency and performance of the RITS for guaranteeing food supply chain security sustainability. The verification experiments with practical food safety datasets show that the proposed method has the best performance in terms of prediction accuracy and stability compared to other models, which takes various indicators reflecting identification accuracy, response speed, and computing consumption of food systems into account. Thereby, this work indeed makes a significant contribution to promoting the security and sustainability of food supply chains, as well as mining important decision support rules to help regulatory authorities, enterprises, and consumers.The proposed methods in this paper can combine other identification schemes for studying new modeling and prediction of dynamical systems –42 and c