What are the main findings?Innovatively designed sensor that uses airflow to adaptively reposition and flatten soybean leaves for optimized imaging results.The developed device can identify the effect of nitrogen treatment under both controlled environments and field conditions.What is the implication of the main finding?The throughput and resolution of obtaining a multispectral soybean image has been elevated compared to current proximal whole leaf imagers.Proximal sensing has the potential to outperform remote sensing because of the higher signal-over-noise ratio.p-value of the data collected by the device (p = 0.008) is significantly lower than that of a remote sensing system (p = 0.239).Image-based spectroscopy phenotyping is a rapidly growing field that investigates how genotype, environment and management interact using remote or proximal sensing systems to capture images of a plant under multiple wavelengths of light. While remote sensing techniques have proven effective in crop phenotyping, they can be subject to various noise sources, such as varying lighting conditions and plant physiological status, including leaf orientation. Moreover, current proximal leaf-scale imaging devices require the sensors to accommodate the state of the samples during imaging which induced extra time and labor cost. Therefore, this study developed a proximal multispectral imaging device that can actively attract the leaf to the sensing area (target-to-sensor mode) for high-precision and high-throughput leaf-scale phenotyping. To increase the throughput and to optimize imaging results, this device innovatively uses active airflow to reposition and flatten the soybean leaf. This novel mechanism redefines the traditional sensor-to-target mode and has relieved the device operator from the labor of capturing and holding the leaf, resulting in a five-fold increase in imaging speed compared to conventional proximal whole leaf imaging device. Besides, this device uses artificial lights to create stable and consistent lighting conditions to further improve the quality of the images. Furthermore, the touch-based imaging device takes full advantage of proximal sensing by providing ultra-high spatial resolution and quality of each pixel by blocking the noises induced by ambient lighting variances. The images captured by this device have been tested in the field and proven effective. Specifically, it has successfully identified nitrogen deficiency treatment at an earlier stage than a typical remote sensing system. The  Plant phenotyping is a critical tool for understanding the complex interplay between genomics, environment and management (G × E × M). In particular, multispectral and hyperspectral imaging technologies offer rich spectral and spatial data, making them valuable for studying plant traits . NowadayRemote sensing technologies offer powerful tools for studying plant traits, with unmanned aerial vehicles (UAVs) being one widely used option. UAVs can be flown between 5 and 200 m above the field, allowing for the capture of crop spectra . StudiesProximal sensors, including soil plant analysis development (SPAD) meters, are commonly used to monitor plant growth and health condition. Multiple studies have shown that SPAD readings are correlated with various crop traits ,22,23. HTherefore, imaging devices that capture the entire leaf are crucial and valuable for advancing plant science research. Such devices can provide researchers with a more comprehensive view of the plant and help to reveal insights into the complex interactions between crops and their environment.Recently, handheld multispectral and hyperspectral whole leaf imaging devices have been developed and proved useful for determining nutrient stress. Zhang et al.  developeHowever, all existing leaf-level multispectral imaging devices require sophisticated operation. Specifically, the users need to approach the sensor to the leaf, then flatten the leaf and clamp it between the light source and the camera to capture images. This operation is time-consuming and laborious, regardless of whether it is in the field or in the controlled environment. This traditional sensor-to-target imaging mode needs to be redesigned to reach higher throughput. Therefore, it is necessary to improve the fundamental mechanisms of these leaf-level imagers to relieve the labor and operation cost to increase the throughput while taking full advantage of proximal sensing. Furthermore, the device also needs to provide consistent and stable lighting conditions to remove the noises induced by the environment.With the considerations described previously, in this work, we present an innovated multispectral leaf-level imaging device for dicotyledon plant leaves that is fast and accurate. This device is designed to provide consistent lighting conditions by using LEDs while blocking out the ambient light. Such design is necessary to obtain informative images. Moreover, this device adopts a new mechanism that uses active air flow to manipulate the orientation of the leaf, keep the leaf flattened and hold the leaf in place against a leaf barrier made by ultra-fine nylon threads. This mechanism creates a new target-to-sensor mode, which relieves the users from clamping and flattening the leaf, thus, the throughput is increased by five times compared to other whole leaf imaging devices for dicotyledon plant. Furthermore, the device is compared with a remote sensing system to compare the capability of separating nitrogen treatments.In this paper, The DC fans used to create the airflow were manufactured by Mechatronics; each of the fans has a nominal 4500 rotation per minute (RPM) max under 12 V input.The leaf barrier attached to the bottom of the imaging chamber consists of a grid of 0.2 mm thick nylon threads anchored to the frame. These fine monofilament nylon threads have low elasticity and high strength that allow them to hold the leaf flat and prevent the leaf from being sucked into the chamber. A black non-transparent felt fabric (150 mm × 225 mm) is attached to the leaf barrier, and it will sandwich the leaf after it is captured and hold against the nylon network to block the ambient light to reduce the noise.This device images a single leaf to obtain the reflectance multispectral images of four wavelengths including blue, green, red and near infrared (NIR) bands. The wavelength of each band 460 nm (blue), 525 nm (green), 630 nm (red) and 850 nm (NIR). The image captured under visible light  and NIR light is helpful to investigate the chlorophyll and nitrogen content. There are two light boards in the device, and each lightboard is covered with 2-mm thick white Teflon panels to diffuse the light for even distribution. Each light board houses eight LED strips  manufactured by Waveform Lightings.A monochrome camera manufactured by Teledyne FLIR LLC is used to image the leaf. The camera has a resolution of 1920 pixels × 1200 pixels (H × V) and a pixel size of 5.86 um × 5.86 um (H × V). A low distortion (−1.0%) lens with fixed focal length manufactured by Kowa Optronics Co., Ltd. is attached to the camera as an object lens. The device is powered by a 12 V rechargeable portable battery. The power source is converted to 5 V to power the Raspberry Pi 4B, which serves as the microprocessor of the device. The LEDs and fans are powered by the same power source that the microprocessor uses, but the power is stabilized via a 12 V–12 V converter to maintain the brightness of the LEDs when the battery voltage decreases. A push button switch wired to the microprocessor is used to initiate the imaging sequence. The microprocessor, programmed using Python3, sends pulse-width modulation (PWM) signal to LED and fan drivers to turn on the LEDs and control the fans to operate at 80% of the max RPM. The microprocessor also powers the camera and controls the imaging timing through onboard USB 3.1 port. The images captured are saved as tagged image file format (.TIFF) in SD card mounted on Raspberry Pi 4B. The power consumption of the device is 30.7 W when imaging and 1.6 W while idling. The command and data flow are illustrated in The flow chart in Once the device is closer than 1 cm from the leaf, the operator presses the push button switch. Two DC fans will create airflow that attracts the leaf to the leaf barrier and hold the leaf against it. The operator then flips the black non-transparent felt fabric beneath the leaf to block the ambient light. The operator does not need to hold the fabric because the fabric is soft and light; the air flow created by the DC fans is also capable of holding it flat against the leaf barrier. With a one-second time delay after pressing the push button switch, the first wavelength of the LED turns on, and the camera captures an image. Such a process is repeated three times until all wavelengths of the LED are turned on once and a corresponding image is captured. Once completing the imaging sequence, two DC fans are turned off, and the leaf is released. Finally, the device transfers a preview image to the smartphone App via Bluetooth, and the operator examines the preview image to assure the entire leaf is captured.The whole imaging process starts from pressing the toggle switch and ends as an image is transferred to the smartphone App, taking no more than five seconds. The time consumption for capturing one image has been drastically reduced five times compared to a current whole leaf imaging device, such as the imager developed by Wang . Though With the image captured by the device, post-processing of the images is necessary before analyzing the nitrogen content. Since there was a gird of nylon thread used to hold the leaf specimen, it is necessary to reconstruct the leaf to recover the information to reach best image quality. Therefore, this section explains the image preprocessing procedure to improve the image quality, including calibration of uneven distribution of the light source, segmentation of the leaf from the backgrounds, the removal of nylon thread and recovery of those missing pixels.White referencing is used to compensate for the non-uniform distribution of the light source. Equation (1) is used to calibrate the raw image.calImage is the calibrated image; rawImage is the raw; and the whiteImage is the image of a white color board taken under each wavelength of light. Therefore, there are four whiteImage images used for corresponding rawImage. Same whiteIamge was used for different samples as the nonuniformity is only related to the location of the LEDs, which was not changed among samples.Greenness of an image using the following Equation (2).Color thresholding segmentation is used to segment the leaf from the background. As the background is pure black, the leaf area can be determined by calculating the After segmenting the green leaf from the background, a dilation operation followed by erosion operation was performed to smooth the edge of the leaf and to remove small objects from the image.After removing the background from the images by color thresholding, the nylon threads were not removed. Because the values of the pixels corresponding to the nylon threads were always higher, due to the color and reflectivity, the thresholds used for segmenting black background were not high enough to segment the threads. It was possible to remove the threads simply by raising the thresholds, but the pixels corresponding to the leaf would be impacted as some leaf may have had intensities like the nylon threads.Therefore, this study used a 2-D Gabor filter  integrated in MATLAB script  to segment the nylon threads with minimum impact to leaf pixels. The Gabor filter transforms the intensity of each pixel to the frequency response of an impulse signal. The frequency response contains two values: magnitude and phase. By varying the parameters of the impulse signal, objects with specific orientation will have different magnitude and phase; thus, the objects could be detected based on the phase and magnitude. In this study, the Gabor filter applies an impulse signal with wavelength parameter set as 5 and spatial aspect ratio parameter set as 0.1 was used to detect the nylon threads aligned at 45 degrees. inpaintExemplar, and it is included in the image-processing toolbox made by MATLAB . This function recognizes the gap area that needs to be inpainted and searches the remaining images for a best-matching patch to fill in the gap. By only using the pixels from the undamaged part of the image, the reconstructed area would have similar intensity with the rest of the image, which is crucial for multispectral imaging as many phenotypic parameters are calculated based on the image intensity under different lights. This function has two input arguments: the image to be inpainted and the mask of target regions. For this study, masks like inpaintExemplar function: the Filling order parameter was set as tensor and the patch-size parameter was set as 9 × 9 pixels. The filling order parameter calculates the priority of each patch in the target region, and the patch size specifies the number of pixels in a patch. By reducing the patch size, the target region will be separated into more patches, resulting in more computing time. Based on our experiments, setting patch size as 9 is most cost-effective. After the pixel containing nylon threads is removed, the leaf becomes fragmentary and is not ideal for phenotyping. Therefore, the segmented image needs to be reconstructed to a whole leaf without missing pixels. Image inpainting has been a traditional technique to reconstruct images that are partially damaged or lost while a major part of the image is intact . PopularWith the completion of the hardware assembly and software deployment, this study investigated the effectiveness of this device in separating nitrogen treatment, both under controlled environments and field conditions.Two rounds of imaging experiments were carried out to validate the capability of separating nitrogen treatments of this sensor. The first round imaged the plants grown in a controlled greenhouse environment. The second round imaged the plants grown in the field. The first round of imaging was conducted using soybeans planted in plastic pots in the Lily greenhouse facility, located at Purdue University . The soybean genotype was Asgrow 29XF1 from BAYER. The experiment used self-mixed growing medium. The medium was a mixture of BM2 seed germination and propagation mix made by Berger and pure sand, and the volume ratio was 2:1, respectively. Two nitrogen treatments with nitrogen parts per million (PPM) as 200 and 800 were applied to each genotype with 10 replicates for each treatment, which totaled to 20 plants. After about 60 days after seeding and the plants were at the V8 stage, the middle leaf of the top fully expanded trifoliate and the middle leaf of the lowest trifoliate was imaged by the newly developed device, resulting in 40 images.NDVI across the leaf can be calculated by using the following Equation (3) [Normalized Difference Vegetation Index (NDVI) has been proved to have strong correlation with the nitrogen content of a plant ,32. Sinction (3) .(3)NDVIThen the whole leaf NDVI was averaged to obtain a single value. This value is used as an indicator to distinguish nitrogen treatment. The results are plotted in box chart shown as t-test between the same genotype was carried out to evaluate the device’s performance on detecting the effect of nitrogen treatment. Based on the indicator obtained from the bottom leaf, the p-value between high and low nitrogen treatment is 5 × 10−6 (p < 0.001). Moreover, the indicator obtained from the top trifoliate for Asgrow is 7 × 10−4 (p < 0.001). Therefore, based on the statistical results, it is reasonable to conclude that the whole leaf averaged NDVI captured from the device is effective on detecting the effect of nitrogen treatment.A two-sample 2 for the fitted model indicates a relatively strong correlation between the chemical reference data and the indicator measured by the newly developed device. After collecting the images for every subject, all the leaves from each plant were cut off and sent to laboratory  for nitrogen content measurement. The laboratory results were used to determine the correlation between the indicator we obtained from the device and the nitrogen content of the plants. For each plant, we averaged the indicator collected from the leaf of top fully expanded trifoliate and the bottom leaf to characterize the whole plant. This set of experiments have validated the device’s capability of detecting soybean nitrogen treatment with relatively high confidence. However, further experiments including more genotypes and treatment will be needed to validate the fitted model. The major source of error is the individual differences between each soybean canopy. As the soybeans are grown in the greenhouse, the microclimate will have effect on the growing of the canopy that could induce large variance to plant grow . AnotherThe second round of imaging took place at field 9C of Purdue Agronomy Center for Research and Education (ACRE) in West Lafayette, IN, USA in 2022. The field layout was shown in The imaging took place at V6–V7 stage, and only the middle leaf of the top fully expanded trifoliate was imaged using the device because the bottom leaves were not reachable due to the density of the soybean canopies. The imaging procedure was the same as imaging in the greenhouse. For each image, the NDVI heatmap was calculated and averaged to obtain a single indicator. Four randomly selected leaves were imaged from each plot. The results of these four images were averaged again to represent the plot to which they belonged.Along with the individual leaf imaging, the field was imaged by an UAV (model: DJI M600) mounted with hyperspectral camera ) to obtain remote sensing results of the field from 44 m above the field. The results are shown in A set of masks were created for each plot to calculate the average NDVI of that plot. Each rectangle mask includes 357 pixels and has identical shape, shown as red and yellow blocks in p-value calculated two-sample t-test. Based on the p-value, the images captured by the device separates the low and high nitrogen treatment, while images captured via remote sensing does not describe the treatment effectively. These results again prove the disadvantages of remote sensing, which includes too many noises from the environment, and the plant canopy could jeopardize the phenotyping results.In this study, the design of a new multispectral whole leaf imaging device is presented along with the experiment to validate the effectiveness of this new device. The purpose of developing this new device is to increase the sampling throughput without compromising the data quality. Compared to the previous developed device ,27 the n2 = 0.19 in the study of correlating NDVI and nitrogen content. Saberioon and Gholizadeh [On the other hand, remote sensing still has higher priority compared to proximal sensing because remote sensing has the data acquisition speed that current proximal sensing technologies cannot meet. However, results from studies on farmland that have related NDVI of nitrogen content are still weak. Tilling et al.  reportedolizadeh  reportedCompared to remote sensing platforms, the newly developed proximal sensing device can effectively describe the nitrogen treatment in the field by only a few data points. Therefore, the total time consumption of using this device to image the field is not significantly increased. Furthermore, proximal sensing showed better capability in analyzing nutrient information, Song et al.  developeHowever, studying a huge field by human use of this device is still unachievable. In the future, with an automation system that can operate this device, the throughput would be further increased to match the throughput of remote sensing while having superior data quality.In this study, a new type of multispectral proximal imaging device for soybean leaves has been developed. The device comprises a high-resolution camera, LEDs with 4 wavelengths and two DC fans. It is designed to block out ambient light while imaging and the LEDs inside the housing will provide consistent lighting conditions, which increases the signal-over-noise ratio and spatial resolution compared to remote sensing technologies. Moreover, this device used DC fans to create active airflow to reposition and flatten the leaves during imaging. This mechanism relieves the operator from laboring and time-consuming operations required by other proximal imaging devices and to improve image quality by removing noise caused by leaf angles. As a result, this device can image a soybean leaf in less than 5 s.p-value < 0.01. (2) Data collected by the device shows a relatively strong linear correlation with laboratory measured nitrogen content. In field experiments, the remote sensing platform failed to distinguish the nitrogen treatment (p = 0.239) while the device successfully detects the effect of nitrogen treatment (p = 0.008). These experiments have proven the images captured by the device are informative for studying plant phenomics.To evaluate the performance of this newly developed device, a greenhouse experiment, and an in-field experiment were carried out. In greenhouse experiments, the results show: (1) The device can clearly detect the nitrogen treatment with This new development has reduced the labor and time consumed in plant phenotyping without compromising the quality of the results. It has elevated the state-of-the-art of leaf-level multispectral imagers, and it could benefit wider populations. However, the data collection process both in field and controlled environment still requires human participation, which is labor and time-consuming. Therefore, it would be valuable to develop a robot system that can operate this newly developed sensor to further relieve human labor for even higher throughput and data quality.


Background: Airway stent has been widely used in airway procedures. However, the metallic and silicone tubular stents are not customized designed for individual patients and cannot adapt to complicated obstruction structures. Other customized stents could not adapt to complex airway structures with easy and standardized manufacturing methods.Object: This study aimed to design a series of novel stents with different shapes which can adapt to various airway structures, such as the “Y” shape structure at the tracheal carina, and to propose a standardized fabrication method to manufacture these customized stents in the same way.Methods: We proposed a design strategy for the stents with different shapes and introduced a braiding method to prototype six types of single-tube-braided stents. Theoretical model was established to investigate the radial stiffness of the stents and deformation upon compression. We also characterized their mechanical properties by conducting compression tests and water tank tests. Finally, a series of benchtop experiments and ex vivo experiments were conducted to evaluate the functions of the stents.Results: The theoretical model predicted similar results to the experimental results, and the proposed stents could bear a compression force of 5.79N. The results of water tank tests showed the stent was still functioning even if suffering from continuous water pressure at body temperature for a period of 30 days. The phantoms and ex-vivo experiments demonstrated that the proposed stents adapt well to different airway structures.Conclusion: Our study offers a new perspective on the design of customized, adaptive, and easy-to-fabricate stents for airway stents which could meet the requirements of various airway illnesses. Airway illnesses, such as tracheobronchomalacia (TBM) and tracheobronchial stenosis (TBS), may cause the patients to cough, wheeze, apnea and sometimes lead to profound airway obstruction which will threat patients’ life . TBM andTo design airway stents used for the above-mentioned clinical practices, several requirements have to be met. First, the diameter and the radial stiffness of the airway stents must be systematically optimized, to keep the airway open and reduce the risk of unnecessary complications simultaneously. Oversized stent or too large radial stiffness will cause the airway tissue injured, e.g., mucosal ischemia, while undersized stent or too weak radial stiffness will result in migration inside airways . Second,Over decades, airway stents has been constantly studied . The mosin vivo molded airway stent was designed. The thermoplasticity-based single helical stent would not induce the growth of the granulation tissue and avoid blocking the mucociliary clearance, and its diameter can also be adjusted during clinical procedures  . The stein vivo molding stents, the STB stent is designed as multi-helix structure, to provide sufficient supporting force and contact points to the airways. From the perspective of design, five conditions must be fully met in this work.The design strategy of STB stents is described in this section. The STB stents are designed to be braided by a single tube filled with UV-curable polymer. The tube we selected is a biocompatible silicone tube, and the softness brought by the silicone material promise the stent to exhibit easy deformation and adaptation to the airway structures. To provide sufficient supporting force to the airway, the radial stiffness is the key design criteria. In this work, the UV polymer filled into the silicone tube is liquid before curing but will turn to rigid material after ultraviolet radiation. In view of the shortcoming of the First and foremost, the stent must be braided by one single tube, to provide customized stents for various airway illnesses, and the stent shall be composed of Although the following theory can be used to design the proposed stents with more than three pairs of curve group structures, the number of curve group structures For a single curve group structure, the two helixes must be symmetric, so the pitch and diameter of the helix are the same and the only parameter determining the length is number of coils for each helix Before deciding the actual number of coils, the relationship between the number of helical coils and the parameters of airways shall be expressed.upported .However, Once the number of helical coils of STB stents have been determined, the rest works to conduct is to decide the pitches and diameters of the stents. The four helixes of the stent are bonded to each other at their interaction points exhibiting numbers of rhombic structures , which wThe stents will be winded onto the mold for consistency, so the diameter of the mold plus the twice the diameter of the tube shall be smaller than the diameter of human main bronchi . The diameter shall be as small as possible, but not too small to unnecessarily enhance the difficulty in manufacturing.Since we have the diameter of mold The length of mold Here, the materials we chose, the fabrication method and curing process are described in this section. In this work, we selected a silicone tube with a thickness of 0.15 mm and the UV-curable polymer we selected with high elastic modulus  and relatively low creep.To braid the proposed STB stent of diverse shapes, we designed a modular molding technology . The modThe braiding process is described in detail in The curing process is described in detail. The ready-to-use stent will be placed on a balloon catheter, with which the stent will be delivered to the airways. We used an hourglass-shape transparent pipe to represent the trachea with stricture, which was designed to be consistent with the anatomical shape of an adult trachea. The balloon catheter and pipe are shown in In this section, we established a model to investigate the radial stiffness of the STB stents, and meanwhile we performed a series of experiments to investigate their mechanical properties. The deformation of the STB stent under uniformly distributed loading was studied in the first set of experiments, to verify the model. The second set of experiments investigated the fatigue performance of STB stents over time under radial loading (water pressure at a depth of 10 cm). The modeling and experiments are described below.The stents used in clinical procedures are usually oversized than the airways to prevent the stent from migration and hence suffer from compressive forces due to the elasticity of the tissue. Breathing and coughing of patients can further improve the compressive force. We established a theoretical model to establish the relationship between the compressive force and the radial deformation of the STB stent. This model shows how the parameters influence the stiffness of STB stents, such as the cross-section area of cured polymer, diameter D and helical angle of STB stent.Based on the design configuration, we assume that the force applied on all helixes are the same. To simplify the calculation, half coil of a single helix is chosen for analysis . The curThe perimeter of the semi-ellipse is kept constant due to the inextensibility of the cured polymer, we can get the relationship of the axes before and after compression: ellipse .The horizontal and vertical component of the uniformly distributed loading force Then, we can obtain the reaction force by Eq. The friction between the beam and the internal wall of the airway was assumed as below  and two parallel plates with curved fillisters, between which the stents are positioned, as shown in After implanted, the stents are suffering from continuously radial loads at body temperature over a long period of time. Therefore, to evaluate the performance of stents in a simulated environment, we performed water-tank tests. The proposed stents were delivered into a phantom tracheal models, which was placed under 37°C water . AccordiThe phantom tracheal model was casted with silicone with an inner diameter of 10 mm and a thickness of 0.2 mm  investigated whether the proposed STB stent could provide support to different airway regions in more complicated scenarios. The T shape stent shown in ex vivo experiments with trachea and bronchi harvested from a swine , would not cause injuries to the tissue, and the geometrical design would not induce the growth of granulation tissues which may require invasive procedures to remove the stents out of the airway. In terms of the comparison with silicon stents in clinical trials, our stents can reduce the impediment on mucus flow and has less possibility to cause mucus plugs. In addition, the proposed stent shares the same advantage with the previous work that the stents can be easily screwed out of airways to reduce the chances to hurt the tissues, but this stent could provide more reliable and uniform supports to the trachea than the previous work. Moreover, the proposed braiding method provides possibility to manufacture all types of airways stents with the same fabrication technique.ex vivo experiment was conducted in this work. To further evaluate the practicability of STB stents in clinic procedures, we still need to conduct in vivo experiments to test the proposed stents in future.Our study demonstrated the potential of STB stents, but there are still limitations which require further investigation in future works. 1) Fabrication: To fabricate a STB stent, we manually braided the tube on a 3D printed mold, which is not a standard fabrication technique. The errors in fabrication will add inaccuracy to the theoretical prediction. Thus, an automatic braiding equipment shall be built to improve the fabrication precision in the future. 2) Modeling: In this paper, to simplify the modeling process, we chose half elliptical beam as the representative for analysis. We also made a few assumptions or used approximated calculations formulas. Even though the modeling showed similar results to the experiments, the accuracy of theoretical model is limited. 3) Experiment: It is normal to find both continual and fluctuant breathes in real breathing. But in the water tank tests, the silicone tube was placed under water with the inner space connected to the atmosphere. The experiment only provided static tests but no dynamic physiological conditions were simulated. 4) Animal test: Only


In this paper, the prediction of flexural strength was investigated using machine learning methods for concrete containing supplementary cementitious materials such as silica fume. First, based on a database of suitable characteristic parameters, the flexural strength prediction was carried out using linear (LR) model, random forest (RF) model, and extreme gradient boosting (XGB) model. Subsequently, the influence of each input parameter on the flexural strength was analyzed using the SHAP model based on the optimal prediction model. The results showed that LR, RF, and XGB enhanced the accuracy of forecasting sequentially. Among the characteristic parameters, the most significant effect on the flexural strength of concrete is the water-binder ratio, and the water-binder ratio shows a negative correlation with flexural strength. The effect of maintenance age on flexural strength is second only to the water-binder ratio, and it shows a positive trend. When the amount of fly ash is less than 40% and the amount of slag or silica fume is less than 30%, the correlation between the amount of supplementary cementitious materials and flexural strength fluctuates and a positive peak in flexural strength is observed. However, at a dosage greater than the above, the supplementary cementitious materials all reduce flexural strength. The interaction interval and the degree of interaction between the supplementary cementitious materials and the cement content also differ in predicting flexural strength. Among the many mechanical properties, strength is one of the most basic indicators. The study of concrete strength can be traced back to the beginning of the invention of concrete materials. Usually, the strength of concrete includes compressive strength and flexural strength, etc.4. For some structural members with special force and support requirements, the flexural strength of concrete must meet the requirements, the in-depth study of which is extremely important6.The mechanical properties of concrete have been of great interest8. These studies have greatly enriched the understanding of the factors influencing the flexural strength. Among the many factors affecting the flexural strength of concrete, the composition of raw materials is one of the most basic and important factors10. Among the raw materials, the connection between cement and strength is more complex and important compared to other raw materials such as aggregates because the hydration products of cement are directly related to strength12. However, the massive use of cement will produce more greenhouse gases such as carbon dioxide, which aggravates environmental pollution and contradicts the current promotion of energy saving and carbon neutrality13. This has led to a gradual shift of attention to the development of cementitious materials with a certain cementing capacity and low carbon emissions. Compared to cement, supplementary cementitious materials such as fly ash, slag, silica fume, and metakaolin are more environmentally friendly both in preparation and use17. This makes it important to study in depth the effect of supplementary cementitious materials on the mechanical properties  of concrete. Currently, there have been many studies on the strength of concrete containing supplementary cementitious materials, as summarized in Table During the study of concrete flexural strength, specimens were usually tested on the basis of changing the raw material ratio, curing conditions, and the curing age34. For example, Nguyen et al.35 studied the compressive strength of concrete by machine learning model. Zeng et al.36 used deep learning theory to predict achieved accurate prediction of compressive strength of concrete. However, most of the aforementioned studies have focused on concrete compressive strength. Few studies have been conducted on the flexural strength of concrete using machine learning, and no studies have been reported on the flexural strength of concrete containing supplementary cementitious materials. Therefore, there is a need to conduct scientific machine learning modeling studies for the flexural strength of concrete containing supplementary cementitious materials to promote the development of flexural strength of low-carbon concrete.The above studies have provided a significant boost to the related field and have led to a better understanding of it. However, most of these studies have been carried out by traditional experimental means, i.e., by controlling specific ages, specific water-binder ratios, etc., and obtaining specific values using a press machine. This will make the concrete containing supplementary cementitious materials in some projects with large volume and very high strength characteristics requirements, which will increase the experimental labor and material as well as time costs. Moreover, due to the influence of multiple factors, the final test results may be different from the actual engineering application. Therefore, the search for a concrete strength research method that can integrate multiple strength influencing factors with high efficiency and accuracy is on the agenda. With the development of computer science, some researchers have introduced machine learning into the study of concrete strength40. The existing studies were synthesized and a database covering the appropriate feature parameters was established. Based on this database, modeling studies were conducted using the linear (LR) model, random forest (RF) model, and extreme gradient boosting (XGB) model. The purpose of the above machine learning model selection is first to verify whether there is a linear relationship between various feature parameters, and then select two integrated machine learning models that are currently widely used for in-depth analysis. On the basis of the optimization model, the characteristic features were investigated based on the Shapley additive interpretation (SHAP).In view of the above, this study selected fly ash, slag, and silica fume, which are three widely used supplementary cementing materials at presentThere are numerous elements affecting the flexural strength of concrete (F_S), including water-binder ratio, aggregate content, etc. For concrete containing supplementary cementitious materials such as fly ash, slag, and silica fume, the admixture of supplementary cementitious materials is also an important influencing factor. Therefore, this study summarizes the factors influencing the strength of concrete with supplementary cementitious materials according to the possible differences in the process of making and maintaining concrete. First, in the preparation stage of concrete with supplementary cementitious materials, cement as the main cementitious material, the influence of cement content (cem_con) on the flexural strength of concrete cannot be ignored. Taking the cement admixture as a benchmark, the admixture of fly ash (FA_con), slag (S_con), and silica fume (SF_con) as alternative cement materials in the concrete is also the focus of this study. As an important participant in the hydration process of cementitious materials that cannot be ignored, the water content has an important effect on the strength. Therefore, the water-binder ratio (w_b) is also a characteristic focused on the study. Aggregate, as the component with the largest proportion of the three-phase composition of concrete, also has an important influence on the flexural strength of concrete, so fine aggregate content (Fine_A_con) and coarse aggregate content (Coarse_A_con) are set as characteristic parameters as well. During the curing of concrete, the hydration process of the cementitious material changes with the increase of curing time, so the curing age (Age) also has an important influence on the flexural strength. It is worth noting that the curing temperature and the curing humidity also affect the hydration of the cementitious materials during the curing of concrete specimens, which in turn affects the flexural strength of concrete, but these two are not considered characteristic parameters in this study. The curing temperatures of the selected concrete specimens were in the range of 20 ± 2 °C and the relative humidity fluctuated in the range of 60 ± 5%.60, and 373 valid data sets were compiled. The data on flexural strength of 373 groups of concrete were counted, and the mathematical characteristics of the input and output parameters were obtained as shown in Table When using machine learning methods for predictive modeling of material properties, a database covering all feature parameters with a sufficient amount of data is required. A good database contributes to the successful implementation of machine learning modeling and helps to improve the accuracy of subsequent feature parameter analysis. In this study, eight sets of input parameters  and one set of output parameters (F_S) were set according to the influencing factors given in "i is data sample; m is the number of i; yi, The requirements for the database are to ensure that, in addition to the quantity and quality of the dataset, there are no significant linear relationships between the feature parameters. When the linear relationship between the feature parameters is strong, it will affect the model prediction accuracy. In order to characterize the linear relationship between different feature parameters, statistical analysis is performed using Pearson correlation coefficient (PCC) in Eq. . The i iThe data hot map of PCC values for all input parameters and output parameters is shown in Fig. Many machine learning models have been successfully validated in the study of performance prediction of concrete materials. The most basic model is the LR model, which has a simple modeling process, simple analytical ideas, and fast computation speed, and was widely used before the integrated machine learning system was proposed. The calculation principle of the LR model is shown in Fig. 17.Integrated algorithmic models, as emerging machine learning models, possess more sophisticated computational procedures that can significantly improve prediction accuracy compared to linear model systems. Machine learning is accomplished by constructing multiple learners that adopt specific strategies. The purpose of the integrated algorithm is to construct a strong learning machine from multiple weak learners. This leads to a better analysis of various parameter features61. When using the bagging method for calculations. During processing, strength learners are obtained by processing many weaker learners. Subsequently, the average value is processed.The base learner chosen for modeling the flexural strength of concrete through the RF model is the decision tree. Autonomous aggregation algorithm (bootstrap aggregation) for learners61.This model is an extended extension of the bag method in which the schematic diagram is presented in Fig. 35. Figure Among centralized machine learning systems, the XGB model is a rapidly developing and widely used efficient learning model because it's fast, flexible and lightweight. The XGB model was also utilized as a decision tree to serve as the basic collector for machine learning. The integrated learning method used in the XGB model is the BoostingV number of tree structures , as shown in Eq. , it is Li and Pi in the above equation are shown in Eq.  is the prediction result i; u0 is the model baseline; xij denotes the j parameter for sample i; g(xij) is the devote in f(xi). f(xi) has the uniqueness17. In PYTHON, the calculation is analyzed by SHAP. Based on the data in the database, the feature values are obtained using the XGB model.The analytical model used in analyzing the effect of various parameters on flexural strength is SHAPn in Eq. :13\documThe SHAP value can characterize both the parameter with the highest degree of influence when analyzing the influence of a parameter and the influence process within the parameter range. In this case, the impact is positively correlated for SHAPM values greater than 0 and negatively correlated for values less than 0.Based on the previous comparison of the computational results of the three machine learning models, it was demonstrated that the XGB model performed optimally in predicting the flexural strength of concrete. Therefore, in this study, the SHAP model was used to analyze the effect of various characteristic factors on the flexural strength of concrete based on the XGB model.The gradient of the strength influenced by the eight input parameters affecting the flexural strength of concrete was first determined and characterized using SHAP mean values, and the results are shown in Fig. To further analyze the effects of various input parameters on the flexural strength of concrete and to provide insight into the trend of their effects on flexural strength from the numerical changes of the input parameters, the SHAP values of the input parameters shown in Fig. Combining Figs. 3, the variation of SHAP values showed fluctuation with extreme values around 370 kg/m3. This indicates that the strength will have a peak value at a cement admixture of 370 kg/m3. However, since in this study, there is cement along with supplementary cementitious materials, the effect of cement needs to be analyzed together with the three supplementary cementitious materials. The SHAP values of the two aggregates show similar trends with the aggregate content. Analyzing Fig. The relation of the effect of various parameters on the strength is illustrated in Fig. Then the effect of the three supplementary cementitious materials on the flexural strength of concrete is analyzed, and it can be found from the figure that FA_con and SF_con show a similar trend to w_b affecting the flexural strength of concrete on some intervals. This indicates that in terms of strength, the above two characteristic parameters have less flexural strength when they are present in concrete in larger amounts. However, in marked contrast to w_b, the fluctuation of the SHAP value of FA_con at less than 40% content is not obvious, and even up to 20% shows an increasing trend. This indicates that the flexural strength of concrete peaks at around 20% of FA_con content. For SF_con, its SHAP value appears to show fluctuations below 30%. and a smaller peak occurs around 10% and 30%, respectively. Subsequently, when the content of S_con exceeds 30%, the value decline. Therefore, combining the numerical changes of the SHAP value of S_con in several content zones, it can be concluded that the effect of S_con on the flexural strength of concrete may have peaks at 10% and 30% when the content is small. However, when the content of S_con exceeds 30%, the flexural strength decreases gradually. As for the characteristic parameter SF_con, when its content is below 18%, the value presents a fluctuating growth tendency. When the value of SF_con exceeds 18%, the trend is downward. The above shows that the maximum value of flexural strength of concrete may occur when the amount of silica fume in the cementitious material is 20%.3 and 400 kg/m3, indicating that the slag and cement interactive intervals are less than FA, and there is a certain effect on the flexural strength of concrete in this interval. In Fig. 3. This indicates that the interaction interval between silica fume and cement is mainly concentrated in the interval when the cement content is small, and silica fume will have an important effect on the flexural strength of concrete in this interval.In order to analyze in depth the effect of composite cementitious materials on the flexural strength of concrete when three supplementary cementitious materials act together with cement, and to clarify the interval of different supplementary cementitious materials acting on cement content. In this study, the SHAP values of the three supplementary cementitious materials were analyzed in combination with the SHAP values of the cement, the interactions were analyzed as shown in Fig. The LR model is the poorest predictor of flexural strength for concrete with supplementary cementitious materials. The XGB model has a higher prediction accuracy than the RF model. The XGB model is recommended for predicting the flexural strength of concrete with cementitious admixtures.Among the characteristic parameters that affect the flexural strength of concrete with supplementary cementitious materials, the highest degree of correlation with the predicted results is the water-binder ratio, followed by the curing age. The weakest correlation with the predicted results was the admixture of silica fume and slag.Water-cement ratio and age of curing have the greatest influence on flexural strength. The water-cement ratio is negatively correlated with flexural strength, while the age of maintenance is positively correlated. The relationship between aggregate content and flexural strength showed a positive correlation followed by a negative correlation. The relationship between flexural strength and cement content was similar to that of aggregate content. The effect of the three supplementary cementitious materials on the flexural strength of concrete can be summarized as the correlation with flexural strength fluctuates at small admixtures  and there is a positive correlation with the optimum admixture. However, the correlation becomes significantly negative at dosing levels greater than the above.3 and 400 kg/m3. Silica fume has an important role in low cement content.In predicting the flexural strength of concrete, different supplementary cementitious materials interact with cement in different admixture intervals. Fly ash can act in the whole blending interval of cement. Slag has an important role between cement h content of 200 kg/mFor the flexural strength of concrete containing supplementary cementitious materials, prediction studies were carried out using three machine learning models. The parameters were analyzed based on SHAP model, with the following main conclusions:


Ensuring road safety, structural stability and durability is of paramount importance, and detecting road cracks plays a critical role in achieving these goals. We propose a GM-ResNet-based method to enhance the precision and efficacy of crack detection. Leveraging ResNet-34 as the foundational network for crack image feature extraction, we consider the challenge of insufficient global and local information assimilation within the model. To overcome this, we incorporate the global attention mechanism into the architecture, facilitating comprehensive feature extraction across the channel and the spatial width and height dimensions. This dynamic interaction across these dimensions optimizes feature representation and generalization, resulting in a more precise crack detection outcome. Recognizing the limitations of ResNet-34 in managing intricate data relationships, we replace its fully connected layer with a multilayer fully connected neural network. We fashion a deep network structure by integrating multiple linear, batch normalization and activation function layers. This construction amplifies feature expression, stabilizes training convergence and elevates the performance of the model in complex detection tasks. Moreover, tackling class imbalance is imperative in road crack detection. Introducing the focal loss function as the training loss addresses this challenge head-on, effectively mitigating the adverse impact of class imbalance on model performance. The experimental outcomes on a publicly available crack dataset emphasize the advantages of the GM-ResNet in crack detection accuracy compared to other methods. It is worth noting that the proposed method has better evaluation indicators in the detection results compared with alternative methodologies, highlighting its effectiveness. This validates the potency of our method in achieving optimal crack detection outcomes. The escalating demands on transportation and the pervasive impact of the natural elements, such weather conditions, has made the occurrence of cracks within transportation infrastructure  a prevalent concern. If the challenge of crack detection is not proactively addressed, it could precipitate a diminution in the service life of the infrastructure. Even more concerning, it might trigger a sequence of safety-related incidents, exacting substantial economic ramifications and human casualties . This unThe conventional approach to road crack detection relies primarily on manual inspection, where personnel visually examine road cracks. However, this method proves time-consuming and labor-intensive, often yielding unsatisfactory outcomes due to environmental challenges, such as dim lighting, shadows and background noise . ContempWhile CNNs proficiently extract crack image features and achieve precise road crack detection, its constrained representation capacity poses challenges in maintaining generalization across varying object scales, sizes and angles. Addressing this, the attention mechanism can allow the model to dynamically modify the weight based totally on the importance of the input data so that it can focus on processing some critical feature information that helps with the detection task, reducing attention on irrelevant redundant feature information, andthus, enhancing the generalization fitting ability and detection performance ,18. The The structure of this paper is as follows: in the second section, the relevant knowledge and the proposed road crack detection architecture of the GM-ResNet are described in detail, and the third section undertakes experimental validation, presenting results from the proposed model and analogous comparative models. The fourth section summarizes the entire paper and prospects for follow-up work.This section starts with ResNet and introduces the concepts of the GAM and the proposition of a MFCNN as a replacement for the ResNet fully connected layer. Subsequently, the focal loss function is introduced. Finally, by amalgamating the GAM and MFCNN within the ResNet-34 framework, we present a novel GM-ResNet architecture tailored specifically for the task of road crack detection.The rapid evolution of deep learning technology has propelled the widespread utilization of a CNN as an exemplary network model. Its pervasive influence spans across a spectrum of domains, including facial recognition , intelliAssuming that By using the equation above, an identity map can be constructed to convert the function, The core concept of the attention mechanism resides in its ability to selectively emphasize relevant feature information while disregarding unessential inputs, thus pinpointing the pivotal feature representations within deep neural networks. This strategy effectively eliminates the detrimental impact of extraneous data on model training outcomes during the training process . Some scThe GAM is refined and enhanced, building upon the foundational concepts of the CBAM, which can not only decrease the reduction and dispersion of the feature information, but also simultaneously accentuate the amplification of global dimensional interaction features . For thiLet the input feature map be The GAM manifests through two independent sub-modules: the Channel Attention Module (CAM) and Spatial Attention Module (SAM). The CAM can extract global feature information by preserving the information of three dimensions. It first performs dimension conversion, and then inputs the dimension-converted feature map into a multilayer perceptron to enhance the cross-dimensional channel–spatial information dependence, where the reduction rate of multilayer perceptron is r. Finally, the dimension is reversed and the output, In SAM, two 7 × 7 convolutional layers are used successively to reduce and increase the number of channels to achieve the purpose of centralizing spatial feature information. This strategic operation can reduce the amount of calculation and fuse spatial information. Meantime, the conventional maximum pooling layer is removed from the SAM to further preserve the spatially useful information feature map. Finally, the output, In the formula, Conv denotes the convolution operation with a convolution kernel size of seven.Recognizing the fact that the fully connected layer in the ResNet model often fails to achieve satisfactory results in processing complex image tasks, we have innovatively substituted this layer with an MFCNN composed of multiple linear, batch normalization and activation function layers. The nonlinear expression ability of the model has been greatly improved by embracing a more profound network architecture and harnessing the potency of the leaky ReLU nonlinearity. Consequently, it can fit complex data distributions and patterns better. Furthermore, the infusion of multiple intermediate layers imparts the model with the capacity to glean multifaceted beneficial features across various levels and strengthen its feature extraction and representation capabilities. The batch normalization layer also helps to prevent overfitting during model training, increasing the crack detection effect and generalization ability. The specific calculation rules for an MFCNN are as follows:y are 1 and −1, representing the foreground and background, respectively.It is widely acknowledged that within a dataset containing crack images, the number of normal images (negative samples without cracks) will be much more than that of abnormal images (positive samples with cracks), and the proportion of cracks in the same image is often very small. At this time, the negative sample portion of the image without cracks and the image without cracks will become the majority category, while the positive sample portion of the image with cracks and the image with cracks will be the minority category, which leads to the occurrence of class imbalance. The traditional cross-entropy loss function has proven its efficacy in image processing tasks. However, in situations characterized by such class imbalances, employing the standard cross-entropy loss may inadvertently skew model predictions towards the majority categories while neglecting the minority categories, which may limit the performance of the model and result in unsatisfactory detection performance. Therefore, we use the focal loss function for model training to alleviate the adverse effects of category imbalance. The focal loss function incorporates adjustable weighting factors, which enable the model to effectively adjust the weight of easily classified samples (non-cracked images) during the training process and reduce its contribution to the loss by reducing the weight. Conversely, increasing the weight of difficult-to-classify image samples (images with cracks) makes the model pay more attention to such image samples, which helps to enhance the generalization learning ability of the model for difficult-to-classify image samples. The following is the calculation rule of the focal loss function:In the final analysis, the focal loss function can help the model better handle the troublesome question of uneven positive and negative image samples and imbalanced image samples that are difficult or easy to classify by introducing two regulatory factors, (1)The crack image dataset containing cracks with images and without images is divided into training and testing datasets according to a certain ratio.(2)The training datasets are input into the proposed GM-ResNet for model training, and the loss function of the training process adopts the focal loss function.(3)If the current training epoch is greater than the preset maximum epoch, the resultant trained model is subsequently extracted and utilized for testing purposes with the testing datasets. Alternatively, if the current training epoch is below the maximum threshold, the process returns to step (2) for continued training iterations.(4)The detection results of the final crack image testing datasets are output.Based on the work above, in order to better realize crack detection and boost the detection performance of the model, we selected ResNet-34 as the backbone network and introduced the GAM into the ResNet-34 model. This pivotal inclusion enables comprehensive global cross-dimensional interactions across both the channel and spatial dimensions and ensures that the model can capture prominent three-dimensional features of the channel, such as its spatial height and width. This synergistic augmentation bolsters the model’s feature extraction potency significantly, thereby elevating the overall crack detection efficacy. Subsequently, we engineered an enhancement by substituting the conventional fully connected layer in ResNet-34 with the meticulously described MFCNN module outlined in This paper uses the Concrete Structure Spalling and Cracking (CSSC) dataset proposed by Li et al.  to rigorWe know that the selection of optimal model parameters bears a significant influence on the ultimate training outcomes of the deep learning model. Therefore, after multiple experimental tests in this paper, we chose the stochastic gradient descent (SGD) with a momentum factor of 0.9 as the model optimizer and set the batch size to 64 and the weight attenuation to 0.001. The training regimen extends across 10 epochs, with the initial learning rate set to 0.01. In order to modulate the learning rate dynamically, the StepLR scheduler was introduced to decrease the learning rate to one tenth of the original every five epochs so as to avoid optimization bottlenecks in the training process that could lead to poor training accuracy. (a)True positive class (TP): the model correctly determines the input positive category samples as positive category samples.(b)True negative class (TN): the model correctly determines the input negative category samples as negative category samples.(c)False positive class (FP): the model mistakenly determines the input negative category samples as positive category samples.(d)False negative class (FN): the model mistakenly determines the input positive category samples as negative category samples.After preparing the datasets and determining the parameters required for training the model, the training model can be started. The GM-ResNet is trained based on the pre-training model of the ImageNet large-scale image datasets. Training based on the weight of the pre-training model can speed up the training speed and make the training effect reach its optimum as soon as possible. As shown in Recall indicates the ratio of the number of true category samples correctly classified in the crack datasets to the whole quantity of all positive category samples. The recall formula is as follows:Precision represents the ratio of the true category samples correctly classified to the whole quantity of positive category samples. The precision formula is as follows:The F1-score is obtained by counting the harmonic average of the precision and recall, and its value range is . When the value of the F1-score is close to one, it indicates perfect precision and recall, and if the precision or recall is zero, the F1-score value is zero. The F1-score is calculated as follows:Therefore, the visualization curves of the three evaluation indicators of the GM-ResNet are shown in In order to fully evaluate the superior performance of the proposed model, we present a comprehensive comparative analysis involving five distinct architectures: AlexNet, DenseNet, SqueezeNet, VGG16 and ResNet-34, as the experimental comparison models of the proposed GM-ResNet model. The training optimizer and learning rate scheduler of all the models are consistent with the previous text. Similarly, the model is trained using pre-training weights based on the ImageNet dataset to ensure fairness in the comparative experiment. The cross-entropy loss function is used as the training loss function for these five models. By inputting the crack image dataset into the models given above for training, we generate In the comparative analysis, recall, precision and F1-score serve as supplemental assessment metrics, corroborating the performance of the six models. In order to better evaluate the training performance of the models, the average values of these three evaluation indicators were taken on each epoch, and the results are shown in In addition, since the values of TP, TN, FP and FN have already been calculated in previous work, it is natural to calculate the confusion matrix for model training through TP, TN, FP and FN and visualize the confusion matrix graph as another indicator for evaluating model performance. Serving as a synopsis of the classification prediction outcomes, the confusion matrix becomes instrumental in assessing the overall efficacy of the model training. Shadows in crack images are considered a special kind of interference, which can lead to some false detection during model detection. Shaded areas can be incorrectly classified as cracks, ultimately resulting in the degraded detection performance of the model and adversely affecting its detection accuracy. In order to verify the detection performance of the proposed model under shadow conditions, we add shadows to the dataset and compare the detection accuracy of the proposed model under shadow conditions and without shadow conditions compared with other models, as shown in Looking at In addition, in order to verify the impact of different modules of the GM-ResNet on the performance of the model, we supplemented the ablation experiment and conducted comparative tests without the GAM, without the MFCNN, without the GAM + MFCNN and the GM-ResNet, respectively. In order to enrich the test results, we also conducted the same test under the shadow condition, and the test results are shown in By observing the ablation experimental results, it can be seen that the different module parts of the GM-ResNet proposed in this paper have a certain impact on the performance improvement of the model. Because it is under shadow conditions, several modules of the GM-ResNet play a great role in resisting shadow interference. The effectiveness and superiority of the proposed crack detection model are further verified through the ablation experiment.Based on the analysis above, it can be seen that the GM-ResNet road crack detection model proposed in this paper has achieved commendable performance during the training process, and the comparative experiments with the other five models have further verified that this model has compelling superiority and viability. After the model training is completed, the test phase begins, and road crack classification and detection are carried out by inputting partitioned testing datasets into the pre-trained model. In a quest to validate the crack detection efficacy of our proposed model, the five models above were compared and classified for the same testing datasets. After the training of the GM-ResNet crack detection model proposed in this paper, the image test set containing cracks is preprocessed and divided into smaller images of fixed area size, which are, respectively, input into the training model for prediction. If cracks are detected in the area, it will draw a colored rectangle in the original image to identify the detected cracks so as to determine the crack location. Based on the above work, we further calculate the crack length and angle information through Canny edge detection. For example, the specific crack information on the last three columns of the crack images in Although the relevant crack information of the crack image can be calculated, it is still a rough estimate of the crack information in the final analysis, and the model can not get very accurate, specific data. Therefore, in our future follow-up work, we hope to develop a crack detection model with higher efficiency and better effect and enable it to accurately calculate a series of related crack information from the crack images.(1)By introducing the GAM attention module to improve the feature extraction ability of the model, this mechanism enables comprehensive three-channel feature extraction and cross-dimensional interaction between the channels and spatial dimensions, avoiding a decrease in the feature extraction ability caused by the omission of useful feature information.(2)Relying on the deep network architecture, batch normalization layer and activation function layer in the MFCNN module to enhance the nonlinear fitting ability of the model and avoid overfitting, the generalization performance of the model is effectively enhanced.(3)The focal loss function is introduced to train the model to overcome the category imbalance problem of road crack detection; thus, the model can better distinguish between positive and negative samples and difficult and easy classification samples and improve the final detection performance of the model.(4)The experimental verification of the concrete crack image datasets also shows that the proposed model is significantly superior to other comparative models in terms of accuracy, recall and precision evaluation indicators, as well as in the output effect of crack detection, and has achieved significant progress, demonstrating the effectiveness and applicability of the model in the field of crack detection.This paper proposes a road crack detection network framework based on a GM-ResNet, designed to significantly enhance detection accuracy. It is summarized as follows:Although the GM-ResNet road crack detection framework proposed in this paper has achieved excellent performance, its detection performance for complex cracks is not yet perfect. Therefore, in future work, we will develop a more suitable detection model framework for the goal of complex crack detection in order to achieve precise complex crack detection results.


One of the main uses of artificial intelligence in the field of orthodontics is automated cephalometric analysis. Aim of the present study was to evaluate whether developmental stages of a dentition, fixed orthodontic appliances or other dental appliances may affect detection of cephalometric landmarks.p < 0.05.For the purposes of this study a Convolutional Neural Network (CNN) for automated detection of cephalometric landmarks was developed. The model was trained on 430 cephalometric radiographs and its performance was then tested on 460 new radiographs. The accuracy of landmark detection in patients with permanent dentition was compared with that in patients with mixed dentition. Furthermore, the influence of fixed orthodontic appliances and orthodontic brackets and/or bands was investigated only in patients with permanent dentition. A t-test was performed to evaluate the mean radial errors (MREs) against the corresponding SDs for each landmark in the two categories, of which the significance was set at The study showed significant differences in the recognition accuracy of the Ap-Inferior point and the Is-Superior point between patients with permanent dentition and mixed dentition, and no significant differences in the recognition process between patients without fixed orthodontic appliances and patients with orthodontic brackets and/or bands and other fixed orthodontic appliances.The results indicated that growth structures and developmental stages of a dentition had an impact on the performance of the customized CNN model by dental cephalometric landmarks. Fixed orthodontic appliances such as brackets, bands, and other fixed orthodontic appliances, had no significant effect on the performance of the CNN model.The online version contains supplementary material available at 10.1186/s12903-023-02984-2. Cephalometric analysis involves identifying common landmarks, quantifying the various relationships between them, and diagnosing the correlations in a patient's craniofacial morphology. However, during the cephalometric tracing, sources of error or inter-observer variability may lead to low reproducibility of the observations –5. SinceIn particular, the null hypothesis that developmental stages of a dentition, fixed orthodontic appliances or other dental appliances do not affect the accuracy of a customized artificial model for automatic detection of cephalometric landmarks shall be tested. For these purposes, a CNN model with commonly used architecture  was deveThis retrospective diagnostic study was approved by the LMU Ethics Commitee (Ref. No 19–863). Cephalometric radiographs were obtained from the archives of the Department of Orthodontics and Dentofacial Orthopedics, University Hospital, LMU Munich. For this study, a Convolutional Neural Network (CNN) was developed for automatic recognition of cephalometric landmarks. The accuracy of landmark recognition in patients with permanent dentition was compared with that of patients with mixed dentition (both groups included radiographs without fixed orthodontic appliances). In addition, this study investigated the influence of fixed orthodontic appliances and orthodontic brackets and/or bands among patients with permanent dentition only. For reporting this study, the guidelines of the Checklist for Artificial Intelligence in Dentistry  and the 2. Out of 1151 images, 251 images were excluded applying the exclusion criteria, 430 were included in the training dataset and 460 images were used as the test dataset.The patient sample was intended to be as comprehensive as possible, therefore exclusion criteria were limited to craniofacial anomalies and to images of poor quality and/or incorrect positioning of the skull, which might affect landmark recognition. Images of growing and adult patients with or without fixed orthodontic appliances, dental restorations and osteosynthesis plates were included. The distribution of data by age, sex and ethnicity are shown in Table Cephalometric analysis included 16 key landmarks for the orthodontic diagnosis of the skeletal and dental anatomy. Since soft tissue cephalometric landmarks are rarely located in the proximity of developing tooth germs or fixed orthodontic appliances, they were not considered in the present study. The positions of 16 cephalometric reference points Table  were manThe training data set consisted of a total of 430 images including patients with both permanent dentition and mixed dentition, as well as radiographs with fixed orthodontic appliances, orthodontic brackets and/or bands, osteosynthesis plates, implants, dental prosthetic restorations and root canal treatments. The images were divided into training images (90%) and validation images (10%). The training images are used to adjust and optimize the model so that the CNN "learns" how to perform its task, while the validation images provide an objective evaluation of the model and its performance. Sets of input data were created which consisted of cephalometric radiographs and a corresponding pair of coordinates  indicating the exact location of each landmark.A total of 460 cephalometric radiographs were used as the test dataset. The performance of the developed CNN was tested on a versatile data consisting of images with various radiographic features  and anatomical structures of patients at different stages of growth. The data were divided into independent subgroups to investigate the impact of the distinctive characteristics Table . RadiogrFor each case, one lateral cephalogram without annotations of reference points and one with identified as well as validated cephalometric landmarks were manually exported from the database. These were anonymised, labelled as a pair, and stored in two folders. For each case, the X and Y coordinates of all marked cephalometric reference points were automatically exported from the annotated X-ray image using a custom Python script and stored in a text file (.txt) labelled to match the corresponding case.Subsequently, the text files were automatically filtered so that only a single pair of coordinates corresponding to a specific reference point was stored in a text file. Since the location of each reference point is distinct, a Python script was written for the extraction procedure for each of the 16 mentioned landmarks. Finally, the plain cephalograms and the text document (.txt) storing a pair of coordinates were used as input for the CNN models, dealing with each point independently.The evaluation of the accuracy of the CNN model was also performed automatically using a Python script. By this means, the trained model was accessed and applied for the detection of the specific cephalometric point. The absolute difference between the predicted point (the point identified by the CNN) and the referenced point (the point positioned by the examiner) was determined. The Results were then imported into an Excel file  where further statistical analysis was performed.A deep learning model, more specifically a CNN was constructed using the open-source deep learning frameworks Keras   and TensThe model had a commonly used CNN Architecture for image classification , with so−4, batch size at 32 and as the number of layers was not consistent for each landmark. A detailed summary of the model for each point is provided in Supplementary file https://doi.org/10.5282/ubm/data.359. The performance in this model was measured with a mean squared error (MSE) cost function that quantifies the error between the real coordinates(input) of the landmark and the predicted ones(output).The input for the training of the proposed network involved a lateral cephalogram from the training dataset and a corresponding file containing the location for the cephalometric landmarks as pairs of X and Y coordinates. Consequently, the output of the CNN was a predicted pair of X, Y coordinates indicating the position of the landmark. For example, the Training Set for the Sella Point included 430 unmarked cephalograms and 430 corresponding text files containing the position of the landmark written as a pair of X and Y coordinates. This data was processed by a convolutional layer, which detects specific features and patterns. After a feature is detected, the information is compressed and passed to the next layers of the network. This process, which is responsible for pattern recognition, is called filtering and the filters used are adjusted throughout the learning process to benefit the performance of the machine learning model. The number of filters varies for each layer, with the first layer having 30 filters, the second 60 filters continuing in ascending order for each additional layer. The learning rate was set to 10 In order to increase the capacity of the model, a non-linear activation function was applied after each convolutional layer Fig. . To avoiFollowing the stack of convolutional layers, a global average polling 2D layer was added. This layer reduced the dimensionality of the learned feature maps by averaging over the special dimensions of the output and yielding a fixed-size vector representation, which can be processed with standard fully-connected layers.Note, that the described composite neural architecture is comparatively large. A major problem in large neural networks is overfitting, that is, the danger of overspecialization to the training set and the resulting limited generalizability to new (upcoming) data. Regularization is a common technique to prevent overfitting and thus poor generalization performance of deep neural networks  To regulAdditionally, to optimize the performance of the model, each point was considered separately, and a modified CNN was created for each landmark. This made it possible to adapt the number of convolutional layers, filters, dropout layers, and activation functions for each landmark depending on the complexity and variety of the features, resulting in a different neural expert for each landmark. This study relied on a standard CNN architecture , which sIn order to quantify the utility of the model, the absolute difference between the predicted point (the point identified by the CNN) and the referenced point (the point positioned by the examiner) was determined along the X-axis , which indicates the percentage of correctly detected reference points in different precision ranges: It should be considered that the deviation of the distance error along a certain axis has greater importance for some points. For example, the accuracy of the B-Point along the X-axis is more significant as it marks the position of the mandible in the sagittal plane. Hence, the distribution of errors in the horizontal and vertical planes were considered separately.Mean radial errors and standard deviations of the 16 used orthodontic cephalometric reference points were collected in an excel file . These numbers were categorized in two main comparing groups:INo fixed orthodontic appliances—Permanent dentition XY – Error (Group I/1) versus No Fixed orthodontic appliances—Mixed dentition XY – Error (Group I/2)IINo fixed orthodontic appliances—Permanent dentition XY – Error (Group I/1) versus Orthodontic brackets and/or bands and other fixed orthodontic appliances—Permanent dentition XY – Error (Group II)p-value < 0.05.A t-test was applied to compare MREs with their corresponding SDs for each landmark in their two categories, to determine whether the means of these two groups are equal to each other. For this purpose, a t-test was run for both abovementioned categories Table , separatThe results for the different groups are presented in Table The descriptive statistics indicating the mean error in the X-axis and Y-axis, the mean radial error and standard deviation, and the SDR  for all studied groups of patients are shown in Table The null hypothesis that developmental stages of a dentition, fixed orthodontic appliances or other dental appliances do not affect the accuracy of a customized artificial model for automatic detection of cephalometric landmarks was partially confirmed. The results of this study indicated that fixed orthodontic appliances had no significant impact on the recognition of cephalometric landmarks. However, growth structures such as tooth germs in the mixed dentition affected the performance of the studied model.p < 0.05) observed in the MREs of Ap-Inferior Point and Is-Superior Point. A recent study also reported a lower accuracy rate of the detection of the root apices [Images of patients with permanent dentition showed showed homogeneous anatomical patterns in the areas of the landmarks to be placed. In contrast, patients with mixed dentition were associated with complex growth structures, varying bone density and uniquely positioned permanent tooth germs. Consequently, the recognition process showed better accuracy for images of patients with permanent dentition, while the overall performance of the model was lower for cases with mixed dentition Table . As the t apices , howeverIn order to eliminate the complexity of growth structures, the influence of fixed orthodontic appliances on the model’s performance was studied only in patients with permanent dentition. Cephalometric radiographs with fixed orthodontic appliances are usually obtained at a later stage of treatment when initial objectives such as crowding, eruption problems, impacted teeth, and occlusal relationship problems have been resolved. Since at this stage of orthodontic treatment the teeth are usually well aligned, the overall detection of cephalometric points is less affected by double contours and superimpositions, but may be affected by metal artifacts. In this study, the overall detection of cephalometric points was more accurate for images of patients with fixed orthodontic appliances, and there was no significant difference in accuracy between cephalometric radiographs of patients with orthodontic brackets and/or bands and other fixed orthodontic appliances and cephalometric radiographs of patients without fixed orthodontic appliances. It should be noted that common fixed orthodontic appliances are made of stainless steel or other alloys and therefore have a different radiographic density than skeletal structures. A similar pattern of results may be seen in of radiographs with other factors associated with comparable density that may affect the performance of the framework, such as artifacts, osteosynthesis plates, implants, prosthetic restorations, and root canal fillings. Nevertheless, the present study did not investigate these aspects due to the limited study data.The distribution of errors in the horizontal and vertical planes was considered independently of each other. By means of a common cephalometric appraisal, the anteroposterior or vertical position of the maxilla and mandible and their relationships to the cranial base and dental structures are evaluated. For this purpose, the image was considered as a coordinate system with its two axes: X and Y. Transferred to the lateral cephalogram, these mark the sagittal and vertical planes respectively. The results for cephalometric points marking important anteroposterior correlations, such as A point and B point, showed overall a smaller distance error on the X axis than on the Y axis Table . It is aDifferences in landmark recognition in the X-axis or Y-axis can be explained by the fact that each landmark is located at a distinct anatomical site that is more accurate to locate in either the vertical or horizontal direction . EspeciaInvestigating the distinct characteristics and the exact position of the dentoskeletal landmarks is essential for the quality of the cephalometric appraisal. Therefore, the focus was set on developing an independent CNN type suitable for the unique characteristics of each reference point. This approach eliminated the expected decline in accuracy with increasing number of detection targets described in a previous study . The numAlthough the performance of the developed CNN was tested on a versatile dataset consisting of images with a variety of radiological features, this study employed a relatively small dataset for images, particularly from Group III ). Indeed, the challenge of limited training data in the health sector was also recognized in a recent review on deep learning . Hence, Another limitation of this study is the annotation procedure used, as it is prone to error with regard to the examiner. In the absence of a gold standard, constructing a reliable reference test capable of reducing bias in the dataset remains a challenge .Finally, since the reference points are used in a further step of the cephalometric analysis to perform angular measurements , a potenThe radiographic appearance of fixed orthodontic appliances such as brackets, bands, and other fixed orthodontic appliances on a lateral cephalometric radiograph did not significantly influence the performance of the model. Complex growth structures may affect the recognition accuracy of dental landmarks, thus detected references should be verified in growing patients and in the mixed dentition.Additional file 1.