Inkjet printing technology is expected to enhance printed display mass production technology in the future. Nozzle-array printheads form the basis for printed display mass production applications. However, jet instability caused by air bubble entrapment and nozzle wettability changes during the printing process is a major challenge in the application of this technology. To adapt to possible nozzle abnormalities, a high-adaptability nozzle-array printing system based on a set covering printing planning (SCPP) model for printed display manufacturing is designed in this study. The study consists of two parts. First, a printing system based on multistep visual inspection and closed-loop feedback is proposed to accurately detect and screen abnormal nozzle positions. Notably, the inkjet printing system can identify nozzles with abnormal ejection characteristics and ensure that the remaining nozzles work accurately and stably. Then, an SCPP model is established for display pixel printing planning by using the remaining normal nozzles on the nozzle-array printhead. This model can output the most efficient printing path and nozzle printing action and can adapt to any pixel pattern, nozzle type, and abnormal nozzle distribution. The system and technology are highly adaptable and scalable for fabricating large-area printed display devices. Figure 4 of an organic light-emitting diode (OLED) device that can be printed. The flow of the printing manufacturing process for this printed display is shown in Fig. 5 (HIL), hole transport layer6 (HTL), emission layer7 (EML), and thin film encapsulation8 (TFE) layer of OLEDs have been prepared by printing.Inkjet printing is a technology that can directly deposit a material solution on a substrate to form patterns at room temperature. This approach is considered a next-generation manufacturing method for printed display fabrication due to its low cost, high manufacturing efficiency, and advantages in manufacturing large-area flexible panels9. However, for inkjet printing to transition from research and development (R&D) to mass production, one of the greatest challenges in nozzle-array printhead applications is the instability of nozzle ejection, as shown in Fig. 10, which ultimately affect the quality of the printed display, as shown in Fig. 11. The sources of these problems are very complex and difficult to directly identify. Thus, printed display nozzle-array printing systems must be highly adaptable to possible abnormal nozzle ejection conditions.One of the key technologies for printed display mass production is the use of nozzle-array printheads, which are usually composed of one or more rows of multiple nozzles. The nozzles in the same row are arranged at equal intervals, and the nozzles in different rows are arranged in a staggered pattern to improve the printing density. Most production printer systems use nozzle-array printheads, which have thousands of nozzles, to improve the efficiency of inkjet printing12. For droplet inspection and printhead adjustment, conventional printing systems reduce the appearance of abnormal nozzles through nozzle hydrophobic treatment, ink formulation optimization, and waveform adjustment of the printhead. Han et al.13 used a selective hydrophobic coating of the inkjet head plate to enhance the uniformity of printing. Huang et al.14 analyzed the effects of metallic nanoparticle sizes in the ink formulation through droplet impact behaviors and determined that this approach could be used to optimize ink formulation. Jiang et al.15 proposed a computational fluid dynamics model to investigate the mechanism of the printing process; the model could guide the quick determination of operation parameters for the desired printing resolution when given a new ink. Cao et al.16 controlled the droplet volume deviation within 5% through waveform adjustment according to the droplet observation results from a nozzle-array inkjet printing system and used this printing system to print an area of 40 × 40 mm OLEDs with high uniformity. Yun et al.17 added a waveform optimization algorithm to a nozzle-array printing system and controlled the droplet velocity deviation within 6% through feedback adjustment based on measurements obtained with a camera. Then, the inkjet printing system used the adjusted normal nozzle information to plan the movement and jetting action of the printing process through a printing planning algorithm and finally achieved OLED printing. The commonly used printing planning methods include raster printing planning algorithms and vector printing planning algorithms. Lin et al.18 proposed a novel printing algorithm based on printhead rotation and an interlaced printing method for OLED display fabrication. Chang et al.19 developed an interlace rotation algorithm that can plan the printing path and nozzle ejection action by rotating the printhead angle, and the results showed that the printing resolution can be continuously adjusted from 100 dots per inch (DPI) to 5080 DPI. Phung et al.20 designed an encoder processing unit and a vector printing algorithm to plan X–Y printing motions and nozzle ejection actions by producing droplets at an equally spaced distance to prevent a nonuniform linewidth near the endpoints where line printing starts and ends. Kim et al.21 presented an algorithm to redefine the printed image resolution according to the pixel pitch of OLEDs and then planned the printing process according to a bitmap pattern.To solve this problem, the traditional printing system mainly includes the following three steps. (1) Droplet inspection and identification of abnormal nozzles; (2) printhead adjustment until all nozzles are adjusted to the normal ejection state; and (3) rotating the nozzle-array printhead to match the nozzle pitch and the pixel pitch and using the printing planning algorithm to obtain the printing path and nozzle ejection action22; therefore, a droplet volume deviation within 5% for all nozzles is insufficient. It is difficult to adjust all nozzles to the normal state through waveform adjustment and ink formulation optimization22. Essentially, an R&D-type operating condition is required for all nozzles to provide normal and stable jets, and this condition is not suitable for high-throughput mass production. Therefore, a display pixel printing system that can be adapted to any jetting conditions is worth studying.However, the above printing systems highly depend on the accuracy and stability of all the nozzle ejection states. These printing systems cannot adapt to printing conditions with abnormal nozzles. If any nozzle undergoes abnormal ejection, it may cause printing defects for the printed display panels. Moreover, mura-free printed display printing results require that the total thickness for all pixels is uniform; for example, the thickness variation of each OLED layer must be ≤  ± 0.6% of the mean thicknessInspired by previous nozzle-array inkjet printing systems, this paper presents a display pixel nozzle-array inkjet printing system based on a set covering printing planning (SCPP) model, which can stably and accurately print any type of pixel pattern when abnormal printhead nozzles are identified and closed. The proposed system consists of two parts. First, a nozzle-array inkjet printing system based on multistep visual inspection is designed to realize full-cycle monitoring and feedback control from droplet ejection to deposition for all nozzles on the printhead and screen abnormal nozzles based on the detection results. Second, an SCPP model for printed display fabrication is proposed to plan the printing path and nozzle ejection actions according to the remaining normal nozzle position and pixel pattern. The issue of nozzle ejection instability is resolved with the two components of the proposed system. The highly adaptable nozzle-array inkjet printing system proposed in this study can satisfy the requirements of commercialization and large-scale printed display fabrication.Printhead and substrate position inspection and calibration The printhead installation position is determined with an up-looking camera, and the substrate position is determined with a down-looking camera. If the installation position error exceeds the relevant limit, the printhead and substrate positions are calibrated. After calibration, the coordinate sets for nozzles and the pixels are established in the same coordinate system as used by the printing system.Abnormal nozzle screening First, a droplet-watching camera is used to measure the volume, velocity, and angle of droplets ejected from each nozzle. Then, the printhead is used to perform trial printing on the blank position of the substrate, and the down-looking camera is used to measure the error between the actual deposition positions of droplets and the designed positions. According to the measurement results, the printhead driving waveform is adjusted, and printing position compensation is performed to enhance the nozzle ejection results. The serial numbers of abnormal nozzles with droplet parameters that exceed the relevant limits are recorded.Model planning According to the recorded abnormal nozzle situation and the printed display pattern requirements, a proposed SCPP model is used to plan the printing process. The output of the model includes the printing motion route, the printhead ejection action, and the mapping between nozzles and pixels. The first two outputs are used to perform substrate printing. The third output is used to find the abnormal nozzle position that causes defects when the defect is found after printing.Panel printing and result inspection According to the planned printing route and nozzle ejection actions, a nozzle-array printhead is used to complete the printing of all pixels on the substrate. After the printing is completed, the printing results are inspected with an automated optical inspection (AOI) camera. According to the positions of the inspected printing defects and the mapping between nozzles and pixels, abnormal nozzles are rescreened and closed in the next printing.The display pixel printing process spans from ejecting droplets from the nozzles of the printhead to deposition in pixels on a substrate. The main objects involved in this process are printheads, droplets, and substrates. The basis of the accurate and stable printing plan includes two points: (1) to determine the coordinates of the nozzle and print pattern and (2) accurately identify the position of abnormal nozzles. Therefore, multistep detection and feedback adjustment must be accurately performed for the printhead, droplets, and substrates. The printing system contains functions such as printhead and substrate inspection, abnormal nozzle screening, printing planning, panel printing, and printing result inspection. The proposed printed display nozzle-array inkjet printing system is divided into four parts according to these functions, as shown in Fig. Based on the design for the display pixel printing process in Fig. 23; it includes two droplet-watching cameras at different angles on the same horizontal plane, synchronized triggering stroboscopic lights, and multiple image processing algorithms to obtain high-quality three-dimensional images of high-speed flying droplets and accurately measure their volume, speed, and angle. The AOI module contains multiple sets of industrial cameras and coaxial light sources and an automatic recognition algorithm for printing defects. This module can quickly collect images of the printing results and obtain the type and location information for printing defects through image processing. The motion module is composed of a multiaxis motion platform, in which the Y-axis motion platform moves the printhead modules, the substrate inspection module, and the AOI module to the working position; the X-axis motion platform transports the droplet measurement module, printhead inspection module, and adsorption platform on which the substrate is placed; the Z-axis motion platform lifts the printhead module and the AOI module; and the θ-axis motion platform rotates the substrate adsorption platform.The printing module includes the printhead, the printing control system, and the ink supply system. The ink supply system guides the ink through the hose and pump to achieve a stable flow from the ink cartridge to the nozzles and ensures that the ink pressure is within a small steady-state error range. The printing control system receives the printhead ejection signal sent from the host computer to achieve ink jetting. The printhead and substrate inspection modules include an up-looking camera and a down-looking camera, respectively. Through these two cameras, the positions of the printhead and the substrate can be detected. According to the position coordinates of the two cameras in the printing system, the position coordinates of all nozzles on the printhead and all pixels on the substrate are established in the coordinate system of the printing system. The droplet measurement module uses a self-developed accurate stereovision-based droplet measurement systemBased on the thickness and uniformity requirements of the printed display functional layer and the manufacturing efficiency of the printing system, the performance indexes of this printing system are as follows: (1) printing speed ≥ 100 mm/s; (2) droplet deposition position error ≤  ± 10 μm; and (3) mura-free printing result. Therefore, each step in the proposed process requires closed-loop detection and feedback adjustment to improve the accuracy and stability of the printing system and to provide accurate screening results for abnormal nozzle positions on the printhead for the next step of planning and printing in the process flow.The closed loops in the printing process flow are designed as shown in Fig. The specific actions and functions of the closed-loop printing process are as follows. Inner closed-loop 1: As shown in part A of Fig. To create a printing system with high adaptability, based on obtaining abnormal nozzle positions on the printhead through a multistep closed loop, the remaining normal nozzles need to be used to complete the printing of all pixels on the substrate. Therefore, a printing planning model is established to plan the movement path of the printhead and the nozzle ejection action so that all pixels can be printed completely in the shortest possible time.24. When all the pixels in the same column in the X direction have been printed, one-pass printing is completed. Then, the substrate returns to its original position, and the printhead moves a certain distance in the Y direction so that the normal nozzles are aligned with other unprinted pixels to start the next one-pass printing process. This process is repeated until all the pixels are printed on the substrate.The printing process is as follows. After the nozzle moves to the starting point of printing, the substrate starts to move in the X direction. When the printable region of the pixel on the substrate passes under a normal nozzle, droplets are ejected from the nozzle and deposited to form pixels. According to our other research results, the printable region is the largest area where a droplet can finally flow to form a pixel after it is deposited on a substrateL and m are the nozzle pitch and the number of nozzles on the printhead, respectively. P and n are the pitch and number of pixel columns in the Y direction of the substrate, respectively. S is the size of pixels in the Y direction, and e is the droplet deposition error in the Y direction. The S and e together define the Y-direction printing allowable range of the pixel. When the normal nozzle position is within the range, the droplets ejected by this nozzle can be deposited into the pixel. Otherwise, the droplet may deposit outside the pixel and cause printing defects.Therefore, the printing planning problem is defined as follows. Figure According to the pixel printing process, the process is composed of multiple printing passes. The normal nozzles are aligned with different pixels during each pass, and as the substrate moves in the X direction during a pass, all pixels in the column where the aligned pixels are located are printed. Thus, when the position of the printhead for one pass is determined, according to the alignment of the normal nozzle and the positions of unprinted pixels, the indexes of the nozzles that need to be used in this pass can be obtained. Therefore, the printhead position printed per pass is the decision variable of the print planning problem. Moreover, the challenge to be solved is to expect substrate printing in the presence of abnormal nozzles, so it is a strong constraint for the model that the substrate printing can be completed normally regardless of the number of abnormal nozzles. For optimization goals, when the printing speed and the number of pixels in the X direction are determined, the printing time required for one pass is known. The fewer the number of printing passes, the less time it takes to complete the printing job. Therefore, the optimization goal of the printing planning problem is to minimize the number of print passes.In summary, the printing planning problem can be defined as follows based on the printing process. Consider a printing system that includes one or more printheads with determined relative positions of nozzles and a substrate with a certain pixel pattern. The printhead starts printing the first pass from a certain position on the substrate. Each normal nozzle can be assigned to jet or not when it is within the pixel range, and all abnormal nozzles are not ejected under any circumstances. When there is a nozzle in the pixel range, all pixels in the same column on the substrate are printed along a linear path in the X direction in this pass. Then, the printhead moves a certain distance in the Y direction to print other columns of pixels and performs the next pass. This process is repeated until all pixels are printed. The object of the printing planning problem is to minimize the number of printing passes required to print all pixels.The problem parameters for the display pixel printing planning problem are shown in Table entclass1pt{minimaThe key to the establishment of model constraints is the calculation of According to the number of pixel columns, several Y-direction stop points for the printhead are set in the printed pixel area of the substrate.The alignment of normal nozzles and pixels on each stop point is expressed by entclass1pt{minimaSince Eq.  usually The principle of the SCPP model is as follows:Therefore, in a mathematical sense, after the input variables are determined and the SCPP model for this printing is established, if the model has a solution, it means that the normal nozzle on the nozzle can achieve the printing of all pixels on the substrate, and the output objective function According to the objective function 26, which means that the printing scheme with the shortest printing time can be obtained through the SCPP model to complete all the pixel printing.In addition, since the SCPP model takes normal nozzle coordinates, pixel coordinates, and error bounds of the pixel as input variables, the model can be applied to different printhead types, different abnormal nozzle numbers and positions, and different printed display pixel patterns by changing the values of these variables. Furthermore, the SCPP model is an integer programming model, so the solution obtained by the exact algorithm can be guaranteed to be the global optimal solution27, column generation method28, heuristic algorithm30, and some commercial optimizers32, have been applied to solve this type of problem. Due to the large printing area, many stop positions need to be set for printing planning, which leads to a large problem scale in actual application cases. Therefore, the popular general-purpose 0–1 integer programming optimizer Gurobi is selected to solve the model, and it is especially suitable for solving large-scale linear problems31.To solve the display pixel printing planning problem, a solution program was developed. The modeling of the display pixel printing planning problem is based on the classic set covering model, which is a classic nondeterministic polynomial hard (NP-hard) problem in integer programming. A variety of algorithms, such as the interior point methodTo verify the adaptability of the proposed model and solution program in different printing situations, a variety of printing cases are used for testing. The default test parameters listed in Table The results for different cases are shown in Fig. The influence of printing parameters on the SCPP model is numerically explored. As shown in Fig. The numerical results verify the adaptability of the SCPP model to different types of nozzles, different PPI substrates, and different proportions of abnormal nozzles on the printhead. Moreover, it also shows that the SCPP model can be used to simulate the printing results of the printing condition, and the system can choose the appropriate nozzle type and determine the abnormal nozzle screening boundary according to the type of substrate.All experiments were programmed in MATLAB 2019b and implemented on a personal computer .To evaluate the practical printing effect and application performance of the proposed method, self-developed printing equipment using the display pixel nozzle-array inkjet printing system and SCPP model is designed and built. As shown in Fig. To ensure the reliability of the printing process, the printing equipment improves the stability of the printing system through the design of the ink supply system, the printhead waveform optimization, and the maintenance operation in the printing gap. These designs and operations enable the equipment to maintain a normal working state for a long time. When the AOI camera does not find defects in the printing result, the equipment can periodically perform printhead detection and printing planning while printing the same substrate. When the AOI camera detects defects in the printing result, the printing system can trace the specific abnormal nozzle position through the mapping produced by the printing planning according to the position of the defects. Based on the current proportion of normal nozzles, the system can automatically choose to suspend printing to close the abnormal nozzles and rerun the printing planning, or stop printing for abnormal nozzle screening.The droplet deposition accuracy is an important index of the display pixel printing system. Because pixel size is determined by substrate, Eq.  shows thTo verify the performance of the proposed inkjet printing system and SCPP model, a comparative device printing experiment with and without the SCPP model was tested. The printing parameters of the comparative printing experiments are shown in Table 2 OLED device printed with the proposed printing system and planning model exhibits no printing defects. The numbers of printing passes are 50, 80, and 90, and the maximum printing time does not exceed 240 s, which meets the time requirements of the display printing processes. These findings verify the high adaptability of the proposed method to different types of pixel substrates.To verify the adaptability of the proposed system for printing different PPI substrates, experiments were carried out on ITO substrates with different pixel patterns using the proposed jet printing system with the SCPP model. The parameters of the substrate and printing process are shown in Table Figure 2, 85 PPI OLED device are achieved. The printing and lighting results are satisfactory, and the system can meet the current application requirements of the display printing and manufacturing industry. The technology is highly adaptable and scalable and can be used in the industrial-grade printing manufacturing of large-area printed display panels by increasing the number of printheads. This technology can also be applied in other sectors of printed electronics manufacturing.In this paper, a highly adaptable display pixel nozzle-array inkjet printing system based on an SCPP model is proposed. This system and planning model can overcome abnormal nozzle ejection issues and can improve the adaptability and scalability of printing technology applied in printed display industrial production. First, a display pixel nozzle-array inkjet printing system based on multistep visual inspection and a closed-loop feedback process is designed to monitor the droplet state and provide feedback during the whole process from preprinting to postprinting and accurately identify the positions of abnormal nozzles. Second, an SCPP model is proposed, which can be applied in printing planning for any abnormal nozzle position, any printing pattern, and any type of nozzle condition. The model uses the results of abnormal nozzle screening and the printing pattern as inputs and outputs of the multipass printing path and nozzle ejection actions. Based on these planning results, the system can realize high-precision, high-adaptability, and high-efficiency display pixel printing by using a nozzle-array printhead. Additionally, display pixel printing equipment is established according to the proposed system and model, and a series of experiments are conducted on the equipment. The experimental results show that the proposed display pixel printing system and printing planning model can achieve a droplet deposition accuracy of ≤  ± 5 μm. With this equipment, the printing of substrates with various pixel densities and the printing and lighting of a 400 cmThe OLED HIL, HTL, and EML ink material was purchased from the DuPont company.The NEJ-PR200 printing equipment was self-developed and built by Wuhan National Innovation Technology Optoelectronics Equipment Company. A QS256 printhead from FUJIFILM Dimatix company is chosen for printing. The printing equipment can realize high-precision visual positioning, accurate stereo-vision-based flying droplet measurement, adaptive control of process parameters, and online AOI defect automatic monitoring. Through these functions, the construction of the proposed printing system and the operation of the SCPP model were realized.The luminance and spectrum data of the OLED printed devices were measured by using a five-axis optical measurement instrument .


The literature reports some recipes used by old Cremonese luthiers as primers, mainly based on alkaline aqueous solutions and protein-based glues. In this manuscript, we analyze the impact of these treatments on the mechanical properties of the material. The combination of two pre-treatments and three sizes is considered on nine different plates. We compare the vibrational properties before and after the application and assess the effects of the different primers, also supported by finite element modeling. The main outcome is that the combination of particular treatments on the violin surface before varnishing leads to changes not only to the wood appearance, but also to its vibrational properties. Indeed pre-treatments, often considered negligible in terms of vibrational changes, enhance the penetration of the size into the wood structure and strengthen the impact of the latter on the final rigidity of the material along the longitudinal and radial directions.It is commonly believed that violins sound differently when finished. However, if the role of varnishes on the vibrational properties of these musical instruments is well-established, how the first components of the complete wood finish impact on the final result is still unclear. According to tradition, the priming process consists of two distinct stages, called  In this regard, classical Cremonese varnishes set the bar in aesthetic terms thanks to the transparent yet colorful effect that is still appreciated by musicians nowadays. Following the tradition, luthiers still pay great attention in refining their recipes to reach a similar trade-off between elegant appearance and desired sound radiation capability.Wood finish is one of the most fascinating and delicate processes in violin making. It is a common belief, in fact, that different aspects of the violin, from its durability to its visual appeal and perceived timbre, depend on how the maker designs the finish and applies it to the instrument. According to Koen Padding2. We will refer to these as pre-treatments. In a second step, the wood is prepared for varnish through the application of a protein-based compound that generally includes animal glues or caseinate. These glues, denser than the previous solutions, can seal the surface pores and avoid a potential soaking of the varnish. The process is known under different names: sizing, sealing, filling or grounding. In this manuscript, we will denote it as sizing. After these priming steps, the violin is ready to be varnished.Because of the important role that this step has in the uniqueness of a violin, makers still keep their own methods and discoveries secret. Nonetheless, some common practices can be outlined. The surface of the tonewood usually undergoes three main types of treatment during the finishing process. First, the plates are pre-treated with alkaline solutions to enhance wood durability and prevent biological attacks6 and musical instruments7, and its impact has been mainly observed in prevention and wood appearance9, i.e. leading to either a darkening or discolouring of the material. Among the physical and chemical pre-treatments available in the art and crafts literature, treating the wood in alkaline medium is one of the most reported. In particular, two main pre-treatments can be considered: on the one hand the ammonia fuming, which simulates the old common practice to place the violin on a smoldering dung pit10 and, on the other hand, a lye-based solution usually applied with a brush or a sponge.The use of primers in the past has been proved by the analytical investigation carried out on historical artworks12. Indeed, it is commonly accepted that varnish does not only contribute to the aesthetic appearance of violins, but it also plays an important role in the wood protection13 and sound production20. Conversely, very few works suggested possible chemical compositions for primers and provided a tentative explanation about their function21. Primers may produce a variation on the wood structure extended in depth to a relevant portion of its thickness. However, how deep a prescribed combination of pre-treatment and size can penetrate inside the tonewood and which changes the primer induces to the elastic properties of the treated material are still open questions that call for an in-depth study from the standpoint of vibrational analysis.To the best of our knowledge, research has focused the attention only on varnish, probably driven by the greater amount of information available on its chemical composition with respect to primers22. The consequent geometrical changes were taken into account inside the FE model. For a subset of the specimens, considerations extracted from simulations were supported with the actual evaluation of the penetration depths through SEM-EDX.By using a multi-disciplinary approach including vibration tests, Finite Element (FE) modeling and Scanning Electron Microscopy with Energy Dispersive X-ray (SEM-EDX) analysis, we studied the penetration of primers in the wood structure and their impact on its material properties. We first applied a selection of two alkaline pre-treatments and three protein-based sizes over one side of a set of Spruce plates, as traditionally done by luthiers which varnish only the outer surface of the violin. Vibration tests were first performed on untreated plates, and then repeated after both pre-treatment and size were applied. The data obtained in terms of modal frequencies and density allowed us to estimate the variation in the specific stiffness characterizing the treated plates with respect to the original ones. Subsequently, we employed FE analysis to fit the measurements, study the penetration depth of each primer and estimate the change in density in the portion of wood affected by treatments. Due to the small value chosen for the thickness of the specimens, and the different moisture exchange rate observed for the two faces and caused by one-sided applications, the so-called ‘flying wood’ or ‘cupping’ phenomenon occurredwhite, as in23. Figure Frequency Response Functions (FRFs) were collected on a pair of fixed points over the specimens Fig. a to studentclass1pt{minima29. Indeed, their correlation with the first modes of the instrument is found to be higher than others30. Moreover, luthiers usually focus on the Young’s modulus along the principal dimensions of the wedge as they can infer them empirically31 and they are used to define acoustic indicators for instruments (e.g. anisotropy ratio15).The impact of pre-treatments and sizing on wood stiffness was assessed by measuring the change in the longitudinal and radial Young’s moduli and in the density of the specimens. We focused on these material properties since they have been proved to be important for determining the vibroacoustic properties of complete instruments26. The relative change in the specific stiffness, i.e. the ratio between the measured Young’s modulus The Young’s moduli were estimated from the frequency of modes  and  using the Caldersmith’s formulaj after the application of each treatment t. Green markers are associated to values obtained after the pre-treatment  are characterized by an average decrease equal to − 6.9(±2.0)% in Changes in the specific stiffness become relevant only when plates are also sized. Indeed, 23, modifying the FE model. The model was defined assuming a change in the parameters of the treated side of the plate with respect to untreated wood, together with the introduction of the arching due to the cupping phenomenon. Figure In order to analyze the penetration of the treatments, we employed the FEM-based measurement fitting explained inWe simulated the vibrational behavior of the virtual plate varying the penetration depth With the new model, the optimization performed did not provide a unique set of optimal parameters for the treated wood section, since the cost function used was severely affected by local minima. Therefore, we analyzed the histogram of the penetration as obtained from the local minima of the cost function.The analysis revealed that a range of ‘preferential’ depths exists for each primer under study. Figure If we look at the penetration depth of the pre-treatments (dashed lines), a range of depths concentrated within 300–650 After the application of the sizing, these ranges shift to lower values. The histogram obtained for the KOH+AG, KOH+AGK and KOH+C primers  shows a slight shift of the region with the largest number of occurrences from 300–650 Plates where sizes were applied directly show small values of Similar considerations on the penetration depth of primers can be formulated by observing the results of the SEM-EDX transversal analysis on cross-sectional samples, which reveals the variation of the potassium (K) relative abundance as a function of depth.Figure It can be seen that when a section of the sample has been treated, the profile shows higher counts down to a certain depth. The profile becomes then comparable with the control for higher depths. We thus interpreted the change in the slope of the K counting profile as the limit of interaction between the portion of wood which probably underwent chemical modification or impregnation and the rest of the sample.The limits of interaction were thus estimated by fitting the decaying part of the profiles with a piece-wise linear (or bilinear) function. The results are shown with vertical lines in Fig. Figure We performed the analysis conducted on the distribution of the simulated penetration depths also on the other control parameters characterizing the material on top of the FE model. The resulting distributions revealed different material changes as a function of the primer.Figure By analyzing more in detail the variation induced after the sizing, it can be seen that pre-treatments induce a density increase which is greater than the variation encountered if plates are only sized. Figure The density changes obtained through FEM modeling are in line with the density values measured after each treatment stage. Indeed, an average variation of In this study, a multi-disciplinary approach consisting of vibrational measurements, FEM modeling and SEM-EDX investigation has been employed to characterize the impact of Cremonese traditional alkaline pre-treatments and sizing on the elasticity and vibration of Spruce tonewood, typically used for the production of soundboards in violin making. To the best of our knowledge, this is the first systematic study in this direction. Pre-treating the wood eases the penetration of sizes into the wood pores, inducing a variation in the material parameters, such as its density and stiffness, which is greater than the one observed for wood which underwent only the sizing stage. As a consequence, also the vibrational properties of the tonewood plate are more affected. These findings may be crucial for makers, so that the contribution of each stage of the finishing to the final sound is taken into account. Indeed, the impact of the first wood treatments has been always considered marginal with respect to the varnish, and less efforts were devoted to clarify this aspect from a quantitative point of view.34, to study whether the particle size or concentration can control these material changes. Moreover, focusing on one treatment combination and increasing the number of specimens used may lead to more informative and representative results.Vibration tests revealed a change in the first modal frequencies of the specimens, and thus in the specific stiffness along the most relevant directions. Although the application of the pre-treatment may not induce relevant modifications by per se, the combined effect with sizes leads to an enhanced stiffening of the whole plate, especially along the wood grain. The effect is more pronounced if kaolin particles are dispersed into the treatment: as an example, the mixture of animal glue and kaolin showed the largest variation in the specific stiffness in both longitudinal and radial directions of the plate. This evidence opens the door to further investigations for characterizing the variety of inorganic particles used in wood finishesFEM modelling and SEM-EDX analysis highlighted the different penetration of alkaline-based treatments, both before and after the sizing. Analyzing the pre-treatment only, 2.When the combination of pre-treatment and sizing is considered, an increase in the penetration depth can be observed if compared to the direct application of sizes on untreated plates, especially when KOH is applied. This result, together with a reduction of the density in the upper portion of pre-treated specimens, and thus of the equivalent density of specimens, may confirm the action of alkaline treatments as an artificial support in wood drying or agingPicea abies L., purchased at Rivolta Wood, Italy) with average density equal to 400 21, two types of alkaline medium were selected to emulate historical pre-treatments33: a 1 M aqueous solution at 21. During the first stage of the campaign, KOH was spread with a brush over one face of the plates, while p) stage have been accomplished. In the second stage (s), three different sizes have been applied with a brush: (i) a solution with 10% of rabbit glue dissolved in water ; (ii) a mixture of 10% rabbit glue enriched with 1% kaolin 36. As a result, nine different combinations of pre-treatments and sizes were studied  and an uniaxial accelerometer  were used to generate an impulsive excitation and measure the harmonic response. The excitation and the measurement points were placed along the axis of symmetry parallel to the plate length, at 45 mm from the edges  as depicted in Fig. 39. Details on the estimations performed starting from measured FRFs can be found in Supplementary Information.The Frequency Response Functions (FRFs) were measured by means of hammer impact testing. A structure made of wood and rubber bands was built to simulate free boundary conditions during the test. Four rubber bands were used for the suspension of the plates, letting the sample in vertical position and minimizing the contact surface32. The reference material used was Sitka Spruce, not a common material in European violin making tradition, but with mechanical properties known and similar to Spruce tonewood. A table resuming the distributions and the nominal values used for the sampling is provided in Supplementary Information  and, at a later stage, the section of the specimens affected by the treatments. In particular, the first step consisted in the assessment of the material parameters  and dry-polished with silicon carbide fine sandpapers (1200–8000 mesh). The analyses were performed on the specimens treated with potassium hydroxide and potassium caseinate. The semi-quantitative line profiling of K was acquired from the up-radial treated surface of the specimen (including around 100 Supplementary Information.


A large number of sensors work in the narrow bandpass circumstance. Meanwhile, some of them hold fine details merely along one and two dimensions. In order to efficiently simulate these sensors and devices, the one-step leapfrog hybrid implicit-explicit (HIE) algorithm with the complex envelope (CE) method and absorbing boundary condition is proposed in the narrow bandpass circumstance. To be more precise, absorbing boundary condition is implemented by the higher order convolutional perfectly matched layer (CPML) formulation to further enhance the absorption during the entire simulation. Numerical examples and their experiments are carried out to further illustrate the effectiveness of the proposed algorithm. The results show considerable agreement with the experiment and theory resolution. The relationship between the time step and mesh size can break the Courant–Friedrichs–Levy condition which indicates the physical size/selection mesh size. Such a condition indicates that the proposed algorithm behaviors are considerably accurate due to the rational choice in discretized mesh. It also shows decrement in simulation duration and memory consumption compared with the other algorithms. In addition, absorption performance can be improved by employing the proposed higher order CPML algorithm during the whole simulation. With the development of the electronic industry, sensors have become more important than ever before. Large numbers of sensors work in narrow bandpass circumstances or individual frequency ,2. AlthoWith the development of sensors, structures have become more complex than ever before. For the fine details simulation, mesh size must be fine enough to satisfy the calculation accuracy. Due to the existence of the Courant–Friedrichs–Levy (CFL) condition, the relationship between time step and mesh size is established . Thus, tBecause the computer cannot solve infinite computational domains, an adequate boundary condition must be employed at the boundaries . To simuIn order to simulate complex sensors and components whose fine details merely exist along one and two dimensions, several algorithms have been proposed which still have their own limitations. The original HIE procedures with PML (NPML) and CFS-PML are proposed in two dimensions which cannot be employed into the practical employment ,34. ReceIn order to efficiently simulate sensors with fine details in narrow bandpass circumstances, the one-step leapfrog HIE procedure is proposed with the CE method and higher order CPML formulation. The proposed algorithm takes advantages of leapfrog HIE procedure, CE method and higher order CPML formulation, in terms of the calculating of fine details along one and two dimensions, simulating bandpass problems and enhancing absorption. Numerical examples and experiments are carried out for the illustration of effectiveness. It can be concluded that results simulated by employing the proposed algorithm show considerable agreement with the experiments and theory resolution. Compared to the other algorithms, the proposed algorithm also shows significant improvement in accuracy, efficiency and absorption.Because massive components exist in Maxwell’s equations, assuming the fine details merely locate along x-direction, quations . AccordiIn order to update the equations, Equation (2) is substituted into Equation , one obtains:The First Time Step:The Second Time Step:According to the leapfrog HIE procedure, the first time step can be given by substituting the magnetic components into electric components, one obtains:n − 1, substituting magnetic components to electric components accordingly, one obtains:By rewriting the second time step to the previous step of According to the leapfrog scheme, by adding Equations (7) and (8) together into a single equation, after some manipulations, one obtains:The auxiliary variables can be given as, for example,(1)Explicitly update (2)Explicitly update (3)Implicitly update (4)Explicitly update auxiliary variables inside PML regions.According to the convexity of Maxwell’s equations, the other auxiliary variables and components can be solved by employing a similar approach. By employing the above-mentioned equations, leapfrog HIE procedure with CPML formulation and CE method can be implemented. The entire update procedure of the described is as follows:The block diagram of the entire update procedure is shown in As can be concluded from The effectiveness of the proposed scheme can be demonstrated through employing the numerical example and experiment including a filter for the sensors system with experiments and remote sensing problems with theory resolution. The computer for the code holds the parameter of i7-11700k and DDR4 128 GB. The program is implemented by Fortran under Visual Studio. The entire computation duration can be described as the following procedure: (1) Import geometry structures with STL files; (2) Set condition for calculation including time step, mesh size and more; (3) Start Calculation; (4) Export results for TXT file.Within microwave components, micro-strip components can be regarded one of the most powerful and widely spread. Among some of them, an extremely thin metal patch is located on the surface of the device. Thus, by employing the unconditionally stable algorithms and conventional explicit algorithms directly, uniform extremely fine mesh must be employed within the entire computational domain which results in significant increments of memory and duration. In order to alleviate such conditions, the HIE procedure is introduced for the fine details along one and two dimensions. Here, a micro-strip filter for sensors system is employed for the illustration. The sketch picture of the structure is shown as The entire filter structure is composed of substrate and patch. The dielectric substrate with the electrical parameter of unded in  are locaThe entire computational domain is shown in For comparison, parameters of CFS-PML regions are Waveforms obtained by explicit algorithms and implicit algorithms with CFLN = 1 are almost overlapped, as shown in The wave reflection at the boundaries of the domain also affects the accuracy of the calculation. The wave reflection can be reflected by the relative reflection error in the time domain which can be defined as:The relative reflection error obtained by different PML algorithms with CFLN = 1 is shown in It can be concluded from 11) and transmission loss (S21) are considered in the frequency domain. The photograph of the filter and its sketch size is shown in The above-mentioned discussion verifies the effectiveness of the algorithm in the time domain. In order to further testify the algorithm in the frequency domain, a filter for the sensors system is simulated and manufactured. The scattering parameter is one of the most important parameters in the evaluation. Here, return loss  The manufacturing process tolerance results in disagreement compared to the size in the simulation. The generation of shifting is caused by such a condition. Thus, the manufacturing process tolerance overcoming the technique becomes increasingly important; (2) The measurement of the filter system is a precision-type instrument, especially for the probe. The pollution of the devices also leads to shifting in the experiment; (3) Due to the existence of the complex and oblique patch, the conventional conformal method in the Yee’s grid becomes inaccurate. Conformal error causes occur at the oblique and complex part. Such conditions also result in the curves shifting. Thus, improvement on the conformal method for the HIE algorithm is also worth investigating.The performance of the proposed algorithm has been demonstrated by employing the above-mentioned numerical example and its corresponding experiment. Through results, its behaviors show considerable performance during the whole simulation. In order to further demonstrate the efficiency in much more complex problems, a metal sphere model is considered. The normal vector of the sphere model ranges from −180 degree to 180 degree which holds the most complexity among all of the structures. Because the metal sphere model in the sensing problem holds theory resolution, calculation accuracy can be demonstrated through the comparison. The entire computational domain holds dimensions of The accuracy of the algorithm can be reflected by the radar cross section (RCS) parameter in the frequency domain. It can be concluded from Due to the existence of a massive absorbing boundary condition, the choice of absorbing boundary condition according to different simulation problems has raised much attention. There are several widely-spread absorbing boundary conditions including perfectly matched layer and transparent boundary conditions (TBC) . It can In order to efficiently simulate devices for a sensors system in narrow bandpass circumstances, the one-step leapfrog HIE algorithm with the complex envelope method and higher order CPML formulation is proposed. Through the implementation, dielectric material with fine details along one and two dimensions can be solved. The numerical example and its experiment are carried out for the demonstration of effectiveness. It can be concluded from the results that the proposed algorithm shows its advantages in accuracy, efficiency, absorption and computational resources, both in the time domain and frequency domain. Meanwhile, the CFL condition can be broken in the proposed scheme which also indicates the efficiency improvement in the entire simulation. Compared with the previous work, it can maintain considerable performance with decrement of computational resources and simulation duration.In future work, the method to reduce memory consumption and simulation duration increment caused by matrix solution in the HIE procedure can be further investigated. Meanwhile, numerical dispersion increases with the enlargement of the time step, which affects the calculation accuracy significantly. The method to decrease numerical dispersion with a larger time step is worth investigating. Finally, conformal errors still exist in the complex and oblique structures. Improvements on the conformal method for the HIE algorithm is also worth investigating.


The international GNSS service (IGS) real-time service (RTS) provides orbit and clock corrections for the global navigation satellite system (GNSS) via the internet. RTS is widely used for real-time, precise positioning and its data is transmitted via the internet. Intermittent data loss can occur and cause positioning accuracy degradation. RTS data has a discontinuity when the issue of data (IOD) changes every two hours. If the signal loss occurs immediately after the IOD change, then the performance of the RTS prediction degrades significantly. We propose an adjustment method to make the RTS data across the IOD change, which makes it possible to use long RTS data for building a prediction model. The proposed adjustment method is combined with a long-short-term memory (LSTM) network to improve long-period prediction accuracy. Experiments with GPS and RTS were performed to evaluate the RTS orbit prediction accuracy. The LSTM with the IOD adjustment outperforms other polynomial prediction methods, and the positioning accuracy with the predicted RTS orbit correction shows a significant improvement. While precise satellite orbit and clock products are necessary to perform the precise point positioning (PPP) technique, any errors that exist in the global navigation satellite system (GNSS) absolute positioning must also be modeled. The international GNSS Service (IGS) real-time service (RTS) correction or IGS ultra-rapid product is mainly used to correct the orbit and clock error components of navigation satellites for real-time applications. However, the IGS ultra-rapid product makes use of predicted information, and the error of the clock increases over time and does not meet the requirements of PPP ,2.The RTS correction provides information about the navigation satellite orbit and clock correction over the internet, and the user can obtain accurate orbit and clock information by applying the RTS correction to the navigation satellite orbit and clock from the navigation message. However, RTS availability is not 100% due to issues such as caster failure or user’s internet connection, and it may cause a problem with applying RTS orbit and clock corrections in real-time. If RTS correction information is not available during the real-time PPP, the positioning error may increase rapidly. Therefore, it is necessary to predict the RTS correction during the signal interruption period to prevent the error from increasing.Several studies have been carried out to predict the RTS correction. First, the prediction has been proposed using a polynomial model in a short time interval . Some stInformation about the issue of data (IOD) is included in the navigation message transmitted by GNSS and RTS. IOD means the issue number for the transmitted message. IOD includes IOD ephemeris (IODE), which is an IOD value for the orbit, and IOD Clock (IODC), which is a value for the clock. The IOD changes if the navigation message is changed every 2 h. The RTS correction is not an absolute value, but an additional value to the broadcast ephemeris and clock in the navigation message. When the broadcast ephemeris is changed every two hours, the RTS correction changes according to the new broadcast ephemeris. When the IOD changes, both the RTS correction and the satellite orbit calculated from the navigation message change discontinuously. A special situation is when the RTS signal outage or loss occurs immediately after the IOD change. In this situation, it becomes very difficult to predict the RTS correction. This is because the prediction method does not have enough fitting data for the prediction. For example, if the IOD changes at 2:00 P.M. and the RTS signal loss occurs at 2:03 P.M., we have only three minutes of fitting data for the RTS prediction process. To solve this problem, the authors proposed using the orbit change value to eliminate the RTS correction discontinuity at the IOD change. Although this special situation is rare, e.g., a few minutes during two-hour constant broadcast ephemeris transmission (between consecutive IOD changes), it is important for PPP that requires continuous orbit and clock corrections to resolve ambiguities. The usefulness of the concept was demonstrated using a polynomial prediction model at the time of IOD change . The polIn this study, we propose a prediction method for RTS orbit correction using the LSTM network by removing the discontinuity of RTS correction when signal interruption occurs after the IOD change. Since RTS clock corrections have white noise characteristics , the perRTS correction is streamed over the internet in the radio technical commission for maritime services (RTCM) state space representation (SSR) format. The internet protocol uses networked transport of RTCM via internet protocol (NTRIP) ,14,15,16RTS corrections are produced by the IGS real-time workgroup (RTWG), as shown in IGS RTS provides several products for orbit and clock correction, such as Since RTS orbit correction is provided in radial, along-track, and cross-track (RAC) coordinate systems, transformation to the Earth-centered Earth-fixed (ECEF) coordinate system is essential for applying to navigation message via the transformation equation asTo analyze the changes in the RTS corrections at the time of the IOD change, the mean change of the RTS corrections at each IOD change was calculated. SSRA03IGS1 (IGS03) data streamed from 28 September to 2 October 2022, was analyzed. At 12 IOD changes per day, a total of 58 changes were considered, excluding the start and end of the data.When the IOD changes, a discontinuity occurs in the orbit generated from the navigation message and the RTS correction. When predicting RTS correction, it is difficult to predict with discontinuous data, so prediction should only be performed with the data received after the IOD change. However, due to the small amount of received data, a large prediction error occurs. If the difference in orbit calculated by the navigation message before and after the IOD change is applied to the RTS correction using this method, then it is possible to predict the RTS correction after the IOD change and the RTS correction becomes continuous.When learning data over a long period of time with a recurrent neural network (RNN), the weights are not updated correctly due to the gradient loss that occurs during the back-propagation process . To solvAs shown in en state . There aen state . The outproblems . FinallyUsing the output gate, we can only export the part we want to send as output.The LSTM network implemented in this study is trained to predict the future with one-step forward. The LSTM prediction experiments were performed on the assumption that the RTS correction outage occurs immediately after the IOD change. The RTS was assumed to be interrupted for 3000 s from 180 s after the IOD change, and since IOD changes every 2 h, a total of 12 data loss periods per day was assumed. The prediction performance of the LSTM network was compared with the polynomial model.To predict RTS orbit and clock correction using a polynomial model, it is necessary to select the best polynomial order for the prediction. In this study, the order was chosen based on references ,25. RTS SSRA03IGS1 data for a total of 4 weeks from 7 July to 3 August 2022, was used to train the LSTM network for the orbit correction prediction. For the pre-processing of LSTM network training data, all data was separated and standardized for each IOD interval. The number of LSTM cells is 50 and they have been trained to predict future data one-step ahead using the adaptive moment estimation (Adam) optimizer. Based on first-order gradients, the Adam optimizer adaptively calculates the learning rate for each weight and is one of the most efficient stochastic optimization algorithms . Adam isTwo types of LSTM experiments were performed with different lengths of update data. One updated the LSTM network at a short data interval (180 s) after the IOD change. The other updated the LSTM network with a long data interval (3600 s) before and after the IOD change. The latter utilized the concept of the orbit difference adjustment after the IOD change. The polynomial experiments have the two types as well—one with a short length of data for polynomial fitting and the other with a long length of data.When considering the IOD change, network update and 4th order polynomial fitting were performed with 3600 s of data before and after the IOD change. When ignoring the IOD change, the network update and 1st order polynomial fitting were performed with 180 s data right after the IOD change. With the short 180 s fitting data, 4th order polynomial fitting is not suitable and 1st order was used instead. After the update or fitting, the RTS prediction was executed for 3000 s. All four cases are summarized in Prediction of the RTS corrections was performed using the four methods, cases 1 through 4. Five days’ worth of data were processed to evaluate the performance of the LSTM prediction method with the consideration of the IOD change.The accuracy of the predicted corrections was calculated by adding the predicted results to the orbit derived by the navigation message and by comparing them to the IGS precise ephemeris. The positioning accuracy with the predicted RTS corrections was evaluated. For the real-time test, only GPS data was used, and the prediction results were applied to the GPS broadcast ephemeris. PPP was performed by RTKLIB, a real-time precise positioning program. RTKLIB is an open-source program developed by Takasu of the University of Tokyo and is now widely used to run PPP . The funThe LSTM network is applied to predict IGS RTS orbit correction during periods of data loss. The LSTM method offers better accuracy than the polynomial method when the prediction period is long. IGS RTS corrections are generated to adjust the orbit and clock information of the GNSS navigation message, which is regularly updated, and the updates are identified by the IOD number. When the IOD changes, the corresponding RTS correction changes abruptly and the RTS correction is no longer continuous after the IOD change. This discontinuity makes it difficult to predict the RTS correction if the signal loss occurs immediately after the IOD change. We propose an offset value to minimize the RTS discontinuity by subtracting the navigation message orbits before and after the IOD changes.The proposed method was evaluated with five days of GPS and RTS orbit correction data. The evaluation focused on the prediction performance when the RTS signal loss occurred immediately after the IOD change. The results of the LSTM network were compared with those of the polynomial method, and the results with the IOD change adjustment were compared with those without IOD adjustments. The prediction interval, corresponding to the signal loss duration, was set to 3000 s. Since the prediction accuracy of the LSTM network for the RTS clock correction was not significant, the constant  prediction method was used for both the LSTM and the polynomial methods. The LSTM network method with the IOD change adjustment outperforms other methods. The reduction of the prediction error by the LSTM method with the IOD adjustment over the 4th order polynomial method with the same IOD change adjustment is 80.6%. The PPP positioning accuracy with the predicted RTS corrections shows similar behavior as the RTS prediction. The LSTM method with the IOD adjustment outperforms other methods; the vertical position error mean is reduced by 51.8% from the polynomial method without the IOD adjustment and by 47.1% from the LSTM method with the IOD adjustment.


Face mask-wearing detection is of great significance for safety protection during the epidemic. Aiming at the problem of low detection accuracy due to the problems of occlusion, complex illumination, and density in mask-wearing detection, this paper proposes a neural network model based on the loss function and attention mechanism for mask-wearing detection in complex environments. Based on YOLOv5s, we first introduce an attention mechanism in the feature fusion process to improve feature utilization, study the effect of different attention mechanisms  on improving deep network models, and then explore the influence of different bounding box loss functions  on mask-wearing recognition. CIoU is used as the frame regression loss function to improve the positioning accuracy. By collecting 7,958 mask-wearing images and a large number of images of people without masks as a dataset and using YOLOv5s as the benchmark model, the mAP of the model proposed in the paper reached 90.96% on the validation set, which is significantly better than the traditional deep learning method. Mask-wearing detection is carried out in a real environment, and the experimental results of the proposed method can meet the daily detection requirements. The spread of COVID-19 endangers public safety, and its known transmission routes mainly include droplet transmission and close contact. Therefore, wearing masks has become the main means of preventing the spread of the virus . The traMask-wearing detection technology is one of the key prevention means supervised by epidemic prevention departments, and the stability and robustness of mask detection algorithms are particularly important. Mask-wearing detection is developed based on face detection and target classification. Compared with simple face detection, mask-wearing detection needs to address many problems, such as different lighting, side faces, occlusion, and density. However, solving these problems and improving mask-wearing detection accuracy are of great significance , 4.Study the effect of different attention mechanisms  on improving the deep network model.Explore the impact of different bounding box loss functions  on mask-wearing recognition.With the development of deep learning technology, mask-wearing detection technology based on deep learning has been widely used , 6. HoweThe rest of the content is arranged as follows: Mask-wearing detection is an object detection task that determines the location and category of each object in the image. Mask-wearing detection methods based on deep learning are divided into two categories. One is a two-stage target detection model based on region extraction, such as RCNN  , Fast R-Attention mechanisms have achieved great success in various computer vision tasks . The purBounding box regression is a mainstream technique in object detection that uses a rectangular bounding box to predict the location of the target object in the image, aiming to refine the predicted bounding box location. Bounding box regression adopts the overlapping area between the predicted bounding box and the ground-truth bounding box as the loss function, which is called the IoU-based loss function. Deep learning optimization suffers when the predicted and ground-truth bounding boxes do not intersect. GIoU  is an imYOLOv5 is a representative of a single-stage detection model, which has the advantages of fast speed and high accuracy. Compared with YOLOv4, the model has fewer parameters, simple operation, and easier transplantation to the mobile terminal. It was proposed by Ultralytics in May, 2020. There are four network models, namely, YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. Among them, YOLOv5s has the smallest network depth and feature map width, and the other three models are continuously deepened and widened based on YOLOv5s.The input includes an adaptive anchor box and an adaptive image scaling.The backbone includes the focus module and the CSP module. The slicing operation in the focus module reduces the number of computations and improves the speed while realizing downsampling. The CSP module is beneficial for improving the network learning ability and reducing the memory cost. Two structures, CSP1 and CSP2, are designed in YOLOv5s. The CSP1 structure and the CSP2 structure are applied to the backbone and neck networks, respectively, to further speed up the inference speed of the network model.The core of the neck adopts FPN and PAN structures. The FPN and PAN structures realize the fusion and complementation of high-level features and low-level features. FPN is top-down and uses upsampling process to transfer and fuse the new type to obtain the predicted feature map. The PAN adopts a bottom-up feature pyramid. In this paper, CBAM is introduced into FPN, and the fused feature map is sent to CBAM to reduce the influence of redundant features after fusion.The prediction includes the bounding box loss function and nonmaximum suppression (NMS). In this paper, CIoU is used as the loss function to locate the target box more accurately. In the target detection prediction result processing stage, the optimal target frame is obtained by using the weighted NMS operation for screening numerous target frames.Based on YOLOv5s, this paper adds the CBAM attention mechanism in the feature fusion process to improve feature utilization and uses CIoU as the bounding loss function to improve the positioning accuracy. The model structure is shown in The convolutional block attention module (CBAM) is a lightweight convolutional attention module that combines channel and spatial attention mechanism modules. CBAM includes two submodules, the channel attention module (CAM) and the partial attention module (SAM), which perform channel and spatial attention, respectively. This not only saves parameters and computing power but also ensures that it can be integrated into the existing network architecture as a plug-and-play module. CAM is an adjustment to the structure of the SE module. Based on the SE module, a global maximum pooling operation is added to the CAM. CAM compresses the feature map into a one-dimensional vector in the spatial dimension, uses global average pooling and global maximum pooling to aggregate the feature information of the spatial map, and performs an element-by-element sum operation on the results by sharing the fully connected layer. The structure setting of the double pooling operation can make the extracted high-level features richer and provide more detailed information. SAM performs the concatenating operation on the result of the CAM operation based on the channel and performs single-channel dimensionality reduction through convolution. Similar to CAM, SAM adopts a double pooling operation. CBAM is similar to the SE module. The module structure mostly uses a 1 × 1 convolution to operate and completes the information extraction of the feature map through the entire channel dimension of the SAM, as shown in Since the CBAM model adds a global maximum pooling operation to the CAM, it can make up for the information lost by the global average pooling to a certain extent. In addition, the generated 2D spatial attention map is encoded using a convolutional layer with a kernel size of 7, and a larger kernel is good for preserving important spatial regions. The YOLOv5s network with CBAM added can not only classify and identify the target more accurately but also locate the target more accurately.A, B)=(A∩B)/(A∩B), where A is the predicted box and B is the ground-truth box. IoU can be used as the distance; then, LossIoU=1 − IoU. The advantage of IoU is that it can reflect the detection effect between the predicted box and the ground-truth box. IoU has two disadvantages: when the prediction bounding box and the ground-truth bounding box do not intersect and IoU = 0, the distance between A and B cannot be reflected. At this time, the loss function is not steerable and IoU loss cannot optimize the situation where the two bounding boxes do not intersect. Assuming that the sizes of the prediction bounding box and the ground-truth bounding box are determined, as long as the intersection value of the two boxes is determined and their IoU values are the same, the IoU value cannot reflect how the two boxes intersect. To this end, the paper adopts the CIoU loss function, and its formula is as follows:b and bgt represent the center points of prediction box B and ground-truth box Bgt, respectively, c represents the square of the diagonal length of the minimum bounding box C, p represents the calculation of the Euclidean distance between the two center points, α is the weight parameter, and v is used to measure the similarity of the aspect ratio. In addition, it can be seen that the CIoU loss function not only considers the overlapping area of the predicted frame and the real frame but also considers the distance between the center points and the aspect ratio of the two. Therefore, in the mask-wearing detection environment, the performance is better than other loss functions.The loss function of the target detection task consists of two parts: classification loss and bounding box regression loss. The most commonly used bounding box regression loss is IoU and its improved algorithm. The full name of the IoU algorithm is the intersection and union ratio, which is obtained by calculating the ratio of the intersection and union of the predicted box and ground-truth box, that is, IoU, and mean average precision (mAP@0.5) are used as model accuracy evaluation indicators, where AP represents the area under the PR curve and mAP@0.5 represents the average precision (AP) of all categories when IoU is set to 0.5. The specific formula is as follows:In the training phase of the YOLOv5s model, the initial parameter batchSize is set to 8 and 8 images are randomly selected for training each time. The epoch is set to 450 rounds, and the rect is true. By setting the same parameters, different loss functions are used to train the network. When the model is trained to 300 epochs, the model begins to converge. After 450 epochs, each model takes the optimal result.To verify the performance of the CBAM, this paper uses SE, CBAM, and CA to conduct comparative experiments. Among them, SE attention is channel attention, to solve the loss problem caused by the different weights occupied by different channels of the feature map in the process of neural network feature extraction. CBAM attention is additional spatial attention based on SE attention. The feature of CA is that the channel attention is divided into two different directions, horizontal and vertical, so that the information of the position and spatial direction of the input feature image can be fused, which can make the model more accurately locate the detection target.By adding three attention mechanisms of SE, CBAM, and CA to different positions of the YOLOv5 network model (such as backbone network and neck network), the paper found that the performance of the same attention in different positions is different. To verify the detection performance of the CIoU loss function, the paper uses IoU, GIoU, DIoU, alpha-IoU , and VarTo further verify the effectiveness of the method proposed in the paper, Pictures of people wearing masks and pictures of people without masks in real scenes are collected, and the YOLOv5s + CIoU and YOLOv5s + CIoU + CBAM network models are used to evaluate the collected pictures. The detection results in the real environment are shown in Aiming at the occlusion and density problems of mask-wearing detection, this paper proposes a mask-wearing detection model based on an attention mechanism and loss function and carries out experimental tests based on different attention mechanisms and loss functions. Taking YOLOv5s as the basic framework, the experimental results show that the CBAM is significantly better than the other two attention mechanisms. Experimental tests on different loss functions show that the CIoU loss function is slightly better than the other three loss functions. The experimental results tested in the real environment show that the proposed model is robust to small targets and occlusion. Future work will further study the new network model to improve the accuracy of mask detection.


Recent developments in pattern analysis research have made this methodology suitable for the study of the processes that are set in motion in psychological interventions. Outcome research, based on the comparison between clinical results from treatment and control groups, has leveraged our empirical knowledge about the efficacy of psychological interventions. However, these methods of research are not precise enough for the analysis of these processes. On the contrary, pattern analysis could be a powerful tool to study moment-to-moment interactions typical of psychological interventions. This is methodology is relevant because clinical psychology is experiencing a paradigm shift from a protocol for syndrome perspective to a principle-based and person-centered intervention. This evidence-based, theory-grounded, and process-oriented paradigm of clinical intervention needs new research methods to thrive . The analysis of the therapeutic relationship built into the verbal interaction between the clinician and the client is one of the cornerstones of this new era of research. So, the purpose of this article is three-fold: (1) to discuss the role of the verbal interaction pattern analysis in the clinical context to the development of the principle-based clinical psychology, (2) to analyze the patterns of verbal interaction in a clinical case, and (3) to compare the results using two different methods. To reach these purposes, using the observational methodology, we have coded the verbal interaction of 16 clinical sessions with a person diagnosed with a borderline personality disorder. We have analyzed the data using sequential analysis (GSEQ) and pattern recognition algorithms . We have been able to detect typical patterns during different phases of psychological intervention . Finally, the conceptual, methodological, and empirical implications of this study will be discussed within the realms of pattern analysis research and principle-based clinical psychology. Pattern is a term that refers to a stable repetition of events that arise from specific circumstances. The recognition of patterns in the natural world has had an evolutionary impact on the animal species, it has allowed us to adapt to the environment by taking advantage of these regularities. Moreover, understanding how these patterns work has led us to control and predict these natural events. Recognition and analysis of patterns are scientific endeavors that have benefited from the efforts of multiple scientific fields  and are useful to improve applied sciences . Furthermore, the development of new definitions of patterns or the application of existing methods of analysis to new areas could solve contemporary problems in several scientific disciplines, we believe that this is the case in clinical psychology. This paper aims to improve clinical psychology research using pattern analysis methods , to do that first we define the state of the art in clinical psychology research highlighting the current challenges, then we analyze the conceptual and practical opportunities that pattern analysis methods open to this field of research, and finally we conduct a proof-of-concept study.Clinical psychology has reached an important stage of development in regards to the scientific agreement about the efficacy of psychological interventions for specific psychological problems . Thanks Specifically, in the psychological interventions with adults, these processes occurred within the verbal interaction between the psychologist and the client. Other important variables explain the therapeutic change of our clients , but the verbal interaction with the client is the main channel that psychologists have to implement their procedures .The interest in the study of verbal interaction in clinical sessions from a functional perspective is present since the early 60 s see . This coFor this observational study we had the participation of a 31-year-old client diagnosed with borderline personality disorder (BPD) for the last 8 years; and a 35-year-old clinical psychologist . Both participants came from a public-funded Vocational Rehabilitation and Employment center (VR&E) in the Community of Madrid. The client has been referred to this center due to mood problems  and substance use problems that directly interfere with the client’s chances of accessing employment/training opportunities; and once obtained, problems in keeping his job or completing the required training. Also, this study involved the participation of two trained observers. Both observers are predoctoral students that have been trained in the same research group . Observer 1 and Observer 2 have, respectively, 4 and 2 years of experience working with the observational coding system used in this research and they helped in the development of it. Before recording the clinical session, the client and clinician have been informed about the use of the data and the purpose of the research. All participants have signed the study informed consent.Clinical Discriminative Stimulus (CD) and Instructional Discriminative Stimulus (ID), Conditioned Motivating Operation (CMO), and Positive Reinforcer (R+) and Aversive Stimulus (AS). The observational system assumes that the verbal behavior of the client has a response function it is not established any specific categories in the observational system. To code the client’s verbal behavior in this case we used eight categories based on the response topography Giving Information (GI), Asking for Information (AI), Following Instructions (FI), Not Following Instructions (NFI), Well-Being (WB), Discomfort (D), Target Behavior (TB) and Problem Behavior (PB). See We used the Functional Coding System for Verbal Interaction in Clinical Contexts  to code Recordings of 16 clinical sessions were obtained through the camera installed on the VR&E psychologist’s computer. The recordings were sent to the ACOVEO research group and were treated following the protocol used in the research group in which the recordings are anonymized following the ethical and legal guidelines of the Organic Law 3/2018 on Personal Data Protection and guarantee of digital rights. These recordings were stored on external hard drives kept under lock and key in the group’s laboratory at the Autonomous University of Madrid.The Observer XT 12 observation software. The data analysis was done in R  and Microsoft Excel. GSEQ , Treatment Phase 1 (T1), and Treatment Phase 2 (T2). These phases were not experimentally manipulated and have been divided considering the protocols and procedures of the VR&E center . The division between the two treatment phases is arbitrary and responds to a need to divide the treatment into, at least, two phases to evaluate differences between various time points.k > 0.70) was achieved while coding similar clinical sessions. These sessions were from the research team’s clinical sessions archive.After the study, both observers received specific training in the observational instrument. The training process was completed when a stable reliability index , which are a normalized index of the extent to which the values of the frequencies observed in each cell of the matrix deviate from their expected values: a value greater than 1.96 indicates that this behavior occurs significantly more than expected and, conversely, a value less than −1.96 implies that it occurs significantly less than expected by chance  was used to calculate the inter-observer and intra-observer reliability of the records. Once the records were obtained, the rate per minute of each variable recorded in each session was calculated. Descriptive data  were obtained to allow a visual inspection of the variables and the patterns through time. To analyze the interaction between variables, we used two types of statistical analysis, which are part of the family of statistical tools for the sequential analysis of temporally distributed data. First, we calculated the Yule’s Q, a contingency index of 2×2 tables, using the GSEQ software , 2016. TT-Pattern Model (ThemeEdu software (T-Patterns) from observational data (T-Data) using the T-Pattern detection algorithm. Pattern detection works in a bottom-up fashion, from the data to the pattern detection. The T-Pattern is a hierarchical, multi-ordinal, and self-similar pattern type that comprises m ordered components , X1.m, recurring in a single discrete dimension, where each component is a T-data category  or a T-pattern  which is an index focused on identifying the differences between two phases of a design A and B; .k values can be interpreted as very good and excellent reliability indexes , positive correlations (Q > 0) are painted in green, and negative correlations (Q < 0) are painted in red. Data indicate that there is a positive correlation between Clinical Discriminative Stimulus with the client’s Giving Information behavior in all phases of the case ; Following Instructions correlates positively with Target Behaviors during Evaluation and T1. Finally, Conditioned Motivating Operation and Target Behaviors in all phases.case see . FollowiPositive Reinforcers and Target Behaviors, Well-being Verbalizations, Following Instructions, Aversive Stimuli, and Problem Behavior. Also Conditioned Motivating Operations were weakly correlated with Target Behaviors and Asking for Information.The consequent sequential analysis indicates a positive correlation between Clinical Discriminative and Giving Information in all phases of the clinical case, this pattern repeats 888 times through different clinical sessions. Also, there are some patterns especially repeated in the early stages of the treatment , and in treatment phases . Moreover, we have detected patterns increasing through the phases .Target Behaviors and Positive Reinforcers . Our analysis reveals similar data with four-term patterns. The most repeated patterns involve Target Behaviors and Positive Reinforcement .Pattern detection has indicated which variables are moment-to-moment correlated in verbal interaction. These results have allowed us to analyze how these variables change along with the psychological treatment. Clinical Discriminative functions among different treatment phases and how are they followed by Giving information verbalizations of the client. As we will discuss these visual inspections could help practitioners to evaluate the clinical relationship or the efficacy of the clinical intervention.For example, Target Behaviors and the therapist’s Positive Reinforcers. To assess the effect of these Positive Reinforcers we have calculated the effect size in the client’s Target Behaviors . In this case, the effect size index indicates that the treatment had a moderate effect on increasing the Target Behaviors .Both sequential analysis and pattern analysis indicate positive correlations between the client’s Target Behaviors of the client. Third, we have compared the results yielded by these two methods of pattern analysis. In this section of the article, we discuss methodological implications in the search for hidden patterns of interaction in clinical settings, the clinical implications of our results, the limitations of our work, and the main conclusions.The aim of the current article was three-fold. First, we have highlighted the potential relevance of pattern analysis methods in the study of verbal interaction in psychological interventions and we have presented the reasons why these types of analysis could help in the development of the next generation of process analysis in clinical research. Second, we have conducted an observational proof-of-concept study to analyze the verbal interaction in a single-case design using two methods of pattern analysis . Using sequential analysis and more complex pattern detection algorithms we could identify more than 25 interaction patterns between a clinical psychologist and his client in different phases of the psychological treatment. These interaction patterns were visually displayed through different sessions, and the effect size of the treatment was measured for the positive reinforcer after a problem behavior of the client , the interpretation could lead to erroneous conclusions, because it did not include the variables that are inside of this correlation, and the software simply correlate two variables that are in a specific distance. That is not the case with THEME, it automatically has detected patterns formed by three and four correlated variables. This feature is essential to the study of verbal interaction because it has been useful to detect the repetition of structured patterns that imply a use by the psychologist of three- and four-term contingencies. In this study, we have detected 7 three-term patterns and 3 four-term patterns. This detection imply that we have increased the precision of this analysis, if we compare it with previous research on this topic e.g., . Thus, tClinical Discriminative Stimulus correlates significantly with the client’s Giving Information behavior in all phases of the case, showing antecedent discriminative control by the psychologist of the client’s Giving Information behavior. Following Instructions correlates positively with the occurrence of Instructional Discriminative Stimulus, but this only occurs in the treatment phases and it is not found in the assessment phase. These differences between the evaluation phases could be explained by the positive correlation found in the evaluation phase between Target Behaviors and Instructional Discriminative Stimulus. This contingency also seems to occur to a lesser extent in the first treatment phase and coincides with the lower correlation between Following Instructions and Instructional Discriminative Stimulus in the treatment phases. Instructional Discriminative Stimulus may have correlated in the assessment with topographies of the Target Behaviors, but this study is not sensitive to such topographies. Also, we have detected a correlation between Target Behaviors and the Conditioned Motivating Operations. This correlation could be clinically explained by the conditioning function of the conditioned motivation operations. These verbalizations have the purpose of changing the client’s motivational value of some stimulus or activities. If the clinician states a verbalization with this function and the client agrees, it is probable that this agreement could be coded as a target behavior. Also, if the clients state a Target Behavior, the clinician could explain more details about why the client is right or relate it to their therapeutic goals, so these verbalizations could have a motivating function. Moreover, this relationship between Target Behaviors and Conditioned Motivating Operations also appears to have a key role in patterns with more complex structures . But, despite all these limitations, we believe in the exploratory value of this paper, it could be useful for the development of new perspectives and methodologies for the study of processes in clinical psychology.The study of the processes underlying therapeutic change is essential to optimize psychological treatments. The identification of patterns of verbal interaction during therapy is a valuable step in understanding how the processes that make psychological treatments work are set in motion. GSEQ and THEME software have proven to be able to detect those patterns in verbal interaction. THEME has proven to be more powerful in detecting complex interaction patterns and more sensitive in detecting low-frequency patterns, yet both have detected predominant patterns in verbal interaction that may underlie clinical change. This implies that pattern recognition methods could be seen as a promising alternative to studying behavioral change processes in psychological treatments. These methods combined with single-case designs and the development of new recently developed effect sizes for this type of studies e.g., , could hhttps://osf.io/ezhmd/?view_only=dbe50c17e90a419e93c87116703ceb27.The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found at: The studies involving human participants were reviewed and approved by Autonomous University of Madrid. The patients/participants provided their written informed consent to participate in this study.All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.This research was supported by grant PSI2016-76551-R from the Ministerio de Economía, Industria y Competitividad of the Spanish Government.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.