The reconfigurable feature of FPGAs (Field-Programmable Gate Arrays) has made them a very attractive solution for implementing adaptive systems-on-chip. However, this implies additional design tasks to handle system reconfiguration and control, which increases design complexity. To address this issue, this paper proposes a model-driven design flow that guides the designer through the description of the different elements of a reconfigurable system. It is based on high-level modeling using an extended version of the MARTE  UML (Unified Modeling Language) profile. Both centralized and decentralized reconfiguration decision-making solutions are possible with the proposed flow, allowing it to adapt to various reconfigurable systems constraints. It also integrates the IP-XACT standard , allowing the designer to easily target different technologies and commercial FPGAs by reusing both high-level models and actual IP-XACT hardware components. At the end of the flow, the implementation code is generated automatically from the high-level models. The proposed design flow was validated through a reconfigurable video watermarking application as a case study. Experimental results showed that the generated system allowed a good trade-off between resource usage, power consumption, execution time, and image quality compared to static implementations. This hardware efficiency was achieved in a very short time thanks to the design acceleration and automation offered by model-driven engineering. Reconfigurable FPGA-based systems-on-chip (SoCs) can be modified several times after fabrication. Dynamic Partial Reconfiguration (DPR), supported by several FPGAs, offers the possibility to reconfigure some parts of the FPGA at runtime . This alModel-Driven Engineering (MDE) can be a solution to reduce design complexity. UML (Unified Modeling Language)  is commoOne of the advantages of using MDE is that it allows for generating a system implementation code from high-level models. To allow this feature, MDE approaches often use a deployment phase that links the components of high-level models to low-level implementations through IPs . All the necessary code for IP communication and system implementation is then generated automatically from high-level models. Furthermore, the code of some high-level components, with non-existent implementing IPs, can be generated from the UML descriptions of their behaviors, such as UML state machines. This can be the case with the reconfiguration controller, for instance. The main issue of the deployment phase of most MDE methodologies in the literature is that the deployed hardware IPs are tightly linked to the implementation technology, which makes both high-level models and actual IPs hardly reusable for various technologies. In this context, a standard representation of the system components is required to enhance design reuse.The IP-XACT specification developed by the SPIRIT consortium offers a standard XML description of IP meta-data independent of the used language. This facilitates system integration using compatible descriptions from multiple component vendors and allows IPs to be easily reused and exchanged between electronic design automation (EDA) tools for SoCs . TherefoThe progress in FPGA technologies has enabled them to embed a growing number of computing resources on one chip, targeting increasingly sophisticated applications. This has led to a more complex design handling different aspects related to runtime adaptation. In this context, it may be difficult to analyze all the control data in a centralized way. Designing a decentralized control by splitting the control problem between autonomous controllers, each handling the self-adaptation of a reconfigurable component, can be beneficial for complex reconfigurable systems. A coordinator that has visibility only on global system requirements and constraints can be used in this approach, leaving local control data management to the distributed controllers. This reduces the complexity of the controllers, facilitating their verification and their reuse as well as the scalability of the control model. Therefore, the design flow proposed in this paper offers the possibility to design both centralized and decentralized control, depending on the requirements of the control problem handled by the target system.This paper proposes an MDE-based design flow and platform for dynamically reconfigurable SoCs aiming at reducing design complexity through the following contributions:High abstraction level modeling and automatic generation of the implementation code without the need for in-depth knowledge of many low-level aspects of the FPGA technology. An extended version of the MARTE profile is used to integrate reconfiguration-modeling concepts. The separation of concerns is used when designing the models  to promote their reuse.Flexible control decision-making, handling both centralized and decentralized solutions that are easily reusable and scalable for various reconfiguration control requirements.Transforming high-level architectural models into IP-XACT files, thus targeting different tools supporting this standard.Using abstract metamodels and model transformations to be easily adaptable to designs targeting other reconfigurable FPGA technologies.Generating FPGA scripts, in addition to the automatic generation of C/VHDL code, so the synthesis and implementation flow can be run automatically to obtain bitstreams ready to be loaded onto the target FPGA.Accelerating design exploration  thanks to the automation of model-driven engineering.The proposed design flow was validated through a reconfigurable video watermarking application as a case study. Experimental results showed that the generated watermarking system allowed a good trade-off between resource usage, power consumption, execution time, and image quality compared to static implementations. The rest of the paper is organized as follows: SoCs are composite and complex systems that can include several hardware and software IPs and various processors for software tasks. Thanks to their reconfigurability, FPGA devices extend the concept of SoCs, making them able to adapt to various changes in user preferences, application requirements, and standards . HoweverIn the literature, many works have proposed high-level modeling to reduce the design complexity of DPR SoCs. In this context, several works used MDE to automatically generate the code necessary for the system implementation from high-level models, thus reducing design complexity and time-to-market. For example, the authors in  presenteThe IP-XACT standard, offering an XML description of the system IPs independently of the used language, is a good solution to face the SoCs design reuse challenge. Some works proposed the integration of IP-XACT and UML in MDE-based design since they are complementary tools for the information they deliver . Indeed,The proposed design flow is called the generic design flow for dynamic partially reconfigurable systems (GDF4DPR). The term generic is explained by the fact that even though not many FPGA vendors provide dynamic reconfiguration, this is a tendency that will cover all FPGA vendors and our design flow also aims to be used for future reconfigurable FPGAs.The flow is based on six steps, where the first four steps imply high-level modeling covering: (1) system constraints and requirements specification, (2) application design, (3) architecture and task allocation design, and (4) deployment modeling. The fifth step is model transformations and code generation, and the sixth one is design exploration, as illustrated in The proposed model-driven design platform was implemented using the Eclipse framework using plugins for high-level modeling, model-to-model transformations, and model-to-text transformations. After realizing the high-level modeling (the first four steps), the designer launches the transformation process, which generates all the system code (in VHDL and C languages). It also generates tool-specific scripts and a tool-specific design file. The designer can then launch the FPGA design tool (such as Vivado for Xilinx FPGAs), and execute the generated tool-specific scripts, which automates synthesis, implementation, and bitstreams generation. During the design exploration step (step 6), designers verify, using FPGA design tools, if the generated system respects the functional and non-functional constraints and requirements specified at the beginning of the design flow. If not, high-level models can be edited, and code can be regenerated automatically. A list of the software and hardware systems used in this work is available in Thanks to the concern separation of the first four steps, the used models can be easily modified and reused for different system specifications or to target different FPGA platforms. In this section, the design flow is described through the used UML/MARTE concepts and the extensions we propose for the MARTE profile. The application of these concepts and extensions is presented in This represents a critical step that directly affects the final system. Indeed, at this stage, the designer must include all the useful information to analyze the need of the system. This information groups together the system requirements, the user requirements, the constraints to be respected, etc. This information can be related to performance, power consumption, quality of service, etc. It is mainly modeled using the <<nfpConstraint>> MARTE stereotype , as will be illustrated in In the second step of the flow, the application is described as a set of interconnected tasks, each handling a specific function. This stage is thus dedicated to the modeling of the application tasks and the communication between them using mainly UML/MARTE components, ports, and connectors. Depending on the needs of the target system, the application tasks can be physically implemented in software or hardware. This choice is made in the next step.At this stage, the static and reconfigurable parts of the system are modeled using UML/MARTE components, ports, and connectors. To model the reconfigurable part of the architecture, we extended the MARTE profile through the <<ReconfigurableHwComponent>> stereotype, which inherits from the <<HwComponent>> MARTE stereotype as shown in This stage also specifies which tasks will be implemented in software by the processor(s) and which tasks will be implemented in hardware using the FPGA logic. Task allocation on processors or static hardware components is performed using the <<allocate>> MARTE stereotype. As for reconfigurable tasks, we propose to use the <<mode>> and <<configuration>> concepts of MARTE together with the <<allocate>> stereotype. Depending on the system requirements, reconfiguration control can be either centralized or decentralized, in which case each reconfigurable region is handled by a separate module. In both cases, state machines are used for reconfiguration decision-making, where each state (<<Mode>>) represents a different configuration of one or several reconfigurable regions. To model these state machines, the <<modeBehavior>>, <<mode>>, and <<modeTransition>> MARTE stereotypes are applied, respectively, to UML state machines, states, and transitions. The MARTE <<configuration>> stereotype is used here to describe the task allocation on reconfigurable regions. Our GDF4DPR design flow offers two ways of modeling <<configurations>>. First, a <<configuration>> can model task allocation on a single reconfigurable region for a given <<Mode>>. In this case, a separate state machine is used to model each region, so a <<configurations>> describes task allocation on this region for a given <<Mode>> of the state machine. In the second case, a <<configuration>> models task allocation on multiple reconfigurable regions. In this case, the behavior of reconfigurable regions can be modeled either by a separate state machine for each region or by using a single state machine to model the different modes of all the reconfigurable regions of the system, so a <<configurations>> describes, in the second case, either a combination of the active modes of all the state machines of the reconfigurable regions or an active global mode in a global state machine.In both centralized and decentralized reconfiguration decision-making, a global controller is required to guarantee that the reconfiguration decisions respect the global system requirements. To model this controller, we propose the <<ReconfigurationController>> stereotype as an extension to the <<RtUnit>>  MARTE stereotype. As shown in At this stage, the implementation details of the application tasks and the architecture components are specified through IPs. To be used in the deployment model, IPs have to be already packaged according to the IP-XACT standard and then transformed into MARTE components and stored in a MARTE components library. Once the software and hardware IPs are specified in the deployment model, model transformations are used to become closer to the implementation code as detailed in the next section.An IP-XACT design file describing the whole architecture is generated from the deployed architecture model and is based on a set of transformation rules: Every instance of a MARTE component having the <<IP>> stereotype is transformed into a “component instance” according to the IP-XACT standard. Connections between the MARTE components in the deployed architecture are then transformed into either IP-XACT interconnections, adHocConnections, or hierConnections depending on the type of the connected ports (single port or bus interface) and the levels of hierarchy between the connected components.After transforming the MARTE deployed architecture into an IP-XACT design, the latter is transformed into a tool-specific design to be imported into an FPGA tool. To implement the system using the Vivado tool for Xilinx FPGAs, the generated tool-specific design can be in form of a .bd (Block Design) file. For Intel FPGAs, a .qsys file can be generated to implement the system using the platform designer tool. To make this transformation easily reusable and extendable for different tools and technologies, we used a model-to-model transformation before generating the tool-specific design file. We defined a metamodel for the tool-specific design. The IP-XACT design file is first transformed into a model corresponding to the defined metamodel. Finally, we translated the obtained model into code. This step corresponds to the generation of the code necessary for reconfiguration decision-making and reconfiguration launching. It consists in translating MARTE <<modeBehaviors>> (state machines) into code for reconfiguration decision-making together with the necessary code to launch reconfiguration. For this, we propose two methods. The first one is purely software-based where decision-making and reconfiguration launching is performed by a processor. The second one is mixed, where MARTE <<modeBehaviors>> are translated into VHDL state machines. These machines are coordinated by the processor which handles also reconfiguration launching. This step will be detailed later in the case study section.The high-level model describing task allocation on the processor is translated into a processor code containing function calls to the software IP linked to these tasks at the deployment phase. This code also contains the code necessary for reconfiguration decision-making and reconfiguration launching.At this step, the proposed platform creates a project skeleton in which it copies: (1) the hardware and software IPs codes specified at the deployment phase of the high-level modeling, (2) the generated tool-specific design file, (3) the generated hardware code for reconfiguration control, and (4) the generated code for the processor. Tcl scripts are also generated at this phase so the synthesis and implementation flow can be run automatically to obtain bitstreams ready to be loaded onto the target FPGA. The generated scripts handle the following steps: (1) importing the generated design file, (2) launching Synthesis for the static part of the system and reconfigurable modules, and (3) loading the Synthesis results in the floor planning tool. Designers can then place the reconfigurable regions as desired. This is the only task that is performed manually since we cannot predict the resource usage of the reconfigurable regions before synthesizing them and therefore their placement is not completed in the high-level models. The rest of the generated scripts are then launched to automatically implement different configurations of the system and to generate the related bitstreams.The objective of this step is to verify that the generated system respects the functional and non-functional constraints and requirements specified at the beginning of the design flow. Different kinds of verifications can be completed at this stage, such as functional verification, through simulation or during runtime execution, resource usage and timing reports verification using FPGA tools, power consumption estimation/measuring, etc. Thanks to these verifications, the designer can make the necessary corrections and modifications in the high-level model of the concerned design phase to respect the system requirements and constraints. The system code can be then regenerated automatically and re-verified, if necessary, which significantly accelerates the exploration phase.In the next section, we describe the video watermarking application that we will use as a case study to validate the proposed design flow for reconfigurable systems.In this section, we use a video watermarking application to validate the proposed design platform. This application is intended for copyright protection to protect the video sequence from hacking and illegal use. The watermarking technique consists of inserting a watermark in a multimedia document  to identify the author of the document while respecting the security and the visual quality that represent the major requirements of the user. We call a watermark , the infFrom a watermarking system implementation point of view, our work presents some novelties. First, the architecture that we use in this paper is the only FPGA-based implementation using motion vectors for watermark insertion. Furthermore, to the best of our knowledge, it is the first dynamic partially reconfigurable FPGA-based architecture for a video watermarking system.The suggested application is composed of five main modules which are the motion estimation, the sum, the parity, the insertion, and the scrambling module, as shown in In the proposed architecture, the motion vectors represent the insertion space of the watermark. These vectors are generated in the motion estimator (ME) module, which is the most critical component in video compression. Its purpose is to minimize the temporal redundancy between the successive frames of the video. This has been used by various video-coding standards such as H.265 and H.264. The H.265, also known as standard High-Efficiency Video Coding (HEVC), is the most used in a wide range of new multimedia applications . The basIn this case study, we use a motion-adaptive watermarking system using two block-matching algorithms to generate the motion vectors, the gradient search (GS) and the four-step search (4SS). Depending on the movement speed in the video, the estimator will implement either the GS or the 4SS algorithm. The objective of reconfiguration here is to have a good trade-off between resource usage, power consumption, execution time, and PSNR . Indeed, previous works showed that, for slow video sequences, the GS algorithm produces a higher PSNR than the 4SS one, though at the cost of a significantly higher execution time. On the other hand, for fast video sequences, these algorithms offer close PSNRs. This makes the 4SS algorithm more suitable for this type of video due to its accelerated execution. Both block-matching algorithms are implemented using nine Processing Elements (PE) running in parallel to generate the SAD (sum of absolute differences) value. In the case of slow movement, the estimator will be reconfigured by the nine PE of the GS algorithm or else by the nine PE of the 4SS algorithm. In our previous works ,23, we pGenerally, there are three types of objects in a video sequence: objects that keep the same motion for certain frames and other objects that move in an almost constant way between two sequential frames. The third type is objects moving in slow or fast motion in the same frame . In thisVideo watermarking is a real-time application that requires high processing capacities. Indeed, a small delay in video stream processing can lead to high degradation in the visual quality. This application can thus benefit significantly from the hardware acceleration offered by FPGA. The change of the motion estimation algorithm according to the video motion speed makes the video watermarking application well-adapted to dynamic partial reconfiguration.<<nfpConstraint>> MARTE stereotype as presented in As the first step of high-level modeling, the designer specifies the requirements and constraints of the system. To model constraints linked to non-functional properties, we used the The application phase consists in modeling the different application tasks and the interconnections between them. In this phase, the tasks are defined abstractly independently of the used technology, which makes the application model easily reusable for other system configurations. The modeling of the case study application is presented in In this case study, we use two motion estimation algorithms (GS and 4SS described earlier) according to the motion speed, so the Estimator task will be implemented with two algorithms as will be specified in the rest of the GDF4DPR design flow.In this phase, we modeled the architecture as presented in Once the architecture components are specified, the designer must detail the reconfigurable part of this architecture. As mentioned above, the motion estimator used will implement either the GS or the 4SS algorithm at a given time. The operation of one algorithm or the other depends on the video motion speed. Each PE is implemented in a reconfigurable region. As shown in After describing the static and reconfigurable parts of the architecture, the allocation of the application tasks on the architecture component is modeled. The allocation in the MARTE profile consists in attaching each abstract component of the application to its execution support in the architecture. The designer performs two types of allocation. The first type is the allocation of static tasks, which will be implemented in the static part of the system, and the second type is the allocation of reconfigurable tasks, which will be implemented in the reconfigurable regions.In this step, the task allocation of the processor and static hardware components is achieved using the <<allocate>> MARTE stereotype. As shown in As mentioned above, both centralized and decentralized reconfiguration decision-making approaches are possible in the proposed GDF4DPR design flow. For this case study, since all the PEs implemented in the reconfigurable regions have to be reconfigured at the same time with the same content, centralized decision-making is used here. In case the target system has reconfigurable regions handling various tasks, allocating a separate controller to each task allows it to make reconfiguration decisions related to the reconfigurable region. These decisions can be either authorized or denied later by a global coordinator to respect global system requirements and constraints. In this case, a separate state machine would be used for each reconfigurable region. The decentralized control would have many advantages in this case, such as decomposing the control design complexity between controllers, providing easier access to local data for decision-making, and enabling easier placement and routing of distributed controllers compared to a centralized hardware controller. In our case study, we modeled one state machine that represents the behavior of all reconfigurable regions using the <<modeBehavior>> and <<mode>> MARTE stereotypes as shown in In this case study, the reconfigurable region performs two modes of the same application task. Therefore, it is necessary to indicate the version of the task to be allocated to a region for each configuration of the reconfigurable region. As we mentioned earlier, in both centralized and decentralized reconfiguration decision-making, a global controller is required to guarantee that reconfiguration decisions respect the global system requirements. In this case study, this task is performed by the processor. To do so, we apply the <<reconfigurationController>> stereotype, described earlier in The deployment phase consists in assigning to each elementary component of the architecture or application a software or hardware IP implementing it. We use the <<implements>> stereotype to link the IP, which has the <<IP>> stereotype, to the application task that it implements. The code files of the used IPs are also specified at this stage using the <<codeFile>> stereotype. As shown earlier in The last step of high-level modeling is the description of the deployed architecture. This step deals with the interconnections between the architecture IPs as shown in At this stage, the deployed architecture is transformed into an IP_XACT design file, where MARTE components having the <<IP>> stereotype are transformed into an IP-XACT “component instance”. Connections between the components of the deployed architecture are then transformed into either IP-XACT interconnections, adHocConnections, or hierConnections depending on the type of the connected ports (single port or bus interface) and the levels of hierarchy between the connected components. The obtained IP-XACT design file is then transformed into a tool-specific design file. In this case study, the IP-XACT file was transformed to obtain a .db (Block Design) file to be used in the Xilinx Vivado tool. To make this transformation easily reusable and extendable for different tools and technologies, we used model-to-model transformation before generating the tool-specific design file.In addition to the generation of the tool-specific design file, our platform generates the reconfiguration control code. For this case study, the modeled state machine modeled in The final step of code generation is to generate tool-specific Tcl scripts, so all the synthesis and implementation flow can be run automatically to obtain bitstreams ready to be loaded onto the target FPGA. Our design flow generates three scripts. The first one handles the following steps: (1) importing the generated design file, (2) launching Synthesis for the static part of the system and reconfigurable modules, and (3) loading the Synthesis results into the floor planning tool. Designers can then place the reconfigurable regions using the floor planning tool. This is the only task that is performed manually since we cannot predict the resource usage of the reconfigurable regions before synthesizing them and therefore their placement is not completed in the high-level models. In this application, the reconfigurable region size corresponds to the size of PEs of the 4SS algorithm since it requires more resources than the GS algorithm. The second script is then launched to automatically implement different configurations of the system and to generate the related bitstreams. The last script compiles the software to be executed by the on-chip processor and generates the whole system image to be loaded on FPGA.After the first generated script is launched, we can check the consistency of our architecture before launching the second script that implements the system configurations. At this phase, we can verify that the static part, comprising interfaces to the reconfigurable regions, is consistent for all the configurations. For this, we use a design rule checker, which allows for detecting and managing errors in a report form. After launching the second script, we can detect if the reconfigurable regions were optimally placed and sized. Designers can easily replace these regions if necessary to have better resource usage results. Other verifications are also possible through simulation or at runtime. If the detected errors or problems are related to the structure of the application, the architecture, tasks allocation, or components deployment, this can be solved by modifying the high-level models and regenerating the system code automatically. This makes design exploration faster and spares the designer from tedious, error-prone, and long development cycles.In a previous work , we implThe proposed reconfigurable watermarking system aims to make a good trade-off between the following requirements: resource usage, power consumption, execution time, and PSNR for various motion speeds. Therefore, in the rest of this section, we present the implementation results of the reconfigurable watermarking system in terms of resource usage, power consumption, and reconfiguration time. We also compare execution time and PSNR results to those of static implementations.In the proposed architecture, we use nine reconfigurable regions. In each reconfigurable region, a PE implementing either the GS or the 4SS algorithm is loaded, depending on the generated SAD average value. The reconfigurable region size is determined based on the largest reconfigurable module (RM) size to be loaded, which in our case is the RM implementing the PE with the 4SS algorithm. The hardware resources used by an RM implementing the 4SS algorithm are summarized in The resource usage of the whole system was compared to three static versions of the system: a static version implementing only the GS algorithm, a static version implementing only the 4SS algorithm, and a third version using adaptive motion estimation proposed in our previous work . The difAs for bitstreams, the GDF4DPR design flow generated one full bitstream and a set of partial bitstreams . The full bitstream is the one to be loaded initially and it contains the static part of the system and the partial bitstreams of the GS algorithm. Using SAD values, the adaptation module of the estimator decides which algorithm has to be executed and sends an interruption to the processor if a reconfiguration is required. The processor then launches the required partial reconfiguration through the PCAP (Processor Configuration Access Port) interface. The 32-bit PCAP interface has a theoretical reconfiguration speed of 400 MB/s when clocked at 100 MHz on a Xilinx Zynq SoC. The total size of the nine partial bitstreams is 1595 Kb, which implies a reconfiguration time of 30 ms. This reconfiguration time is not negligible. This is because the total size of the reconfigurable module is quite high. Since the reconfiguration is based on two thresholds applied to the average SAD produced after processing each frame, this implies a small number of reconfigurations. This is because successive frames have, most of the time, similar average SAD values, unless the type of the motion changes significantly, which we manage using the SAD thresholds.Our watermarking system was tested using three video sequences of 300 frames in the QCIF format. These sequences have three different motion behaviors: slow, medium, and fast. The average power consumption of the reconfigurable watermarking system while processing the three video sequences was 140 mW. The same video sequences were executed in the non-DPR version of the watermarking system using the adaptive motion estimator proposed in . The aveSince one of the objectives of using two different motion estimation algorithms is to have a good trade-off between execution time and video quality (represented by the PSNR), we compared the PSNR and processing time given by our DPR system with the same three non-DPR versions described earlier: the static version implementing only the GS algorithm, the static version implementing only the 4SS algorithm, and the version using adaptive motion estimation proposed in our previous work  , the DPR version provides better PSNR results than the other systems. It also has a faster execution time than the GS version. This is because this video sequence has more fast-motion frames than slow-motion ones, depending on the used SAD thresholds, which makes the DPR version apply the 4SS algorithm more than the GS one for this video sequence. Compared to the version in , the DPRFor the fast sequence (Silent), the DPR version provides better PSNR results than the other systems. Its execution time is between the GS and the 4SS execution times since it applies the 4SS algorithm more than the GS one for this video sequence. This is because, for fast sequences, the 4SS algorithm has faster execution times than the GS one with comparable PSNR results. Compared to the version in , the DPRThe implementation results described in this section show that the reconfigurable watermarking architecture generated by the proposed design flow makes a good trade-off between execution time, video quality, resource usage, and power consumption. The proposed model-driven approach shows a significant design acceleration thanks to the high-level modeling and the automatic generation of VHDL/C code and tool-specific files and scripts.In this paper, we propose a model-driven design flow and platform for dynamic partially reconfigurable architectures aiming at accelerating the design of reconfigurable systems to be implemented on FPGAs by reducing design complexity, automating code generation, and facilitating design reuse. The GDF4DPR is based on high-level modeling that covers several design steps from system specification to physical implementation. As for reconfiguration decision-making modeling, the proposed design flow handles both centralized and decentralized control solutions to adapt to various reconfigurable systems constraints. It proposes an automatic transformation of high-level models into IP-XACT designs to ensure IP reuse and to target different technologies and commercial FPGAs. The proposed flow enables the automatic code generation of the whole system, including tool-specific files and scripts, to increase designer productivity. As a case study, we used a video watermarking application. It can be concluded that the proposed flow supports all the design stages of dynamically reconfigurable architectures. Its high-level modeling allows the designer to easily target different technologies. The implementation results show that the proposed platform allowed us to automatically generate a dynamic partially reconfigurable watermarking architecture that provides a good trade-off between adaptivity, performance, resource usage, and power consumption. In future work, we intend to enhance the proposed platform to support more complex reconfigurable systems and offer more automatic generation possibilities.Although the tool-specific design files generated by the design flow are currently targeting Xilinx FPGAs, we designed the tool-related metamodel and model transformation in a way that makes them easily reusable and extendable to target different reconfigurable FPGA technologies, which is one of the future directions of our work. In the presented case study, the performance/energy consumption trade-off was handled by fixing SAD thresholds to minimize the reconfiguration number and save energy. In future work, we intend to integrate more sophisticated power/performance optimization techniques into the proposed design flow to target various reconfigurable systems constraints. Dynamic task partitioning is also an interesting feature to add to the proposed design flow to cover more requirements of reconfigurable systems.


K-IoU loss function is defined to strengthen the penalty of the wrong segmentation, which rationalizes the loss of mis-segmentation and effectively prevents wrong segmentation. Finally, an ablation experiment is designed to evaluate the effectiveness of the chromosome instance segmentation network based on regression correction, which shows that our proposed method can effectively enhance the performance in automatic chromosome segmentation tasks and provide a guarantee for end-to-end karyotype analysis.Precise segmentation of chromosome in the real image achieved by a microscope is significant for karyotype analysis. The segmentation of image is usually achieved by a pixel-level classification task, which considers different instances as different classes. Many instance segmentation methods predict the Intersection over Union (IoU) through the head branch to correct the classification confidence. Their effectiveness is based on the correlation between branch tasks. However, none of these methods consider the correlation between input and output in branch tasks. Herein, we propose a chromosome instance segmentation network based on regression correction. First, we adopt two head branches to predict two confidences that are more related to localization accuracy and segmentation accuracy to correct the classification confidence, which reduce the omission of predicted boxes in NMS. Furthermore, a NMS algorithm is further designed to screen the target segmentation mask with the IoU of the overlapping instance, which reduces the omission of predicted masks in NMS. Moreover, given the fact that the original IoU loss function is not sensitive to the wrong segmentation,  Chromosomes are essential carriers for genetic information, and their abnormalities may result in congenital genetic diseases . HealthyTraditional automatic chromosome segmentation methods are mainly based on geometric morphology  and threRecently, more researches have constructed deep learning methods to accomplish medical image processing tasks, which can effectively avoid the occurrence of the aforementioned issues. Similar to natural image segmentation, chromosome segmentation methods based on deep learning are mainly divided into semantic segmentation  and instIt can be seen that the accuracy of the target detection box is extremely important in the chromosome instance segmentation tasks. Generally, when detecting clustered targets, the classification confidence of the target box is often high, but the actual detection result is poor, which leads to a decrease in the AP score with high IoU threshold. To address this issue, Jiang et al. constructed IoU Net, which predicts the IoU of the regression box and the ground truth box to replace original classification confidence, which eliminated the screening error caused by the misleading classification confidence, thus improved the target detection performance . Wu et aPBox and IoUMask with positioning accuracy and segmentation accuracy are achieved without extra head branches to achieve better correction of the classification confidence. PBox is the predicted probability based on the regression box, and IoUMask is the predicted IoU based on the mask.1) Considering high classification confidence but poor detection and segmentation performance in reality, more relevant confidence of 2) Considering that the traditional non-maximum suppression algorithms based on the overlap screening of prediction boxes, which may result in missing or wrong target boxes, a non-maximum suppression algorithm based on instance mask screening is proposed to improve the segmentation of instances.K-IoU loss function is designed. It divides the area to be segmented into K parts and calculate the weight of each part to the overall segmentation loss according to the proportion of the area to be segment in each part to the total area, which improves the sensitivity of the network to error segmentation and makes the penalty reasonable.3) Since the traditional IoU loss function is not sensitive to the wrong segmentation area, This study proposes a chromosome instance segmentation network based on regression correction to achieve precise segmentation in the Giemsa-banding chromosome images. The main contributions of this study are summarized as follows.The multitask supervised learning method is known to make good use of valuable information to obtain more accurate results for each task. Its effectiveness lies in the correlation between all tasks. However, the predicted result of the regression branch is the offset of the regression box rather than the actual coordinates. There is no direct correlation between the offset and the IoU score, which makes it not reasonable enough. In addition, the use of IoU score to modify the classification confidence will cause the drop of classification confidence, thus worsen the subsequent non-maximum suppression operations. Therefore, Wu et al. and Chen et al. proposed regression branches to predict IoU scores under a multitask supervised learning framework, but the results showed low correlation with the real IoU scores. To address this issue, we propose here a regression correction-based instance segmentation network for chromosome segmentation, as shown in PBox is introduced, as shown in PBox is predicted through a fully connected layer with 1,024 output nodes. It helps make the prediction process of PBox more reasonable, which shows stronger correlation with positioning accuracy.T(·) is tanh function, and LReg is the regression loss, which is calculated by the Smooth L1 loss function.First, a regression confidence IoUMask, the output of the mask branch acts as input, and the IoUMask is predicted by the fully connected layer with 1,024 output nodes, instead of multitask supervised learning, which helps make the prediction process more interpretable, as shown in Due to the direct correlation between the output of the Mask branch and PBox, as well as IoUMask, which is more relevant with the segmentation accuracy, are used to correct the classification confidence. Thus, both the detection score and the segmentation score are considered simultaneously to achieve better instance segmentation performance.Finally, the regression confidence For overlapping target detection, the non-maximum suppression algorithm should be further improved due to its poor effect on severe overlapping . TherefoTherefore, a mask-based non-maximum suppression algorithm is proposed here for overlapping chromosome segmentation tasks. The algorithm aims to remain as many prediction boxes as possible before the prediction box fed into the mask branch and then calculates the IoU of each prediction mask and other prediction masks. Finally, traverse the classification confidence from high to low and remove prediction masks that have an IoU score greater than that of the threshold with the current prediction mask. It makes use of the IoU of the mask as a threshold to help select overlapping targets, which can effectively prevent missing and misjudged overlapping targets, thus improve segmentation performance.IoUMask represents the IoU score between the predicted mask and its ground truth.There are multiple metrics for segmentation performance evaluation. Among them, IoU is the most widely used one, and better segmentation performance expects higher IoU score. Thus, the IoU loss function  is oftenLK-IoU, is proposed for the incorrectly segmented region, as shown in K-IoU loss function, the network has better segmentation performance for difficult-to-segment regions.K indicates the number of different parts that the ground truth mask is divided into. As shown in K is 4 and the shape is 2 × 2, the ground truth mask is equally divided by two vertical center lines to obtain four parts. As shown in δi indicates the proportion of the ground truth in the i-part over the entire ground truth, and IoUi indicates the IoU of the predicted mask and the ground truth in the i-part.However, IoU can only represent the overall segmentation quality of the prediction results. It cannot adequately represent the segmentation quality of some key regions. Under chromosome segmentation scenarios that chromosomes exhibit variable shapes, fuzzy edges, and severe overlaps, the difficult-to-segment regions are the key regions that call for more attention. The segmentation quality of key regions may better help karyotypists to diagnose, thus provides more reliable information for physicians’ choice of medical regime. Thus, a more effective and reasonable loss function, δi scores of the four parts are demonstrated, with the striped area being the predicted mask. In LK-IoU (δi = 1) is used. Compared with the IoU loss function, better segmentation performance can be obtained when the loss converges to the same value, and the sensitivity of the network to incorrect segmentation is dramatically improved.As shown in LK-IoU (δi = 1) are the same. It may thank LK-IoU that corrects the loss of each part through the weight δi, as shown in However, it is not necessary to blindly increase the sensitivity of the network to incorrect segmentation. When the proportion of the ground truth mask in a certain part to the entire ground truth mask becomes lower, the influence of this part on the whole is smaller. Comparing LBox is composed of three parts, which are defined in LCls is calculated by the cross-entropy loss function, and PBox obtained by LSeg is also composed of three parts:LMask is the binarized cross-entropy loss function, and LK-IoU, calculated by Then, we define the multitask loss on each proposal as the sum of the losses from Box head and Seg head, as shown in Wein the next conducted five-fold cross-validation experiments on 985 real chromosome Giemsa-banding chromosome images of 1,600 × 1,200 pixels. A total of 60% of the data was allocated for training, while the remaining 40% images were equally partitioned and referred to as validation and test sets. These images were first scaled and padded to 512 × 512 and data augmentation was also involved to better train the models.Mask RCNN , PANet , IoU NetM  and the method of MS RCNN, both use the predicted IoUMask to correct the classification confidence. The segmentation performance of the former is better than that of MS RCNN. This verifies from the side that the method in this study makes the predicted correlation between IoUMask and Mask stronger and is more helpful to correct classification confidence.By comparing the method of directly using the output of the Mask branch to predict IoUBox instead of classification confidence, is ineffective and even leads to a decrease in APM. It is due to the fact that the correlation between the output of the regression branch and IoUBox is not strong enough. Therefore, this article uses a more relevant head branch to predict the regression confidence PBox to correct the classification confidence (the eighth row). Compared with IoU Net, APM has a more significant improvement, which means that the regression confidence PBox can modify the positioning accuracy of the prediction box more than IoUBox. Finally, this study considers both the positioning accuracy of the prediction box and the segmentation accuracy of the instance (the ninth row). The APM has been further improved to 83.11 with an increase of 2.73%.The IoU Net–based method, which uses PBox of the prediction box and segmentation accuracy IoUMask of the instance at the same time to improve the segmentation performance of the network. However, multiplying the two directly with the classification confidence may not be the best choice. Therefore, PBox and IoUMask are exponentiated, and the APM scores obtained are shown in IoUMask is “√2” and PBox is “√,” the specific calculation method is shown in The design of confidence weight: This study considers the positioning accuracy PBox is calculated to the sixth power, APM reaches the highest score of 83.35%. Moreover, the improvement is more significant than the effect brought by the exponentiation of IoUMask. It can be seen that APM is more sensitive to PBox, further verifying the effectiveness of PBox.We can see that when PBox and IoUMask, which are more relevant to the positioning accuracy and segmentation accuracy and achieve a better correction of the classification confidence. A non-maximum suppression algorithm based on mask is proposed, which uses the overlap of the instance as the basis for judgment, which effectively prevents the missing and incorrect segmentation of the chromosomes. Moreover, a K-IoU loss function is proposed, which improves the network’s sensitivity to incorrect segmentation while fully considering the impact of the incorrect segmentation on the whole so that the penalty is reasonable. The experimental results show that the method in this study greatly improves the accuracy of instance segmentation on the baseline Mask RCNN, and it also has a good effect on PANet. Since the implementation of PBox and IoUMask does not require additional head branches and the structure is relatively simple, it is expected to be extended to other models which aim at instance segmentation.This article focuses on improving the segmentation accuracy of chromosome instances in real chromosome datasets, significantly overlapping chromosomes. We respectively use the output of the regression branch and the mask branch to predict two confidences,


The sudden decline in the attitude control performance is a common abnormal situation during shield tunneling. When the problem happens, the shield driver will have difficulty controlling the shield's attitude, which will cause the shield to deviate from its design axis and affect the quality of the tunnel. The causes behind poor control performance are usually complicated, so how to choose appropriate countermeasures is a challenging problem. Based on the above issues, this paper proposes the Interpretable Predictive Model for Shield attitude Control Performance (IPM_SCP). The model first predicts the current shield control performance through the extreme gradient boosting (XGBoost) sub-model and then uses the Shapley additive explanation sub-model to interpret the model output. The model was tested on the left-line tunnel of the Hangzhou–Shaoxing railway project in the Ke-Feng section. The results reveal that the model could effectively predict the control performance of the shield and give the most influential parameter and the direction in adjusting the parameter to improve the shield's attitude control performance when the control performance decreases. Therefore, IPM_SCP gives the correct parameter adjustment instructions when the shield’s attitude control performance declines, and eventually improves tunnel construction quality and efficiency. For example, in a subway project in Nanning, the shield had difficulty lowering its head, which led to the segments’ dislocation and damage and seriously affected the tunnel quality2. Another example is that during the construction of a high-pressure underwater subway tunnel, the vertical deviation of the shield machine was too large due to improper handling of its attitude, which prevented the project from progressing properly3.The shield tunneling method is widely used in tunneling projects owing to its fast excavation speed, high degree of automation, and low impact on the surrounding environment. However, shield tunneling still faces many challenges; posture instability and tunneling misalignment are two common construction risks during tunneling. Posture instability means that the attitude of the shield cannot be kept to the control target, and tunneling misalignment means that the position of the shield machine deviates significantly from the design axis. In most circumstances, shield misalignment is caused by posture instability. Excessive misalignment can cause segment cracking, segment offset, and even water leakage4. There are five main types of posture discord: (1) the shield machine tilts upward uncontrollably, (2) the shield machine tilts down uncontrollably, (3) the shield drifts upward (the position of shield rises as a whole)5, (4) the shield body has difficulty turning horizontally in a certain direction, and (5) the rotation angle of the shield is too large.Posture instability will seriously affect the construction progress, increase project cost, and cause severe consequences, such as segment damage, tunnel water leakage, excessive ground settlement, and even tunnel reroutingThe main reason for the posture discord is that the driver cannot easily control the posture of the shield by just adjusting regional oil pressure. Therefore, researchers have attempted to use various methods to resolve the problem. Most studies approach the shield attitude control problems in three ways.6. They concluded that the upper and lower limits of shield elevation adjustment would drop as the downhill angle increases. The sliding force was the leading cause of shield tilting down. They also made several suggestions to prevent the shield tilting down. Li analyzed the factors influencing the shield's attitude on soft ground and gave advice on the shield's potential remedies for lousy attitude in Suzhou metro tunnel project7. However, these studies are mainly based on the empirical analysis from technicians; the findings are not sufficiently clear and definitive to be of use as practical guidance.The first type of research focuses on shield attitude anomaly analysis in specific tunneling projects. Guan et al. analyzed the shield attitude adjustment of an inclined shaft project8. Yue et al. modeled the hydraulic system and built a simulation system to simulate the variation of shield attitude9. However, the theoretical method cannot cover most of the influence of various factors because the model is too simplified. When encountering real-world situations, the model's accuracy will be significantly decreased.The second type of research approach is to model the process of shield attitude adjustment based on the analysis of the forces acting on the shield. The theoretical method examines the force between the shield and the soil as well as its impact on the shield’s attitude. Alsahly et al. established a load model to represent the interaction between the surrounding soil and the shield body and developed a computational framework that can obtain the change of the shield attitude during excavation10. Shen et al. incorporated wavelet transform into a Long short-term memory (LSTM) network to predict the attitude and position of the shield11. This type of method improves the accuracy of attitude prediction. However, owing to the model’s lack of interpretability, it is almost impossible to clarify the relationship between the features and the model's output, so it is impractical for on-site personnel to use the model as a guide to solve the problem of deteriorating shield control performance despite the high accuracy the models have. Furthermore, the method can only be applied in specific conventional engineering contexts because most of the methods primarily consider the common situations and omit the unusual ones. Furthermore, once complications occur in shield attitude control, it is not easy to use the models directly to improve the control power of the shield.The third research type is the data-driven approach. Compared to the theoretical approach, the data-driven approach is based on actual engineering data and uses machine learning algorithms to predict key control parameters. Li et al. used gated recurrent units (GRU), a variant of recurrent neural networks that utilized real-time tunneling parameters to predict the tunneling attitude of the shield7, most studies emphasize attitude control optimization in the normal state.From the perspective of controlling the shield, the major manifestation of posture instability is that the shield's attitude control performance does not meet the expectations. Therefore, how to evaluate and improve attitude control performance is a significant issue. However, except for a few studies that have focused on studying shield attitude control under abnormal situations for specific projects12. Avunduk and Copur and Arbabsiar et al. established multiple regression models to predict the thrust, cutter disk torque, and other tunneling parameters of shields under different soil conditions, as well as performance indexes such as instantaneous cutting rate and energy consumption14. Martins and Miranda and Mahdevari et al. used the artificial neural network and support vector regression (SVR) models to establish a hard rock excavation speed prediction model16. Fattahi and Babanouri also used the SVR model to predict the excavation rate and used a variety of heuristic algorithms to optimize the parameters of the model17.At present, there are few studies on shield attitude control performance. Nevertheless, some researchers revolved around the tunneling equipment's excavation performance to improve the shield machine's tunneling capacity. Wang et al. established the rock mass rating (RMR) system and the empirical formulas of the shield tunneling rate, construction progress, utilization rate, and penetration index using the regression methodInspired by the excavation performance studies, this paper focuses on evaluating, analyzing, and optimizing shield attitude control performance to fill the research gap. We designed an interpretable attitude performance prediction model as well as an index for shield attitude control performance. The model can not only accurately indicate the current shield's attitude control performance with the help of the index, but also can show the effects of the input parameters for the decline of the attitude control performance so that construction personnel can take effective countermeasures eventually.18 is used to evaluate each player's contribution fairly. Suppose we regard each control parameter of the shield as a participant. In this case, we can refer to the Shapley value to quantify the marginal contribution of each control parameter for attitude control to gain a clearer understanding of which control parameters have the most significant impact on the current control performance and how the parameters influence it. Then, the control parameters can be adjusted more precisely. Based on this idea, combined with the extreme gradient boosting (XGBoost) model’s advantage in prediction accuracy, this paper proposes an Interpretable Predictive Model for Shield attitude Control Performance (IPM_SCP).In cooperative game theory, the Shapley valueThe rest of the paper is organized as follows. Section The shield attitude control studies can be divided into two types: theoretical analysis based and data-driven.19. Similarly, Yue et al. established a kinetic mechanical model and used a numerical simulation to predict the effect of the various attitude correction strategy20. Sugimoto et al. demonstrated a finite element model and used it to predict the attitude change of the shield21. Yue et al. made a model for shield attitude and the electromagnetic hydraulic system9. Accordingly, a jack control strategy used for shield correction was proposed.Methods based on the theoretical analysis establish models that describe the interaction between the shield and the surrounding soil. Shield weight, force upon the shield tail, thrust, excavation surface soil resistance, and shield surface resistance are the key considerations for these models. Sramoon et al. established a shield load model based on the forces mentioned above and analyzed the factors affecting the shield attitude control based on the force balance equation6. The model was used to describe the shield attitude adjustment ability under inclined shaft conditions. He concluded that the greater the downhill angle (the sliding force of the shield), the lower the bottom limits of the shield's vertical adjustment capability. He then gave suggestions on how to prevent the shield from pitching forward too much. Li established a load model and analyzed the influence factors of the shield’s attitude in the soft soil layer through simulation, and then gave recommendations on the attitude correction strategies of shield7.This method is also used in the analysis of shield attitude control performance. Guan et al. developed a mathematical model based on the Winkel elastic foundation model and shield loadIn summary, while methods developed from the theoretical analysis have great interpretability and can reflect the key factors that affect attitude control, the generalization of the model is usually poor because these models rely heavily on assumptions of the forces, soil descriptions, and parameter settings. Therefore, it is difficult to directly guide the tunnel construction with these models.22. Xia et al. and Wang et al. used the XGBoost model to predict the deviation of the shield24. The model's input is composed of control parameters such as cutter speed, cutter torque, total thrust, and advance speed. Then, the control parameters are optimized based on experience and rules if the model output yields poor results. Many researchers have also used deep learning models on shield attitude prediction. Zhou et al. used the combination of convolutional neural networks and LSTM networks, that is, the CNN + LSTM method, to extract the features of the time series of the shield parameters and predict the attitude deviation25. Hu et al., Li et al., and Zhang et al. used recurrent neural networks to predict the attitude deviation27. Data-driven methods can reflect the correlation characteristics of the parameters and have high short-term prediction accuracy, but they are prone to overfitting. At the same time, machine learning models lack interpretability and cannot explain the relationship between the model input and output. Therefore, these models are only used for attitude prediction instead of control attitude directly.With the rise of machine learning, more researchers have chosen data-driven approaches to study the attitude control of shields. These methods use the shield parameters as the input of the model and the deviation of the shield posture as the model output. SAKAI and HOSHIYA applied the Kalman filter theory to the attitude control of the shield and established an autoregressive predictive control modelThe traditional modeling method based on the theoretical analysis has fewer factors, and its model parameters are hard to obtain, which causes its weak prediction performance. In contrast, data-driven modeling methods used in attitude deviation prediction have the advantage of high accuracy. However, owing to their lack of interpretability, it is difficult to use them in attitude control because they still depend on experience and rules for the on-site workers to select a control strategy when the shield encounters abnormal conditions. Although some researchers have studied the causes for the difficulty of shield attitude control, there are hardly any universal index for shield attitude control performance and these studies have often been confined to a specific project or stratum condition, so their conclusions can hardly be applied to other projects.Therefore, this paper aims to design a universal index that can indicate shield attitude control performance and identify a model that is both accurate and interpretable to analyze the influence of various shield parameters related to attitude control performance and the interaction between parameters. In this way, the model can genuinely guide the shield attitude control as the model can provide direction in adjusting specific parameters to improve the attitude control performance of the shield when the attitude control performance is weak.28. This phenomenon is also the direct cause of most posture instability and misalignment problems.The shield attitude is mainly controlled by the jacks. To simplify the control process, jacks are grouped according to the diameter of the shield, usually forming four or six groups. The shield driver controls the shield posture by adjusting the oil pressure of each group. Take the four-group shield machine as an example. The way the jacks are grouped is shown in Fig. 6. This index is only suitable for inclined shaft tunneling. Therefore, to improve the universality and accurately reflect the shield's control performance, we introduce a shield control performance index based on the way shields are controlled. We use the ratio of the oil pressure difference (the same direction to the attitude change) to the shield attitude change as the Shield Attitude Control Performance Index (SCPI) as shown in Eq. , which combines XGBoostIPM_SCP is divided into four steps: (1) Preprocess the shield tunneling data according to the attitude control and feature independence principle; (2) Use historical data to train a predictive model for predicting SCPI based on the model input and output; (3) Based on the shield attitude control performance prediction model, use SHAP to analyze the relationship between input features and the model output as well as the magnitude and direction of the influence of the features; (4) If the current attitude control performance is weak, analyze each feature's importance and direction of its influence with the model and the in situ data, formulate adjustment strategies, and optimize the control parameters to improve the attitude control performance.Based on the shield attitude control performance index, we conduct a preliminary selection of the input features of CPPSM. The features crucial to shield control performance according to Sect. Figure 29, this paper uses the XGBoost algorithm to establish the relationship model between the shield control performance and shield attitude. The input feature of the model is the feature set selected in Table Because the XGBoost algorithm can achieve better accuracy in multiple regression tasks and is not easily prone to overfittingThe objective function of the model is shown in Eq. .4\documeEquation  is compo33 and count how many splits in the decision tree originated from the same feature. However, these methods may have inconsistencies 34, which makes the result unreliable. Also, these methods cannot reflect the direction of each feature influence on the model output, nor can they reflect the interaction between features.There are several ways of getting the feature importance from an XGBoost model. For instance, based on the basic decision-tree structure of the XGBoost model, one can sum up the gain of each feature split on each node34. It regards the features in the machine learning model as participants in Shapley, and the predicted value as the goal of all participants. By calculating the Shapley value of each feature, the contribution of each feature to the predicted value can be calculated. Based on the principle of SHAP, the output of CPPSM can be obtained by adding the contribution value of each feature, as shown in Eq. . This model combines the XGBoost algorithm and SHAP to predict a shield attitude control performance. It takes advantage of the SHAP value to determine the contribution of each input control parameter for the attitude performance, while XGBoost predicts the actual control performance with precision. The results can accurately reflect the influence that the various control parameters have on the attitude control performance. The model also provides a basis for the formulation of attitude control strategy for the shield. This method was applied to the left line of H–S Project, where it successfully solved two attitude control issues during the tunneling. Although the symptoms of two occasions both appeared to have difficulty raising the head position, two opposite methods of adjusting earth pressure were applied for the shield machine based on the explanatory analysis of the model, which both achieved good results and successfully got rid of the engineering dilemma. Therefore, applying the explanatory model can help accurately grasp the real cause of poor posture control so that personnel can adopt precise control strategies, avoid making decisions blindly, and improve construction quality and efficiency.IPM_SCP emphasizes nonconventional attitude control and focuses on shield control performance. It also proposes the shield attitude control performance index and an interpretable shield control performance prediction system. It determines the adjustment direction of each control parameter by analyzing the interactive relationship among all the input control parameters as well as their influence on shield control performance. Compared with other methods, IPM_SCP can quickly find out the factors that cause abnormality when the shield control is poor so as to guide the excavation of the shield better.Admittedly, although IPM_SCP achieved some degree of success in practical applications, it still has limitations. For example, the interpretability of IPM_SCP is highly dependent on the accuracy of its sub-models. However, because of the complex conditions of the tunneling environment, it is necessary to retrain the model when a new project starts. Lack of data at the beginning of a project will affect model performance. Therefore, in future research, we need to make better use of the trained model by using transfer learning or other methods. If the model can achieve better results on a smaller dataset, the reliability of interpretation will be further improved, and the model will also have a broader application.


A geodetic reference frame is a fundamental element in geoinformation fields such as autonomous navigation and digital twins. The international terrestrial reference frame is established and maintained using several space-geodetic techniques, including very long baseline interferometry (VLBI) and satellite laser ranging (SLR). For several decades, geodesists have been devoted to connecting these two sensors at a site . The reference point of the VLBI antenna and SLR telescope, called invariant point (IVP), should be precisely determined to connect these two solutions. We developed an innovative integrated model to estimate the IVP, which is composed of spherical and conical models, depending on the rotational axis. In this model, all target points in 3D spaces were directly connected to the IVP; thus, the stability and robustness of the system were secured. Furthermore, all inherent errors in the coordinates were predicted by applying the total least-squares approach. The international earth rotation and reference systems service (IERS) has contributed to establishing and maintaining the international terrestrial reference frame (ITRF) by analyzing and combining data from five different space-geodetic systems such as global navigation satellite system (GNSS), satellite laser ranging (SLR), lunar laser ranging, very long baseline interferometry (VLBI), and Doppler orbitography and radiopositioning integrated by satellites ,2,3,4.The “co-located sites” are the stations where more than one of the five instruments are in operation simultaneously. They play a pivotal role in the combination of data observed by different physical sensors located away from each other via a local tie vector. A local tie vector represents a three-dimensional (3D) baseline vector between the reference points of two space-geodetic instruments ,6. ThereIn general, an IVP is physically located inside the antenna structure, and its coordinates cannot be estimated directly from ground surveys. Therefore, the indirect approach has been widely used for the determination of the IVP by observing targets attached outside the VLBI antenna during rotational sequences ,9.The coordinates of the targets, together with their variance–covariance information, were obtained from ground surveys by applying the least-squares method. Then, the IVP estimation procedure is performed using the geometric relationship between the targets and IVP. The 3D circle fitting method is commonly used to estimate the IVP. The trace of a target on the antenna rotating about an independent axis can be expressed as a 3D circle that can be defined using seven parameters, including three parameters for the center of the circle, three parameters for the unit normal vector, and one parameter for the radius of the circle ,12,13,14The trace of each target due to the horizontal rotation presents a full circle in normal surveying conditions, which increases the stability for parameter estimation. However, the trace of a target by the vertical motion in elevation makes only a quarter circle because of the limited motion of the rotational axis. Consequently, it leads to an unstable linear system being solved. Moreover, each rotational sequence of the elevation axis yields seven parameters for modeling each arc. The increase of rotational sequences requires more parameters to be estimated, which is one of the disadvantages of the 3D circle fitting method.In this study, we propose a new model for the determination of the IVP from the coordinates of targets estimated based on terrestrial observations. The basic concept of the new approach is that the trace of horizontal  rotation becomes a 3D circle in space. However, the motion in elevation creates a cone shape with an apex at the IVP, where the slant distance of the cone is the only unknown for a vertical target. Therefore, a new approach, called the sphero-conical model, connects all target positions in the 3D space to a common IVP, which significantly reduces the unknowns compared to the conventional methods. In other words, the primary unknowns are the coordinates of the IVP and an additional parameter for each target, representing either horizontal or vertical motions.Setting aside the small number of observations, the conventional model based on a 3D circle may lead to an ill-conditioned linear system when circle arcs are incomplete; as a consequence, the quality of the estimated parameters will be degraded . HoweverIt should be noted that the coordinates of the observed targets are observations; hence, they have inherent random errors. Therefore, the adjustment computation must be performed by estimating not only the IVP, but also the coordinates of the targets on the antenna. The total least-squares (TLS) approach was adopted in this study because it minimizes random errors in all data variables .In the following sections, detailed descriptions of the new sphero-conical model for IVP estimation and the numerical test results with in situ measurements are provided.Although the space-geodetic sensors are located at the same station, they are physically different from one another. Therefore, the connection between two different sensors can only be accomplished by ground surveying, such as slant distance and horizontal and/or vertical angles, using total stations and/or theodolites. For IERS to accurately combine solutions obtained from both GNSS and VLBI techniques, the coordinate vector between their respective antennas  must be precisely determined. As the VLBI antenna rotates in any direction, its point of rotation remains fixed in location, which is the antenna’s IVP. Thus, we must determine accurate coordinates of the VLBI IVP.The IVP location (coordinates) with respect to the continuously operating reference station (CORS) antenna was determined from measurements made during local ground surveying see . HoweverThe main task of the project is to connect the GNSS and VLBI antennas by measuring and adjusting the ground survey data. In the following, we often refer to these two antennas, or their locations, as “GNSS antenna” (or just GNSS) and “VLBI antenna” (or just VLBI), respectively. We developed a new model that relates all target coordinates to a single IVP; thus, every single point in the 3D space is connected to the IVP. In addition, the set of coordinates for each target is obtained from the ground survey; thus, each component of the coordinate set has an intrinsic error that should be considered in the adjustment process. Therefore, we applied the TLS approach based on the Gauss–Helmert model (GHM) to account for all error components in the target coordinates ,17,18.The VLBI IVP, coordinates of the pillars, and targets were estimated based on the mathematical model developed in this study. The estimation process is based on the TLS within the GHM, in which the coordinates of the pillars/targets are estimated simultaneously.In the final step, the VLBI IVP should be represented in the global reference frame , which can be performed using the Helmert transformation typically performed in geodetic reference frames ,22,23. TGround surveying is generally conducted in a local frame, whose vertical axis is naturally determined from the local plumb line. Although the origin of the local frame can be set up anywhere within the site, CORS SEJN was chosen as the origin of the local reference frame. This is because it should eventually be aligned to the ITRF to be combined with other space-geodetic solutions, which is performed using the GNSS campaign at local pillars, including CORS SEJN. Therefore, the local tie vector is represented with respect to the GNSS pillar (SEJN). The orientation of the local reference frame will be defined using the local geodetic coordinate system of the CORS (SEJN); thus, there might be an offset called the “deflection of the vertical” between the geodetic vertical direction and the local gravity vector . HoweverAll possible slant distances and horizontal/vertical angles between the pillars were observed with a total station (an electronic theodolite integrated with an electronic distance-measuring instrument) . BecauseTo determine the local tie vector in the ITRF, the coordinates of the eight pillars and the CORS (SEJN) ARP should be estimated in both the local and global frames. A GNSS campaign was conducted by simultaneously collecting data from GNSS receivers mounted on all eight pillars and the CORS. Data were collected at a 30 s rate over seven 24 h sessions for one week. The GNSS data collected at the pillars were processed together with those from 51 international GNSS service core stations worldwide, resulting in a set of GNSS vectors aligned to ITRF 2014. The resulting vectors, along with their covariance matrices, were used as observations in the subsequent network adjustment.A total of 13 targets were mounted on the VLBI antenna, which consisted of seven prisms and six sheet targets at different locations ,27. All Likewise, all possible horizontal and vertical angles between the pillars and targets on the VLBI antenna were measured together. The coordinates of the targets and their covariance matrix were estimated via a least-squares adjustment of data. Minimally constrained adjustments were first performed using PMINOLESS to minimize the norm of the parameter vector .All targets were attached “outside” the VLBI antenna (the opposite side of the concave area). Two different motions are possible for the VLBI antenna: (a) azimuthal rotation through 360∘ see , and (b) azimuth . TheoretAssume that the local and VLBI-specific frames are aligned with each other. The azimuthal rotation of the targets is depicted in k, are highly correlated. Therefore, it is desirable to estimate only the radius of the circle, which is related to the horizontal components. The mathematical model for the circle on the horizontal plane traced by target k through points The radius of the horizontal circle  in As can be seen in Equation , the unkx-axis (east direction), where height information was adopted from a spirit leveling survey (orthometric height). The estimated IVP values were based on the 3D circle fitting of the conventional LESS for each target, which was averaged to generate the final solution.To evaluate the completeness and/or compatibility of the mathematical models, numerical verification was performed using the survey results from March 2019 . The iniThe observations are the coordinates Numerical results show that the TLS method with the GHM works well in this experiment. As can be seen in The estimated parameters, along with the precision of the estimates are summarized in As can be seen in the table, the radii of each motion for the targets were estimated with a precision of <1 mm. For the IVP, the horizontal components were estimated to be even better than the vertical components, in terms of the dispersion of the estimates. This is because all horizontal and vertical targets were related to the horizontal component . In other words, all measured target positions are directly connected to the IVP; thus, no single target is in vain during several steps of 3D circle fitting, as in conventional methods. The total number of parameters is significantly reduced in the proposed method, especially for the traces of vertical targets , resulting in considerable instability in the estimation process. As explained earlier, ground survey measurements, such as slant distances and horizontal/vertical angles, can be directly used in a sphero-conical model. In addition, when combined with TLS within the GHM, we can estimate the errors in all components of the target coordinates in the 3D space.It should be noted that the vertical offset of the antenna axis must be considered, although it was assumed to be parallel to the plumbline in this study. There might be two types of offsets: (a) the horizontal offset of the vertical axis or (b) the inclination of the vertical axis. These offsets should be modeled to complete the estimation of the invariant point . BecauseLastly, two motions  were modeled differently in this study: either the radii of the horizontal circles in the 3D space or the slant distances of the cones with the apex at IVP. However, a unified, homogeneous model that incorporates two motions makes the system simpler and more consistent, which will be discussed in future research.


Ultrasonic nondestructive testing (NDT) provides a valuable insight into the integrity of stainless steel structures, but the noise caused by the scattering of stainless steel microstructure often limits the effectiveness of inspection. This work presents a novel adaptive filtering approach to enhance the signal-to-noise ratio (SNR) of a measured ultrasonic signal from the inspection of a stainless steel component, enabling the detection of hidden flaws under strong noise. After the spectral modeling of the noisy ultrasonic NDT signal, the difference between the spectral characteristics of a flaw echo and that of grain noise is highlighted, and a reference spectrum model to estimate the frequency spectrum of the echo reflected by any possible flaw is developed. Then, the signal is segmented and the similarity between the spectra of data segments and the reference spectra is evaluated quantitatively by the spectral similarity index (SSI). Based on this index, an adaptive time-frequency filtering scheme is proposed. Each data segment is processed by the filtering to suppress the energy of noise. The processed data segments are recombined to generate the de-noised signal after multiplying weighting coefficients, which again is determined by the SSI. The performance of the proposed method for SNR enhancement is evaluated by both the simulated and experimental signal and the effectiveness has been successfully demonstrated. Excellent mechanical properties such as toughness, high strength as well as resistance to corrosion and heat have made stainless steel important building material widely applied in civil and marine engineering. For example, introducing stainless steel tubes to replace the carbon steel tube in concrete-filled-steel-tube systems can fundamentally solve the corrosion problem . StainleHowever, stainless steels are coarse-grained, heterogeneous and anisotropic. The interaction between incident ultrasonic waves and the grain microstructure of stainless steel results in a type of complex acoustic noise, often called grain noise, leading to low SNR of the measured signal and poor detectability of flaws. The instrumental noise of the NDT system could further lower the SNR.For the effective inspection of stainless steel, SNR enhancement of the measured ultrasonic signal is of great importance, and numerous methods have been proposed for it. Widely explored methods include wavelet-based processing ,11,12,13When an ultrasonic pulse propagates inside a stainless steel component, the pulse will interact with the grain microstructure, resulting in the distortion of the pulse due to scattering and attenuation. In stainless steel, as reported in , the ratependent . On the ependent . The difBased on the above analysis, this work presents a novel methodology to enhance the ultrasonic NDT of stainless steel materials by exploiting the time-frequency characteristics of the ultrasonic signal. Specifically, the commonly used pulse-echo testing mode will be considered here. It is important to note that the echo reflected by a defect has a finite-time duration and the distortion of the waveform is dependent on the traveling distance. Taking these characteristics into consideration, it is desirable to exploit the time-frequency features of the signal rather than analyzing the signal as a whole. We start with the spectral modeling of grain noise and a typical flaw echo, and how the two spectra evolve with traveling distance is established. The difference between the spectral characteristics of a flaw echo and that of noise is highlighted. A reference spectrum model based on this difference is developed for the echo reflected by any possible flaw. Next, the signal is segmented and the similarity between the spectra of data segments and the corresponding reference spectra is evaluated quantitatively by a term defined as the spectral similarity index (SSI) in a moving window fashion. For each data segment, its SSI will determine the shape of the filter as well as the weighting coefficient which is used in the recombination stage where all the filtered data segments are recombined to generate the de-noised signal.The rest of this paper is organized as follows. In When conducting ultrasonic NDT on a stainless steel component in pulse-echo mode, the measured signal In pulse-echo mode where the transducer is functioning as both a transmitter and a receiver, the scattered wave by a grain scatterer in the time domain can be viewed as the convolution of the impulse response of a grain scatterer In the frequency domain, the above expression can be written as:d phases . Rf is tSimilar to grain noise, the echo reflected by a flaw can be viewed as a convolution between the impinging ultrasonic pulse and the impulse response function of a flaw, considering the effect of frequency-dependent attenuation: By comparing Equations (3) and (5), it is obvious that the spectrum of a flaw differs from that of grain noise, represented by From Equation (5), it is seen that once the attenuation function itten as ,24:(6)Afforehand . Generalith segmented data of which the time delay is The procedure of the proposed method is illustrated in To evaluate the degree of similarity between orted in , by mappA kernel function Substituting normalized variables into Equation (11), one hasFrom Equations (10)–(12), it is easily noticed that Equation (12) is equivalent to computing the correlation coefficient between Note that the calculation of SSI above combines two commonly used similarity criteria, i.e., the correlation coefficient and the sum of squares of deviations.ith data segment and its corresponding reference spectrum. The proposed time-frequency filtering is expressed as follows:ith data segment after the filtering process, Essentially, the reference spectrum ing form :(16)Wn=1Because the spectrum of a flaw echo is assumed to process a Gaussian shape where the energy of the signal is distributed in a finite bandwidth centered at the center frequency, the width of the Tukey window From Equation (15), it can be observed that if the spectrum of a data segment After the filtering stage, inverse FFT is applied to Algorithm 1. Procedure of the proposed method. Input: noisy signal Output: de-noised signal Step 1: Data segmentation.Successive segmentation of the signal, such that:(1) Segment length (2) The overlapped length of adjacent segments is Step 2: Filtering of each data segment.ith segment whose time center is For the (1) Calculate its frequency spectrum (2) Obtain the reference spectrum: (3) Calculate its SSI value (4) Let (5) Calculate the filtered spectrum (6) Calculate the weighting coefficient Step 3: Signal reconstruction.Reconstruct the de-noised signal The implementation of the proposed method is summarized in Algorithm 1.Considering the typical inspection scenario shown in jth grain scatterer which is For the inspection scenario above, the received signal contains two flaw echoes and grain noise backscattered by the grain microstructure of stainless steel as well as white Gaussian noise from the circuits of instruments. Employing the model in , the freThe simulated noisy signal is the superposition of grain noise To evaluate the SNR enhancement performance of the proposed method, normalized signal-to-noise ratio (NSNR) is introduced which is calculated as :(22)NSNRFor the highlighted signal segment in In this section, the proposed methodology is employed to process signals obtained from ultrasonic inspection of a stainless steel block in the laboratory to demonstrate its effectiveness. The specimen being tested is a heat-treated Type 304 stainless steel block with two fabricated flaws (side drilled holes with a diameter of 2 mm) as shown in Longitudinal sound velocity in the block is measured first on the time-of-flight principle. The thickness of the specimen is measured ten times using a Vernier caliper, and the average thickness of the specimen is about 81.4 mm. The round-trip travel time of the ultrasonic pulse reflected by the bottom of the specimen is 28.4 μs; therefore, the longitudinal sound velocity equals In In this work, a novel time-frequency filtering approach is proposed to enhance the ultrasonic inspection performance of stainless steel structures by suppressing noise induced by the grain microstructure of stainless steel; thus, increasing the SNR of the measured signal. Spectral modeling of a flaw echo and grain noise shows that, in the same time frame, the spectrum of a flaw echo and that of grain noise exhibits different characteristics which can be exploited to suppress the energy of noise. As such, the spectral similarity index is introduced, which is used to determine the shape of the Tukey window in the filtering stage and to determine the weighting coefficients in the signal recombination stage. This approach has been validated both in the processing of simulated signals and signals measured from experiments. In both cases, the enhancement of SNR is significant. The high fidelity of the de-noised signal enables accurate flaw depth detection, showing good performance of the proposed method. Moreover, the method is also effective for scenarios where multiple flaws exist.


Text document clustering is one of the data mining techniques used in many real-world applications such as information retrieval from IoT Sensors data, duplicate content detection, and document organization. Swarm intelligence (SI) algorithms are suitable for solving complex text document clustering problems compared to traditional clustering algorithms. The previous studies show that in SI algorithms, particle swarm optimization (PSO) provides an effective solution to text document clustering problems. This PSO still needs to be improved to avoid the problems such as premature convergence to local optima. In this paper, an approach called dynamic sub-swarm of PSO (subswarm-PSO) is proposed to improve the results of PSO for text document clustering problems and avoid the local optimum by improving the global search capabilities of PSO. The results of this proposed approach were compared with the standard PSO algorithm and K-means algorithm. As for performance assurance, the evaluation metric purity is used with six benchmark data sets. The experimental results of this study show that our proposed subswarm-PSO algorithm performs best with high purity comparing the standard PSO and K-means traditional algorithms and also the execution time of subswarm-PSO comparatively takes a little less than the standard PSO algorithm. In recent years, there has been a tremendous increase in the volume of text documents on the Internet and modern applications which affect the text analysis process  ,2. Text SI algorithms include simple unintelligent agents that follow some simple rules to accomplish very complex tasks. SI algorithms are suitable for resolving complex text document clustering problems compared with traditional clustering algorithms . The PSOThe next section provides an overview of the research work focused on the improvements in the PSO algorithm. This section shows the previous research works  related This section describes the main process of text document clustering. This process includes text document collection, pre-processing, document representation, clustering, and cluster validation as shown in t represents the number of unique terms in the document collection, and d represents the number of documents. The text document clustering algorithms (PSO and subswarm PSO) directly use this matrix to convert the document data sets into meaningful sub-collections.Pre-processing is an important step to enhance the performance of the clustering algorithm. As shown in the As shown in This subsection shows the evaluation metrics of text document clustering. Clusters can be evaluated using the metrics such as purity, homogeneity, accuracy, completeness, entropy, F-measure, V-measure, and adjusted rand index (ARI). In this study, we use purity to evaluate the quality of text document clustering algorithms.N. The ideal clusters will get a purity value is 1. Equation  and (3), in  ,21.AlgoThis section describes the proposed subswarm-PSO approach. The main focus of this study is to optimize the result of the PSO algorithm for the text document clustering (Algorithm 2) by applying dynamic sub-swarm techniques to the PSO algorithm as shown in n) and other parameters of PSO. Each particle of the PSO algorithm is randomly initialized with the k cluster centers. Then the algorithm computes text document clustering around the cluster centers and executes the fitness function.The initialization section of the proposed subswarm-PSO algorithm is the same as the PSO algorithm and its starts with the initialization of populations ) P for the next iteration as shown in the Algorithm 3. The final result of the algorithm includes the global best solution.Algorithm 3: Dynamic Sub-Swarm Approach of PSO Algorithms for Text Document ClusteringThe sub-swarm Equation , the subEquation ). The suEquation  and assiThis section shows the details of benchmark data sets and experimental conditions that were used for this experiment.In this study, six different benchmark machine learning data sets were used that are constructed from BBC sports news , 20 newsAs shown in In this study, the population size of each SI algorithm is assigned as 10. Here, for each algorithm, we performed experiments for the iterations such as 10, 20, 30, …, 100. Five simulations are used for each iteration number to handle the stochastic results of algorithms. The performance of this algorithm is evaluated using the evaluation metric purity and these results were compared with the algorithms K-means and PSO.In this experiment, our proposed approach is applied with a different number of abandoned particles with the least solutions such as 1, Here, to compare the performance of PSO and proposed subswarm-PSO, the same default parameters terature ,25.This section shows the experimental results of finding the optimum number of abandoned least solutions of particles and the comparative results of the proposed subswarm-PSO algorithm with the optimum number of abandoned particles with the least solution for the text document clustering problem.This section shows the experimental results of a different number of abandoned particles of PSO for the subswarm-PSO approach to find the optimum number of abandoned particles with the least solution. Here, we experiment and compare with 1, This experiment includes the following.1 particle with the least solution will be abandoned and reinitialized randomly. The remaining Here, Here, The results from This section shows the performance comparison results of the proposed subswarm-PSO algorithm with an optimal number of abandoned particles The comparative results and average running time of all algorithms for the data sets 1 to 6 are shown in the Based on the above results, we can say that the proposed algorithm subswarm-PSO is performing best to find the optimal solution for most of the data sets by comparing standard PSO and K-means algorithms in text document clustering. The average running times of this proposed subswarm-PSO algorithm are The standard PSO algorithm has more ability to solve complex optimization problems such as text document clustering but is usually trapped into the local optimum. Our proposed algorithm ignores half of the least solutions and reinitializes the particles which include the least solution in each iteration. This proposed approach increases the global search capability of PSO and enhances the performance of the PSO algorithm. During each iteration, only half of the particles which include the best solutions are moved to the next solution, and others are reinitialized randomly. Random initialization takes less time compared with updating particles to the next position. This might be the reason for the reduced execution time of our proposed algorithm.In our proposed method, we choose half The standard PSO algorithm is more effectively solving complex problems like text document clustering compared with other standard SI algorithms and traditional K-means algorithms. Still, PSO needs to be improved to avoid the problems such as premature convergence and local optima. In this study, we propose a subswarm-PSO algorithm to increase the global search capabilities of PSO and avoid the local optimum to improve the results for text document clustering. Our proposed algorithm is to select potential solutions and re-initialize particles dynamically. Out of a number of experimental observations, selecting half of the best solutions in each iteration has provided performance achievement. The results of this proposed approach were compared with the standard PSO algorithm and the K-means algorithm. Here, evaluation metric purity is used with six benchmark data sets. The experimental results show that our proposed algorithm subswarm-PSO performs best and uses less execution time than the standard PSO algorithm. Previous literature shows that the PSO is also performing well in a few other problems such as clustering, feature selection, and scheduling . Our pro