In the platform gravimeter, the stabilization accuracy of the gravimetric stabilization platform is crucial to improve the accuracy of gravity value measurements due to its uncertainties, such as mechanical friction, inter-device coupling interference, and nonlinear disturbances. These cause fluctuations in the gravimetric stabilization platform system parameters and present nonlinear characteristics. To resolve the impact of the above problems on the control performance of the stabilization platform, an improved differential evolutionary adaptive fuzzy PID control (IDEAFC) algorithm is proposed. The proposed enhanced differential evolution algorithm is used to optimize the initial control parameters of the system adaptive fuzzy PID control algorithm to achieve accurate online adjustments of the gravimetric stabilization platform’s control parameters when it is subject to external disturbances or state changes and attain a high level of stabilization accuracy. The results of simulation tests, static stability experiments, and swaying experiments on the platform under laboratory conditions, as well as on-board experiments and shipboard experiments, all show that the improved differential evolution adaptive fuzzy PID control algorithm has a higher stability accuracy compared with the conventional control PID algorithm and traditional fuzzy control algorithm, proving the superiority, availability, and effectiveness of the algorithm. The Earth’s gravitational field is one of the essential fundamental physical fields of the Earth ,2. GraviAs shown in Then, the gravity measurement error can be expressed as:In Equation (1), the second term is a steady-state error, which exists even if the carrier does not move.If the gravity static measurement accuracy of 1 mGal is to be achieved, the horizontal misalignment angle must be less than 4.8′ when the first item in Equation (1) is ignored. The magnitude of the first term in Equation (1) depends on the horizontal acceleration of the carrier and the extent of the error angle of the platform. If the gravity sensor’s damping mechanism is not considered, the maximum dynamic gravity measurement error should not exceed 1 mGal. When the horizontal acceleration of the carrier is As large-weight instrumentation, a gravimeter requires a gravimetric stabilization platform with a hefty load capacity (more than 30 kg). The related research mainly studies the control algorithm of a small load, three-axis or airborne stabilization platform. There is less research on the control of high-precision stabilization platforms with enormous load capacities. Domestic and foreign research scholars have conducted relevant studies to reduce the stabilized platform’s control system error and improve the stabilized platform’s stabilization accuracy. In Ref. , a robusThe gravimetric stabilization platform in this paper was developed in the Marine/Aerospace Gravimeter project, which is a primary national science and technology project with a two-axis structure (pitch axis and roll axis). The mechanical structure is shown in Inertial coordinate system Earth coordinate system Geographic coordinate system Carrier coordinate system Outer frame (roll frame) coordinate system The inner frame (pitch frame) coordinate system The relevant coordinate system is defined as follows .InertiaThe structure of the gravimetric stabilization platform was designed so that the IMU is fixed directly under the gravimeter base. When the carrier moves or is otherwise disturbed, the inner and outer frames will shake slightly, resulting in the stabilization platform not coinciding with the local geographic level. Let where The angular velocity of the transverse roll axis for the geographical coordinate system is obtained based on the relevant coordinate transformations.Similarly, the angular velocity of the pitch axis for the geographical coordinate system can be obtained.To maintain the gravimetric stabilization platform plane at all times within the local geographic level, the angular velocity of the inner frame relative to the geographic coordinate system must be 0 after the initial righting, and the following conditions must be met.Equations (3) and (4) can be obtained by the two torque motors to produce the angular speed, meeting the following conditions.When the control result satisfies Equation (5), the gravimetric stabilization platform table surface will remain parallel to the local geographical level.The servo control system is the core of the gravimetric stabilization platform. Additionally, it ensures algorithm implementation and platform stability accuracy, so a scientific and reasonable control system structure is essential. The gravity measurement stable platform’s hardware structure consists of a pitch axis and a roll axis with the same design, without considering the influence of coupling factors between the two axes. Only one of the axes can be analyzed when analyzing its control law, and the pitch axis is used as an example in the text with no further explanation. Given that the stabilized platform must install heavy equipment such as an IMU and a gravimeter and has stringent real-time requirements, its control system employs a multi-stage composite control scheme, which utilizes dual position and velocity rings to achieve composite control. Where The gravimetric stabilized stage is a servo-controlled system with strong nonlinearity. Due to its uncertainties, such as random disturbances, friction, mechanical coupling, and gyroscopic drift, the exact mathematical model of the stabilized platform cannot be determined. In contrast, fuzzy control is a kind of control based on fuzzy rules, which is used to realize the control of a nonlinear system without a precise mathematical model . Figure Fuzzy control can adjust the torque motor’s control voltage according to the fuzzy rules. However, its steady-state accuracy generally does not meet the accuracy requirements of the gravimetric stabilization platform. Therefore, the control of the stabilized platform is usually realized by the PID controller, which can achieve high control and stabilization accuracy with good real-time accuracy and strong robustness when it has suitable control parameters. To meet the requirements of high precision and robust system stability, a combination of fuzzy control and PID control can realize the real-time adjustment of the control parameters using the control system according to the error and the differential of the error. Then, the adaptive fuzzy PID control of the system can be determined.The accuracy of the initial parameter-setting of adaptive fuzzy PID control significantly impacts the control performance. In the conventional adaptive fuzzy PID control, the selection of initial control parameters is often determined based on experience or traditional parameter tuning methods, and its accuracy is inadequate. The differential evolution algorithm is an intelligent algorithm that can perform automatic optimization searches. An adaptive differential evolution algorithm based on a multivariate strategy with variable parameters (ADE-MMVP) is attempted to optimize the initial control parameters of an adaptive fuzzy PID control and realize the accurate control of a stable platform.(1)The ability to deal with nonlinear, non-differentiable, and multimodal functions;(2)The ability to process intensive cost functions in parallel;(3)It is easy to use and can realize algorithm optimization while making good use of control variables;(4)Good convergence, which can converge to the optimal global value in continuous independent tests.Differential Evolution (DE) was proposed by Professor Rainer Storn and Professor Kenneth Price in 1995 ,19. DE iVariation strategy of ADE-MMVP algorithmDue to the single variation strategy, fixed variation factor, and crossover probability factor of the conventional differential evolution algorithm, the algorithm is prone to slow convergence and failure to converge to the global optimum. To solve this problem, we propose an ADE-MMVP algorithm with multiple variation strategies, various factors, and crossover probability factors that can adaptively change with population evolution .Variati2.Variation factor of ADE-MMVP algorithmAmong all DE variation strategies, DE/rand/1 is the most widely used and most beneficial for maintaining population diversity, as in Refs. ,24 DE/be3.Crossover probability factor of ADE-MMVP algorithmThe variation factor The crossover probability factor To solve the problem regarding the accuracy of the initial control parameters of the adaptive fuzzy PID controller,  ADE-MMVPIn the platform mathematical model, the highest numerator and denominator of the speed loop transfer function are the same, and PI control is used in the speed loop control system to ensure the system’s stable operation. The structure of the speed loop control system is shown in Structure of the position loop control system is shown in When the system error When the system error When the system error Set several rules for PID controller control parameter adjustment according to the control system requirements.The response of the speed loop under the two signals is given in The experimental results show that the IDEAFC algorithm is superior.Based on the simulation test results, this chapter presents the platform experiments that were conducted to further verify the algorithm’s superiority. The stable platform control system had a TMS320F28335 digital processor as the control core. The traditional PID control algorithm, AFC algorithm, and IDEAFC algorithm were used in laboratory static and dynamic rocking experiments . The electronic self-collimator and IMU data were collected to control each algorithm. The effect of each algorithm was compared by collecting data from the electronic self-collimator and IMU. The control effects were compared by collecting data from the electronic self-collimator and the IMU. The IDEAFC algorithm was also used for onboard experiments and shipboard experiments, and the control effects were analyzed using the data collected from the IMU.The gravimetric stabilization platform was subjected to a shaking table static stabilization experiment in a laboratory environment. The angular offsets in the investigation were collected from IMU attitude data and TriAngle electronic self-collimator data, respectively. The physical objects of IMU and the electronic self-collimator are shown in In the static experiment, the angle measured by the electronic self-collimator is the angle deviation value of the overall stabilized platform, which can truly reflect the overall accuracy of the forum. In contrast, the angle deviation value measured by the IMU can reflect the control accuracy of the control algorithm. GNSS positioning-assisted calibration was used throughout the experiment to avoid the influence of measurement errors caused by the IMU’s own drift in the test results.When running the tests, multiple experiments were conducted, utilizing the conventional PID method, the AFC algorithm, and the IDEAFC algorithm. The set closest to the mean value was chosen as the outcome. As seen from the comparison in The data collected by the electronic self-collimator and IMU were compared separately to study the relationship between the overall stability accuracy and control accuracy of the platform. Since the electronic self-collimator is a high-precision optical goniometer, its reflector is small and fixed, so it can only be used under static test conditions. The relationship between the platform’s overall stability and control accuracy were evaluated from the comparison experiment under static conditions, and then used to analyze whether the stability accuracy of various algorithms meets the requirements.In the static experiment, the pitch angle and the roll angle measured by the electronic self-collimator are shown in A comparison of the control effects of the three control algorithms can be obtained from the comparison chart, as shown in The improvement in accuracy was calculated as follows.Relative comparison algorithm precision improvement ratio = /comparison algorithm precision.The dynamic swaying experiment of the gravity measurement stable platform was conducted in the laboratory using a swaying table with adjustable longitudinal and transverse swaying amplitudes. The maximum value of longitudinal swaying was ±45°, and the total value of transverse swaying was ±30°. In the natural gravity measurement environment, whether vehicle-mounted, on the lake, at sea, or under the sea, the swaying angle generally did not exceed ±10°. To test the stability of the platform, when conducting the platform swaying experiment, the angle of both longitudinal and transverse swaying was set to ±15°. At the same time, to avoid the impact caused by the different swing speeds, the multi-grade swing experiment was conducted separately. The swing speed of each gear was different, and the period setting for each pack is shown in The comparison chart shows that the control effect of the algorithm is the same under different shaking table operation gears, so the impact of other gears on the control algorithm were not considered in the subsequent comparison test.A comparison of the data measured by IMU under the three control algorithms during the longitudinal rocking test is shown in The comparison graphs show the control effects of the three control algorithms in the longitudinal rocking experiment, as shown in The comparison of data under the three control algorithms was measured by IMU during the swing test, as shown in Using the comparison graphs, we can compare the control effects of the three control algorithms in the swing experiment, as shown in The gravimetric stabilization platform was tested in-vehicle to verify the stability performance of the IDEAFC algorithm in real application scenarios. The designed route in the experiment was Wuhan city road, the vehicle driving speed was 30 km/h, and an onboard experimental environment was built, as shown in The vehicle-based experimental driving city route was as shown in The pitch and roll angle data were collected during the experiment, as shown in During the experiment, the standard deviation of the pitch angle of the gravimetric stabilized platform was 7.56″, and the standard deviation of the roll angle was 7.09″, both within 10″.A gravimetric stabilization platform using an IDEAFC algorithm was used for a boat experiment conducted at Mulan Lake, Wuhan, using a small electric pleasure boat. The boat is shown in The experimental waters contained light waves, with a wind speed of about 1 m/s; the speed of the boat in the lake was 15 km/h. From the docked experimental dock, sailing experiments were conducted around Mulan Lake; the whole route map is shown in The pitch and roll angle data obtained throughout the cruise are shown in During the whole shipboard experiment, the standard deviation of the gravity measurement stabilized platform pitch angle was 2.11″, and the standard deviation of the roll angle was 3.69″, both within 10″.An improved differential evolutionary adaptive fuzzy PID control method was proposed to reduce the influence of multi-source perturbations on the gravimetric stabilization platform, achieve high precision control, and improve stabilization accuracy. As a comparison, conventional PID and adaptive fuzzy control were also applied to the platform control. Simulation experiments and platform tests were carried out for different algorithms. The experimental results show that the improved differential evolution algorithm can significantly improve the stabilization accuracy of the gravimetric stabilization platform. In the laboratory swing experiments, the control accuracy was enhanced by more than 25% under static conditions and more than 18% under dynamic conditions. In the onboard and shipboard experiments, the improved differential evolution adaptive fuzzy PID control algorithm pitch and roll angle standard deviation were within 10″. This lays a foundation for subsequent gravity measurements under static or dynamic conditions and creates the necessary conditions for the realization of high-precision dynamic gravity measurements.


Health monitoring of structures operating in ambient environments is performed through operational modal analysis, where the identified modal parameters, such as resonant frequencies, damping ratios and operation deflection shapes, characterize the state of structural integrity. The current study shows that, first, time-frequency methods, such as continuous wavelet transform, can be used to identify these parameters and may even provide a large amount of such data, increasing the reliability of structural health monitoring systems. Second, the identified resonant frequencies and damping ratios are used as features in a damage-detection scheme, utilizing the kernel density estimate (KDE) of an underlying probability distribution of features. The Euclidean distance between the centroids of the KDEs, at reference and in various other cases of structural integrity, is used as an indicator of deviation from reference. Validation of the algorithm was carried out in a vast experimental campaign on glass fibre-reinforced polymer samples with a cylindrical shell structure subjected to varying degrees of damage. The proposed damage indicator, when compared with the well-known Mahalanobis distance metric, yielded comparable damage detection accuracy, while at the same time being not only simpler to calculate but also able to capture the severity of damage. Structures made of conventional materials are being extensively replaced by fibre-reinforced polymer (FRP) composite materials in the aerospace, automotive, energy and other industries owing to their superior specific strength , negligiOutput-only SHM involves the detection and possible characterization of damage by analysing only the response signals collected from sensors mounted on the structure. Structural excitation can be provided if conditions for operational modal analysis (OMA) are fulfilled . In pracTime-frequency analysis methods, such as short-time Fourier transform, Wigner–Ville distribution and Hilbert–Huang transform and wavelet transform (WT) , among oAfter the extraction of DSFs, an anomaly detection algorithm is employed to identify outliers supposedly originating from damage or changes in environmental conditions . A populBuilding on the concept introduced in , the aimThe current study is based on the concepts described in detail in . WaveletA quick recap of the methodology from  is givenCWT on the recorded response signals Wavelet ridges . Then, an operator estimates the modal parameters of the structure. The resulting parameters together with the possible external operational factors, such as static loads, temperature or speed of rotation (blades), are input to a modal passport ,22, wherThe damage detection algorithm proposed for an SHM system originates from an anomaly detection field. It has three distinct phases, namely, Phase I: signal collection, Phase II: feature extraction, and Phase III: statistical control, as shown in Vibration signals as a response to structural excitation are measured with In Phase II, a time-frequency analysis of the averaged responses is performed using CWT. CWT analysis involves the identification of wavelet ridges from the ridge condition in Equation (2) in the time domain signal analysed. Wavelet ridges represent the oscillatory modes, which comprise the components of vibration decay signals. Subsequently, this ridge information from the ridges identified at Phase III is concerned with performing statistical control of the feature values. The first stage is the feature-value filtering, which is carried out by using the interquartile range (IQR) rule. The goal is to remove the outliers from the features. Unlike the threshold set to mean plus/minus two or three standard deviations, the IQR approach for outlier removal is appropriate for data that does not necessarily follow a normal distribution. The frequency values for the ridges identified at The next step involves exploring the underlying probability distribution of the filtered features. For this purpose, the kernel density estimate (KDE) is computed. The reason is that a kernel distribution representation of the probability density function (PDF) of the data does not make any assumptions on the underlying distribution. The KDE is defined by a smoothing function and a bandwidth value that controls the smoothness of the resulting density curve. The kernel density estimator of the data at hand  for each value of the bandwidth parameter. Then, compute an error Calculate the cross-validation error as a mean-squared-error of the errors in Equation (9) across all folds for each value Find the optimum bandwidth parameter by calculating the minimum of these cross-validation errors across all bandwidth valuesAfterwards, the KDEs with these optimized bandwidth parameters are calculated for reference and all monitoring cases. Then, a centroid value of the KDE for both features is calculated according to6.Consider all available structures of the same type at their reference state.7.Perform a modal parameters estimation to form feature vectors and calculate their centroid values for each structure.8.Calculate the Euclidean distance between centroid values in all possible combinations of structure pairs.9.Calculate the median value of these Euclidean distances and confidence bounds asOnce the Euclidean distance between the reference and all monitoring cases is calculated, a threshold value is established according to the following scheme:The algorithm proposed is validated on the modal parameters extracted from five glass-fibre-reinforced polymer composite specimens with a cylindrical shape manufactured in the scope of an SHM system prototype research project. Cylindrical structures mimic the structural components of serial production, such as a helicopter tail boom, for which the current anomaly detection algorithm is intended.2 fibreglass fabric with a fibre orientation of 45° and LG 385 epoxy resin (HG 385 hardener). The weight of the specimen with the upper and lower flanges is 4.37 kg. Photo of a specimen is shown in Item 1—composite cylinder made of fiberglass and epoxy resin;Items 2 and 3 —top and bottom annular flanges for cylinder fixation made of laminated plywood (30 mm thickness), respectively;Item 4—a network of 48 piezoelectric strain sensors;Item 5—wires connecting the sensors;Item 6—4 D-SUB type connectors at the places for connector fastening.Testing objects are the structures in the form of cylinders fabricated from a composite material with the flanges made of plywood rings. The specimens are made of 300 g/mThe dimensions of the specimen are as follows: nominal diameter of 300 mm, nominal length of 710 mm (with the flanges of 773 mm) and wall thickness of 1.45 ± 0.05 mm.Each specimen includes its own sensor network, which outputs the signals during testing to the measuring system, providing a registration and storage of the signals.The sensor network of each specimen comprises 48 polyvinylidene fluoride piezo film sensors connected to four conductor terminals and wiring harnesses with connectors. These films are flexible, lightweight and have piezoelectric properties . Due to After the instrumentation installation, each specimen is fixed on a U-shaped modal testing stand, which, in turn, is mounted on a vibration isolation base see . StructuAll test specimens manufactured were visually inspected to check for defects or damage, as well as their compliance with the specifications. Local deviations and small differences between the specimens were identified to be mainly due to the factors of hand-made technology. These factors include uneven filling of the specimens and flange joint, resin leaks on the specimen surface, air bubbles in the protective layer, undercut on the flange of specimen No. 5, slightly different location of the first hole on the flange for each specimen, resin pouring out at different locations on the inner and outer surfaces of specimens, and differences in wire layouts for sensors.The test cases realized in the current study involve reference state and progressing damage, namely, a circular hole drilled through the thickness of the cylinders in the same location. Diameter of the hole is increased in five stages as follows—4, 8, 16, 24, and 32 mm. The location of the hole schematically is shown in An example of a response signal (specimen No. 1 from measurement channel 1) at a reference state is shown in CWT scalograms at the reference state for the averaged time domain responses are shown in The estimation of resonant frequencies from wavelet phase is shown in The procedure of damping ratio estimation is shown in The results of the frequency filtering for the vibration mode at 185 Hz  at a reference state for specimen No. 2 are illustrated using histograms in After feature filtering, the next step is computation of the KDE of the filtered features. It was shown that feature values do not follow a clear normal distribution; therefore, KDE is an appropriate tool for the estimation of the underlying probability density. In the bandwidth optimization routine, bandwidth values were set from 0.01 to 1, with a total of 100 values for testing. The bandwidth parameter optimization results for specimens at the reference state are shown in The optimized KDEs for a specimen No. 1 are shown in The threshold values for the Euclidean distances of centroids at reference are estimated from all five cylindrical specimens. Hence, the total number of combinations is 10 if that particular vibration mode is identified for all structures. The Euclidean distances calculated for three modes of vibration at 185, 176 and 106 Hz are shown in The median values and their confidence bounds according to Equation (14) of the calculated Euclidean distances are presented in The damage detection results are displayed in Limitations of the algorithm proposed are directly associated with the quality of the vibration signals recorded, which, in turn, strongly depend on the noise robustness of the sensors to the environmental conditions in which the structure is operating. Secondly, the algorithm does not consider the operational deflection shapes (ODSs) as an additional feature. It is anticipated that information on the ODSs could potentially enhance the discriminative power of the algorithm. It is possible to identify the ODSs with continuous wavelet transform. However, it would significantly increase the complexity of the algorithm, which was not the aim of this study. Thirdly, damage detection accuracy depends on an accurate threshold estimation. Obviously, according to statistics, a larger sample of structures would have yielded a more accurate threshold value for the Euclidean-distance damage indicator. However, there is a trade-off of accuracy and number of structures to be manufactured for such threshold calculations involving increased financial and time resources.χ2) distribution [χ2 cumulative distribution function with υ degrees of freedom and a selected probability P: The damage identification results obtained were compared to the well-known Mahalanobis distance (MD) metric. The MD values follow a chi-squared .The vibration mode at 185 Hz is the least favourable for damage detection (accuracies are low for both methods), while both other modes are roughly equal in this regard. Overall, the accuracies are comparable between both methods. In cases where there is 100% accuracy for the Euclidean distance, the corresponding accuracy for the Mahalanobis distance is usually lower, since not all observations have passed the threshold. The advantages of the method proposed are that computation for the Euclidean distance is simpler and requires less computational power since the covariance matrix does not need to be computed.The Euclidean distance of the centroids of the modal features KDEs between the reference and damage states can be used to detect damage.The damage indicator proposed shows an upward trend for damage progression, meaning that it is effective in detecting increasing severities of damage.Some vibration modes are more sensitive to damage than others. Therefore, multiple vibration modes have to be identified in order to increase the reliability of the damage detection. For example, features originating from the vibration modes at 106 and 176 Hz have significantly higher deviations from the reference than the vibration mode at 185 Hz. Therefore, these vibration modes were more effective in damage detection. On the other hand, these vibration modes could not be identified for all damage cases, while the vibration mode at 185 Hz was present in all scenarios.There is a significant scatter of the feature value deviations from reference among the test samples. For the most part, this is due to inconsistencies in the sample design and instrumentation, as mentioned in The damage indicator proposed was compared to the Mahalanobis distance metric for damage detection. Both methods yield comparable damage detection accuracy. Therefore, there is no reason to use a more computationally costly Mahalanobis distance approach.In the current study, an anomaly detection algorithm based on output-only structural vibration responses of structural components was proposed. The algorithm detects changes of modal parameters caused by structural degradation, such as the progression of damage. Phase I deals with the acquisition of vibration response signals from the sensors mounted on the structure and subsequent signal averaging and fusion. Phase II uses the modal parameters as the features identified with continuous wavelet transform routine. Phase III is carried out for decision-making regarding the state of integrity of the structure in question based on the statistical descriptors of the extracted features. Here, the feature filtering scheme based on IQR Rule is adopted to remove outlier feature values. Feature filtering revealed that the distribution for some specimens is, in fact, bimodal, while for others it is skewed and not classical Gaussian. The probability density function of the filtered features is estimated using a kernel density estimate (KDE) to avoid the assumption that the underlying distribution is normal. The damage indicator is based on the Euclidean distance between the centroid of KDE for both features at reference state and any state of damage. The following can be concluded:Future research will be devoted to the consideration of an influence of environmental and operational factors on the structural characteristics and modal parameters for a practical SHM system. To reduce the influence of the ambient temperature, for example, some papers consider methods for suppressing this effect as interference , in othe


Software defined networking (SDN) improves the flexibility and programmability of the network by separating the control plane and the data plane and effectively realizes the global control of the network infrastructure. However, the centralized structure design of SDN exposes the controller to potential threats. Attackers have used the active flow table delivery mode to launch distributed denial of service (DDoS) attacks on the SDN controller, resulting in the controller failure and seriously affecting the network performance. To overcome this problem, this paper proposes a defense framework called CC-Guard. The framework consists of four modules: attack detection triggering, switch migration, anomaly detection, and mitigation. Among them, the attack detection trigger module improves the system’s timely response to DDoS attacks. The switch migration module effectively unclogs the controller congestion problem and provides convenience for network flow transmission. The anomaly detection module uses a coarse-grained method for two-stage detection, which improves the detection accuracy. The mitigation module uses the idea of cross-domain cooperation of the controller to clear the abnormal flow in the blacklist. Experimental results show that our proposed CC-Guard has real-time DDoS attack defense capability and high detection accuracy, as well as efficient network resource utilization. In recent years, with the rapid development of internet technology, the distributed control and management mode of the traditional network makes it difficult to upgrade to new network functions, resulting in the difficult implementation of network innovation . As a neThe flexible network architecture of SDN provides convenience for network management, but its security problems have become increasingly prominent. OpenFlow  is a souAt present, there are many DDoS attack detection and defense methods for SDN ,6,7. HowIn this paper, we propose a multi-domain comprehensive cooperative guard framework called CC-Guard to defend against DDoS attacks at the control layer. CC-Guard is divided into four modules: the attack detection trigger module, the switch migration module, the anomaly detection module, and the mitigation module at the control layer. When the ratio of the number of packet_in messages in the SDN network to the processing capacity of the controller is greater than a certain threshold, the system starts the switch migration module. The system selects the best target controller from the neighborhood and migrates some switches in the overloaded control domain to the target control domain to avoid controller overload in the attacked domain. Then, the anomaly detection module uses a two-stage detection method to detect the DDoS attack flow. In the first stage, it determines whether there was a DDoS attack based on the packet_in message characteristics. If the anomaly is detected, the DDoS attack flow is judged by the second stage of the deep learning method based on flow characteristics. Finally, the mitigation module deploys the abnormal flow blacklist in the controller and informs the neighborhood controller cross-domain through the east–west interface of the SDN controller, and then the attack host is isolated by issuing the flow rule command to prevent the attack from happening again. The following are our contributions with this paper.This paper proposes a multi-domain controller cooperation defense framework, CC-Guard, against DDoS attacks on SDN controllers. The mechanism includes the attack detection trigger module, the switch migration module, the anomaly detection module, and the mitigation module.In this paper, the DDoS attack detection trigger module is proposed to overcome the limitation of the fixed period detection method and realize adaptive fast detection.To prevent controller failure and impacts to the the operation of the detection module, this paper proposes the switch migration module for the first time to provide enough CPU computing resources for the subsequent start-up of the detection module.To improve the DDoS attack detection accuracy and reduce the false positive rate, this paper proposes a two-stage attack detection method. In the first stage, the entropy value of the source IP and destination IP is used for the coarse-grained judgment of whether the DDoS attack exists. If it exists, the CNN-GRU-Attention deep learning model is used for fine-grained detection of the DDoS flow. However, as DDoS attacks become more intelligent, the entropy method is not foolproof.To effectively mitigate DDoS attacks, this paper proposes a DDoS mitigation mechanism for attack blocking and flow cleaning. The SDN characteristics (query network topology and information of each switch) were used to identify the attack path and source of the attack flow. Finally, the OpenFlow protocol is used to send the flow table to the attack edge switch to clear the attack flow.The rest of this paper is arranged as follows. SDN architectures are becoming a new network architecture of interest in academia and industry because of the manageability, flexibility, and programmability they provide to network infrastructures, enabling operations and maintenance staff to efficiently perform network monitoring, resource allocation, and dynamic provisioning. At the same time, the SDN architecture introduces many new types of security threats to the network. In particular, the problem of controller failure due to SDN-based DDoS attacks is of great concern to researchers. Researchers have proposed many solutions to mitigate DDoS attacks, mainly in terms of attack detection and SDN defense architecture design.To date, a large number of SDN-based DDoS attack detection methods have been proposed, mainly based on the analysis of network flows. Nam et al.  combinedSDN-based DDoS attack defense schemes are considered from the overall architecture of SDN, such as adding additional devices in the data plane and control plane or using nearby idle network devices in the domain to alleviate the impact of SDN DDoS attacks so that the network can work normally. Shin et al.  proposedTo sum up, the methods to prevent DDoS attacks against SDN controllers still have the following shortcomings. First of all, most detection and defense methods only consider how to improve the detection efficiency, without considering the time delay when the controller starts the detection module, which will cause a large number of abnormal flow packets to congest the controller before the controller detects them, affecting the normal work of the controller. Secondly, most DDoS attack mitigation methods based on SDN controllers only consider the distinction between normal and abnormal flow, without considering the situation of sudden normal flow, which can easily lead to a high false detection rate. Finally, most works in the literature only consider how to detect DDoS attacks and do not consider how to effectively alleviate abnormal flow. To solve the above problems, this paper proposes the CC-Guard defense framework from the perspective of multi-domain controller cooperation, which can not only ensure that the controller has enough computing resources to drive the detection module, but also reduce the false positive rate of DDoS attacks and launch effective mitigation measures against DDoS attacks.The DDoS attacks in SDN are different from traditional network attacks. In OpenFlow networks, attackers can exploit the communication overhead between the data layer and the control layer to launch attacks. Attackers use malicious software to invade one or more network hosts and launch TCP/UDP flood attacks on the attack targets through the invaded hosts to generate a large number of fake data packets  that falsify fields in the OpenFlow network in a short time. However, these table_miss packets fail to match the flow rules in the switch, which triggers the switch cache to send a large number of packet_in messages with table_miss packet header information to the controller. To make things worse, the OpenFlow specification v1.4 , with a limited output range and stable optimization. Finally, the softmax function is used to classify the network flow.A.CNN layerThe convolutional neural network, as a feedforward neural network, uses the idea of local correlation and weight sharing to reduce neural network parameters and achieve higher training efficiency. CNN is mainly composed of three parts: convolutional layer, pooling layer, and fully connected layer. The convolution layer extracts the feature information of the data by performing correlation operations on the matrix of each channel through the convolution kernel and increases the expression ability of the model through the non-activation function. The convolutional layer Equation is as follows:ith feature in the output of layer In Equation (19), The role of the pooling layer is to reduce the size of the feature map and simplify the complexity of network calculation. At the same time, the features are compressed to retain salient features and prevent overfitting. Pooling layers are usually sandwiched between successive convolutional layers. The function of the fully connected layer is to integrate the features calculated by the convolutional layer and the pooling layer into the sample space. Fully connected layers are usually placed at the end of the convolutional neural network architecture.B.GRU layerAfter multiple convolutions and pooling to extract more accurate frequency and spatial domain features, the GRU layer is used as the input data of the GRU network. Among them, the update gate of GRU is close to 0, which means that some information about the hidden state of the previous layer is forgotten in this hidden layer. The update gate is close to 1, indicating that this hidden layer continues to be retained. When the reset gate of GRU is close to 0, it means that some information at the previous time is forgotten in the current memory content. The reset gate is close to 1, indicating continued retention in the current memory content. Then, the two pieces of information are added and passed through the tanh activation function to normalize the result to between . Therefore, the memory content of this moment is composed of two parts: one is the reset gate to store the important information related to the past, and the other is the important information input at the current moment. These two parts compose all of the memory content of the current moment to extract the stream feature. The structural diagram of the GRU unit is shown in Two gates are obtained by the last transmission state In Equations (20)–(23), C.Self-attention layerFor each flow, not all bytes are equally important for classification. To make the relatively important information in the stream have a strong influence on the classification results, this paper introduces the attention mechanism. In the attention mechanism, the core operation is the calculation of the weight of each element. In this paper, the feature vector In Equations (24)–(26), Extract the information. Once the attack flow is detected, the attack source IP, destination IP, switch ID, switch ingress port, and the timestamp when the switch sends packet_in messages to the controller are extracted from the flow table and put into the blacklist in the controller.Locate the attack source. The flow data in the blacklist are grouped according to the source IP and destination IP fields, and then each group of flow data is sorted in descending order according to the timestamp field to find the smallest timestamp flow information and locate the attack host according to the switch ID and the switch ingress port in_port. If the attacking host exists, proceed to the next step. If the host does not exist, the neighbor controller is notified through the east–west interface of the controller, and the flow table information of the same source IP and destination IP field is searched in the edge switch, and the switch and switch port in_port that the attack host is connected to are located. The pseudocode of the algorithm is shown in Algorithm 3.Flow rule generation. The flow rules are constructed with the matching field of in_port = the port to which the attack host is connected and the Action domain: Drop.The flow table is issued. The flow rules constructed in step 3 are issued to the switch connected to the attacking host to achieve the isolation of the attacking host.Algorithm 3: Attack source location algorithmInput: blacklist, network topology;Output: the edge switch port to which the attack source is connected(1) Extract_features  from the blacklist (2) Results = Search (SrcIP & DstIP)(3) Infor1 = Sort_stamp (Results)(4) Infor2 = Extract_max_stamp (Infor1) (5) Extract switch_ID & in_port from infor2If switch[i].inport is connected with a host (6) then the connected host is the attack host(7) else inform the other neighbor controllers about this flow and repeat step 2~7(8) end If(9) In the cross-domain scenario, an attacker can construct many attack packets using the forged IP addresses or MAC addresses, so that the switch generates packet_in messages from the attacking host to the target host. Finally, many packet_in messages are gathered in the controller, causing the controller to fail. As shown in This experiment was conducted in the environment of Mininet , RYU 2929 was seThis experiment was simulated on Mininet, and the cbench  tool wasThrough the switch migration strategy in this paper, we determined that the optimal migration scheme was as follows: migrating switch S1 under controller C1 to controller C2. To verify the superiority of the proposed switch migration strategy, it was compared with the EOSDM method  and DPCLIn this paper, in order to effectively distinguish DDoS attacks in the SDN control layer, a two-stage attack detection method was used. In the first stage of detection, the experimental design was carried out according to In the first stage, the method of information entropy was used for coarse-grained detection. By calculating the entropy value of each feature vector in several windows, whether an anomaly occurred in the network was judged according to whether the entropy value exceeded the threshold. Firstly, the source IP address and the destination IP address of the packet_in messages collected under flash events and DDoS attacks comprised 50,000 data flows each. Then, the feature entropy values within each window  were calculated by Equations (11) and (12). Finally, the confidence intervals of the entropy of source IP and destination IP were obtained by Equation (13), as shown in Through the above thresholds between the source IP entropy and the destination IP entropy, we could determine whether an anomaly occurred. If it were a flash event, the data flow would be forwarded normally. Otherwise, the system would adopt a deep learning approach for flow-based fine-grained detection.In the second stage, the deep learning CNN-GRU-Attention method was used to detect DDoS attack flows. The model structure and parameters are shown in AC), precision (P), recall (R), F1) and false positive rate (FPR) of the different classifiers were also analyzed : indicates the number of samples correctly judged by the detection model as a percentage of the total number of input samples.Accuracy rate (P): indicates the percentage of the number of samples that the detection model determines to be DDoS attack types that are actually DDoS attack types.Precision (R): indicates the number of samples correctly determined by the detection model to be DDoS attack types as a percentage of the number of samples of all DDoS attack types.Recall (F1): represents the summed average of precision and recall, enabling a more accurate assessment of model performance.F1-score (FPR): represents the percentage of the number of samples wrongly judged as the DDoS attack type to the total number of normal samples.False positive rate  classification algorithm, Adaboost classification algorithm, and CNN and GRU classification algorithms. The accuracy  metric. AUC is the area of the receiver operating characteristic (ROC) curve. The ROC curve refers to a graph where the horizontal axis is the false positive rate and the vertical axis is the true positive rate. The maximum AUC value is 1, and the minimum AUC value is 0. The closer the AUC is to 1, the higher the accuracy; If its value is around 0.5, the prediction is not ideal enough.Then, we compared the ROC with the AUC values for the five detection algorithms. Confusion matrix: The confusion matrix is a metric to judge the results of the model and is used to judge the merits of the classifier. It is composed of two dimensions, namely “actual” and “prediction”, and both are composed of “true positive (TP)”, “true negative (TN)”, “false positive (FP)”, and “false negative (FN)”, as shown in The confusion matrices of the five classification algorithms are shown in Aiming at the SDN architecture to avoid DDoS attacks, this paper proposes a security defense scheme, CC-Guard, with multi-domain controller cooperation. CC-Guard consists of four modules: the attack detection trigger module, the switch migration module, the detection module, and the mitigation module. The attack detection trigger module uses the controller’s ability to process packet_in messages as a prerequisite for triggering the CC-Guard system. This module can effectively reduce the time delay of the polling mechanism and execute the defense operation in time. In order to prevent controller congestion, the switch migration module can migrate the switches in the overloaded controller domain to the controller domain with idle resources, thus effectively avoiding controller overload and ensure the normal operation of the SDN network. The detection module uses a two-stage detection method. In the first stage, information entropy is used to distinguish flash events from DDoS attacks. Once an anomaly is suspected, the second phase of abnormal flow detection will be applied. In addition, to quickly detect attacks and detect them early, we use multiple IDS for parallel detection to improve detection efficiency. Finally, the mitigation module is started, the controller’s east–west interfaces are used for cross-domain traceability, and the OpenFlow protocol is used to issue flow rules to block the abnormal port of the switch. In order to effectively verify the defense effect of CC-Guard, this paper used Mininet and RYU controllers for simulation, and the experimental results verified the effectiveness of the CC-Guard defense system. In the DDoS attack environment, the system can not only accurately identify abnormal flow, but also effectively use the SDN controller for effective mitigation. Even if the SDN controller is congested, it can be guaranteed to work effectively.As DDoS attacks become more and more intelligent, it is not foolproof to distinguish DDoS attacks and flash events by using the information entropy method in this paper. Therefore, it is urgent to propose a favorable solution that can accurately identify DDoS attacks and flash events. In addition, the network topology experiment in this article was small-scale and could not validate multiple IDS collaborative parallel detection schemes. In future work, we will conduct large-scale topology experiments to verify their effectiveness, and plan to deploy a physical SDN experimental platform for further research, attempting to improve the versatility and scalability of the CC-Guard framework.


In recent years, the field of robotic portrait drawing has garnered considerable interest, as evidenced by the growing number of researchers focusing on either the speed or quality of the output drawing. However, the pursuit of either speed or quality alone has resulted in a trade-off between the two objectives. Therefore, in this paper, we propose a new approach that combines both objectives by leveraging advanced machine learning techniques and a variable line width Chinese calligraphy pen. Our proposed system emulates the human drawing process, which entails planning the sketch and creating it on the canvas, thus providing a realistic and high-quality output. One of the main challenges in portrait drawing is preserving the facial features, such as the eyes, mouth, nose, and hair, which are crucial for capturing the essence of a person. To overcome this challenge, we employ CycleGAN, a powerful technique that retains important facial details while transferring the visualized sketch onto the canvas. Moreover, we introduce the Drawing Motion Generation and Robot Motion Control Modules to transfer the visualized sketch onto a physical canvas. These modules enable our system to create high-quality portraits within seconds, surpassing existing methods in terms of both time efficiency and detail quality. Our proposed system was evaluated through extensive real-life experiments and showcased at the RoboWorld 2022 exhibition. During the exhibition, our system drew portraits of more than 40 visitors, yielding a survey outcome with a satisfaction rate of 95%. This result indicates the effectiveness of our approach in creating high-quality portraits that are not only visually pleasing but also accurate. Robotic portrait drawing has attracted considerable attention from researchers due to its fascinating challenges in both technical and creative domains. With the increasing adoption of collaborative robots, numerous robotic portrait drawing systems have emerged. However, achieving a balance between the quality of the output drawing and the efficiency of the drawing process has been a significant hurdle. Many approaches have prioritized either producing highly detailed drawings, which can be time consuming, or simplifying facial features to complete the drawing quickly.In this paper, we propose a novel approach that combines these objectives by leveraging advanced machine learning techniques and a variable line width Chinese calligraphy pen. The proposed approach adeptly balances the trade-off between drawing speed and output quality, thereby offering a unique solution. Our system emulates the human drawing process, involving sketch planning and creation on the canvas. Preserving facial features poses a major challenge in portrait drawing, which our system overcomes by utilizing CycleGAN, a technique that retains essential details such as the eyes, mouth, nose, and hair. The proposed system employs a human keypoint detection algorithm to identify and crop the dominant human face from a video feed. Subsequently, the CycleGAN algorithm is applied to perform style transfer, transforming the image into a black-and-white sketch. From this sketch image, line extraction and path optimization algorithms are utilized to generate optimized waypoints to guide a robotic arm during the physical tracing process. Finally, a six DOF robotic arm equipped with a Chinese calligraphy pen is employed to draw the portrait, dynamically adjusting the pen pressure to vary the stroke width.Drawing with variable stroke widths utilizing a Chinese calligraphy pen.Style transfer to variable-width black and white pen drawings using CycleGAN.Effectively balancing between time consumption and portrait quality.High level of satisfaction from the public demonstration volunteers.To assess the effectiveness of our system, we conducted extensive real-world experiments involving various volunteer groups. The results demonstrate that our system can produce high-quality portrait drawings with an average drawing time of 80 s while preserving most of the facial details. Furthermore, the system was showcased to the general public at the RoboWorld 2022 exhibition, where it successfully drew portraits of over 40 visitors, achieving a satisfaction rate of 95%. In contrast with existing methods, our proposed system offers the following key contributions:The remainder of this paper is organized as follows: The utilization of robotic systems for drawing and painting has gained significant attention among researchers in recent years, as demonstrated in various papers. For instance, ref.  presentsHowever, the use of machine learning in robotic portrait drawing poses several challenges. The primary difficulties include preserving fine details and achieving short drawing times. In previous methods, various drawing tools such as paint brushes ,22,23,24To overcome these challenges and improve the efficiency of robotic portrait drawing, researchers have increasingly turned to machine learning techniques. For example, Gao et al. in  employedTo provide a comprehensive understanding of our approach, we have included a detailed comparison section that compares our system to other research papers in the field of robotic portrait drawing. The comparison includes various aspects such as the robotic arm’s degree of freedom, drawing time, and the drawing tool used. We divided the comparison into two tables to provide a clear overview. The first In summary, our proposed system provides a solution to the challenges faced by previous methods in robotic portrait drawing. By utilizing advanced machine learning techniques and a path optimization algorithm, we can achieve efficient and detailed portraits. The comparison section provides an overview of the current state of the art in the field and highlights the advantages of our approach.The proposed system is designed to achieve the objective of generating high-quality portraits on a physical canvas using a four-module approach. The system consists of four main modules seen in To gain a deeper understanding of each module’s related work, separate sections will be dedicated to discussing their functionality, strengths, and limitations. Through this comprehensive analysis, the proposed system’s performance and potential for future improvements can be better evaluated.The incorporation of the OpenPose algorithm  is an obAdditionally, apart from its ability to crop images, we have implemented the OpenPose algorithm to retrieve the precise position of the eyes seen in Y). The mapping is designed to preserve the consistency of essential facial features, such as haircuts, face shapes, and eye shapes, while learning the relationship between the two domains. This enables the system to generate high-quality portrait sketches that accurately reflect the real face images, without the need for any labeled data or supervision.Our study introduces a novel system for generating sketch-style avatars from real face images, which utilizes the CycleGAN  to learnA significant advantage of the CycleGAN is its ability to handle a diverse range of facial features and styles, making it an effective tool for generating sketch-style avatars that preserve the consistency of the features, even when the real faces and avatars have significant differences in appearance. The flexibility of the CycleGAN allows for the generation of sketches that are faithful representations of the real faces, which makes the system ideal for use in various applications.Y represents the generated sketches. To convert face images to sketch images, we employ a generator network CycleGAN can be used in many situations due to its unsupervised approach, which enables it to be highly flexible and adaptable. The structure of the CycleGAN is depicted in D, To ensure the accuracy of the generated sketches, we use a drawing discriminator The primary objective of this module is to develop an efficient and accurate method for extracting generated sketch lines through the implementation of advanced machine learning techniques and optimized line extraction. In order to achieve this goal, we have concentrated on extracting the minimum number of lines necessary to retain all facial features. The extracted lines are then converted into waypoint data that can be utilized by the robot motion module. By utilizing this approach, we aim to create a highly effective method for generating portrait sketches that provides both precision and speed.In order to generate smooth and accurate drawing motions, it is necessary to employ a series of preprocessing steps on the sketch. One such crucial step is the application of morphological transformation to extract motion waypoints from the sketch. The extraction process involves simplifying the sketch into a structure of lines, which can be easily processed. To achieve this, two different morphological transform algorithms are used for skeletonization: the opening algorithm and the closing algorithm.The opening algorithm is primarily used to extract the majority of the sketch lines. This algorithm involves performing erosion followed by dilation, which helps to smooth pixel edges and eliminate isolated pixels. This step significantly simplifies the sketch structure and allows for better feature extraction in the subsequent steps. On the other hand, the closing algorithm is primarily employed to improve the accuracy of the system for eye sketches. This algorithm involves performing dilation followed by erosion, which fills small holes and provides better details. This step helps to produce more natural-looking portraits.The pixel-to-pixel Algorithm 1 was employed prior to extracting lines from a sketch image. This analysis was conducted on a down-sampled version of the cropped image, where the width was reduced to 400 pixels. This technique entails a search for the nearest pixel to the current pixel and appending it to a line array if it is connected. Pixels that are not connected are saved to a new line array. The pixel-to-pixel algorithm is highly effective and accurate when it comes to extracting lines from sketches, enabling their further processing in the system. The method facilitates the extraction of crucial details and features of the sketch, including facial features, while minimizing the number of lines and optimizing the processing time. It is a crucial step in the creation of a high-quality portrait sketch.By implementing the pixel-to-pixel algorithm, we were able to extract important features and details from the sketch image accurately and efficiently. The algorithm’s process of searching for connected pixels and adding them to a line array enabled the isolation of key features of the sketch, which could be processed further in the system. Additionally, the algorithm’s ability to identify non-connected pixels allowed us to save them to a new line array, ensuring that no important features or details were lost in the extraction process. This approach enabled us to extract critical facial features from the sketch image with minimal noise, resulting in a high-quality portrait sketch.In this proposed system, we have implemented a line clustering algorithm that categorizes lines based on their spatial proximity to one another with the aim of minimizing the number of lines that need to be drawn by the robotic arm. By recognizing clusters of lines that are situated in close proximity, we are able to merge them into a single line. This approach resulted in a reduction in the average number of lines drawn to 49%, leading to more efficient and faster production of high-quality drawings. As an illustration, the lowest line count recorded was reduced to 45 lines, resulting in a drawing time of 75 s.Once the lines have been clustered, we use a waypoint generation algorithm to extract the necessary information from the sketch image. We have set the waypoint generation algorithm buffer to 250 waypoints per line, which allows us to maintain a high level of precision while also optimizing the drawing process. The waypoints generated by this algorithm not only contain information about the path that the arm should follow, but also hold information about the thickness of the lines being drawn.Algorithm 1: Extract Line: Pixel to PixelAfter the waypoints have been generated, they are passed to the robot motion control module, which uses them to guide the arm along the desired path. This allows the arm to smoothly trace the lines in the sketch and produce high-quality drawings in a timely manner.Version 1: Direct Sketch Features ExtractionIn our initial attempt at extracting lines from sketch images, we utilized the Closing Algorithm seen in Version 2: Replacing eye pupil sketch with circleTo address the issue of detail loss in the eye sketch, we implemented a first approach that involved replacing the eye pupil sketch with a circle. This process involved the use of the Open Pose algorithm to identify the approximate position of the eyes in the sketch, followed by the clearing of the sketch at these positions. The remaining lines were then extracted and processed to generate waypoints, with additional waypoints in the form of circles added at the positions of the eye pupils. While this approach successfully preserved the details of the eye sketch, it was sensitive to errors in eye position recognition, leading to a less natural appearance in the final drawing compared to the original sketch. To address this issue, we introduced a third version of the system.Version 3: Eye lines extractionTo further improve the accuracy and natural appearance of the eye sketch, we implemented an approach that involved separating the eye sketch from the rest of the image and processing them separately. We used the Opening algorithm for the eye sketch seen in We implemented a path optimization algorithm and a communication system, using ROS, to improve the efficiency and precision of the robotic arm’s portrait drawing process. The path optimization algorithm calculates the most efficient path for the arm to reach the next starting point, minimizing unnecessary movements and reducing the drawing time. To ensure that the robotic arm had enough information to accurately draw a single line, we set the waypoint navigational buffer to 250 points, as determined by the previously discussed waypoint generation algorithm. The communication system, using ROS, allowed for smooth and precise control of the robotic arm during the drawing process. Together, these techniques resulted in a smooth and efficient drawing process, producing high-quality portrait sketches in a timely manner.Our hardware setup consists of a robotic arm mounted on a table as seen in The ZEUS robotic arm , is an iThis gripper is specifically designed to securely hold calligraphy brush pens in a fixed position, preventing them from slipping during the drawing process seen in In addition to the custom-designed gripper, we have also implemented a 3D printed canvas pad to hold the A5 paper steady and keep it in place during the drawing process. By 3D printing the canvas pad, we are able to customize its shape and size to perfectly fit the dimensions of the paper, ensuring that it remains firmly in place throughout the drawing process. The use of 3D printing technology allows us to achieve a high level of precision and customization in the design of both the gripper and the canvas pad.In our experiment, we took great care in selecting the appropriate calligraphy pens for use. We used extensive testing to find out which pens would best serve our needs and produce the desired results. We utilized Kuratake Bimoji Calligraphy Brush pens XT3-5s, XT3-10s and DK150-25B as suitable options for our needs. We used the XT3-5s producing 0.6–3.0 mm line thickness refer to a, the XTThrough analysis, we found that the DK150-25B with up to 15.0 mm width, which has a tip made of horse tail, produced blot lines when applied to thick lines, as shown in In order to accurately and consistently reproduce the desired line thickness in our robotic portrait drawing system, we conducted a series of calibration tests to determine the relationship between pen height and line thickness. Using calligraphy brush pens, we measured the resulting line thickness at different heights above the canvas. The results of these tests were plotted on a correlation graph, as shown in The computational average time per sketch is a crucial aspect with which to evaluate the efficiency and practicality of the proposed robotic portrait drawing system. In order to assess the efficacy and precision of our system, a series of experiments were carried out. These experiments involved the use of our robotic arm to create portraits of a diverse range of volunteers, as illustrated in In November 2022, the fully tested system was showcased at the RoboWorld exhibition, as reported in . During The proposed system for robotic portrait drawing, which combines machine learning techniques such as CycleGAN with morphological transformations and path optimization, has been shown to be highly effective at generating visually appealing and accurate sketches in a rapid and efficient manner. Real-world experiments at the RoboWorld exhibition in November 2022 demonstrated the effectiveness, stability, robustness, and flexibility of the proposed system. Our system drew portraits of over 40 volunteers, achieving a high satisfaction rate of 95% among the participants. The success of our system at the exhibition was recognized with the Korean Intellectual Property Office Award. The results suggest that the proposed system has great potential for practical application in a range of settings, including galleries, studios, and educational institutions.The high satisfaction rate of 95% among participants indicates the effectiveness and reliability of the system in producing high-quality portrait sketches in a time-efficient manner. The proposed system represents a significant advancement in the field of robotic portrait drawing due to its ability to generate visually appealing and accurate sketches in a rapid and efficient manner. The real-world experiments at the RoboWorld exhibition have demonstrated the effectiveness, stability, robustness, and flexibility of the proposed approach, with an average drawing time of 80 s per portrait.In summary, the proposed system has demonstrated its capability to generate visually appealing and accurate sketches in a rapid and efficient manner, making it a significant advancement in the field of robotic portrait drawing. The high satisfaction rate achieved among the participants at the RoboWorld exhibition suggests that the proposed system has great potential for practical application in various settings. This research presents a promising direction for the development of robotics and machine learning technologies in the field of portrait drawing.Furthermore, investigating the adaptability of the proposed system to different artistic styles and mediums, such as watercolor or charcoal, would broaden its applicability and artistic versatility. This could involve exploring different data representations, training methodologies, or incorporating style transfer techniques to allow users to customize the output according to their artistic preferences.


This paper investigates the effects on the material microstructure of varying the Inter-Layer Cooling Time (ILCT) during the printing process in laser powder bed fusion (L-PBF) multi-laser machines. Despite these machines allowing higher productivity rates compared to single laser machines, they are affected by lower ILCT values, which could be critical for material printability and microstructure. The ILCT values depend both on the process parameter sets and design choices for the parts and play an important role in the Design for Additive Manufacturing approach in L-PBF process. In order to identify the critical range of ILCT for this working condition, an experimental campaign is presented on the nickel-based superalloy Inconel 718, which is widely used for the printing of turbomachinery components. The effect of ILCT on the microstructure of the material is evaluated in terms of porosity and melt pool analysis on printed cylinder specimens, considering ILCT decreasing and increasing in the range of 22 to 2 s. The experimental campaign shows that an ILCT of less than 6 s introduces criticality in the material microstructure. In particular, at an ILCT value of 2 s, widespread keyhole porosity (close to 1‰) and critical and deeper melt pool (about 200 microns depth) are measured. This variation in melt pool shape indicates a change in the powder melting regime and, consequently, modifications of the printability window promoting the expansion of the keyhole region. In addition, specimens with geometry obstructing the heat flow have been studied using the critical ILCT value (2 s) to evaluate the effect of the surface-to-volume ratio. The results show an enhancement of the porosity value (about 3‰), while this effect is limited for the depth of the melt pool. Additive manufacturing (AM), contrary to the conventional machining methods in which the product is manufactured by removing the unwanted material from the blank , allows Laser powder bed fusion (L-PBF), according to ISO/ASTM 52900 standard, is an AM process in which thermal energy selectively fuses regions of a powder bed. This technology is revolutionizing the manufacturing approach to obtain near-net-shape components, especially for products with high geometric complexity ,9,10. Th3/h, lower than that estimated for electron beam melting (EBM) and direct energy deposition (DED) of 70–100 and 140 cm3/h, respectively [Despite all these advantages compared to the conventional machining methods, this process is affected by low efficiency in terms of material handling, high maintenance costs and low production volume . The buiAs a result, the need to make this process more productive and competitive with other technologies is becoming the most crucial goal in this field.-Process parameters optimization ,18 using-New printing strategy as Hull-Core .-Increasing the number of heat sources using multi-laser machines .Increasing productivity ,16 meansAmong all the proposed solutions, using multi-laser machines is the best, allowing print volumes for the job to be doubled or even tripled.Although these machines allow sharp increases in print volumes, they suffer from very low Inter-Layer Cooling Time (ILCT) ,22,23.The ILCT is calculated as the time between the exposure of one layer and the next, so it is different from the Inter-Layer Time (ILT)  constantSince the process is essentially thermally driven, the difference in thermal history between layer parts or from layer to layer could affect the mechanical properties and microstructure ,36,37,38As previously mentioned, the variation of this parameter becomes predominantly critical on multi-laser machines where the exposure times for each layer are considerably shorter than in single laser printers as multiple lasers work in parallel on the same printing platform.In multi-laser printers, a value of ILCT equal to a few seconds can be reached, hypothetically increasing the substrate temperature. So, what is theoretically expected is a variation of the material’s printability window  process parameters chosen according to the results reported in the literature , as showThe experiments were carried out in a high volumetric energy zone to study the effect of ILCT variation in the area close to the keyhole melting regime, where defect formation is promoted. This choice was made to understand whether a short ILCT could increase the tendency of defect formation or not. In addition, working in this VED region was interesting to avoid high residual stresses in the component as the remelting of several previous layers was obtainable.Investigation of ILCT’s variation effect on the material microstructure.Evaluation of obstructing heat flux geometries on critical ILCT values.The presented analysis of ILCT’s variation effect on material microstructure consisted of two main phases:This phase investigates if a critical level of ILCT negatively affects the material microstructure. The effect on the material microstructure was evaluated as a modification on porosity level and melt pool characteristic dimensions and shape ratio.ILCT has been studied, considering the increasing and decreasing order, to reproduce the heating and cooling effect on a component due to geometry section variations. The range of ILCT investigated was from 22 s to 2 s in steps of 4 s.A cylinder specimen was printed for each ILCT level tested, and each ILCT level was kept constant for ten millimeters in height. Thus, 11 specimens were made that were exposed to the ILCTs shown in Single tracks and single laser exposition on the powder bed were also made on the top surface of each specimen  in orderThe planned ILCT for the specimens was obtained using customized ghost parts characterized by zero laser power exposition.Considering the process parameters and laser assignment, the build file was created using Renishaw Quantam Software Version 5.3.0.7105. The specimens were placed on the building platform, as shown in The printed specimens were detached from the building platform using a wire EDM machine (ECUT EU MS Genesi) and machined to obtain a mockup with a height of ten millimeters, starting from the top base. Then, the mockup was sectioned along an axial plane, as shown in -A porosity analysis was performed to estimate the densification level and the presence of defects in the material. This analysis was performed using a Leica Leitz DMRME  optical microscope, and the percent porosity was evaluated by analyzing six fields for each specimen. Then, the acquired images were post-processed using ImageJ 1.53  software. In order to measure the porosity percentage, a thresholding image processing was applied to the acquired images.-A melt pool analysis was performed on the specimens using the Leica Leitz DMRME , and then a chemical etching with oxalic acid was carried out to highlight the melt pool boundaries. For each analyzed configuration, five random single tracks were measured in terms of melt pool width and depth .The specimens were then etched with oxalic acid to highlight the melt pool shape. The microstructural analyses were then carried out as follows:This phase aims to investigate whether a geometry that obstructs the heat flux criticality affects microstructure for the critical value of ILCT identified in the previous phase . This anThe specimens were modelled as cylinders and inverted truncated cones in function of the area to be evaluated, as shown in The usage of inverted truncated cone specimens was set to reproduce very hostile heat transfer conditions. This geometry could help the element to overheat progressively as each layer has a larger area to be exposed compared to the one below . This eThe dimensions and the investigated area for each specimen are listed in In order to assess and calculate the melting regime for each tested configuration, several single tracks were printed on the top surface of each specimen .As in The build file was created following the procedure described in The porosity results in terms of average and standard deviation for each specimen are summarized in The porosity average and standard deviation results for each specimen are tracked in In order to evaluate the size of defects that affect the specimens’ microstructure, The results of the melt pool analysis are summarized in Upon visual inspection of the specimens after printing, it is possible to state that the sections of the specimens exposed with two seconds of ILCT show a different visual chromatic appearance than the others . The difThe porosity analysis confirms this first indication. The results  show thaBy virtue of these results, it is possible to state that, consistent with the material and machine type used to carry out the tests, the ILCT range from 6 to 2 s presents a criticality in terms of densification level.It is important to note that these ILCT values are difficult to achieve with a single laser printer but easy to achieve with a multi-laser one.The melt pool analysis confirms this range of ILCT values’ criticality . In factRelative to the analysis of the effect of overheating and cooling on the specimen by decreasing and increasing the value of ILCT, there are no significant differences in terms of porosity and melt pool shape. For example, specimens 5 and 7, exposed with the same ILCT value, have a similar average melt pool depth . Similarly, the effect of ILCT on the melt pool width is not significant.By the evaluation of the width/depth ratio , it is oMoreover, the width/depth ratio results suggest a possible review of the criteria provided by Johnson et al.  to calcuThe visual inspection of the specimens indicates a different coloring for samples number 4 and 5 . This inThe melt pool analysis  indicateFrom the analysis of By analyzing the melt pool shape , it is p-An effect of mitigation of porosity level and reduction of melt pool depth is observed as the surface-to-volume ratio decreases, keeping the printed area for low ILCTs constant. This is evident when comparing the porosity level and melt pool depth measured for specimen number 6  and spec-An effect of amplifying porosity level as the exposed area range increases is registered for an ILCT of 2 s . This efUnder these results, two observations can be pointed out:The research investigates the effect of ILCT variation during printing on the material’s microstructure  on specimens of nickel-based superalloy Inconel 718. Moreover, the effect of customized geometries that obstruct the heat flux combined with critical ILCT values is investigated.Considering the obtained results, it is possible to state that extremely low ILCT values can affect the material microstructure regarding densification level and melt pool shape. In particular, an ILCT of less than 6 s introduces criticality in the material microstructure, and a value of 2 s is characterized by widespread keyhole porosity (close to 1‰) and critical melt pool depth (about 200 microns). This effect promotes the expansion of the keyhole region in the printability map.Moreover, considering geometry obstructing heat flow (low surface-to-volume ratio), exposed with a lower ILCT value (2 s), this effect is strongly amplified, and a very high porosity is measured (about 3‰).Various processes or design changes can avoid critical ILCT values. In fact, ILCT can be increased by changing geometry, introducing ghost parts during the printing process or using different part orientations. It is important to emphasize that the first solution is often not feasible due to design constraints on component function ; the second results in an evident loss of productivity; and the last could lead to difficulties in the printing process.A different approach is used to develop a specific process parameter set for the component sections with critical values for ILCT. In developing this process parameter set, the amplification effect of the keyhole region for short ILCT values must be considered. This approach might be easier to implement and might not have a negative impact on productivity.An experimental campaign using different materials and boundary conditions, such as L-PBF machines, building platform temperature and process parameter sets, could be interesting to fully characterize this phenomenon and optimize the thermal design of a component in the L-PBF process. Similar results could probably be obtained using other nickel-based superalloys because of their similar printability properties in the L-PBF process.Moreover, another important aspect to be investigated in future works is the possibility of measuring temperature substrate variation as a function of ILCT values to evaluate the modification of the printability map. This information will be necessary to develop process parameter sets that will be robust to substrate temperature variation.


This study introduces the Spacetimeformer model, a novel approach for predicting stock prices, leveraging the Transformer architecture with a time–space mechanism to capture both spatial and temporal interactions among stocks. Traditional Long–Short Term Memory (LSTM) and recent Transformer models lack the ability to directly incorporate spatial information, making the Spacetimeformer model a valuable addition to stock price prediction. This article uses the ten minute stock prices of the constituent stocks of the Taiwan 50 Index and the intraday data of individual stock on the Taiwan Stock Exchange. By training the Timespaceformer model with multi-time-step stock price data, we can predict the stock prices at every ten minute interval within the next hour. Finally, we also compare the prediction results with LSTM and Transformer models that only consider temporal relationships. The research demonstrates that the Spacetimeformer model consistently captures essential trend changes and provides stable predictions in stock price forecasting. This article proposes a Spacetimeformer model combined with daily moving windows. This method has superior performance in stock price prediction and also demonstrates the significance and value of the space–time mechanism for prediction. We recommend that people who want to predict stock prices or other financial instruments try our proposed method to obtain a better return on investment. With the progress of society and changes in the economic environment, investment and finance have become mainstream trends in modern society. Among numerous investment options such as bonds, stocks, funds, and futures, stock trading is a popular choice for many people. In stock trading, quantitative trading is an important area that utilizes extensive historical data and market information to predict stock price trends ,2,3. ThrIn addition to neural network models, deep learning models are currently the most important models in the field of artificial intelligence, especially in time series forecasting ,11,12,13Traditional LSTM and recent Transformer models only focus on the temporal relationship between data points, and predict future information by capturing the dependencies between before and after data collection. They overlook the spatial relationships that exist among variables, which can be complex and dynamic in the context of stocks ,35. For Based on the research background and motivation mentioned above, this study aims to explore the impact of spatiotemporal relationships on the stock price predictions. To capture the relationships between different stocks at different time points, we employ the Spacetimeformer model , which ihttps://www.yuanta.com.tw/eYuanta/Securities/Stock, accessed on 1 May 2022), which is updated every minute. In order to effectively demonstrate the performance of the Spacetimeformer model, we selected six important constituent stocks from the constituent stocks of the Taiwan 50 Index . EMC has written many brilliant records in the history of container shipping. So far, it has taken the leading position in the world in terms of fleet size and container carrying capacity. FCFC is one of the main members of the Formosa Plastics Group. Whether it is textile, fiber products or petrochemical products, the company has a leading position in Taiwan and Asia. YFH is a financial holding company that develops on the dual axes of securities investment and commercial banking. Its market share in each business is one of the main market leaders, and it has long been recognized by investors.We use the ten minute interval trading prices of these six stocks as input data for the model, and the data range is from 1 June 2022 to 18 November 2022. In order to avoid the sharp fluctuations in stock prices caused by “opening” and “closing”, the trading time interval of this study is locked from the original  to  for discussion. That is to say, we only consider twenty-seven time cut-off points, 9:01, 9:11, …, and 13:21, every day. The descriptive statistics of the stock prices of these six companies are shown in Predicting stock prices using LSTM models has gained significant attention in financial markets due to their ability to capture complex temporal dependencies in historical price data. LSTM models are a type of recurrent neural network designed to overcome the vanishing gradient problem, making them particularly well suited for time series forecasting. The architecture of an LSTM model is composed of distinct layers, each serving a crucial role in processing sequential data. This design is particularly effective for capturing intricate temporal relationships within the data. A fundamental characteristic of LSTM models is their ability to stack multiple LSTM cells shown in  on top oWithin each LSTM cell, a series of gates regulates the flow of information throughout the sequence. Their primary function is to control how information is introduced into the network, stored within the cell state, and ultimately released for prediction. The forget gate, in particular, is responsible for filtering out information that the model deems irrelevant or outdated. Only information deemed pertinent and aligned with the model’s learning objectives is retained, while less relevant data are actively discarded. This selective retention and discarding of information through the forget gate enables LSTM models to focus on the most meaningful aspects of the input sequence, enhancing their ability to make accurate predictions and capture underlying patterns effectively.t. We also use Let In the subsequent phase of the LSTM process, we encounter the input gate and the new memory network. Their primary purpose at this stage is to determine what fresh information should be integrated into the network’s long-term memory, known as the cell state. This decision hinges on a careful evaluation of both the previous hidden state and the current input data. Similar to the forget gate, the output value from the input gate holds significant meaning. A low output value signals that the corresponding element of the cell state should remain unaltered, indicating a decision to not update that specific aspect of the memory. Crucially, the new memory update vector serves as a blueprint for adjusting each component of the long-term memory, the cell state. It essentially guides the LSTM in determining how much each memory element should be modified based on the most recent data, ensuring the model’s adaptability and responsiveness to evolving information. Let In the concluding phase of an LSTM’s operation, the pivotal task is to derive the new hidden state by leveraging the recently updated cell state, the preceding hidden state, and the incoming input data. The output gate essentially acts as a decision-maker, regulating which parts of the updated cell state and the previous hidden state should contribute to the final hidden state. This filtration mechanism ensures that only the most pertinent and contextually relevant information is incorporated, enabling the LSTM model to maintain a precise and informative hidden state while avoiding unnecessary complexity. The output gate is calculated asTime series forecasting plays an important role in many fields, including weather forecasting, traffic conditions, and financial forecasting. In the past, LSTM had excellent performance in NLP tasks. However, the input of LSTM is a vector, and the input must be processed step by step. Moreover, due to the recursive structure, LSTM cannot capture the long-term correlation in the sequence. Compared with LSTM, the input of the Transformer is a matrix which can eliminate the order of the input so that each Token in the sequence can be processed in parallel. Therefore, the Transformer has been more widely used in time series forecasting tasks recently. A Transformer basically consists of a series of encoder and decoder layers whose input is a matrix. The encoder uses the attention mechanism to understand the correlation between Tokens, while the decoder uses the information obtained from the encoder to produce task-specific predictions.k steps are used to predict future target values for h steps. We assume that t , and that N is the number of variables, represents the response vector at time t. Given the timestamps x) and response vectors  and root mean square error (RMSE) as the indicators of the performance evaluation among the three models. The formulae of MAPE and RMSE arei”, i = 1, 2, …6, represents the ith step predictions through each model. At the same time, we use MAPE (shown in First, we draw the trend chart of stock price predictions, as shown in shown in  and RMSEFirstly, let us discuss the performance of the stock price forecast for TSMC. From For the performance of UMC’s stock price prediction, we can observe from Next, we discuss the respective predictions of the three models for DELTA’s stock prices. From Regarding the stock price forecast of EMC, from From Finally, we discuss the stock price forecast of YFH. From According to the above results, the Spacetimeformer model can predict the stock price in the next ten minutes, twenty minutes or even sixty minutes with the smallest error. It significantly outperforms the other two models for stock price prediction at all steps. The Transformer model is next, and the LSTM model has the highest error. Most of the predictions from the Spacetimeformer model fall close to the true value, despite errors from the true data. However, these errors can be explained as natural fluctuations. Although the Transformer model outperforms the LSTM model, the predictions at some time points are significantly biased. It shows that its smoothness processing is poor, and it also confirms again that the Spacetimeformer model with the concept of time and space performs better in stock price prediction. The Spacetimeformer model can continuously capture important trend changes in stock price forecasts and provide relatively stable forecast results. In the predictions of multiple stock prices, both the Transformer and LSTM models show very unstable performances, which further proves that the Transformer model combined with the daily moving windows method may perform better in long-term forecasting.Investment and financial management has become an important topic in modern society, and stock trading is an investment project that people have taken much interest in. In stock trading, quantitative trading is a field worthy of research. We can use a large amount of historical data and market information to predict the stock price trend in order to obtain better trading opportunities. In recent years, with the development of artificial intelligence and deep learning, the LSTM and Transformer models have shown good performance in stock price prediction. The traditional LSTM and the latest Transformer models mainly focus on the time-to-time correlation but ignore the spatial relationship between stocks. There are often complex and dynamic relationships between stocks, such as AI concept stocks, electric vehicle concept stocks, etc. It is important to consider the mutual influence between stocks, but LSTM and Transformer models mainly focus on the time series characteristics of individual stocks. The interaction between multiple stocks is relatively less explored. Therefore, this article uses the Spacetimeformer model with a space–time mechanism as a model for predicting stock prices. The model is trained through the interaction mechanism of space and time to capture the relationship between different stocks at different times.This study uses TSMC, UMC, DELTA, EMC, FCFC, and YFH from the constituent stocks of the Taiwan 50 Index as our research targets. The stock price every ten minutes from 1 June 2022 to 18 November 2022 is used as our research data. At the same time, we avoid the influence of cross-day forecasts, and use the method of daily moving windows to define the training set and testing set. In addition, through multi-time-step forecasting, we can predict the stock prices every ten minutes in the next hour. We compare it with the Transformer and LSTM models that only consider the temporal relationship, and finally use the MAPE index to evaluate the performance of the three models.The research results show that the Spacetimeformer model with the space–time concept performs better in stock price prediction than the Transformer and LSTM models. The Spacetimeformer model can continuously capture important trend changes in stock price forecasts and provide relatively stable forecast results. Furthermore, the Spacetimeformer model’s predictions for the next ten minutes, twenty minutes, and even one hour are very close to the true values. This highlights the excellent performance of the Spacetimeformer model in both short- and long-term predictions. In contrast, the predictions of the Transformer and LSTM models at different time points might vary, indicating a less stable performance. This also further demonstrates the superior performance of models based on the Spacetimeformer architecture in both short- and long-term predictions. The experimental design of this study adopts a daily moving window approach to avoid the problems of cross-day forecasting and using long-term models for prediction. This allows us to more accurately assess model performance and leverage the adaptive nature of deep learning. In summary, the Spacetimeformer model combined with daily moving windows demonstrates a superior performance in stock price prediction compared to the Transformer and LSTM models, indicating the significance and value of spatiotemporal concepts for predictive modeling.Therefore, we suggest that people who want to predict stock prices or other financial instruments use the Spacetimeformer model with a time–space interaction mechanism to obtain better results. Based on the above research results, we hope to further verify whether the forecasting performance is also excellent for the stock markets of different countries in the future. We hope to use it to discuss different financial instruments, such as foreign exchange, futures contract, cryptocurrency, etc. This will help us provide more diverse investment forecasts and insights.