Cooperative perception, or collective perception (CP), is an emerging and promising technology for intelligent transportation systems (ITS). It enables an ITS station (ITS-S) to share its local perception information with others by means of vehicle-to-X (V2X) communication, thereby achieving improved efficiency and safety in road transportation. In this paper, we present our recent progress on the development of a connected and automated vehicle (CAV) and intelligent roadside unit (IRSU). The main contribution of the work lies in investigating and demonstrating the use of CP service within intelligent infrastructure to improve awareness of vulnerable road users (VRU) and thus safety for CAVs in various traffic scenarios. We demonstrate in experiments that a connected vehicle (CV) can “see” a pedestrian around the corners. More importantly, we demonstrate how CAVs can autonomously and safely interact with walking and running pedestrians, relying only on the CP information from the IRSU through vehicle-to-infrastructure (V2I) communication. This is one of the first demonstrations of urban vehicle automation using only CP information. We also address in the paper the handling of collective perception messages (CPMs) received from the IRSU, and passing them through a pipeline of CP information coordinate transformation with uncertainty, multiple road user tracking, and eventually path planning/decision-making within the CAV. The experimental results were obtained with manually driven CV, fully autonomous CAV, and an IRSU retrofitted with vision and laser sensors and a road user tracking system. Autonomous vehicles (AVs) have received extensive attention in recent years as a rapidly emerging and disruptive technology to improve safety and efficiency of current road transportation systems. Most of the existing and under development AVs rely on local sensors, such as cameras and lidars, to perceive the environment and interact with other road users. Despite significant advances in sensor technology in recent years, the perception capability of these local sensors is ultimately bounded in range and field of view (FOV) due to their physical constraints. In addition, occluding objects in urban traffic environments such as buildings, trees, and other road users impose challenges in perception. There are also robustness related concerns, for instance, sensor degradation in adverse weather conditions, sensor interference, hardware malfunction and failure. Unfortunately, failing to maintain sufficient awareness of other road users, vulnerable road users (VRU), in particular, can cause catastrophic safety consequences for AVs.In recent years, V2X communication has garnered increasing popularity among researchers in the field of intelligent transportation system (ITS) and with automobile manufacturers, as it enables a vehicle to share essential information with other road users in a V2X network. This can be a game changer for both human operated and autonomous vehicles, which would be referred to as connected vehicles (CVs) and connected and automated vehicles (CAVs), respectively. It will also open many doors to new possibilities with peer-to-peer connectivity. The connected agents within the cooperative ITS (C-ITS) network will be able to exploit the significant benefits that come from sharing information amongst the network. For instance, the standardised Cooperative Awareness Messages (CAMs) enable mutual awareness between connected agents. Nevertheless, there are other types of road users such as non-connected vehicles, pedestrians, and cyclists that have not been included in the C-ITS services yet. The detection of these non-connected road users in this case becomes an important task for road safety.The major standardisation organisations such as European Telecommunications Standard Institute (ETSI), SAE and IEEE have made a significant effort to standardise specifications regarding C-ITS services, V2X communication protocols, and security. This is essential to facilitate the deployment of C-ITS in road transportation network globally. The collective perception (CP) service is among those C-ITS services that are currently being standardised by ETSI. The CP service enables an ITS station (ITS-S), for instance, a CAV or an intelligent roadside unit (IRSU) to share its perception information with adjacent ITS-Ss by exchanging Collective Perception Messages (CPMs) via V2X communication. The ETSI CPMs convey abstract representations of perceived objects instead of raw sensory data, facilitating the interoperability between ITS-Ss of different types and from different manufacturers. A CAV can benefit from the CP service in terms of improved awareness of surrounding road users, which is essential for ensuring road safety. Specifically, it facilitates a CAV to extend its sensing range and improve sensing quality, redundancy, and robustness through cross-platform sensor fusion, i.e., fusing its local sensory data with other CAV and IRSU information. In addition, the improved perception quality as a result of the sensor fusion potentially relaxes the accuracy and reliability requirements of onboard sensors. This could lower per vehicle cost to facilitate the massive deployment of CAV technology. As for traditional vehicles, CP also brings an attractive advantage of enabling perception capability without retrofitting the vehicle with perception sensors and the associated processing unit.Over the last two years, we have been working with Cohda Wireless on CP and CAV. We are particularly interested in the safety implications the CP service is bringing into the current and future transportation network, and how the CP service will potentially shape the development of intelligent vehicles. To this end, we have developed an IRSU and tested it with the Australian Centre for Field Robotics (ACFR) CAV platforms in different traffic scenarios. The IRSU is equipped with a multi-beam laser scanner and multiple cameras for road user detection and tracking, and the vehicles are retrofitted with a suite of local sensors to implement full autonomy. The IRSU and CAV have the V2X communication capability through Cohda Wireless MK5 Road-Side Unit (RSU) and On-Board Unit (OBU), respectively.The first and also the main contribution of the paper is to investigate and demonstrate through three representative experiments how a CV and a CAV achieve improved safety and robustness when perceiving and interacting with VRU, using the CP information from an intelligent infrastructure in different traffic environments and with different setups. These experiments do not use the CAV’s internal perception capabilities so as to highlight the benefits of using intelligent infrastructure and CP service in the traffic environments. The received perception data from the IRSU are used as the only source of information for multiple road users tracking and path planning within the smart vehicle we tested with. The first experiment was conducted on a public road in an urban traffic environment, and the CV was able to “see” a visually obstructed pedestrian before making a turn into an alleyway. It is demonstrated in the next two experiments that a CAV navigated autonomously and safely when interacting with walking and running pedestrians in proximity in simulated and real lab traffic environments, respectively. The second contribution of the paper is to address the coordinate transformation of perception information considering the respective uncertainties, and to analyse the influence of ITS-S self-localisation accuracy through numerical simulations. The uncertainties associated with the perceived object information are essential for performing consistent state estimation for road users, and probabilistic fusion of perception information from multiple ITS-Ss in CP.The remainder of the paper is organised as follows. The concept of CP has been extensively studied in the ITS research community over the last two decades. Initial CP related work proposes to share raw sensory data between two mobile agents, such as images , lidar pMore theoretical and experimental work on CP was conducted as part of the Ko-FAS  researchBased on the work in , the autSimilarly, there are early stage standardisation activities in SAE advanced application technical committee to standardise messages and protocols for sensor data sharing in SAE J3224 [Considering the limited communication bandwidth and avoiding congestion in the wireless channel, more recent studies in the CP area tend to focus on the communication aspect of the technology, weighing up provided CP service quality and the V2X network resources. The work in  investigThere is also work conducted to explore the use cases, benefits, and challenges of CP. The authors of  provide Veins (Vehicles in Network Simulation) [Artery [A significant proportion of the existing work conducts the analyses of V2X communication and CP in simulated environments. For instance, the work in ,28 is caulation) ,50, whiculation)  for modeulation) ,30,32,52 [Artery ,54 frame [Artery  for CP r [Artery ,56 is co [Artery . DespiteThe IRSU developed is comprised of a sensor head, a processing workstation, and a Cohda Wireless MK5 RSU . The sensor head is mounted on a tripod for easy deployment in different testing environments, as shown in In terms of information processing, the workstation first performs sensory data fusion of images and lidar point clouds for pedestrian and vehicle detection. Specifically, the raw images from cameras are first rectified using camera intrinsic calibration parameters. As illustrated in The detection results are then encoded into ETSI CPMs and broadcast by the Cohda Wireless MK5 RSU at 10 Hz. Details are available in To assess the position tracking accuracy of the IRSU, an outdoor test was conducted at the Shepherd Street car park at the University of Sydney. x- and y-directions.As mentioned previously, the road user detection in the IRSU is achieved through camera-lidar sensor fusion. Despite a centimeter level of ranging accuracy obtained from the multi-beam lidar, the bounding box accuracy in visual object detection decreases dramatically as the target moves further from the cameras. In addition to the noises from the sensors themselves, there are errors introduced in the data processing and sensor fusion, such as lidar-to-camera projection, point cloud segmentation, and clustering. Therefore, based on the comparison with ground truth data and considering the detection range of the IRSU, the overall position noise contained in the pedestrian detection is modelled as a zero-mean Gaussian variable with a standard deviation of 0.2 m in both As presented in ented in . In addiented in  for moreLanelet2 map is built for every experiment site, which includes road network, lane layout and traffic rules such as speed limits, traffic lights and right-of-way rules.Although the CAV can achieve full autonomy using its various built-in sensors, its perception sensors were not used for road user detection in the experiments presented in the paper, in which case, the CAV had to rely on the CP information from the IRSU when interacting with pedestrians in the traffic environments. This is to highlight the benefits of using the CP service in CAV operations. The multiple cameras were used for video recording purposes, and the multi-beam lidar was enabled only for aiding self-localisation within the map. Lidar feature maps of the experiment sites were built using a simultaneous localisation and mapping (SLAM) algorithm. The maps are based on pole and building corner features extracted from lidar point clouds, which are essential for localisation since GNSS alone cannot provide the desired level of accuracy in the experiment environments. Interested readers can refer to  for moreWhen the CAV receives an ETSI CPM through the onboard Cohda Wireless MK5 OBU, the received perceived objects information is first decoded from binary ASN.1 encoding, and transformed with its uncertainty to the local frame of reference of the CAV, as presented in Following the coordinate transformation with uncertainty, the perceived objects information from the IRSU are fused into a multiple road user tracking algorithm that is a variant of the GMPHD filter running within the local frame of the receiving CAV. The tracked states of road users include their position, heading, and speed. The general formulation of the GMPHD tracker is to use a Gaussian mixture to represent the joint distribution of the group of tracked targets. The GMPHD approach is considered attractive due to its inherent convenience in handling track initiation, track termination, probabilistic data association, and clutter. Compared with the naive GMPHD algorithm, the road user tracker running in both the IRSU and CAV is improved with measurement-driven initiation of new tracks, and track identity management. In addition, note that an instance of the tracker is required for each type of road user, which effectively reduces the overall computational cost.Lanelet2 map, (2) obstacles picked up by local perception sensors, and (3) current and future estimates of road users detected and broadcast by IRSU through V2I communication, such that it can plan a smooth, safe, and kinematically feasible path to avoid collision with any other road users the CAV becomes aware of. It should be noted that, in the experiments presented in the paper, the use of local perception information in the navigation was minimised or disabled. Currently, the path planner is designed for low-speed scenarios, as our CAV platforms are intended for low-speed operations in shared traffic environments. However, this does not affect the demonstration of CP.The navigation subsystem in the CAV is responsible for path planning, monitoring and controlling the motion of the vehicle from the current position to the goal. A hybrid A* path planner presented in our recent work  is runniCPM Management Container, which indicates station type, such as a vehicle or an IRSU, and reference position of the transmitting ITS-S.A Station Data Container, which provides additional information about the originating station. This includes the heading, speed, and dimensions when the originating station is a vehicle.An optional Sensor Information Containers, which describe the type and specifications of the equipped sensors of the transmitting ITS-S, including sensor IDs, types, and detection areas.Optional Perceived Object Containers, each of which describes the dynamics and properties of a perceived object, such as type, position, speed, heading, and dimensions. These perceived object descriptions are registered in the coordinate system of the originating station.Optional Free Space Addendum Containers, which describe different confidence levels for certain areas within the sensor detection areas.Optional The overall system diagram of the IRSU and the CAV in a C-ITS configuration is illustrated in The CPM generation rules support different abstraction levels for perceived object descriptions for the implementation flexibility, which can derive from low-level detections made by individual sensors, or the results of the high-level data fusion. In the developed IRSU, the road user detections as a result of camera-lidar sensor fusion are considered in the encoding of CPMs, as depicted in The overall perception information flow from the IRSU to CAV is illustrated in msg_bridge node running within each MK5 serves as a bridge between ROS CPMs and ETSI CPMs. It populates an ETSI CPM given information from an ROS CPM and based on the ASN.1 definition of the ETSI CPM. In the meantime, it decodes an ETSI CPM received from other ITS-Ss back into the ROS CPM format. The transmission and reception of ETSI CPM payload are handled by Cohda’s V2X stack in MK5. We keep upgrading the ROS msg_bridge node to support the features required in the CP demonstrations in accordance with the status quo of ETSI CPM standardisation.A ROS Perceived Object Containers of the CPM specification.Perception uncertainty in the sensing ITS-S. Every commonly used perception sensor in ITS area has its own strengths and limitations. For instance, RGB images are useful in detecting object instances and classification of road users with an estimated confidence level using a visual classifier algorithm. Doppler RADARs produce both position and velocity measurements but are prone to noise and interference from the environment. Lidars have a high range resolution to observe physical extent and shape of objects. Nevertheless, the point density decreases dramatically along with range. These sensors all produce measurements corrupted by noise, and thus should be modelled with uncertainty. Combining multi-modal sensory information not only improves robustness of the perception system, but also increases accuracy, which means a lower level of uncertainty. The produced estimates of perceived objects with uncertainty have been represented in CPM Management Container and Station Data Container in the CPM definition.Self-localisation uncertainty of sensing and receiving ITS-Ss. A vehicle ITS-S has to constantly localise itself within a global frame of reference such as a map or UTM frame for navigation and safety reasons. However, accurate self-localisation for a moving platform is known to be one of the existing challenges in ITS applications such as urban navigation. Using GNSS as the only source for localisation often cannot achieve satisfactory accuracy, in particular in GNSS-degraded or even GNSS-denied environments. There are various existing solutions that can provide higher localisation accuracy such as RTK, GNSS and inertial/encoder data fusion, feature based localisation based on existing map, etc. Nevertheless, even with the same localisation approach, the uncertainty magnitude in the localisation can vary significantly depending on certain external conditions, such as GNSS satellite visibility in the sky, and the quantity of observable features and their qualities in the surroundings. The location of an IRSU, although deployed static in a traffic environment, is not immune from localisation error either when set either by GNSS or through a surveying process. The localisation uncertainties of both sensing and receiving ITS-Ss therefore have to be considered in the perceived objects coordinate transformation since it cannot be completely eliminated regardless of the self-localisation means employed. The originating ITS-S information including its pose with associated uncertainty has been contained within the The perceived objects information broadcast through CP service is produced from the perspective of the sensing ITS-S. When a piece of perceived object information is received, it is not usable for the receiving agent until it is transformed into its local coordinate system. In , the cooG. The sensing ITS-S S observes multiple road users within its local frame with some level of sensory measurement uncertainty labelled with purple dashed lines. The road users measurement uncertainties grow when transformed to a global frame G due to self-localisation uncertainty of ITS-S S, as labelled with gray dotted lines. The road users measurement uncertainties grow further, as labelled by orange solid lines, when transformed to the local frame of the receiving ITS-S R as its self-localisation uncertainty is also incorporated.Consequently, the transformed object state estimates contain uncertainty that is mainly a combined result of uncertainty in the localisation of both ITS-Ss and that in the sensory perception. As illustrated in Both works ,62 perfoThis section presents mathematical formulation of the coordinate transformation of perceived objects information a recipient ITS-S receives from a sensing ITS-S. We propose to use unscented transform (UT) in the coordinate transformation with uncertainty, which here is recognised as a nonlinear and non-deterministic process. The formulation presented is intended for 2D transformation, as it is reasonable to assume a planar road surface for the road user detection and tracking problem discussed in the paper. Nevertheless, the formulation can be easily extended to transformation in 3D space. It is important to note that although the paper is mainly focused on CP service provided by intelligent infrastructure, the formulation presented here is intended to provide a generic form for two arbitrary ITS-Ss in a V2X network, which includes both V2V and V2I scenarios. Before proceeding further, some definitions about coordinate systems are first given to facilitate discussions.S is attached on the platform body and is represented as R is attached on the platform body and moving with the platform. It is denoted as The global frame, which is the fixed coordinate system attached on the ground, is represented as p in the x-direction with respect to t. A variable without pre-superscript is defined in a generic way to not specify a particular coordinate frame.The pre-superscript is employed to describe the coordinate frame in which the corresponding variable is expressed. For instance, R at time t and its estimate are denoted ast. The state estimate is represented by a multivariate Gaussian asThe state of the receiving ITS-S S at time t is also represented as a Gaussian variable:S, t, t in , along with the movement of the receiving CAV. Test 2, which is with different uncertainty in position estimates of the CAV(s). It can be seen from both figures that the confidence ellipses of those perceived objects, after being transformed into the local coordinate system of the receiving CAV, are bloated and distorted to different extents, depending on their relative poses with respect to both ITS-Ss. Please note that, in both figures, the first set of ellipses on the left in each subfigure represents the uncertainties in the transformed position estimate of the sensing ITS, while the receiving CAV is located at the origin, facing the x-direction.x- and y-directions, as shown in In In addition, the two scenarios of an IRSU and a CAV acting as the sensing ITS-S are compared in each of It should be noted that the perceived objects and the sensing ITS-S are set to be static in the simulation just for the convenience of analysis. In fact, the bloating of the position uncertainty in the coordinate transformation is independent of moving speeds of the participants in the simulation, mainly because only pose estimates of the perceived objects and ITS-Ss are involved in the transformation formulated in The demonstrations consist of three experiments conducted in different traffic environments, namely, an urban traffic environment, the CARLA simulator, and a lab traffic environment. The configurations of these experiments are compared in The experiment was conducted in a real urban traffic environment next to the ACFR building located at Chippendale, Sydney. In the experiment, the IRSU was deployed near the intersection of Abercrombie St and Little Queen St, monitoring the traffic activity at the intersection and broadcasting the perception information in the form of ETSI CPMs in real time, as depicted in Instead of autonomous driving, the CAV in the experiment was driven manually for ensuring public road safety in the real traffic scenario. The CAV platform therefore practically functioned as a CV in the experiment and is referred to as the CV in the remainder of the section. While the CV was performing self-localisation within the experiment area, it was able to transform the perceived objects contained in CPMs into its local coordinate system with uncertainty and have the road users tracked in real time. The CV identifies itself and its driver based on the position of the transformed perceived objects in its local frame to avoid self-tracking.In the experiment, as labelled in In addition, the CV localisation quality varied depending on the lidar feature quality at different locations of the experiment environment. It is illustrated by comparing ros-bridge package. It is considered a safe, realistic, and repeatable environment for experimenting with autonomous driving systems, in particular, for conducting CAV operations that have safety concerns to test in the real world.The experiment demonstrates the autonomous operation of a CAV using the perception information received from an IRSU in CARLA simulator as the only source for pedestrian estimation. CARLA  is an opThe experiment is setup with the IRSU detecting multiple walking pedestrians within its sensor range. The information of perceived pedestrians is then broadcast in the form of ETSI CPMs and is received by the CAV. The information is then transformed into its local frame of reference, and taken into account as the only perception data in the autonomous navigation of the CAV. The experiment is to demonstrate how a CAV interacts with VRU at non-designated crossing areas with the perception information from an IRSU. It is strongly recommended to conduct these experiments with higher risk CAV maneuvers in a simulator before performing them in the real world.Town01 map of CARLA 0.9.8 simulator with participants labelled in Lanelet2 map was built for the experiment environment in the simulator, which is essential for the CAV to stay on the road and plan drivable paths for autonomous navigation. The map includes the lane markings and the network of drivable regions as per right hand traffic rules. Accordingly, the CAV respects right hand traffic rules in the demonstration. On the road where the CAV is navigating, the CAV has right of way over pedestrians and is allowed to cross the broken dividing line to overtake other road users when it is safe to do so.The experiment was setup in the The IRSU in the simulator is configured as close as possible to the real IRSU developed, which is equipped with two cameras and a 16-beam lidar, as revealed in In the experiment, one of the pedestrians was crossing the road at a non-designated crossing area while the other is walking along the road in the opposite direction to the CAV. In Lanelet2 map was built for the environment and the CAV was able to localise itself within the pre-built map of lidar features, such as poles and building corners. The pedestrian crossing area was encoded in the Lanelet2 map, which is stored locally in the CAV. The expected behaviour of the CAV is to drive through the crossing or stop before the crossing to give way depending on whether there is any pedestrian crossing activity detected by the IRSU.The last experiment presented in the paper investigates the CAV interacting with a pedestrian running towards a pedestrian crossing area. It was set up in a lab traffic environment with a 55 m stretch of straight road, a virtual pedestrian crossing area, and the IRSU deployed next to the crossing, as illustrated in The CAV onboard perception sensors were not used for pedestrian detection throughout the experiment. However, there are a few measures in place to ensure safety when conducting autonomous navigation related activities in real-world experiments. First of all, the experiment was conducted in a closed, controlled environment and the CAV speed is limited to 11 km/h when in autonomous mode. In addition, there is always a safety driver sitting in the CAV closely monitoring any hazards and has the ability to trigger an e-stop of the CAV in case of an emergency. Furthermore, the virtual bumper subsystem based on lidar points obstacle detection is enabled on the CAV, which brings the vehicle to an e-stop if any obstacle breaches any of the pre-set distance thresholds to the vehicle. However, the virtual bumper treats every surrounding object as static obstacles and therefore did not take part in the decision-making process of the CAV in The paper is mainly focused on safety and robustness implications the CP can bring to the operations of CAVs. It presents the IRSU and CAV platforms developed by the joint research team from the ACFR and Cohda Wireless over the last two years. The contributions of the paper include (1) investigating and demonstrating through three representative experiments the safety benefits associated with the use of CP service for CV and CAV operations alongside VRU in different traffic contexts, and (2) the formulation and analysis of coordinate transformation of perceived objects information with uncertainty.In the first experiment conducted on a public road, the CV was able to track a pedestrian visually obstructed by a building with CP information from an IRSU deployed on the road side. This was achieved seconds before its local perception sensors or the driver could possibly see the same pedestrian around the corner, providing extra time for the driver or the navigation stack to react to a safety hazard. The second experiment conducted in the CARLA simulator shows the autonomy of a CAV using the CP service. The experiment highlights the autonomous navigation of the CAV and its safe interaction with walking pedestrians, purely responding to the perception information provided by the IRSU. The last experiment demonstrated the expected behaviour of the CAV when interacting with a pedestrian rushing towards the designated crossing area in a lab environment. The CAV managed to take preemptive action, that is, braking and stopping before the crossing area for the pedestrian based on the kinematic prediction of the pedestrian. The pedestrian tracking, prediction, path planning, and decision-making in the CAV in the last experiment were operating based on the perception information received from the IRSU.We believe that these demonstrations can assist engineers and researchers in the relevant fields in achieving a better understanding of the safety implications the CP is bringing to the current and future transportation systems. The CP enables the smart vehicles to break the physical and practical limitations of onboard perception sensors, and, in the meantime, to embrace improved perception quality and robustness along with other expected benefits from the CP service and V2X communication. As importantly, the CP can also reduce the reliance on the vehicle’s local perception information, thereby lowering the requirement and cost for onboard sensing systems. Furthermore, it is demonstrated that, when properly used, IRSU perception data can be used as another reliable source of information to add additional robustness and integrity to autonomous operations. These potentially trigger more in-depth discussions in the research community and industry related to the future development directions of intelligent vehicles and other C-ITS services and applications, such as maneuver intention sharing and cooperative driving.


The application of gyroscopes in harsh environments has always been a hot topic. As a high-quality material for manufacturing gyroscopes, quartz crystals need to be designed and optimized to meet the normal operation of gyroscopes in harsh environments. The Micro Electronics Mechanical System(MEMS) quartz tuning forks resonant gyroscope is one of the quartz gyroscopes. The elastic structure (V-shaped beam) between the anchor support point and tuning forks plays a vital role in the MEMS quartz tuning forks resonant gyroscope. This structure determines the natural frequency of the gyroscope, and more importantly, determines the shock resistance of the gyroscope structure. In this article, the MEMS quartz tuning forks gyroscope with different V-shaped beam thicknesses are simulated and analyzed by finite element analysis simulation software. After the modal analysis and shock simulation  and duration of 2 ms in the six shock directions), the results show that when the beam thickness is 80 μm, the maximum stress is 94.721 MPa, which is less than the failure stress of quartz crystal. The gyroscope’s shock resistance is verified through shock testing. A gyroscope is a precision instrument that can sense the angular rate of rotatory objects. It is widely used in many fields, such as aerospace, navigation, automotive industry, and military ,2,3. DueRecently, with the rapid growth of microelectronics technology and semiconductor process, Micro Electronics Mechanical System (MEMS) has gradually grown and emerged. At the same time, MEMS gyroscopes have also attracted the attention of researchers. The MEMS gyroscopes are cheaper , smallerAt present, there are many companies in the world researching MEMS quartz tuning forks resonant gyroscopes and putting them into the market, including Microcomponents SA , Watson In this paper, a gyroscope and the structure connecting the tuning forks and the anchor point is introduced and analyzed. In addition, the measuring range is related to the shock resistance. The larger the range, the stronger the shock resistance. For this gyroscope, the range of the gyroscope is 300°/s. This gyroscope is used in fields such as automobiles and navigations. The V-shaped beam is a vital structure in the MEMS quartz tuning forks resonant gyroscope, it connects the elastic structures (drive tuning forks and detect tuning forks) and anchor support point and determines whether the gyroscope can work normally in a shock environment. The effect of the thickness of V-shaped beam on the gyroscope stability, natural frequency and shock resistance is simulated and analyzed in this paper. The thickness is determined by finite element simulation and verified by shock test after the simulation.m is vibrated in the driving direction (x-axis). When the system is subjected to angular velocity, due to the Coriolis force, the mass m is vibrated in the detection direction (z-axis).In the ideal gyroscope model, the drive axis and the detection axis of the miniature quartz tuning fork gyroscope are orthogonal to each other. The mass-spring-damping system model of the ideal-gyroscope is simplified as shown in x-axis) near the resonance frequency of the driving mode by electrical signals. When the gyroscope rotates around the y-axis at an angular velocity Ω, the driving tuning fork is subjected to Coriolis force and generates vibration in a direction perpendicular to the plane of the tuning forks (z-axis). This Coriolis force movement is transmitted to the detection tuning fork by the supporting beams (V-shaped beams), which makes the detection tuning fork vibrate perpendicular to the plane of the tuning forks. The vibration signal of the detection tuning fork is converted into an electrical signal through the piezoelectric effect, and the electrical signal is proportional to the input angular velocity Ω. The electrical signal can calculate the input angular velocity through the signal conditioning circuit. For the micro quartz tuning forks gyroscope, the gyroscope structure of each part is shown in The micro quartz tuning forks gyroscope is composed of two tuning forks, the driving tuning fork and the detecting tuning fork. Each tuning fork can be regarded as composed of two masses, as shown in Fabrication of the micro quartz tuning forks gyroscope draws on MEMS process technology. The process mainly consists of the following steps: Cleaning, photolithography, magnetron sputtering, peeling and etching. The detailed steps are shown in Assuming that the quartz gyroscope structure is directly fixed on the shock table, as shown in Q, respectively. x and y are the absolute displacements of the gyroscope and the shock table, respectively, so the displacement y of the shock table is,Define z = x − y, and z(t) can be obtained,Define R(t) is related to iy, iω, ω, and ξ. These parameters are very critical in the simulation process and need to be added in the simulation software.From the above formulas, Shock usually refers to a single interaction between the shock object and the research object, which can provoke a sudden change in the force, acceleration, velocity, or displacement of the transient disturbance and release a considerable amount of energy in a small space. The outstanding feature of the mechanical system’s response to shock excitation is that there is still a mechanical motion after the input action ends. The motion generated during the mechanical system and excitation is called the initial shock response; the motion that still exists after the end of the excitation is called the residual shock response. The shock motion process is distinguished from the vibration process by its single-time, non-periodic, and non-zero amplitude characteristics.A is the amplitude of the half-cycle sine shock pulse, and D is the pulse duration.Generally, force, displacement, or acceleration are used to express the characteristics of shock. Among them, in most cases, acceleration is used to represent shock. The most common single pulse acceleration waveforms are half-cycle sine wave, rectangular wave, triangle wave, and clock wave. In this shock environment test, half-cycle sine shock pulse is used, and In the actual situation, when the shock pulse acts on the gyroscope, a waveform like a half-cycle sine wave pulse emerges at the beginning. However, these do not immediately vanish, and a series of instantaneous oscillation waveforms emerge before the waveform gradually disappears. This process is difficult to express with a clear mathematical expression, and it is usually necessary to convert the expression of the shock motion in the time domain into the frequency domain-that is, Fourier transform of the shock function. Its Fourier transform is:maxa of the shock response is directly related to the damage and failure caused by the shock of the device, which leads to the maximum shock response spectrum.Shock response spectrum is a concept commonly used in engineering . The IntA is 50 g, and the duration D is 11 ms) shock response spectrum of an undamped single degree of freedom linear system given in the International Electrotechnical Commission (IEC) international standard shock test method.There are two ways to obtain the shock response spectrum, which are measured through experiments, and calculated by the known half-cycle sine response spectrum. A and D values. The converted expression is,amax is acceleration maximum response, A is shock amplitude, D is shock duration, and fn is natural frequency of the system.The generalized frequency axis and the normalized maximum response axis are located below the frequency axis, and left of the maximum response axis, respectively. The shock response spectrum of normalized coordinate can be used to convert the half-cycle sine shock response spectrum of different Vibration modes are inherent and integral characteristics of elastic structures. Through the modal analysis method, the characteristics of the main modes of the structure in a certain susceptible frequency range can be analyzed, and the actual vibration response of the structure under various external or internal vibration sources in this frequency band can be predicted. Therefore, modal analysis is an important method for structural dynamic design and equipment fault diagnosis.The sensitive unit of the quartz tuning forks resonant gyroscope fabricated by MEMS process is shown in rf) will be superimposed on the driving frequency  to generate another frequency. As rf is a random value between 20 Hz and 2000 Hz, the superimposed frequency (r + ffth7) is also a random frequency within a frequency range. Significantly, if there are sixth resonance frequency (fth6) and ninth resonance frequency (fth9) within this range, it is possible to generate an output signal when the gyroscope has no angular velocity input. To ensure that the gyroscope is not affected by random vibration, fth6 and fth9 should not appear in this frequency range. To better ensure that the gyroscope works normally under random vibration, this frequency range is defined as (7thf − 2500) to (7thf + 2500). From As there is random vibration in the frequency range of 20~2000 Hz in the working environment of the gyroscope, the frequency of the random vibration  in Before analyzing the shock resistance of this tuning fork gyroscope model, it is necessary to analyze the modal of this model and find the natural frequencies of the first four order modals. Then the shock pulse of the gyroscope model in six shock directions  is simulated. Three of the six shock directions are shown in amax and fn of this shock response spectrum is:According to the Formula (6), the Input this shock response spectrum is inputted into COMSOL Multiphysics, then the spectrum is drawn as shown in This part introduces transient dynamics simulation and shock response spectrum simulation. Transient dynamic analysis is to determine the displacement, stress, strain and force of the structure under the random combination of static load, transient load and simple harmonic load. Shock spectrum analysis provides an analysis function that is different from transient response. In the analysis, the excitation of the structure is represented by small components, and the response of the structure to these components is the combination of the maximum response of each mode of the structure.In transient dynamics simulation, the time range of transient simulation is 0~10 ms, the shock is 2 ms and the half-sine shock pulses in six shock directions, the shock pulse starts from 0 ms and lasts for 2 ms. When the shock is loaded on the model within the first 2 ms, the deformation generated by the model increases, and the deformation is the largest at around 1 ms. After the shock duration has passed, the model still vibrates and the deformation becomes smaller and smaller, as shown in The shock response spectrum of 2 ms and six shock directions half-cycle sine shock pulse are applied to the structure with the V-shaped beam of many kinds of thicknesses . Through the simulation analysis, the stress distribution and maximum stress value of these gyroscope modals during the half-cycle sine shock pulse are obtained.When the V-shaped beam thickness is 100 μm and the structure is exerted on the shock response spectrums of six shock directions, the structural deformation and stress distribution are shown in When the V-shaped beam thickness is 60 μm, the structural deformation and stress distribution are shown in The maximum stress value with different V-shaped beam thicknesses are shown in 7thf) of the MEMS quartz tuning forks resonant gyroscope with a V-shaped beam thickness of 80 μm analyzed and tested by the E4990A Impedance Analyzer is 11.104 kHz. The driving frequency (7thf) obtained by the simulation is 11.095 kHz.Before undergoing the high-acceleration shock test, the driving frequency  in six shock directions are obtained by simulation and analysis of many kind of V-shaped beam thicknesses of the MEMS quartz tuning forks resonant gyroscope. Simulation results show that when the V-shaped beam thickness is 80 μm, the driving frequency is 11.104 kHz, and the maximum stress value emerged by the shock is 94.721 MPa. It is verified that the gyroscope with a V-shaped beam thickness of 75 μm is failed after the shock by experiments. The experimental results are consistent with the simulation results. The existing shock resistance technology is compared in The simulation of shock resistance is carried out in six directions, and the shock resistance experiment is tested in the same six directions. The stress distributions and deformations exerted by the half-cycle sine shock pulse (


Purpose: The use of a mandibular advancement device (MAD) in the treatment of obstructive sleep apnea (OSA) is a consolidated therapy. This study aimed to evaluate the predictive value of awake upper airways (UA) functional endoscopy in identifying the outcome of MAD therapy. Methods: This observational prospective study included 30 adult OSA patients, all patients underwent pre-treatment awake UA functional endoscopy, during the exam subjects were instructed to advance their mandible maximally, and they were divided into three different groups according to the response of the soft tissue, group A (expansion), group B (stretch), group C (unchanged). The results of this test were used in combination with other noninvasive indexes to predict the treatment outcome in terms of apnea-hypopnea index (AHI) reduction. Results: We found that a substantial AHI reduction occurred in group A and group B while e slight AHI reduction was measured in group C. Conclusion: Based on our experience the awake UA endoscopy is a valid prognostic exam for discriminating responder and non-responder patients; in addition our results indicate the possibility of predicting a range of post-treatment AHI index values. Several treatment options are available for the effective management of obstructive sleep apnea (OSA). Surgery plays an important role and various authors have described many techniques with good results ,2,3,4,5.Many authors ,9 considThis observational prospective study included adult OSA patients that were referred to the UOC MaxilloFacial Surgery, IRCCS Foundation Policlinico A. Gemelli of Rome-Italy, from June 2017 to December 2019.The study was conducted on 30 patients, including 16 males and 14 females, with a mean age of 52.93 ± 8.26 years.The study included adult patients with OSA symptoms who had an apnea-hypopnea index (AHI) > 5 (on PSG).All patients were CPAP intolerant and refused surgical any treatment.Patients on CPAP treatment, patients undergoing diet therapy for weight loss, or that had previously undergone surgical treatment for OSA were excluded.All patients were aged >18 years.All patients had an indication for MAD therapy after evaluation by a dentist expert in sleep medicine.After recording the medical history, all patients underwent a general, dentist and upper airway examination, Epworth Sleepiness Scale (ESS) analysis, and Obstructive Airway Adult Test (OAAT); the OAAT questionnaire has shown to be a valid tool for the diagnosis of mild , moderate OSA , and severe OSA  . These eEndoscopy was performed using a flexible nasoendoscope (FN) . The computerized system included a camera and a light source. The software enabled video recording.The patients were invited to lie down on the ambulatory bed. After the lubrication of the FN, we inserted it into the nasal cavity and proceed with the exploration of the UA. In O space (oropharynx), we invited the patient to simulate snoring. This maneuver gave us an indication of what happens at level O during snoring. Furthermore, this maneuver allowed the patient to relax, accustom himself to the instrument’s presence and stay less contracted.-with a significant (>25%) widening of the airspace [enlargement-expansion (e)];-with a stretch (<25%) of tissue only slightly affecting the airspace [stretch (s)];-without a response, i.e., the airway’s volume remains unchanged [unchanged (U)].We invited the patient to advance the mandible maximally and we observed the soft tissue response. The soft tissues of level O can respond to advancement in the following ways:We performed the Muller maneuver to evaluate the lateral collapsibility of the UA. We stopped for a few seconds at level H  to allow the patient to relax and get ready for the descent of our instrument. At level H, we evaluated the position of the lingual base towards the epiglottis and both structures towards the posterior wall of the pharynx. We invited the patient to advance the mandible maximally and observed what happened at the base of the tongue and epiglottis. The soft tissues of level H can respond to advancement in the same way as those of the O space. During the examination, we evaluated any signs of gastroesophageal reflux disease (GERD) and evaluated the presence of any neoformations to report to an ENT colleague for further diagnostic investigations. Returning to the O level, the patient was again invited to advance the mandible maximally, and we observed the soft tissue response. We determined whether the response of the soft tissues to mandibular advancement was improved or remained unchanged. After removing the flexible endoscope from the explored nostril, we introduced it into the contralateral nostril and performed the examination only at level N. To evaluate the obstruction of the UAs, we have applied the NOHL classification presented by Vicini et al.  in 2012.Patients underwent standard full-night PSG . The obtained tracks were interpreted by a qualified sleep disorder technician using the diagnostic criteria established by the AASM in 2007 and then revised in 2012 . The PSGA two-piece appliance used to move the mandible forward was custom-made for each patient. Mandibular advancement was set at 60% of the maximum mandibular protrusion at first. The patient was then re-evaluated every 30 days and the appliance was incrementally titrated to the level of protrusion that allows the best improvement of snoring and daytime symptoms without causing discomfort. After 6 months PSG was repeated. If the patient reported no improvement of snoring and daytime symptoms, the MAD advancement was increased by 1 mm, and the patients were re-evaluated every 15 days, if at 90% of the maximum protrusion the patient still reported no improvement, a confirmation PSG was performed and therapy was suspended.Statistical analyses were performed with the software R  (3.5.2 release) . The R pIn this study, we evaluated the possibility of identifying OSA patients who would benefit from MAD treatment by performing an awake UA functional endoscopy. For this purpose, during the awake UA functional endoscopy subjects were instructed to advance their mandible maximally, and they were divided into different groups according to the soft tissue response. Three types of responses were identified: enlargement (group A), stretch (group B), and unchanged (group C). The results of this test were used in combination with other noninvasive indexes to predict the treatment outcome in terms of AHI reduction. These indexes included the Epworth scale, snoring index, OAAT score, and the AHI assessment before the MAD treatment. The possible effects of age and sex were also evaluated. The main demographic and clinical characteristics of the patients are summarized in p-values for group A and group B support this finding but also reflect a larger sample size, especially for group A.In To investigate the extent of the MAD-induced modifications more in-depth, we compared the achieved AHI perceptual reduction A and theThese quantities are defined as follows:p-value (0.28), the AHI difference in group A has a larger range than that in group B. The absence of such an effect in the AHI percentage difference , which is significantly different from groups A and B. Interestingly, we did not observe any significant difference between groups A and B, which remarkably showed a very similar AHI percentage reduction with almost overlapping IQRs. A closer inspection of fference A probablfference .For the sake of completeness, we also investigated the presence of correlations among the different markers, taking into account the determination performed before and after the intervention. For this purpose, Spearman’s correlation coefficients were computed and displayed as a correlation map . This maAn attempt to predict the extent of the MAD-induced modifications in terms of AHI reduction was made using a linear regression analysis. For this purpose, we first included in our model all the markers measured before the intervention as independent variables. The results of the advancement mandibular test and sex were included as dummy variables. The regression model pointed out that the only significant variables were the AHI value before the intervention and class membership in group A (expansion), B (stretch), or C (unchanged). This finding was also strengthened by a stepwise regression, which selected the same two variables (see Patients and Methods for more details).Following this analysis, we showed the AHI reduction as a function of the initial AHI for the three groups A. A lineInterestingly, intercepts are consistent with zero at the 0.05 confidence level in all cases. This might indicate that, in the absence of obstructive sleep apnea (AHI = 0), the treatment is not expected to cause any improvement, on average. The AHI reduction increased linearly with the AHI value before the intervention, indicating that a large reduction is possible only in the case of a large initial AHI. A dramatic difference between the slope trend of group C and those of groups A and B reflects a larger AHI reduction for patients belonging to the latter classes. The confidence bands in The prediction of the effectiveness of MAD in the treatment of OSA is a major problem that significantly limits the use of this therapeutic approach. Other aspects that limit the expansion of MAD use in the treatment of OSA are the difficult calibration of the device, the need for dentists specialized in sleep medicine and the high cost of the device associated with the uncertainty of the therapeutic result. In our experience, the most common concerns of patients regarding MAD therapy are related to the high cost of the device  and to the uncertainty of therapy success. Many authors suggest diagnostic analysis or exams to predict the outcome of MAD therapy and exclude non-responder patients. However, different authors ,20 affir-it is based on drug-induced sleep, and there are documented problems and concerns about the safety, efficacy, and methods of administration of these drugs;-patient hospitalization is needed for procedure execution;-it is performed during an intermittent short sleep cycle and does not permit examination of multiple airway levels simultaneously during the same cycle;-there are no clear standards for therapy with MAD planning based on data of the degree/pattern of obstruction;-the validity and reliability of DISE results are yet uncertain;-it is costly and not widely available for many patients.In the literature ,22, it iTo avoid the aforementioned limitations of DISE, we think that functional endoscopy of the upper airway is a simple, low-cost, convenient, informative, and noninvasive method. Many authors have tried to demonstrate the association between functional endoscopy of the upper airway and MAD therapy with few results. In the last few years, many authors have begun to demonstrate the capacity of functional endoscopy of the upper airway to discriminate responder and non-responder patients. In 2014, Sasao ,26 affirBased on our experience the awake UA endoscopy is a valid prognostic exam for discriminating responder and non-responder patients. In our experience, the Muller maneuver has no predictive function and is found to be completely unreliable. Our results demonstrated that there are significant differences among the three groups in many terms. We observed a significant reduction in the AHI index in patients with a severe apnea index compared to those with a moderate or mild apnea index. Interestingly, the data show that a substantial AHI reduction occurred for group A (expansion) and group B (stretch), while a modest, albeit significant, decrease was measured for group C (unchanged). Interestingly, we did not observe any significant difference between groups A and B, which, remarkably, showed a very similar AHI percentage reduction. In our opinion, this could indicate that in many patients, the muscular contraction due to the presence of an FN in the UA hides the enlargement of the tissue so that only minimal movements and a stretch of the soft tissue is visible.Our statistical analysis demonstrates the correlation between the pre- and post-treatment AHIs and improvements in the OAAT and Epworth Sleepiness Scale scores.Finally, our results indicate the possibility of identifying responder patients and predicting a range of posttreatment AHI index values. We believe that the main limitation of this study is the subjectivity of the analysis of the airway response to mandibular advancement. In the future, it would be interesting to consider the correlation between other assessments, like Friedman’s classification, and the response to MAD therapy .We think that UAs awake functional endoscopy can be a discriminant exam to distinguish responder vs. non-responder patients in OSA therapy with MADs. The results obtained in this study, supported by results evidenced in the international literature, allow us to compare the predictive value of this exam with the results obtained with DISE but with minor risks, costs, and problems for the patients. Finally, the possibility of predicting the posttreatment AHI can help clinicians understand whether MAD therapy is an optimal choice for patient treatment.


As one classical anomaly detection technology, support vector data description (SVDD) has been successfully applied to nonlinear chemical process monitoring. However, the basic SVDD model cannot achieve a satisfactory fault detection performance in the complicated cases because of its intrinsic shallow learning structure. Motivated by the deep learning theory, one improved SVDD method, called ensemble deep SVDD (EDeSVDD), is proposed in order to monitor the process faults more effectively. In the proposed method, a deep support vector data description (DeSVDD) framework is firstly constructed by introducing the deep feature extraction procedure. Different to the traditional SVDD with only one feature extraction layer, DeSVDD is designed with multi-layer feature extraction structure and optimized by minimizing the data-enclosing hypersphere with the regularization of the deep network weights. Further considering the problem that DeSVDD monitoring performance is easily affected by the model structure and the initial weight parameters, an ensemble DeSVDD (EDeSVDD) is presented by applying the ensemble learning strategy based on Bayesian inference. A series of DeSVDD sub-models are generated at the parameter level and the structure level, respectively. These two levels of sub-models are integrated for a holistic monitoring model. To identify the cause variables for the detected faults, a fault isolation scheme is designed by applying the distance correlation coefficients to measure the nonlinear dependency between the original variables and the holistic monitoring index. The applications to the Tennessee Eastman process demonstrate that the proposed EDeSVDD model outperforms the traditional SVDD model and the DeSVDD model in terms of fault detection performance and can identify the fault cause variables effectively. With the increasing scale and complexity of modern industrial processes, timely fault diagnosis technology is gaining its importance because of the high demands for plant safety and process continuity. Due to the application of the advanced data acquisition and computer control systems, huge volumes of data are collected so that data-driven fault diagnosis methods have been one of the most popular process monitoring technologies in recent years ,2,3. IntSupport vector data description (SVDD) is firstly proposed by Tax and Duin for the one-class classification problem ,7. For tThe unsupervised learning characteristic of SVDD is especially suitable to the data-driven fault diagnosis problem. In the industrial processes, most of the collected operating samples are normal while the faulty data are very rare. Even if some fault data exist, they are often unlabeled. Therefore, the statistical process monitoring can be also viewed as the anomaly detection problem. In fact, there have been some scholars applying SVDD algorithm to industrial process monitoring and fault detection. By regarding the process monitoring as one-classification problem, Ge et al.  designedAlthough the present studies demonstrate the practicality of SVDD in some fault diagnosis cases, there are some related problems deserving deep studies. One problem is how to improve the feature extraction capability for the high-dimensional nonlinear chemical process data. The conventional SVDD may not provide accurate data description in the complicated cases. The main reason is its intrinsic shallow learning structure. It is investigated that the SVDD only involves one feature extraction layer based on the kernel function. Therefore, the data description ability of SVDD is limited. In recent years, deep learning theory is thriving in the machine learning and data mining field. Deep neural networks learn the abstract representations of data automatically by multiple feature extraction layers, and have achieved great success in speech recognition, face recognition and drug discovery etc. ,22,23. SMotivated by the above analysis, this paper proposes one ensemble deep SVDD (EDeSVDD) method for nonlinear process fault detection. The contributions of this method are three-fold. (1) we introduce the idea of deep SVDD into the process monitoring field and build one deep SVDD (DeSVDD) based fault detection method for deeper data feature representation. To our best knowledge, there is no studies introducing the fault detection applications of DeSVDD. (2) we present a model ensemble strategy based on Bayesian inference to enhance the monitoring performance of DeSVDD. One problem of DeSVDD based fault detection is that the model structure and the initial weights may affect the model training results. It is difficult to select the optimal model structure and initial parameters in the unsupervised case. So we design the ensemble DeSVDD model by integrating the different models for a holistic monitoring. (3) A novel nonlinear fault isolation scheme based on the distance correlation is designed to identify the possible cause variables. This scheme indicates which original variables are responsible to the occurrence of fault by measuring the nonlinear association. Finally, in order to validate the effectiveness of the EDeSVDD, the applications on the benchmark Tennessee–Eastman (TE) process are illustrated and discussed.The rest of this paper is organized as follows. SVDD is a widely used anomaly detection method. Its main idea is to first map the training data in the original space into a high-dimensional feature space and then seek a minimum-volume hypersphere as a boundary of the training samples ,7. The dn samples, where m variables, SVDD projects the original data onto feature space by introducing a nonlinear transformation Given the normal training data set as R, which is written by: In the feature space, an optimization objective is designed to find the minimal hypersphere with the center To solve the above optimization problem, a Lagrange function is constructed as follows:plied as :(9)K desc desc8) ds (USVs) . The hypFor any test vector given by :(12)Dt=|By combining Equation , the SVDR, we can judge the condition of process data.If In the SVDD algorithm, the feature extraction and data description procedure can be depicted as shown in The deep learning based SVDD model was firstly discussed in Ruff et al.’s work ,25, whicn samples For the training data The goal of deep SVDD is designed as Equation , which iion item .(15)minIn this optimization objective, the training samples are mapped to the features close to center Usually, we can learn the network parameter set be noted .Remark 1.The determination of hypershphere Equation , it is sRemark 2.The hidden layer activation functions should have no bias item Remark 3.The unbounded activation function such as ReLU should be applied to avoid the hypersphere collapse. If the deep SVDD network is equipped with the bounded activation function, it may be saturated for all inputs.For the testing sample w is the smoothing parameter, KDE is an effective non-parametric tool for estimating the probability density function (PDF) of a random variable . For themated as ,27:(17)fBy the KDE technique, we get a reasonable threshold for the monitoring index R is dependent on the pre-determined parameter R has different values. This way it is difficult to explain the probabilistic meaning of the data description. Therefore, even for the SVDD model, KDE is also very suitable and adopted in this paper.In the SVDD model, the anomaly detection based on Although the DeSVDD provides the potential to extract the more meaningful features for fault detection, its shortcomings should also be noted. One is about the model structure determination. That is, how many layers should be used for the given data set and how many nodes should be used in each layer? In fact, this is an open problem in the deep learning category. Up to now, the present deep learning studies do not give a conclusion about this issue. In many supervised learning cases, researchers can adopt the trial-and-error strategy to choose a reasonable but not optimal network structure. Another problem involves the weight initialization. As the deep SVDD network adopt the stochastic training algorithm, the trained models may be different for each running. This is not a serious problem in the supervised case as the users can adopt the best one according to the classification performance. However, the DeSVDD is intrinsically dealing with one unsupervised problem and it is difficult to determine the optimal initial parameters based on the training data.Considering the unsupervised property of the DeSVDD, the ensemble learning strategy is applied to build the improved DeSVDD method for dealing with these two above issues. In this method, a two-level ensemble framework is designed, which is shown in K groups of the model structures and the second level provides M groups of the initial parameter sets for each structure, the total number of DeSVDD models B is computed as In the strategy ,29.b-th (For the testing vector puted by :(20)PbρBased on the distance correlation coefficient, we can obtain the associations between the monitoring index and the original variables. For the variable The value of Offline modeling stage:Collect the historical normal data and divide them into the training data set and validating data set;Normalize all the data sets by the mean and variance of the training data set;Apply the normal data set to pretrain the Train the multiple deep SVDD networks as the basic monitoring sub-models.Compute the monitoring indices Construct the ensemble statistic Online monitoring stage:Collect testing sample and normalize it with the mean and variance of the training data set;Project the normalized data onto each deep SVDD model and obtain its monitoring indices Compute the ensemble index When a fault is detected, the fault isolation map is built to identify the fault cause variables.Process monitoring based on the EDeSVDD mehod can be divided into two stages: offline modeling and online monitoring. In the offline modeling stage, the normal data are collected and trained for the EDeSVDD modeling, while in the online monitoring stage, the new data are collected to project onto the built model and the monitoring index is computed for process condition judgement. The specific steps are as follows:The Tennessee–Eastman (TE) process was applied to evaluate the proposed method. This process is from a real chemical process and has been one benchmark system used to test the different control and diagnosis methods ,35,36,37This process had 41 measurement variables and 12 operational variables. A total of 52 variables were involved in fault diagnosis and 21 fault patterns are designed for fault diagnosis method testing. The detailed fault descriptions are listed in Three methods of SVDD, DeSVDD, and EDeSVDD were applied to detect the faults. The SVDD parameters were set as terature . ELU funThe fault F10 was taken as the first example, and its monitoring charts of three methods are listed in Another illustrated example is Fault F19, which was one unknown small-amplitude fault. Regarding this fault, the SVDD monitoring chart is shown in A complete FDR comparison of three methods is listed in Besides the fault detection performance, the false alarming performance is also one important evaluation aspect. In all the tested fault data sets, the first 160 samples belonged to the normal condition. We computed the false alarming rate (FAR) on these samples and tabulated in the In this section, three methods of SVDD, DeSVDD, EDeSVDD were applied for fault detection and their characteristics are summed up as follows. For the traditional SVDD method, it used a shallow learning model with only one kernel mapping layer. This method had no random weights, so ensemble learning was not involved. As to the DeSVDD method, it applied the deep learning model with multiple feature mapping layers. However, DeSVDD may lead to unstable monitoring results due to different random weight settings. For the EDeSVDD, it combined the deep learning and ensemble learning to overcome the disadvantages of these two mentioned methods. It should be noted that the EDeSVDD method had the highest model complexity. This is understandable in view of the no free lunch theorem. For achieving better monitoring performance, a more complicated model is needed.Next, we discuss the results of fault isolation. Fault F10 is taken as one example. When this fault occurred, we collected the corresponding fault data and computed the fault association degree with the monitoring index by the Pearson correlation The fault isolation results of fault F19 are shown in In order to provide effective monitoring on the chemical processes, this paper designs an improved SVDD based fault detection method by combining the deep learning and ensemble learning. Deep learning is applied to present a deep feature description for the SVDD modeling, while ensemble learning is utilized to overcome some essential disadvantages of deep neural networks. At the same time, a fault isolation map based on the distance correlation is applied to locate the fault cause variables. We test the proposed method on the benchmark TE process. The application results show that the deep learning is effective and the DeSVDD model can improve the FDR of the basic SVDD model for most of the fault cases, and the EDeSVDD model can provide a better monitoring performance with the higher FDR and the lower FAR. However, some problems deserve the further studies. One is about the declining performance in some fault cases. Although DeSVDD outperforms the SVDD in general, it does worse in some cases. A possible reason involves the optimal deep model. To get a more rational reason and further give some viable solution is one valuable topic.


The unavailability of large amounts of well-labeled data poses a significant challenge in many medical imaging tasks. Even in the likelihood of having access to sufficient data, the process of accurately labeling the data is an arduous and time-consuming one, requiring expertise skills. Again, the issue of unbalanced data further compounds the abovementioned problems and presents a considerable challenge for many machine learning algorithms. In lieu of this, the ability to develop algorithms that can exploit large amounts of unlabeled data together with a small amount of labeled data, while demonstrating robustness to data imbalance, can offer promising prospects in building highly efficient classifiers. This work proposes a semisupervised learning method that integrates self-training and self-paced learning to generate and select pseudolabeled samples for classifying breast cancer histopathological images. A novel pseudolabel generation and selection algorithm is introduced in the learning scheme to generate and select highly confident pseudolabeled samples from both well-represented classes to less-represented classes. Such a learning approach improves the performance by jointly learning a model and optimizing the generation of pseudolabels on unlabeled-target data to augment the training data and retraining the model with the generated labels. A class balancing framework that normalizes the class-wise confidence scores is also proposed to prevent the model from ignoring samples from less represented classes (hard-to-learn samples), hence effectively handling the issue of data imbalance. Extensive experimental evaluation of the proposed method on the BreakHis dataset demonstrates the effectiveness of the proposed method. Breast cancer is one of the most frequent cancers among women and the second most common cancer globally, affecting about 2.1 million women yearly. Statistics from a global cancer report recorded that an estimated 627,000 women died from breast cancer in 2018 . ComputeExisting manual methods for breast cancer diagnosis include the use of radiology images in identifying areas of abnormalities. These images, however, cannot be used to accurately determine cancerous areas . Biopsy Deep convolutional neural networks (CNNs) have achieved tremendous successes in several disciplines including but not limited to object detection , 10, segIn spite of these successes, it is also pertinent to note that the deep layers associated with CNN models imply the fact that they require large amounts of well-labeled data during training to achieve satisfactory results. Training on relatively small amount of data leaves the models prone to overfitting and, subsequently, poor generalization. In the medical imaging domain, obtaining abundant labels for image samples is a major challenge, not to mention that a large amount of image samples are also required to aid in a model's ability to generalize well on data. Again, the process of labeling image samples is a time-consuming and an expensive one, requiring expertise knowledge. Existing methods mentioned in the literature that perform classification of histopathological images resort to training CNN models with random initialization and data augmentation techniques in a bid to improve a model's performance , 25, 26.In , the autIn , the aut  We propose a novel semisupervised learning framework that utilizes self-training with self-paced learning in classifying breast cancer histopathological images by formulating the problem as a loss minimization scheme which can be solved using an end-to-end approach.  We introduce a novel pseudolabel generation and selection algorithm for selecting pseudolabels with relatively high-confidence probabilities to augment the training samples for retraining the model. In retraining the model, the optimization process begins by selecting pseudolabeled samples with relatively higher confidence (“easy” samples) then gradually adds “hard” samples to the training data. This ensures the selection of pseudolabels with high precision and prevents mistake reinforcement.  To tackle the issue of class imbalance associated with self-training methods when generating and selecting pseudolabels, we implement confidence scores that use class-wise normalization in generating and selecting pseudolabels with balanced distribution.  We obtain significant accuracy performance on the BreakHis dataset compared to the state-of-the-art approaches.To this end, this work proposes a novel semisupervised learning framework that uses self-training and self-paced learning (SPL)  to classWe provide an overview of the formulation of the problem as a loss minimization scheme which can be solved using an end-to-end approach. The concepts of self-training and self-paced learning as applied to the proposed scheme are also presented.xs, ys and the target data xt, yt:For a given number of sample classes, the classification task is defined as a standard softmax loss on the labeled source data as inputs θc that can correctly classify target samples at the time of testing, with minimal loss. Nonetheless, based on the assumption that there is usually a limited amount of labeled target data , effective transfer of representations becomes limited. Consequently, a classifier abandons the less-represented class samples in the learning process, focusing only on well-represented class samples. This ultimately impedes the classifier's ability to learn robust representations. The two key issues of learning the classifier lie in an effective formulation of a score function and a robust formulation of the loss function. Again, the robustness of a learner depends on the formulation of the loss function to relieve the influence of noisy and confusing data [Il denotes the image in the source domain indexed by l=1,2,3,…, L. 𝒴l,n represents the true labels for the nth image  for Il. W denotes the network weights. Pn is the softmax output containing the class probabilities. Similar definitions hold for It, 𝒴t,n and pn during evaluation. This problem formulation is different from [In equation , the aiming data . Moreoveing data , 41 provent from  where thIt. Similar to [Semisupervised learning approaches typically adopt self-training to utilize unlabeled samples –45. Basemilar to , all unlTo incorporate the self-paced learning and self-training scheme, the loss function is modified as follows:𝒴 is assigned to zero, implying that kc is introduced. The selection of a large quantity of pseudolabels is synonymous to a large value of kc. Adding kc in equation  w.r.t. L1 regularizer in a bid to avoid the scenario where most of the pseudolabels are ignored. In solving the pseudolabel framework optimizer, the work in [The introduction of a class-wise bias by normalizing class-wise confidence scores distinguishes this formulation from the one proposed in  where th work in  utilizedpn). Inherently, such an approach does not handle the issue of class imbalance. To resolve this, equation )/(exp(−kc)) in equation  is equivalent to the probability ranked at iteration (p∗Nc), with Nc being the number of images predicted as class C. For each unlabeled sample, the maximum output probability M was taken in descending order and these probabilities are sorted out across all samples. Optimizing the pseudolabels resulted in the p × 100% most confident pseudolabeled samples to be used in training the model . Such a scheme ensures that the probability ranked at p × 100% is taken independently from each class to (1) threshold the confidence scores and (2) normalize the confidence scores. p is first initialized with 10% of the most confident predictions and at each additional round, the top 5% is added to the next pseudolabel generation and selection process.To regulate the amount of pseudolabeled samples to be selected to update the model in each training iteration, We have carried out experiments on the BreakHis dataset . The Breβ1=0.9, β2=0.99 and an initial learning rate of 0.001 which is decayed via a polynomial decay scheduling  were used. We present and discuss results for both binary and multiclass classification tasks.The experimental outcomes for the binary classification task are shown in Kc pseudolabeled samples via the proposed schemed was a vital key in controlling and determining the amount of pseudolabeled samples to be selected in updating the model at the next iteration. The selection scheme, coupled with the self-paced learning and self-training approach ensured that classes with the least representations which would have otherwise been ignored, was still selected and added to the training samples. This proved to be an effective and efficient step in the learning process. Again, the results in Kc pseudolabels proved to be a more effective approach rather than using all the pseudolabeled samples. The accuracy results obtained with the proposed approach show significant accuracy gains.The generation and selection of the top The accuracy and loss plots for 40X and 100X are shown in Figures The pretrained model used as the baseline model was trained on the ImageNet dataset  as against the BreakHis dataset which contains breast cancer histopathological images. As such, at the start of supervised training stage, the model begins to learn the relatively new patterns from the target domain (breast cancer images) resulting in the spikes as depicted in the plots. However, past epoch thirty, a drastic drop in loss value is observed and the accuracy values increase steadily. At the end of epoch fifty, the loss value is greatly reduced and the training and validation accuracy (for both the supervised learning stage and the self-training stage) are almost aligned. This is an indication that the proposed approach also effectively curbs overfitting. The imbalanced nature of the BreakHis dataset implies that accuracy alone cannot be used to access the performance of the model. Results for precision, recall, and F1-score values are also presented in Kc pseudolabels to augment the training samples in the next training iteration proved to be more effective than selecting all the pseudolabels. This outcome further rubber-stamps the significance of Kc in the proposed approach.The accuracy results for the multiclass classification are summarized in The plots for loss and accuracy (for images scanned at 40X and 100X) are shown in The confusion matrices also bring out the imbalance in the dataset. The ductal carcinoma class has more samples than the remaining classes with the adenosis class having the least number of samples. As a result, these two classes represent the most and least number of samples, as depicted in Kc in determining the proportions of pseudolabels to be used in updating the model in each training iteration and also prove that adding samples in an “easy-to-hard” approach ensures that even the least-represented samples are still considered in the training process. Overall, these schemes resulted in the model being very versatile and robust even in the face of the similarities and coherence between the images samples in the dataset.The accuracy, precision, recall, and F1-score values as well as the confusion matrices all show the effectiveness of using We compare the performance of the proposed approach with other works mentioned in the literature as shown in In , the autIn , the autIn this work, we have used 70% of the data for training at the supervised learning stage and the remaining 30% was added to the test set which was used as unlabeled data for the self-training stage. The selection of the most confident pseudolabeled samples to augment the training sample has been proven effective in providing the model with reliable samples, and ultimately expanding the training set, thereby making more data available to the model (to satisfy the hunger of deep models for more data). The effectiveness of the proposed method is evident in the results obtained, which depict significant accuracy improvements compared to the abovementioned methods which are mostly supervised learning approach where only labeled data was used. The proposed algorithm has been tested on breast cancer histopathological images since it is in line with our research objective. Therefore, we are quick to add that, the significance of the proposed algorithm is not limited or specifically designed for breast cancer classification. Based on the results obtained, we are confident that this algorithm can be extended to other classification tasks in medical imaging or computer vision that seek to employ semisupervised learning techniques in solving various tasks.Obtaining a significant amount of well-labeled data in the medical domain is a challenging task and more tedious is the task of accurately providing labels to data. In this work, we have proposed a semisupervised learning scheme that integrates self-paced learning paradigm and self-training for training a model on both labeled and unlabeled data. Self-paced learning plays a vital role in curbing the issue of mistake reinforcement, where wrongly generated pseudolabels are reinforced into the training sample. In the light of selecting pseudolabels with the most confident probabilities, we show a novel selection algorithm was proposed to present the CNN model with only the most confident pseudolabels. Experimental results obtained using the top 5%, 10%, and 20% generated pseudolabels for training showed significant accuracy improvements for both binary and multiclass classification task when compared with state-of-the-art approaches. For future work, we intend to incorporate diversity into the self-paced learning scheme and as well as incorporate the similarities in feature space of histopathological images. A combination of these elements into the self-paced learning scheme will result in a versatile and robust learner.


A 14-year-old girl presented with an insidious onset of left hip pain, limp, and intermittent fever for a 3-month duration. Patient had a history of toe walking since childhood which continued into adolescence. On radiographic investigations, she was found to have a dysplastic hip with fluid collection around the hip which was surgically drained. The microbiological investigations proved the presence of Mycobacterium tuberculosis (TB). Accordingly, she was started on anti-TB chemotherapy as per drug sensitivity. TB infection in a previously neglected dysplastic hip is not reported as per our knowledge and poses unique diagnostic and management difficulties. Tuberculosis (TB) of the hip is the second most frequent site of occurrence after spine in osteoarticular TB. Many patients get diagnosed in the advanced stage of the disease owing to atypical clinical presentations despite the presence of an extended spectrum of modalities. In most cases of TB of the hip, there is a dilemma of accurate diagnosis owing to its presenting complaints of painful and restricted movements of the hip because several pathologies may mimic this presentation. In the pediatric age group, diseases such as juvenile idiopathic arthritis, Perthes disease, and pyogenic arthritis are differentials. Largely based on clinical suspicion, multiple radiological and microbiological investigations help in the early diagnosis of TB of the hip. With late presentations and diagnoses, the extent of the disease becomes increasingly severe with profound destruction of the joint. We present a case of a 14-year-old girl with TB of hip in a pre-existing unstable hip secondary to a neglected dysplastic hip.Statement of informed consent: The patient was informed that the data including clinical photos concerning the case would be submitted for publication, and a written consent was taken accordingly.Mycobacterium and liquid broth culture and antibiotic sensitivity was also carried out. The results came back positive for TB with sensitivity to all first-line antitubercular drugs. Pulmonary TB was ruled out by a chest radiograph and sputum microscopic examination and culture in both solid and liquid media. We do not perform QuantiFERON-TB Gold in tube for our patients owing to its questionable role in high endemicity countries and high rates of false positives. A skin tuberculin testing was not performed because of the history of Bacillus Calmette-Guerin (BCG) vaccination. The patient was started on weight-based first-line antitubercular chemotherapy with isoniazid, rifampicin, pyrazinamide and ethambutol, and hematinics for a period of 3 months. She showed both clinical and serological signs of recovery, gradually over the next 3 months. This chemotherapy was later converted to a two-drug regime with weight-based isoniazid and rifampicin for a period of 7 months titrated with serological and clinical recovery as per the institutional protocol. The antitubercular chemotherapy was stopped after a period of 10 months when the patient had resolution of pain. The lurch persisted, and the patient was counseled regarding the need for a replacement options for the dysplastic hip. The patient was concerned regarding the ability to squat and sit cross legged. At the 1-year follow-up, the patient resumed all her activities including squatting and sitting cross legged and remained painless with residual lurch and limb shortening on the left side  with derotation, and limited varization if needed, tight capsulorrhaphy, and appropriate pelvic reconstruction  with an average age at operation around 10.6 years (range 8 to 18 years) reporting good results in 79% cases.3 They offer that older children presenting with unilateral or bilateral dislocation of the hip should be treated, unless there is an underlying systemic disorder contraindicating the operation. Addition of TB to this already existing controversy further complicates the matter and limits the treatment options to few. The pain, loss of movement, and progressive development of deformity results in the loss of function of an already compromised hip with a poor overall function. Early diagnosis is of paramount importance in the treatment of TB and its outcome. In the past 3 decades, extensive research was performed on the diagnosis and management of TB.4 Besides clinical features, serological, radiological and molecular diagnostic methods including polymerase chain reaction are available easily. Osteoarticular TB still comprises a significant number of patients among the total TB burden in our country. The cases of multidrug-resistant TB is also on a rise secondary to inappropriate antitubercular chemotherapy without drug testing. Synovial TB is primarily considered to be a medical disease and surgical intervention is reserved for advanced joint destruction. In this case, surgical options regarding reconstruction of the hip were not available owing to a previously dysplastic state, and hence, more stress was given to the medical mode of treatment. The morphological classification proposed by ShanmugasundaramDevelopmental dysplasia of hip encompasses a wide spectrum of anatomic hip abnormalities that affect the development and stability of the hip. These abnormalities can range from subtle acetabular dysplasia to complete subluxation or dislocation of the femoral head. Developing countries do offer the other end of spectrum of this disease for a dislocated hip, many presenting after the walking age. Presentation in the periadolescent age group is also not infrequently found. The aim of treatment usually in cases presenting early is to achieve a stable and concentrically-located joint and satisfactory development of the hip which is achieved surgically. It is assumed that the outcome of treatment is inversely related to the age at presentation. Left untreated, these dysplastic hips lead to an abnormal gait pattern and osteoarthritis by early adulthood.ng up to  cm with 6 Until that time adequate counseling, reassurance and periodic monitoring are all that is required to be performed. To our knowledge, this is the first such case being reported in the literature.Double attack at the hip with dysplasia and TB poses double set of problems, both for the patient and the treating surgeon. The primary aim should be targeted toward an early diagnosis of the infection, early institution of medical management, and uneventful resolution of the disease so as to save adequate anatomy for future reconstruction of the hip.