Today, the Intelligent Transportation Systems (ITS) are already in deep integration phase all over the world. One of the most significant enablers for ITS are vehicle positioning and tracking techniques. Worldwide integration of ITS employing Dedicated Short Range Communications (DSRC) and European standard for vehicular communication, known as ETSI ITS-G5, brings a variety of options to improve the positioning in areas where GPS connectivity is lacking precision. Utilization of the ready infrastructure, next-generation cellular 5G networks, and surrounding electronic devices together with conventional positioning techniques could become the solution to improve the overall ITS operation in vehicle-to-everything (V2X) communication scenario. Nonetheless, effective and secure communication protocols between the vehicle and roadside units should be both analyzed and improved in terms of potential attacks on the transmitted positioning-related data. In particular, said information might be misused or stolen at the infrastructure side conventionally assumed to be trusted. In this paper, we first survey different methods of vehicle positioning, which is followed by an overview of potential attacks on ITS systems. Next, we propose potential improvements allowing mutual authentication between the vehicle and infrastructure aiming at improving positioning data privacy. Finally, we propose a vision on the development and standardization aspects of such systems. Today, the technology is penetrating most of the modern digital systems . BillionDifferent standardization communities are already working hard to standardize the technological requirements, thus aiming for the same goal: to develop a unified ecosystem that would allow reliable, fast and secure communications between vehicles and roadside infrastructure. Such technologies as Dedicated Short Range Communications (DSRC) in the USA ,11 and EPreviously, V2X development was not so widespread as smart cars were limited in numbers, highly priced and thus inaccessible to the majority of the world’s population. In this paper, we will focus only on the vehicular positioning privacy in an urban scenario where a conventional Global Navigation Satellite System (GNSS) can be affected negatively by propagation in complex environments issues . NowadayIndeed, GNSSs proved themselves to be inefficient in urban-canyon operation scenario  also knoMany solutions were proposed to mitigate the challenges of efficient vehicle location determination. Simultaneously, operation in the city allows broad communication possibilities due to high coverage of various wireless technologies including conventional cellular and other heterogeneous techniques ,25.Overall, several radio-ranging-based cooperative positioning (CP) techniques were already proposed to enable vehicular localization in urban environment . NonetheThe internode distance is commonly measured using radio-ranging or range-rating techniques such as the time of arrival (TOA), time difference of arrival (TDOA), received signal strength (RSS), and other techniques . The infSince vehicular positioning mostly relies on the data provided by the Location Solution Provider (LSP), many concerns arise in regard to controlling personal information of the user . Potentian overview of existing privacy-related V2X solutions for infrastructure-based ITS systems;a modified solution for data privacy enhancement based on well-known protocol;a discussion of possible cybersecurity attacks on mentioned systems;an overview of present standardization and General Data Protection Regulation (GDPR) related activities.From the V2X positioning perspective, researchers have proposed a considerable number of privacy-preserving protocols for data exchange between the vehicle and the surrounding nodes . Most ofThe rest of the paper is organized as follows. First, we survey the existing solutions for vehicular localization in In this section, we consider several approaches for the localization techniques and the corresponding benefits and drawbacks. First, we list various potential ways to determine vehicle location with a simplified representation given in One of the GNSSs is the Global Positioning System (GPS) which is a technology, network, and service owned and maintained by the USA . The GPSThe second GNSS giant is the Russian Global Navigation Satellite System (GLONASS) which serves the same role despite some minor differences . GLONASSFinally, GNSS positioning technology widely used in the European Union is GALILEO starting 2005 . GALILEOThe fundamental principle of using the GNSS system is to determine the location by measuring the timing of the reception of a synchronized signal from the navigation satellites ,48. DespTo start with, the material penetration characteristics limit the use of GNSS in hard-reach places such as underground parking, tunnels, etc. Next, because of the near-spherical shape of the Earth calculating an accurate distance between two points requires the use of spherical geometry and trigonometric math functions. However, many applications calculate an approximate distance using simplified ones. Thus, the GNSS-based calculation of the distance between two objects can lead to errors of 10 percent or more . The mosSome vehicles are already equipped with conventional IEEE 802.11 wireless modules commonly referred to as Wi-Fi. By using previously estimated maps together with sensed Received Signal Strength Indicator (RSSI) along with the Base Station (or access point) Identificator (BSID), the vehicle can approximate its location in the complex metropolitan environment, which is generally a challenging task for GNSS . This meThe main drawback of the technique is the need for continually updating the RSSI maps since the changes in the environment  affect the RSSI, and Wi-Fi is operating in unlicensed spectrum affected by many interference sources. Therefore, the Wi-Fi network faces changes over time, and these changes need to be monitored continuously in order to rely on Wi-Fi positioning. Nonetheless, another drawback of using Wi-Fi for vehicular positioning is the lack of centralized infrastructures and the difficulty to achieve a high quality of positioning system, due to significant fluctuations in RSS of Wi-Fi APs. The advantage of using a Wi-Fi network is the possibility to serve in the metropolitan scenario with complex GNSS propagation characteristics, where GNSS is likely to fail. Similar issues could be encountered while utilizing any short-range wireless technology, such as Bluetooth Low Energy (BLE)  or ZigbeThe other technique that could be utilized by smart cars is so-called cellular-positioning that has been used for localization of phones over the decades. Already in 2G, the Cell-ID-based positioning with accuracies on the order of a few hundred meters was feasible . The effThe evolution of wireless communications brought us to the next frontier of positioning as well . Moving This method enables non-instrumented vehicles to determine their locations by collecting data from neighboring vehicles through V2V direct communications, Light Detection and Ranging (LIDAR) techniques, radars, etc. The node-centric approach specifies the routing path as a sequence of connected nodes. Every vehicle has an opportunity to communicate and sense some or all of its neighbors as depicted in a Fog Internet of Vehicles paradigm .As an example, to localize a vehicle among its neighbors, the authors in  propose Despite actual communication devices utilizing mmWave band, radars open another prospect for vehicle localization, especially significant for autonomous driving since most of the vehicles of this type are already equipped with some LIDARs and radars. Trailblazing work  shows thHuman-centric localization is targeted at people carrying gadgets ,69. As sThere have been numerous studies focusing on human-vehicle interaction ways of driver identification using sitting postures were investigated in , the intAs positive aspects, we can distinguish the high accuracy in location determination owing to the availability of base station cell IDs and, consequently, the availability of data for calculation of the approximate vehicle location . On the Generally, verifiable multilateration is a technique based on the measurement of the difference in distance between two or more stations at known locations by broadcast signals at known times. Recently, researchers have proposed a number of positioning and distance estimation techniques in ,77. ThisSimultaneous reception: In this method, the vehicle communicates with at least three base stations in order to obtain its current coordinates. Certainly, in the real world, it is impossible to get three signals perfectly synchronized.Security: In the communication process, it is vital not to allow the attacker to receive the transmitted data. In ,79 the pConfidence or trustability of the access nodes: Before transmitting the information, it is required to ensure that the base station is trustworthy valid by employing mutual authentication. Some of the approaches to mutual authentication were described in ,81. ThisAnonymity: In some situations, the base station is not supposed to obtain any information about the vehicle, neither the identification nor the location. In this paper, we show several protocols fulfilling mobile node anonymity requirement.This approach poses a number of problems, namely:Previously listed methods allow for the design of an effective hybrid system meeting the necessary security requirements and being both accurate and fast in determining the location . A numbeThe grouping could be done as following. As for the first strategy, we assume having ‘beacon’ and ‘common’ nodes in the system. The modification is such that the beacon node is a node where a passenger with a smartphone is present. Furthermore, the smartphone can identify the location more accurately due to cellular signal presence. Common nodes are the nodes where the smartphone does not transmit any information about the location. In this case, the beacon nodes are polled to receive it. At the same time, all vehicles monitor their surroundings for infrastructure units and corresponding RSSI values. The protocol execution example is shown in A beacon vehicle has a GNSS receiver and a list of available infrastructure access points (APs). At this moment, a common node is location-unaware. Hence it requests the beacon node for a list of available infrastructure APs. In the reply, the beacon node also transmits an approximate location in addition to requested data.Indeed, the positioning techniques as we know them today are relying on their own collected location data. The systems of tomorrow would also rely on neighbors and infrastructure thus moving the information security aspects to an entirely new level.First, the ‘indirect’ distance to the static trusted nodes obtained from the known units (cellular or infrastructure units) is estimated. Mutual authentication also takes place during this phase.Next, the distances are utilized to estimate the location of the node through classical geometry by, for example, triangulation.In this section, we consider possible protocols for data exchange between a vehicle not possessing reliable location information and its surrounding roadside elements and other traffic participants carrying location data. The crucial feature of these protocols is the presence or absence of anonymity of the vehicle requesting location information. The anonymity of the request for location is an essential property of the protocol, and its availability requires significant complication of the protocol. The anonymity in this work is considered as the process of ‘hiding’ the actual vehicle location. Generally, the localization process could be viewed as follows.We further list the potential options for securely executing the first phase.V—the vehicle, and S—the base station. Key constructs utilized by this work are given in In 2004, authors proposed a distance-bounding protocol in , we furtm is only feasible for the vehicle and the BS.First, a shared symmetric key must be generated between the vehicle and the infrastructure node for secure exchange of information. Key generation algorithm depends on the computing power modules installed in the vehicle and base station. Now, it is presumed that such a link has already been established and the data exchange between the station and the vehicle takes place via the secure medium. It is also presumed that a shared symmetric key was delivered after the devices have established a secure link employing, for example, the well-known Diffie-Hellmann protocol. Thus, common pairwise key h  . This ten—is a message length; Here, we compare the discussed protocols and also elaborate on the improved one. Further on, D5)—O(n) ; Signatun) [O(n) ; Verificn) [O(n) ; Measuren) [O(n) ; Calcula(log(n)) ; XOR—O(n)) [O(n) ; and Hom[O(λ3.5) .From the In this section, we list the attacks explicitly dangerous for location data exchange with another nodes. Note, we further elaborate not only on the attacks explicitly related to the exchange of the data between the vehicle and infrastructure node.Attackers are mainly focusing on two main sides while penetrating ITS systems. First, to get unauthorized access based on weaknesses in social engineering and/or physical protection of infrastructure elements and onboard controllers. Second, to exploit the design of the security protocols and communications mainly in terms of message modification or replaying previously received messages . Most ofConventional and Infrastructure-Related AttacksMan-in-the-middle attack (MITM):  is an active attack where an eavesdropper can intercept and modify fully or partly the positioning signaling between two vehicles or between a vehicle and the terrestrial infrastructure used for positioning, such as cellular signaling [ignaling . IncreasReplay attack: This attack is similar to the previous one. Here, the attacker can replay the messages traveling between vehicles or vehicles and infrastructure thus affecting the correctness of the positioning itself instead [ instead .Sybil Node (Rogue AP): This attack and hardware/software unauthorized modifications are another active attacks, where BS or AP would transmit intentionally erroneous positioning-related data [ted data . To copeted data .Jamming attack: The attacker can be either a stationary or a moving jammer. Given the nature of the mobility of the vehicular networks, it is rational to have a moving jammer that tracks the desired node and causes regular interference. Utilizing spread spectrum techniques could assist in addressing this attack. As one of the options, the Frequency Hopping Spread Spectrum (FHSS) makes the incoherent signal period impulse-noise to the eavesdroppers. On the other hand, Direct Sequence Spread Spectrum (DSSS) symbolizes each data bit in the original signal by multiple bits in the transmitted signal [d signal  which isPassive Eavesdropping attack: In this type of passive attack, an attacker can listen to the positioning signaling between two vehicles or between a vehicle and the infrastructure [tructure . If the Message modification attack: This attack is targeted at altering the message during or after transmission (active attack) [ attack) . The advKey and/or Certificate Replication: This attack involves the system endorsement with a node of similar identity. The execution of the attack relies on the key management or certificates replacement, in order to forbid the identification and abuse the authorities [horities .While most of the security threats could be mitigated by employing the integration of stronger information security systems, privacy-related attacks could not be overcome by mere modifying the hardware/software side of the system.Attacks Related to Distributed OperationCongestion attacks: Generally, urban traffic congestion is considered as one of the major problems from the vehicular systems’ operation perspective. Modern cars are already available to report the congestion-related data either with integrated wireless modules or via hand-held user devices. Focusing on the first case, Vehicular Ad-Hoc Networks (VANETs) can provide timely information to the surrounding nodes which result in more effective route selection (around a sudden traffic jam) using the information obtained from other cars. From the information security perspective, the V2V radio connectivity of autonomous vehicles will offer attackers opportunities to combine multiple compromised vehicles into botnets of cars, which will lead to other serious security consequences [equences .From the infrastructure perspective, such attacks may potentially lead to incorrect traffic light signaling. As a result, the optimization algorithm running in the ITS cloud may operate based on the incorrect data, leading to congestion .Repudiation attacks: This attack is an example of adequately tracking and recording the user actions, which can maliciously manipulate or fake the identification of new nodes [ew nodes . This atRouting Loop Attack: In this type of attack, an internal attacker receives a message, updates it and sends it back to one of the previous forwarders (or the source) even if there is a better node in its routing table that is available to be the next forwarder according to the routing strategy. This attack aims to delay or prevent the delivery of a message [ message . The pri message .Sinkhole Attack: In this type of routing attack, the attacker  attempts to announce the nodes in the network with a different location in its beacon messages followed by drops of any other packets [ packets . It resuWormhole Attack: This type of routing attack corresponds to cases of two terminals having a link in between and they resend messages received by one of them to another [ another . TherefoUnauthorized use of location data and location-based services: This threat could be considered as unauthorized use of tolled highways pretending to have a position of a neighbor vehicle which is not situated on a highway or unauthorized access in a car park [car park .Disclosure of unwanted information: As an example, consider a case such as if a person leaving the house empty for a long trip by car or by boat (which may enable house burglaries); how often an employee, supposed to be at the office, is visiting places by car ; how often a spouse is visiting places that he/she has never mentioned to his/her relatives (which may cause family crises), etc.Tracking malware applications: A vehicle-installed software could contain malware to “steal” the tracking patterns of the users inside the vehicle, i.e., the user positions and speeds in time; such information, if sold to car dealers, could adversely affect the price of a vehicle, e.g., in function of the types of roads where vehicle and how often it was driven, etc.Scrambler attack: An additional measure to achieve a higher level of anonymity in vehicular systems is the utilization of dynamically changing identifiers (from MAC to application layer), so-called pseudonyms [eudonyms . The effAs compared to security threats, privacy issues always had more involvement by the actual humans. One option to improve privacy in ITS systems is to utilize the solutions with strong anonymity properties ,113 or fThis section provides an overview of the future development of privacy strategies in positioning from EU regulations perspective in addition to conclusions of this work.profiling and processing the location information of an individual could be done concerning personal aspects which are naturally related to person’s movement and location data. According to [personal location data should be treated more carefully especially for cases when the identifiability of the said individual could be established. This way, the GDPR aspects related to the protection of personal data also should be taken into consideration while operating with personal location data.Recently introduced General Data Protection Regulation (GDPR) policy broadly states that rding to , it coulIn the ITS case and focusing on conventional GNSS systems, all the supported systems are controlled by the authorities, namely military for GPS and GLONASS and, in contrast, civilian Galileo in EU, which makes the implementation of GDPR more complicated. In order to overcome this issue, European Telecommunications Institute (ETSI) is actively developing a standard EN 303 413  aiming tIndustrial giants, such as Huawei, already rise the question on how to address the challenges of private positioning in future networks . One of Aiming to achieve the above-mentioned, we have shown different location estimation techniques for ITS scenario in this paper. We analyzed the existing protocols for the information exchange between the vehicle and the base station during the localization process. Subsequently, we presented an improved protocol for the data exchange, where much attention was paid to the security of signaling information transmission. Using the modified protocol, the vehicle can also rely on the location data without disclosing the identity, and thus the privacy could be generally improved.In real life, the selection of the protocol highly depends on the application needs. If the system architect wants to have maximum anonymity, it is recommended to utilize the modified protocol, but it is the most difficult one for implementation. The optimal option is to use a protocol with a property of anonymity but BS-based. The complexity is not much higher compared to others, and there are no complex operations.


As ultrasonic wave field radiated by an ultrasonic transducer influences the results of ultrasonic nondestructive testing, simulation and emulation are widely used in nondestructive testing. In this paper, a simulation study is proposed to detect defects in a circular tube material. Firstly, the ultrasonic propagation behavior was analyzed, and a formulation of the Multi-Gaussian beam model (MGB) based on a superposition of Gaussian beams is described. The expression of the acoustic field from a linear phased-array ultrasonic transducer in the condition of a convex interface on the circular tube material is proposed. Secondly, in order to make the tapered probe wedge better fit the curved circular tube material and carry out the ultrasonic inspection of the curved surface, it was necessary to pare the angle probe wedge. Finally, acoustic field simulations in a circular tube were carried out and analyzed. The simulation results indicated that the method of ultrasonic phased-array inspection is feasible in circular tube testing. Tube materials with different curvatures need different array element lengths and widths to get the desired focused beam. Tubular parts have been widely used in petroleum, natural gas, chemical industry, aerospace, ship, construction, and nuclear industry. These industries play an important role in national economy, national defense construction, and high-tech-related fields ,2,3. ThePhased-array ultrasonic imaging has developed as a result of the improvement of electronic information technology and has made great progress in the medical field. However, in the industrial field, although, to a certain extent, the problems existing in conventional ultrasonic have been solved, it still faces many challenges. These include the fact that the object structure and the material of industrial ultrasonic imaging are very different, the large amount of industrial phased-array ultrasonic imaging data, the complex hardware architecture , the badIn actual testing, wedges are always used to couple the array to the tested material. In the inspection of circular tube material, according to its convex shape, the probe wedge needs to fit different inner diameters. So, acoustic field simulations are carried out under the convex cylindrical interface between the wedge and surface of the circular tube material. In the present study, a phased-array ultrasonic transducer was placed on the surface of the material, scanning along the circumferential direction of the material, as shown in In order to simulate the acoustic field, various models have been developed ,14,15,16In this paper, a simulation method based on MGB is proposed to detect the defects in a circular tube material. Firstly, the expression of the acoustic field in the condition of a convex interface was derived according to MGB and ray acoustics theory, and then acoustic field simulations in the circular tube material were carried out. In order to make the tapered probe wedge better fit the curved circular tube material, it was necessary to pare the angle probe wedge for ultrasonic inspection of the curved surface. Finally, the acoustic field distribution when the probe was placed in water and in the condition of a convex cylindrical interface between the probe with a tapered wedge and the circular tube material was obtained. The main contributions of our article include: (1) A complete model was established to detect the defects in circular tube material. The model was divided into two sub-models: the acoustic field simulation model based on MGB and the model to pare the angle probe wedge for ultrasonic inspection of curved surface. The two sub-models were then combined, so as to be applicable for the detection of internal defects in the circular tube material. (2) Acoustic field simulations were carried out in the circular tube material. The simulation results showed that, to obtain optimal focusing beams, the tube material with different curvatures needed elements with different sizes.1) and the detected circular tube material was the solid medium (V2). That is to say, issues associated with a phased-array ultrasonic detection with a wedge can be simplified into liquid–solid interface problems. The transmission of a Multi-Gaussian beam through the interface in the x–z plane is shown in The solid wedge is considered as an equivalent fluid medium because the liquid couplant between the wedge and the detected circular tube material cannot carry any shear stress. Thus, in our model, the wedge was perceived as liquid medium −iwunAn ultrasonic phased array pulses the individual probe elements at slightly different times according to different time delay rules, so that the individual wave fronts generated by each element in the array combine with each other to add or cancel energy in predictable ways that steer and shape the sound beam. So, the acoustic field from a linear phased-array transducer in liquid (tapered wedge) and solid  media can be expressed respectively as:The linear phased-array ultrasonic probes are usually used with tapered wedges. Their characteristics are described below.The time delay in a tapered wedge is the time required for ultrasonic propagation in the wedge, as shown in According to Snell’s law, the wedge internal incident angle The phased control probe midpoint (effective launching point) height The wave propagation distance in the wedge is Finally, the delay time in the wedge can be written as:Time delays are calculated based on ray acoustics theory. Phased-array probes are used to transmit and receive ultrasonic beams . The lauAccording to Snell’s law, the time of wave propagation trough the distance The delay time of the nth element relative to the first one can be written as:The focusing beam from the linear phased-array transducer in the tapered wedge is shown in From Assume the total propagation time of the first pulsed element is The ultrasonic angle probe is used to detect material with a small radius curvature . The refIn 3 kg/m3, and the velocities in the wedge and in the material were 2730 m/s and 3230 m/s, respectively. The wedge was made of Plexiglas, whose density is 1.18 × 103 kg/m3. The center frequency of the linear phased-array ultrasonic transducer was 5 MHZ, the element width 2a was 4 mm, the element length 2b was 13 mm, and the inter-spacing between the centers of adjacent elements d was 5.9 mm. The distance between the probe and the incident point was 17.1 mm. In this paper, we implemented the simulation method in MATLAB R2012a and windows 7.Considering a longitudinal wave, acoustic field simulations were carried out in a circular tube material using the model described above. The material was steel, whose density is 7.8 × 10The acoustic field distribution of the beams is shown in There are two factors affecting the beam steering and focusing behaviors: one is the surface curvature of the circular tube material, the other is caused by the length and the width of the element. The effect of the surface curvature of the circular tube is discussed first.To see how the curvature parameters affect the characteristics of the propagating beam, we artificially changed the curvature parameters of the circular tube material in the case that the length and the width of the element were 13 mm and 4 mm, respectively. Clearly, something changed when the curvature was between the 1/0.6 and 1/0.35. Different diameters of the tube affected the focusing behavior of the beam because of their different surface curvature, as shown in In addition to the beam focusing ability of the linear phased-array affects caused by changing the curvature of the circular tube material, the length and the width of the element also play an important role in beam propagation. Thus, the effect of the length and width of the element is discussed. The simulation results showed that the tube material with different curvatures needed elements with different sizes to get optimal focusing beams. In future work, we will perform inspection experiments of the defect on the tubular parts using the linear phased-array ultrasonic transducer, and further research will be carried out.Furthermore, the probe was simulated on the basis of the finite-element method. The acoustic field in the case of a probe with two elements is shown in The experimental apparatus employed in this work consisted of five components: the excitation and acquisition equipment, the probe, the tested sample, a water injection device, an encoder for recording the UT data, as illustrated in In order to meet the strong demand of automated quantitative ultrasonic nondestructive testing for circular tube materials, the MGB, based on the superposition of Gaussian beams, and the expression of the acoustic field from a linear phased-array ultrasonic transducer under condition of a convex interface on the circular tube material were proposed. Ultrasonic phased-array inspection technology was proposed to detect defects in circular tube materials. Acoustic field simulation in the detected circular tube material was carried out. The simulation results indicated that a linear phased-array ultrasonic transducer could produce routine focusing beams in the material, which indicated that the method of ultrasonic phased-array inspection is feasible in circular tube material testing. The simulation results showed that circular tubes with different curvatures and elements with different widths and lengths can produce different focusing beams. Finally, we performed inspection experiments to identify defects in the circular tube material using the proposed linear phased-array ultrasonic transducer and showed that the proposed model allows a reliable, accurate, and detailed defect characterization.


The glass transition in soft matter systems is generally triggered by an increase in packing fraction or a decrease in temperature. It has been conjectured that the internal topology of the constituent particles, such as polymers, can cause glassiness too. However, the conjecture relies on immobilizing a fraction of the particles and is therefore difficult to fulfill experimentally. Here we show that in dense solutions of circular polymers containing (active) segments of increased mobility, the interplay of the activity and the topology of the polymers generates an unprecedented glassy state of matter. The active isotropic driving enhances mutual ring threading to the extent that the rings can relax only in a cooperative way, which dramatically increases relaxation times. Moreover, the observed phenomena feature similarities with the conformation and dynamics of the DNA fibre in living nuclei of higher eukaryotes. Glass transition in soft materials can be affected by the topology of constituent particles, but the detail remains elusive. Here, Smrek et al. show that the interplay between circular topology of ring polymers and their active segments generates a new state of matter, namely active topological glass. Ultra-soft particles can be realized experimentally using polymers, such as long polymeric stars or rings, which are highly deformable but possess a fixed topology imposed during the synthesis. While it is known that the polymer topology has strong impact on the stress relaxation mechanism3, it has been questioned if the topology can independently induce a glass transition4.Tremendous interest has been devoted to understanding of the glass transition driven by an increase in packing fraction or a decrease in temperature in soft and deformable systems5. The rings cannot cross each other, but are known to thread—one ring piercing through an eye of another ring, which temporarily topologically constraints the motion of the two rings. Mutual threadings of many rings can yield a conformation that is relaxed only if a cascade of threadings is sequentially undone, which could give rise to very long relaxation times, e.g., exponential in the ring length6. The glassy behavior in an equilibrium melt of rings has been observed in computer experiments, but only under a pinning perturbation, which immobilizes a fraction 5. Unfortunately, the conjectured critical length, 90 entanglement lengths, of unpinned rings is currently beyond the reach of experiments and, although the pinning deepens our theoretical understanding of the glass transition9, creating it experimentally to drive the topological glass transition would be challenging.Such a topological glass has been conjectured for the system of long, unknotted, and nonconcatenated polymer rings in dense equilibrium solutions10. While, intuitively, activity opposes glassiness by enhancing mobility of the particles, some active models can exhibit a more complex behavior as a function of the active control parameters. For example, increasing the persistence time of the active Ornstein-Uhlenbeck particles can either glassify or fluidize the active system, depending on the particular state point, as a result of nontrivial velocity correlations in the system14. Indeed, some system properties, such as the time-dependent effective temperature are pertinent to active fluids and render also the corresponding glass transition distinct from the passive one. In particular, the location and the existence of the glass transition of active fluids are dependent on the microscopic details of the activity mechanism. Nevertheless, close to the transition region, universal features of the passive glassy dynamics have been found recently for active spin-glasses10 and self-propelled particles in the non-equilibrium mode-coupling theory (MCT)14. For instance, the scaling of the relaxation time with activity control parameters is governed by the same exponent as in the passive MCT.Whereas many questions remain open in the traditional glass transition of passive Brownian particles, recently a whole new research direction has been opened by considering system composed of active particles that are driven by non-thermal fluctuations19. There, a range of dynamic phenomena arises as a consequence of the competition between directed flows, characteristic for active matter in euclidean space, and the ‘hedgehog theorem’20 that asserts the existence of topological defects in a smooth vectorial field on a sphere. In contrast to these studies, where the topology is a global property of space, here we focus on a system where the active particles themselves are not point-like but feature intrinsic circular topology, the embedding space being Euclidean with periodic boundary conditions.The impact of topology on the active glassy states has been studied almost solely in the context of active particles confined to a topologically nontrivial space, in particular a sphere21 to collective cell migration in tissues22. While on the subcellular level, glass-like properties have been attributed to the high crowding, as well as size and interaction heterogeneity of the constituents23, the confluent tissues modeled using vertex models exhibit a new type of rigidity transition at constant density without24 and with active motion25, attributed to an interplay of cells’ shape and persistence of motion. Although there is no topology involved in these models and, therefore, they are inherently different from our present work, the transition occurs due to shape changes at constant density similar to the case studied here.Glassy dynamics also appears in various biological contexts ranging from the bacterial cytoplasm26, where the monomers are arrested due to their nearest neighbors, here the centers of mass of the whole chains are inhibited due to multi-body, long-range effects of topological constraints. After detailed account of the physics of the active topological glass, we discuss its relevance for the organization and dynamics of chromosomes in living eukaryotic cells.For dense solutions of ring polymers, we show that making the rings locally more mobile by introducing a moderate segmental activity, the system reaches a glassy state with dramatically slowed-down relaxation. This novel state of matter, the ‘active topological glass’, is a consequence of the interplay between internal polymer topology, activity, and the crowded polymer environment. In contrast to the conjectured equilibrium topological glass, here no imposed pinning is necessary and only relatively short rings are sufficient to observe the transition. Moreover, contrary to well-studied polymer glasses28. A consecutive segment of length We start with a large, well-equilibrated, concentrated solution of 31. Simultaneously, we observe a gradual but dramatic conformational change of the rings as revealed by the mean radius of gyration After switching on the activity, the initial equilibrium uniform spatial distribution of the active (hot) and the passive (cold) segments alters Fig. a. They pWe observe that the hot segment is usually localized at one of the ends of a tree-like, doubly folded conformation Fig. b. This sentclass1pt{minimaentclass1pt{minimaentclass1pt{minima28. The following regime shows again the directional dynamics of the rings and explains its origin. While the passive segment, and hence the whole center of mass, move superdiffusively , whereas for longer lag times .We now show that the mutual ring threadings are responsible for the glassiness. We analyzed the ring threadings using computationally spanned minimal surfaces on the ring contours Fig. a. An int47. We found that more ring pairs are involved in threadings and that they are progressively deeper compared with equilibrium . To prove at least that ring topology and threadings are essential for the glass transition, we took a configuration of the system in the glassy state and cut the bond connecting two cold monomers in the middle of the cold segment on each ring. We further simulated the system which now consisted of 50. As the non-topological properties of the system remained unchanged, we conclude that the phase segregation is not a crucial element stabilizing the glass. The glassiness is driven by the enhanced threading due to the ring topology and the violated detailed balance29.Although in equilibrium the threading depth is correlated with the diffusion slow-down of individual rings⟩∕t Fig. b that th51. In the continuous case, the non-ergodicity factor grows smoothly from zero to finite values. This second scenario is less common and its realization requires the presence of additional control parameters, such as porosity and randomness38, tunable attraction widths in the interactions53 or confining periodic potentials54 for which the amplitude and the wavelength can be independently varied. In such cases, lines of continuous glass transitions have been found in parameter space, and they are associated with higher-order singularities of the We now return to the question of the order (continuous or type-A vs. discontinuous or type-B) of the glass transition for the system at hand. The usual control parameters driving vitrification in molecular or colloidal systems are the temperature and the density, and the typical glass-transition scenario there is discontinuous: the intermediate scattering function in the ergodic state develops a plateau, which grows in height and extends longer in time approaching the glass transition as the control parameters are changed54. The presence of a subdiffusive regime in the mean-square displacements  of the chromatin fiber also varies. While conformations inferred from contact probability measurements exhibit knots65, the knot analysis66 of the direct observations of fluorescently labeled chromatin segments finds mostly unknotted segments. Unable to refine the scales of the topological constraints completely at this point, we assumed their existence for the typical observation times and examined the consequences. Therefore, we mention phenomena on chromatin at various scales, for which the interplay of the topology and the activity could be relevant.While our model system is interesting from the fundamental physics point of view, it is also inspired from biology and may well bear important ramifications for the organization of the DNA fiber (chromatin) of higher eukaryotic cells. The large-scale static properties of the equilibrium melt of unknotted rings, such as the territorial organization, the scaling of mentclass2pt{minim67 as opposed to energy-deprived cells, and has its origins in the energy dependent processes, such as chromatin repair or remodeling, DNA transcription, or loop extrusion. We conjecture that the phenomena observed in our partly active system could also be relevant in biological context, on the basis of the following similarities with our model: genes exhibit size increase upon transcription decoupled from the chromatin decondensation68; a highly transcribed gene shows directed motion69; overall chromatin loci exhibit heterogenous subdiffusive dynamics67 reminiscent of glassy behavior70; the active and inactive chromatin are spatially segregated71; and the chromatin exhibits a doubly folded structure at small scales72. Naturally, these effects have also alternative explanations. The phase separation and glassy dynamics could be observed for a copolymer models where different chromatin segments have different interaction potentials based on their epigenetic state74 or by interaction with binding proteins75. The double folded structure is likely to be attributed to supercoiling due to the torsional stress induced on the fiber in the process of transcription or loop extrusion77. Nevertheless, we show that the activity in combination with topological constraints at the fiber level can complement the observed phenomena and should be considered in a more complete picture of the chromatin organisation79. As the models above typically do not consider topological constraints and activity, our findings represent a completely new mechanism for the observed phenomena. Certainly, more work is required to determine the relative contributions of the different mechanisms in the various cases. Potentially, the different nature of the observed glassy states, namely the monomer glass due to caging in copolymer models70 and topological glass here, could be used to discern which one could be present or dominant in the chromatin context.The segmental activity with thermal-like fluctuation spectra has been measured to be stronger in the normal living cell nuclei2, the superdiffusive motion is connected to a major particle deformation at high density. In contrast, however, in our work, the superdiffusivity is triggered by the active noise. Second, a novel state of matter—the activity-driven topological glass—arises from the activity-enhanced ring threading. In contrast to the conjectured equilibrium topological glass5, our present model allows the creation of a topological glass for rings of accessible lengths (80 or synthetic ring polymers3, driven by molecular motors fueled by ATP hydrolysis. The effective temperature ratio 81. However, as our preliminary results suggest, even smaller temperature ratios can be sufficient. Other means of selective heating could be attempted by fluctuating external fields or infrared irradiation selectively coupling to individual groups in the polymer. The latter mechanism can yield a fluid material with reversible vitrification upon light exposure. What is the proper topological order parameter of the glass transition, or what are the fragility properties of the active topological glass, are just a few intriguing questions to be addressed in the future. The present work paves the way for a development and investigation of these novel, molecular, topology-based, responsive materials.Activity complemented with topological constraints at the microscopic level can lead to rich system dynamics. We have demonstrated this on a model system of dense, unknotted and nonconcatenated ring polymers with active segments. First, the directionality of polymers in a dense environment arises from the topological constraints and the isotropic noises of unequal strength. Similar to ref. All particles interact via a purely repulsive Lennard-Jones potential82. The equations of motion were integrated with the time step The studied systems are monodisperse at fixed volume with the total monomer density mentclass2pt{minim83 we know that such values of 31. Our preliminary studies show that even a weaker temperature contrast is sufficient for the glassy state to occur. We leave a more detailed characterization of the phase diagram for the future study.To model heterogeneous activity of the rings, on each polymer a consecutive segment of length 28 at 28, The employed model was frequently used not only for the melt of linear chains but also for the melt of ringsTo quantify dynamical evolution of the system, we consider squared displacements of the centers of mass of the rings, The relaxation dynamics of the system is also characterized by the self-part of the intermediate scattering function defined asWe define ergodicity breaking parameter as84 and followed the protocol in47 ensuring that for these ring lengths the area of the final surface is close to the minimum.The minimal surfaces are spanned on the fixed contours of the rings from the molecular dynamics simulations and then minimized using the overdamped surface tension evolution under the constraint of fixed disc topology. We used the Surface Evolver softwareThe separation length is defined assee Fig. a. \documThe threading cluster analysis is performed in the following way. At first, each ring is analyzed for threadings with other rings. Two rings are assigned to the same cluster if at least one of them threads the other one with To characterize the dynamics of threadings we compute the threading gain A related quantity characterizing the threading correlation is In addition, we compute the threading survival time distribution Supplementary InformationPeer Review FileDescription of Additional Supplementary FilesSupplementary Movie 1Supplementary Movie 2Supplementary Movie 3


The non-uniform response in infrared focal plane array (IRFPA) detectors inevitably produces corrupted images with a fixed-pattern noise. In this paper, we present a novel and adaptive scene-based non-uniformity correction (NUC) method called Correction method with Statistical scene-based and Interframe Registration (CSIR), which realizes low delay calculation of correction coefficient for infrared image. This method combines the statistical method and registration method to achieve a better NUC performance. Specifically, CSIR estimates the gain coefficient with statistical method to give registration method an appropriate initial value. This combination method not only reduces the need of interactive pictures, which means lower time delay, but also achieves better performance compared to the statistical method and other single registration methods. To verify this, real non-uniformity infrared image sequences collected by ourselves were used, and the advantage of CSIR was compared thoroughly on frame number (corresponding to delay time) and accuracy. The results show that the proposed method could achieve a significantly fast and reliable fixed-pattern noise reduction with the effective gain and offset. Infrared focal plane array (IRFPA) sensors are commonly used in military and civilian applications such as disaster assessment, security monitoring , medicalNUC methods are mainly divided into two categories: calibration-based correction methods (CBNUC)  and scenTo address the above disadvantage, many SBNUC methods were proposed ,14. The Statistical methods can make good use of the spatial information from the focal plane of each detector, and then fine tune the gain and offset to achieve NUC. The LMS (Least mean square) method  proposedCompared with statistical methods, registration-based correction method is more reasonable and converges more quickly . This meIn addition, the neural network-based SBNUC methods receive extensive attention because they not require high linearity and stability of detector coefficients. Hardie  proposedBased on the analysis of the advantages and disadvantages of existing methods, this paper proposes a hybrid correction method based on both statistics and registration. We use the statistical method to get gain coefficients and then use these gain coefficients as the initial values for registration methods to obtain offset coefficients more rapidly and accurately. In this way, both the spatial information between pixels and temporal motion estimation are considered in the process of non-uniform correction. Specifically, a block-based correction method based on statistical consistency of adjacent pixels is used to obtain gain coefficients and a registration method utilizing the phase-correlation algorithm that minimizes the mean square error between two properly aligned images is used to obtain the offset coefficient.This paper is organized as follows. The related work is introduced in Suppose ar model  as folloetectors ,19. WhenThe main purpose of the NUC algorithm is to estimate the value of In the correction method based on inter-frame registration , there iThe purpose of inter-frame registration is to minimize the error At this time, As with other optimization algorithms, using fewer pictures means lower convergence rate. Many methods can achieve lower convergence rate and the easiest way is to use appropriate initial values of w and n. Generally, consistent w  and b , or w and b corresponding to two points correction results are used as initial values for the registration method. However, these initial values need many frames to achieve relatively low NUC error because the correction results might not be appropriate for the current scenario , which wThe statistical methods usually achieve NUC by only correcting the gain coefficient. In the NUC method based on consistency of adjacent pixels , it is cFor the convenience of formula deduction, the ratio between a pixel and its adjacent pixels is defined asSetting the median of Obviously, this method uses iteration to calculate the gain coefficient of any big error in the picture sequence, which is caused by uncorrected flash pixel or occasional large shot noise that can lead to error transfer. As a result, the proposed statically method itself is not good enough. However, when combined with the registration method, the accuracy can be improved, which was proved in the experiments. Overall, the gain coefficient is obtained by recursion as follows.In the registration-based method, there is usually no research on the initial value of parameters, and the results of NUC mainly depend on the effectiveness of the registration. The statistical method’s purpose is to obtain the gain coefficient; it only processes the spatial information of single image, ignoring the sequential information between frames. Therefore, our paper innovatively combines registration-based and statistical correction-based methods together. The overview of our algorithm is presented in In the registration-based correction methods, the initial values of gain and offset coefficients are important for iterative convergence. Therefore, setting an optimal initial correction coefficient is the key factor for obtaining the result. If the coefficients can be set as appropriate values before iteration, the time for iteration would be shortened and the convergence value would be more accurate. In the method of IRLMS, the initial gain is set as an all-one matrix and the initial offset is set as an all-zero matrix. Different from this, our approach generates the initial gain coefficient by a modular pixel consistency method for the registration-based NUC method.The modular-based pixel consistency NUC method is mainly used to obtain the initial gain coefficient by analyzing the spatial relationship between pixels. Different from the previous methods, we utilize the statistical correction method in modules. It not only calculates the pixels of the whole image, but also divides the image into four blocks according to the center point for spatial information computation. This method is more reasonable and meticulous and can make full use of the spatial relationship between pixels. The details of the method are as follows.The calculation of the pixels in the whole image is shown in The calculation between adjacent pixels in modules is shown in Apart from considering the spatial information, we also integrate the interframe registration NUC method into our approach. The gain coefficient obtained as shown in Since the application of this paper is long range sky background and the optical focal length is relatively long, the images used in the correction method have little high frequency information. Thus, phase correlation  is used Firstly, the two inter frame images If the peak of the map Then, Equations (8) and (9) are used to iteratively converge to the optimal gain and offset coefficients to achieve NUC. Unlike the usual iterations, the initial gain is estimated in advance by the pixel consistency strategy, which could effectively improve the NUC performance.In this section, the proposed SIR method is compared with two representative NUC techniques: two-point NUC method  and IRLMIt is known that reducing the standard deviation of the image can improve the signal-to-noise ratio of the target and provide help for detection, by reducing false alarms or detecting longer distances. Thus, the global and local mean standard deviations were used to quantitatively evaluate the NUC performance on real infrared sequences. We tested the represented approach and our approach on three real infrared sequences collected by ourselves. In The results of the global mean standard deviation on each sequence are shown in These results also show that statistical methods have better accuracy in the two-picture sequence because they use the scenario itself to correct the result. However, in Sequence 2, statistical methods ahd poor performance, which was caused by flash pixel, while the CSIR method could improve the accuracy of the statistical methods in these three sequences.For a more detailed comparison, we also evaluated our method on the local mean standard deviation using Equation (21). By cropping the same position of the corrected image and calculating the average standard deviation of the area, we compared the corrective effect of different methods on the details. The results of the local mean standard deviation over each sequence are presented in To verify more clearly the advantage of low delay time (corresponding to frame number) for proposed method, we used three different scenario pictures and different frame numbers to test the two-point method, the IRLMS method, and the CSIR method.The IRLMS method, in contrast, shows poor performance when using relatively few pictures in the sequence. In this case, the IRLMS method can hardly remove the non-uniform noise. With the continuous iteration of IRLMS, the influence of non-uniformity on the image can be gradually eliminated. However, convergence time is important in many real-time situations. In such cases, this method converges slowly, and cannot be used.In contrast, the proposed method has the best visual effect. With this method, the effect of non-uniformity of focal plane can be corrected stably and clearly. At the same time, with the combination of inter-frame registration method, with the iteration of video sequence, there is no ghost or tail phenomenon during the correction process, as shown in In this paper, a novel statistical scene-based non-uniformity correction method with interframe registration is proposed. This method innovatively combines the registration-based method and the statistics-based method to realize NUC. Statistical-based correction methods are mainly used to obtain gain correction coefficients. We make full use of the spatial relationship between the pixels based on modular pixel consistency method. This method provides great help for the final correction by providing appropriate initial values for registration method, thus reducing convergence rate and improving NUC correction accuracy. Detailed experiments demonstrated our great performance. However, there are still some shortcomings to our approach: when flash pixel exists, the registration method is not good enough to have good NUC correction performance. In the future, we will combine more machine learning algorithms to improve the NUC performance and achieve better performance when flash pixel exists.


Electric power line equipment such as insulators, cut-out-switches, and lightning-arresters play important roles in ensuring a safe and uninterrupted power supply. Unfortunately, their continuous exposure to rugged environmental conditions may cause physical or electrical defects in them which may lead to the failure to the electrical system. In this paper, we present an automatic real-time electrical equipment detection and defect analysis system. Unlike previous handcrafted feature-based approaches, the proposed system utilizes a Convolutional Neural Network (CNN)-based equipment detection framework, making it possible to detect 17 different types of powerline insulators in a highly cluttered environment. We also propose a novel rotation normalization and ellipse detection method that play vital roles in the defect analysis process. Finally, we present a novel defect analyzer that is capable of detecting gunshot defects occurring in electrical equipment. The proposed system uses two cameras; a low-resolution camera that detects insulators from long-shot images, and a high-resolution camera which captures close-shot images of the equipment at high-resolution that helps for effective defect analysis. We demonstrate the performances of the proposed real-time equipment detection with up to 93% recall with 92% precision, and defect analysis system with up to 98% accuracy, on a large evaluation dataset. Experimental results show that the proposed system achieves state-of-the-art performance in automatic powerline equipment inspection. Powerline equipment such as insulators, cut-out-switches, and lightning arresters plays vital roles, on either the power generation or the distribution side, for the safe delivery of electricity to the end user . Since tRecent studies and advancements in the field of image processing and computer vision, especially the development of Convolutional Neural Networks (CNN) have enabled improving the performance of image analysis at the level of near human intelligence. On the other hand, with the rapidly expanding scale of power grids, traditional methods of manual inspection using electromagnetic field meters is far from satisfying the growing demand, due to the vulnerability to bad weather conditions and high prices ,5,6. ConZhang et al.  presenteEdge-based feature extractors were proposed in ,21 to deZhao et al.  proposedLi et al.  used verEven though there are several proposed systems for automatic insulator detection from aerial images, the high price, and low stability make them less practical. Moreover, aerial vehicles are more susceptible to ill weather conditions . Therefore, the ground vehicle-based systems were proposed for insulator detection ,27,28,29Oberweger et al.  presenteDeveloping a successful insulator monitoring system is a challenging problem due to the large variations in the appearances of insulators caused by scale, viewpoint, color and occlusion . ClutterIn this paper, we propose a complete real-time electrical equipment monitoring system which can detect various types of insulators from live video, taken from a dual camera system mounted on the rooftop of a ground vehicle, and subsequently analyze the images for potential defects. Since most of the electrical equipment discussed in this paper are insulators, we will use the term insulator to represent all other electrical equipment throughout this paper. The camera system contains two cameras inside its casing: (1) a low-resolution (LR) camera that takes a long shot image covering many objects of interests , and (2) a high-resolution (HR) camera that focus on to these objects of interest by panning, tilting and zooming. In order to assist the camera system to detect insulators in low-resolution and high-resolution images, we trained multiple CNN-based insulator detectors. To the best of our knowledge, there has been no study where CNN was effectively used to detect and classify different types of powerline insulators Unlike earlier studies that use handcraft features, we explore the robustness of the CNN features and use them for the task of multi-type insulator detection in a highly cluttered environment.(2)We present an ellipse detection method specifically designed for segmenting the caps from various types of insulators.(3)We propose a novel insulator rotation normalization method that normalizes the in-plane rotation of insulators, irrespective of their types.(4)We propose a novel defect analysis method that can detect gunshot defect in polymer insulators.(5)We present a complete automatic real-time multi-type insulator detection and defect analysis system.After the detailed review of the related work and introduction of the proposed method, we highlight the main contributions of this study as the following:The rest of this paper is organized as follows: in This section presents a detailed overview of the proposed system. A brief introduction to the types of insulators our system can detect is given in The overall system diagram of the proposed real-time multi-type insulator detection and defect analysis system is shown in The rotated high-resolution images still contain extra space around the insulators, therefore, another CNN-based multi-type insulator detector named HR_2 is applied to the rotated high-resolution images, which finally give the precise bounding boxes around different types of insulators, as shown in In contrast with the handcraft features used in the previous studies, the proposed system uses CNN for the detection of different types of powerline insulators. We believe that the lack of availability of the annotated insulator images is one of the reasons why CNN was not used in previous studies. In our work, we acquired and annotated a large and diverse set of insulator images  to train the CNN-based multi-type insulator detectors.We used Darknet’s open source neural network framework and object detection system, You Look Only Once (YOLO) version 2 , which iThe LR detector is trained to detect different types of insulators from long shot low-resolution images taken by the LR camera. The detector divides the entire image into fixed sized B × B regions to detect objects. We clustered the ground truth bounding box sizes into 5 groups and set the size of B = 7 based on the average size of the bounding boxes in the major clusters. Different types of COS insulators  appear very similar in the long shot images of low-resolution, as different types of LP, LA and add-on insulators. Moreover, the class information is ignored by the next step in the pipeline In order to find the best rotation angle, the edge map of the detected insulator in the high-resolution image is computed using the Canny edge detector. The edge map is rotated to all possible angles between 1 to 180 degrees.(2)Low-level visual features of all the edge maps are analyzed exhaustively; feature points are extracted and clustered by their appearance similarities.(3)The appearance similarity between any two feature points is computed by folding the edge map with respect to the center of the two feature points and then applying convolution.(4)High convolution score implies that the feature points in the two folds have high appearance similarity. Hence, we do max voting on convolution scores to find candidate point cluster which is consistent with the geometrical relationship of the insulator shape.(5)Finally, a tight bounding box encapsulating the maximum voted point cluster is returned, whose longest side represents the final orientation of the insulator.Rotation normalization of insulator images is an essential process in our proposed system. The novel rotation normalization algorithm is designed in such a way that it can estimate the orientation of the insulators regardless of their types. We exploited the appearance symmetry property of the insulators’ shape and the spatial context information to design a robust algorithm for estimating the orientation of the insulators. The algorithm is comprised of the following steps:The rotation normalized images are fed to HR_2 detector, which returns a tighter bounding box around the insulator body. The tightly cropped insulator image is then passed to the novel ellipse detector module for individual cap segmentation, which is covered in the next section.In order to find the defects on insulator caps, we need to analyze each and every cap separately, for which the caps of the insulator are segmented. In contrast with the color  based seith edge, Our ellipse detection algorithm starts with finding the edge map of the insulator image. We use the Canny Edge Detector with adaptive thresholding. In contrast with automatic thresholding used by ,42, we fIt is obvious that One prior information about all the types of insulators is that a core passes through the center of the caps. The core has horizontal edges in the edge map that are sometimes connected with the edges of the caps, as shown in ith edge point in the edge map, characterized by its position and edge gradient as x, y, and xy directions, respectively, then Next we apply edge refinement steps to obtain arcs of good quality for robust ellipse detection. Let kth arc represented by Next, we compute the convexities and directions of all the arcs according to , and labIt is clear from The tightly cropped insulator image returned by the HR_2 detector helps in the implementation of the second constraint which allows the selection of the arc triplet, present within a possible geometrical proximity. As shown in (i)(ii)(iii)(iv)Let The two constraints imposed by the algorithm not only enable better arc selection but also cut down tremendous processing time.Next, we fit an ellipse with direct least square fitting of ellipses method  using a Once the proposed algorithm selects arc triplet, we pick five equally spaced points from each of the three arcs. Let to refer  for moreE and the points ith point on nth arc triple, oth predicted ellipse, then we define a function If the set We empirically set Next, we combine multiple detections by computing the similarity between the ellipses, as:The final step again exploits the symmetric structure of insulators. As same sized caps are regularly spaced to form an insulator, we cluster the candidate ellipses into a number of groups based on their similarity intervals. The cluster with the largest number of candidate ellipses is selected as the final cluster, and candidate ellipses in the cluster are considered as the detected ellipses. Again, the methods in ,42 do noIn Defects in powerline insulators can occur due to inferior design, use of low quality materials in manufacturing, improper manufacturing processes, misapplication of the insulator or extreme stresses from weather , vandalism, wildlife, extreme electrical activity or mishandling. These defects can alter the appearances of insulators, such, as color, shape and texture. The proposed defect analyzer provides quantitative measures of defects with insulators. Since we use optical images for the defect analysis, we only address the defects that alter the appearance of the insulators. To be more specific, those defects occurring internally without changing the outer appearance are out of the scope of this paper.(1)Ellipse detection to segment caps from insulators.(2)Masking to remove the extra background.(3)Edge detection followed by some morphological operations to remove noisy edges.(4)Labeling all the edges inside the cap.(5)Connected component analysis for blob detection.(6)I points on the curve, then The bullet shots normally create circular shaped holes on insulator caps. Hence, in order to differentiate between noisy edges and edges caused by gunshots, we compute the circularity  of every(7)A priori information related to the color intensity of the hole created by bullet shot puts a final constraint in classifying the detected contour as bullet shot or not. We observed that in every insulator image with bullet shot damage, the color intensity of the area inside the hole of bullet shot is lower than the surroundings.(8)Let (9)G contours classified as gunshot damage, then the final percentage of damage per cap due to the gunshot is computed as:ith gunshot contour.If there are total The defect analyzer performs the following steps to detect gunshot damage:In this section, we present experimental results of the proposed insulator detection and defect analyzer system.There is no publicly available insulator image dataset and only few studies presented quantitative evaluation result of the system performance using self-acquired small databases. In our work, we gathered a large, unconstrained dataset of insulator images and presented our evaluation results based on that. We believe that the dataset is unbiased, unconstrained and sufficiently large enough to validate the reliability and effectiveness of the proposed system. We acquired a large dataset of long-shot low resolution and close-shot high-resolution images of equipment. Among them, we manually annotated 667 low resolution and 5533 high-resolution images . Detailed numbers of each insulator types used for training and test are provided in We used the well-known Pascal Score  to evaluAccording to the criteria in , an objeFor long-shot low-resolution images, our objective is to detect as many insulators as possible. We keep the detection threshold lower (= 0.10) to allow some false positive detections because the false positives are removed at later stages by the high-resolution detectors. Moreover, even if the detection returns a wrong classification label, the detected insulator has a high chance to be correctly classified by the high-resolution classifiers. Since the high-resolution classifiers are trained with high-resolution images, they learn a better shape model as compared with low-resolution classifiers. We found that most of the false positives detected by LR detector are corrected by HR_1 detector, and only 0.001% of the false positives detected by LR detector remain after HR_2 detector. We used 70% of images of each type of equipment for the training and the rest for the test.Moreover, we found that the n images . ConsideThe purpose of rotation invariant HR_1 detector is to detect insulators in the close-shot high-resolution images taken by the camera module. Similar to LR detector, we keep the detection threshold low (= 0.30) to allow more detections. As our rotation normalization method utilizes symmetry property of the shape of insulators, the false positives that contain asymmetric shapes, such as trees, pole edges, wires, insulator covers, etc. can also be suppressed in the rotation normalization step. We verified that 95% of the false positives detected by HR_1 detector are suppressed by our rotation normalization step, while out of the all false positives detected by HR_1 detector, none is passed to the defect analyzer.The final CNN-based detector is responsible to detect insulators in rotation normalized images. The performance of HR_2 detector is summarized in In order to find the accuracy of the rotation normalizer, we used the ground truth orientation information saved during the annotation process of close-short high-resolution images. We allow a tolerance of Unavailability of publicly available dataset restricts the comparison of our results with those of state-of-the-art methods. Among the published studies, only ,30 preseAs mentioned in The PR curve shown in We present the performance of the defect analyzer in The two thresholds also prove to be a strong geometrical constraint in the detection of true bullet holes and significant rejection the false positives (noisy edges), because, usually the bullet shots leave a circular hole in caps, and the size of that hole is relatively small as compared with the size of the cap. If bullet shots break/tear off the major portion of the cap, then we consider this case as a separate defect, called broken shed/cap, which is not covered in this manuscript. As shown in Even though, the proposed CNN-based electrical equipment inspection system present state-of-the-art detection performances, it contains room for improvement. Hence in the following subsections, we discuss the presented results along with the challenges and limitations faced by the proposed system.The CNN-based detection frameworks usually  require a large amount of training data to properly learn the convolution filters. We observed an overall performance improvement of 3% in precision and 0.7% in recall when using 50% of the augmented data   in our eThe potential cause of the less precision and recall values for the ceramic insulator, LP, COS, LA and add-on in As the CNN-framework used in this paper does not contain a region proposal network , during the annotation process, we marked such occluded equipment as “ignore” (not used as positive or negative samples during the training process). But the trained CNN network is highly robust against occlusion, and it still detects occluded objects with high confidence. Although, these detections are valid , our testing routine consider these detections as false positives, and hence reduce the overall reported performance of the system. As discussed earlier, our ellipse detector algorithm requires that the rotation of the input insulator image be normalized, and the insulator body tightly cropped, due to which we use a rotation normalizer and HR_2 detector. Hence, in the proposed system, there is a trade-off between the accurate ellipse detection and the computational resources used by the additional rotation normalizer and the HR_2 detector.Furthermore, even though the proposed ellipse detector is robust against noisy edges, still the ellipse detector algorithm relies on the quality of edges. As the ground vehicle taking the videos of the electrical equipment is continuously moving, there are cases where the detected electrical equipment is blurred out or occluded in such a way that their important edges are not retrieved. In such cases, the ellipse detector struggles to detect the caps, as shown with some example cases in Images of caps damaged by gunshots or any other damage are scarce and due to this reason, our testing set is highly imbalanced (95% of the segmented caps have no gunshot defect). Consequently, the precision drops more drastically as we try to improve the recall. A 100% precision and 75% recall shown by the gunshot defect analyzer means that the system is able to reject 100% false positives in 523 healthy caps, at the cost of rejecting only six faulty caps as false negatives, which is one false negative per 87 images (still a quite good ratio).In this paper, a novel automatic electrical equipment detect and inspection system has been proposed. The proposed system can detect 17 different types of electric power line equipment from videos taken by a ground vehicle and analyze one type of defects in polymer insulators. Compared to previous methods, the proposed system is superior in many aspects: (a) powerful CNN-based insulator detection versus handcrafted features, (b) capability to detect 17 types of insulators, (c) use of close-shot high-resolution images for defect analysis, enabling more precise defect analysis of insulators, (d) evaluation of detection performance on a large diverse dataset of long-shot (204 test samples) and close-shot images (807 test samples), which proves the robustness of the proposed system against occlusion, lighting conditions, in-and-out of plane rotations, viewpoint changes, color, shape, texture and cluttered environment.The proposed insulator rotation normalization and ellipse detection methods enable the system to recognize cap level defects. In order to support real-time operations, the proposed system is accelerated by a GTX1080 GPU  and uses OpenCV, CUDA and cuDNN libraries along with Darknet’s CNN framework. The proposed system is tested on a large, unbiased, unconstrained evaluation dataset of insulator images and state-of-the-art results are presented. Comparisons with previous studies verified the superior performance of the proposed system.The system pipeline presented in this work provides a natural guide to future research, which includes pushing existing CNN models to learn different types of insulators at deeper levels and formalizing for detection of additional types of insulator defects. The future research will also consider studying a specific and unified CNN architecture for insulator detection and insulator defect analysis.


In the wild, wireless multimedia sensor network (WMSN) communication has limited bandwidth and the transmission of wildlife monitoring images always suffers signal interference, which is time-consuming, or sometimes even causes failure. Generally, only part of each wildlife image is valuable, therefore, if we could transmit the images according to the importance of the content, the above issues can be avoided. Inspired by the progressive transmission strategy, we propose a hierarchical coding progressive transmission method in this paper, which can transmit the saliency object region  and its background with different coding strategies and priorities. Specifically, we firstly construct a convolution neural network via the MobileNet model for the detection of the saliency object region and obtaining the mask on wildlife. Then, according to the importance of wavelet coefficients, set partitioned in hierarchical tree (SPIHT) lossless coding is utilized to transmit the saliency image which ensures the transmission accuracy of the wildlife region. After that, the background region left over is transmitted via the Embedded Zerotree Wavelets (EZW) lossy coding strategy, to improve the transmission efficiency. To verify the efficiency of our algorithm, a demonstration of the transmission of field-captured wildlife images is presented. Further, comparison of results with existing EZW and discrete cosine transform (DCT) algorithms shows that the proposed algorithm improves the peak signal to noise ratio (PSNR) and structural similarity index (SSIM) by 21.11%, 14.72% and 9.47%, 6.25%, respectively. Wildlife monitoring is crucial for the balance and stability of the whole ecosystem ,2. ImageDue to the limitations of WMSNs with low processing capability, power consumption restrictions and narrow transmission bandwidth, the wildlife monitoring images collected encompass high resolution and large information data characteristics, which poses a challenge for the WMSN transmission process . For traTo solve the problem of unsuccessful transmission effectively, the image progressive transmission method ,13,14 isSaliency object detection can provide the mask of wildlife for the progressive transmission process, which is utilized to separate the object and background information. Conventional saliency detection algorithms, such as the human-computer interaction , visual This paper proposed a progressive transmission method for wildlife images based on saliency object detection. Firstly, regions containing saliency objects (wildlife) are detected from the original source by convolution neural networks, and then the mask of the saliency object region was generated. After the generation of the mask, set partitioned in hierarchical tree (SPIHT)  losslessWMSN is widely used in wildlife monitoring systems to capture wildlife image materials with industrial grade cameras, which consist of WMSN terminal nodes, coordination nodes, gateway nodes and a data storage center (Back-end Sever). The wildlife monitoring system is acknowledged to present remote, real-time, all-weather and friendly monitoring merits, and the schematic diagram with detailed configurations are shown in Cervus Elaphus, Lynx, Capreolus pygargus, Sus Scrofa and Naemorhedus goral, and so on. Cervus Elaphus and Lynx are secondary national-protected species Saliency region extraction based on convolution neural networks (CNN) , which a(2)The maximum displacement method is applied to ensure the saliency region, in another words, the wildlife region is placed in the highest priority of compression transmission based on SPIHT coding.(3)To guarantee the transmission efficiency, the EZW coding algorithm is utilized to transmit the background region when the image information of the saliency region is received.A novel hierarchical coding progressive transmission method is proposed to process the wildlife monitoring images with high resolution and large information data, as shown in Redundant background regions may cost undesired transmission bandwidth consumption, as background regions always account for a large percentage of the area of wildlife monitoring images. Therefore, extracting the wildlife region out of the background is the most significant procedure in our experiment. However, the high resolution and complex content in wildlife images aggravate the difficulty in completing saliency object detection and the mask image generation.In our algorithm, the CNN is applied in the saliency object detection step, as it has been validated to detect the most visually distinctive object regions in the monitoring image. We selected 700 images as the training set, 100 images are used as the validation set and 200 images as the testing set, all of which are with 256 × 256 pixel size. The saliency object region is marked with pixels for model training, and the gray value 0/1 represents the background region/saliency object region.Novel network architecture based on the MobileNet model  is utiliTo obtain more detail and texture feature, skip layer short connections  which fuIn the network architecture, the side loss function and fusion loss function are calculated asied from . σ Thus, we can get the final loss functionHyper-parameters used in this section are learning rate (le-7), weight decay (0.0003), momentum (0.9) and loss weight for each side output (1). Our fusion layer weights are initialized at 0.2, and the group size is set to 32.Regarding saliency object detection and extraction, the corresponding coefficient in the saliency and background regions are set to 1 and 0 respectively to obtain the binary mask image, as shown in After generating the mask of the wildlife region, we should determine which wavelet coefficients belong to the saliency region or the background region in the different levels of wavelet decomposition. In this paper, the mallat wavelet decomposition algorithm is utilized to obtain the wavelet coefficients, which are calculated byTo ensure the integrity of the reconstructed image edge, the edge detection based on Canny operator is utilized in this paper, which aims to measure the convolution of the gaussian smoothing filter Then the partial derivative is obtained by calculating the first-order finite difference of the filter result.Among them, Finally, non-maximum suppression is completed through seeking amplitude maximum of the matrix along the gradient direction. The pixels with maximum amplitude are considered as the edge pixel. To make the image edge close, this paper selects double appropriate threshold (high threshold and low threshold). As a consequence, the non-edge points that do not satisfy the threshold condition are removed. Then the connected domain is expanded to get the final edge detection result.In the process of wavelet decomposition, the mask image is updated simultaneously shown in For the preferential transmission of the important region, the maximum displacement transmission is applied in this section. The researchers are most interested in the wildlife target while the background region is a supplement for wildlife in the monitoring images. In this paper, we think highly of all wavelet coefficients in the saliency object region over the wavelet coefficients in the background regions. The background information is transmitted after the saliency object region information is completed, which helps to identify the wildlife species in the initial stage of transmission. The maximum displacement transmission strategy firstly calculated the bit plane layer When the value of the bit plane layer is determined, the wavelet coefficients in the saliency region are multiplied by Wildlife protection and wildlife scientific research, such as wildlife species identification and wildlife individual identification, both require high-resolution and high quality wildlife monitoring images, especially the wildlife itself. Therefore, after the bit plane layer of the saliency region is lifted, we utilized the SPIHT algorithm to achieve lossless compression transmission for saliency region coefficients, which guarantees the transmission quality of important region.The spatial direction tree structure expressed in Where T is the current threshold and the initial threshold is selected as The function Considering the importance of wavelet coefficients according to In the wavelet coefficient scanning process, three lists are utilized to store the relevant information of the wavelet coefficients according to their importance: LIP (less significant pixels), LSP (significant pixels) and LIS (less significant sets).(1)If the coordinate in (2)All elements in (3)If there are important elements in (4)The symbol (5)Step (5) is performed periodically for each newly generated spatial direction tree until all the important elements are stored in the LSP.The SPIHT rules set in this section include:Finally, the wavelet coefficients are refined, coding by successive approximation quantization. The wavelet coefficients are continuously coded according to the update threshold To improve the transmission efficiency, the lossy compression EZW algorithm is implemented to encode background information, which has some similarities with the SPIHT coding algorithm.In this section, the zero tree structure, whose definition is that the root nodes and all offspring nodes are unimportant coefficients, we can use an encoding symbol to represent all coefficients in a zero tree, thus greatly improving coding efficiency. In the EZW encoding process, the wavelet coefficients are divided into four symbols by comparing with the threshold T, as shown in When the wavelet coefficient dividing process is finished, the wavelet coefficient scanning and refined coding is completed with the SPIHT coding algorithm in the last section until all coefficients in the background region are coded.To verify the adaptability and effectiveness of the proposed algorithm, transmission analysis of field-captured wildlife monitoring images is presented. The result is evaluated by several evaluation criteria and it is compared to other conventional algorithms for image transmission.Both peak signal to noise ratio (PSNR) and structural similarity index (SSIM)  are utilMN is the total number of pixels in the sample image. PSNR is the ratio of the signal maximum possible power to the destructive noise power based on the mean square error (MSE) , which aThe SSIM is another measure of the similarity between reconstructed and original images. It achieves this by calculating the image distortion degree according to the change of image structure information.We applied our algorithm to the filed-capture wildlife monitoring images with high resolution, high noise interference and complex background selected from own image dataset . ExperimAs a proposed method, the saliency region is firstly transmitted through a lossless approach and then the background region in a lossy way , which aIn the process of image transmission, the background region adopts lossy compression transmission to improve the efficiency of image transmission. We selected the PSNR and SSIM of the full reconstructed image to evaluate the proposed algorithm in this paper. The experimental results are shown in According to above transmission result in To verify the transmission effect, the EZW and DCT algorithms are compared with our proposed algorithm in this section. As shown in The average running time comparison result is shown in In this paper, we proposed a novel hierarchical coding progressive transmission method for wildlife images, which can achieve progressive transmission of saliency region and background separately. We firstly utilize a convolutional neural network based on MobileNet model to detect the saliency object region for a wildlife image with a complex background. When we obtain the mask image of the wildlife region, a progressive transmission strategy using the maximum displacement method aims to transfer the wavelet coefficients of the saliency region preferentially, and then lossless coding transmission is performed on the saliency region and lossy coding performed on the background region. To demonstrate the efficiency and validation of the proposed method, the images from the field-captured wild monitoring database are processed. Comparison results show that the proposed algorithm has better performance than existing classical algorithms, that is, EZW and DCT. Specifically, the average PSNR and SSIM are increased, respectively, by 21.11%, 14.72% and 9.47%, 6.25%.