The paper covers one of the communication technologies used in wireless sensor networks. We have presented improvements in existing radio frequency identification (RFID) systems to address the problem of the phase selection in active load modulation (ALM). The phase selection affects the interoperability of communication devices and has to be addressed in the design phase of a new tag. A novel transmission method is presented to make the phase selection irrelevant for device interoperability. A second solution is shown to improve the existing system synchronization, which allows operation with arbitrary selected phase. A mathematical analysis of signals present on the antenna was used together with the reference reader model to perform an analysis of proposed improvements. We proved that the proposed transmission method is less affected by phase selection. Furthermore, we demonstrated that existing system improvement allows synchronization and operation at an arbitrarily selected phase despite the continuous transmission and large signal-to-interference ratio. Usage of wireless sensor networks (WSNs) has increased over the last decade, with multiple implementation options. WSNs differ in communication distance, network topology, device complexity and several other parameters. Each topology and technology used has their own advantages and disadvantages, which need to be taken into consideration when designing a WSN for a specific application. One of the studied communication technologies for WSNs is radio frequency identification (RFID) in the high-frequency (HF) band ,2. One oA communication pair of devices is composed of a reader and a tag in which the reader generates a magnetic field, which is modulated as a means of communication with the tag. The tag can use the reader’s magnetic field for its own supply and modulates the magnetic field to respond to the reader’s requests according to standard . The impTo fulfill the requirement that the reader recognizes the tag’s magnetic field as modulation of its own magnetic field, the tag needs to guarantee the phase and frequency alignment between both magnetic fields. The core of the ALM system is a circuit for phase and frequency synchronization. In its use today, the circuit is a phase-locked loop (PLL) where the loop is locked and is tracking the reader’s magnetic field phase when the tag is not transmitting. During transmission, the PLL phase tracking is stopped, and oscillator control is placed on hold. Depending on the oscillator stability, the loop needs to relock before the PLL phase drifts too far away from the reader’s magnetic field. The synchronization performances and system architecture are determined by the stability of the PLL frequency and PLL phase in hold mode, and the phase noise of the oscillator. The goal is to minimize the phase drift between the reader’s magnetic field and the tag’s transmitted magnetic field. For this purpose, a method has been proposed to measure the phase drift of a tag using ALM .Recently, the majority of research in HF RFID and NFC has been focused on the power transfer, antenna modeling and use cases of the technology ,10. In tAuthors in work  presenteA circuit presented in  uses an A different approach is proposed in , where tThis paper addresses the issue of phase selection of the system presented in . First, This paper is organized as follows. In A tag performs active load modulation by transmission of the magnetic field, which induces voltage on the reader antenna. Induced voltage from ALM is added to existing driving voltage present in the reader antenna. To achieve only amplitude modulation (AM) of voltage on the reader antenna, the appropriate phase difference between the reader driving voltage and the tag driving voltage needs to be set. The exact phase difference to achieve AM-only modulation between the two signals depends on the sum of multiple phase shifts in the system, such as phase shift in the power amplifier, reader antenna tuning and EMC filter (electromagnetic compatibility). Antenna detuning occurs when a resonant circuit, such as a passive tag, is placed in close proximity of the antenna or when a conductive material is placed in close proximity of the antenna. Inside a conductive material eddy, currents occur, which induce their own magnetic field and reduce the inductance of the coil composing the antenna. To accommodate for the phase shift, a fine-tuning of the phase shift is performed in the design phase of a tag to guarantee interoperability with targeted readers. The exact phase shift depends on the objective of the design. Usually, the target to achieve good interoperability is to have ALM signal on the reader side seen as part AM and part phase modulation (PM). This may not be sufficient, as the environment changes dynamically and introduces an additional phase shift by detuning of the reader antenna. Therefore, the signal can become AM only or PM only. This can present a problem for certain readers and can influence the communication quality. Several readers have, for historical reasons, implemented only AM receive channels or performed selection of only one channel for the duration of transmission frame.  The firsSubcarrier modulation of a carrier is shown on the I-Q diagram given in The solution to avoid having only I or only Q channel modulation on the reader side is to use two intermediate modulated states MS1’ and MS2’, which are orthogonal to existing states MS1 and MS2. The I-Q diagram with additional modulation states is shown in Certain protocols define binary phase shift-keying (BPSK) modulation of a subcarrier. In these cases, MS1 or MS2 may be followed by the same modulation state, resulting in modulation, which is only present on I or Q channel during the repeated modulated state. To solve this problem, transmission of an intermediate modulation state is added between two repeated modulated states. This is visualized in Modulation in time domain is given in A, CA, RA) for use in simulation. The capacitors C2 and C4 are specified as variable capacitors used to tune the antenna. A test method is described in the NFC specification to use a network analyzer and s-parameter measurement to trim the variable capacitor. Cadence SpectreRF sp analysis, which calculates the s parameters, was used. Values of obtained circuit elements are shown in For the reader antenna model, we have used the NFC Forum specified Poller 0 antenna. The reader antenna schematic and model are shown in To model a realistic reader I and Q channels, we used two mixers with 90° shifted clocks and analog filters. The filters are a series connection of a second-order low-pass filter with cut-off frequency at 3 MHz and a first-order high-pass filter with cut-off frequency at 50 kHz. Together, the filters form a band-pass filter and are needed to remove unwanted products of the mixing.The tag antenna model is shown in The second solution to the described problem of phase selection is to improve the system presented in  to allow1 and represents the differential voltage at nodes vtxp and vtxn of circuit in ANT, and we can write a transmitted signal on the antenna as T’1 = T1 |HANT|eγj, where γ is the phase shift of the antenna transfer function. The reader signal R on the antenna is added to the transformed transmitted tag signal T’1 and together they form the antenna voltage A1, which is the differential voltage at the nodes vantp and vantn of circuit in The settled voltage on the antenna at the end of the subcarrier half-period can be presented by phasors as shown in γ between T1 and T’1 is the argument of antenna transfer function HANT and is composed of the phase delay of antenna resonance, antenna matching circuit and EMC filter. The transmitted signal T’1 is larger than the received reader signal R, and by taking this into consideration, we can write the phase difference α’ between T’1 and A1:The phase difference α’) with α’. While the phase φ is not constrained, the phase α’ has a small value regardless of phase φ if the reader amplitude |R| is smaller than the tag’s antenna voltage |T’1|.Equation (1) can be further simplified by omitting the cosine function in the denominator and approximating tan:In the next modulated state MS2, the transmitted signal Tβ is:The measurable phase α or β is obscured by the antenna phase shift γ. The movement of the tag or self-heating can change the electrical characteristic of the tag antenna and can introduce a change in the phase shift γ. The time difference between the measurement of phase α and β is one-half period of the subcarrier and is considerably shorter compared to the dynamics of self-heating or the tag movement. Therefore, the change of antenna transfer function HANT between two consecutive measurements is negligible. The transmitted signals amplitudes |T1| and |T2| can also be considered constant and the resulting transmit power can also be considered constant during this short time difference. We can replace |T’1| and |T’2| with |T’|, which is the transmitted signal amplitude increased by gain |HANT|. By calculating the phase difference α − β we remove phase shift γ and double the sensitivity:Measurement of single phase The amplitude of antenna signal at the end of modulated state MS1 and MS2 can be measured using a similar approach:2 sin2(φ), and by assuming that the first term is always positive, we can square root both sides of the equations. If we use the approximations and calculate the difference between (7) and (8) we obtain:As the transmitted signal T’ is considerably larger than the reader signal R, we can neglect the second term |R|1| and |A2|, we can retrieve the amplitude of |T’| and use it for the division of Equation (9) to obtain the same multiplier as in Equation (6). The results of the above operation are two measurements; one is the sine function of the phase difference φ and other is the cosine function of the same phase difference. At this point, an arbitrary method can be used to determine the phase difference. In case of an analog solution, this can be an angle interpolator, while in the form of a digital solution, a CORDIC  or any other algorithm to retrieve the phase difference can be used. In comparison to the phase measurement, the multiplier of the cosine function is not the same as the multiplier before sine function in Equation (6). By averaging measurements of |AWe performed a computer simulation of the existing and proposed orthogonal phase transition shaping method with Cadence Spectre simulator. The NFC Forum type B protocol modulation was used for transmission with prescribed 13.56 MHz carrier frequency and 16 times smaller subcarrier frequency. The data was transmitted with 106 kbit/s, BPSK modulated subcarrier. To measure the effectiveness of different transmissions methods, the rms voltage was measured on I and Q channels of our reader model as a function of the tag transmission phase. Analysis for three different transmission methods were performed. The first was on-off keying of the carrier, according to the standard modulation scheme. This transmission mode was called AND mode and is shown in To validate the proposed transmission method, we performed an experiment with a commercially available reader (ST25R3916), where we observed I and Q channel outputs. For transmission signal generation, we used Keysight 33500B arbitrary waveform generator and tag antenna, according to the circuit presented in comp1 was placed on the differential output of the power amplifier and extracted the transmission signal represented by phasors T1 and T2 in comp2 was connected to the coil of the tag antenna between nets vantp and vantn. The extracted clock is represented by phasors A1 and A2 in α and the second to measure phase β. The difference calculation of the two integrated voltages was implemented with an analog difference amplifier. The output of the difference amplifier corresponded to the result of Equation (6). To perform the measurement of the phase between the reader and the tag based on the method presented in Amplitude measurement was implemented with the sampling circuit shown in To verify the circuit performance, we designed the circuit in 90 nm CMOS technology . The simulations were performed with Cadence Spectre simulator. In simulation, we introduced a small frequency displacement between reader and tag clocks to obtain a linear phase drift. The analysis outputs are the phase and amplitude measurements shown in The phase detector performance was verified for different reader-to-tag signal ratios with a second analysis, performed over a full input phase range. The results are shown in The proposed OPTS transmission modulation has shown promising performance gain on reader receiver side across a full range phase shift compared to existing AND and XOR transmission modulations. It was shown that the method guarantees signal on the reader I and Q channel for all phase differences between the tag and the reader. The benefit of the OPTS modulation is improved interoperability and communication reliability. This advantage is important for the cases where the environment causes detuning of antennas, which results in the phase shift of tag signal on reader side, especially on the existing installed equipment that is not operating according to latest standards. The OPTS modulation can also be used on the tag with a clock synchronization system that does not have a circuit for fine phase adjustment or if it cannot operate at arbitrary phase shift. The presented transmission mode can simplify the design of a new active tag antenna, as the designers can omit the fine phase tuning design step. The drawback of this use case is a somewhat smaller ALM signal seen on the reader side. Our second presented solution, the improved phase detector, is capable of operation during continuous transmission. The dynamic range of the ratio between the received reader and the transmitted tag signal was shown to be between −40 and −18 dB. In this signal ratio, the range phase detector had limited phase error at the output over a full range of input phases. This phase detector operation can support a clock synchronization system to maintain a lock on reader signal regardless of phase shift between reader and tag. The ability to select the transmission phase enables the operation with XOR transmission modulation, where the tag transmission phase has to be carefully selected to guarantee the interoperability with existing equipment. The benefit of using XOR modulation is increased signal amplitude observed on reader side and increased communication range. The drawback is an additional design step in the development of a tag to guarantee the interoperability. By implementing both presented solutions, the end user designing a complete tag system with ALM has the option to decide between a simpler solution or a more complex one to increase communication range. For future work, we foresee further development of proposed techniques and implementation on an integrated circuit.An overview of existing ALM solutions was given in the paper, focusing on signal synchronization between the received reader signal and the transmitted tag signal. The emphasis was placed on the limitation of the existing solution, which is able to synchronize during the continuous transmission without pauses for synchronization. The limitation of the existing system was identified as the phase detector, which has a limited input phase range. We have presented two solutions. One solution is the improved phase detector circuit for which we showed operation over a full input phase range. The phase detector has shown an acceptable phase error for the ratios between the received reader and the transmitted tag signals in the range from −40 to −18 dB. The second solution is a new transmission modulation with orthogonal phase transmission shaping (OPTS) to improve the signal reception on the reader side. We proved that the new modulation produces signal on I and Q channel inside the reader receiver, regardless of the signal phase between the reader and the tag. This is an improvement over existing solutions for which we showed that both exhibit a phase setting where the signal inside the reader receiver is present in only one, I or Q channel. This improvement is important for high interoperability with existing equipment.


We present a protocol for the design, fabrication and characterisation of laser-induced ultrasound transmitters with a specific, user-defined frequency response for the purpose of ultrasound tomography of large-volume biomedical samples. Using an analytic solution to the photoacoustic equation and measurements of the optical and acoustic properties of the materials used in the transmitters, we arrive at a required mixture of carbon black and polydimethylsiloxane to achieve the desired frequency response. After an in-depth explanation of the fabrication and characterisation approaches we show the performance of the fabricated transmitter, which has a centre frequency of 0.9 MHz, 200% bandwidth and 45.8 In one such example an optical fibre was fitted with a selectively absorbing film for LIUS excitation and an integrated optical sensor based on a ring resonator, to allow miniaturised PA/LIUS sensing and imaging Most of the applications of LIUS in biomedical imaging from the literature share the characteristic that they tend to focus on imaging relatively small targets like zebrafish and mice, or intravascular imaging. In these cases it is advantageous to use LIUS sources with high optical absorption, to both maximise the generated peak pressure and bandwidth of the generated acoustic pulses to improve spatial resolution. Recent work by Joo et al. The current work presents a detailed look at the design, fabrication and characterisation of unfocused LIUS transmitters for large-volume ultrasound CT of, for example, the human breast. The use of LIUS can be beneficial for the prevention of crosstalk, because the transmitters are intended to be used with a high density of highly sensitive PZT detectors. Further, this decoupling of transmission and detection allows for the independent optimisation of both. Since it is important, in this case, to detect the LIUS signal with as many detectors as possible per shot, a large opening angle in addition to a lower centre frequency for transmission through around 10 cm of tissue are desirable features. We present the design considerations in terms of geometry and dimensions and an analytic approach to estimating the output characteristics of such a transmitter. A protocol for fabricating such a transmitter will be presented in detail. We also show the methods used to optically and acoustically characterise the materials in the active element and the response of the finalised transmitters. While the transmitters presented here are designed for a specific imaging application and have the properties to match, the design and fabrication approach can be easily adapted to attain any desired set of transmitter properties using low-cost, readily available materials.22.1The transmitters are designed to match with a reasonable precision to the response and sensitivity of a custom detector . The detector frequency response peaks at 1 MHz with a bandwidth of roughly 100%. The remaining pressure after transmission across the imaging volume, a distance of some 30 cm, needs to be above the minimum detectable pressure, around 0.5 Pa. An opening angle of at least 30For this particular transmitter design the physical dimensions are restricted by the need to use them with an existing set up 2.22.3The properties of the generated acoustic pulse can be influenced, within our design, by the optical absorption coefficient (2)The first term is due to the direct pulse from the initial pressure distribution while the second term denotes its reflection off the back surface of the absorber, with The pressure detected at a given point as a function of time can be described as follows, by solving the photoacoustic equation with the Green’s function approach for an optical delta pulse (7)The two ranges of In the following we assume the laser spot size to be significantly smaller than the diameter of the absorber, which is thus treated as though it extends infinitely in the lateral dimension (11)Where the following constants and variables are defined, for compactness’ sake: Evaluating Eqs. 2.42.4.1The base material for the LIUS transmitters is polydimethylsiloxane , which is mixed with an appropriate amount of carbon black  for optical absorption, 45 g of PDMS with 3.63 g of CB for a weight percentage of 8.07%. The transmitter casings are made of machined polymethylmethacrylate (PMMA), and the backing consists of a disc-shaped borosilicate glass microscopy cover slip with a 5 mm diameter and 2.4.2Next, in a separate container or dish, a desired amount of stock mixture is weighed, denoted After adding the required weight of clear PDMS a disposable plastic pipette is used to add curing agent of the desired 10% of the total PDMS weight, after which it must be well mixed with a small spatula until it appears homogeneous, as the volume is too small to use a mechanical mixer. To facilitate the production of multiple transmitters the mixture is cured in a slab by transferring it to a flat-bottomed rectangular container, degassing in a vacuum vessel at −1 bar and regulating the thickness with a PMMA plate on a translation stage. Any excess liquid will flow past the edge of the plate and is cut off after curing. The mould is then placed in an oven at 70 °C for 90 min to cure.Once cured and cooled, 10 mm diameter discs are cut from the slab with a mechanical hole punch and a hammer. A small drop of cyanoacrylate adhesive is deposited on one face of a cut disc and smeared out into a thin layer with a needle tip. A glass backing disc is gently but firmly pressed onto the glue for tight adhesion to the disc. The tube segment and fibre mount need to be bonded coaxially, for which purpose a cylindrical alignment piece was machined, shown in image III on the right-hand side of 2.5Before fabrication of the transmitters could begin, the dependence of the optical properties of the CB/PDMS mixture in the concentration of CB had to be determined. Once known, the acoustic properties of a sample with the required optical properties also had to be characterised. This section will describe the various characterisation approaches taken.2.5.1d is the particle diameter and −20 mm−1 for the CB concentrations in the range of interest, many orders of magnitude below The required weight concentration of CB is determined by measuring the optical transmission spectrum for a range of samples of PDMS with CB contents between 0.03% and 0.28%, beyond which the transmitted light is insufficient for an accurate measurement. Any higher concentrations could not be measured by the spectrophotometer . The samples are prepared by mixing a range of concentrations from the previously prepared stock solution and placing a drop inside a circular, −1] is the speed of sound in water.Once the proper CB-percentage for the desired photoacoustic response is known, a precursor slab for a desired number of transmitters is fabricated according to the description in Section While the acoustic attenuation between roughly 0 and 4 MHz can be determined from transmission measurements on this block, it was deemed preferable to measure it over a broader frequency range. For this purpose a pair of disc-shaped samples of 2.5 cm diameter and 3.97 and 5.97 mm thicknesses was also cast. In this manner the sound speed can be measured with greater accuracy due to the smaller relative uncertainty in the measured sample thickness, while the attenuation will be known over a broader frequency range. The acoustic attenuation in dB per cm as a function of frequency can be determined from b are known they can then be used to estimate the impact of acoustic attenuation on a transmitter of a given thickness.Once the values of 2.5.2A fibre-optic needle hydrophone system  is used to characterise the response of the transmitter under illumination by a 3.6 mm diameter (1/e) Gaussian spot from a compact diode-pumped pulsed Nd:YAG laser  emitting 4.7 ns pulses at a 100 Hz repetition rate at a 1064 nm wavelength. The light from the laser is coupled into a multimode optical fibre , in a fibre multiplexer described in more detail in previous work 33.1−1 and 10 mm−1 at a depth of 300 mm with an optical pulse energy of 1 mJ and −2 gives the signals and spectra shown the insets in −1, with a bandwidth from 0 to 5.5 MHz and peak-to-peak pressure of 49.3 Pa at 300 mm in water. The signal and frequency spectrum for −1 are shown in −1.Evaluating Eqs. 3.23.2.1−1 requires a CB concentration of 0.075% by weight. Calculating the expected value of the scattering coefficient −1, 20 orders of magnitude below 3.2.23.3−2, at a distance of 6.2 mm from the transmitter surface. An overview is shown in −2), exhibiting oscillations in the rising flank of the negative peak currently suspected to indicate detachment of the backing from the absorbing layer. There appear to be no oscillations visible in the pre-damage curve in The generated peak-to-peak pressure was measured using illumination pulse energies between 809 −2 should allow for damage-free operation.The detachment may well have worsened at higher pulse energies, partially explaining the decrease in generated pressure. At higher pulse energies the amplitude of the internal-reflection peak at −2. For further measurements, using an undamaged transmitter, a liberal safety margin to prevent damage was maintained. The transmitter was illuminated with pulse energies of 950 −1 e−1 points at each depth, from which the opening angle of 45.83.4The LIUS transmitters presented in this work are still very much prototypes. Were they to be made for wider distribution and application some further improvements would be advised. The glass backing is quite fragile in the current design, meaning even fairly light mechanical pressure applied to the front face of the transmitter can lead it to break or detach. This immediately deteriorates the signal quality, if not necessarily the pressure, in unpredictable ways. One way to make it more robust would be to have the backing not simply attached to the PDMS, but integrated with the casing. This could be done by using a wider glass disc or machining the PMMA casing to incorporate a thin layer of material on top of the air pocket. To lower the chances of the backing detaching a different bonding mechanism may also be considered. In microfluidics highly robust bonding of PDMS is regularly performed to both glass, through oxygen plasma treatment While PMMA is a low-cost material that is also easy to machine, a further simplification of the fabrication process might be achieved by 3D-printing the casings instead. As long as the precision is high enough to allow for the placement of the optical fibre with variations of no more than As described above, the current method of fabrication includes a considerable potential for variability in transmitter properties, due to the hand-made nature of the transmitters. In applications using multiple transmitters in concert, it is therefore important to characterise each transmitter well, and to be sure that their properties do not change during an measurement. Even if there is an appreciable variation in transmitter pressure output, or even centre frequency, knowing exactly what these variations are will let them be taken into account. Ideally, in future improvements to the fabrication process any variability would be minimised by more standardised and automated fabrication steps, as well as by ensuring robust bonding methods that do not vary with time and are not susceptible to optical damage in the course of normal operation.The calibrated fibre-optic hydrophone used for the characterisation measurements was well-suited to determining the field geometry, the active sensor having a diameter of For integration into a hemispherical large-volume PAT/USCT system the positioning of the transmitters needs to be considered carefully. Depending on the requirements of the PAT illumination and the density of detecting elements, a minimum number of LIUS elements for sufficient-quality images should be arrived at. Considering the desire to insonify as much of the imaging target as possible with each LIUS shot, it is likely that a narrow ring nearer the rim of the bowl would be optimal. In this manner a large range of angular positions could be covered by perhaps a few tens of transmitters. With such a number of transmitters in mind, the illumination method should also be addressed. One method to efficiently illuminate such an array of LIUS transmitters sequentially could be through the use of a fibre-optic multiplexer based on a matrix of fibre inputs, selectively illuminated by a laser via galvanometer mirrors and a scanning lens, as we have presented in previous work A final consideration for the integration of the LIUS transmitter into a multimodal system with a PAT capability is the generation of parasitic signals. When illuminating the imaging volume for PAT measurements, light hitting the front face of the LIUS transmitters is absorbed and can generate unwanted signals orders of magnitude stronger than the actual PA signals. This is highly undesirable and has to be addressed by preventing optical absorption in the front face of the LIUS transmitters. One of the easiest ways to do this, with minimal impact on the LIUS signal quality, would be to place gold foil stickers on the front faces of the transmitters, which would reflect most of the light and drastically, if not completely, reduce parasitic signals. A similar approach has already been shown to work in preventing parasitic signals from optically absorbing ultrasound detectors in a photoacoustic mammoscope For future work, several options present themselves to further improve control over the output of the LIUS transmitters. For one, the addition of a broadband acoustic matching layer 4We have presented an approach to designing laser-induced ultrasound transmitters for large-volume ultrasound computed tomography. The transmitters produced according to our method behave in a satisfactory and reproducible manner, though some room for improvement in the process and result still remains. The LIUS transmitters produced here have a centre frequency of 0.94 MHz with a bandwidth from 0.17 to 2.05 MHz, producing pressures between 180.17 kPa and 24.35 kPa for a range of depths between 7.42 and 62.25 mm. The ultrasound field has a cylindrical symmetry about the acoustic axis and an opening angle of 45.8The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.


We express the task for finding changes as a network anomaly detection problem on temporal networks. We propose the use of a curvature measure based on the Forman–Ricci curvature, which expresses higher-order correlations among two connected nodes. Our theoretical result on comparing this Forman–Ricci curvature with another well-known notion of network curvature, namely the Ollivier–Ricci curvature, lends further justification to the assertions that these two notions of network curvatures are not well correlated and therefore one of these curvature measures cannot be used as an universal substitute for the other measure. Our experimental results indicate nine critical edges whose curvature differs dramatically in brains of adhd patients compared to healthy brains. The importance of these edges is supported by existing neuroscience evidence. We demonstrate that comparative analysis of curvature identifies changes that more traditional approaches, for example analysis of edge weights, would not be able to identify.We analyze networks of functional correlations between brain regions to identify changes in their structure caused by Attention Deficit Hyperactivity Disorder ( Such heterogeneous networks may vary in diversity from simple undirected networks to edge-labeled directed networks. One such class of network models are temporal networks3 (networks whose edges vary over time) where elementary components of the network (such as nodes or edges) are added and/or removed as the network evolves over time. Examples of such networks include biological signal transduction networks with node dynamics, biochemical reaction networks, infectious disease contact networks, and time-evolving correlation networks3. Typically, such networks may have a set of critical elementary components  whose presence or absence alters a significant global property of these networks between two time steps. Finding such a set of critical components in the context of temporal networks is more popularly called the anomaly detection or the change-point detection problem in statistics, computer science or data mining literature5, and prior widely used application areas of these problems include medical condition monitoring7, weather change detection9 and speech recognition11.It is by now a common research practice to study the properties of complex interconnected systems by representing them as heterogeneous networks and then using various network-theoretic tools for their analysisinterchangeably. We provide a formal definition of the network anomaly detection problem following a mathematical framework similar to what is described in12. To identify critical components of a temporal network, one first needs to provide details for the following four specific items: (i)model under consideration,the network (ii)elementary components of the network, anda definition of the (iii)changes over time,how the network (iv)property of the network that will be used to identify critical components.the  Let (i)’complete undirected graph n nodes V, a set of  edges non-negative real number t. We will use the notations t. Note that alternatively we could also view G(t) as a The temporal network model considered in this paper is a (ii)’The elementary components are the edges of the network, or equivalently following the convention as described above, they are every pair of nodes in the network.(iii)’same set of nodes. Note that a change of the weight of an edge from zero to a non-zero value is interpreted as the addition of a new edge, whereas a change of the weight of an edge from a non-zero value to zero is interpreted as the deletion of an existing edge.The network changes over time by modification of these elementary components, i.e., the network changes over time by changing the weights of edges while keeping the (iv)’G given by a suitable version of the Forman’s combinatorialization of Ricci curvature for networks (henceforth will be referred to simply as the “Forman–Ricci curvature”)17. For now, assume that the Forman–Ricci curvature for a complete graph t is a real-valued scalar function G at time t to a real number with the assumption that symmetric with respect to its first two arguments, andThe property of the network studied in this paper is the curvature of In this paper, we will use the two terms “graph” and “network” network anomaly detection problem can be defined as follows.Then, our u, v such that the value of Given two complete graphs Attention Deficit Hyperactivity Disorder (adhd). adhd is one of the most common neuro-developmental disorders of childhood impacting parts of the brain that help us plan, focus on, and execute tasks. adhd impacts approximately 11% of children and 5% of adults in the US alone. It is usually first diagnosed in childhood and often lasts into adulthood. Children with adhd may have trouble paying attention, exhibit controlling impulsive behaviors, or be overly active. Unfortunately, the causes and risk factors for adhd are still unknown, and as of yet there is no single clinical test that helps diagnose adhd before its onset. There are several published neuroimaging studies that link the behavioral symptoms of adhd to altered connections between brain regions. In the last few years, network analysis methods have been extensively used for studying properties of human brain networks21. The human brain can be divided into different regions based on functional or anatomical properties22. One can consider these regions as nodes of a brain network and define the edges as functional correlations among brain regions. Two prior graph theoretical studies of adhd24 reported changes at the global level of the entire brain but did not study any altered connection patterns between different regions in the brain. In this paper we use our curvature-based network anomaly detection algorithms to detect statistically significant altered connection patterns between different regions of the brain.In the above formulation of the network anomaly detection problem, we want 25 characterized the brain networks affected by Autism Spectrum Disorder (asd) using Ollivier–Ricci curvature. Individuals with asd have altered white matter developmental patterns compared to individuals without asd. White matter development can be altered by neuro-inflammation, which in turn is associated with abnormalities in cerebrospinal fluid circulation. Autologous cord blood infusion, a potential therapy, is believed to reduce neuro-inflammation and promote white matter development, thus triggering a reconfiguration of connectivity patterns in the brain. Simhal et al. 25 used the Ollivier–Ricci network curvature (based on the mass transportation distance) to quantify the changes in the brain network after administering asd patients a single infusion of autologous umbilical cord blood. They calculated the Spearman correlation between changes in clinical behavioral scores and changes in curvature following treatment, and identified a relationship between clinical improvement and altered curvature in three white matter pathways that are implicated in social and communication abilities. We, on the other hand, use the combinatorial Forman–Ricci network curvature to determine and quantify the local and global changes in the brain network due to the onset of adhd and highlight the target nodes and edges which undergo major alterations due to the disease. We believe that our work in localizing the changes in the brain network will aid biomedical science in targeting specific treatment of the disease before the behavioral symptoms manifest.Another related workt from the argument of G, t in its argument if it is completely clear from the context.We may omit the time variable e with not present in the given graph at time t.We may interpret an edge If we use the above conventions, then we will further simplify exposition by renaming the two (complete) input graphs For simplified exposition and ensuring that notations are minimalistic, we will use the following conventions in suitable places through the article including supplementary documents 27. However, extensions of these curvature concepts to graphs and hyper-graphs are quite non-trivial for several reasons such as the discreteness and the lack of a preferred geometric embedding for combinatorial objects. There are several ways previous researchers have attempted to formulate such an extension in addition to what is used in this paper; we briefly review two such major approaches. We note that the references cited at various places in the paper also show that these kinds of network curvature measures can encode non-trivial topological properties that are not expressed by more established graph-theoretic measures such as degree distributions, clustering coefficients or betweenness centralities. Moreover, some of these references also show that these curvature measures can explain many phenomena one frequently encounters in real network-theoretic applications that are not easily explained by other measures.Various notions of curvature are already widely used in disciplines such as physics and mathematics to study properties of high-dimensional objects of certain types Ollivier’s discretization of Ricci curvature (the “Ollivier–Ricci” curvature)31. Roughly speaking, the Ollivier–Ricci curvature is calculated by defining a probability distribution on the neighborhoods of two nodes of an edge and then calculating the difference between the weight of the edge and the The first kind of extension that gives rise to network curvatures involves some appropriate discretization of the Ricci curvature for a Riemannian manifold to capture metric properties of the manifold. Typically, these extensions defines a curvature value for every edge of a graph. In addition to the Forman–Ricci discretization used in this paper, another relevant and more direct discretization is the 32. There is a large body of research works dealing with various aspects of this measure, e.g., see37. The Gromov-hyperbolic curvature is a global measure in the sense that it assigns a scalar value to the entire graph, and therefore is not directly suitable for comparing components of two graphs.In another direction and historically much before the discretizations of Ricci curvature were formulated, Gromov and others defined a notion of network curvature (the ”Gromov-hyperbolic” curvature) via geodesic triangles that captures properties of the set of exact and approximate geodesics of the entire network38 provided a notion of network curvature based on circle packings.There are also other notions of network curvatures explored in past literatures. For example, Chow and Luo30. Assume that n nodes and m edges. For a formal definition we require some basic concepts from topology as available in introductory textbooks such as40; for convenience of the reader, we summarize these concepts in supplementary document 13. To define such a topological association, the following definitions and assumptions are used for a simplicial complex:parallel  cycle of length (number of edges) at most d containing the edges e and Let mentclass2pt{minimmentclass2pt{minimIfGhas no hanging edges with respect toethennot depend on the weights of the faces. Some examples of graph classes that satisfy the condition in Proposition not necessarily list every possible graph class that satisfy the condition):completek-partite graphs for any The class of mentclass2pt{minimcomplete bipartite graphs with every partition having at least two nodes satisfy the condition in Proposition The class of d satisfy the condition in Proposition Any biconnected graph of circumference at most 43 satisfy the condition in Proposition k-dimensional hypercube where e and The class of standard hypercube graphsd allows more graph classes to satisfy the condition in Proposition d, namely when Increasing mentclass2pt{minim44G, namely the class of random graphs, parameterized by p, in which each possible edge independently for inclusion in G with a probability of p for some H) denote the diameter of a graph H. The random graph G by deleting the two nodes u and v and all the edges incident on these two nodes is itself an Erdös-Rényi random graph on p (since edges are selected independently). Note that G satisfies the condition in Proposition u and v. Thus, using known extremal results for w.h.p.)” in probabilistic methods refers to a probability of at least n tends to infinity):nG satisfies the condition in Proposition w.h.p. for all 45.If mentclass2pt{minimp is a fixed constant less than 1 then for all sufficiently large nG satisfies the condition in Proposition 1w.h.p. for all d. For this result we need to use the bounds in Corollary 10.11 in46.If Consider the classical Erdös-Rényi random graph modelmentclass2pt{minimExistence of a cycle d edges containing the edges e and v of v. As mentioned in the “not present in the graph. Viewed in this context, satisfying the condition in Proposition tantamount to satisfying the following claim: for every ordered sequence of  The total number n and d, it is unlikely that each of these n and d are relatively small or unless the connectivity matrix A has a large number of zeroes. For the connectivity matrix considered in this paper, A proof of Proposition mentclass2pt{minim47 as available via the website http://umcd.humanconnectomeproject.org. This is a web based data repository with specific data analysis tools. The data here are in the form of connectivity matrices derived from neuroimaging data. The site is administered by MGH/UCLA Human Connectome Project. The source of the data available via various links in the website are of various imaging modalities such as fMRI, DTI, structural MRI and EEG, however the imaging modality for the specific study that we use is only fMRI. The data is also divided into various subject groups, ages, gender and spans different disorders such as Alzheimer’s, Autism, and adhd. Further details are available in supplementary document The data for our empirical analysis was collected from the UCLA Multimodal Connectivity Databased edges, the running time is prohibitive for arbitrary correlation matrices and arbitrarily large d. Thus, to reduce the time complexity we made the following decisions so that curvatures and z-scores can be computed within reasonable time:We selected very long time to compute the edge curvatures. However, for a threshold of 0.4, the total number of non-zero entries was 25, 220 and the time taken was reduced to a manageable level. We also observed that for a higher threshold of 0.57 the graph corresponding to the non-zero entries in the thresholded connectivity matrix became disconnected, and the total number of non-zero edges was significantly reduced to 8800.The original connectivity matrix had 200 nodes with a total of 34, 036 non-zero entries (edges). In order to reduce the computation time, we sparsified the given correlation matrix by zero-ing out all entries that are less than a threshold of 0.4. We observed that for a threshold of 0.3, the total number of non-zero entries was 27, 701 and our algorithm took a The computation of The weight v, i.e., v.The weight The weight of a triangle (3-cycle) The weight Since computation of entclass1pt{minimau, v such that the absolute value of G in the sense that the random network keeps the relation between the nodes u and v , and preserves the first-order topological characteristics (such as the degree sequence) of G but randomizes higher-order statistics (such as distribution of paths) of G. Due to the absence of accurate generative null models for brain networks, we use the well-known and widely used Markov-chain algorithm for generating random networks48 that starts with the given network G, and repeatedly swaps randomly chosen pairs of connections up to 50. Given such a subroutine for generating random networks, we use the following generic method which is by now standard in the network science literature dealing with various types of biological networks51. We repeat the following step 100 times: during the Z-score52 of the observed value Z-score as the criteria for being statistically significant. Using a Z-score to relate a property of an actual network to the distribution of the same property in an ensemble of randomized networks is a well established and popular method in network science, e.g., see53.As briefly mentioned in the “adhd) patients, respectively, and henceforth will be referred to as the “control network” and the “disease network”, respectively. We implemented a program in PYTHON to efficiently compute u and v.For our results, 54 provide precise mathematical definitions of the Ollivier–Ricci curvature of a graph; for the convenience of the reader we do summarize these definitions in supplementary document e of a given graph G since 54 but n and m via the topology of G. However, under the standard assumption that the zero value of a curvature represents the default flat curvature, one would expect the signs of the two curvatures to be not correlated if they indeed represent different discretization of the Ricci curvature for a Riemannian manifold. For further discussion, we recall the standard definition of the sign function References such asThe following theorem shows the lack of correspondence of the signs of the two curvatures.For everyand for everythe following claim is true: there exists an infinite familyof graphs suchthat for every graphit holds thatandforsome edgeeofG.55; we therefore provide the proof in supplementary document The proof of Theorem 17empirically compared 56). However, as Samal et al. 17 themselves cautioned, their results should not be construed as implying that one of these curvature measures can be used as a universal substitute for the other measure, but merely that for some real networks using one of those measures that allow faster implementation may suffice. Our results in Theorem no contradiction between our theoretical results and the empirical observations in56. To provide an analogy, consider the standard simplex algorithm for the linear programming problem. It has been theoretically proved that there exists examples of inputs for which for which simplex takes exponential time57. However, it is also a well-known observation that simplex runs in polynomial time for many empirical data58, and there is no contradiction between these two kinds of knowledge.On a related note, Samal et al. not significantly different due to many edges of similar curvature values.To begin, we display the first-order statistics of the curvatures of edges for the two networks in Fig. average and the standard deviation of the u, v which satisfied u, v correspond to an edge both the control network outlier edges”). These edges and their corresponding curvature differences in the disease and control networks are shown in Table occipital region and to the parietal lobe region of the human brain.The top 2 outlier edges  control groups.In accordance with the framework of our network anomaly detection problem as defined in the “mentclass2pt{minimz-scores of the z-score of absolute value at least 2 is statistically significant, we note that 4 of the 9 outlier edges in Table (a)The curvature values of all the four edges are higher in the disease network compared to the control network.(b)occipital region and the superior parietal lobule region of the brain.One of the four edges, namely the topmost edge in Table occipital region of the brain is involved in visual processing activities such as object recognition, memory formation and distance perception, whereas the superior parietal lobule is important in planning movements, spatial reasoning and attention. As we have already mentioned before, the importance of these regions in adhd development is consistent with previous studies by Wang et al.59.The (c)occipital region and the frontal operculum region of the brain.The remaining three edges, namely the second, third and fourth edges from the top in Table frontal operculum plays a role in thought, cognition, and planning behavior; for example, see the paper61 regarding the role of operculum in cognition and the subsequent changes in adhd patients. Studies such as62 suggest that the frontal operculum is a critical node in a brain network controlling activities in other brain areas to perform a wide array of cognitive tasks. Activities in the frontal operculum are involved in predicting the efficacy of future tasks, and also in cessation of engagement in a task. Decreased functional connectivities between the left frontal inferior operculum and other areas of brain were studied in63.Generally speaking, the We next focus our attention to the top 5 outlier edges (based on the entclass1pt{minimaentclass1pt{minimaIn addition, for illustrative purposes, we show in Fig. is indeed an outlier with respect to the curvature differences of this edge in the random networks.R_Superior_Lateral_Occipital_Ctx_Superior_Parietal_Lobule_Precuneous_Ctx_143. As can be seen, the actual curvature difference of this edge why not simply use the edge-weights to find out significant differences between the disease and control network?” Previous studies such as64 have indicated that edge weight by itself is not a sufficiently informative measure, for example the set of nodes with the highest sum of edge weights do not capture all the hubs in the brain networks. It is thus of interest to see the differences in edge weights between the corresponding edges in the two networks, and check if a measure as simple as the edge weight difference could also encode the higher order correlations that the curvature method encodes. Let us denote this difference of edge weights between the control and disease networks by fair comparison between the not necessarily the case for the normalize” the e.A natural question that arises here is the following: “significantly smaller than the absolute values of their corresponding normalized curvature differences, and therefore edge weight differences would not have detected the same critical edges as our curvature differences.Figure adhd patients versus healthy humans that exhibit statistically significant changes of the curvature measure. Our usage of the Forman–Ricci curvature measure focused on 2-complex based associations of order 5 with a specific scheme for computing weights of faces. The empirical analysis can be further improved by using k-complex based associations of order d for higher values of k and d. However, a practical limitation of the extent to which such refinements can be carried out in practice stems from the computational complexity issues. Another improvement may come from using different weighting scheme for the faces; for example, for our case Heron’s formula could not be used to provide weights of 3-cycles because it leads to complex numbers, but this obstacle may be avoided for other kinds of data.Using the Forman–Ricci curvature measure as given by  in the fk time steps spanning the progress of a disease in a patient. This provides a series of networks 66 showed that seizures evolve from a more random to regular and then back to random network structure before termination, but curvature analysis methods are likely to provide further non-trivial insights into the temporal structure of these networks.Another direction for extension of our works in this paper would be to apply the Forman–Ricci curvature measure to analyze networks for other diseases such as schizophrenia, alzheimer’s disease or epilepsy. A further but equally important extension in this direction would be to use curvature measures to analyze the progression of a disease when suitable temporal data are available. For example, suppose that we have the brain network data over 17 did empirically compare the Ollivier–Ricci curvature to some version of the Forman–Ricci curvature for a few real networks, we believe a more comprehensive comparison may reveal salient distinguishing characteristics of these two curvatures for brain networks.A third direction of future research would be to compare another discretization of Ricci curvature, namely the Ollivier–Ricci curvature that was mentioned in Theorem Finally, a further future research direction could be to investigate the biomedical significance of the curvature changes of the critical connections identified by curvature analysis methods such as ours.Supplementary Information


Probabilistic linguistic term sets (PLTSs) play an important role in multi-criteria decision-making(MCDM) problems because it can not only describe objects with several possible linguistic terms, but also represent the proportion of each linguistic term, which can effectively avoid the distortion of decision information to a greater extent and ensure the credibility of decision results. First, to compare PLTS more simply and reasonably, we define a new score function that takes into account partial deviations. Then considering the superiority of the classic combinative distance-based assessment (CODAS) method in the complete representation of information, it is extended to the probabilistic linguistic environment. Subsequently, we improved the classic CODAS method and proposed the PL-CODAS method. Finally, we apply the PL-CODAS method to a cases of venture investors choosing emerging companies, and we compare the proposed method with PL-TOPSIS method, PL-TODIM method and PL-MABAC method to verify its applicability and effectiveness. When evaluate the real things, the accurate data are difficult to obtain or the cost is too high to achieve this goal and the decision-makers (DMs) may prefer to assess the objects using the linguistic terms rather than the numerical values  has become a very influential classic financial risk measure, which is called the volatility method. The most important features of the variance calculation is simple and has a fairly mature theory. However, this method also accounts the deviations above the mean when calculating fluctuations, which is unreasonable. Based on this idea, we propose to consider only the deviation of the linguistic terms value below the mean, i.e. partial variance. Subsequently, we define a comprehensive score function on PLTS by the concept of partial variance, which not only considers the mean and hesitation of linguistic terms, but also considers partial variance.In the case of risk assessment, define a score function on the probabilistic linguistic term set, which comprehensively considers the mean and partial variance. The score function can achieves one-step comparison of probabilistic linguistic term set.Put forward an extended probabilistic linguistic CODAS (PL-CODAS) approach to handle MCDM problems where the criteria weights on the probabilistic linguistic term set are completely unknown.Utilize the extended PL-CODAS method to a cases of venture investors choosing emerging companies.Compare with the three existing methods to demonstrate the rationality and availability of the proposed approach.To achieve this goal, the rest of this paper is organized as follows. Section The CODAS  and so on.S should satisfy the following conditions . The current academic community generally believes that the importance ratio is 9, then we can obtain  of the evaluation values and propose the novel score function of PLTS.After the information is represented in the form of probabilistic linguistic variables, how to effectively compare them is a very important issue in the decision-making process. The most common comparison way is based on their score function, and aroused great enthusiasm for scholars. Pang et al.  defined n et al.  uses theLet where entclass1pt{minimaS. Assume that the linguistic terms in S are equal probability, namely, Let If If Let The classic CODAS method was originally proposed by Ghorabaee et al. . As an eIn view of describe the shortcomings of classic methods, we use a simple situation with three alternatives and two criteria to illustrate. These values are dimensionless and between 0 and 1. Figure Here, we consider the origin to be the negative ideal solution. The points in the figure are expressed as To overcome the shortcomings of the classic CODAS method described in Sect. In the classic CODAS method, the valve function for constructing the relative evaluation matrix is expressed as follows: Here, the valve function takes 0 to indicate that the relative evaluation value is the difference between the Euclidean distances of the two alternatives, and the valve function takes 1 to indicate the relative evaluation value is the sum of the difference between the Euclidean distances of the two alternatives and the Hamming distance. Therefore, the meaning of the above function is as follows:if the Euclidean distance difference x between the two alternatives is less than the given threshold, the relative evaluation value is the Euclidean distance difference. If the difference x between the Euclidean distance of the two alternatives is greater than or equal to the given threshold, the relative evaluation value is the sum of the difference between the Euclidean distance of the two alternatives and the difference of the Hamming distance.Because of the shortcomings of the CODAS method proposed in Sect. x between the Euclidean distances of the two alternatives is less than the given threshold, it is considered that only considering the difference between the Euclidean distances is not enough to distinguish the two alternatives. So the Hamming distances needs to be added to distinguish the two alternatives. When the difference x between the Euclidean distances of the two alternatives is greater than or equal to the given threshold, the Euclidean distance is considered to be sufficient to distinguish the two alternatives, and Hamming distance does not need to be considered. Based on the above description, we made the following improvements to the definition of valve function.However, the description here is inconsistent with realistic thinking. In general, when the difference In this section, we introduce PL-CODAS method. First of all, we determine the criteria weights on the basis of the existing weight decision method. Then based on the improved CODAS method, we extended it to the probabilistic linguistic environment and propose PL-CODAS method for handling the MCDM problems.Determining the criterion weights is critical in the whole decision-making process, which can directly affect the outcome of the decision. In many cases, criteria weights are given directly by DMs and it will result in a more subjective outcome. To make up for this deficiency and get more reliable results, refer to Zhang et al.  for the A given standardized decision matrix ith alternative with respect to jth criterion, which is a probabilistic linguistic term set form and has been standardized by Definition Step 1  Transform the PLTS into crisp numbers and obtain new matrix Step 2  Based on the idea of minimizing the deviation from the positive ideal solution and maximizing the uncertainty of the criteria weights, then we construct the following single-objective optimization model to determine the weights of criteria.Step 3  Solve the optimization model and the weight of each criterion is obtained asm alternatives, and n criteria. The specific steps of probabilistic linguistic CODAS method are shown as follows:In the following, we consider the MCDM problem with probabilistic linguistic information. Let Step 1  Construct the probabilistic linguistic decision matrix D, which is shown as follows:ith alternative with respect to jth criterionStep 2  Unify the criteria type to get the decision matrix Step 3  Normalize the decision matrix Step 4  Determine the negative ideal alternative under each criterion as follows:j.mentclasspt{minimaTo better understand the calculation process of the negative ideal solution under each criterion, an example is given below for illustration.Suppose the evaluation values of the four alternative under criterion 1 are  {{s_2}0.,{s_1}..Step 5  Step 6  Compute the weighted normalized decision matrix R.peration  of DefinStep 7  Calculate the Euclidean and Hamming distances of alternatives from the negative ideal alternatives, shown as follows: by Eqs. , respectStep 8  Construct the relevant evaluation matrix Q, shown as follows:Step 9  Calculate the comprehensive evaluation value of each alternative shown as follows:Step 10  Sort the alternatives by the comprehensive evaluation values To better understand the steps of PL-CODAS method, we draw its flowchart as shown in Figure In this section, we use an example H and E bIt can be seen from Table In this section, we demonstrate the rationality and effectiveness of this method by comparing with other existing methods. So far, there is no research on the CODAS method in the probabilistic linguistic environment. Therefore, we use probabilistic linguistic information to compare existing methods for MCDM problems. In the same context, we use existing methods to handle the same case, i.e., PL-TOPSIS method  from PLBAA as displayed in Table Step 7  Calculate the Step 8  Rank the alternatives In this section, we compare the proposed method with the above existing methods. The results are shown in Table From Table The PL-TOPSIS method determines the optimal alternative by calculating the shortest distance to the probabilistic linguistic positive ideal solution (PLPIS) and the longest distance to the probabilistic linguistic negative ideal solution (PLNIS), and considers the influence of the PLPIS and the PLNIS on the selection of the alternatives. However, in the literature (Lu et al. Based on the prospect theory, the PL-TODIM method determines the optimal alternative by considering the cognitive behavior of the decision-maker. This method can capture losses and gains in uncertain situations from the perspective of a reference point. Through comparative analysis, it is found that the ranking results are same obtained by the PL-TODIM method and the proposed method, which shows the effectiveness of the proposed method in this paper.The PL-MABAC method obtains the optimal alternative by calculating the difference between the approximate area of the multi-attribute boundary and the alternative. In the literature (Wei et al. Based on above analysis, the score function that considers the mean and partial deviation proposed in this paper is more comprehensive when comparing probabilistic language term sets. In addition, considering both Euclidean distance and Hamming distance to calculate the gap between probabilistic linguistic term sets makes the ranking results more accurate. In the above two aspects, the developed method is superior to the existing methods.Defined score function on the probabilistic linguistic term set under risk assessment situation.Improved the classic CODAS method and extended it to PLTS.Proposed the PL-CODAS method and applied to actual cases.Compared and analyzed the PL-CODAS method with the PL-TOPSIS method, the PL-TODIM method and the PL-MABAC method, and demonstrated the effectiveness and applicability of the PL-CODAS method.It is noteworthy that this study still has some shortcomings that warrant further research. First, in this paper, we directly quotes the existing literature methods when calculating the criteria weights, and further research on the calculation of criteria weights is needed in the future. Moreover, we have only proposed an expert decision-making model and method. In the future, we will apply this method to more practical situations, such as the prediction the number of people infected with COVID-19.In this paper, we expand the CODAS method and combines it with PLTS information to deal with MCDM problems in uncertain environments. First, we have a brief overview of the definition, operational laws, the normalization and distance measure of PLTSs. Then in consideration of the mean of linguistic terms, partial variance and hesitation of PLTS, we define a new score function. Moreover, we improved the classic CODAS method and proposed the PL-CODAS model. Through a cases of venture investors choosing emerging companies, the effectiveness and rationality of the proposed method are illustrated. The major achievements of this paper are listed below:


With the rapid development of unmanned combat aerial vehicle (UCAV)-related technologies, UCAVs are playing an increasingly important role in military operations. It has become an inevitable trend in the development of future air combat battlefields that UCAVs complete air combat tasks independently to acquire air superiority. In this paper, the UCAV maneuver decision problem in continuous action space is studied based on the deep reinforcement learning strategy optimization method. The UCAV platform model of continuous action space was established. Focusing on the problem of insufficient exploration ability of Ornstein–Uhlenbeck (OU) exploration strategy in the deep deterministic policy gradient (DDPG) algorithm, a heuristic DDPG algorithm was proposed by introducing heuristic exploration strategy, and then a UCAV air combat maneuver decision method based on a heuristic DDPG algorithm is proposed. The superior performance of the algorithm is verified by comparison with different algorithms in the test environment, and the effectiveness of the decision method is verified by simulation of air combat tasks with different difficulty and attack modes. From a macro point of view, air combat decision making refers to one party in air combat providing corresponding control instructions to fighter jets after analyzing and judging battlefield information so that it can complete the dominant attack position occupying the enemy. Decision making is the core of air combat, and its rationality will determine the final outcome of air combat .In recent years, with the continuous improvement and development of deep learning (DL) theory, the deep reinforcement learning (DRL) algorithm combined with deep learning and reinforcement learning has become a research hotspot in artificial intelligence. With no training samples, not limited by specific models, and able to take into account the long-term impact of actions and other advantages, deep reinforcement learning methods have gradually received attention in the research of air combat maneuver decision making. Deep reinforcement learning can be divided into two types: value-based reinforcement learning algorithms and policy-based reinforcement learning algorithm. –4.Q, forming the deep Q network algorithm (DQN)  and the state change, the drag parameter D, the lift parameter L, and the thrust T need to be solved; however, as the drag, lift, and thrust are influenced by various factors, such as altitude, atmospheric density, aerodynamic shape, and flight speed, and are strongly coupled and nonlinear, their parameter expressions are difficult to derive through traditional mechanics. In this paper, the relevant aerodynamic parameters of a publicly available storm shadow UAV  and Lmaxu and Lminu are the maximum and minimum firing ranges of UCAV airborne weapons, respectively.Distance is an important factor for UCAV platform situation assessment and weapon launch conditions. When the relative distance between two aircraft meets the maximum missile launch distance, the maximum distance factor can be obtained, which is defined asv=vu − ve represents the relative speed of the UCAV and enemy aircraft and δv is the allowable relative speed difference from the optimal attack speed.When the distance between the two planes reaches the maximum launching distance of the missile, the UCAV sees the speed of the enemy aircraft as the best attack speed. When the distance between the two planes is relatively far, the UCAV should maintain a large flight speed to rapidly form a favorable situation and maintain a kinetic energy advantage with the help of high speed and maneuverability. The speed factor is established as follows:rENVt is constructed as follows:xlimit=,ylimit=, and zlimit= represent the range of the operational airspace on the X, Y, and Z axes of the inertial coordinate system, and vlimit= represents the extreme value of the safe flight speed of the UCAV.When the UCAV air combat strategy is learned through reinforcement learning, in addition to making the UCAV capable of attacking enemy aircraft, the more important prerequisite is that the UCAV has the ability to adapt to the battlefield environment and maintain a safe flight altitude. Therefore, to train the air combat strategy with both air combat capability and safe flight capability, it is necessary to set negative returns in the form of punishment for dangerous flight maneuvers, so the environmental factor Based on the above analysis, the total reward function isThis section constructs an exploration strategy that is more effective than traditional Gaussian noise or OU noise. At present, the traditional exploration strategy such as OU usually acts directly on the actions of the strategy network output and makes the actions randomly disturbed in the form of addition to realize the exploration of unknown space. In an air combat environment, unmanned combat aircraft control the amount of high dimensionality and large amplitude range; therefore, the DDPG algorithm is based on the strategy of OU explores noise and is likely to create many blind spots in the search. The serious influence training effect, at the same time, is based on the limited performance and flight safety, UCAV variation and volume control of each dimension has a strict limit. When the output action of the policy network is close to the boundary of its scope, it is blind and ineffective to implement exploration by adding noise directly.πe and action generation strategy π are highly similar and can be improved by applying various forms of random noise to the action generation strategy. However, this setting is not conducive to the expansion of state exploration. In contrast, the space that has not been explored in the past will be explored only when there is a significant difference between exploration strategy πe and action generation strategy π. The amplitude of Gaussian noise is an important parameter that has been discussed for a long time, even in the method of using Gaussian noise as the exploration mode. Therefore, it is of great theoretical value and engineering significance to construct an adaptive exploration system method to replace the traditional probabilistic method.A large number of current research exploration methods that potentially set the exploration strategy πe can be decoupled from the action generation strategy π. The specific idea is to construct a more efficient heuristic exploration strategy acting on the experience replay space to have a more positive role in the training of the action generation strategy π.As the DDPG algorithm is a typical off-policy learning method, its exploration process and learning process are independent from each other, so the exploration strategy πe acts as the heurist and generates a set of heuristic information D0 during each iteration, and the action generation strategy π learned by the DDPG algorithm acts as the heurist and receives the D0 heuristic strategy πe and carries out training and learning. Therefore, the decisive factor is changed to adaptively improve the exploration strategy πe to generate optimal value heuristic information D0 according to the learning efficiency of the DDPG algorithm so that DDPG can learn as quickly and effectively as possible.The framework proposed in this section can be regarded as a heuristic learning framework, in which the exploration strategy D0 can be considered as the action ae performed by the exploration strategy πe, and its related reward function should be defined as the improvement of the DDPG algorithm through heuristic information D0:π′=DDPG represents the new strategy obtained by one or more updated steps of the DDPG algorithm on the basis of heuristic information D0, Rπ′, and Rπ represent the cumulative rewards of the DDPG algorithm obtained by strategy π′ and strategy π interaction with the environment, respectively, which have no relationship with exploration strategy πe. R represents the extent to which the heurist (exploration strategy πe) helps the heurist  in the learning process.The generation of heuristic information πe can be parameterized by parameters θπe. Similar to the traditional reinforcement learning method, the gradient J(πe) of parameters θπe can be calculated as follows:P(D0|πe) represents the probability of generating heuristic information D0 : ={st, at, rt}t=1T in the search strategy πe, and its distribution can be decomposed intop is the state transition probability and p(s1) is the initial distribution. Since the probability p has no relationship with the exploration strategy parameters θπe, a gradient of approximately θπe can be obtained as follows:Referring to the parameterized representation of the policy network in traditional DDPG, the policy This value can be calculated from the data that the DDPG algorithm interacts with the environment.R, a heuristic strategy is adopted in this paper. The heuristic is realized by calling the DDPG algorithm one step or n steps in advance. First, a new action strategy is obtained by calling DDPG based on heuristic information D0, and then heuristic information is obtained by using the newly obtained action strategy π′=DDPG. The cumulative reward value π′ can be estimated through heuristic information D1 so that the reward of travel value can be estimated as follows:π, which is obtained by the previous iteration of the cycle.To estimate the difference reward value R is obtained, the following parameters of the exploration strategy πe are updated along the gradient direction of . The specific After the exploration strategy D1, the high efficiency D1 brought to the overall algorithm can compensate for this shortcoming. The construction of DDPG can not only be used to evaluate the improvement of learning performance but also participate in the update of action strategy π.Although the improvement of the DDPG algorithm in this section increases the amount of computation in the calculation of heuristic data Q(s0) curves of the three algorithms are shown in To test the performance of the improved algorithm proposed in this paper, Half Cheetah-v1, a Mujoco robot control environment in the OpenAI Gym toolkit, is selected as the test environment. Considering that the UCAV air combat in this paper is a decision-making process with air combat status information as input, without considering the image input, the RAM version of the environment is chosen and the state information is obtained directly, rather than the RGB version with the game graphics as input. To reflect the performance of the algorithm, 20 Monte Carlo simulations were performed for each algorithm. The Q(s0) distribution. The solid lines of the three colors are the average values of their distribution data. Through comparison, it can be seen that the heuristic DDPG algorithm has a stronger scoring ability after strengthening the exploration performance, which reflects a stronger ability to explore the optimal solution. Simulation comparison tests verify the effectiveness and superiority of the proposed algorithm.In x, Δy, Δz and ΔV are the relative position coordinates and relative flight speed of the two aircraft, respectively.To increase the generalization ability of strategic networks, this paper considers the relative relationship between the enemy and the UCAV in the selection of state variables and takes the three-dimensional relative position coordinates of two aircraft, the relative flight speed, AA, and ATA as state variables; that is, the state variables areu= of the UCAV platform in consideration of generating the smoothness of the maneuver trajectory.κi, Δαi, Δϕi represents the change in throttle lever, change in the angle of attack, and change in roll angle, respectively. The control variable at of the strategy network output at time t acts on the environment to produce the state st+1 at the next step. Together with the state st at time t, the reward rt constitutes the state transfer information .In the selection of the action, it is designed as the variation of the model control variable s=, so the input layer with 12 nodes is set. The output is the maneuvering action control variable ai= in the current state; therefore, the number of nodes in the output layer is 3. The structure of actor network is shown in Combined with the maneuver decision scheme, the actor network and critic network structures in our algorithm are designed. The structures of the actor network and actor target network are the same, and the input value is the state input s= and the change rate of the control value ai= generated by the current actor network. Therefore, the input layer of 15 nodes is constructed, and the network output is the action value function Q. The structure of critic network is shown in The critic network has the same structure as the critic target network. The input value is the combination of the state input The neural network training platform is a TensorFlow open-source deep learning computing platform based on an NVIDIA GeForce GTX 1080Ti GPU in an Ubuntu 16.04 system. The specific hyperparameter settings of the H-DDPG algorithm are shown in To verify the effectiveness of the algorithm, it is assumed that the enemy fighter and the UCAV adopt the same platform model and the same maneuverability constraints. The decision method of enemy adopts the rolling time-domain maneuver decision method proposed in reference . In ordeD ≤ 2500 m&0 m ≤ hr − hb ≤ 1000 m. The air battle trajectory is shown in The UCAV weapon is stronger in the head-on situation, and the launching distance of the UCAV weapon is superior. The winning conditions of the UCAV are as follows: ATA ≤ 30°&200 m ≤ As shown in D ≤ 2500 m&0 m ≤ hr − hb ≤ 1000 m. Air battle trajectory is shown in The enemy weapon is stronger when the firing distance of the enemy weapon is dominant; the UCAV winning conditions are as follows: ATA ≤ 30°&AA ≥ 90°&200 m ≤ As seen from Under this task, the enemy makes intelligent maneuvers using the rolling time-domain maneuver decision method proposed in reference , which iD ≤ 2500 m&0 m ≤ hr − hb ≤ 1000 m. After training, the maneuvering UCAV strategy gradually converges. Under this strategy, the air battle trajectory is shown in Mission setting: the enemy weapon is stronger. Under this mission, the UCAV adopts the rear attack mode to attack the enemy aircraft. The enemy does not need to go around the rear but adopts an omnidirectional attack strategy. The UCAV winning conditions are as follows: ATA ≤ 30°&AA ≥ 90°&200 m ≤ As shown in In this paper, a continuous action space air combat decision-making technology for UCAVs based on reinforcement learning is studied. Starting with the UCAV continuous action space model, a continuous action space air combat model is constructed based on the aerodynamic parameters of the unmanned stealth fighter. Focusing on the problems of weak exploration ability and low data utilization rate of the DDPG algorithm, a heuristic exploration strategy was introduced to propose a heuristic DDPG algorithm to improve the exploration ability of the original algorithm. The effectiveness and superiority of the proposed algorithm are verified by the Monte Carlo simulation in a typical continuous motion control environment . In the simulation verification stage, two subtasks with increasing difficulty, random maneuvers, and intelligent attack maneuvers are adopted for enemy aircraft, and the results show that the method presented in this paper can accomplish maneuver decisions under various tasks as well.


In this paper, we propose an underwater target perception architecture, which adopts the three-stage processing including underwater scene acoustic imaging, local high-order statistics (HOS) space conversion, and region-of-interest (ROI) detection. After analysing the problem of the underwater targets represented by the acoustic images, the unique cube structure of the target in local skewness space is noticed, which is used as a clue to develop the ROI detection of underwater scenes. In order to restore the actual appearance of the ROI as much as possible, the focus processing is explored to achieve the target reconstruction. When the target size and number are unknown, using an uncertain theoretical template can achieve a better target reconstruction effect. The performance of the proposed method in terms of SNR, detection rate, and false alarm rate is verified by experiments with several acoustic image sequences. Moreover, target perception architecture is general and can be generalized to a wider range of underwater applications. Interpreting target information from acoustic images has long been an active research field  in oceanOne solution is to construct a local background distribution model and take the discontinuous isolated singular as the clue to correlate ROI. Higher-order statistics (HOS) is considered as a possible local background transformation method because it has been proven to be sensitive to outliers  and is sCan the local HOS space be a good representation of targets with low SNR in underwater scenes?Can the target information obtained in the local HOS space be directly used for ROI detection?In this paper, we address the challenges of perceiving underwater targets in local HOS space of underwater scenes and, starting from our previous research , 20, we The structure of the paper is organized as follows: A target in the underwater scene is usually represented as a set of specific pixels in acoustic images. The echo intensity, shape, and contour are used to describe this group of specific pixels, which become clues to indicate the presence or absence of the target in the image. Due to the complex and changeable characteristics of the underwater acoustic channel medium and its boundary, the highlight area associated with the potential target will also change significantly. Acoustic images of the same underwater scene collected at different time periods are shown in The above analysis shows that the shape and contour features are not stable. In order to identify the target in the acoustic image sequence formed by the time-varying underwater scene, one solution is to explore the invariable local features , 22 and Sw is calculated byα is the ratio of the target to the background in the window, mB(r) is the r-th origin moment of the background, and A is the average echo intensity of the target. In calculation, A is commonly replaced by the SNR, which is defined as the average power ratio of the target and background echo intensity:μB and σB are the mean value and mean square error of the background, respectively.The background model of the acoustic image is established, which is transformed into HOS space, and the discontinuous distribution region is regarded as the potential target. According to previous research, Weibull distribution is used to represent the background of water imaging and skewness is chosen as the HOS. A rectangular window is designed to traverse the entire acoustic image, and the local skewness of each unit in the window To = 4, and the background followed Weibull distribution. Scale parameters k=5.2 and shape parameters λ=0.42 were estimated from the actual data. As seen in Tw = 7 is set to transform the original acoustic image to the local skewness space, and the deviation correction estimation of the local skewness is given byn is the total number of pixels in the window.Local skewness can be regarded as a function of the target-to-background ratio and the SNR in the window. In the selection of sliding window size, a tradeoff must be made between obtaining high target contrast in the local skewness space and stable background distribution. A simulated acoustic image of 100 × 100 pixels was established, including a square target SNR = 12 dB with a side length SW, and as the target pixels increase in the calculation window, SW gradually decreases.In the local skewness space, the target displays a cube structure as shown in To = 8. Change the SNR to 6 dB, 8 dB, 10 dB, and 12 dB from left to right in Pfa = 0.01, and the detection result is shown in Once the SNR of the acoustic image is relatively low, the target will be submerged by the background, and it is difficult to identify the potential target directly by the detection algorithm. Create a 100 × 100 pixel simulated acoustic image containing a square target of size Tw = 10; convert the original acoustic image in It is considered to implement detection in local HOS space through transformation. Let the window size The simulation results show that the cube structure formed by the potential target in the local skewness space can be used as a clue of ROI, but the SC-CFAR algorithm will have a high false alarm rate in case of low SNR. In addition, the use of large computing windows will form a larger cube structure, which will cause problems in actual target positioning.TM is related to the calculation window size Tw and the target size To, and its expression is as follows:The problem of ROI detection in the local HOS space mentioned above can be solved by target reconstruction. The focus processing is used to reconstruct the target for restoring the original appearance, and the local HOS image and target theoretical template are processed for correlation. The target theoretical template size To = 12 was built. The size of the calculation window was selected as Tw = 15, and TM = 26 could be obtained from equation  target representation in space; (ii) ROI detection performance. Calculate the target SNR of the original image and the local HOS image by equation , and evaTw = 12, and the false alarm rate is set to an equivalent 0.01. Method-II uses a certain theoretical template with a target size To = 9, while Method-III uses an uncertain template with a target range To = [6∼12].The proposed method is compared with other methods in terms of ROI detection performance. The method description and parameter setting are shown in The indoor pool experiment is designed to simulate the underwater scene in which the sonar system is fixed in the safe area near the port to monitor the approaching threat target. A large amount of actual data including real targets was collected, and the acoustic image sequences were generated by water imaging.2. It describes 5.2 × 10.8 m2 water scene parallel to the water surface, in which two targets marked T1 and T2 move along the direction of the track simultaneously. The typical original acoustic image is shown in Acoustic image sequence I contains 33 frames of 261 × 541 acoustic images with a resolution of 0.02 × 0.02 mThe three methods described in Pd, and Pfa of the above three images and the average of the entire image sequence. From the original image to the local HOS space, the average SNR of the target is improved by 30.33%. Comparing Method-I and Method-II, Method-III achieved the highest Pd = 91.35% and the lowest false alarm rate Pfa = 0.03%.The detection statistics of the image sequence I are shown in 2. It describes a water scene of 6.2 × 10.2 m2 parallel to the water surface, containing two groups of targets, one set of relatively stationary targets marked S1 and S2 and another set of moving targets marked T1. The typical original images shown in Acoustic image sequence II contains 48 frames of 125 × 205 acoustic images with a resolution of 0.05 × 0.05 mThe detection results with Method-I shown in Pd = 76.87% and the lowest false alarm rate Pfa = 0.13%.The detection statistics of the image sequence II are shown in Pd and Pfa. Moreover, the target reconstruction restores the cube structure in local HOS to its original appearance, and the uncertain template is used to achieve better focusing effect than the certain template. The target area has a slight but negligible offset, mainly due to the sliding window. Further improvements can be optimized through window location estimation.In summary, the qualitative comparison of the target representation and detection results is given in Figures 10When the original acoustic image is mapped to the local skewness space, the target presents a special cube structure and the SNR is enhanced. Therefore, the target with low SNR is easier to identify in the local skewness space.The cube structure formed by the target in the local skewness space will cause the problem of target positioning. The focus processing can restore the actual appearance of the target as much as possible and determine the position of the target. When the target size and number are unknown, using the uncertain target template can achieve a better effect.An underwater target perception architecture based on layered processing mechanism is proposed and verified by experiments with multiple sets of real data. Experimental results show that compared with the traditional method, the proposed method has a higher detection rate and a lower false alarm rate.This paper studies target perception in local HOS space, which is difficult to interpret in an original acoustic image. The main conclusions are as follows:Future research will promote the application of a layered target perception architecture to threat target tracking in warning areas such as wharf, port, and nearshore.