In recent years, several deep learning approaches have been successfully applied in the field of medical image analysis. More specifically, different deep neural network architectures have been proposed and assessed for the detection of various pathologies based on chest X-ray images. While the performed assessments have shown very promising results, most of them consist in training and evaluating the performance of the proposed approaches on a single data set. However, the generalization of such models is quite limited in a cross-domain setting, since a significant performance degradation can be observed when these models are evaluated on data sets stemming from different medical centers or recorded under different protocols. The performance degradation is mostly caused by the domain shift between the training set and the evaluation set. To alleviate this problem, different unsupervised domain adaptation approaches are proposed and evaluated in the current work, for the detection of cardiomegaly based on chest X-ray images, in a cross-domain setting. The proposed approaches generate domain invariant feature representations by adapting the parameters of a model optimized on a large set of labeled samples, to a set of unlabeled images stemming from a different data set. The performed evaluation points to the effectiveness of the proposed approaches, since the adapted models outperform optimized models which are directly applied to the evaluation sets without any form of domain adaptation. Although there have been several breakthroughs in the area of medical image analysis based on deep neural networks in the recent years, there are still a lot of issues to be dealt with in order for this specific type of technology to be effectively applied in a clinical setting. One of these issues is related to the generalization ability of trained deep neural networks on medical images stemming from different medical centers or recorded with different protocols. This is of particular interest in cases where the images share the same labels but have dissimilar appearances due to the different protocols used to record the data. The domain shift occurring in such settings causes a huge performance degradation of the optimized models, thus preventing a widespread adoption of such approaches in health care. Hence, a lot of efforts are being pulled in the specific area of domain adaptation    to generalize to unseen samples is that both the training set and evaluation set are independent and identically distributed. However, depending on several factors such as dissimilar data recording procedures, the data distribution of the evaluation set can significantly differ from the data distribution of the training set, thus causing the observed performance degradation when evaluating the optimized model on the evaluation set. Hence, various domain adaptation (DA) approaches  approach by adding an adversarial loss characterized by a Domain Discriminator model c+λr+λd = 1. Following the optimization of the models, the resulting domain invariant feature representations can be used in order to perform the classification task in the target domain.with λIn the current work, the source domain classification loss c being the number of classes), wk corresponding to the weight of the sample xk , bs∈ℕ>0 standing for batch size. The adversarial loss is basically an inverted label loss as proposed in Tzeng et al.  data set , collected within a study at the Department of Diagnostic and Interventional Radiology of the Ulm University Medical Center, in Germany. The study was compliant with regards to the health insurance portability and accountability act (HIPAA). The ethics board of the Medical Faculty and the University Hospital approved this retrospective data evaluation study and waived the informed consent requirement (No. 115/21). Two radiologists verified and relabeled the data set. Images were labeled as cases of cardiomegaly based on a CTR threshold of 0.55. A stratified random sampling of the data set in training (50% of the entire data set), validation (25% of the data set) and testing (25% of the data set) sets is performed and subsequently used for the evaluation of the proposed approaches. The data distribution of the resulting sets is depicted in Before being fed into the designed neural networks, CXRs have to be pre-processed in order to significantly reduce the amount of noise within the images, which hinders an optimal optimization of the parameters of the neural networks, resulting into overall sub-optimal classification performances. Moreover, since the data used for the evaluation of the trained deep neural networks stem from another domain, the pre-processing steps also help homogenizing the structure of the data at a certain extent (since the data stemming from both source and target domains go through the exact same pre-processing steps). In the current work, the pre-processing consists of localizing and extracting the chest cavity from each CXR and subsequently using this specific area to perform the evaluation of the proposed approaches. First, CXRs are converted into gray-scale images see . SecondlIn order to perform the assessment of the proposed unsupervised domain adaptation approaches, several experiments are conducted. A baseline of the classification performance specific to each data set is computed by performing a transfer learning classification experiment using uniquely the labeled training set specific to each data set and evaluating the trained model on the corresponding testing set . The architecture of the source classifier consists of two dense layers (fully connected layers). The first layer uses a ReLU activation function and the second layer uses a Softmax activation function.The domain discriminator takes as input the features of the samples from both domains and performs a discrimination between the target domain and source domain (samples from the source domain are labeled as 1 while samples from the target domain are labeled as 0). Its architecture also consists of two dense layers. The first layer uses a ReLU activation function and the second layer uses a Sigmoid activation function. The details of the architectures can be found in −6, for a total of 200 epochs. Due to memory constraints, each architecture is also trained with a fixed batch size of 16. During the domain adaptation experiments, since the size of the labeled source data set is different from the one of the unlabeled target data set, batch training is implemented in such a way that at each single epoch, CXR images are evenly sampled from each data set (source and target). The same amount of samples (the specified batch size bs = 16 in the Equations 3–6) is randomly selected from each data set and fed into the architecture to optimize its parameters. The next training epoch begins, once each sample of the training set specific to the source domain has been used at least once for the optimization of the parameters of the deep neural network, during a single epoch. In the case of the deep reconstruction domain adaptation (DRDA) approach, the weighting parameters defined in Equation (1) are set empirically as follows: λc = λr = 0.5. In the case of the regularized deep reconstruction domain adaptation (RDRDA), the weighting parameters defined in Equation (2) are set empirically as follows: λc = λr = 0.45;λd = 0.1. The implementation and the evaluation of the proposed approaches are performed with the libraries Tensorflow   can be seen in During the unsupervised domain adaptation experiments, the described data sets (see Section 4.1) are used either as the source data set  were chosen empirically, it is believed that an improved and systematic optimization of these specific parameters can further improve the performance of the RDRDA approach. The depicted results also show that the overall performance of the classification task depends on the pre-trained model used as an encoder, with the InceptionV3 model outperforming the other assessed models in most cases. A depiction of the activation map corresponding to a case of cardiomegaly and generated by using an RDRDA approach (the InceptionV3 model is used as backbone in this case) with the OpenI data set as source domain and the Ulm data set as target domain, is presented in The previously described results show that in a cross-domain setting, the domain shift negatively affects the overall performance of a classification model, particularly in cases where there is a huge dissimilarity between the data distributions of both the target domain and the source domain. In such cases, applying a model previously optimized on the source domain uniquely, without any form of domain adaptation, would result in very poor classification performances on the target domain. The proposed unsupervised domain adaptation approaches can effectively generate domain invariant feature representations, that can be subsequently and effectively used in both target and source domains to perform the underlying classification task. The results of the performed experiments have shown that the overall performance of the classification task can be substantially improved by applying either domain adaptation approaches. Moreover, in most cases, the unsupervised domain adaptation approaches also outperform models that are trained on a labeled set stemming from the target domain. Normally the baseline results (As a summary, the proposed unsupervised domain adaptation approaches have proven to be effective in a cross-domain setting with domain shift, since a substantive performance improvement can be observed when adapting a model trained on the source domain to the target domain. In general, both approaches yield similar results. However, the experimental results show that the regularized deep reconstruction domain adaptation approach slightly outperforms its deep reconstruction domain adaptation counterpart. However, there is still some room for improvement and a thorough optimization of the regularization parameters of the loss functions is believed to be able to improve the overall performance of the RDRDA approach. Even if the proposed approaches are very simple and straight forward, other approaches involving generative adversarial networks such as Cycle-Consistent Adversarial Domain Adaptation (CyCADA Hoffman et al., https://bimcv.cipf.es/bimcv-projects/padchest/; https://openi.nlm.nih.gov/gridquery?it=xg&coll=cxr&m=1&n=100.Publicly available datasets were analyzed in this study. This data can be found at: PT, LL, and HK designed the experiments. PT designed, implemented, and evaluated the unsupervised domain adaptation approaches and also collected and pre-processed the data used for the evaluation. CK, DB, AL, and MB collected, annotated, and assessed the data from the Ulm Medical Center. PT and HK wrote the first draft of the manuscript. MB and HK funding acquisition. All authors read and approved the manuscript.


The swift advancement in MANET technology is the driving force behind this rising adoption rate. Routing over MANET is a critical problem due to the dynamic nature of the link qualities, even when nodes are static. A key challenge in MANETs is the need for an efficient routing protocol that establishes a route according to certain performance metrics related to the link quality. The routing protocols utilised by the nodes in WMNs and MANETs are distinct. Nodes in both types of networks exchange data packets through the routing protocols. For this highly mobile network, the ad-hoc On-Demand Distance Vector (AODV) routing protocol has been suggested as a possible solution. Recent years have attracted researchers’ attention to AODV since it is a routing technique for ad-hoc networks that prevents looping. The architecture of this routing protocol considers several factors, including the mobility of nodes, the failure of connection links, and the loss of packets. In this systematic review, one of the key focuses is bringing attention to the classic AODV, which was developed after discussing the recent development of several versions of AODV. The AODV routing protocol performs a path strength check to generate a more reliable and secure route between the source and destination nodes. In AODV, investigations demonstrate advances in both the format protocol approach and the network simulation-2 (NS-2), and these improvements were made in the same scenario used to revitalise AODV. It has been discovered that the AODV is more effective in several aspects, such as throughput, end-to-end delay, packet delivery ratio (PDR), energy consumption, jitter, packet loss ratio, and network overhead. Furthermore, this paper presents this systematic review based on AODV modifications in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). It also provides a methodological framework for the papers’ selection.Mobile  WMNs have become an important way to deal with the problems of next generation networks, such as giving service providers and businesses flexible, adaptable, and re-configurable architectures that are also cost-effective , which is used in WMNs , Optimized Link State Routing Protocol (OLSR), Destination-Sequenced Distance Vector (DSDV), etc. This article proved the claim that the AODV is an essential routing protocol. In addition, this work involves the recent advancements, review analysis, and extensions of the AODV and the concepts of modification to the AODV Routing Discovery Mechanics. Moreover, this work aims to give students and researchers a better outline of the present state of the art in this way. This work takes a systematic approach with the hope that it will help the researcher learn more about the problems that still need to be solved in the field.Another goal of this work is to present a conceptual framework for MANET routing protocols based on various technologies and concepts. Several ad-hoc networks. The mobile and dynamic nature of WMNs could result in link breaking. This link breakage occurs when the speed of WMN mobility increases. Link breakage is especially common in reactive routing protocols like AODV. In the reactive protocol, the message of the broadcast would have found its way to the destination. When many nodes attempt to find a path along one route, the situation is called “route overloading.” The result of this overloading is the breakage of the link. When the link is broken down at the intermediate position of the path, then it finds a new local path that does not sort at the end-to-end route. Finding a new local path is advantageous for the message. Local repair process approaches have been covered by several scholars, including  is crucial on any network. Agent-Based AODV (AB-AODV) manages trusted information locally with a minimum of extra messages and time delays . Each noad-hoc network. AODV does not take into account the node energy in the route discovery process. Nodes with critically less energy can cause transmission failure and packet dropping. That ultimately re-initiates the RREQ and consumes more energy. To address this issue, an energy-aware protocol was proposed by The authors have modified the traditional working mechanism of AODV as suggested in The Hop-Power based AODV (PH-AODV) was presented by The Optimized Adaptive Multi-path AODV (OAM-AODV) proposed in ad-hoc networks are infrastructure-less, self-administrated architectures. The administration of the network is highly controlled by the flood of messages. Communication between nodes cannot be done without energy consumption.Energy-efficient protocols are important due to limited energy participants in a network, especially when Blue-tooth devices are part of the network. Network life time and route stability are directly affected by energy constraints and the mobility of the nodes in hybrid networks. These facts are typically not considered in conventional routing protocols. In Localization is a very important task for any Mobile Low-Duty Wireless Sensor Network (MLDC-WSN). etc (Neighborhood density AODV (ND-AODV) helps in reducing the overhead of ever-changing WANETs, which are highly dynamic and scalable in nature. An expected transmission time (ETX) is used to improve the reliability of the conventional AODV protocol instead of simply a hop count. The RREQ is greatly reduced by considering the neighbours’ density. The derivation of optimal density is empirically shown in the literature. The proposed method is compared with AODV and probabilistic AODV on various performance metrics such as overhead, PDR, throughput, jitter, etc .The Internet of Vehicles (IoV) is getting more attention day by day with the technological advancement in the Internet of Things (IoT). VANET makes communication possible between moving objects in an infrastructure-less environment. Drones and other unmanned aerial vehicles (UAVs) are useful for a variety of applications such as surveillance, shooting, security, and monitoring. The authors have presented drone assisted DSDV (DA-DSDV), drone assisted OLSR (DA-OLSR), and drone assisted AODV (DA-AODV). Experiments were conducted on grids of sizes of 300 × 1,500 m and 300 × 6,000 m. The drone assisted protocols have achieved quite good results over conventional algorithms .etc. etc.The Internet of Underwater Things (IoUT) is gaining popularity due to factors like exploring the ocean surface, security, industry expansion, military etc. It is also constrained by a lot of other things, including the energy of the node, latency, and reliability. At any given moment, the node would be in one of the following states: transmit, receive, idle, or sleep. The node consumes the most energy while transmitting data. All nodes are battery operated and have limited energy backup.A decision factor (DF) based modified AODV (Mo-AODV) is proposed in etc., which leads to performance degradation.Energy-efficient protocols are always a need for MANET. Enhanced AODV (ENH-AODV) focuses on route selection by taking into account the quality of links and upcoming nodes . Each noTo address the issues of congestion and energy, Path congestion and path robustness were used to evaluate the fitness of each solution. Path congestion reflects the load of the route, and path robustness reflects the life span of the route. The survival time of a path is defined by the minimum energy level of each node in the route.Improved Priority Aware AODV (IPA-AODV) targets the issue of the velocity of mobile nodes in the network. The wireless network does not allow mobility beyond 2 m/s. The proposed method restricts any node from participating in the routing if its velocity is greater than the predefined velocity threshold .In ad-hoc on-demand distance vector (QAODV) driven routing protocol has been proposed. A power constraint factor is used with the distance parameter. Experiments have shown an improvement of 14.28% in throughput, 27.83% in delay, and 11.6% in packet loss compared to conventional AODV. RREQ packets are broadcast throughout the network, and power and hop count are updated at each node. On receiving back the RREP, the destination node checks the power of the node. The routes that require the least amount of energy are eliminated. The route is selected for further communication if its power is greater than a predefined threshold. If the power of a neighboring node goes below a threshold, route discovery is initiated again. Work can be extended further for mobile nodes. AODV protocol’s real-world implementation in RFC  has been the focus of recent attention by n in RFC .e.g., energy, delay, and bandwidth). Many reports are incomplete and have the minimum details of protocols because many protocols are present in Traffic load effects, source-destination pairs, and mobility in different stages have been discussed. Reports in relation to routing protocol reviews and surveys can be found in the literature. There are many reports on popular routing protocols; this protocol depends on different routing mechanisms . Some reThe survey study is based on the AODV and its analysis. In short, it has been defined that many survey reports cover famous extensions and the formation of the AODV protocol. This search report gives a quick overview of how the AODV routing protocol is used in WMNs and in other research. Agents are the tiny packets in-network, which move across the network and collect the data bout networks and the node.etc. These characteristics allow the agent to freely move, communicate, and gather information. Authors in literature presented an improved version of AODV, which is known as Agent-based AODV (A-AODV), which operates on three principles: routing metric, route discovery, and routing algorithm. Simulation results show that A-AODV always has better PDR compared to AODV and DSR  protocol, etc . This contains the hops that are needed to reach the destination route. All nodes participate in sending the reply, and then the source node makes it to the destination of the forward route. This creation is not meant to create a complete route compared to the source route but to form the hop-by-hop state destination of the source route. The Optimised Link State Routing Protocol (OLSR) knows all the present routes. OLSR is an advanced type of “pure link-state protocol.” The topology changes, and then the OLSR overflows to the topological information of an active node. An Multi-point Relay (MPR) is used to reduce the overhead of network protocols .The MBR decreases the copy re-transmission of the forward broadcast packet. In this process, re-transmission bounces to the small neighbouring nodes . Insteadad-hoc networks  protocol: To solve network blockage generated by the protocol of AODV, a modification called the PAODV protocol has been developed  protocol: Some new features have been added to PAODV that solve the problem of the PAODV protocol called the Neighbour-PAODV protocol. The new features in PAODV relate to the addition of a parameter. This parameter shows the probability of rebroadcasting of the RREQ routing packet in the discovery routing process. It is evaluated by the following formula, as described in the study  and the number of neighbors (Nb). The p of the node relates to the Nb of that node. The routing table has Nb numbers  is the older version of the Network Simulator-3. Network Simulator-2 is widely used because it has some new and different tools. The implementation of NS-2 depends on discrete-event simulation.This search paper is not completed, but this section references initial cases to the readers. The open-source simulators category describes the report of Network Simulator-2 (NS-2) and Network Simulator-3 (NS-3), which are actual examples of literature. The Network Simulator-2 . The PRISMA is a minimum list of things that must be reported in systematic reviews and meta-analyses. This list was made based on the evidence that was already available.The first step of current structured analysis of compositions was comprised of describing the publications that were examined. The current search analysis was restricted to the databases containing papers from conference proceedings and the documents considered the most important to the fields of AODV and relevant literature. Besides, broader searches of the SCOPUS and WoS database have been carried out to include related efforts from other origins not contained in the starting list. These databases include Scopus, Science Direct, Taylor Francis, The ACM Digital Library, Springer, IEEE, and Emerald Publishers. These seven databases that have the papers in professional journals and at international conferences are worth discussing, as shown in i.e., the summary introduced at the conference), compositions, and incomplete articles were excluded from the analysis.The Systematic Review Database, Scopus, and Google Scholar were used to identify the modifications in the AODV routing discovery mechanism for wireless mesh network work already published in the open access publishing/online publishing and reviewed publications. Grey writings .Each article was assessed in-depth at the eligibility level. The researchers extracted the proposed recommendations and the specific studies conducted to validate or confirm those recommendations for each paper. The papers were eliminated from the current work if the suggested design guidelines contained design standards too broad for this work or too confusing for the researcher to understand (coders).Several data processing tools were applied to keep records ordered during the identification and filtering phases. Online Google spreadsheets were used as a communication method. This made it possible for the researchers to work together. They could store and lookup records from the reviewed papers, analyze the data, and add notes.ad-hoc changes to AODV routing discovery mechanics have been steadily going up since the turn of the century.Many researchers examined a qualitative analysis of the information extracted to better categorise the design guidelines and prepare the coded data to address the research questions that guided the systematic literature review. Although our study focused on a review of AODV studies published since the early 1900s, it nonetheless provides a snapshot of current research in this area. Based on this research, it is clear that several publications about WMN and Inclusion and exclusion criteria were used to categorise relevant and non-related research, as all modifications to the AODV routing discovery mechanics in wireless mesh networks were included in the sample population. Furthermore, the current systematic review includes several recent articles about AODV modification in MNAET relevant to this review.Inclusion requirements were based on a focused research topic as a prime precondition, modification to the AODV routing discovery mechanics in WMNs and execution while studying, a publication report available to the public or held between 1900 and 2022, recognisable rates of users, and promoted to all levels of users. A qualitative analysis concentrated uniquely on the abstracts of the papers, which did not fulfil any of the terms for implication and did not fall under the measures of exclusion. The current work was entirely checked for eligibility criteria to be calculated.During the last phase of the procedure for collecting data, all of the necessary information was gathered, and any disagreements that existed amongst the authors were first discussed and then settled upon. Therefore, the authors summarised these articles. The process of interpreting and classifying the articles assisted us in obtaining a wide variety of essential and compelling hints. As a direct result of this, many potential future initiatives and recommendations have been proposed.It is worth noting that the research investigation was conducted carefully, and it offered a detailed basis concerning modification to the AODV routing discovery mechanics in the wireless mesh networks. It is also imperative to keep in mind that the most challenging part of adopting the PRISMA methodology was coming up with indirect ways to describe the approaches in the abstracts and methodological parts of the publications.Therefore, the authors had to go through the entire content of the papers and look at more knowledge to determine the particular approach used for changing the AODV routing discovery mechanics in assessing the requirements for wireless mesh networks. In this section, our literature review’s findings on our initial research questions have been discussed.The current section represents the formal validity of the AODV. Also, the formal validity represents results in the form of a summary. Furthermore, the selection and extraction of articles will be outlined in the following subsection.A total of 627 papers were selected from over 1,000 for the initial search and identification process. The database lists of all the studies are reviewed systematically to do such work. A full-text review of the 175 articles was carried out to extract information from the correspondence in the eligibility process. This review omitted 50 more papers because they were too general or ambiguous to provide valuable research data on modifying the AODV routing discovery mechanics in wireless mesh networks.This process resulted in 125 papers being marked as applicable to the AODV routing discovery mechanics in the wireless mesh networks for modification. In the end, in the course of the information withdrawal (‘Contained’ in the chart of PRISMA), the concluded set of articles was further checked to remove the related modification data found in the AODV routing discovery mechanics in WMNs and specifics of the studies that generated and validated the design guidelines. After carefully going through all the titles and abstracts, there were only 125 reviews that could be useful.In The data collected by the researchers was collected separately using the proper procedure and documentation. Interviews with the third analyst resolved disagreements, and a consensus was reached following the discussion. The required information was collected in the final stage of the data collection process, and any differences between the authors of this study were addressed and resolved. Then, the authors compiled the selected 125 articles. By translating and sorting papers, we were able to get a wide range of essential ideas.As a consequence, many possible future initiatives and guidelines have been suggested. It should be noted that the study investigation was carefully carried out and provided a thorough basis for modification to the AODV routing discovery mechanics in wireless mesh networks. There was an issue with papers’ abstracts and methodological sections not explicitly expressing methods used during the PRISMA technique’s implementation. Therefore, the writers must go through the entire content of the articles and take a more in-depth look at more information to determine the exact form of the modification to the AODV routing discovery mechanics in the WMNs criteria assessment. This part of the selection process took a long time, but it ensured that the most relevant publications were chosen.The research group used personal feedback to develop and refine the paper selection criteria. The analysis was performed according to the standards established in the past analyses. Studies were categorised as either “important” or “not-important” based on the inclusion and exclusion criteria, such as informal literature . Also, as mentioned above, several papers have been left out because they were published in unknown indexes.The number of articles discussing AODV has increased dramatically in recent years, as shown in e.g., qualitative, quantitative, expository). More details are presented in The previous studies related to the modification of the AODV routing discovery mechanics in wireless mesh networks and MANET, which have been concluded from the last stage, have been summarised, and their characteristics are presented in ad-hoc on-demand distance vector (AODV) protocol is the widely used hop count protocol, which operates on need and utilises an efficient route discovery mechanism. The route from source to destination is usually determined by the route discovery mechanism, where the address of each node is traversed to reach its destination. It is a common routing protocol to create the topology of a network. It responds quickly to dynamic link conditions, low processing and memory overhead, and low network utilisation and determines uni-cast routes to destinations in the ad-hoc network. It uses destination sequence numbers to prevent a loop and thus prevents sending the same packet to the node from another node.The A standard method of AODV routing transmits the route request to all nodes . AnalysiOnly one possible route includes the node if the RREQ is obtained from an intermediate node in its route list. These mentioned nodes form part of this route list if the chosen nodes can only transfer the route request packet. The intermediate node’s neighbourhood density is regarded in this proposed scheme as the factor in the forwarding plane of RREQ for the in-between node. This implies that the possibility of transmission of any node will become low if the neighbour’s count of nodes increases. Finally, the above information will decrease. The picking of nodes improves the likelihood of complete network coverage at random from the neighbourhood set. More significant savings could be achieved by choosing nodes for transmission using a range-dependent strategy. This can only be done at the expense of expanding the scope .The protocol, AODV, is for reactive routing based on route exploration and path conservation protocols. The path discovery process is established only when a node has a package to deliver. A node that wishes to transmit packets creates RREQ packets and broadcasts them when it finds a path. For this purpose, modifications are required to the AODV protocol relating to the route discovery procedure .As a reactive algorithm, AODV has the advantage of not needing to keep track of routes for nodes that are not actively communicating. Bellman-Ford’s “counting to infinity” problem is stopped by AODV, which tells the affected nodes about the status of the current link. Employing sequence numbers per request helps the nodes execute the route request they have already processed. The multi-cast group’s IP address should be set as the destination node for the multi-cast route. After the route is discovered, the source node must uni-cast a Multi-cast Activation (MACT) message to activate the route . The advThe reported simulated routing protocols have been analysed by In the proposed work of Although some studies , have suThe relationship between the destination node and the source ties of the three pacts and the number of nodes in the region has also been shown in In The experimental results show that the Fortified Ant protocol’s connection search speed is remarkably higher than the conventional ACO. The protocol’s norm is end-to-end postponements, and the transference performance fare is better recommended than the two AODV and AOMDV networks. The simulation aims to examine the impact of increasing the network size, end-to-end delay, and packet delivery ratio on different VANET environment routing protocols  using the NS-2 simulator.More attention has focused on the provision of AODV with his suggested work of It can be shown that the packet delivery fraction has continuously improved in all kinds of densities with the use of DP-AODV compared to normal AODV. The proposed technology limits density by managing transmission power, which accounts for the increase from 12% to 31%. The dropped packet rate is higher in basic AODV than in DP-AODV, and many packets are lost due to fixed transmission power, producing extensive interference between nodes. Also, the delay has decreased with DP-AODV for most densities. Currently, it is at 52%, down from 54%. Therefore, DP-AODV is typically inferior to standard AODV when looking for routes with a higher success probability and lower data transmission latency.In contrast to AODV, where many packets are lost due to fixed transmission power and severe interference between nodes, the proposed DP-AODV limits density by regulating the power used for transmission. The author presented the results obtained from the experimental studies. The DP-AODV was found to be better in both terms of throughput and jitter. The suggested protocol becomes more acceptable and produces high performance in terms of throughput and jitter network parameters.The DP-AODV is applied to the single-path transmission in the study scenarios. However, the study still lacks knowledge of the behaviour of the new DP-AODV with the multi-path transmission. Furthermore, the DP-AODV race has an overhead as the number of nodes in a network grows. It is a noticeable overhead that various node densities will increase with the increasing number of nodes. The author compares overhead with various nodes in e.g., NS3 and NS2). In already documented studies, different mobility models were considered to construct the network scenario more like the real world. Studying various routing protocols provided us with the experience of using the correct protocol under different network conditions in multiple instances.The comparison was carried out with simulation tools  definition and autonomous and connected vehicles leads to new innovative technologies with various service quality requirements, thus increasing the new challenging data transfer problems. So far, several routing protocols have been proposed for such connections. Ad-hoc on-demand Distance Vector (AODV), Dynamic Source Routing protocol (DSR), Optimum Link State Routing (OLSR), and Destination-Sequenced Distance-Vector Routing Protocol (DSDV) are among the most common ones. This analysis tests the routing mechanisms of AOVD for WMNs and their modifications.It is critically assessed that ad-hoc networks. AODV is ideal for networks with modest computational power and relatively light traffic as a reactive protocol. The advantage of AODV is that it does not create overhead for inactive nodes, which turns out to be a disadvantage as the traffic load increases. The lack of network topology information causes AODV to under-perform. Reviewed literature shows that conventional AOVD was slow in performance and information changed with time, and conventional could not get that information appropriately. So, we have conducted a literature review to highlight the modification in the AOVD routing protocol in wireless mesh networks.The analysis starts with the implementation of WMN architectures and routing function basics. Thanks to their most significant pledge to incorporate various descendants’ wireless networks, the WMNs have been the subject of research in recent years. Modern communication systems have been the subject of recent studies on WMNs, motivated by rich and high-speed content access. In contrast, the suggested protocols have comparatively received little attention. There is a broader field of applications. Effective routing protocols may offer significant advantages in efficiency and reliability for mobile Previous studies also noted that AODV had been modified for transmitting route request messages. The updated version was called AODV EXT. Unlike possibility-based strategies, where a fixed probability can be allocated to each node that does not guarantee complete network coverage, the methodology suggested in previous studies and this paper review incorporates eliminating repetitive re-transmissions on routing paths, principles from complete range routing algorithms with node pruning. Still, it provides interconnection and is more robust in the route path than wireless network guarantees.i.e., Energy Reversed ad-hoc On-Demand Distance Vector, which contains up to 1.7% more power in comparison with that of ad-hoc On-Demand Distance Vector Request EXT). Compared to the standard AODV, AODV EXT increases data throughput by more than 19% and 10% more than the ER-AODV.The findings of the present study can also be compared to the routing protocol proposed by i.e., AODV) demands a solution and has previously suggested eliminating inefficient re-transmissions depending on the transmitted node. Neighbourhood densities have shown pledging outcomes. This research has shown that for ad-hoc networks to be as efficient as possible, protocols would need to be fine-tuned to fit different traffic or applications.The conclusions The authors conducted a systematic literature review identifying research patterns on AODV routing discovery dynamics modification in wireless mesh networks. This research review article concludes that modifications to the AOVD routing protocol have been presented. For this highly mobile network, the AODV protocol is proposed, but AODV only saves a minor hop route to the destination. The path that increases the network overload needs to be rediscovered when the connection breaks. In this review paper, the researchers highlighted the traditional AODV previously invented and addressed a new addition to the AODV (AODV routing protocol) that inspects path strength to create a more secure route between the destination and source. Studies show improvements in the format of RREQ and Hello messages to track the time of sending and path stability in the new versions of AODV. A comparison and review of previously published simulation results were performed to determine the efficacy of the AODV. It has been found that the AODV shows improved efficiency in several aspects.ad-hoc network with few resources in future work.The study presents an outline of all the essential aspects of AODV, which will be helpful for academics working in this field who are looking into this topic. In conclusion, based on this survey, the current routing protocol mentioned above, AODV, has been extensively researched. However, there are still ways to improve the protocol to ensure quality in an unreliable mobile


In this work, a visualization experiment for rectangular channels was carried out to explore gas–liquid two-phase flow characteristics. Typical flow patterns, including bubble, elastic and mixed flows, were captured by direct imaging technology and the corresponding measurements with fluctuation characteristics were recorded by using an electrical conductivity sensor. Time-domain and frequency-domain characteristics of the corresponding electrical conductivity measurements of each flow pattern were analyzed with a probability density function and a power spectral density curve. The results showed that the feature vectors can be constructed to reflect the time–frequency characteristics of conductivity measurements successfully by introducing the quantized characteristic parameters, including the maximum power of the frequency, the standard deviation of the power spectral density, and the range of the power distribution. Furthermore, the overall recognition rate of the four flow patterns measured by the method was 93.33% based on the support vector machine, and the intelligent two-phase flow-pattern identification method can provide a new technical support for the online recognition of gas–liquid two-phase flow patterns in rectangular channels. It may thus be concluded that this method should be of great significance to ensure the safe and efficient operation of relevant industrial production systems. Rectangular channels (RCs) with high heat-transfer efficiency and small size have been widely used in small reactors , compactThe many approaches to flow-pattern identification can be divided into direct observation and indirect measurement, according to various applicable principles . Direct Compared with this direct method, the indirect measurement method is more widely used, especially in the industrial production process, which does not require direct observation of flow patterns in most cases. The indirect measurement method obtains signals such as capacitance , pressurWith the recent development of computer science, machine learning algorithms have been widely applied with good classification ability ,22. In fIt is worth mentioning that various gas–liquid mixing scenarios can result in various measurements, time series, or signals see . With anInspired and motivated by all of the above studies, in this work, the issue we needed to address was how to identify and classify the various flow patterns simply and accurately with characteristic parameters of electrical conductivity measurements of the gas–liquid mixing process in rectangular channels. Accordingly, the aim of this work is to propose a two-phase flow-pattern recognition model or framework combining the time-domain and frequency-domain characteristics and a support vector machine. In fact, a unqualified flow pattern not only greatly reduces production efficiency, but also causes serious harm to equipment. This investigation on gas–liquid two-phase flow-pattern identification is of great significance for understanding the heat-transfer mechanism in rectangular channels and ensuring the safe and efficient operation of relevant industrial production systems. The contribution of this work is two-fold. First, using flow data analysis, this article responds to a number of current growing needs for the in-depth mining of sensor data. It provides a systematic solution for the integrative analysis of conductivity measurements, as the electrical conductivity and the flow images were measured and obtained instantaneously. Thus, the subjective experience of operators can be avoided as much as possible for flow-pattern recognition in rectangular channels. Second, this study used a classification methodology to provide a novel and broad framework for combining a feature vector reflecting time–frequency characteristics with a support vector machine. A large number of models and extensions are potential outcomes within this framework, as several signals—such as conductivity, temperature, and pressure in industrial processes—can be input into the algorithms proposed in this work. Hence, the universality of this work can provide a demonstration for various investigators in related fields, including multiphase flow.The rest of this paper is organized as follows. In The schematic of the gas–liquid mixing process in a vertical rectangular channel is illustrated in 3 to 1.27 kg/m3 was used as the gas phase. An NaCl solution with different concentrations was used for the liquid phase. Thirty sets of experiments were designed in order to obtain different flow patterns, as shown in During the experimental procedure, normal-temperature air with density ranging from 1.21 kg/mIt is worth mentioning that electrical conductivity varies with the changing of various flow patterns. In fact, the surface tension and viscosity of fluids can indeed affect the evolution of flow patterns, but may not have a remarkable effect on the measurement of electrical conductivity and the identification of flow patterns. In addition, changing the original measurements of electrical conductivity is so very complicated and irregular that it is difficult to understand the flow pattern accurately according to the original direct measurements. However, the abnormal information  of the original measurements could be reduced by extracting the features of conductivity measurements to identify the flow patterns in the rectangular channel.The probability density function has often been used to describe the expected values of random variables from a sample in time–frequency domain analysis . AccordiThe power spectral density, which is a measure of the mean square value of a random variable, belongs to a probabilistic statistical method and the power spectral density curve shows the relationship between power spectral density and frequency . Accordi in Ref. .A support vector machine was originally a binary classification algorithm that supported both linear or nonlinear classifications , as showIn addition, The flow pattern images of the gas–liquid mixing process in rectangular channel under a variety of working conditions were recorded. At the same time, the conductivity of the gas–liquid mixing process was recorded by a conductivity sensor. It is worth mentioning that the conductivity of the gas–liquid mixture changes with various flow patterns. In other words, the fluctuant conductivity measurements are recorded as bubbles, gas–liquid mixtures, and liquids flowing through the conductivity sensor. As shown in In fact, the mean of the conductivity measurements of different flow patterns was also different. It can be observed from In order to extract the time-domain features of the conductivity time series, the probability density function was first used to analyze the conductivity measurements in a vertical rectangular channel. The probability density function curves of the conductivity measurements of different flow patterns were obtained via calculation software and are shown in 2 and 6.03 × 103, indicating that mixed flow was an unstable flow pattern. In general, the difference between the standard deviations of these three flow patterns was relatively large. Hence, the standard deviation In order to further quantify the time-domain characteristics of the typical flow conductivity measurements, the standard deviation −2 to 2.49. Obviously, the skewness The specific distribution of the standard deviation and the skewness of the three flow patterns in the rectangular channel are shown in −3 Hz, showing that the power distribution of the three flow patterns was very concentrated. Meanwhile, it can also be seen that the peak of the bubble flow was the largest. The peaks of the elastic flow and the mixed flow were relatively small, and the difference between them was not big. Compared with those of the bubble flow and the slug flow, the power distribution of the mixed flow behaved more uniformly. The difference in peaks did not suggest different flow patterns. However, other characteristic parameters need to be introduced to assist when the peaks are close. On the basis of time-domain analysis, the conductivity measurements were processed by the power spectral density with frequency-domain analysis. The power spectral density curves of the three typical flow patterns were obtained and are shown in In order to further quantitatively extract frequency-domain features of conductivity measurements, three characteristic parameters—frequency corresponding to the maximum power Moreover, the ranges of distribution parameters of the power spectral density of the conductivity measurements in the rectangular channel are shown in The time-domain and frequency-domain characteristics of the conductivity measurements of three typical flow patterns during gas–liquid two-phase mixing in rectangular channel were analyzed and quantified. It can be seen that the five quantized characteristic parameters—First, several five-dimensional feature vectors consisting of In this work, the gas–liquid mixing process in the rectangular channel was measured, and the flow pattern images and conductivity measurements of the bubble flow, the slug flow, and the mixed flow were obtained. The time-domain and frequency-domain characteristics of the conductivity measurements were analyzed by the probability density function and power spectral density. The feature vector describing the flow pattern in the rectangular channel was constructed by introducing characteristic parameters. The classification and recognition of gas–liquid two-phase flow patterns in the rectangular channel was realized by using a support vector machine, and good results were obtained. The main conclusions can be summarized as follows: (1) The relationship between the gas–liquid flow patterns and electrical conductivity in the rectangular channel was discussed in detail. The bubble flow, the slug flow, and the mixed flow were measured in different conductivity while passing through the conductivity sensor. The evolution of the flow pattern could be qualitatively analyzed by observing the fluctuation of the conductivity measurements. (2) The probability density function and power spectral density were used to process the conductivity measurements of the gas–liquid two-phase mixing process in the rectangular channel for the first time. The five-dimensional feature vector describing the flow pattern was constructed by introducing feature parameters, including


In the autonomous driving process, the decision-making system is mainly used to provide macro-control instructions based on the information captured by the sensing system. Learning-based algorithms have apparent advantages in information processing and understanding for an increasingly complex driving environment. To incorporate the interactive information between agents in the environment into the decision-making process, this paper proposes a generalized single-vehicle-based graph neural network reinforcement learning algorithm . The SGRL algorithm introduces graph convolution into the traditional deep neural network (DQN) algorithm, adopts the training method for a single agent, designs a more explicit incentive reward function, and significantly improves the dimension of the action space. The SGRL algorithm is compared with the traditional DQN algorithm (NGRL) and the multi-agent training algorithm (MGRL) in the highway ramp scenario. Results show that the SGRL algorithm has outstanding advantages in network convergence, decision-making effect, and training efficiency. In autonomous driving, the decision-making system is mainly used to produce advanced actions of vehicles, such as lane changing, acceleration, braking, and so on. Tactical decision-making for autonomous driving is challenging due to the diversity of environments, the uncertainty in the sensor information, and the complex interaction with other road users ,2. TradiThe operational space of an autonomous vehicle (AV) can be diverse and vary significantly. Due to this, formulating a rule-based decision-maker for selecting driving maneuvers may not be ideal . With thThe realization of the decision-making is based on the understanding and analysis of high-dimensional environmental information. However, the traditional reinforcement learning algorithm only has a good capacity for decision-making based on low-dimension features . It willInteraction between vehicles in common public transport scenarios is necessary and pervasive. However, there are relatively few studies on how autonomous vehicles interact in public environments with reinforcement learning. Realizing coordination between vehicles in a shared environment is challenging due to the unique feature of vehicular mobility, which makes it infeasible to apply the existing reinforcement learning methods directly. Chao Yu et al. proposed using a dynamic coordination graph to model the continuously changing topology during vehicles’ interactions and developed two basic learning approaches to coordinate the driving maneuvers for a group of vehicles .Cooperative vehicle–infrastructure systems and automatic driving technology are developing rapidly . The gra1.Multi-agent training simultaneously increases the computational network complexity, resulting in a higher overall training time cost. Therefore, each parameter modification requires a more prolonged verification time, which is not conducive to the development and adjustment of the algorithm.2.The reward and punishment offset each other in multi-agent overall training, resulting in poor network convergence in the training process. Due to the mutual influence of multiple agents’ reward values, it cannot accurately evaluate the current state, resulting in the very unstable fluctuation of the loss curve.3.Through the test of the final training model, the final task success rate of the GCQ algorithm is maintained at around 4.The GCQ decision-making model stays in the lateral lane change and cannot control the longitudinal behavior of the vehicle at the same time. This is an incomplete driving control model, which leads to a low success rate, serious collisions, and low traffic efficiency.However, the above multi-agent training-based GNN reinforcement learning (MGRL) has the following problems in the actual verification process (the scenario of highway ramp exiting):1.Based on retaining interactive feature extraction (GNN), the trained object is transferred from multi-agent to single-agent. Through the internal processing of the network model, the training results of single-agent can be applied to the application scenarios of multi-agent. This training method can significantly reduce the time cost of model training and eliminate the interaction of rewards and punishments between agents to improve the training effect.2.An inductive reward function is designed to improve the convergence speed of the model. The reward function incorporates driving intention, collision, lane change frequency, and vehicle speed into the calculation. The trained model simultaneously performs well in terms of task success rate, safety, driving stability, and traffic efficiency.3.The dimension enhancement of the action space is realized by changing the network structure. Adding vehicle longitudinal velocity control gives the model a more robust control ability for task success rate, safety, and traffic efficiency.4.In the training process, the real-time screening of stored data can improve the training speed. The random generation mechanism of vehicles’ number, type, and position in the training scenario can improve the model’s generalization ability and avoid overfitting.Given the above problems, this paper proposes an improved single-agent GNN-based RL algorithm . This paper has the following contributions:The paper is organized as follows. The proposed SGRL methods are used to model the Markov Decision Process (MDP). The agents can explore the environment by observing states, taking actions, and receiving rewards, as shown in The implementation core of the SGRL method is based on the preprocessing of input data, the structure of the deep neural network, and the setting of the output end.t, the input of the training model can be divided into two parts: feature matrix i. The position of the vehicle is represented by three-bit binary coding. For example, vehicles in the rightmost lane are denoted as i. The data type is similar to In each time step in the training process, the vehicles in the scenario can be divided into human-driven vehicles (HVs) and autonomous vehicles (AVs). At each step i of the matrix represent the relationship between vehicle i and other vehicles through binary data. The value j within the set distance of the target vehicle i will be set to 1, as shown in Based on GNN, we can introduce a correlation matrix Considering that the number of vehicles in the scene is dynamic, this situation has been considered when setting the feature matrix and the correlation matrix. The vehicles in the environment are divided into HVs and AVs, located in the feature matrix’s upper and lower parts. When the number of vehicles is less than N, the positions in the matrix are occupied by 0.t cannot reach the predetermined matrix size. To ensure the standard calculation of GNN, matrix data filling is needed. Matrix segmentation prevents data confusion and error correspondence caused by random packing, as shown in To ensure the authenticity of the simulation process, all vehicles appear and disappear successively in the scenario, so the feature information and correlation information obtained at the step In the whole network structure, the function of graph convolution is to obtain the interaction information between vehicles. The function of the full connection layer is to parse the matrix information. Each newly obtained matrix in the network needs to be input into the full connection layer for recording and information analysis. In addition, the number of nodes in each layer also plays a decisive role in the model’s performance. The optimal number of nodes is selected through comparative experiments.Data records for comparative tests are shown in t, the feature matrix At each step Next is the calculation of graph convolution. The input of the convolution function is the newly obtained feature matrix Equation ).(2)Kt=The original feature information and the new information obtained by graph convolution are fused by matrix stitching online as the total input of the subsequent neural network Fully Connected Network (FCN): Graph Neural Network (GNN): Q Network: Output: The feature data density changes at each stage are as follows:The overall structure of the SGRL algorithm is shown in Since the vehicle longitudinal control model built into the simulation environment is still rule-based, it cannot incorporate the interaction between vehicles into the control model. The SGRL algorithm fuses longitudinal and lateral control into the same model by increasing the output dimension, which significantly simplifies the complexity of the control process and solves the coupling problem of two directions.The SGRL algorithm is trained based on DQN, so the output action space can only be discrete. For AVs, the longitudinal control is mainly reflected in the acceleration, and the lateral control is primarily reflected in the lane change direction. The improvement of the output action resolution is conducive to improving the control sensitivity, but it also increases the computational complexity and reduces the control frequency. In the algorithm of this paper, a compromise solution is selected. The longitudinal control is set to 11 discrete values in the interval The setting of the reward function needs clear guidance and a strong correlation with training objectives. In the SGRL algorithm, the task success rate, security, and traffic efficiency should be considered simultaneously. The corresponding reward values are set as intention reward, crash reward, and speed reward.To guide autonomous vehicles to complete driving tasks from the corresponding ramps out of the highway, the SGRL algorithm constructs reward gradients for different lanes, as shown in To ensure practical guidance for all locations, the reward value t and output the number The safety of autonomous driving is the basis of all other characteristics. The simulation platform SUMO can detect collisions at step Equation ).(3)RC−To control the consistency of the reward scale, the speed reward value Equation .(4)RS−tThe general method to calculate the total reward is a weighted summation of each component. Direct addition will lead to mutual coverage of rewards, resulting in poor training effects. The SGRL algorithm uses a new aggregation method, as shown in Equation .(5)Rt=ωThis calculation method takes the task success rate and safety as the primary considerations and considers traffic efficiency simultaneously. The problem of vehicle parking for intention rewards can be completely avoided. Moreover, the weight relationship between rewards can be adjusted by parameters The most prominent feature of the SGRL algorithm is training for a single agent, but the obtained model can be applied to a multi-agent environment. For trained agents, the vehicles around them can be considered the same category—surrounding vehicles. Therefore, the work done by the GNN-based reinforcement learning model can be interpreted as planning and decision-making based on the characteristics and relationships of the target agent and its surrounding vehicles. The model obtained by single-agent training can be directly transplanted to other agents in the same scenario, as shown in To prevent the overfitting of the reinforcement learning process, we randomize the distribution of training vehicles and the task of autonomous vehicles in each episode based on fixed rules, as shown in Algorithm 1 SGRL Q Learning Steps.Initialize the reply memory R to capacity NInitialize the weights for the SGRL Net  doFor step e choose random action for agent i:  With the probability  With the probability   State decoding:   Double FCN:   GNN + FCN:   Feature Stitching:   Double FCN:   Compute Q values:   Select  Execute  Store in the buffer Set ## Training model at training step ##Sample a batch from the buffer and calculate the Q target: Get average Update parameters n steps #### Updating Target Net every Algorithm 1 shows the detailed steps of training.Algorithm 2 SGRL Testing Steps.Initialize the simulation environmentT doFor step  State decoding: do For AV   Move other AV’s features to the back of HV:   Calculate Q based on trained model:   Select  Get the action matrix  Execute  Set Algorithm 2 shows the detailed steps of testing.Two baseline models are introduced for comparative analysis in the simulation: the traditional DQN algorithm (NGRL) and the multi-agent training algorithm (MGRL). In the NGRL algorithm, the GNN part is removed, and the correlation matrix is used as input by splicing with the feature matrix.The model of the MGRL algorithm is consistent with the SGRL proposed in this paper in terms of network structure, but its training process is based on multi-agent environment interaction.By comparing the three models, the specific effects of the GNN structure and single-agent training of SGRL can be effectively analyzed. The simulation scenario is shown in The simulation scenario is a long highway with three lanes. There are exit ramps at one-third and two-thirds of the total length respectively. The speed limit for the whole road is set as 20 m/s (76 km/h) for all the vehicles. The AVs and HVs are put into the scenario at the probability of 0.1 and 0.4 from the left side of the road. And the initial speed and lane position are random.There are six types of vehicles in the scenario, including HVs that travel straight through the highway, vehicles (HVs and AVs) that want to exit from ramp_0 and vehicles (HVs and AVs) that want to exit from ramp_1. The simulation environment controls the driving of HVs, and the AVs are completely controlled by the reinforcement learning model SGRL in real time.The number of vehicles in the experiment is set as shown in All three models were trained for 1000 episodes. The symbolic data of the training process are the reward, average Q value and loss value. Average rewards are obtained by averaging rewards against steps. The specific changes are shown in For the comparison of reward values in the training process, under the same reward value calculation, the reward values and average reward values of the SGRL algorithm can converge faster and have better final convergence.The changing trend of the Q value and loss value of the SGRL algorithm in the training process is consistent with the learning process. According to the loss results, SGRL has a faster convergence speed.To compare the task success rate, security, and traffic efficiency, the average velocity, number of collisions, success rate, and average steps per episode must be collected. The results of the above training data are shown in The SGRL algorithm has obvious advantages regarding task success rate and average vehicle speed. In terms of collision times and average training step length, SGRL also meets driving safety requirements.In addition, SGRL has apparent advantages in training efficiency under the premise of the same training episodes. The specific hardware parameters and training time are shown in The trained model needs to be verified by the test process. To fully verify the algorithm’s effectiveness, we adjust the total length of the road under the premise of the same traffic flow (20 vehicles per episode). The three algorithms are tested on 1000 m, 750 m and 500 m roads to simulate different traffic flow and congestion levels.Each test process includes 1000 episodes. In the test process, the reward value can still be used as an essential evaluation of model performance. The simulation results of reward value and average reward value are shown in For the most complex and congested 500 m highway scenario, the longitudinal motion spatial distribution of the three algorithms throughout the test cycle is as shown in The longitudinal control of autonomous vehicles will become more and more complex with the increase in road congestion, so the test results of the 500 m scene are the optimal reference. The test results show that the action output of the SGRL algorithm is mainly concentrated near 0, and the probability of large acceleration is acceptable.To compare the task success rate, security, and traffic efficiency, the average velocity, number of collisions, success rate, and average step per episode must be collected.The data of different methods are listed in It can be seen from the figure and table that SGRL has apparent advantages in task success rate and average velocity. In terms of the number of collisions and the test steps, although the SGRL value is not the best, it is consistent with the best value.It can be seen that the SGRL algorithm has outstanding advantages over the MGRL algorithm and the NGRL algorithm. The SGRL algorithm can converge faster during the training process and achieve better data performance. In the testing process, the SGRL algorithm has outstanding data performance in terms of task success rate, average vehicle speed, and security.The MGRL algorithm directly sums the reward value of all autonomous vehicles in the process of reward value calculation, so there is a phenomenon in which the excellent performance of vehicle behavior and the poor performance of vehicle behavior offset each other, which is not conducive to the adequate updating of parameters, and also causes the final convergence speed to be slow, and thus the convergence effect is not good.The NGRL algorithm lacks the calculation consideration of the interaction process between vehicle individuals. Therefore, in the decision-making process, due to the relatively simple understanding of the environment, it cannot obtain sufficient data support, so the data performance is poor.The SGRL algorithm considers and solves the above problems and optimizes the network structure. According to the comparison of data, it can be verified that the improvement of SGRL is obviously effective.This paper proposes a generalized single-vehicle-based graph neural network reinforcement learning algorithm . This algorithm combines GNN with deep reinforcement learning to solve the vehicle planning problem in the scenario of highway driving out of the ramp. The SGRL model is trained for a single agent and can be tested in multi-agent scenarios. At the same time, the algorithm sets up an improved reward function to provide a clear direction for training.Firstly, a training mode for single-agent training extended to multi-agent scenarios is proposed and verified in terms of training effectiveness and performance. The algorithm improves the analytical ability of the DRL by increasing the number of network nodes, thereby increasing the control dimension of vehicle decision-making to longitudinal and lateral dimensions.Secondly, the proposed SGRL algorithm simplifies the training mode and improves the training efficiency without affecting the training effect. This helps to adjust complex parameters and reduce time costs.Thirdly, SGRL is more sufficient in the training process to achieve a better convergence effect. The fluctuation is the smallest after the training data are stable. This shows that the proposed SGRL algorithm has outstanding training ability and is more suitable for decision-making based on reinforcement learning in multi-agent scenarios.Finally, the newly designed reward function effectively solves the problem of mutual influence between longitudinal and lateral control. SGRL can achieve higher task success rates and average velocity in the training and testing process. This shows that the new reward function, the training method for a single agent, and the incorporation of GNN effectively improve the decision performance of the model.Comparing the three algorithms, the conclusions are as follows:In future research, the continuity of model action space can be added to this algorithm, which will effectively improve the driving fluency of vehicles. In addition, the relationship between multiple agents in the scenario should not be limited to physical characteristics: decision-making, driving intention, and task priority can also be incorporated into the calculation process.


The spatial–temporal prediction of traffic flow is very important for traffic management and planning. The most difficult challenges of traffic flow prediction are the temporal feature extraction and the spatial correlation extraction of nodes. Due to the complex spatial correlation between different roads and the dynamic trend of time patterns, traditional forecasting methods still have limitations in obtaining spatial–temporal correlation, which makes it difficult to extract more valid information. In order to improve the accuracy of the forecasting, this paper proposes a multi-scale temporal dual graph convolution network for traffic flow prediction (MD-GCN). Firstly, we propose a gated temporal convolution based on a channel attention and inception structure to extract multi-scale temporal dependence. Then, aiming at the complexity of the traffic spatial structure, we develop a dual graph convolution module including the graph sampling and aggregation submodule (GraphSAGE) and the mix-hop propagation graph convolution submodule (MGCN) to extract the local correlation and global correlation between neighbor nodes. Finally, extensive experiments are carried out on several public traffic datasets, and the experimental results show that our proposed algorithm outperforms the existing methods. With the rapid increase in the number of vehicles in cities, the rational planning of urban transportation has become an important challenge. Intelligent transportation systems (ITS), as a vital intelligent traffic management system in intelligent cities, can provide new solutions to urban road traffic problems. In this paper, we study one of the most representative spatial–temporal forecastings, traffic flow forecasting. Traffic flow is a part of the intelligent transportation system (ITS)  and refeAt present, the problem of traffic flow prediction based on spatio-temporal data has attracted extensive attention from researchers ,4,5. In At the same time, deep learning has made great breakthroughs in the field of traffic flow prediction ,9. For eWe propose a dual graph convolution framework with graph sampling and aggregation (GraphSAGE) and mix-hop propagation graph convolution (MGCN) to capture spatial information. By fusing the neighbor nodes information extracted with these two methods, the capability of capturing spatial relations can be further improved.We propose a multi-scale temporal convolution with a gated mechanism as a temporal block, in which the temporal correlation of traffic data at different scales is extracted using convolution kernels of different sizes, and the obtained features are fused and adjusted by an efficient pyramid split attention module (EPSA).These experimental results conducted on four public datasets show that our proposed algorithm outperforms the existing methods.To address these challenges, we propose a multi-scale temporal dual graph convolution network (MD-GCN). First, we use kernels of different sizes on the temporal convolution module, which can complement the multi-scale temporal dependence and avoid the problem of gradient bursts. After the output of temporal convolution, we use the gating mechanism to filter unnecessary information. From the spatial perspective, as the traffic network becomes more and more complex, the change of traffic flow is obviously affected by its topology, and the traffic flow data between adjacent roads and between roads with a longer range are obviously closely related. However, in previous studies ,15,16, rThe subsequent work of this paper is organized as follows: Traffic flow forecasting has long been regarded as an important part of ITS to help alleviate unexpected rising traffic flow, and traffic flow forecasting is a classic time series forecasting task. Compared with the traditional time series and machine learning models, deep learning-based models , e.g., LIn recent years, deep learning models have been widely used in traffic flow prediction , which mZhao et al.  proposedHowever, the existing graph convolution models only change the ways of constructing the graph and cannot effectively capture the deep spatial information from the perspective of aggregating nodes. In this work, we design the dual graph convolution module with GraphSAGE  and an MRecurrent neural networks (RNNs) have often been used for time series prediction. However, traditional RNN-based methods are inefficient when training longer sequences, and their gradients are more likely to explode when combined with graph convolution networks. Therefore, researchers ,34,35 beE is the set of edges between nodes representing a connection between two nodes (sensors), the adjacency matrix n is the number of nodes. If there are two nodes In this work, we define the traffic topology as t for all the nodes D is the number of traffic features. Given a traffic network graph G and the histroical traffic flow, the traffic flow prediction can be defined as a mapping function f,S time steps and T time steps to be predicted.We define a feature matrix N spatial–temporal blocks and a complete fully connected layer as the output block. In MD-GCN, each spatial–temporal block consists of a spatial block and temporal block. The temporal block is mainly a multi-scale gated temporal convolution module and an efficient pyramid split attention module. The spatial block is composed of a graph sampling and aggregation (GraphSAGE) module and mix-hop propagation graph convolution (MGCN) module. The main innovation of this model is that it constructs modules separately to extract spatial correlation and temporal correlation. For the mining of temporal relations, we use a channel-centered multi-resolution gated temporal convolution model to improve time data processing ability. For the mining of complex spatial relationships, we use the spatial information extracted by the GraphSAGE module and MGCN module to enhance the ability to summarize the information of neighbor nodes. The following sections describe the detailed structure of each module.The structure of MD-GCN presented in this paper is shown in Due to the different traffic conditions at different times in the future, the temporal information extracted by using temporal convolution in TCN  is oftenk as the number of layers of the current temporal convolution module with F is the number of time features of the output, and C is the number of channels.In recent years, the temporal convolution model has been widely used in time series analysis. We propose a multi-scale gated temporal convolution module (MGTCN) as shown in g parts represented as After MGTCN combines different convolutions by splicing, the channel attention module is introduced to capture the correlation between channels. In this work, we use the efficient pyramid split attention module (EPSA) , mainly g group convolution splicing. We extract channel attention weights for data at different scales by We adaptively select the size of the group according to the size of the convolution kernel, where the relationship between the group and the convolution kernel is For transportation networks, traffic conditions in adjacent locations influence each other, and the spatial relationship between roads can be captured to predict traffic more accurately. In previous studies, the correlation was usually captured from the global aspect of nodes, and the local correlation of nodes was not fully considered, but transportation networks often contain different dependencies. The spatial module uses the graph sampling and aggregation module and the mix-hop propagation graph convolution module to extract spatial features and hidden spatial dependencies in parallel. The details of the module are defined in the next two sections.In this section, we use the GraphSAGE module to spatially model the road network. The module generates node embeddings as follows: given a node L is number of layers for graph convolution, and In this module, we uses the mix-hop propagation graph convolution module as shown in In this section, we verify the effectiveness of our proposed model on four real datasets. We will introduce the experiments in detail from the aspects of experiment setup, baselines, convergence analysis, parameter study, experiment results, ablation experiment, and case study.We evaluate the preformace of our proposed model and baseline models on four widely used traffic datasets. The properties of the datasets are summarized in METR-LA ,15: It iPEMS-BAY ,15: It iPEMS04 ,35: It iPEMS08 ,35: It iS and T are set equal to 12, the first S time steps are our input data, and the last T time steps are considered to be our actual label values. Using 12 consecutive time steps from the past, we predicted 12 successive time steps in the future. In each dataset, all experiments were repeated ten times. The number of layers N for the entire spatial–temporal block is set to 3; the number of layers L of the spatial blcok is set to 2; and the number of layers K of the temporal block is set to 3. In the model proposed in this paper, all the convolution operations are set with 64 filters . In the spatial–temporal block, the size of the hidden layers was set to 64. The initial value of the expansion factor was set to 2. In the training stage, we use adam to optimize the model, the batch size is 32, and the learning rate is set as We divided the dataset into a training set, validation set, and testing set in the ratio of 7:1:2 and used the same hyperparameters on four datasets. R is the total number of samples, and We use three evaluation metrics commonly used in baseline papers to evaluate the predictive effect of the model, including mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE). The formula is shown below:We selected the latest research methods to compare our models.FC-LSTM : This moT-GCN : This moGraph WaveNet : This moSTFGNN : This moSTSGCN : This moDCRNN : This moSTGCN : The modASTGCN : This moMTGNN : This isIn order to explore the convergence of our proposed model, we show the error between the ground truth and the prediction results preduced by MD-GCN in the training and validation process on the four datasets in In the section, In the spatial block, as the number of network layers deepens, node representations of the same connectivity graph tend to have the same value; it is impossible to distinguish between different nodes (over-smoothing). In order to solve the problem, we set an initial node information retention factor The number of layers in the spatial block will have different effects on the extraction of spatial information, so we use an experimental comparison to select the most suitable number of layers for the spatial block. As shown in Compared with ASTGCN, STFGNN, MTGNN, and GraphWaveNet, the MD-GCN model proposed in this paper adopts the method of constructing spatial–temporal information mining hidden structures. In a temporal block, we use channel attention mechanisms and temporal convolution networks to combine the characteristics of data at different scales. Our spatial block adopts the method of graph convolution and graph aggregation sampling dual graph fusion to integrate the spatial information extracted in different ways. To further investigate the effect of our model, we show the training error at each time step of the two datasets METR-LA and PMES08 in w/o GraphSAGE: In the mixed hop propagation graph convolution module, we remove the GraphSAGE module.w/o EPSALayer: In the temporal module, we remove the efficient pyramid split attention module.w/o MGTCN: We replace the multi-scale gated temporal convolution module with a normal time convolution module.In order to verify the effectiveness of each module in the model, we performed ablation tests on four datasets, and the main process is as follows:In our experimental setup, we first verify the validity of the dual graph convolution module and then use the graph convolution module alone to extract the spatial structure information. Second, we validated the need for the channel focus mechanism by removing the EPSAlayer module. Finally, we choose the traditional temporal convolution module to verify the MGTCN module. As shown in In this section, we plotted the predictions of MTGNN and our model 60 min ahead against the actual values on both datasets. We randomly selected the prediction of two sensors over time from two datasets, and the final result is shown in From the experimental results, we can see that our proposed MD-GCN model is able to obtain performance improvements in terms of the evaluation metrics: RMSE, MAE, and MAPE. Compared with our dual graph convolution module, MTGNN and Graph WaveNet only use adaptive graph convolution to extract spatial features, which makes it difficult to show good results in both long-term and short-term prediction. Our proposed model can enhance the ability to extract hidden spatial information by integrating two graph convolution methods to aggregate node information of different ranges. Compared with our MGTCN module and EPSA module in a temporal block, STSGCN and STFGNN use the temporal convolution to extract time information, and the predictions on average time steps are also not as effective as our model. Our proposed temporal module can extract time features at different ranges and adjust the features using channel attention to obtain more effective time correlation. From the results on these representative evaluation metrics, our model shows more stable and better results in traffic flow prediction than these popular baselines.From the results obtained by the ablation experiment, we can find that our proposed dual graph convolution module and multi-scale gated temporal convolution module, as well as the EPSA module, can improve the accuracy of prediction, which also explains the necessity of our work. From the comparison of real road data and forecast data in the case study, we can intuitively observe that our model shows better stability and accuracy in the face of complex traffic data than other baseline models.In this paper, we propose a novel spatial–temporal model (MD-GCN) to predict traffic conditions. Specifically, in terms of time dependence, we propose a gated temporal convolution module based on multi-scale channel attention combined with an “inception” structure. By expanding the width of the convolution network and combining the receptive field of temporal convolution at different scales, the temporal relationship capture ability of the model is effectively improved. For spatial dependencies, we combine two modules: the GraphSAGE module and the mix-hop propagation graph convolution module. The spatial information extracted by fusing the two modules improves the ability of the model to obtain feature relationships of different ranges in traffic networks. Finally, we choose to verify the validity and stability of the model on four datasets METR-LA, PEMS-BAY, PEMS04, and PEMS08. In addition, the ablation experiments again validate the effectiveness of our model. For future work, we will consider the influence of various external factors to further improve our work.