Fully Depleted Monolithic Active Pixel Sensors (FD-MAPS) represent an appealing alternative to hybrid detectors for radiation imaging applications. We have recently demonstrated the feasibility of FD-MAPS based on a commercial 110 nm CMOS technology, adapted using high-resistivity substrates and backside post-processing. A p/n junction diode, fabricated on the detector backside using low-temperature processing steps after the completion of the front-side Back End of Line (BEOL), is reverse-biased to achieve the full depletion of the substrate and thus fast charge collection by drift. Test diodes including termination structures with different numbers of floating guard rings and different pitches were fabricated together with other Process Control Monitor structures. In this paper, we present the design of the backside diodes, together with results from the electrical characterization of the test devices, aiming to improve understanding of the strengths and limitations of the proposed approach. Characterization results obtained on several wafers demonstrate the effectiveness of the termination rings in increasing the breakdown voltage of the backside diodes and in coping with the variability of the passivation layer characteristics. A breakdown voltage exceeding 400 V in the worst case was demonstrated in devices with 30 guard rings with 6 μm pitch, thus enabling the full depletion of high-resistivity substrates with a thickness larger than or equal to 300 μm. Additionally, we show the first direct comparison for this technology of measured pixel characteristics with 3D TCAD simulations, proving a good agreement in the extracted operating voltages. Monolithic Active Pixel Sensors (MAPS) are considered promising candidates for the replacement of standard hybrid pixel radiation detectors in medical imaging, high-energy physics and space applications due to their advantages in terms of production costs and low material budget ,2,3,4. EIn the last few years several monolithic radiation detectors were developed following different approaches. The MAPS produced for the upgrade of the ALICE Inner Tracking System at CERN represent one of the first examples of monolithic sensors applied to a large high-energy physics experiment . In thesHigh-Voltage CMOS (HV-CMOS) processes provide an alternative solution, fully compatible with foundry standard production flows, for the fabrication of FD-MAPS . SeveralSensors reaching several tens of μm depleted thickness with small pixels and small-capacitance electrodes have also been demonstrated ,13. For 12 cm−2, whose concentration cannot be accurately controlled, and a concentration of surface states typically larger than the one present with thermal oxide. Thus, the design of the backside diode termination structures was carried out taking into account the possible high variability in the characteristics of the backside interface and oxide charge. The validation of the backside structures presented in this work is essential to fully understand the possible issues and the characteristics of the employed production process in view of its future use in the fabrication of thick detectors.In order to provide a cost-effective solution, which could be easily portable to a more scaled process node, we focused on the development of FD-MAPS fabricated with a standard CMOS production process, with a backside junction obtained with low-temperature processing steps carried out after the completion of the entire front-side process . In the In this study, we present the extensive experimental results obtained on the test structures embedded on the backside of a batch of samples produced with a modified 110 nm commercial CMOS process. The influence of the termination structure geometry on the breakdown voltage of the backside junction was quantified by measuring a group of test diodes specifically designed for this purpose. These results were compared with the breakdown voltages estimated from TCAD simulations to validate the simulation models. Gated diodes were characterized to extract the surface generation velocity, while the capacitance-voltage (C–V) characteristics were recorded on backside MOS capacitors to estimate the oxide charge concentration. Finally, we measured the current–voltage (I–V) characteristics of a pixel array without embedded electronics to extract the voltages associated to the full depletion of the device substrate and to the onset of the punch-through between the frontside and backside p-type implants.The paper is organized as follows. The simulation domains and the models employed in the TCAD simulations are described in A schematic cross-section of the proposed sensor including the backside termination structures is shown in unctions .A dedicated set of TCAD simulations was carried out to select the most promising geometries for the termination structures in order to prevent breakdown in the operating voltage range. For the considered devices, the lower bound of this range is represented by the voltage needed for the full depletion of the sensor substrate, while the upper bound is related to the voltage at the onset of the punch-through between the frontside pwells/deep pwells, and the backside The design of guard rings for silicon detectors is a known art, covered for example in ,19. Howe® Sentaurus TCAD tool. In the first set of simulations we considered a 2D domain representing a simplified cross section of the device and including a variable number of floating rings with different pitches. The simulations were performed with the SynopsysA thin passivation layer was included on top of the backside surface. The thickness of the insulator was chosen according to the value obtained experimentally from process-control structures. Outward metal field plates were included on top of the guard rings, in order to lower the local electric field peaks and further increase the breakdown voltage of the diode. The surface generation velocity −2, since the value of this parameter in the deposited layers is subject to large process variations, and the termination structures should be functional within the entire range of variability.We considered different concentrations of charges trapped inside the surface passivation layer For the estimation of the diode breakdown voltage, we have used the Van Overstraeten and De Man ionization coefficient model  based onThe two graphs reported in The breakdown voltages were extracted from the I–V curves, considering a threshold on the diode current equal to TCAD simulations were also performed to estimate the voltage range that needs to be applied to the backside contact for the correct operation of the sensors. As reported in , this opIn the simulations, we exploited the pixel matrix symmetry to reduce the simulation domain size and thus the overall time and resources required for the computation. The operation voltage range was extracted from I–V curves obtained with quasi-stationary DC simulations. A negative voltage sweep was applied to the backside 2, chosen accordingly to the acceptable power dissipation. It can be observed that the operating range can be extended by increasing the voltage applied to the sensor nodes, and an applied voltage of 0.8 V is sufficient to provide a depletion voltage significantly smaller than the onset of punch-through currents. On the contrary, the current measured at the backside As mentioned in The test structures include pixel arrays with integrated electronics and 50 μm pixel pitch and passive test structures made of pixel arrays with 50 μm and 25 μm pitch, having all the sensor nodes connected in parallel to a common metal pad. In addition to active and passive pixel arrays, several test structures have been designed to qualify the characteristics of the backside layers and of the termination structures. More specifically, a set of backside Most of the measurements were performed at a probe station on diced chips using a Keithley 4210 Parameter Analyzer. High-voltage measurements were carried out using a Keithley 2410 Source Meter.The I–V curves of a group of test diodes were acquired to extract the breakdown voltages. These values were compared with the voltages obtained from the TCAD simulations  in It is worth noticing that, thanks to the multiple-guard-ring termination structure surrounding the backside In this work, floating guard ring termination structures for the backside junction needed to bias FD-MAPS detectors have been presented. The structures were designed with the help of TCAD simulations, fabricated with post-processing on front-side processed CMOS wafers and verified experimentally, demonstrating the feasibility of the proposed approach.A good agreement was found between the breakdown voltages obtained from the simulations and the ones extracted from the I–V curves of the test diodes.The measurements of the gated diodes and of the MOS capacitors provided an estimation of the surface generation velocity and of the surface oxide charge concentration that can be expected in the future productions realized with this technology.We are confident that a good radiation hardness can be achieved by the backside diodes realized with this technology, since the test structures proved to be properly working with levels of Finally, the operating voltage range of integrated pixels was analyzed both in simulations and experimentally. We investigated the effect of the voltages applied to the frontside nwells and n-guard ring on both full depletion and punch-through voltages. Both the simulations and the measurements showed that increasing the applied


Dynamic calibration was performed in the laboratory on two catching-type drop counter rain gauges manufactured as high-sensitivity and fast response instruments by Ogawa Seiki Co. Ltd. (Japan) and the Chilbolton Rutherford Appleton Laboratory (UK). Adjustment procedures were developed to meet the recommendations of the World Meteorological Organization (WMO) for rainfall intensity measurements at the one-minute time resolution. A dynamic calibration curve was derived for each instrument to provide the drop volume variation as a function of the measured drop releasing frequency. The trueness of measurements was improved using a post-processing adjustment algorithm and made compatible with the WMO recommended maximum admissible error. The impact of dynamic calibration on the rainfall amount measured in the field at the annual and the event scale was calculated for instruments operating at two experimental sites. The rainfall climatology at the site is found to be crucial in determining the magnitude of the measurement bias, with a predominant overestimation at the low to intermediate rainfall intensity range. Surface precipitation measurements have been recognized as “indispensable, despite advances in several areas of remotely sensing of precipitation” . Direct However, surface precipitation measurements are affected by measurement errors themselves. Catching-type precipitation gauges are subject to environmental and quantification errors, environmental conditions at the collector (including wind-induced undercatch and wetting/evaporation losses), to their instrument mechanics and measuring principle  . QuantifThe World Meteorological Organization (WMO) held dedicated intercomparison campaigns, both in the laboratory  and in tThe WMO recommendations  and new One example of a catching-type precipitation gauge is the Drop Counter (DC) rain gauge. The DC rain gauge collects rainfall in a funnel and conveys the rainwater towards a calibrated nozzle. Water is then dispensed as drops to a sensor that counts using optical or acoustic principles ,17,18.−1. This allows the instrument to rapidly sense any variation of the rainfall rate during precipitation events.Due to the small volume of the dispensed drops, these instruments are characterized by a very high sensitivity, suitable to detect very light precipitation events, which is two orders of magnitude lower than for typical tipping-bucket rain gauges. The time needed by the drops to fully develop and detach is very short even during low precipitation events, e.g., approximately three seconds for an instrument with a sensitivity of 0.004 mm at RI = 5 mm hDC gauges were recently employed as the reference instruments for in-field rainfall measurement campaigns, to assess the performance of the more common tipping-bucket rain gauges (TBRs) ,20. In b−1, but no correction was proposed.However, these instruments are not exempt from systematic measurement biases. A recent experimental study was conducted in the field by Chan et al. , at the In the present work, two different DC rain gauge models were tested in the laboratory using dynamic calibration to evaluate their performance at various rainfall intensities. An innovative derivation of the calibration curves based on the actual drop volume generated by the gauge nozzle is proposed rather than using the constant drop volume approach provided by manufacturer. Suitable calibration curves were derived for both instruments to account for the drop volume variation as a function of the drop dispensing frequency. Corrections were applied in post-processing to calculate the RI values and to evaluate the improved performance obtained with the correction. Moreover, the operational limitations of the two DC gauge models were investigated, and the threshold RI value for the validity of the involved measuring principle identified.Calibration curves were then applied to 20 years of field data available from the Chilbolton Facility for Atmospheric and Radio Research (CFARR) in the UK and 2 years of data from the Hong Kong International Airport (HKIA) field test site. This allowed to quantify the impact of the constant drop volume assumption vs. full dynamic calibration on the measurement of the total precipitation amount at the annual scale.In this study, two DC rain gauges manufactured by Ogawa Seiki Co. Ltd.  and the Chilbolton Rutherford Appleton Laboratory  are considered see , named i−1. The drop water volume, calibrated by the manufacturer, is assumed to be constant and equal to 0.0625 and 0.0600 cm3 for the OSC and CRAL, respectively. The area of the collector, measuring range, calibrated drop volume and sensitivity of each instrument are summarized in The two DCs have similar measuring ranges, as documented in the manufacturer specifications, with a maximum rainfall intensity value equal to 200 mm hThe sensitivity of the two gauges is equal to 0.005 mm and 0.004 mm for the OSC and CRAL, respectively, which is two orders of magnitude lower if compared with traditional tipping-bucket rain gauges. This characteristic makes this kind of instrument particularly suitable to measure very low rainfall occurrences.The time resolution of the two instruments is different: the OSC provides the number of drops counted every 10 s, while the CRAL was set to count the drops falling over a one-minute time interval.The laboratory tests were performed at the WMO/CIMO Lead Centre “B. Castelli” on Precipitation Intensity, according to the recommended calibration procedures developed during the WMO Laboratory Intercomparison of Rainfall Intensity gauges  and desc−3, the equivalent RI reference value was calculated.The calibration system was composed by a constant head water tank that feeds two volumetric pumps . The two pumps generate a wide range of known and constant flow rates, which are dispensed into the gauge collector to simulate different rainfall intensities. The water measured by the gauge during the test was then collected in a tank and weighed by means of a precision balance  to measure the total amount of water provided by the system. Assuming a constant value of the water density of 1000 kg mA dedicated software was developed to automatically control the flow rate generation and to measure the reference volume of water, whereas the DC rain gauge measurements were acquired using a dedicated data-logger.−1).Following the European standard EN 17277:2019 , all mearel), defined as:meas) is calculated by adopting the constant drop volumes provided by the manufacturers of each instrument, while the reference rainfall intensity (RIref) is provided by the calibration system. Calibration results for each test are then reported in the form of a non-parametric representation using boxplots, where the lower and upper dots represent the 5th and 95th percentiles, 80% of the data are contained within the two whiskers (10th and 90th percentiles), and half of the data are enclosed within the grey box. The horizontal thin and bold lines indicate the median and the mean of each sample, respectively.The instrument performance was evaluated using the percentage relative error  for each event measured by the three investigated DC gauges starting from the number of drops measured every minute. A variable drop volume was adopted, calculated as a function of the drop frequency recorded by the gauge.The calibration curves obtained from the laboratory tests were adopted to calculate the corrected RI values (RIdiff) between the two approaches was calculated as follow:With the objective to quantify the effect induced by the constant drop volume assumption, the RI relative difference , beyond which it abruptly decreases and the relationship between RIref and the drop frequency becomes undefined. This indicates that the generated drop volume is not constant, and the mean volume of the drops is significantly larger than the declared one. Beyond 300 mm h−1, the performance of the DC gauge abruptly changes, indicating that the operational limit of the rain gauge is reached  RI is not known apriori, the use of a co-located traditional rain gauge  is mandatory for any operational application of this type of gauge.rele|≤ 5%) for most of the operational range. An even worst result is obtained for the CRAL gauge (rele larger than 10% (class C of the European standard) for RI lower than 10 mm h−1. Both gauges overestimate precipitation for low RI values, with positive relative errors, then rele decreases with increasing RI, until reaching a minimum underestimation value (rele < 0). Beyond this, the error starts growing again until the instrument operational limit is reached, where both gauges overestimate the actual precipitation (rele > 0).Dynamic calibration results a, assumiAL gauge b, which −1). In this case, due to the very low RI value and the small number of generated drops, the sampling error contributes to the variability of the measurement and therefore degrades the resulting precision.The width of the boxplots reported in DV) against the measured drop frequency (DF) and interpolating the experimental data with a third-order polynomial  of the European standard and meet the WMO recommendations for most part of the operational range (−1).By using the actual volume of the drops, expressed as a function of the measured drop frequency according to the calibration curves, to calculate the RI, the performance of the two instruments falls within the limits of the Class A  of the selected events . This isAlthough this behavior is evident at the two examined field test sites, it appears more prominent at the CFARR site, where a CRAL gauge is employed and the total overestimation of RA ranges between 15.1 and 17.7% . At the RA) at the event scale.As shown in In DV approaches and classified according to the mean RI obtained using a variable drop volume. Relative RA differences are calculated per each rainfall event as follows:const is the total amount obtained under the constant drop volume assumption and RAvar is the total amount obtained by letting the drop volume to vary with the rainfall intensity. The variability of RAdiff over the whole set of events analyzed is reported in the form of boxplots in mean.The rainfall events measured at the two field test sites were interpreted using the constant and variable Dynamic calibration analysis of the two DC gauges showed that their performance does not fulfil the WMO recommendations when the constant drop volume calibration suggested by the manufacturers is adopted. Indeed, for some RI values within the declared measurement range, the relative percentage error exceeds 10%, a limit that excludes the gauges even from the low performance class of the recent European standard EN 17277:2019 [rele < 3%). However, the instrument performance in the field may be lower than those observed in the laboratory, due to errors induced by the atmospheric conditions, installation, status of maintenance, etc.The performed laboratory tests showed that the volume of the drops generated by the instrument varies as a function of the drop frequency, and therefore of the measured RI. For both the investigated DC gauge models, a relationship between the drop volume and drop frequency was derived. Using a post-processing adjustment algorithm, based on the correction curves obtained from dynamic calibration in the laboratory, the precision of the instruments was improved, and the obtained results are fully compatible with the WMO required maximum admissible error . Also, tmean). The largest overestimation is observed at the lowest RImean classes, then overestimation decreases as the RImean increases and underestimation occurs at high RImean. This reflects the fact that overestimation is evident in the calibration curve at the low to medium one-minute RI values, which typically characterize most part of any rainfall event, while high RI showers have a relatively short duration.When the overall bias of the total rainfall accumulation was investigated in the field, the three gauges exhibited a similar behavior upon varying the one-minute mean rainfall intensity (RImean than those at the CFARR site (The local climatology, therefore, plays a dominant role, since the gauges at the HKIA site a are subARR site b. LimiteARR site  with resARR site .Due to the internal drop generation mechanism, laboratory tests also revealed the operational limit of these instruments, given by the RI threshold at which the water flux from the internal nozzle starts to be irregular and stepwise continuous. This leads to the failing of a one-to-one relationship between the measured drop frequency and RI. For this reason, although these instruments are very useful to measure rainfall at low RI, a stand-alone installation is discouraged and a co-located rain gauge, using a different measuring principle, is required to avoid large underestimation of severe rainfall events.


Collaborative reasoning for knowledge-based visual question answering is challenging but vital and efficient in understanding the features of the images and questions. While previous methods jointly fuse all kinds of features by attention mechanism or use handcrafted rules to generate a layout for performing compositional reasoning, which lacks the process of visual reasoning and introduces a large number of parameters for predicting the correct answer. For conducting visual reasoning on all kinds of image–question pairs, in this paper, we propose a novel reasoning model of a question-guided tree structure with a knowledge base (QGTSKB) for addressing these problems. In addition, our model consists of four neural module networks: the attention model that locates attended regions based on the image features and question embeddings by attention mechanism, the gated reasoning model that forgets and updates the fused features, the fusion reasoning model that mines high-level semantics of the attended visual features and knowledge base and knowledge-based fact model that makes up for the lack of visual and textual information with external knowledge. Therefore, our model performs visual analysis and reasoning based on tree structures, knowledge base and four neural module networks. Experimental results show that our model achieves superior performance over existing methods on the VQA v2.0 and CLVER dataset, and visual reasoning experiments prove the interpretability of the model. Visual question answering (VQA) is an intersecting field of computer vision and natural language processing, which has just been proposed in recent years. The meaning of visual question answering is to fully understand the features of images and texts and make the appropriate decision. In the field of visual question answering, visual reasoning is a very difficult but extremely important problem to be solved. However, most of the existing methods ,2,3,4,5 The layout generated by these subtasks is considered as a reasoning map for predicting correct answers. However, the disadvantage of these manually pre-defined subtasks is that the questions and images provide very few features, and thus they are short of some common sense knowledge relationship to fully understand the features in the process of reasoning.In this paper, we propose a reasoning model of a question-guided tree structure with a knowledge base (QGTSKB) for addressing these problems. The inherent textual structure of the question is parsed into a tree structure. As shown in In order to solve complex reasoning tasks comprehensively and effectively, we propose four modular networks with different functions, which are the attention module, gated reasoning module, fusion reasoning module and knowledge-based relation module. The attention model not only uses the word encodings of the current tree node, but also fuses the attention map of the child node with the relationship between words from the knowledge base to extract local visual evidence for explicit reasoning. The gated reasoning model is inspired by Gated Recurrent Neural Networks. The advantage of the gated reasoning model is that the information flow can be modulated according to different tasks without a separate memory cell, which can reduce the memory consumption during calculation. Since the gating reasoning model has the function of forgetting and updating, it can delete and update the features passed by the child nodes according to the features of the current node. The fusion model combines the attention features of the current node and the features of the child nodes, which uses bilinear fusion to increase the variability of the fused features so as to enhance the flexibility of the model. The combination of external knowledge and visual information helps humans answer various questions. Therefore, the knowledge base in natural language processing has been gradually paid more attention.The knowledge-based facts are extracted from the three commonly used knowledge base datasets which are DBPedia , WebChilWe conduct extensive experiments to prove that our model has superior performance on CLVER and VQA2.0 public dataset. In addition, the attention map of each node is considered as a qualitative experimental result in the process of explicit visual reasoning, which further shows that our model is interpretable and has strong adaptability to different tasks.The main contributions are summarized as follows:We propose an interpretable and practical VQA model which unifies the visual, textual, knowledge-based factual representations from different modalities. Given the question-guided parsing tree structure, it performs remarkable reasoning based on the attention mechanism with the help of a knowledge base.We propose a tree-structure reasoning model based on modular network structure, which has four notable advantages: First, the modular network effectively mines and merges features of different modalities and implements parallel reasoning over tree nodes for logical reasoning. Second, the knowledge base provides some common knowledge other than images and texts to facilitate collaboration and communication between child nodes and parent nodes. Third, the attention mechanism has good interpretability of clearly performing explicit logical reasoning, and the gated reasoning model can tailor and update features.The proposed model has achieved excellent results on benchmark datasets, including VQA2.0, CLVER and FVQA, which indicates that our model has strong adaptability and versatility for different tasks. Through ablative experiments, it can be seen that the contribution of each module network to improving the overall performance of the model.The structure of the paper is organized as follows: In Visual question answering (VQA) is an intersecting topic that has emerged in the field of computer vision and natural language processing in recent years. Given the image–question pair, the VQA model needs to infer the correct answer in natural language according to the visual information provided by the image, which requires in-depth mining of the characteristics of the image and the question itself, and a full understanding of the fusion features of the image and text. Previous common VQA solutions used CNN-RNN-based ,16 archiMost of the above reasoning models focus on images and questions, ignoring the external knowledge base in the field of natural language processing. Moreover, the optimal reasoning structure can adapt to different types of questions, so it is appropriate to have inference structures that are specific to the input question. According to the structure of the text itself, our model uses a neural modular network and attention mechanism for visual reasoningA knowledge base (KB) stores complex structured fact entries in some hierarchical type such as tuple form , where S means subjects, R means the relationship between S and T, and T means targets. The application of knowledge base in visual question answering has attracted increasing attention. Wang et al.  proposedNeural module networks (NMN) play a significant role in generating layouts that constructs deep networks with flexible computational expressions for addressing compositional visual reasoning. Instead of adopting a fixed network structure, this method assembles a task-specific network from a set of predefined sub-models according to various tasks. Hu et al.  proposedWe propose a model of a question-guided tree structure with a knowledge base (QGTSKB) for explicit reasoning in visual question answering tasks. The overall structure of the proposed model is shown in iatt and the summed knowledge-based reasoning feature iv. The knowledge-based fact model , where v represents visual features, q represents question embedding, T denotes the nodes in the tree structure, which contains the word embedding i(i=w:N)1 from the question, and K denotes knowledge-based supporting facts in the form K = .As shown in As shown in v are extracted from the conv4 features of the ResNet-101  denotes concatenation operation.In this section, we introduce the internal structure of the gated reasoning model and fusion reasoning model. As shown in zf generates fusion reasoning features iz based on the attended visual features iv and the summed knowledge-based reasoning feature w4, w5, w6, w7 are learned parameters and As shown in i, ik is the concatenation of supporting facts and the question. The supporting fact ik  can be obtained by extracting the entity in the question and querying the entity in the knowledge base. For example, given the question ”What color is the hat of the girl riding a horse?”, the supporting fact  indicates the spatial relationship between the hat and the girl and the supporting fact  presents the size difference between a girl and a horse. The supporting facts provide sufficient clues to locate the related objects for answering questions and logical reasoning.The knowledge base compensates for the shortage of features of images and texts by providing facts related to the entities in the question. In the leaf node ik and the fusion reasoning feature iz go through the fully connected layer and the activation layer (ReLU) which performs spatial transformation and obtains the same dimension, respectively. The knowledge-based reasoning feature w8, w9 are learned parameters.As shown in w10, w11 are learned parameters.Traversing each node of the tree from bottom to top, our model obtains the final result M represents the batch size and N represents the number of candidate answers. Since there is only one candidate labeled as the ground-truth answer and the rest of the candidates are annotated as negative answers in each training batch, it is efficient to adopt binary cross-entropy loss  to deal CLVER  is a synThe VQA v2.0  is a widThe FVQA dataset contains 193,499 facts corresponding to the questions. The knowledge base is constructed by querying top visual concepts from three knowledge bases: DBPedia, ConceptNet and WebChild.In our experiment on the CLVER dataset, each image is resized to 224 × 224. Furthermore, we extract the 1024 × 14 × 14 feature maps by forwarding the 3 × 224 × 224 resized image through the res4b22 layer of the Resnet-101 with the kernel size of  and  trained on the ImageNet classification. For improving spatial reasoning, we connect two coordinates beddings . The seqDuring supervised training, our model is trained by Adam optimizer  with 12 We evaluate our model and the previous model on each question type in the CLVER dataset. As shown in We compare our model with previous methods on the VQA v2.0 in In this section, we introduce the process of visual reasoning based on the tree structure parsed from the question. The CLVER dataset is generated by codes for reasoning. Thus, the knowledge base provides limited supporting facts for the nouns in the question. As shown in We also evaluate the interpretability of the model on the VQA v2.0 dataset which contains open-ended questions and realistic images. Questions require an understanding of features of images, questions and commonsense knowledge to answer. As shown in In this paper, we propose a reasoning model of a question-guided tree structure with a knowledge base (QGTSKB) for visual question answering. To the best of our knowledge, this is the first visual reasoning model that leverages the combination of the entire language structure, visual features and knowledge base. Our model can automatically perform interpretable visual reasoning over a parsing tree structure from the question, which has strong flexibility and universality of adjusting the structure according to different types of questions. Compared with previous methods, it does not rely on annotations or manual rules to set the network layout. The knowledge base provides supporting facts making up for the lack of the original visual features and question features for reasoning. The neural modular network simplifies a single large-scale into small and manageable modules. Each module of the neural module network has specific functions and is independent of other modules. Due to parameter sharing, a modular neural network reduces computation cost and improves the efficiency of the model. Our model obtains excellent results and performs visual logical reasoning on the VQA v2.0 and CLVER dataset.Our model uses convolutional neural networks to extract visual features of objects as representations of images. However, it cannot accurately capture the relationship between objects. We are going to adopt the method of graph network to model the fully connected graph according to the relationship of objects in the image, in which the objects in the image are represented as nodes of the image and the relationship between objects is represented as edges of the image. How to deeply mine object relationships in images to establish a graph neural network for reasoning on visual question answering is essential.


Predicting the travel demand plays an indispensable role in urban transportation planning. Data collection methods for estimating the origin–destination (OD) demand matrix are being extensively shifted from traditional survey techniques to the pre-collected data from intelligent transportation systems (ITSs). This shift is partly due to the high cost of conducting traditional surveys and partly due to the diversity of scattered data produced by ITSs and the opportunity to derive extra benefits out of this big data. This study attempts to predict the OD matrix of Tehran metropolis using a set of ITS data, including the data extracted from automatic number plate recognition (ANPR) cameras, smart fare cards, loop detectors at intersections, global positioning systems (GPS) of navigation software, socio-economic and demographic characteristics as well as land-use features of zones. For this purpose, five models based on machine learning (ML) techniques are developed for training and test. In evaluating the performance of the models, the statistical methods show that the convolutional neural network (CNN) leads to the best performance in terms of accuracy in predicting the OD matrix and has the lowest error in terms of root mean square error (RMSE) and mean absolute percentage error (MAPE). Moreover, the predicted OD matrix was structurally compared with the ground truth matrix, and the CNN model also shows the highest structural similarity with the ground truth OD matrix in the presented case. The development propagation of cities and transportation infrastructures together with population growth have produced a wide variety of travels with different purposes at different times. Population growth and the spatial dispersal of physical developments in cities have created new centers that generate intra-city trips. The generation of new trips in new areas along with increased travels in other areas has exacerbated the problem of traffic congestion in cities. Therefore, predicting the travel demand plays a vital role in urban transportation planning.The prepared travel behavior questionnaires and annual censuses are the cornerstones of many studies and transport models ,2. The tThe data collected from ITSs are becoming increasingly complex and have the characteristics of big data. Large companies such as Microsoft and IBM classify the data with the three characteristics of volume, variety, and velocity as big data. Working with this big data and taking advantage of machine learning (ML) techniques provides novel solutions to solve transportation problems .The methods of data collection for estimating the OD demand matrix are being shifted from traditional survey techniques  to pre-collected data stored by ITSs, e.g., data obtained from GPS, automatic number plate recognition (ANPR) cameras, smart fare card of the public transportation system, inductive loop detectors, etc. These data sources have greater temporal and spatial coverage than origin–destination survey data. Additionally, the samples collected from these data have larger dimensions . In addiIt should be noted that there are several methods for comparing the predicted OD matrix with a ground truth OD matrix. These methods are mostly designed based on the statistical comparison between the value of each cell of the matrix, such as root mean square error (RMSE)  and meanThis paper mainly aims to develop a model using ML techniques and the data collected from ITSs to predict trips between traffic analysis zones (TAZs) of metropolitan areas. The travel demand matrix, obtained from comprehensive urban transportation studies (CUTSs), is used as the ground truth OD matrix to test and validate the presented model. The proposed model uses big data sets collected from ITSs such as ANPR camera data, Loop detector count data at intersections, smart fare card data, GPS data of a navigation application, etc. In this study, the development of the model is conducted in the Tehran metropolis as a case study. This paper further intends to train, evaluate, and compare different ML methods for predicting the hourly OD demand matrix. In this study, different ML models are evaluated based on the prediction accuracy of test data. Furthermore, the structural similarity of the matrix estimated by ML methods based on the ground truth OD matrix is also measured. It should be mentioned that although previous studies tried to solve some parts of the OD matrix estimation problem with ML, none of them used ML methods to solve the whole problem entirely. The novelty of this paper is to consider big data sources and ML techniques to solve the whole problem of OD estimation, as well as evaluating the results by both the statistical indicators and structural comparison with the ground truth. In the following, the literature regarding the prediction of OD matrices and various techniques are addressed. The data description for developing models is then explained, and consequently, the detailed development processes and methodology are explained. Statistical and structural comparisons of the proposed models followed by conclusions are discussed in the remaining sections of this paper.Estimating the OD demand matrix has always possessed a fundamental problem in transportation engineering. Although various methods have been presented to solve this problem, ML and deep learning-based methods have also been implemented to solve various transportation problems in terms of big data collected by ITSs. This section reviews the methods for estimating the OD matrix, also, ML and deep learning-based methods to solve problems related to transportation.Many studies have dealt with the estimation of static OD matrices. The static OD matrix generally represents travel demand in a proposed region for a given period of time . Many stAs mentioned earlier, most of the methods use classical models based on statistical approaches to estimate the OD demand matrix. However, innovative methods based on artificial intelligence, such as artificial neural networks in deep learning, have emerged as novel alternative methods to the previous ones. These methods are discussed in the following.ML techniques are usually acting as the mastermind of a system. In recent years, deep learning approaches created a significant breakthrough in various fields such as computer vision, speed recognition, and natural language processing. These methods have broken accuracy records in several areas. ML and deep learning methods have been used in various areas in the field of transportation planning, such as traffic flow prediction ,34,35,36ML techniques related to travel choice models have been used for a long time (since the 1990s) . Since tThis study attempts to evaluate the potential use of different ML methods to predict hourly OD demand matrix by complementing previous studies. Using pre-collected big data from ITSs, the ability of various basic and developed ML techniques to predict a demand matrix is evaluated, and finally, the best model to predict dynamic OD demand matrix is presented.The data collected for this study can be classified into six categories: (1) traffic analysis zones (TAZs) of Tehran along with spatial, demographic, socio-economic, and land-use characteristics, (2) automatic number plate recognition (ANPR) camera data, (3) Loop detector counts at intersections data, (4) smart fare cards data, (5) GPS of navigation software data, and (6) OD matrix from comprehensive urban transportation studies (CUTS) in Tehran. All of them are stored completely without duplicate data. It should be noted that the data reliability, means the data completeness and accuracy, is tested.The characteristics of the data and the preparation process of the model are described in the following sections. It should be mentioned that the data for categories 2 to 5 were collected over a 6-month .The metropolis of Tehran is the capital and most populous city of Iran, with a population of 8.7 million and an area of 1200 square kilometers . Tehran 9 matrix .Studies have shown that the TAZs characteristics can estimate the number of trips that occur between OD pairs . TherefoThe metropolis of Tehran has been struggling with traffic congestion issues for a long time in the Central Business District (CBD) of the city. With the population growth and the increasing willingness of people to use private vehicles, this problem has been manifesting more seriously in recent years. Transportation and traffic experts in Tehran have designed and established traffic monitoring systems in the CBD of the city to apply demand management policies such as congestion pricing. Traffic demand restriction policies are applied in two layers in the CBD of Tehran. As shown in Many intersections in the metropolis of Tehran are timed by Sydney Coordinated Adaptive Traffic System (SCATS). SCATS is an adaptive real-time traffic signal control system that was first implemented by the Road and Traffic Authority (RTA) in Sydney in the early 1970s . The sysTo facilitate mobility and increase the optimal use of public transportation systems, smart cards are used in many cities for paying travel costs . The metToday, citizens are equipped with extensive and highly diverse portable gadgets such as smartphones, laptops, and GPS-based navigation devices that enable researchers to track their owner locations continuously. These activities can reveal travel patterns and the daily behaviors of people. Presently, researchers use these data to extract valuable information from movement patterns in cities . Neshan The comprehensive urban transportation studies (CUTS) of Tehran determined the observed OD matrix in this study. This matrix was designed on a 24-h basis and calculated based on the latest national household travel survey conducted in 2014. The OD demand matrix of citizens in the study year (2019) was calculated using activity-based models in CUTS and used as ground truth in this study. This matrix represents person trips on the network. As shown in The development of an efficient model significantly depends on the selection of parameters that affect the model output. Choosing the best combination of input data can significantly influence the accuracy of the final estimation of the model. Thus, the ANalysis Of VAriance (ANOVA) method is considered for finding the proper model independent variables . The depb. Therefore, when the adjusted In this ANOVA test, due to the large volume of input data and the inability of the personal computers (PCs) to analyze the multiple interactions between the data, only the direct relations between the input data were considered, and the 2-way and 3-way interactions were ignored. When the adjusted According to ML techniques are basically divided into two categories: supervised methods and unsupervised methods. In supervised learning, the data intended to be predicted is also included in the model’s input data. However, there is no prior estimation of the model output in the unsupervised method, and the model is expected to determine outputs itself. As we have the ground truth, we have trained our model based on it, so, this study focused on supervised ML methods. The steps of this study are shown in According to K-nearest neighbor (KNN) regression is one of the simplest ML methods. The prediction based on the observed value of KNN is performed using the distance parameter  in the multi-dimensional space of the variables. This value can be mean, median, maximum or any other statistical parameter obtained from KNN. In this study, the Scikit-Learn library in Python was used . In thisThe random forest (RF) regression method was designed based on a set of random decision trees . A decisThis framework is a fast and high-performance gradient boosting mechanism based on decision tree algorithms. It is used for ranking, classification, and many other tasks in ML. This efficient algorithm was designed for distributed applications, with the following advantages: high training speed, more efficiency, less memory usage, better accuracy, and large-scale data management capability . In thisThere are various approaches based on artificial neural network algorithms, and the MLP is one of the most common ones. This method matches the input variable layers to target variable layers. In theory, there can be an infinite number of hidden layers between the two layers. Each hidden layer contains a certain number of nodes or neurons that the user should determine. The artificial neural network allows training a highly nonlinear model; however, the initial selection of weights in the layers puts the models at risk of becoming stuck in local minima. Additional details on these models can be found in previous studies ,31,36,42The convolutional neural network (CNN) is a deep learning algorithm that receives an input image and assigns importance (learnable weights and biases) to each of the objects/aspects in the image and can distinguish them from each other. The CNN algorithm requires less pre-processing than other classification algorithms. Although the filters of basic methods are performed manually, CNN acquires the ability to learn filters/features after enough training. CNN can successfully capture temporal and spatial dependencies to an image with appropriate filters. Moreover, it performs better filter architecture on image data set due to the reduced number of parameters involved and reuse of weights. In other words, the network can be better trained to learn complex images. The structure of an OD matrix can be viewed as an image where each matrix cell represents an image pixel. In summary, each CNN consists of three layers: convolutional layer, pooling layer, and fully connected layer. For more details, refer to previous studies on this topic ,38,39,45lemented .The data set examined in this study consists of 11,726,424 hourly origin–destination records. Each record includes the number of daily trips between an origin and destination every hour. The model’s dependent variable is the OD matrix of CUTS, and the 21 variables listed in As shown in Statistical indicators, such as root mean square error (RMSE) and mean absolute percentage error (MAPE), were used to evaluate the performance of models. In addition, to evaluate the possibility of overfitting in the models, the As shown in As shown in In addition to statistical indicators (such as RMSE and MAPE) for comparing the predicted and ground truth OD matrices, other methods have also been developed to understand the OD matrices’ structural similarity/dissimilarity. In these methods, in addition to numerical comparison of cells in two matrices , the distribution of travels from different origins to different destinations or, in other words, the structure of the two matrices is also compared. One of these methods is the mean structural similarity index method (MSSIM) . AccordiAccording to Among the developed models, the best results belong to the CNN and MLP models due to higher Comparing the models developed based on the statistical indicators, the CNN model has higher accuracy in estimating the OD matrix of trips for the test data according to the RMSE and MAPE values. The CNN model error is 10% less than the MLP model and 16.5% less than the LightGBM model. The superiority of the CNN model over the rudimentary ML models such as KNN was distinctly observed. The MLP model again ranked second in this comparison.The structural comparison of two matrices reveals new aspects of similarity or difference between the matrices. In addition to the statistical indicators, this study compared the matrices structurally predicted by the ML models with the ground truth OD matrix. For this purpose, the MSSIM index was used to calculate the structural similarity between the two matrices. The highest structural similarity in the developed models was observed between the OD matrix of the CNN model and the ground truth OD matrix.In this research, the convolutional neural network (CNN) is investigated as the most accurate model for estimating the origin–destination demand matrix in the Tehran metropolis. This study used the big data analysis concept based on the gathered data of ITS sources to estimate the hourly (dynamic) OD matrix of trips between TAZs. Data used in this study includes traffic flow in zones by ANPR cameras, entries to and exits from subway stations by SFCs, Loop detector traffic count in intersections by SCATS, the OD matrix from GPS data of Neshan navigation software, and characteristics of TAZs. The diversity and the large volume of usable data made ML methods useful to determine the most efficient model for predicting the hourly OD matrix of trips. In this regard, five different ML models (from primary to advanced models based on deep learning techniques) were trained on a target OD matrix in terms of person trip, adapted from CUTS in Tehran. The results of the development of the mentioned models are summarized as follows:Accordingly, it could be concluded that neural networks have an undiscussable superiority in predicting the OD demand matrix among various ML techniques. Among the neural networks developed in previous studies, according to the structural nature of the convolutional neural network and the results of this study, these models make the most accurate predictions. This study shows that the CNN model provides the best results numerically  and structurally to estimate the OD matrix.It should be noted that ML models act as a black box, and thus, no accurate interpretation of the relationship between input variables and output data can be extracted. Therefore, the interpretation of models under varying conditions is always prone to difficulties. As the process of data collection and processing becomes apparent to transportation experts, modelers may be able to use new techniques to analyze the data. Furthermore, completing additional information  along with these data may allow for better analysis.


To properly interact with our environment, the brain must be able to identify external stimuli, process them, and make the right decisions all in a short time. This may involve several brain regions interacting together by sharing information birectionally via rhythmic activity. Such flexibility requires the functional connectivity between the areas to be dynamic, and a key question is the relevant parameter and operating regimes that make this possible in spite of fixed structural connectivity. Working towards this goal, we consider two coupled brain regions, each of which exhibits a noisy rhythm, a commonly observed type of neural activity. Such rhythms can be induced by the stochasticity in the neural circuitry, or be autonomously generated through nonlinearities and not necessitating noise. For these two types of rhythms, we computed the amount of information shared between the brain areas and the preferred direction(s) of sharing. We found that without the coupling delay, the flexibility needed by the brain to perform cognitive tasks requires the rhythms to be autogenerated rather than noise-induced. This is the case even with asymmetry or heterogeneity. This suggests that the importance of the dynamical regime has to be taken into account when modeling interacting neural rhythms from an information theoretical point of view.Brain areas must be able to interact and share information in a time-varying, dynamic manner on a fast timescale. Such flexibility in information sharing has been linked to the synchronization of rhythm phases between areas. One definition of flexibility is the number of local maxima in the delayed mutual information curve between two connected areas. However, the precise relationship between phase synchronization and information sharing is not clear, nor is the flexibility in the face of the fixed structural connectivity and noise. Here, we consider two coupled oscillatory excitatory-inhibitory networks connected through zero-delay excitatory connections, each of which mimics a rhythmic brain area. We numerically compute phase-locking and delayed mutual information between the phases of excitatory local field potential (LFPs) of the two networks, which measures the shared information and its direction. The flexibility in information sharing is shown to depend on the dynamical origin of oscillations, and its properties in different regimes are found to persist in the presence of asymmetry in the connectivity as well as system heterogeneity. For coupled noise-induced rhythms (quasi-cycles), phase synchronization is robust even in the presence of asymmetry and heterogeneity. However, they do not show flexibility, in contrast to noise-perturbed rhythms (noisy limit cycles), which are shown here to exhibit two local information maxima, i.e., flexibility. For quasi-cycles, phase difference and information measures for the envelope-phase dynamics obtained from previous analytical work using the Stochastic Averaging Method (SAM) are found to be in good qualitative agreement with those obtained from the original dynamics. The relation between phase synchronization and communication patterns is not trivial, particularly in the noisy limit cycle regime. There, complex patterns of information sharing can be observed for a single value of the phase difference. The mechanisms reported here can be extended to I-I networks since their phase synchronizations are similar. Our results set the stage for investigating information sharing between several connected noisy rhythms in neural and other complex biological networks. The brain is a complex biological system composed of many subsystems. To perform important tasks such as perception, cognition and behaviour, its subsystems must coordinate information in a dynamic and flexible manner. Rhythmic activity is found in several brain areas such as the primary visual cortex ,2 and obFrom a dynamic point of view, two distinct mechanisms can give rise to oscillations. One is the self-sustained periodic rhythm  with a cThe basic stochastic nature of quasi-cycles may limit their ability to synchronize with other brain areas displaying rhythms. Alternately, it may endow them with more flexibility compared to noisy limit cycles. We will see below that the issue of flexibility is a subtle one, even if the connection delays between two individually rhythmic subsystems are negligible as assumed in our work. In fact, the shape of the delayed mutual information function (dMI—defined below) between two gamma rhythm-generating subsystems, including the number, location and heights of its peaks, will be shown below to critically depend on the origin of the rhythms, the degree of heterogeneity of the networks and the asymmetry of their coupling.The synchronization of gamma oscillations has been mostly studied using noise-perturbed limit cycles, for which an analysis in terms of phase synchronization is effective . Phase si-cycles .The sign of the mean phase difference value signals the leadership between the two oscillators. Outof-phase phase synchronization between two identical and symmetrically coupled limit cycle oscillators may occur due to spontaneous symmetry breaking beyond a certain critical parameter ,22. WhicIn contrast, phase synchronization has been much less studied for quasi-cycles in spite of the obvious relevance ,11,12. OAdditionally, it is still not clear what is the precise relationship between phase difference dynamics and information flexibility. Can the pattern of information sharing or communication be inferred from the pattern of phase synchronization? Do noisy limit cycles differ in important ways from quasi-cycles in terms of flexible communication, and if so, what are the critical parameters and essential ingredients? In particular, how is the pattern of information sharing shaped by biophysically more realistic network features such as connection asymmetry or heterogeneity of the subsystems? These questions motivate our work. They are addressed using two subsystems, each exhibiting its own rhythm via the Pyramidal Interneuron Network Gamma (PING) mechanism. The two subsystems representing neighboring—and perhaps co-localized—brain areas are coupled through long-range excitatory connections. We study information sharing between these subsystems by computing the dMI between their phases ,28. The Zero-delay-coupled quasi-cycles can exhibit robust phase-synchronization even in more realistic cases where asymmetry and heterogeneity are considered;The system of two coupled quasi-cycles oscillations does not show dynamic connectivity. The information is predominantly shared from one network to the other in the asymmetric inter-areal connectivity case and in the heterogeneous population case. Therefore it lacks flexibility in information sharing;When the system is in the noisy limit cycle regime, we may observe dynamic connectivity highlighted by the presence of two out-of-phase locking states for a single and fixed value of the inter-areal coupling. Information can then be shared from one network to the other and vice versa: there is a flexibility in information sharing. Such flexibility persists in the presence of asymmetry and heterogeneity but with some bias.We first present the model, then the statistics used to quantify the phase and mutual information. Results are shown first for the quasi-cycles; these results are then confirmed by analyzing the phase differences and flexibility using a recently proposed envelope-phase description of the coupled quasi-cycle rhythms. Then follow our findings for the coupled noisy limit cycles. For each of these dynamical regimes, we investigate the effect of asymmetry in the anatomical coupling and heterogeneity in each population. The paper ends with a discussion and outlook onto future work. The results can be summarized as follows:i arrives instantaneously at a neuron of the network j with i and the E population of network j denoted We consider a system composed of two E-I networks  connectefined by :(2)g1(EiThe Gaussian white noises have the following properties:The dynamics of a single isolated stochastic Wilson–Cowan network is given by Equations  and 2)  2) withoλ=−ν±jω0 a, with i)<100Hz) b. WithouSynchronization level: To quantify the ability of the model to generate sustained and coherent oscillations, we compute the “synchronization level” as the mean of the differences between the peak and the trough during each period over a very long simulation of the excitatory activity E(t) see a. In theQuasi-cycle regime: If quasi-cycle regime. Note that we choose to work with only the excitatory variable Noisy limit cycle regime: For noisy limit cycle regime. The time series Mean frequency; The parameters were chosen such that oscillation frequencies belong to the higher frequency gamma-band (30–100 Hz). To obtain the mean frequency, we first computed the autocorrelation function of the excitatory activity In our work, we make sure that when two stochastic Wilson–Cowan networks operating in the quasi-cycle regime prior coupling are coupled through long-range excitatory connections (quations  and 2)))LEEij≠0 H defined as:P signifies the Cauchy principal value. The envelope of the stochastic signal is then We are interested in the patterns of phase-locking and information sharing  between phase signals of the reciprocally coupled networks. We extracted the phases using the Hilbert transform. For example, the analytic signal corresponding to Phase difference distribution: Phase locking can be analyzed by first computing the difference in the phases of the time series for each network. The phase difference is given by:Phase cross-covariance function: In the quasi-cycle regime, oscillations appear as short epochs of transient synchrony called bursts. Due to this transient nature, an alternative and more appropriate way to compute the phase-locking is to use the cross-covariance function between the phase signals defined as:t denotes a time average, and s varies from T the mean period of the oscillation. The advantage of using the cross-covariance is that it takes into account the transient nature of the signals and therefore efficiently captures the correct phase-difference values. Using the cross-covariance function returns the most probable time-lag between the two signals. We convert this time-lag into a phase value using the mean period of the signal. The phase differences to which the system locks are then given by:bsystems ,26,30. IAnother way to infer the “effective connectivity” between the networks is to compute the delayed mutual information (dMI) using the phase signal of each network :(9)dMI1,d) copy d, independent of how this information is encoded or decoded [It measures the amount of information shared and transferred between the phases  decoded . When th decoded ,28. For Flexibility in information sharing: dynamic effective connectivity: When the dMI curve exhibits a peak at both a positive and a negative location, the coupled networks reciprocally share information in both directions, in one direction at a time. The “effective connectivity” fluctuates in directionality from one epoch of time to another, and for the stochastic dynamics of interest here, these changes occur randomly, as described in [flexibility in information sharing, or a dynamic effective connectivity, which is likely a necessary property for bi-directional communication between brain areas. In this picture, the more positive and negative peak locations there are, the more flexible the information sharing is. Note that this is fundamentally different from information sharing with a peak at zero, which does not imply the same flexibility.ribed in . Such beAmount of information shared: The values of the peaks of the delayed mutual information describe the maximum amount of information shared between the networks. It also measures the level of phase synchronization between the networks. When there are several peaks at positive and negative locations, the system of coupled networks can still share information towards a dominant direction provided that one peak is higher than the other. The amplitudes of the dMI peaks will be studied in more depth elsewhere.Symmetric case: We first consider the symmetric case of two identical networks coupled with long-range excitatory connections  to zero a showing to zero b. Thus t to zero c, a sign to zero d, which  to zero b,c.Asymmetric-coupling case: We move to two identical networks but which are asymmetrically coupled. The coupling 1>LEE12) e. The meThe network with the higher frequency leads. The location of the dMI peak increases from negative to positive, meaning that information is preferentially shared from the second to the first network when the first network leads, and conversely when the second network leads f. The va studies ,26,30 bu studies ,24,26. IHeterogeneous case: Finally, we focus on the heterogeneous situation where the two networks are not identical. The inhibitory external input ituation i. The loituation j. The dMituation k. InformThus for coupled quasi-cycles without propagation delay, phase synchronization is robust even in the presence of asymmetry and heterogeneity. However, communication is not flexible and the effective connectivity is not dynamic. This means that for a fixed structural connectivity or phase relation  could be written in the form:The dynamics of the envelopes od (SAM) ,21. We cThis leads to the following envelope A complete and detailed review about the derivation of the envelope-phase dynamics Equation  can be fThe results obtained for the symmetric, asymmetric, and heterogeneous cases in Spontaneous symmetry breaking [To better understand the mechanism of information sharing reported for the case of noise-induced rhythms, we extend the previous analysis to the case of noisy limit-cycle oscillations. Parameters are such that, prior to coupling and without noise, each network lies in the limit cycle regime . We note that a recent study has also showed the presence of symmetric out-of-phase locking states in coupled noisy limit cycles oscillators . HoweverFor identical and symmetrically coupled quasi-cycle oscillations, there is no flexibility in the absence of any delay. This likely follows from the fact that delays have been shown to be crucial for the presence of multiple out-of-phase locking (OPL) states, and without them, the range of phase differences between the networks is smaller. The same holds in the more realistic cases where asymmetry or heterogeneity is included in the model.In contrast, for noisy limit cycle oscillations, flexibility is observed in the absence of coupling delay. Coupling delay is not necessary for dynamic changes in phase synchronization and flexible communication. However, the critical biophysical parameter responsible for dynamic out-of-phase locking observed with our parameter settings is not identified here. Flexible communication persists in more realistic situations where asymmetry or heterogeneity are included. This takes the form of two peaks in the dMI, and bistability in the phase difference.Surprisingly, we found that the pattern of information sharing can not always be inferred from the phase relation. Even a single value of the phase difference can lead to a complex pattern of communication with two routes see e,f. ThisWe also found that the relation between phase synchronization, structural coupling, frequency difference and information sharing depends on the specific coupling scenario and the dynamical regime of interest. More precisely, we showed that in the quasi-cycle regime, information could be shared from the laggard to the leader (in asymmetric coupling), in contrast with the noisy limit cycle regime where information was shared from the leader to the laggard. The relation between the phase difference and the frequency difference was similar in the quasi-cycle and noisy limit cycle regimes. In these cases, the network with the higher frequency was the leader, whereas the one with the lower frequency was the laggard. The fact that information could be shared from the phase laggard to the phase leader in quasi-cycles stands in clear contrast to previously known results on phase synchronization and information sharing between coupled limit cycle oscillators where the opposite was established ,24,30. TWe used numerical simulations to compute phase synchronization and the dMI between phase signals. Phases were extracted through the Hilbert transform and from previously derived envelope-phase equations (SAM) for quasi-cycles. The results from the Hilbert transform and the SAM equations were qualitatively in good agreement. However, we observed a quantitative discrepancy between the amount of information shared and the exact locations of the peaks of the dMI curves. These discrepancies may be associated with the absence of nonlinear terms to obtain the SAM dynamics. In fact, SAM dynamics are derived from the linearized dynamics of the former nonlinear stochastic equations. Additional information could be contained in the nonlinear terms which were neglected to obtain the SAM dynamics to the order shown here. We also found (not shown) that the results obtained when using only the phase obtained directly from the SAM dynamics were identical to those obtained when the phases were converted to Hilbert phases. This suggests that all the information is contained in the stochastic part of the Hilbert phases.Some results obtained here are in good agreement with previous results on flexible information transfer in noisy limit and quasi-cycles. These results allow us to gain a better insight into the mechanisms of flexible information sharing or dynamic effective connectivity between brain areas. However, complex biological, biochemical, genetic, mechanical, or chemical networks are often made up of more than two interacting subsystems. The results obtained here could be extended to such complex realistic networks. Furthermore, from a theoretical point of view, an extension of the envelope-phase dynamics to include nonlinearity and to cover limit cycle oscillations will be an important avenue for future analysis, along with the effect of external stimulation aimed at controlling the properties of the rhythms in isolation or under coupling.We have addressed the question of flexible information sharing through neural oscillations. How can two coupled brain areas dynamically exchange information of a fast timescale provided that the structural or anatomic connectivity between them remains fixed? We tackled this question by considering oscillatory brain areas that independently exhibit rhythms in the fast gamma band (30–100 Hz) prior to being coupled. The two networks are connected through a bidirectional long-range excitatory connection but without conduction delay. For each of these two dynamical regimes, we computed the phase-locking states and the delayed mutual information between excitatory signals of the two brain areas. Our minimal criterion for flexible information sharing was two peaks in the delayed mutual information curve and phase difference distribution. We found that when the system of the two coupled brain areas was in the quasi-cycle regime, flexibility in information sharing was not observed. However, when the networks were in the noisy limit cycle regime, we observed flexibility in information sharing, i.e., sharing in both directions but not at the same time. Clearly, the effective connectivity was dynamic such that during a short epoch of time, only one network sent the information to the other which is passive, and the reverse situation happens during another epoch of time. Therefore, the ability of zero-delay coupled brain areas to dynamically exchange information while keeping the structural coupling fixed depends on the operating regime of the system. Including a coupling delay could induce more complex phase-locking patterns as well as several routes of information sharing even in the quasi-cycle regime, a possibility that further studies could establish.


The complex interaction between rain, vegetation, soil, and climate can only be understood by studying form–function relationships of leaves in a meaningful ecological context. Water shedding from leaves is a complex process depending on multiple leaf traits interacting with rain, wind, and air humidity, and with the entire plant and surrounding vegetation. Here, we synthesize current knowledge of the physics of water shedding with implications for plant physiology and ecology. We argue that the drop retention angle is a more meaningful parameter to characterize the water-shedding capacity of leaves than the commonly measured static contact angle. The understanding of the mechanics of water shedding is largely derived from laboratory experiments on artificial rather than natural surfaces, often on individual aspects such as surface wettability or drop impacts. In contrast, field studies attempting to identify the adaptive value of leaf traits linked to water shedding are largely correlative in nature, with inconclusive results. We make a strong case for taking the hypothesis-driven experimental approach of biomechanical laboratory studies into a real-world field setting to gain a comprehensive understanding of leaf water shedding in a whole-plant ecological and evolutionary context. In times of global change, with increasing frequency of high-intensity rainfall events  aggravatThe effects of surface water retention on plants are, however, strongly context dependent. On the one hand, persistent wetness on leaves can impede transpiration and photosynthesis ; howeverIn general, it appears that temporary water cover on leaf surfaces can have benefits, but long-term wetness tends to be disadvantageous. Therefore, plants have ubiquitously evolved adaptations to promote water shedding from their leaves. These can be simplified into two general mechanisms: (i) increased water repellency of the leaf surface and (ii) steeper leaf inclination angle. In the following, we will explore both strategies in detail and discuss their interaction with each other as well as trade-offs with other leaf functions, and implications for leaf ecology and evolution. We will first consider the (simpler) case of a droplet or water layer on a static leaf, as might occur after rain or as a result of condensation, before exploring the more complex effects of drop impacts during rain. Finally, we will take a look at some specialized adaptations such as the ‘drip tips’ on the leaves of a diversity of tropical species, and anisotropic surface structures promoting directional water transport.Two factors dictate how easily water is shed from a leaf: (i) the water repellency and (ii) the inclination of the surface. In brief, the more easily a drop can move and the steeper the surface, the more easily water will be shed. Both factors are intuitively captured in the drop retention angle—the angle at which a drop starts to roll off a surface when the surface is gradually tilted , D. NoteAs natural leaves are never perfectly flat and smooth, their wettability is always based on a combination of surface chemistry and topography . SurfaceTraditionally, most studies on biological surfaces report static contact angles; however, the above examples illustrate that this may not be the best variable to explain water shedding from leaves. This is further aggravated because biological surfaces tend to be inhomogeneous in texture and chemistry . Even onSo far, we have considered the relatively simple case of a sessile drop on a static leaf. However, rain drops hit the leaf with impact forces of >1000 times their static mass . Recent We already showed for the static case that the inclination angle determines the transition between drop retention and run-off for a surface of a given water repellency. For the dynamic case, it has been shown that the wettability strongly influences the transition points between adhesion, bouncing, and splashing. For hydrophobic leaves, higher inclination angles broaden the range of drop sizes and impact velocities at which bouncing occurs; that is, the steeper the leaf inclination, the less likely the drop will stick or splash ; Fig. 3BNarrow elongated leaves and extensions to the leaf apex—so-called ‘drip tips’ ; Fig. 2HUp to here, we have ignored the fact that leaves can move. This includes both slow deformations as a result of loading, and rapid impact responses on a subsecond time scale. Upon impact, rain drops transfer momentum to the leaf, leading to a deformation of the leaf itself and/or the supporting petiole. The transfer of energy depends on the impact rate and location, the size and speed of the drop (which is influenced by wind and surrounding vegetation), and whether the drop bounces or spreads on the leaf. Drop behaviour on the surface in turn depends on drop size, speed, and impact angle, and on the leaf’s surface properties, size, rigidity, and angle of inclination, as well as presence or absence of water already on the surface. If part of the impacting water remains on the surface, the added mass will change the inclination angle, with the amount of change depending on the biomechanical properties of the leaf  leaves favour water movement along the leaf blade (Lolium sp.) leaves result in a 3-fold lower drop retention angle in the proximal than in the distal direction along the leaf  can curl up under wind load, thereby reducing drag and fluttering. This not only reduces the total exposed surface area but also mainly exposes the more hydrophobic abaxial surface . This could provide a mechanism to shift harmful UV radiation into the visible spectrum, thereby increasing light availability for photosynthesis. Synergy effects are likely between water repellency and protective functions against herbivore and pathogen attacks. Epicuticular wax crystals not only impede drop adhesion but equally prevent insects from attaching to the surface  leaves; however, both climate parameters were also intercorrelated, making further interpretations difficult. The authors suggest that drip tips might be selected against in drier climates where retaining nightly accumulating dew on the leaves may reduce transpirational water loss in the morning.A few studies provide direct evidence linking the prevalence of drip tips to wetter habitats , I. MengWe already saw that inferring functional or evolutionary relationships from the presence or absence of certain leaf traits in different (micro-) habitats can be problematic for various reasons. Maybe the most imminent problem for such survey-based studies is that virtually all of the traits we considered in the context of water shedding are highly variable not only between individuals of the same species, but also between leaves of the same individual, and even within the same leaf over time. Epicuticular wax crystals can be reduced or absent in plants growing under low light conditions  or underDrip tips are both more common and more pronounced in saplings than in mature trees of the same species . How driBy far the most plastic of the leaf traits considered in this review is leaf inclination angle which can be highly variable depending on leaf and plant age ; Fig. 2GMimosa pudica touch-me-not plants  with biomechanical measurements  and quantification of whole-plant fitness correlates  in a natural field setting. While this approach is certainly challenging, we have now, for the first time, the necessary technology to take research on water shedding to the next level (