Unhealthy lifestyles may hinder bariatric surgery outcomes. This non-randomized controlled study aimed to evaluate the effects of an integrated post-operative exercise-based educational and motivational program in improving behaviors, quality of life, anthropometric features, cardiorespiratory and physical fitness in bariatric patients respect to the only surgical intervention.2max, strength and flexibility were assessed at the start and at the end of the study in both groups.A group of adult sedentary bariatric patients chose to attend a 12-month exercise program integrated with diet education and motivational support, or to receive usual care. Dietary habits, binge eating disorder, physical activity, obesity-related quality of life, Body Mass Index, waist and hip circumference, VOOn a total of 82 patients enrolled, follow-up measures were obtained from 28  and 42  patients included in the intervention and control group, respectively. All the behavioral and physical outcomes improved significantly in the participants to the intervention, while the control group showed lesser changes, especially regarding quality of life and physical fitness.Notwithstanding the self-selection, these results suggest that a timely postoperative behavioral multidisciplinary program for bariatric patients may be effective in establishing healthy behaviors which can lead to better surgery outcomes. Currently, the epidemic of obesity represents a public health challenge and many efforts are required to control its determinants, mainly unhealthy diet and physical inactivity , 2. PersSeveral studies reported a high prevalence of psychological disorders and eating disorders in bariatric surgery candidates, which may be associated with greater difficulties in the adaptation to the new condition after surgery –15. LiteIn addition, the stability of weight loss after surgery may be influenced by behavioral factors such as compliance to the new nutritional regimen and to an active lifestyle, as recommended by guidelines , 21, 22.Physical Activity (PA) is widely recognized as an effective instrument and recommended as an important component of multidisciplinary care programs aimed to improve psycho-physical conditions and surgery outcomes in bariatric patients –24. In pLiterature shows that, when implemented, exercise-based educational interventions are effective in improving weight loss maintenance in the post-operative period: therefore, structured multidisciplinary interventions including the contribute of various figures  may provide to the patients the needed support and education to maximize the success of surgery , 31–33. The aim of the present study was to evaluate the effects of an integrated post-operative exercise-based educational and motivational program implemented immediately after surgery on lifestyles, quality of life, anthropometry, cardiorespiratory fitness, muscular strength and flexibility respect to the only surgical intervention in a sample of Italian sedentary bariatric patients.This was a prospective, controlled and non-randomized study performed over the years 2016–2019 at the Evangelical Hospital Villa Betania in Naples. Participants were recruited progressively since November 2015 to March 2017; follow-up took place since March 2016 to January 2019.The institutional review board of the Hospital Villa Betania approved the study protocol. The investigation was carried out in compliance with the ethical principles of the Helsinki Declaration for medical research involving human subjects. A written informed consent was obtained by all the participants and the anonymity of personal data was guaranteed. The study was registered to the ISRCTN registry with the identifier ISRCTN95430070.Adult sedentary subjects who had undergone a first bariatric intervention no more than 6 months before the start of the intervention were considered. Individuals reporting any physical limitation due to underlying conditions such as cardiovascular or musculoskeletal (including arthritis) disorders, substance dependence, pregnancy, cognitive or psychiatric disorders were excluded. The eligible patients were invited to take part to the study in the hospital during the routine post-operative examination with the bariatric physician by a researcher who presented the aims and the design of the study. Those who accepted to participate in the study were not randomly allocated in the intervention or in the control group: patients who wanted to and could participate in the scheduled activities underwent the multidisciplinary intervention; those who did not want or were unable to take part in the proposed meetings because of other commitments, but were available to provide their clinical and physical information, underwent treatment as usual (TAU) and constituted the control group. TAU protocol consisted of a meeting with bariatric surgeon at 1 month and at 12 months after surgery. During these meetings patients received advice regarding diet and PA. They could see a dietitian upon request.TAU patients were progressively matched to intervention patients in order to obtain at least a 2:1 ratio. Considering an expected difference in weight loss between groups of at least 14 kg , the recThe intervention included an exercise program, a dietary educational program, and motivational support. All these activities were free of charge and were provided for twelve months by the same staff in a hospital gym facility.The exercise program consisted of 60-minute training sessions carried out two times per week and supervised by exercise specialists with expertise in adapted physical activity. Training protocols were developed on the basis of the American College of Sports Medicine guidelines for persons with obesity and tailored to participants’ conditions; the intensity of exercise was periodically increased according to the advances of the subjects . Each seThe motivational program was carried out through periodical series of bi-weekly group meetings lasting 90 minutes and guided by a psychologist with expertise on motivational interviewing for behavior change. The first sessions were focused on the reciprocal introduction of participants and on their barriers, problems and readiness for behavior change; subsequently their thoughts, attitudes and beliefs regarding PA and diet were explored. In the course of the program, according to the social cognitive theory, patients were asked to set personally-meaningful goals, providing feedback, and exploring current and imagined futures regarding lifestyles in order to increase their psychological skills and enhance their self-efficacy . The nutTAU protocol for control patients consisted of periodical routine medical examinations by the surgeon. TAU patients were contacted by phone to participate to follow-up after 12 months since their recruitment.Sociodemographic features were collected for all the subjects who adhered to the study.0) and at the end (T1) of the 12-month intervention in participant and control patients in the same facility where the activities took place. All the measurements were performed by investigators who were blinded to the participants.All the following outcomes were measured at the start  or week ; the number of the days in a week patients consumed breakfast was also evaluated.Dietary patterns of participants were estimated by asking them to record the type and amount of food and beverage consumed for 7 days before TIn order to explore eating behaviors and feeling/cognitions regarding binge eating episodes, we administered to both groups the 16-item Binge Eating Scale (BES), an instrument usually employed to assess the severity of binge eating among obese persons . Three-fThe levels of habitual PA were assessed through the short format of the International Physical Activity Questionnaire (IPAQ) , which aThe Obesity-Related WELL-being questionnaire (ORWELL-97) was administered to all the participants in order to detect possible variations in obesity-related quality of life and particularly to analyze their attitude towards PA . The queVariations in BMI were investigated by measuring height and weight using a medical-certified scale and a stadiometer. Waist and hip circumference  were assessed using a non-stretchable tape and expressed in centimeters to the nearest 0.1 cm. WC was measured at the end of a normal expiration between the lowest border of rib cage and the upper border of iliac crest; HC was measured at the widest part of the hip, at the level of the greater trochanter.2max, which represents the highest amount of oxygen an individual can take in and utilize to produce ATP aerobically during exercise. The Rockport 1-mile walking test (RWT), which is the most commonly used field test to assess cardiorespiratory fitness and to predict aerobic fitness, was used to this aim [2max value was estimated through a formula that includes participants’ body weight, age, gender, time to complete one mile, and post-exercise heart rate [2max was expressed as mL/kg/min.Cardiorespiratory fitness was evaluated through the estimation of VOthis aim . Brieflyart rate . VO2max The strength of lower limbs was evaluated through the chair squat test. Participants were asked to squat down from the standing position until they lightly touch a chair with their back. The number of repetitions performed by participants was considered as outcome . A hydraIn order to detect possible changes in flexibility, participants’ passive Range of Motion (ROM) of shoulder, elbow, ankle and knee joints was assessed through a standard goniometer and expressed as degrees . All joit test for independent samples, while categorical variables were compared using the chi-square test.A descriptive analysis was carried out to evaluate the sociodemographic characteristics of the two groups at the start of the study. Age was expressed as mean value ± SD, while gender, education level and type of surgical intervention were expressed as number and percentage. Continuous variables were compared between groups through the Student’s The mean number of servings consumed per day or week ± SD for each food type and the mean number of breakfasts consumed per week ± SD were calculated for each group.The mean BES values ± SD reached by intervention and TAU groups at the two times were calculated.The mean IPAQ total score ± SD was calculated to express the medium level of habitual PA at the beginning and at the end of the study in both groups.As for the quality of life, the mean total scores of the ORWELL-97 ± SD obtained from the two groups at the two times were calculated.0 and T1 for each group were reported as mean values ± SD.Physical outcomes measured at Tt test for paired samples or the Wilcoxon Signed-Ranks test depending on the variables distribution. The comparisons between groups were carried out through the ANCOVA, using age, gender and baseline values of the dependent variable as covariates and including all their interaction terms in the models.All the variables were tested for normality through the Shapiro-Wilk test. Within-group changes from baseline to follow-up were analyzed using the Student’s Statistical significance was declared at the 0.05 level. Statistical analyses were performed with the software IBM SPSS version 26 for Windows .Age, sociodemographic characteristics and type of surgical intervention were not significantly different between the two final groups .As for the diet, at the end of the intervention significant changes regarding fruits, vegetables, and sweets consumption were observed in the intervention group; TAU patients showed only a significant decrease in sweets consumption .p<0.01). The presence of BED was diagnosed at the start of the study in the majority of the patients who accepted to participate to the intervention  and in all but two of the control patients . At the end of the study, only one among treated patients showed moderate BED, while 25 moderate and 10 severe BEDs were found among TAU patients  and psychological outcomes  of bariatric patients .Participants to the intervention increased also their levels of habitual PA. This confirms the effectiveness of an exercise-based motivational program in supporting sedentary individuals to adopt an active lifestyle, which is fundamental to maintain weight loss over time after surgery –27.As for the physical outcomes, weight loss between the start and the end of the study was greater among participants to the intervention than among TAU patients. This allows to hypothesize that the behavioral changes induced by the lifestyle intervention were effective in preventing long-term weight regain. In fact, the most common causes of weight regain after bariatric surgery are the return to pre-operative eating habits, lack of exercise, or psychological issues . TherefoThe intervention group showed also better improvements than controls in the other physical outcomes considered. These differences were registered both in anthropometric measures, which are directly related to weight loss, and in cardiorespiratory fitness, strength and flexibility, testifying the role of exercise in enhancing physical function of bariatric patients. The improvements registered among TAU patients testify the role of surgical intervention in improving the physical conditions of patients; however, the greater and significant changes shown by participants to the educational intervention suggest that an integrated post-operative support may further improve these aspects, as reported by other studies –29.This study has some important limitations. First of all, the lack of randomization could have generated a selection bias in the constitution of the two patients’ groups, favoring the participation to the intervention by individuals more inclined for physical activity or better disposed to share their personal information. This consideration is also supported by the lower dropout registered among the participants to the intervention. The self-selection bias might have allowed to obtain a higher follow up rate and better outcomes in the intervention group than in the controls. However, it should be noted that many compliant subjects had to decline the invitation to participate to the proposed activities, or did not take part to the follow up measurements, due to their work or family duties. Moreover, since the exercise program was performed through group sessions and the proposed activities were not adapted to the characteristics of each participant, a consistent number of patients with certain cardiovascular conditions or musculoskeletal disorders requiring an individualized approach were excluded from the study. Unfortunately, these categories represented about the 70% and the 25% of the non-eligible patients respectively, and this would affect the external validity of the study. Consequently, the sample size was small and did not allow to analyze the outcomes with respect to the type of surgery. Considering that the magnitude and durability of the effects of Sleeve Gastrectomy and Laparoscopic Gastric Banding can be considerably different, this variable could have influenced the results [Due to these limitations, the present investigation cannot allow to express definitive conclusions. In particular, the non-randomized design of the study limits the representativeness of our sample and the generalizability of our results. Further studies involving wider, randomized samples of patients are needed to confirm our findings.However, the obtained results testify a general improvement in several aspects related to physical and psychological conditions of subjects who participated to the intervention. This indicates that the timely implementation of a structured program which integrates educational and motivational paths with exercise paths, providing patients with correct information regarding diet and physical activity, and supporting them in facing the physical and psychological changes following surgery, may be effective in establishing healthy behaviors which can lead to better surgery outcomes.The early identification of wrong beliefs, lack of knowledge and possible psychological disorders which determine unhealthy lifestyles in obese patients seeking for bariatric surgery may be fundamental to address their post-operative behaviors through a comprehensive approach targeting diet, physical activity and psychological disorders. The review by Kalarchian and Marcus  and the This study suggests that a timely post-operative lifestyle intervention may be useful for bariatric patients in order to help them in enhancing and maintaining surgery outcomes. As the recourse to bariatric surgery is becoming very common, it is fundamental that it could be considered a starting point for the establishment of a healthy lifestyle rather than an ending goal in obesity treatment. In this perspective, a multidisciplinary team composed by different professionals who sustain patients in adopting new behaviors may be fundamental in order to guarantee the long-term effectiveness of the surgical intervention. Adapting the post-operative support to the individual health and organizational needs of the patients may enhance their compliance. Due to the limitations related to the sample self-selection adopted in our investigation, further controlled randomized studies in this direction are needed.S1 Table(PDF)Click here for additional data file.S2 Table(PDF)Click here for additional data file.S3 Table(PDF)Click here for additional data file.S4 Table(PDF)Click here for additional data file.S1 Checklist(PDF)Click here for additional data file.S1 File(PDF)Click here for additional data file.


Immunofluorescent confocal microscopy uses labeled antibodies as probes against specific macromolecules to discriminate between multiple cell types. For images of the developmental mouse lung, these cells are themselves organized into densely packed higher-level anatomical structures. These types of images can be challenging to segment automatically for several reasons, including the relevance of biomedical context, dependence on the specific set of probes used, prohibitive cost of generating labeled training data, as well as the complexity and dense packing of anatomical structures in the image. The use of an application ontology helps surmount these challenges by combining image data with its metadata to provide a meaningful biological context, modeled after how a human expert would make use of contextual information to identify histological structures, that constrains and simplifies the process of segmentation and object identification.We propose an innovative approach for the semi-supervised analysis of complex and densely packed anatomical structures from immunofluorescent images that utilizes an application ontology to provide a simplified context for image segmentation and object identification. We describe how the logical organization of biological facts in the form of an ontology can provide useful constraints that facilitate automatic processing of complex images. We demonstrate the results of ontology-guided segmentation and object identification in mouse developmental lung images from the Bioinformatics REsource ATlas for the Healthy lung database of the Molecular Atlas of Lung Development (LungMAP1) programWe describe a novel ontology-guided approach to segmentation and classification of complex immunofluorescence images of the developing mouse lung. The ontology is used to automatically generate constraints for each image based on its biomedical context, which facilitates image segmentation and classification. We describe the development of ontology-guided segmentation and object identification for immunofluorescent images from the Molecular Atlas of Lung Development Program (LungMAP) consortium . LungMAPTo identify specific structures in immunofluorescent microscopy images of the developing lung, a human observer would make use of experimental, imaging and anatomical contextual information. The experimental context includes information such as species and developmental stage. The imaging information includes the magnification and labeled probes used. Finally, the anatomical context includes the list of structures typically found in the appropriate developing stage of the lung that are identifiable using a particular set of probes, as well as the hierarchical nesting of substructures within structures and spatial relationships between structures. Our application ontology captures the types of contextual data a human observer would use, in a way that can be incorporated into the image processing workflow.http://geneontology.org/) has been used to annotate large collections of literature and data, facilitating data integration and discovery. In turn, this has led to the development of software that can query diverse sets of experimental data curated by different groups of experts and stored in independent databases. In the biomedical disciplines, ontologies provide rigorous, unambiguous descriptions of biological entities and the relationships among them using standardized and well understood formats . The Open Biomedical Ontology (OBO) Foundry (http://obofoundry.org) ontologies are designed to describe orthogonal biological features, utilizing a common standard to ensure interoperability. The OBO Foundry consists of foundational ontologies that define the terms of interest in a domain. Application ontologies combine and take advantage of the terms defined in the foundational ontologies for use in a specific application.An ontology is a controlled, structured vocabulary that provides logical definitions of terms representing distinct categories and their relationships in a particular domain. With strict control of unique term identifiers and logical consistency, ontologies enable the interoperability of heterogeneous data and are widely used as a tool to facilitate the sharing of knowledge. For example, the Gene Ontology , 3  –17. ExamThe immunofluorescent images within the LungMAP project are heterogeneous, including variation in species, developmental age, magnification and labeled probe combination used. We first used the ontology categories to group images into image sets with the same species, development age, magnification, and probe combination. This process is automated using the LungMAP SPARQL API, which returns this grouping metadata in the Resource Description Framework (RDF) labeled graph format for each queried image. The grouping simplifies image segmentation, since images within a set are now homogeneous with respect to extrinsic variables.Manual segmentation of training data is time-consuming and prone to subjective contour boundaries. Most manual segmentation tools also require the user to provide the region labels, potentially requiring review and modification of the labels to match the those in the classification pipeline. To address these issues, we developed a Graphical User Interface (GUI) utility to create training data Fig. . Since tThe tool provides an interface for an expert to visualize the segmented structure candidates generated and accept or reject them. Labels from an ontology-constrained label set can be easily applied to the accepted regions. Regions covering more than one structure can automatically be split using a recursive binary segmentation routine. Finally, any structure missed by the automatic segmentation can be manually drawn as polygons, allowing images to be comprehensively segmented. This is an important feature as our training process is designed to automatically create a “background” class from non-segmented regions in these comprehensively segmented images.the greatest number of probes are prioritized as the primary object(s) to identify. In addition, the pipeline attempts to identify any other structures that have at least one targeting probe. For example, in any image of the lung from a mouse at the developmental stage E16.5 with the probe set ACTA2, SOX-9, and SFTPC, the target structure is the distal acinar tubule bud with 2 matching probes (SOX-9 and SFTPC), and the remaining identifiable anatomical structure classes with a single probe are the bronchiole, the proximal acinar tubule, the pulmonary artery, and the pulmonary vein. The application ontology also plays an important role in providing contextual information regarding the location of probes within structures, indicating whether a probe color is on the periphery of a structure or found within it  traversal of the probe-protein-cell-tissue-structure graph of the application ontology to limit the classes of structures that are identifiable given a specific probe combination. In the LungMAP data sets, a particular set of probes  is used to target a particular anatomical structure. The ontology allows ranking anatomical structures by the number of probes used in the image set that can label a candidate structure. The anatomical structures targeted by The customizable color profile and the configurable segmentation stages provide flexible strategies for the analysis of a large range of immunofluorescent microscopic data sets. However, we found certain strategies more effective for the LungMAP application, and this drove the development of the ontology to automate the creation of segmentation configurations. For example, the first stage should target the most prominent visual structures found in the image set. These structures correspond to those that are targeted by the greatest number of probes. Querying the ontology provides this information.We also found that knowing whether a probe targets the periphery, or the interior of a structure could further improve the segmentation results, and this information was then added to the ontology. Structures targeted by a probe that is found in the periphery of a structure are best placed at the end of the segmentation stage sequence. The rationale for this is that if placed at an earlier position of the segmentation sequence, then the more visually subtle structures in the image could be partially segmented. Using saturation-based segmentation for the intermediate stages proved more effective at completely segmenting these more concealed structures, with the priority towards larger structures. Larger structures are targeted with more aggressive (larger) blur kernels.To label the candidate ROIs, a labeled training set is necessary, and the use of classical machine learning requires only a modest number of training examples for each structure class. The ROIs in the training data are labeled, the feature matrix is calculated for each ROI, and the resulting feature metric data is used to train an XG-Boost  classifiAn application ontology can improve image segmentation and labeling by providing contextual information that is implicit when a human expert evaluates the image. We use the application ontology to construct flexible semantic queries using the SPARQL protocol to generate relevant contextual information for the image analysis pipeline. For immunofluorescent images of the developing lung, the context provided includes general information about the different developmental stages, the hierarchical organization of anatomical lung structures down to the cellular and molecular level, image metadata such as the magnification level, and the experiment-specific information of which fluorescent dyes are mapped to specific antibody probes. This contextual information constrains possibilities and simplifies inference. An additional benefit is that the application ontology guarantees that all terms are unique and derived from more foundational ontologies. This means that all labeled entities provide their own definitions and can be related to other entities. Using a standard vocabulary ensures consistent naming of structures following well-defined standards and also ensures that our labels can be unambiguously linked to other data sets and reference databases. For example, single cells identified in an image can be linked with other data for that cell type  in the BREATH databases, providing spatial resolution to other assay data. The ontology was developed collaboratively by an ontologist and a software developer, using an iterative process to ensure that each addition or modification to the ontology was driven by the intended use of the image analysis pipeline and could be queried programmatically.There are limitations to using an ontology-driven approach for image segmentation and object classification. First, the construction of an application ontology is an iterative process and requires someone with expertise in biomedical ontologies collaborating closely with bioinformaticians who know how to use tools such as SPARQL for querying the ontology. Second, application ontologies are, by nature, constructed for a particular purpose, unlike more general domain  ontologies. Hence, elements of the application ontology must be adapted for new applications, for example, by adding new antibody specificities for a different probe set, or new anatomical relations for different organ systems.While the particular image processing routines described are specific to the lung development context, the framework of logical relationships established by this application ontology is generalizable to object identification in other biomedical imaging experiments. By providing contextual information in a machine-accessible format, application ontologies increase the feasibility of automated image segmentation and object identification. In biomedical settings, an ontology-guided pipeline can be used in a semi-supervised approach to significantly reduce the amount of manual labor while still benefiting from expert oversight. Ontologies are commonly used in a static way for linkage and knowledge sharing, but image segmentation and object identification open new opportunities for ontologies to be used in a dynamic way for machine inference. This manuscript provides a template for integrating an ontology-guided workflow into image segmentation and object classification for complex biomedical images and we hope that it will inspire developers of state-of-the-art image processing applications to consider such an approach.In summary, application ontologies can be powerful tools to pair with immunofluorescent image processing, segmentation and object identification. In addition to their traditional virtues of providing a standard vocabulary and term definitions, logical inference over ontologies can make use of biomedical contextual information to constrain and simplify machine learning. We have demonstrated this approach for developmental lung studies, but believe that it is widely applicable, for example, to study cancer-immune interactions in the tumor microenvironment.https://basic-formal-ontology.org/) formalism and conforms to the requirements of OBO Foundry ontologies. The ontology was specified in the Web Ontology Language and developed using the Protégé (version 4.5) ontology editor. We searched OntoBee (http://www.ontobee.org/) and BioPortal  for equivalent concepts in existing ontologies. To avoid duplication, we imported and used these equivalent terms. If no equivalent term was found, we created new terms and definitions and submitted them to an appropriate OBO Foundry ontology to consider for inclusion. We relied on Uberon [The application ontology was developed following the Basic Formal Ontology , and connected it to the targeted protein by the use of the relation recognizes adopted from REO and antibody ontology AntiO [has_part relation. For example, cell X has_part some protein Y and Antibody Reagent Z recognizes some protein Y. In turn, tissues link to cells, and higher-order anatomical structures link to tissues via the same relationship. The graph illustrating these relationships is shown in Fig. All the relevant data elements were collected from the Bioinformatics REsource ATlas for the Healthy lung (BREATH) database and transformed into terms and relational expressions, mostly reused from other ontologies , we used discrete categories for age. For other applications where a continuous time model is desirable, we would define overlapping intervals at which each entity can be found in the development process.To capture the temporal nature of the development process, the ontology also includes age information. We used an The ontology is used in the process of labeling training data, image segmentation, and mapping of image segments to named anatomical entities. When labeling training data, the ontology provides the list of possible anatomical structures for a given development age. For segmentation, the ontology provides contextual information regarding the location of probes, such as whether a probe is located on the periphery of a structure or found within it. For classification, the ontology constrains the anatomical entity candidate labels allowed given the inferred knowledge about the image and specific segment.The inference over the ontology requires chained SPARQL queries between a set of probes and their related structures that can be time intensive. Hence, relationships between probes and structures are first determined using a set of SPARQL queries and then saved as a look-up table (LUT) for quicker access in the image analysis application. These LUTs can be re-generated programmatically when the ontology is updated. The integration of the ontology into the image processing pipeline is illustrated in Fig. The image analysis pipeline we developed is tailored to the unique properties of immunofluorescent microscopy images, particularly targeting cases where regions of interest are densely packed and numerous. We chose to target these types of images as they are difficult to segment accurately using current computer vision pipelines and highly laborious to segment manually. The pipeline was developed in the Python programming language and is available as an open-source package named odifmap .Our pipeline makes use of metadata associated with each image sample, including species, development age, image magnification, names of the antibody probes, and the colors associated with those probes. Together, this information defines distinct image sets that are analyzed in separate runs of the pipeline. Training data for a single entity consists of an ontology-defined label and a list of vertices given as x–y coordinates that defines the boundary of a polygonal region.Classical machine learning approaches require the reduction of pixel-level data to image feature vectors. The limited colors and gradients in immunofluorescent images can be usefully exploited to generate a limited set of quantitative feature metrics useful for distinguishing between biologically relevant segmentation classes. These features are based on color composition and distribution, region and sub-region shape statistics such as eccentricity and convexity, and simple texture statistics like hole area ratio (HAR).We began the development of features using manually segmented regions and then extracted these regions into groups based on their labels. The extracted regions were visually inspected and analyzed using histograms of their HSV channels. The colors present in all of the images in the LungMAP database are blue: representing the nuclei stained with DAPI, black: representing non-cellular empty space, or "background" due to dark-field microscopy, and red, green, and white: typically used as fluorescent colors in immunofluorescent microscopy. Additionally, we observed that for small areas where a targeted antibody was expressed, the fluorescent color would interact with the blue DAPI stain to produce composite colors.Based on these observations and the histograms of the hue channel in the regions we manually segmented, we partitioned the hue range into 3 main sections we term "major" colors, targeting red, green, and blue. We also binned the value channel into 3 monochromatic colors of black, gray, and white. Finally, we defined "minor" colors as the composite colors of violet, cyan, and yellow. With these 9 color categories established, the entire HSV space was partitioned such that any combination of hue, saturation, and value belongs to a unique color class. The complete definition of HSV ranges for the color classes are shown in Additional file In total, there are 179 feature metrics calculated for each region, consisting of 19 metrics repeated for each of the 9 color classes and 8 global contour features. While the LungMAP data sets employ 3 fluorescence colors, more colors can be used in immunofluorescent microscopy . With thOnce an image set is selected for analysis, its images are preprocessed to be homogeneous with regard to intrinsic variables, such as illumination and color. In microscopy images, illumination inhomogeneity effects occur both within an image and between images. Intra-image variation in light intensity results from a focal light source and manifests as a non-uniform intensity across the field of view (FOV), with higher intensities near the center of the image and trailing out toward the edges. Intra-image FOV variation can be significant, with up to 30% variation in intensity from the center to the darkest edge . Inter-iCorrecting both inter- and intra-image variation is critical for both the segmentation and classification of ROIs. Regions of a particular structure class may be under- or over-segmented due to their location within the FOV of an image and the image to which they belong. The number of pixels in a region due to the quality of segmentation, coupled with the differences in hue, saturation, and value of pixels within a region can greatly affect the feature metric calculations. The feature metric variance for regions within a class can, in turn, then affect the performance of the classification results. Figure To correct the intra-image variation due to the single light source, we create a mask from all the blue pixels in the image. Blue pixels represent the DAPI-stained cell nuclei and are the most common and evenly distributed color in the microscopy images we analyzed. The blue mask is applied to the value channel of the image and the masked value data is fitted using a bivariate Gaussian. The Gaussian fit is then inverted and sampled at each pixel coordinate of the image, and then summed with the original unmasked value channel. This process is repeated for every image in the image set.Following value gradient correction, the inter-image color variation is corrected, starting with the selection of a reference image from the image set. The blue mask is extracted from each gradient corrected image to calculate the mean hue value of those pixels. The image with a mean hue closest to the "true blue" of 120  is chosen as the reference image. The images are converted to International Commission on Illumination (CIE) L*a*b space to transfer the color profile from the reference image to the other images in the image set .The homogenized images are processed to enhance visual contrast of the structures targeted by the probes. In the LungMAP images, blue indicates DAPI staining of nucleated cells and is not utilized as a fluorescent probe color. By clipping the blue channel in RGB color-space to the maximum of the red and green channels, areas of high blue intensity is suppressed. This reduces the saturation and value of regions not targeted by any fluorescent probes, thus increasing visual contrast of the borders in adjacent anatomical structures.Each contrast enhanced image is then converted to a hue, saturation, and value (HSV) color space, where the value channel is used as input to a difference of Gaussians routine to generate a binary signal mask that highlights boundaries in visually contrasting areas. The boundaries of neighboring structures within the signal mask are often connected, and single structures may be incomplete, preventing the use of the signal mask to segment regions directly. Instead, the HSV image is used for segmentation of candidate regions of interest (ROI) based on color and saturation, and those regions are dilated and compared to the signal mask to infer the boundaries of the final ROIs Fig. .Fig. 5a The segmentation pipeline uses a customizable sequence of segmentation stages. Regions generated in each stage are removed from subsequent stages, i.e. the residual is used as the input for the next stage ensuring that there are no overlapping regions generated Fig. . There aThe contours found at each segmentation stage are accepted or rejected by iteratively dilating them against the signal mask. This is necessary as the individual segmentation stages target probes, which are generally bound to sub-cellular components, and the resulting regions may be under-segmented with respect to the complete cellular and structural boundaries. As described previously, the signal mask provides more accurate boundaries for regions of visual contrast but cannot be used to directly generate regions of interest due to its inter-connection with adjacent structures and broken borders of single structures. The final size for the ROI is determined by calculating the percentage of “on” pixels from the signal mask and choosing the iteration where the signal percentage falls off rapidly Fig. .The contours generated by the automated segmentation stages often contain more detail than is necessary for quantitative analysis. This is manifested by an excessive number of vertices defining the perimeter of the region, resulting in a significant increase in the storage size of the arrays defining the regions and a less appealing visual appearance. Thus, the region boundaries are smoothed using the Douglas-Peucker algorithm  as impleThe pixel data in the post-processed regions is extracted and utilized to calculate the feature metrics described previously. The feature matrix is then scaled and passed to a classical machine learning pipeline for classification. After evaluating multiple classification strategies, including several deep learning neural networks, as well as more traditional machine learning techniques such as SVM, the most performant classifier was XG-Boost (XGB) .The classification pipeline returns the predicted label along with the probabilities for each structure class. The probabilities can then be used to optionally accept or reject regions, for example, by choosing the class with maximum posterior probability. As noted above, if any of the images in the training data were comprehensively segmented, the prediction results will also include the probability of belonging to the image 'background'. Since all classified regions will likely have a category where the probability is highest (probability ties are rare), the inclusion of a background class reduces the occurrence of false positives due to partial segmentation or gross over-segmentation of structures, as well as ROIs that were generated in regions where no structure exists. Regions classified as “background” are then easily filtered from the final results.Once regions are classified, an optional final step in the pipeline is to sub-segment the structure into its component cells. Here, the ontology also plays a role in determining how to segment the structure into cells. The ontology provides the list of tissues and cell types found within a structure as well as whether the probes present in the structure are on the periphery or in the interior of the structure. Together with the intracellular DAPI staining and the fact that black represents non-cellular areas, the non-peripheral probe colors are used to segment areas most likely to contain cells. We developed a recursive binary segmentation procedure to identify nested structures within a region, where each region is recursively partitioned into two sections using spectral clustering, stopping at a configurable minimum size Fig. b. The ceAdditional file 1:  Supplementary Figure 1. Comparison of results between a) our odifmap method and b) Mask R-CNN. Supplementary Figure 2. Screenshot from Protégé showing the ontology relations for the distal acinar tubule bud.   Supplementary Figure 3. Example segmentation configuration annotated with descriptions of available options.  Supplementary Table 1. Confusion matrices comparing our odifmap method (top) with Mask R-CNN (bottom). Supplementary  Table 2. a) Represents a list of domain ontologies and the number of terms from each of those ontologies used in the lung application ontology. b) represents a list of relations and the relative ontology name and ID that have been used in the lung application ontology.  Supplemental Table 3. Ontology query results for the probes present in an image, demonstrating the linkage between probes and anatomical structures, as well as their contextual information regarding location where they are present in the structure. Supplementary Table 4. Color definitions utilized in the image analysis pipeline. Colors are defined by partitioning the complete HSV color space such that each HSV value occurs in only a single color label. The hue range spans 180 values, saturation spans 256 values, and value (intensity) spans 256 values. Color labels are classified into 3 groups: major colors spanning 40 hue values, minor colors spanning 20 hue values, and monochrome colors.


Experimental characterization and validation of skin components in aircraft entails multiple evaluations  and expensive campaigns. They require different rigs and equipment to perform the necessary tests. Two of the main dynamic characterizations include the energy absorption under impact forcing and the identification of modal parameters through the vibration response under any broadband excitation, which also includes impacts. This work exploits the response of a stiffened aircraft composite panel submitted to a multi-impact excitation, which is intended for impact and energy absorption analysis. Based on the high stiffness of composite materials, the study worked under the assumption that the global response to the multi-impact excitation is linear with small strains, neglecting the nonlinear behavior produced by local damage generation. Then, modal identification could be performed. The vibration after the impact was measured by high-speed 3D digital image correlation and employed for full-field operational modal analysis. Multiple modes were characterized in a wide spectrum, exploiting the advantages of the full-field noninvasive techniques. These results described a consistent modal behavior of the panel along with good indicators of mode separation given by the auto modal assurance criterion (Auto-MAC). Hence, it illustrates the possibility of performing these dynamic characterizations in a single test, offering additional information while reducing time and investment during the validation of these structures. Aircraft structures have to meet challenging requirements for performance and security and are thus subjected to strict controls and validation procedures. Experimentally, this involves a huge investment in complex facilities and equipment as well as long development and execution periods with multiple feedback iterations. During operation, they undergo multiple static and, especially, dynamic forces from different sources that have to be studied . In thisRecently, new methodologies and techniques have been evaluated by different authors to perform full-field modal analysis that improves the spatial performance of the current instrumentation methodologies and provides a powerful tool for numerical model validation . High-spMost of these studies evaluated the bending modal behavior of plate-like elements as the technique is quite appropriate for wide and thin flexible elements. Some of them are aircraft structural elements ,26,27. TAlthough the characterization of the impact performance in terms of brittleness and energy absorption is typically evaluated under a single projectile impact, a more realistic characterization of the degradation and cumulative damage can be done using multihitting. Two scenarios can be considered here, depending on whether the stress-wave interaction of each projectile impact occurs or not: simultaneous multi-impact or sequential impact, respectively. The next works are good examples of that. The effect of strength and ductility under sequential impacts was studied by Russel in stainless steel plates . The resThe dynamic response after an impact is the resulting vibration of a broadband excitation ,4. TradiHowever, the multihitting entails unknown force conditions owing to the difficulty of describing the force transmitted by every projectile and the location of every single impact. Therefore, operational modal analysis (OMA) has to be considered, which is a response-only methodology that assumes that the spectral excitation level is constant in the band of interest. Previous studies have explored OMA on HS 3D-DIC measurement. In , the autIn this work, the possibility of performing modal identification from a high-velocity simultaneous multi-impact test with a gas gun, intended for impact characterization, is explored to offer a wide dynamic characterization from a unique test in composite skins that can be employed for multiple purposes like model validation or damage detection. HS 3D-DIC was employed to measure the full-field displacement response of a CFRP aeronautical panel to such impact excitation, with unknown quantitative information of the excitation. The full-field information was employed for OMA using the polyreference least-squares complex frequency-domain method (PolyMAX) to estimate the modal parameters. The analysis succeeded in obtaining the modes with good quality indicators.The structural test was conducted on a specimen of a stiffened composite panel representative of cockpit fuselage skin for a regional aircraft. This panel was designed, manufactured and tested by Airbus as part of the Clean Sky program , of whicThe specimen was a 2.3 mm thick CFRP aeronautical panel, shown in During the test, the panel was impacted on its nonstiffened surface with multiple polymeric spherical projectiles (12.6 mm in diameter and 1.3 g) at 50 m/s using a gas gun. The projectiles were shot in a gathered way aiming for the lower right bay of the panel, as highlighted in The projectiles were expected not to impact at the same time instant. However, the lag between the first and last one (of the order of 10 µs ) is manyThe response to such impacts was monitored on the other side of the impacted surface by two high-speed cameras (Photron SA4) to provide stereovision, as shown in Good contrast of the speckle is one of the most important parameters for achieving a successful and accurate correlation. Generally, ambient light conditions are not enough to perform satisfactory measurements, but, especially, high-speed recording reduces the light received by the sensor and produces very dark images. During this test, two light sources were employed to achieve a homogeneous light intensity on the panel. The whole setup is illustrated in a schematic arrangement in The resulting images were afterward processed through 3D-DIC to obtain quantified displacement fields. The processing was performed in VIC 3D software by Correlated Solutions. The analysis included the omega stiffeners but not the aluminum frames owing to the larger difference in depth. These aluminum frames were also a physical obstacle for the vision of the cameras. These issues compromised the correlation area, and hence the region of interest was originally divided into three masks to optimize the correlation around the frames. The correlation analysis was performed using the same parameter for every mask, namely facets of 21 pixels and 3-pixel spacing between the central pixels of adjacent facets. A small step was chosen to improve the spatial resolution in narrow areas such as the omegas. The region of interest is shown in Bending deformation is the consequence of the perpendicular impacts in the panel. Therefore, in-plane deformation was neglected and only out-of-plane displacements, perpendicular to the plate, were considered for analysis. During the inspection of the time series, such as in The results were then processed using LMS operational PolyMAX for OMA, which is a polyreference version of the least-squares complex frequency-domain method and is known to produce very clear stabilization diagrams . For theiψ} and {jψ}, applied to the set of mode shapes of a single estimation:To check the linear independence of the mode shapes, the auto modal assurance criterion (Auto-MAC)  was emplH indicates the Hermitian complex conjugate transpose. The MAC takes values from 0 to 1 depending on the consistency of the evaluated modal vectors. A high level of consistency would produce a value close to 1 and vice versa. Considering a single set of modes, a low value of Auto-MAC would indicate linear independence. Hence, when the Auto-MAC of a modal vector against itself is determined, it takes the value of 1.The superscript As a result of the test previously described, a time series of displacement maps was obtained concerning the full event. After this first inspection, the full-field time histories were gathered and prepared for OMA. The mode identification was initially performed through the stabilization procedure. The stability of the resonances as the model order increases is an indicator of the poles of the system. The analysis of these results ends with checking the quality and linear independence of the set of mode shapes through the Auto-MAC, according to Equation (1). The result is presented in The aircraft industry makes remarkable efforts to increase the efficiency of testing procedures during the development of structures and complex materials to reduce time and costs. This study presents an innovative alternative by including modal identification in impact behavior characterization performed in simultaneous multi-impact testing, originally intended for studying the effect of hail, birds or other unidentified projectiles. Hence, it takes advantage of this complex setup and expensive facilities. This has been shown through an operational modal analysis in an aeronautical composite panel using the response to a high-velocity simultaneous multi-impact test measured by HS 3D-DIC in a full-field manner, assuming linear conditions. In particular, the small strain condition was checked through the level of displacement after the impact. Despite the challenging conditions, the operational modal analysis provided the natural frequencies, damping ratios and full-field mode shapes of different modes of the panel up to 600 Hz. The modal parameters were coherent with the multiple order bending deformation of a plate-like structure, also influenced by the stiffener configuration of the panel. Furthermore, Auto-MAC indicated independence of the obtained shapes. Therefore, a unique test rig can be employed for impact analysis and modal characterization under valid assumptions. Additionally, this combination may offer new possibilities in future studies such as evaluating the cumulative damage produced by multiple simultaneous multi-impacts using traditional local damage assessment methodologies and also monitoring the global effect on the modal parameters for structural health monitoring.


A key issue in saliency detection of the foggy images in the wild for human tracking is how to effectively define the less obvious salient objects, and the leading cause is that the contrast and resolution is reduced by the light scattering through fog particles. In this paper, to suppress the interference of the fog and acquire boundaries of salient objects more precisely, we present a novel saliency detection method for human tracking in the wild. In our method, a combination of object contour detection and salient object detection is introduced. The proposed model can not only maintain the object edge more precisely via object contour detection, but also ensure the integrity of salient objects, and finally obtain accurate saliency maps of objects. Firstly, the input image is transformed into HSV color space, and the amplitude spectrum (AS) of each color channel is adjusted to obtain the frequency domain (FD) saliency map. Then, the contrast of the local-global superpixel is calculated, and the saliency map of the spatial domain (SD) is obtained. We use Discrete Stationary Wavelet Transform (DSWT) to fuse the cues of the FD and SD. Finally, a fully convolutional encoder–decoder model is utilized to refine the contour of the salient objects. Experimental results demonstrate that the presented model can remove the influence of fog efficiently, and the performance is better than 16 state-of-the-art saliency models. There is great influence on the visibility of the human tracking in the wild under foggy environments on account of how dust particles suspend in the air. Therefore, the foggy images typically have low contrast and faded color features, in which the main objects are difficult to be recognized. Saliency detection is advantageous to this task, and it is a cognitive process that simulates the attention mechanism of human visual system (HVS) ,2,3, whiIn the past several years, the detection of visual salient objects has drawn much attention to most image processing applications. Saliency detection in foggy images acts a pivotal part in fields such as human tracking in the wild, object recognition, object segmentation, remote sensing, intelligent vehicles, and surveillance. So far, all kinds of defogging techniques ,5,6,7 haAt present, image processing methods in foggy weather can be split into image enhancement and image restoration methods.Image restoration methods include Dark channel prior algorithm , Visual Due to the low-resolution and low-contrast characteristics of foggy images, traditional spatial or frequency-based saliency models have a poor performance under fog environment. In view of this problem, this paper presents a frequency-spatial saliency model based on the atmospheric scattering distribution of foggy images, which can obtain effective information under foggy weather. Since the traditional machine learning method leads to the loss of boundary information, the object contour detection method of deep learning is added to enrich the edge information of the saliency map. As illustrated in In this paper, traditional methods and deep learning methods are combined to effectively detect salient objects for human tracking in the wild. In step one, the frequency domain (FD) and the spatial information are fused by DSWT. We utilize the object contour detection method of deep learning to obtain the map of the edge of the object at step two. Last, we obtain the final saliency map of the foggy image by fusing the two maps. Specifically, in step one, the foggy image is transformed into HSV color space first, and the amplitude information of FD is utilized to obtain feature maps in each channel. Then, segmenting the image into superpixels and computing the saliency of each superpixel by the local-global spatial contrast. Finally, the DSWT is applied to fuse the FD and spatial domain (SD) saliency maps, and the Gaussian filter is employed to refine the results. The flow diagram of the presented method is shown in Saliency detection is generally driven by low-level knowledge and high-level cues. Therefore, visual saliency computation under foggy environments for human tracking in the wild can be typically categorized into two classes: Saliency computational models and object contour detection approaches. Traditional saliency computational models are data-driven and primarily utilize low-level image features; while top-down object contour detection models are task-driven and usually utilize cognitive visual features.From the perspective of information processing, traditional saliency models can be divided into two categories: SD and FD based models.The SD saliency models are usually based on the contrast analysis to establish the algorithms. Itti et al.  presenteThe saliency models of the FD develop an algorithm by converting the to a spectrum. Hou and Zhang  employedObject contour detection is a traditional computer vision problem with a long history. The traditional computer vision methods include Roberts, Prewitt, Sobel, canny, and other algorithms.In the process of object contour detection, Roberts’ algorithm does not smooth the image, so the image noise is generally not well suppressed, which also affects the loss of a part of the edge when calculating the positioning. However, Roberts’ algorithm has higher positioning accuracy and better effect on steep low-noise images. Prewitt algorithm can suppress noise. The principle of noise suppression is pixel average, which is equal to low-pass filtering of the image. Thus, Prewitt’s algorithm is inferior to Roberts’ algorithm in edge positioning. The practical application of the Sobel edge detection algorithm  is when With the development of deep learning, the fast edge algorithm, HED and RCF algorithms are introduced. Fast edge algorithm  uses ranIn our previous work, we trained an encoder-decoder network with full convolution using Caffe to optimize the performance of saliency detection. The proposed fully convolutional encoder-decoder network can learn the object contour to better represent saliency map in low contrast foggy images. The key contributions of this paper are summarized below: (1) We compute the saliency map via a frequency-spatial fusion saliency model based on DSWT. (2) This framework is further refined by a fully convolutional encoder-decoder model based on fully convolutional networks  and decoIn this paper, we propose a frequency and spatial cues based traditional method through DSWT and a deep learning-based edge detection method fused salient object computational model to obtain the saliency map in foggy images effectively.This section first analyses the features of foggy images, including the imaging model and effect of fog distortion on images in Under fog conditions, there are a lot of tiny water droplets and aerosols in the atmosphere, which seriously affect the spread of light, resulting in a decrease in image clarity and contrast in foggy days. Especially for color images, it also produces severe color distortion and misalignment. From the respective of the computer vision, there are plentiful models ,35 whichIn Equation (1), enuation  and air enuation , respectd.The results show that the scene brightness decays exponentially with the scene depth (1)The original information of the image is destroyed by the fog, and the structural information of the image is regarded as the high frequency component with enough energy in the image. The generation of fog destroys the structural information of the image and affects the details and texture of the scene object.(2)The fog adds some information to the image. The existence of fog can be seen as adding the relevant channel information of the image and making the overall brightness of the image rises.(3)Fog distortion combines with the original information of the image to generate some information. Due to the interaction between fog particles and the information of the image itself, the foggy image adds some multiplicative information, such as fog noise. It blurs the image, which reduces the contrast of the image.The degraded effect of fog on the image  is calleGiven a foggy image, it is transformed into HSV color space firstly, which has shown strong stimuli to human visual cortex in foggy image , thus thThen, the H, S, and V channels are converted into FD respectively by conducting the Fast Fourier Transform (FFT) as:phase spectrum (PS), respectively. And they can be computed via:For foggy images, the low amplitude in FD can be regarded as a cue of the object, and the high amplitude can represent the fog background. Therefore, restraining the high amplitude information to highlight the object region in other words, the salient object can be extracted by removing the peaks of the AS via:Next, it can compute a new FD map via:The FD map is then transformed back to SD by performing the Inverse Fast Fourier Transform (IFFT) via:The saliency maps , saliency values In the end, the saliency value of each pixel The presented model mainly employs 2-levels DSWT to remove the noise of the saliency map and to accomplish the wavelet decomposition on it.Low-pass filter and high-pass filter of the 1-level conversion are represented as According to above steps, the saliency map based on the FD Next, the 2-level fusion is calculated using the following formulas:The 1-level fusion is calculated using the following formulas:Then, the fusion image can be calculated using the following formulas:In the end, the proposed method utilizes a Gaussian filter to generate a smoothed saliency map.Object contour detection model  can filtThe layers up to ‘fc6′ from VGG-16  are usedWe trained the network using Caffe. The parameters of the encoder are fixed when training the network, while only the parameters of the decoder are optimized. This maintains the generalization of the ability of the encoder and enables the decoder network to be easily combined with other tasks.Datasets: Abundant experiments are executed on two datasets to assess the performance of the proposed saliency model.https://drive.google.com/file/d/1aqro3U2lU8iRylyfJP1WRKxTWrrFzizh/view?usp=sharing. The other one is the BSDS500 Dataset. It includes 500 natural images with carefully annotated boundaries by different users. The dataset is divided into three parts: 200 for training, 100 for validation and the other 200 for testing. Object contour detection is utilized to optimize the saliency map which was obtained by traditional machine learning methods of salient object detection. Due to the use of traditional methods, the edge information of the saliency map is incomplete.A foggy image dataset (FI) was collected from the Internet, which contained 200 foggy images. We also provide the corresponding manual labeled ground truths. The FI dataset can be downloaded at Evaluation Criteria: For quantitative evaluation, the average computation time, the mean absolute error (MAE) score, the overlapping ratio (OR) score, the precision-recall (PR) curve, the true positive rates (TPRs) and false positive rates (FPRs) curve, the area under the curve (AUC) score, the F-measure curve, the weighted F-measure (WF) score, and various saliency models are computed, respectively.The precision, recall, TPR and FPR values are generated by converting the saliency map into binary map via thresholding to compare the difference of each pixel with ground truth. eriments ,22.TS and DS denote true salient pixels and detected salient pixels by the binary map, respectively.The ratio of the number of salient pixels correctly labeled to all salient pixels in this binary map is defined as the precision. In other words, precision refers to how many of the samples that are positively judged by the model that are true positive samples. The recall rate refers to how many positive samples are judged as positive samples by the model in the ground-truth map:The TPRs represents the probability that have a right classification of positive examples, and the FPRs represents the probability of splitting a negative sample into a positive sample.F-measure value, denoted as ested in .Given a ground truth main subject region G and a detected main-subject region D. The OR score is the ratio between two times the correctly detected main-subject region to the sum of detected and ground truth main subject region.The percentage of area under the TPRs-FPRs curve is called as the AUC score. It intuitively reflects the classification ability of ROC curve.S is predicted saliency map and G is ground truth, the width and height of saliency map S are presented as W and H.The MAE score to calculate the average difference of each pixel between the saliency map which is predicted and ground truth. It is acquired by:The presented method is compared with 16 well-known saliency detection methods including: IT , CA [16]It can be seen from the three figures that the proposed model is superior to other saliency models, which validates that our saliency result is robust in foggy images.The greatest three results in The IT, NP, IS and BL models find it difficult to suppress the fog background. The map did not highlight salient objects but detected the fog background together. It is treated with fog as the foreground. As can be seen from Saliency maps of the MR, BSCA and GP models show that the fog background areas are too bright, and background and foreground are marked as salient regions at the same time. Therefore, saliency maps are blurred. However, the results are relatively better than IT, NP, IS and BL.The FT, LR, PD, SC, CA models detect salient objects in the foreground while also clearly detecting non-salient objects such as trees, streetlights, and roads in the back-ground. Such an algorithm cannot achieve the purpose of saliency detection and is meaningless for tracking humans in the wild.Although the fog background has less interferential in the SR than others, the salient objects are also not detected. It is the worst model on test dataset. Due to the features they used are ineffective in foggy images.The SO, SMD and MIL have poor performance for foggy images with a slightly complex background. Although salient objects in the foreground are detected, the brightness of the fog in the image affects the final detection results. In other words, these models are not robust in fog environment.(1)The local and global information of the images are utilized, so that the salient objects can complement the information in the FD and the SD. However, the traditional machine learning method leads to the loss of edge information. Thus, causes the boundary of the saliency map to be blurred.(2)The object contour detection method of deep learning is added to enrich the edge information of the saliency map. It can suppress the interference from the fog background and acquire the edge of salient objects more precisely.(3)In the meantime, traditional based method and deep learning-based method are combined to effectively detect salient objects. The proposed method can not only retain the edge more accurately via object contour detection, but also ensure the salient objects’ integrity. By this means, can obtain a more precise and clearer saliency map.The experiment results show that other models cannot detect the salient objects well under foggy weather. It is evident that the proposed method can better detect the salient objects in foggy images and more effective than other models. The reasons are summarized as follows:In our study, we present a high-efficiency model to handle the salient object detection of foggy images. The proposed model combines traditional machine learning based frequency-spatial saliency detection algorithm and deep learning-based object contour detection algorithm to cope with the matter of salient object detection under fog environments. In traditional saliency detection method, the saliency map is acquired by fusing the frequency and spatial saliency maps via DSWT. Then, a fully convolutional encoder–decoder model is utilized to improve the contour of the salient objects. Experimental results on foggy image dataset demonstrate that the proposed saliency detection model performs obviously better against other 16 well-known models.


The unmanned aerial vehicle (UAV) cluster is gradually attracting more attention, which takes advantage over a traditional single manned platform. Because the size of the UAV platform limits the transmitting power of its own radar, how to reduce the transmitting power while meeting the detection accuracy is necessary. Aim at multiple-target tracking (MTT), a joint radar node selection and power allocation algorithm for radar networks is proposed. The algorithm first uses fuzzy logic reasoning (FLR) to obtain the priority of targets to radars, and designs a radar clustering algorithm based on the priority to form several subradar networks. The radar clustering algorithm simplifies the problem of multiple-radar tracking multiple-target into several problems of multiple-radar tracking a single target, which avoids complex calculations caused by multiple variables in the objective function of joint radar node selection and power allocation model. Considering the uncertainty of the target RCS in practice, the chance-constraint programming (CCP) is used to balance power resource and tracking accuracy. Through the joint radar node selection and power allocation algorithm, the radar networks can use less power resource to achieve a given tracking performance, which is more suitable for working on drone platforms. Finally, the simulation proves the effectiveness of the algorithm. Unmanned cluster has strong battlefield viability, and radar networks based on unmanned clusters have received much attention. Compared with traditional monostatic radar, radar networks based on clustered drones have a high degree of freedom in array layout and can be adjusted according to the battlefield environment flexibly ,2.The resource management of the radar networks includes the management of the composition of the radar networks and the allocation of the radar transmit resource . For theThe layout structure of the radar networks is spatially dispersed. So, the resource management of radar networks can jointly allocate the radar node and transmit resources. Aiming at multiple-radar tracking multiple targets, Yan proposed a radar clustering and power allocation algorithm that can select a specific number of radars to cluster tracking each target, and optimize radar transmit power on the basis of tracking accuracy . AndargoThe above papers lay a solid foundation for the research of radar networks resource management and provide many ideas for follow-up researchers, but there are several shortcomings: (1) The above research consider the radar cross section (RCS) of the target as a deterministic variable. Due to the changeable environment and unknown target information, the RCS of the target is uncertain. Under uncertain conditions, it is difficult to guarantee the robustness of the algorithm by using a deterministic model to allocate resources . (2) In Aiming at the above problems, this paper proposes radar node selection and power allocation chance-constraint programming(CCP) algorithm based on fuzzy logic reasoning(FLR) in the case of multiple-target tracking. At each moment, the algorithm first uses FLR to obtain the priority of each target to the radar. Then fusion center clusters the radars and the targets according to the priority to split the multiple-radar tracking multiple-target system into several subsystems of multiple-radar tracking single target, which can avoid the occurrence of two variables—radar node and transmit power—in the optimization function. For the management of the transmit power, a transmit power CCP model is constructed. The CCP model can not only deal with the uncertainty of target measurement, but also ensure the robustness of the algorithm . This alThe remaining structure of the article is as follows. The system model is constructed in the M radars tracking N targets. Each radar only tracks one target at a time. In every sampling period, each radar sends the target information obtained from the echo signal to the fusion center. The fusion center uses the information to guide the operation of the radar system at next moment. The flow chart of the system is shown in Assume a scene with q at the kth sample interval. Assume that all targets move in a straight line at a constant speed, the expression is as follows:ntensity .kth sample interval, the nonlinear measurement equation of the target q observed by the radar i is:Radar obtains target range, azimuth and Doppler information from target echoes. At the q. c is speed of light. Assuming that there is no deviation between the radar beam irradiation direction and the azimuth of the target q, the expression of the signal-to-noise ratio of the echo of the target q received by the radar i is [i. i.represents signal strength attenuation along the transmission path.Where ectively , and thedar i is :(9)SNRq,kth sample interval, the measurement of target q can be obtained from the fusion center:q at the kth sample interval respectively.Clustered radars send the measurement information to the fusion center for further processing. At the The radar networks based on unmanned cluster can adjust the form of radar networking according to the battlefield environment. This paper uses FLR to reasonably cluster radars at every moment. To ensure that each target is always observed by a fixed number of radars, the algorithm is designed as shown in MT is the preset number of radars that track each target.Where FLR is a method for uncertain reasoning problem. It is based on fuzzy logic and uses information such as target attributes and the knowledge base in view of the intuitive and expert considerations to calculate the priority of a target online. The FLR system is shown in σ is a positive number, whose size is proportional to the anti-interference ability of the triangle fuzzy method. Here we consider three factors that are important to the target as input variables for fuzzy logic reasoning: (1) distance; (2) speed; (3) target identity. The detailed description of these three variables is shown in the Fuzzification is essentially a concept of map introduced in mathematics, which describes the relationship between the exact numerical value and the fuzzy sets . This paThe fuzzy rule base provides control rules for fuzzy reasoning, which is related to the distribution of fuzzy values. The more fuzzy the values, the more rules. The form of the fuzzy logic rules in this paper could be represented as:Based on intuitive and expert considerations, 75 if-then fuzzy rules are obtained, as shown in the ith fuzzy rule can be obtained [Firstly, the fuzzy inference needs to match the appropriate rules for the current input value, then use the fuzzy rules to operate on the input value, and finally get the fuzzy output value. According to the Mamdani fuzzy inference method, the fuzzy relation matrix obtained .(14)μRiR constituted by all fuzzy rules iR is:D’ is:The fuzzy relation So, the membership function of D’. The geometric property of fuzzy inference is to use ith fuzzy set, The results obtained by fuzzy inference are fuzzy, which cannot be used to directly measure the priority of the target. So, the fuzzy value need to convert to the exact numerical value. Defuzzification can be regarded as the mapping of fuzzy space to exact numerical space. This paper uses the center average defuzzifier for defuzzification. In this method, after weighted average, the output numerical value of each element is taken as the execution quantity. Where, the weight is the height of the corresponding fuzzy set . The numkth sample interval is constituted into M is the number of radar, N is the number of target), then the radars are clustered by the following methods.km corresponding to kn. Set all values of the matrix km to zeros.Select the highest priority value MT, set all values in the column of the target in matrix Repeat step 1 to assign radars to each target. If the number of radars assigned to a target has reached the preset require number The priority of each target for different radar at the By using the above radar-clustering algorithm based on FLR at each sample interval, each radar can track the target with high priority to itself. N (the number of target) multiple-radar tracking single-target subsystems. Then, aiming at the subsystem, the transmitting power CCP submodel for multiple-radar tracking single-target is established. The submodel is solved by the hybrid intelligence algorithm, which is formed by embedding the stochastic simulation into the genetic algorithm.In order to deal with the uncertainty of the environment and target information, this paper treats target RCS as uncertainty information. So, the power resource CCP model for multiple-radar tracking multiple-target is constructed. The Bayesian Cramer-Rao low bound (BCRLB) is used to measure the tracking accuracy of the targets. With the radar-clustering algorithm based on FLR proposed in the previous section, the multiple-radar tracking multiple-target system is divided into q [q. m is the number of radars clustered to observe the target q in the same times.In Bayesian estimation, for the target of a single motion model, the BCRLB matrix provides a lower bound for the MSE of the state estimation of the target :(20)Εξq,target q . Zq,k={zkth sample interval, there are m radars for clustering observation of the target q, so the BIM matrix q is [At the get q is :(21)J: Let Step (2): Generate N sets of RCS vector samples Step (3): If Step (4): Let Step (5): If Use Algorithm 1 to verify whether the decision vector  level a .Algoritkth sampling time, the transmit power of the radar at the next time is predicted by Algorithm 2:Algorithm 2. Stochastic simulation algorithmStep (1): According to the target motion equation Step (2): Initialize population S and stain length N, then verify the feasibility of chromosomes using random simulation.Step (3): Hybridize and mutate chromosomes, use random simulation to verify whether the chromosomes meet the constraints, and correct those chromosomes that do not meet the constraints.Step (4): Calculate the objective function value of all chromosomes, and calculate the fitness function value of each stain according to the objective function value.Step (5): Use roulette to select chromosomes. If the requirements of the stopping rules are not met, go to (3). If the stopping rules are met, go to step (6).Step (6): Returns the best chromosome as the optimal radar transmit power At the Algorithm 3. UKF algorithmStep (1): Let k = 1, initialize each radar transmit power Step (2): The transmitting power of each radar is Step (3): Construct the 2L + 1 sigma point set Where, ith column of the square root I represents the dimension of the state vector.Step (4): Map the sigma point set Step (5): Map the sigma prediction point set Step (6): Calculate the gain matrix Step (7): According to the intelligent hybrid algorithm proposed above, predict the radar transmission power kth moment, let The state vector of the target is estimated from the radar measurement information. Since measurement equations are highly nonlinear, this paper uses Algorithm 3 to predict the state vector of each target.This section designs a scenario where multiple radars are tracking multiple targets to verify the effectiveness of the above algorithm. In order to simplify the analysis and eliminate the influence of radar deployment on the tracking performance, the positions of the 12 radars in the scene are not changed, and all radars form a square network. There are three targets in the scene, and the starting position of each target is set randomly. The radar parameters of each radar are the same, as shown in the following The position parameters of radars are shown in The status parameters of the three targets are shown in the The distribution of radar and targets is shown in There are 40 frames in the simulation process. Within each frame, the radar clustering method based on FLR is first used to implement the selection of radar node. Then, the CCP model is used to allocate the transmitting power of each radar.In order to facilitate comparison, this paper extracts all the numerical results According to the above numerical values in After obtaining the clustering situation of radars for targets at each moment. The hybrid intelligent algorithm is used to solve the power CCP model for multiple-radar tracking single-target. From this, the total transmitting power of each radar can be obtained.In order to clearly show the distribution of the transmit power for each radar, It is found from the three diagrams in In order to show the tracking performance of the algorithm in this paper, we compare the root mean square error(RMSE) of target tracking error obtained by this algorithm with the RMSE of equal power allocation in This paper provides a new solution to the issues of joint radar node selection and power allocation for radar networks based on UAV. An innovative radar-clustering algorithm is proposed to select the radar node online. In the paper, the target RCS is treated as a random variable to ensure the stability of the model in real scenarios. Aiming at the problem that the size of the UAV limits the radar transmitting power, the CCP model is used to optimize radar transmit power. Simulation results show that the algorithm in this paper can reasonably schedule radar node and effectively save the radar power resources.The future work will be directed to the following aspects: The paper assumes that the position of each drone is fixed. This does not fully take advantage of unmanned cluster. In the future, we can rationally schedule the layout of UAVs based on the target-tracking situation to achieve better tracking performance.