Appearing traces of bias in deep networks is a serious reliability issue which can play a significant role in ethics and generalization related concerns. Recent studies report that the deep features extracted from the histopathology images of The Cancer Genome Atlas (TCGA), the largest publicly available archive, are surprisingly able to accurately classify the whole slide images (WSIs) based on their acquisition site while these features are extracted to primarily discriminate cancer types. This is clear evidence that the utilized Deep Neural Networks (DNNs) unexpectedly detect the specific patterns of the source site, i.e, the hospital of origin, rather than histomorphologic patterns, a biased behavior resulting in degraded trust and generalization. This observation motivated us to propose a method to alleviate the destructive impact of hospital bias through a novel feature selection process. To this effect, we have proposed an evolutionary strategy to select a small set of optimal features to not only accurately represent the histological patterns of tissue samples but also to eliminate the features contributing to internal bias toward the institution. The defined objective function for an optimal subset selection of features is to minimize the accuracy of the model to classify the source institutions which is basically defined as a bias indicator. By the conducted experiments, the selected features extracted by the state-of-the-art network trained on TCGA images , considerably decreased the institutional bias, while improving the quality of features to discriminate the cancer types. In addition, the selected features could significantly improve the results of external validation compared to the entire set of features which has been negatively affected by bias. The proposed scheme is a model-independent approach which can be employed when it is possible to define a bias indicator as a participating objective in a feature selection process; even with unknown bias sources. Histopathology examination is typically required when a definitive diagnosis is needed due to the invasive nature of biopsy techniques2. Additionally, pathology is used to diagnose a wide range of inflammatory conditions, infections, and metabolic and autoimmune illnesses6. A new avenue for the application of computer vision techniques used in computer-aided diagnostic (CAD) systems has been created by the digitization of standard tissue glass slides into Whole Slide Images (WSIs)7. One of the prospective uses of computational pathology to aid pathologists is image search10. It provides search through a WSI database for images with equivalent histomorphological characteristics or a specific region of interest. End-users may discover their first diagnosis, acquire more about the patient’s prognosis, and ultimately arrive at the final diagnostic interpretation when integrating additional data using the information acquired from comparable instances. As a result, content-based image search can aid in the growth of precision medicine.Under a light microscope with various magnification settings, histopathology is the study of alterations in tissue samples11. Then features, as the representatives of histopathological patterns of images should be extracted. Analogous to other computer vision fields, histopathology has also been heavily influenced by the emergence of Convolutional Neural Networks (CNNs) which have become an integral part of digital pathology12. Nowadays, many researchers prefer to use pre-trained CNNs for feature extraction13. The quality of features plays a crucial rule in the performance of a digital pathology tasks such as image search and classification. In the medical domain, in order to evaluate the generalization capability of any model, including a trained deep model, “external validation” is conducted, in which the model is tested on a set of unseen data. The necessity of external validation in medical image analysis is widely accepted as the accuracy of the model is significantly degraded in many reported cases14. One of the possible reasons of this crucial validation process is the potential existence of bias in medical images. Avoiding biased data and biased training are crucial challenges in AI16. Many machine learning methods are unsuccessful on unseen data from similar domains while they achieve highly promising results on training or test sets within the same domain. Additionally, the decisions of an AI algorithm should not reflect discriminatory behavior toward certain groups or populations17; various types of bias can negatively affect a data-driven model simultaneously18. In a recent study, Howard et al.19 reported that the distribution of clinical information in the TCGA data, such as survival and gene expression patterns, remarkably differs among samples provided by various clinics and laboratories. They showed that some models detect source sites instead of predicting prognosis or mutation states; which is neither expected nor desired. Many works have been conducted to eliminate these site-specific signatures to enhance the reliability of histologic image analysis, some through correcting the differences in slide staining between institutions20. Some research works tried to utilize the methods proposed by Reinhard et al. and Macenko22 to decrease color variation across images. Additionally, other works24 utilized color augmentation, where the color channels are altered at random during training to prevent a model from learning stain characteristics of a specific site. Most research works on stain-normalization and augmentation techniques have focused on model performance in validation sets, rather than elimination of the site-specific signature that may lead to a biased model26. In addition, bias may exist in any type of medical images. DeGrave et al.27 showed that the trained models on radiographic images are more likely to learn medically irrelevant shortcuts, usually attributable to bias in data acquisition, instead of the actual underlying pathology related information. In28, the authors revealed that an approach may be subject to bias if the feature extractor is trained on specific institution datasets and potential hidden biases are not accounted for. Factors such as scanner configuration and noise, stain variation and artifacts, and source site patient demographics are more likely potential reasons for the observed biases28. The bias detection and recognition would be more challenging when we are faced with multiple types of biases simultaneously. In order to overcome bias in DNNs, some works29 tried to find the relationship between the attributes existing in an image to discover those that cannot be well learned by CNN. Authors proposed a method to discover potentially biased representations hidden in a pre-trained CNN. In addition, a representation-based bias may be removed by resampling technique30. The proposed method looks for a set of weights at the example level that penalizes samples that are simple for a classifier created using a given feature representation. This is accomplished by learning an independent linear classifier and employing a DNN as a feature extractor for the desired representation. Then, minimizing the difference between this classifier’s loss on the reweighted dataset and the uncertainty of the ground truth class labels is equivalent to the minimizing bias.WSIs are generally gigapixel images 31 was investigated. It was established that the deep features extracted from the images are able to accurately distinguish the WSIs based on their acquisition site. Therefore, samples provided by each institution apparently have similar clinical characteristics possibly stemming from various factors, such as digital scanner configuration and noise, tissue stain variation and artifacts, and source site patient demographics. This reveals that DNNs are perhaps unexpectedly detecting the specific irrelevant patterns of source site rather than histomorphologic patterns which results in failure against an external validation. Surprisingly, the bias issue with features is regardless of whether a pre-trained DNN such as DenseNet32 is utilized as the feature extractor or a state-of-the-art DNN trained on cancer subtypes such as KimiaNet33. In other words, even if a DNN is not specifically trained on histopathology images, it is able to discriminate the sources of the samples. This is likely because the utilized data for extracting the features is still bias-contaminated. Therefore, in addition to the challenge of biased training, apparently there exist some tissue source-site specific patterns in TCGA images  which can be even reflected in features extracted by pre-trained networks. That is possibly related to the non-optimal design of the DNNs structure which leading to extracted non-optimal set of features34. Consequently, it seems that DNNs have a tendency to learn irrelevant shortcuts patterns related to data acquisition sites, rather than the actual underlying morphological information28.In a recent study37. However, this cannot be an efficient solution for pre-trained networks or for DNNs which have been already trained on biased data. There are many DNNs that are presently being used for different applications despite of considering the effects of biased data and/or biased training38.In order to tackle these bias issues, three categories of solutions can be imagined. Since the bias originates from the institutions that contributed WSIs to the dataset, the issues such as variation in tissue processing, tissue stain protocols, stain quality, color intensity, scanning hardware platforms, and imaging protocols should be considered for filtering the bias; This, however, does not seem to be an easy task to enforce standardization guidines for general bias filtering, at least in foreseeable future. Alternatively, the training methods could probably be modified to avoid incorporating of hospital-identifying factors into images and consequently to improve the generalization of the modelbias-contaminated” features.Motivated by the above considerations, elimination of biased features from the extracted original features can be an efficient and applicable approach as a post-processing solution for downstream tasks. In other words, the features that encompass the knowledge about the source institution can be removed so that the remaining features discriminate the cancer types through relevant information of histopathologic patterns rather than hospital-identifying visual clues. In this direction, we propose a feature selection framework to select a set of optimal features among the output of a feature extractor. An optimal set of features are those that discriminate the cancer types accurately but do not distinguish the images based on their acquisition source, i.e., the hospital. In fact, during the feature selection, we will explicitly enforce removal of the “In this work, we propose three vital objectives for feature selection as an optimization problem. Similar to all feature selection processes in machine learning, the main goal is to remove redundant and irrelevant features to achieve more accurate image analysis results. Thus, the first objective is defined as the maximization of accuracy of image search. But in addition to the performance of image search, it is desirable to reduce the bias toward source site in optimal feature set. Hence, the accuracy of classification on institute labels is assumed to reflect the bias and consequently is considered as the second objective to be minimized; in fact, the minimization of this objective will manage to remove the bias-contaminated features as much as possible by considering the trade-off between the two objectives.Finally, shortening the feature vector representing a WSI enables us to sample more tissue patches for each WSI as digital pathology tasks such as image search can be sophisticated with current techniques. Accordingly, A small representative code will be essential for both greatly speeding up image retrieval and utilising less memory. The main contribution of this paper is to design a framework to alleviate the bias as a post-processing technique. To the best of our knowledge, this is the first time that a proposed framework can effectively address the bias issue for pre-trained networks. We tailor the evolutionary algorithms for this purpose. The innovation of this study is not only in designing such a framework but also in proposing three objective functions to formulate the bias in medical images. The discovery of bias formulated by the capability of institution classification using histopathological features and proposing a solution to address it are pivotal for the research and pathologist community. The proposed framework can be utilized as a complementary approach with other bias control methods which work toward cleaning data or reducing bias during training phase; it is model-independent (pre-trained or newly trained ones) and can be utilized when defining a bias indicator is possible with no need to know about the source of the bias.multi-objective optimization with three major objectives. We tailored an evolutionary feature selection algorithm to select the best subset of features to increase the generalization performance of a compact representative code for histopathology images by reducing the internal bias and increasing the accuracy of image search simultaneously.Briefly, the proposed feature selection can be modeled as a We utilized TCGA repository which consists of 32,072 WSIs for 32 primary diagnoses. The labeling of these images is at the WSI level  and includes information such as ‘morphology’, ‘primary diagnosis’ and ‘tissue or organ of origin’.40, in which a deep network called “KimiaNet” has been trained for the purpose of histopathology feature extraction, only the permanent section  diagnostic slides from the TCGA repository were used . Cases with no diagnostic, morphological or magnification information reported in addition to the ones scanned at a magnification lower than Based on a previous study41, images of TCGA are categorized into 12 tumor sites. These categories include: endocrine, hematopoietic, pulmonary, breast, brain, gastrointestinal tract, melanocytes, gynecological, prostate/testis, liver/pancreaticobiliary, urinary tract, and mesenchymal. Each tumor site except mesenchymal, hematopoietic, and breast, consists of more than one primary diagnosis which enables classification task. Table According to40 as a network specifically trained on histopathology images and “DenseNet-121”32 as a pre-trained network. DenseNet-121 is a compact architecture with almost 7M parameters. The network is trained by 1.2 million natural images from ImageNet42. The architecture of KimiaNet is the same as DenseNet-121 but re-trained from scratch on TCGA histopathology images. The features are extracted from the last pooling layer.For the specific purposes of this work, we extracted two sets of features from the patches of the whole dataset, using “KimiaNet”high cellularity patches, which impose homogeneity, MFV is relevant in our application.On high-cellularity patches of TCGA images, KimiaNet is trained using soft labels. For the training, validation, and test datasets, respectively, the authors split the patches into 7126, 741, and 744 diagnostic WSIs during pre-processing processes. In order to extract the features based on the 30 primary diagnoses, the network is trained using a set of patches. As a consequence, a WSI is characterized by 135 feature vectors, each in size of 1024. The mean of feature vectors (MFV) over all patches is utilised as a useful representation in order to reduce the amount of data needed for WSI representation. In this manner, a WSI is represented by the mean of feature vectors that only have 1024 values before to the feature selection. Because KimiaNet was trained on In order to distinguish the primary diagnosis throughout the full dataset, as was already indicated, feature vectors are extracted . However, the primary objective of an image retrieval system is to identify the images from a particular category of tumour type that are most similar to a given query image.There exists significant bias toward the source institutions in extracted features by deep networks. This implies that DNNs may have learned to distinguish image source institutions as a form of biased shortcut to classify cancer types. If we assume that the features comprise of both relevant histologicall information and irrelevant non-morphologic tissue source site patterns, then, an optimal feature selection may be a logical approach to eliminate the biased features. Therefore, the proposed method aims to decrease the undesirable bias by selecting the optimal features set which distinguish the cancer types independently from institutes. Accordingly, our solution builds upon the concept of a post-processing phase of learning regardless of the type of data-driven feature extractor. Figure As mentioned previously, the features are extracted from the images of TCGA repository which includes 12 tumor type categories each of which encompasses several primary diagnoses (tumor subtypes) resulting in 32 primary diagnoses in overall. For each tumor type, an optimal set of relevant features are selected among all extracted features to accurately classify the primary diagnosis related to corresponding tumor type. By this way, the multi-objective optimizer tries to eliminate all redundant, irrelevant, and biased features which is tailored for each tumor type accordingly.43. An optimization model is made up of four main components: the objective functions that we intend to minimize or maximize, the problem encoding, the constraints of the problem (if any), and the optimization algorithm.In the light of the aforementioned points, feature selection is defined as a multi-objective optimization problem which can be solved by an evolutionary algorithm. Evolutionary algorithms have strong global optimization capabilities and can consider the combination of features in terms of multiple objectivesThe fallowing three objectives are defined to select the optimal features in order to decrease the bias resulting from acquisition sites:Maximization of the image search quality.  Mainly, the evolutionary feature vector is expected to increase the efficiency of image retrieval. In order to evaluate the quality of selected features, the most similar slides to a query WSI are retrieved. By removing redundant and irrelevant features, the best match of query image can be found. Evaluation of the image search is accomplished by calculating the F1-score of k-Nearest Neighbor (KNN) method on primary diagnosis labels33. As mentioned previously, one of the prospective uses of computational pathology is image search which provides case matching through a WSI database for images with equivalent histomorphological characteristics or a specific region-of-interest. We utilized the kNN technique to determine the top matched  features when we searched through the features. However, to evaluate the performance of search, we have used KNN as a search-based classifier to quantify the efficiency of features. The KNN is one of the commonly used methods in this field. Therefore, the F1-score of image search is considered as the first objective, S. F1-score is a harmonic mean of precision and recall measures defined as the follows44.Precision can be calculated asRecall can be given asMinimization of the number of required features.  When building a classification model on high-dimensional data, the “curse of dimensionality” is a crucial problem, according to45. A shorter code can reduce the complexity of memory needs while simultaneously speeding up retrieval in datasets with many gigapixel pictures. Even when there are less characteristics, there may still be insufficiently relevant information for the learning system to accurately categorize WSIs46. As a result, it is thought that the size of the feature vector and classification accuracy are two conflicting objectives. In the optimization phase, a string with a length equal to the number of features is used to represent each individual for feature selection. A feature’s status is represented by a cell in the vector, where a value of ‘0’ denotes a feature’s rejection and a value of ‘1’ denotes a feature’s selection.Therefore, the number of selected features is equivalent to the total number of ‘1’s in the vector. Hence, given a set of all features represented by a binary vector, S, as the second objective, D is the total number of features.Minimization of the bias of image acquisition source. On the flip side, the bias toward the acquisition source can be defined by the accuracy of institute classification. Therefore, the third objective of feature selection can be reasonably defined as the minimization of institute classification accuracy over the selected features. Ideally, the extracted features should not be the representatives of their source institute. Hence, accurate classification of acquisition sites using the features reveals the existence of bias and may inevitably lead to considerable drop in accuracy for processing of external data . In order to avoid the bias, the optimization algorithm explores the search space of features to find the optimal set of features with minimum accuracy of institute classification. Similar to what we done for image search accuracy, the quality of selected features for constructing a model to accurately classify the institutions is assessed. To this effect, each image is labeled with the cooperated institution instead of primary diagnosis. The best of features should not represent the information related to institutions, hence, we select a set of features with minimum accuracy of institution classification. Hence, the third objective, S, is calculated as follows:In this section, we explain the conducted experiments in order to assess the proposed feature selection for gigapixel pathology images to decrease the bias. The series of experiments investigate the effectiveness of selected features in terms of the value of objectives. The main goal of experiments is to observe whether the optimizer is able to select an optimal set of features with minimum bias toward image acquisition source.In this study, 12 independent tumor-based optimization problems are defined to decrease the number of features and to select the optimal subset specialized for each tumor type category by decreasing the bias, increasing the image search accuracy, and decreasing the number of selected features, simultaneously.k-nearest neighbour technique47. In order to assess the potential solutions during the optimization process. To get the average F1-score across all primary diagnoses, three images that are the most close in terms of Euclidean distance are obtained. The second goal is to determine how many features were chosen. In order to classify the images based on their institutes of origin, we need to employ one of the state-of-the-art classification algorithms. To this effect, a one-versus-all Support Vector Machine (SVM) with linear kernel is trained on the training images. The SVM is a machine learning method that has become exceedingly popular for its simplicity, flexibility, and effectiveness for addressing a range of classification problems. SVMs stand out for their ability to provide balanced predicted performance, even in research with potentially small sample numbers48.Following some preliminary trials, the population size and the number of fitness calls were set at 50 and 512,000, respectively. The validation set images are searched among the training set using the 49, the experiments are conducted as 31 independent runs resulting in 31 Pareto-fronts. The final feature subsets on Pareto-front are evaluated on the test set. Finally, the Wilcoxon statistical test50 is conducted to analyze the significance of the generated results.The corresponding subset of features is selected in training set and then the accuracy is obtained on validation set as the third objective. Due to the stochasticity of evolutionary algorithms, according toFigure In addition, Table From Table  Similar results are obtained using DenseNet features. Although DenseNet is a pre-trained network with no learning process on histopathology images, the classification accuracy of institutions is still high ; just 6% less than the KimiaNet case which was trained with the biased data. This suggests that not only the learning process of deep networks can be biased by information of acquisition sites but also the TCGA images contain the clues of source institutions which are picked up by network during the feature extraction. Surprisingly, the accuracy of institution classification indicating the bias is even higher than the average F1-score of tumor subtype search. As it can be seen from the Table According to the experimental results, the existence of bias toward the information of tumor site sources is not deniable. However, the elimination of such bias in extracted features can be accomplished as a post-processing phase applicable on any data-driven features extraction method. The proposed evolutionary algorithm selects those features that contain less discriminative patterns of source site institution and instead keep the expressive information to distinguish the cancer subtypes. By decreasing the undesirable correlation between the cancer type information and their source sites, we expect the selected features to lead to more accurate external validation, i.e., higher generalization. Therefore, even if models learn transiently useful but histologically irrelevant information from the images, the proposed feature selection can be effective to eliminate those features irrelevant to primary diagnosis patterns and consequently to prevent their negative impacts on WSI representation. The reported results clearly demonstrate that the idea of embedding bias reduction objective in the multi-objective feature selection process can promisingly reduce the bias up to 40% and 36% over the considered two state-of-the-art DNNs as the case studies.In order to transparently show the performance of selected features, it is essential to evaluate the capability of the optimized features to distinguish different tissue types of WSIs. K-means clustering was applied to cluster tissue patches of a renal clear cell carcinoma sample, using only the optimized subset of features to represent each tissue patch. The k was arbitrarily set to 3. The result is shown in Fig.  As mentioned previously, the destructive effect of existing bias in a dataset utilized for training a DNN is mostly on external data, i.e., the unseen samples from an institution that is not participated in training or test. We showed that we could reduce the bias on TCGA data by considering it as an objective function in our optimization. However, the institutions of images in test data had also participated in training and optimization phases. In order to investigate the benefits of the proposed method on generalization for the search, it is required to conduct external validation. Hence, the performance of the proposed method is validated using a set of external Kidney images which have not participated in neither training nor optimization process. The dataset consists of 142 samples of Kidney cancer subtypes from the Ohio State University. There exist 50 samples from KIRC subtype, 45 from KICH, and 50 from KIRP according to subtypes presented in Table In this work, we proposed a novel feature selection framework to reduce the effect of the biased features of DNNs when operating on histopathology images. The proposed framework is applied to the extracted features from the images as the output of the DNNs after they have been trained. Therefore, the presented bias reduction strategy is independent of the utilized dataset and the DNN itself since it operates as a post-processing stage of a learning framework. Feature selection aims to eliminate the irrelevant and redundant features that degrade the image analysis performance and remove the bias-contaminated features carrying the image acquisition site-specific patterns unintentionally present in digital slides. To this effect, three objectives, including maximization of image search accuracy, minimization of institution classification accuracy, i.e., defined as the bias indicator, and minimization of the number of features for compactness, are considered. Two sets of experiments were conducted on features extracted using a pre-trained network and a DNN trained on histopathology images. The experimental results on NIH’s TCGA repository showed that the optimal features result in significantly lower classification accuracy of institution labels, alleviating the feature vector’s internal bias. However, the resultant representation code is highly compact and accurate for primary diagnosis discrimination. As one of the valuable experiments, we conducted external validation to clearly show the performance of selected optimal features on the unseen data from an institute that is not contributed to any part of the development of the method. This experiment clearly showed that the selected optimal features take advantage of the bias reduction and can search the images more accurately than all features. In conclusion, we have demonstrated in this paper that the proposed evolutionary framework can be beneficial in improving the generalization capability of feature extractors by alleviating the degrading impact of biased data on generalization and medical image analysis. This work demonstrated the potential for new complementary post-processing to overcome bias in deep learning.52 will be more beneficial to address this issue. Clearly, in this direction, a reduced number of features will make the attempt of explainability more easier.The only assumption we make is that we can select the optimal features to minimize the accuracy of institute/source classification. The group structure of genetic data might be useful in finding the relation between the genetic information and cancerous patterns but most likely after removing the bias. In this study, we just tried to select the best features with focus on alleviating the bias. The proposed method has formulated three objective functions which can be effective in decreasing bias and increasing the cancer type discrimination at the same time. This framework is designed based on the limited information that we could obtain to interpret the existence of bias in TCGA dataset. Definitely, as a future perspective and with more discovery on the source of bias, the complementary informationProposed feature selection framework is designed based on a multi-objective optimization process. Accordingly, we need to define the objective functions based on which features are selected. The process of evolutionary feature selection is explained in the following subsections. By this type of process, a set of random combinations are initialized as a population and as the algorithm iterates, optimizer improves the individuals to find an optimal set of features considering the objective functions. The improvement of individuals  can be done using a couple of generative operators to create new combinations.53. The population-based nature of these algorithms results in generating a set of candidate solutions at each run of the algorithm. Collaboration of individuals to make an optimal “Pareto-front”49 is the core reason for the success of population-based algorithms. The mathematical definition of multi-objective optimization problem can be defined as54M is the number of objectives, d is the number of decision variables , and the value of each variable, MOO targets handling two or more conflicting objectives. The use of evolutionary algorithms has been very promising for solving such problemsDue to the conflicting of objective functions in a multi-objective optimization problems, the definition of the optimality is not as simple as the single-objective case. Therefore, it is required to find a trade-off among objective functions. One of the commonly used concepts for comparing candidate solutions in such problems is the concept of dominance which is defined as follows: If 55 is one of the popular selection strategies which works based on the dominance concept. It ranks the solutions of the population in different ranks of optimality, called Pareto. The algorithm starts with determining all non-dominated solutions in the first rank. In order to identify the second rank of individuals, the non-dominated vectors are removed from the set to process the remaining candidate solutions on the same way. Non-dominated solutions of this step make the second level of individuals (second Pareto). Thereafter, the second ranked individuals will be removed to identify the third Pareto. This process will continue until the all individuals are grouped into different ranks of Pareto.This concept defines the optimality of a solution in a multi-objective space. Candidate solution An optimization model is made up of four main components: the objective functions that we intend to minimize or maximize, the problem encoding, the constraints of the problem (if any), and the optimization algorithm. In the following, each component for a multi-objective feature selection problem is defined. In addition to the defined objective functions in the previous section, the optimization requires an encoding scheme to be considered as the representation of variables of the problem. The encoding can represent each individual in the population, demonstrating a set of features. To this end, a binary vector in size of all features indicates the status of a feature, where 0 reflects the absence of a feature and 1 is indicative of feature’s presence. Thus, the optimizer explores the search space to find the best value  for each variable. The overall structure of the proposed framework is illustrated in Fig. We now highlight the steps of the algorithm; similar to all population-based evolutionary algorithms, as it is presented in Fig. 49, the performance of this algorithm has been compared with several algorithms to demonstrate the effect of adding the reference points to improve the distribution and quality of the resulting Pareto front.The next step of the optimization is generating the new individuals in order to explore the search space. One-point crossover is utilized to combine the two individuals in the population to create a new candidate solution. By this operator, a random combination point is selected for both parents individuals. The variables after the combination point are swapped with each other. Two new individuals  are generated. Then bit-wise mutation is utilized in which a number of bits flip at random positions. The new individuals compete with the old population while the best solutions  are selected for the next generation. We applied the well-known many-objective optimization algorithm to find a subset of features to alleviate the impact of bias. Among these algorithms, the NSGA-III is one of the state-of-the-art algorithms that has been proposed to address the shortcomings of its predecessor NSGA-II. InThe NSGA-III selection strategy chooses the top ranked feature subsets based on the NDS algorithm and the reference lines distributed uniformly in the search space to provide the well-spread out candidate solutions.Those subsets of features  with maximum accuracy of primary diagnosis classification and minimum source institution classification  are selected among the population. In fact, the elite individuals which are the best subsets of features are selected to make the next generation. The evolutionary process leads to optimize the defined objective function for the goal of feature selection. At the end of process, a set of trade-off solutions are obtained. A decision maker can pick one of the solutions according to the desired objective or criterion.


We introduce a general framework for monitoring, modeling, and predicting the recruitment to multi-center clinical trials. The work is motivated by overly optimistic and narrow prediction intervals produced by existing time-homogeneous recruitment models for multi-center recruitment. We first present two tests for detection of decay in recruitment rates, together with a power study. We then introduce a model based on the inhomogeneous Poisson process with monotonically decaying intensity, motivated by recruitment trends observed in oncology trials. The general form of the model permits adaptation to any parametric curve-shape. A general method for constructing sensible parameter priors is provided and Bayesian model averaging is used for making predictions which account for the uncertainty in both the parameters and the model. The validity of the method and its robustness to misspecification are tested using simulated datasets. The new methodology is then applied to oncology trial data, where we make interim accrual predictions, comparing them to those obtained by existing methods, and indicate where unexpected changes in the accrual pattern occur. Efficiently recruiting patients to clinical trials is a critical factor in running clinical trials and hence delivering new medicines to patients as quickly as possible. Late-stage clinical trials are commonly run across many sites, and successfully managing and running trials and subsequent processes require accurate forecasts of trial recruitment.Lasagna’s Law . For this reason, we recommend restricting the choice of The flexibility of the model, however, can result in potential identifiability issues. Inference methods, such as maximum likelihood, can run into numerical instabilities when Biostatistics online.We aim to construct a framework which can provide reliable predictions whilst capturing uncertainty in the estimated parameters and in the underlying model itself. We employ the Bayesian paradigm since it naturally incorporates the distribution of the random effects, Biostatistics online provides more details on these quantities; see also Given a parametric statistical model, the Bayesian paradigm starts from a prior distribution for the parameters, here denoted Biostatistics online provides further details of this method, as well as of effective sample size (ESS), a diagnostic which indicates the reliability of the Monte Carlo estimates; see also For the models under consideration for trial-recruitment data, neither the posterior model probabilities nor the posteriors for the parameters for any particular model are tractable, and so we employ importance sampling to obtain Monte Carlo samples In Sections We base our prior specification on a maximum likelihood meta-analysis of 20 oncology clinical trial recruitment datasets. The trials studied were for seven different types of cancers: ovarian, prostate, breast, small and non-small lung, bladder, and pancreatic. The number of centers ranged from 58 to 244 with a median of 140 and total enrollments ranged from 245 to 4391 with a median of 1035. In all cases, the parameter estimators were close to orthogonal justifying the use of independent priors: We found that the The Biostatistics online.The difference between the homogeneous  and the  paradox  warns ths. Given , the resBiostatistics online.There are two complementary properties for which predictions might be required: the distribution of future recruitments within a set time interval, and the distribution of time until the target number of recruitments is reached. In this section, we focus on the former; details of the latter appear in Suppose we are interested in sampling the recruitment, denoted Marginalizing over the random effect posterior, we arrive at the negative binomial distribution:The length of interval to If simulations for multiple distinct time periods are required for a given center, osterior . We thent}, from .When predicting the enrollments using a fitted model, we implicitly assume that a single model best reflects reality; however, prediction methods should consider the uncertainty in the models used for inference. We shall, therefore, use model averaging for making predictions, that is, take a weighted average of predictions made by each model. Working in the Bayesian paradigm provides us with an intuitive choice for weights in the form of marginal likelihoods of the models.Before making any statements in regards to the future recruitments, we should validate that the fitted model does indeed capture the true data-generating process sufficiently well. Since the true process is unknown, we compare the observed data to the modal model (the model with the highest posterior probability) fixed at posterior parameter means Firstly, we wish to assess that the chosen hierarchical structure is reflected in the data. The distribution of posterior means of the individual random effects should approximately follow the hierarchical According to the model, the counts in any initial period given in  but usinBiostatistics online.We demonstrate our flexible framework through a simulation study, using simulated data sets to illustrate model fit and prediction and to highlight the effect model misspecification can have on predictions. In practice, patterns in center initiation times can vary greatly between trials. For presenting the methodology, we consider an initiation schedule similar to that observed in a typical trial. We test the robustness of the method using a uniform initiation schedule, with another type of schedule examined in Our historical data set do not include the initiation times of the centers, so instead, to accurately reflect the historical data used in the meta-analysis and what is often available to researchers, we take the first recruitment time of a center as its initiation time and adjust the models to include a single deterministic recruitment at the initiation time of each center followed by stochastic recruitment as described in Section We simulate a study over a course of mentclass2pt{minimll model . The “noBiostatistics online) confirms at least approximate pairwise orthogonality between the parameters, as anticipated from Sections Biostatistics online.Performing the LRT and BST from Section Biostatistics online) still showed bias due to the choice of a single model, although the contrast with We repeated the analysis with a different distribution of initiation times, making the center initiations “clump” roughly every 2 months. The resulting forecast predictive distribution can be seen in To further test the robustness of the framework, we first consider the random effects Biostatistics online). The robustness of predictions comes from the fact that the random effects for initiated centers use re-estimated data-driven distributions, reducing the importance of the random-effect prior; thus the main source of forecasting error comes from the incorrect random-effect prior for new centers. Similar plots for the “clumped" initiation schedule, provided in the Biostatistics online, show the same pattern. This mixture distribution of random effects represents the (extreme) scenario where roughly half of the centers recruit the vast majority of patients, with the remaining sites recruiting little to none each. When the ratio of the two means is closer to 1, the model still produces reliable predictions.We considered data generated using the same Biostatistics online).We also consider the effect of curve-shape misspecification on predictions, generating data using an intensity proportional to the Weibull density functionWe fitted the same set of models to a recruitment dataset of a prostate-cancer clinical trial. The recruitment was carried out across 244 sites. The accrual is presented as the proportion of the total number enrolled. Similarly, time is given as the proportion of the total recruiting period. Biostatistics online. Forecast completion time from 6 different census points and can be seen in Biostatistics online.We compare the proposed framework to the standard homogeneous PG model  as well a priori that a particular global period would be unrepresentative then this time span, and the concomitant recruitment, could simply be removed, albeit at the cost of lower power.We have introduced a general, flexible framework for modeling and predicting recruitment to clinical trials. We suggest two tests for detecting decay in recruitment rates; comparing them both with respect to power and robustness. The particular form of the test statistic allows for a single, simple trial-level test. Alternative forms, such as splitting according to a global time, would either require a test for each center, massively reducing the power, or estimates of all of the individual center intensities which would introduces several layers of additional complexity because of the hierarchical connection between the center intensities. If it were believed The parametric curve-shape forms chosen for the intensity were based on the features encountered in oncology trials. We found that the model was still robust to moderate model misspecifications in the distribution of the random effect and intensity shape. Other therapeutic areas such as pulmonary or cardiovascular diseases experience more frequent recruitments and different curve-shapes may be appropriate. As shown in Section As seen in the data example in Section For curve-shape parameter prior construction, our choice of the quantity of interest Biostatistics online, we demonstrate how recruitment can be predicted using our methodology when there is uncertainty in the initiation schedule. For illustration, we imagine a Weibull-distributed delay to each center’s initiation, but any other initiation model could be incorporated in a similar manner. We stress that full prediction intervals should take this uncertainty into account.In presenting the method, we condition the inference and prediction on known initiation schedules for the centers. Incorporating stochastic center initiation models, such as those in In this work, we focus on patient recruitment regardless of the numbers of dropouts observed. In practice, screening failure and patient withdrawal are both prevalent in clinical trials. Assuming the dropouts are independent of the recruitment process, existing survival analysis techniques such as Cox’s proportional hazard model  or accelhttps://github.com/SzymonUrbas/ct-recuitment-prediction.Software in the form of R code is available at kxaa036_Supplementary_DataClick here for additional data file.


Radiation (RT) remains the most frequently used treatment against cancer. The main limitation of RT is its lack of specificity for cancer tissues and the limited maximum radiation dose that can be safely delivered without damaging the surrounding healthy tissues. A step forward in the development of better RT is achieved by coupling it with other treatments, such as photodynamic therapy (PDT). PDT is an anti-cancer therapy that relies on the light activation of non-toxic molecules—called photosensitizers—to generate ROS such as singlet oxygen. By conjugating photosensitizers to dense nanoscintillators in hybrid architectures, the PDT could be activated during RT, leading to cell death through an additional pathway with respect to the one activated by RT alone. Therefore, combining RT and PDT can lead to a synergistic enhancement of the overall efficacy of RT. However, the involvement of hybrids in combination with ionizing radiation is not trivial: the comprehension of the relationship among RT, scintillation emission of the nanoscintillator, and therapeutic effects of the locally excited photosensitizers is desirable to optimize the design of the hybrid nanoparticles for improved effects in radio-oncology. Here, we discuss the working principles of the PDT-activated RT methods, pointing out the guidelines for the development of effective coadjutants to be tested in clinics. Cancer is one of most threatening diseases to the human population. To eliminate it, in recent decades radiotherapy (RT) has been progressively developed and refined, becoming one of the most diffused and affordable treatments of oncological pathologies in clinics ,2. IonizPhotodynamic therapy (PDT) has been proposed as an alternative to RT due to its high selectivity and minimal invasiveness . The PDTA further step to overcome the PDT drawbacks was made in 2006, with the introduction of the concept of energy transducers to transform the energy deposited by X-rays into optical luminescence that can activate the PDT moieties, thus achieving an enhancement of the RT effect by X-ray-activated PDT (X-PDT) . The eneThe aim of this work is to highlight the general key parameters to optimize the performance of any multicomponent nanomaterial proposed for X-PDT applications. Indeed, appropriate design of the system, in terms of its composition and concentration of the constituents, as well as type of linkage strategy among the components can affect the interaction with the ionizing radiation, the activation of the photosensitizers, and the production of ROS, and thus the overall therapeutic efficacy of the X-PDT agent. Diverse potential archetypal architectures of multicomponent hybrid nanomaterials for X-PDT are discussed, pointing out a possible guideline for the realization of efficient materials that are exploitable in future clinical trials.aq- (hydrated electrons), HO• , H• , H2 (radiolytic hydrogen), H2O2 (hydrogen peroxide), and HO2• . After further reaction mechanisms, these products can create additional ROS, such as •), hydroperoxides (ROOH), and singlet oxygen . The delivered dose, in this case, depends on the radionuclide activity and on the number of radionuclides successfully accumulated in the tissues of interest [According to diverse factors involving the type and size of cancer, the location in the body, the volume in relation to the normal tissue and healthy organs, and the medical history of the patient, the RT healing dose can be delivered to the tumors by external or internal methodologies . The intinterest ,43. Convinterest . On the interest . The curinterest . In thiseristics . The lack of selectively is the major drawback of the RT approach. In this regard, the progress in external RT technology has allowed the establishment of different methodologies for more precise dose delivery while saving the normal organ structures from undesirable irradiation. Advanced beam managing techniques allow to act accurately on deep parts of the body. For example, γ-ray beams are exploited for the treatment of lesions and tumors located in the brain, where multiple photon beams (Gamma Knife) are precisely focused to deliver the therapeutic dose with extremely high accuracy in space ,49. Thre(−) that subsequently react with molecular oxygen to form superoxide anion radicals (31Photodynamic therapy (PDT) is a clinically approved cancer therapy based on photochemical reactions. It relies on the interplay between light-triggered molecules so-called photosensitizers, photons, and molecular oxygen. The reaction of a light-activated photosensitizer with the latter created radicals or reactive oxygen species (ROS) , which c imaging  or populeactions ,54. The sfer ET,  to promooxic ROS .The effects of PDT can be differentiated in three main areas. The first one is the direct consequence of the presence of high amounts of ROS, which result in cytotoxicity as both products superoxide and singlet oxygen can directly react with and damage biomolecules such as lipids, proteins, and nucleic acids . The supThe second effect of PDT is on the vasculature. A proper vascularization/angiogenesis is a key process in cancer development , the impThe third mechanism of PDT-induced tumor destruction is the initiation of an inflammatory response that is followed by host tumor immunity. PDT-induced oxidative stress can upregulate the expression of heat-shock proteins, transcription factors related to inflammation, and the release of inflammatory cytokines . MoreoveSince its regulatory approval over 30 years ago, PDT has been the subject of numerous studies and has proven to be an effective form of cancer therapy. For example, Kaposi’s sarcoma (KS) is an angioproliferative neoplasm of endothelial origin, associated with human herpes virus 8 and human immunodeficiency virus (HIV), that has been effectively treated with PDT . Severaln, with n ≅ 3–4 [vide infra, As discussed above, the energy of X-rays used in clinical RT ranges from hundreds of keV to MeV. Consequently, most of the best photosensitizers that can be used for cancer PDT cannot be activated in these conditions because their low density does not enable efficient interaction with these highly energetic photons. The basic concept of the X-PDT approach is to exploit a physical transducer to harvest the X-ray energy and, consequently, to trigger the photosensitizers to produce the cytotoxic species required for the destruction of the cancer cells . As illu n ≅ 3–4 . Thus, dIn the case of resonance between the nanoscintillator emission and photosensitizer absorption spectra, the fluorescence from nanoparticles can be absorbed by the photosensitizers, thus further enhancing the generation of additional ROS by an active radiosensitization, which is the core of the first X-PDT concept .2 or ZnO) and lanthanide oxides (Ln2O3) as well as fluorides and silica-based nanostructures have been proposed for X-PDT application. Indeed, they can present intrinsic and extrinsic dopant emissions in the blue–green spectral range, suitable to excite organic conjugated photosensitizers such as fluorescent porphyrins [−2 while the fluence from external beams radiation is about mJ cm−2 [Inorganic nanomaterials such as metal oxides  in the tissues . AuNPs’ −3, the maximum calculated concentration of spherical particles with a reasonable radius of 50 nm that can be injected is around 10−4 μM. This means that water is still the most abundant material in any tumor tissue, and consequently, the primary absorber of the incident X-ray radiation, since the probability that the primary event occurs in the nanoparticles is extremely low. Upon primary interaction with water, the X-ray photons are converted in the swarm of secondary high-energy diffusing charges that move within the medium, depositing their energy through a series of random inelastic scattering events. During their diffusion, the charges can lose their energy by directly triggering the formation of the ROS species or activating the luminescence when encountering a dense nanoscintillator. Therefore, the indirect energy sharing through secondary events is the principal mechanism for the activation of the luminescence of nanoscintillator suspensions. Even in the case of a direct primary event, simulations demonstrate that with gadolinium oxide nanoparticles with a diameter of 10 nm, only a very small percentage of the deposited X-ray energy is stored in the nanoparticle, while most of the energy is transferred to the medium outside the nanoscintillator to distances of up to one hundred nanometers [To better understand the radiosensitization mechanism, we should recall that we have a limit to the amount of material that we can safely inject into organisms. For nanoparticles, these amounts are typically up to few μg each milliliter . Thus, tnometers . The effAnother pathway to sensitize the PDT agents is by the direct recombination of diffusing secondary charges generated around the nanoparticle on the PDT moiety A. For th−1. The LY value depends on several parameters including the type and energy of the ionizing radiation and the photoluminescence quantum yield of the nanoparticle, which can be as large as an increase near to 100% in several types of scintillating nanomaterials [−1, [An alternative energy-sharing pathway that does not strictly depend on relative distances is the radiative energy transfer process , which iaterials ,100,101.als [−1,  thus theThis issue can be partially addressed by exploiting a non-radiative energy transfer mechanism between the nanoscintillator and the photosensitizer. Here, the energy is transferred from an energy donor system to an energy acceptor without the production of electromagnetic radiation. In general, the non-radiative energy transfer can be distinguished in Förster-type or Dexter-type mechanism, depending on the physical mechanism that rules the interaction in the donor/acceptor pair. The Förster mechanism operates by an electrostatic coupling of the transition dipole moments of the donor/acceptor pair. It can be represented as a simultaneous non-radiative de-excitation in the energy donor and excitation of the acceptor. Its rate falls off with the inverse sixth power of their distance, depends on their mutual orientation, and requires an energetic matching of the two transitions involved . Quantit6-DBA and Hf12-DBA aniline), showed an energy-transfer-mediated enhancement of hydroxyl radical generation of 33.6% or 55.3%, compared with water, with an X-ray dose of up to 10 Gy. As shown in ad hoc spatial arrangement of photosensitizer molecules, allows the boosting of the nanosystem performances, demonstrating how the non-radiative transfer from the nanoscintillator can be an efficient pathway to activate the photosensitizer moiety [Similar results have been obtained with other hybrid nanosystems of biocompatible chrysotile for RT implementation. The results revealed that a tailored surface functionalization, including an r moiety . AccordiThe understanding and engineering of the physical mechanism occurring in the X-PDT systems (radiosensitization and non-radiative energy transfer processes) are crucial for the design and development of optimized X-PDT agents for the enhancement of RT’s therapeutic effect. Hereby, we propose an affordable and feasible guideline for the manufacture of cutting-edge X-PDT systems, based on considerations from the point of view of the physical phenomena occurring within the nanosystems. Notably, the following considerations are valid for both the cases in which the nanoscintillators are activated by an external source of ionizing radiation  or if raAccording to the working principle of non-radiative energy transfer mechanisms, the electronic properties of the nanoscintillator must be engineered to have a photoluminescence yield as close as possible to 100% and to maximize the resonance between its luminescence spectrum and the absorption of the selected PDT moiety . In suchHowever, we should recall the imbalance between nanometric sizes, and the dimensions of the process involved in the scintillation mechanism, such as the mean distances of the cascade phenomena involved  discussed above. It is worth noting that there is not yet any univocal proof that the activation of PDT by non-radiative energy transfer is the best strategy to maximize the radiosensitization and therapeutic effects. Current research needs to carefully investigate this point, because otherwise, the X-PDT architecture can be further simplified. If the non-radiative ET can be neglected as a sensitization channel, the PDT moiety can simply be connected to the nanoscintillator in order to be confined in a volume corresponding to the region in which we have the maximum energy deposition probability upon secondary events and charge recombination, as can happen in composite bulk scintillators . IdeallyInorganic, organic, and hybrid nanoparticles and nanoscintillators are being considered for a plethora of biomedical applications due to their biocompatibility and low toxicity as well as the tunability of their size, porosity, crystallinity, and shape. Therefore, extensive research has been developed to identify the surface modification and functionalization strategies for a more precise control of drug loading, particle dispersion, blood circulation, accumulation, and specific targeting of nanoparticles for drug delivery and diagnostic and therapeutic applications ,111. Specifically, tumors are characterized by structural and molecular peculiarities that differ from those of the healthy tissues. Thus, this diversity is exploited to develop targeting strategies of nanoparticles to be used for several oncological therapies with a minimal impact on healthy tissues and reduced side effects: due to the structural features of the tumor vasculature, involving chaotic growth, a disordered structure, irregular shape and diameter, and poor lymphatic drainage, nanometric particles can penetration through the blood–tissue barrier and accumulate in the tumor tissue  via the In this paper, we summarized the physical and photophysical mechanisms that are in involved in RT and PDT mechanisms, which are currently under investigation to achieve a combined X-ray-enhanced PDT for the improved treatment of deep tissue cancer. A critical view of the physical aspect of the process is mandatory to identify the appropriate guidelines for the development of optimized multicomponent nanomaterials to be tested in clinics. The imbalance between the nanoparticle dimensions and the dimensions of phenomena involved in the scintillation mechanism indeed leads to fundamental differences in the scintillation mechanisms when compared to monolithic bulk materials. The nanometric dimensions of the dense particles that are exploited to locally maximize the radiosensitization localized in sick tissues are indeed too small to be able to store a significant fraction of the energy transported and dissipated by secondary charges created during scintillation. This suggests that a classical design based on nanoscintillators functionalized with PDT moieties may be not the best one to properly sensitize the PDT effect, even if the system architecture is optimized to maximize the energy transfer yield from the nanoscintillator to the PDT moiety. We propose a general alternative layout of the X-PDT system in which the PDT functionality is placed around the nanoscintillators at distances of up to 100 nm to realize a harvesting network for the energy deposited outside the nanoparticle. A detailed spectroscopic and in vitro investigation is therefore necessary to validate this material design strategy, which can pave the way to the realization of an efficient X-PDT system and that could help shed light on the peculiar interaction of the ionizing radiations in the biological environment at a nanoscale. It should not be forgotten that the proposed nanomaterials should also possess good biocompatibility properties and must be designed in order to show selective accumulation properties in tumor tissues to maximize the therapeutic effect. Indeed, it is worth noting that despite huge progress being made in this direction in the last decade, the targeted delivery of nanoparticles is still one of the most challenging problems for the nanotechnologies that are developed for the treatment of malignant tumors and other diseases, as well as for targeted imaging and diagnosis. Success in fabricating efficient X-PDT agents and developing efficient delivery strategies would represent a breakthrough in the treatment of oncological diseases. The availability of an effective RT coadjutant would revolutionize the method currently employed for the treatment of many cancers, since the X-PDT agents may in principle be implemented in any RT treatment, including advanced methods such the Gamma Knife technique, to improve both the specificity and efficacy while limiting therapeutic side effects. This latter point is crucial as it contributes to the improvement of the patient’s quality of life, thus reducing the detrimental social/economic consequences ascribed to the need for additional care and collateral costs of the disease.


Many deep learning frameworks have come up for retinal blood vessels segmentation tasks. However, the complex vascular structure and uncertain pathological features make blood vessel segmentation still very challenging. This paper proposes a novel multimodule concatenation via a U-shaped network for retinal vessels segmentation, which is based on atrous convolution and multikernel pooling. The proposed network structure retains three layers of the essential structure of U-Net, in which the atrous convolution combining the multikernel pooling blocks are designed to obtain more contextual information. The spatial attention module is concatenated with the dense atrous convolution module and the multikernel pooling module to form a multimodule concatenation. And different dilation rates are selected by cascading to acquire a larger receptive field in atrous convolution. Adequate comparative experiments are conducted on these public retinal datasets: DRIVE, STARE, and CHASE_DB1. The results show that the proposed method is effective, especially for microvessels. The code will be released at  The retina is one of the most important parts of the eyes . On the The unsupervised method and the supervised learning method comprise the retinal vessel segmentation method. The difference between them is whether the input data have manually segmented labels. Oliveira et al.  used twoDuring the past few years, convolutional neural network (CNN) has made outstanding achievements in the automatic segmentation of retinal vessels. Compared with traditional machine learning, there are many layers of neural networks in deep learning, which has strong nonlinear modelling ability and feature representation ability. In particular, since the U-Net  was propWe proposed a multikernel pooling based on the U-shape network that retains three layers, the essential structure of U-Net, but the atrous convolution combining the multikernel pooling blocks are designed to obtain more contextual information.We design a multimodule concatenation network to contain local and global information for retaining small vascular and advanced features.The spatial attention module in the network is concatenated with the dense atrous convolution module and multikernel pooling module, which can further enhance the significance of the target.We evaluate and analyze the proposed MC-UNet on the challenging task of retinal blood vessels segmentation. According to the results of experiments, our method reaches the state-of-the-art level on the public datasets.Although these U-Nets and their improved networks have been used in retinal vessels segmentation so widely, those suffer from many limitations and deficiencies. The encoder-decoder structures receive the information feature and its transmission in the same layer by jump connections, which may cause the loss of small and fragile vessels owing to the limited comprehensive features. In order to alleviate the problems, we propose a multimodule concatenation network based on a U-shape network called MC-UNet for retinal vessel segmentation, which retains local and global information about the retinal main blood vessels and capillaries. Furthermore, the contributions that this paper can make are summarized as follows:In this section, we will introduce our proposed MC-UNet shown in SA is obtained by multiplying the input feature F and attention map σ(·), which is shown in the following formula:f7 and σ represent 7 ∗ 7 convolution operation and the sigmoid function, respectively. The illustration of spatial attention module is shown in The spatial attention module  generater represents the dilation rate and k is the length of the filter w. In particular, when r = 1, formula  and background (negative). By comparing the segmentation maps with the true value of the label, four indexes can be obtained: TP, TN, FP, and FN. P represent the number of white pixels in true images; N represents the number of black pixels in the true image; T for true; and F for false. TP represents the number of white pixels correctly predicted by optic disc, while TN represents the number of black pixels correctly predicted by the optic disc. The values of TP, TN, FP, and FN are calculated according to the total number of pixels in the ground-truth images.On the basis of these four basic indexes, accuracy (ACC), sensitivity (SEN), specificity (SP), area enclosed by the coordinate axis under the ROC curve (AUC), and F1-score can be calculated . In our On the three datasets, we train and evaluate our method by using the manual annotation marked by the first expert. The segmentation result examples from the DRIVE, STARE, and CHASE_DB1 datasets are shown in We also compare the segmentation results on the three datasets with other methods by the five evaluation criteria shown in And There are three skip connections and four layers in our proposed method, compared with four skip connections and five layers in the original U-Net. Although our network has added multiple integrated modules, it has a much smaller number of parameters compared with the original U-Net with 23 convolutional layers and is a lightweight network. The proposed network can enhance the specificity of the image effectively and reduce the blood vessels rate of false positives in the fundus image by integrating the DAC and MKP modules into SA-UNet. However, the limited images available in the dataset restrict the performance of the algorithm. In the experiment, we set a certain number of iterations to avoid overfitting. And we only consider the solution of the same data domain. The domain adaptation method can be introduced to solve domain shift for cross-training and verification for the robustness of the algorithm.In order to solve the limited comprehensive features extracted by the encoder-decoder structure in the U-shaped network, which may lead to the segmentation loss of small fragile capillaries, a novel U-shape network is proposed named multimodule concatenation U-Net (MC-UNet) based on atrous convolution and multikernel pooling for retinal vessels segmentation. The network retains local and global information about the main retinal vessels and capillaries. The DAC and MKP modules are introduced to increase the receptive field for improving the sensitivity of the algorithm and retain more detailed feature information for improving the accuracy of retinal vascular segmentation. Experimental results prove the effectiveness of the method, especially for microvessels. However, for more severe lesions in image data, a robust framework is still needed to be studied and discussed.


The inter-fiber failure of glass fiber-reinforced epoxy specimens with four different fiber angles was analyzed. Flat specimens were subjected to static and fatigue loading considering different load levels and load ratios. Damage investigation in terms of crack density measurement was performed by transmitted white light imaging using a digital camera and LED illumination from the back of the specimen on a servo-hydraulic testing machine. Static and fatigue results were examined with respect to crack initiation and crack growth, considering the special case of bonding yarns parallel to the fiber directions. The bonding yarns act as stress concentrations, influencing the early cracking behavior, and complicate the detectability of cracks growing underneath or next to the bonding yarns. In cyclic loading, the influence of load level, load ratio, mean stress, fiber orientation, and ply thickness was the focus of the experimental campaign. Cyclic cracking behavior in terms of initiation and growth was analyzed based on the applied loading conditions and laminate configurations. It was found that halving the ply thickness nearly doubled the amount of microcracks in case of high loads. For low loads, no such effect was observed up to  The fatigue behavior of fiber-reinforced plastics (FRP) is of high interest for many technical applications due to the excellent material properties and low weight of the structural components. They are used in many industrial applications, both for structural load bearing purposes in naval or wind power sectors and for porous materials ,2. Many It is known that an embedded ply forms multiple sub-critical cracks prior to final failure, and that these grow along the fiber direction through the entire width of the specimen ,11,12,13Great effort has been put into the analysis of cross-ply laminates, which are a special case of fiber alignment rarely used for structural applications. However, the analysis of cross-ply laminates has yielded fundamental findings on the cracking process in FRP laminates, and a variety of models have been developed based on these findings ,25,26,27Berthelot [Weibull, distributions with varying parameters were used to describe the cracking process in three different material systems with four different lay-ups, respectively. For each configuration, a unique parameter set for the fracture toughness distribution was found to describe the cracking process for static loading conditions. In [Weibull distributions were applied to SN-curves for crack initiation and to Paris-relations for crack growth in order to account for the stochastic nature of the fatigue cracking process within the models.Specifically, erthelot  used twoerthelot  Weibull,ions. In ,30,31,32On the contrary, the amount of experimental data for the analysis of the cracking process in laminates comprised of layers with fiber angles ublished ,31,32,33ublished , which iublished . Howeverf, that artificially increases the crack spacing used for calculations. It is therefore presumed that crack growth mainly takes place in areas with crack spacings greater than the equidistant distance, although this has not been experimentally verified.For modeling purposes, it is essential to know what the driving forces of damage are and whether the model parameters have a physical meaning. Current models  E-glass consisting of dry fabrics with an areal weight of 220 g/m2 bonded on a single-side by polyester yarns. As matrix material, an epoxy resin system RIMR135 in combination with a hardener RIMH137 at a mixing ratio of 100:30 was used. The laminate lay-ups were stacked manually and consisted of twelve individual plies. The dry stacks were infiltrated in terms of resin transfer molding (RTM) and cured at 80 °C for 15 h. To alleviate variations in the glass transition temperature, The glass fiber-reinforced polymer laminates investigated in this research were made of Scotch Weld DP-490 epoxy-based adhesive to minimize clamping effects, except for the VNRS specimens, where tabs are not mandatory. The results of the material characterization in terms of the mean value, The UD-laminates were used for the determination of the material’s elastic and strength properties on a Zwick 1465 universal testing machine equipped with a 50 kN load cell and traveling extensometer for strain measurement. Specimens were cut from the cured composite plates by water jet cutting. While the edges were not polished after cutting, no undesired initial cracks or delaminations due to the cutting process were observed during visual inspection. The specimens were tested for axial tension and for in-plane shear in terms of the V-notched rail shear (VNRS) according to the applicable standards. Aluminium end tabs with a 45°-chamfer towards the gauge length were bonded to the UD specimens using Furthermore, laminates with four different lay-ups see  were manIn accordance with the UD tensile tests, the laminates were equipped with aluminium end tabs as well. The specimens were cut to their final dimensions using a high-precision abrasive cutting machine, and the edges were not polished after cutting. An image of a representative specimen used for matrix cracking analysis with its final dimensions is shown in In contrast to the UD material characterization, static and fatigue laminate tests were performed on a servo-hydraulic testing machine see . DetailsThe number of 0°-plies was equal for all laminates. However, in the case of multiaxial lay-ups the 0°-plies were placed between the The crack counting experiments were conducted on an Instron 8801 servo-hydraulic testing machine equipped with a 100 Damage characterization was performed in terms of automated crack counting. The experimental setup used for the evaluation of matrix cracking is shown in Based on the presented detection algorithm in In order to determine the static initiation stress for matrix cracking, The static experiments primarily served to analyze the cracking behavior under static loading conditions and to define the fatigue stress levels for the cyclic experiments. Each lay-up was tested at four different load levels for any of the three stress ratios. The maximum applied stress, ented in ), are prk-th image, the sum of the crack lengths was normalized by the area of observation in terms of2. The definition of crack density used here is therefore in accordance with previous research [Considering the fatigue experiments, the tests were automatically paused ten times within every decade of load cycles in order to take the images. The specimens were not unmounted from the testing machine and the experiments were continued automatically after the image was taken. In accordance with static testing, an initial image was taken before any cyclic loading was applied. Consequently, a set of 41 images per fatigue test was analyzed by the proposed detection method. To determine the crack density, research ,9,16.The results of the experiments obtained by automated crack detection provide insight into the general influence of fiber orientation, layer thickness, and load level on the cracking process in terms of cyclic loading. Further details of the analysis and evaluation of the matrix cracking state by TWLI are provided below.One goal of the presented research is to facilitate and partially automate the data acquisition process for counting matrix cracks during static and fatigue experiments. Compared to manual analysis, this is advantageous because of the user-independent quantification of the crack density, potential reductions in the time required for experiments, more precise measurement of data, and prevention of possible misalignment due to repeated specimen removal and mounting.The image acquisition procedure is based on transmitted white light imaging (TWLI), which is only applicable to transparent materials, e.g., glass fibers with a translucent resin. Several approaches for automated matrix crack counting in translucent materials have already been reported in the literature ,19,37,38Image acquisitionDisplacement compensation and image normalizationAnisotropic image filteringThresholding and crack measurementIn the course of the cracking process, internal surfaces are generated which deflect transmitting light. The cracks appear as dark lines, whereas undamaged regions remain bright. Unfortunately, bonding yarns, which can act as crack nuclei, and matrix cracks may grow along or underneath them, appear darker than the surrounding crack free areas as well. In this particular case, it is nearly impossible to detect the matrix cracks from a single image by either observation with the naked eye or by conventional image processing methods, as depicted in To improve the general visibility of the cracks, the images must be taken while the specimens remain loaded in tension during static and fatigue loading, respectively, to ensure opening of the cracks. As a consequence, the cracks appear wider, and even small cracks are visible. The corresponding reference images were taken at the same level of tension loading.The specimens elongate in the course of the experiments due to specimen damage and creep as well as to uncertainties in the control loop of the testing machine, leading to residual displacements. Using image compensation, the corresponding displacements of related pixels within the image, Matlab (available as of Matlab R2014b) is used to estimate the required transformation. While the details of the image matching process are out of the scope of this study, they can be found in [x- and y-direction for each pixel within an image Pixel-based displacement compensation is performed for the entire image instead of for distinct areas, and does not require the identification of distinct features within the field of view as long as displacements are small. Therefore, a non-parametric image registration technique, proposed by Thirion  and alrefound in ,40. The Finally, the matched image, As a result, differences between the images, e.g., matrix cracks and delaminations, are highlighted, whereas similar regions, e.g., bonding yarns and crack-free areas, are attenuated, as depicted in Gabor filtering [In order to distinguish between cracks in different orientations, the images are filtered with respect to the fiber direction. iltering ,37,41 haGabor filter emphasizes discontinuities within the filtered images; hence, it highlights the cracks that are perpendicular to the filtering direction. Regarding specimens with two layers in the same orientation (The Otsu’s method [Otsu’s method defines an individual threshold for each image and might change with increasing amount of matrix cracks. From the authors’ point of view, a change in the threshold value depending on the crack density is not reasonable, and such a detection procedure might omit small cracks in the latter course of the experiments.In order to separate the cracks from the image background, a thresholding procedure based on the grayscale values within the magnitude images is applied to obtain a black and white image. After normalizing the pixel values of the magnitude images to a range of 0 to 1, the cracks are determined by setting the threshold to 10% of the maximum value. All pixels with lower values are treated as matrix cracks. Other thresholding methods, e.g., s method , might bGlud et al. [regionprops function in Matlab, and the individual crack length is calculated from the major axis length divided by Finally, the individual cracks have to be extracted from the black and white images. d et al.  used a pThe aforementioned procedure is performed for every image taken throughout the experiment with a fixed set of parameters that is calibrated once. All images are evaluated in the same way, ensuring comparability between different experiments. The validity and sensitivity of the developed procedure are checked in the next section.It is not possible to validate the procedure by manual crack counting from the original images because of crack propagation along the bonding yarns, which is nearly invisible to visual inspection. In this case, manual crack counting would lead to erroneous results and crack densities might be greatly underestimated. Hence, manual crack counting was carried out by means of the normalized images, where the cracks are visible more clearly. Additionally, micrographic imaging was used to prove the growth of cracks along the bonding yarns.First, the cracks were analyzed along the specimen edges for certain specimens. The specimen edges were polished after testing and micrographs were taken from the whole specimen edges along the area of observation. To avoid crack closure due to the polishing process, the specimens were cut in two parts and embedded in coloured resin for polishing. As illustrated in The number of cracks counted along the specimen edge cannot be directly compared to the crack density evaluated by TWLI, because for angle-ply laminates the crack direction is not perpendicular to the specimen edge, and it is not clear whether the cracks span the full specimen width. However, the micrographs in In a second step, the cracks within two different laminates were counted manually from the normalized images to further check the accuracy of the algorithm. Therefore, five normalized intensity images, It should be noted that the task of manual crack counting is cumbersome and significantly affected by the user’s expertise. Very good agreement between manual and automated crack counting was achieved in the case of a cracked 90° layer for all of the five analyzed images. Regarding the thin and thick layers within the In the case of The results of the static LUR procedure are shown in None of the specimens showed a crack density higher than eclining a to neareclining b to expoeclining c can be Fikry et al. [The same behavior can be observed when analyzing the stiffness degradation due to crack formation, as depicted in y et al. .From the static LUR experiments, the stress for crack initiation, Carraro et al. [Following o et al.  the fatiThe detrimental effect of shear stresses on crack initiation, as discussed in ,34, was In n = 452,062 load cycles due to widespread delamination and subsequent ply splitting of the 0° layer. All other specimens represented in The highest stiffness loss was measured in the case of 90° off-axis layers, whereas the lowest stiffness loss was found for the In The crack density evolution is very sensitive to a high number of influencing factors. The effects of the load level, load ratio, fiber orientation, and ply thickness are summarized in From Regarding the influence of the load ratio, it was found that high load ratios cause delayed crack growth and lead to lower crack densities at the end of the tests cf. b. ThrougIn both cases, specimens having a load ratio of The detrimental effect of additional shear stresses on the initiation and growth of cracks can be seen from lure cf. b,c.Variations of the ply thickness affects the growth of microcracks within cyclic tests as well. From It is known from the literature that adjacent cracks begin to interact when their distance is small ,24. UsinCrack spacings for several averaged crack densities were measured manually from the images of cracked angle-ply laminates acquired by the automatic procedure to investigate the different crack growth behavior in thin and thick layers. The averaged crack densities assume that all cracks have the same distance to their neighbors, hence, they have equidistant spacings. This research addresses whether cracks grow in areas of equidistant crack spacings or not, which is of high importance for any fatigue damage model to accurately perform stress or energy calculations when crack growth relations are established .Therefore, the crack distance, 2 L, of two neighboring cracks next to a growing crack was measured for ten individual cracks from every image under consideration, with each image corresponding to a certain equidistant crack density. The results shown in Liu and Nairn [f, in the range of 1 to 1.44 in order to appropriately account for crack formation at high crack densities.In nd Nairn , who useDespite the particularly high scatter of the data, we found that the cracks tend to grow within bigger crack spacings. This clearly makes sense, as the effect of crack shielding is less pronounced within these regions and thus the normal and shear stresses within the uncracked regions between two cracks gain higher levels and promote fatigue crack growth.Krasnikovs and Varna [In examining the transition point nd Varna  have preThe research presented here investigated the damage behavior of Therefore, a TWLI methodology for the detection of matrix cracks was used in order to allow for the layer-wise determination of crack densities and the detailed analysis of individual crack growth. The validity of this method was checked by manual crack counting and microscopic analyses. This method is able to extract cracks that are situated next to bonding yarns with the same orientation as the cracks, and is therefore superior to manual counting.The fatigue results were presented in terms of SN-curves for crack initiation, crack density evolution vs. load cycles, and cyclic stiffness loss. It can be stated that high load ratios lead to delayed crack initiation for each of the tested fiber orientations, while shear stresses promote crack initiation under fatigue loading. This is in agreement with previous findings by .Based on our crack density evolution results, several effects which influence damage behavior under fatigue loading could be analyzed.Increasing the maximum applied stress leads to faster and more pronounced evolution of cracks in fatigue loading due to higher stresses and resulting higher strain energy within the off-axis plies. The observed crack densities after High load ratios lead to less crack growth, even when the specimen is loaded with the same maximum stress, which is an effect of the smaller range of elastic strain energy stored in the material within a cycle. It is therefore recommended to adopt the range of the strain energy release rate, Fiber orientations other than 0° and 90° lead to shear stresses which have a detrimental effect on crack growth in fatigue, leading to higher crack densities when experiments with nearly the same normal ply-stresses are compared. On the contrary, the detrimental effect has been observed in static loading, where lower fiber angles lead to less crack initiation and growth up to specimen failure. This might be due to increased fracture toughness from the higher mode mix.Ply thickness affects the number of cracks forming in fatigue, and has little influence on fatigue crack initiation. This might be due to the bonding yarns, which can be seen as initial defects. Such defects trigger failure at very low loads and loading cycles.In case of high crack densities (tions in , and proIn the future, these results can serve as the basis for the development of an energy-based damage model for static and fatigue loading. Furthermore, the presented data can be used by other authors to establish or validate their own models.