Image quality, resolution and scanning time are critical in digital pathology. In order to create a high-resolution digital image, the scanner systems execute stitching algorithms to the digitized images. Due to the heterogeneity of the tissue sample, complex optical path, non-acceptable sample quality or rapid stage movement, the intensities on pictures can be uneven. The evincible and visible intensity distortions can have negative effect on diagnosis and quantitative analysis. Utilizing the common areas of the neighboring field-of-views, we can estimate compensations to eliminate the inhomogeneities. We implemented and validated five different approaches for compensating output images created with an area scanner system. The proposed methods are based on traditional methods such as adaptive histogram matching, regression-based corrections and state-of-the art methods like the background and shading correction (BaSiC) method. The proposed compensation methods are suitable for both brightfield and fluorescent images, and robust enough against dust, bubbles, and optical aberrations. The proposed methods are able to correct not only the fixed-pattern artefacts but the stochastic uneven illumination along the neighboring or above field-of-views utilizing iterative approaches and multi-focal compensations. The integration of digital imaging in medical diagnostics first started in radiology and, due to the advantages of the digital file, such as the possibility of sharing, integration and archiving, the same request appeared in routine histopathology. The digital revolution started with the introduction of whole slide imaging (WSI) technology in pathology. Different scanner devices creating large files have been introduced, presenting tissue structures in an adequate resolution with a high color fidelity . NowadayThe area scanning process is quite like the widely used panorama photography and the base of the applied stitching methods are common. The moving camera takes several pictures of one object following a lane path. A software composes them into a large object connecting the small parts into one integrated object. These methods typically use an oversized FOV recording process, which results an overlapping area in each fitting side of the image rectangle. There are four overlapping stripes along all sides of one FOV rectangle in area scanners which contain a few tens or hundreds of common pixel columns and rows. These areas are adequate to create the fine alignment (stitching), and this can be used also for intensity compensation .The images that come directly from the scanner camera output channel contain FOVs presenting heterogeneous intensity even though they were recorded with the same exposition time. This inhomogeneity originates from the structural heterogeneity of the tissue sample and from the optical features of the scanner hardware, such as the rapid stage movement, complex optical path or the camera and illumination characteristics ,9,10. ThThere are existing techniques in the field of digital microscopes or in other fields of science for calibrating the intensity inhomogeneities . For insIn the field of digital microscopy a few studies are researching solutions for the intensity inhomogeneities like the BaSiC tool  which isSimilar research found in the literature can be divided into two groups. Studies in the first group try to improve locally the boundaries of the neighbouring images but do not consider the wider environment. This approach is useful if we only want to compensate the images with their immediate environment, with their distant neighborhood independent from them. The other category of approach is to try to estimate background intensities and optical characteristics from a series of images and then compensate for the images. The latter group does not have local compensation, it cannot compensate the adjacent fields of view relative to each other. Our proposed solutions have both properties, and they are specialized to the field of digital microscopy.In this article we introduce five approaches which can be used as potential solutions to reduce tile effects of multi-focal fluorescent and brightfield digital slides. After implementation, we investigate the advantages and disadvantages of each approach and make attempt to find an adequate solution that meets to WSI needs.Routine breast cancer slides were used from the archive of the 1st Department of Pathology and Experimental Cancer Research of the Semmelweis University, Budapest, Hungary. Brightfield and fluorescent stained samples randomly collected. The survey was performed with the permission of the Institutional Review Board and Regional Ethics Committee of the Semmelweis University (permit no. 7/2006).TM P250 Flash and PannoramicTM P1000 digital slide scanners  that utilizes Plan-Apochromat objective with 20× (for IHC) or 40× (for FISH) magnification  and a CIS color camera  for brightfield and PCO.edge Scientific Complementary Metal-Oxide Semiconductor camera  for fluorescence image acquisition.Slides were digitized using a PannoramicThe sizes of the digital slides of brightfield samples ranged from 0.8 to 2.4 GB and the number of pixels from 50,000 × 50,000 to 100,000 × 170,000. The sizes of the used fluorescent slides were between 9 and 10 GB; the pixel numbers were around 85,000 × 75,000.The proposed methods are compensating the image channels independently. It means red-green-blue channels in case of brightfield samples and N channel in case of fluorescent slides. All slides in the validation set have 8 bit per channel bit-depth. The methods are not sensitive to whether the background intensity is black (fluorescent) or white (brightfield).In our first approach we try to compensate the tiles using the classical histogram matching algorithm. Since it calculates a function for the whole intensity range, it can match the nonlinear differences: for example, the intensity change is exponential: N intensities:in is the number of pixels with intensity i and N is the number of intensities .The histogram function for an image with cdf) [i:The Cumulative Distribution Function (cdf)  gives thh1 and h2). Following, we calculate their cumulative distribution functions (cdf1 and cdf2). For each intensity value (i1) we find i2 for which cdf1(i1) = cdf2(i2)For compensation, we calculate the histograms for the two overlapping areas hmi1On Executing the histogram matching based method we can compensate the differences which are caused by the stochastics of illumination—these differences are homogeneous along the whole area of field of views. However, if the necessary compensation is different on the parts of FOV, a more adaptive method is needed.cdf pairs.The simplest approach would be to execute the histogram matching method for each row . HoweverAs an intermediate solution we can divide the overlap area into patches, where the size of patches are small enough to compensate the differences inside the FOVs but large enough to contain sufficient information for interpolation and extrapolation.p,in is the number of pixels with intensity i in the patch p and N is the number of intensities. After we calculate the histograms for two neighbouring patches (ph1 and ph2), we can calculate their cumulative distribution functions (pcdf1 and pcdf2):We extend the histogram function for patches:phm)phm is calculated from the p patch using the pcdf1 and pcdf2 distribution functions. We extend the histogram matching for patch-based usage .In general, polynomial regression is a form of regression analysis in which the relationship between the independent variable We can combine the patched based histogram matching with polynomial regression:The polynomial regression modelThe vector of polynomial regression coefficients  is the linear interpolation (t based) value between a and b:W is the width (or height) of tiles; N is the number of patches horizontally or vertically.Let H) and a vertical (V) compensation function:T and B are the top/bottom overlapping areas of a tile. Similarity we can define the vertical case:L and R are the left/right overlapping areas of a tile.We are able to interpolate between the patches. Now, we can define a horizontal (iw) gives the linear interpolation weight  is the intensity on the input image at x,y position and dst is the compensated intensity in this position.To apply the interpolated adaptive histogram matching to the source image we use the following calculation:A brightfield scanning usually uses three channels  while in case of fluorescent illumination the channel number can be up to ten or more. Applying the proposed compensation methods, we can compensate each channel independently ,12,19.After executing the compensation, the intensity differences along the borders of tiles are eliminated, however local brighter and darker tiles are still perceptible.We can improve the algorithm using iterative approach to compensate the tiles considering their farther neighbors.During the iteration we compensate using the above methods, but in each iteration, we use simulated annealing, when we spread the compensation values to the neighbors, with decreasing strength. Applying the proper iteration steps the tiles are compensated by not only their adjacent pairs, but by using farther tiles, reaching a more global optimum.ph(i)Calculate the histogram functions for each patch of overlap areas: phmCalculate the histogram matching for the patches: prEstimate the polynomial regression for them: N iteration and continue with Step 5.Stop the loop after p’ are the patches of the tile and N is the number of patches in a tile.Calculate the average histogram matching values along the patches of each tile:w):Spread the average histogram matching with decreasing strength Compensate the tiles using the compensation function: Iteration steps:On Z layers) are different. It is possible to enhance the method with inter-layer compensation.When we compensate the layers of a multi-focal scanning independently one-by-one, we get homogeneous 2-dimensional layers. However, when we compare these layers with each other, we see that the intensities along the focal layers  pairs, but it has pairs also in Z order .. The idThe validation process was done by using 10 slides of 10 clinical cases. The slide set was selected based on image quality. Only slides with tile effect issues were selected by using manual investigation. 1200 FOVs were used from the selected slide set with an average of 2-megapixel-FOV-size. The methods were executed native resolution.(error)Average intensity error (heterogeneity)Heterogeneity of tile intensities (zerror)Intensity error between focal layers Three different aspects were investigated in the validation protocol:error) was measured in each tile, that was calculated based on the average intensity differences in the overlapping area of the tiles :The average intensity error , but we need to measure the heterogeneity between the tile intensities.One aim of the optimizations is to compensate the tiles in a way that make them look homogenous. We calculated the differences under the overlapping areas .Let oa,bdiff is the original (uncompensated) difference of a and b tiles, and let ca,bdiff is the compensated difference.Let heterogeneity is average difference ratio between the compensated  and uncompensated  tiles:c is the case index, a,b is a tile-pair of the case c, cn is the number of cases and c,a,bn is the number of tile-pairs of case c. This formula gives relative heterogeneity for the differences: before the compensation this value is 1.0 and the compensation needs to decrease this value. A lower value means that the differences after the compensation are lower.Let We can extend the error function for analyze the differences between focal layers using their overlap areas:a,bzerror is the average intensity error of a tile-pair which are located one above the other:i and j are two FOVs in a neighboring focal layers, x and y are the pixel coordinates on the overlap areas, n is the number of pixels and i,jo is the pixels of overlap area of the FOV i with the FOV j.c is the case number, i,j is a tile-pair of the case c, cn is the number of cases and c,i,jn is the number of tile-pairs of case c.The Analyzing the heterogeneity , we can Examining the errors between the focal layers , the inpThe purpose of this study was to investigate how different histogram matching-based compensation methods perform in the task of eliminating the illumination differences in a whole slide imaging system. The present results suggests that the most accurate compensation can be obtained by the patch-based polynomial regression method extended with the iterative and multi-focal approaches, however, all five methods have its own benefits.The methods are designed to reduce the perceptible intensity differences along the tile boundary. In general, using more sophisticated algorithms to do the compensation, the probability of causing undesirable artefacts at the middle of the tiles are increasing. The simplest histogram matching-based method calculates the compensation factor about the four overlapping areas of field of views, so doing a false compensation is minimal. The other end is the iterative patch-based approach, where the more parts we divide the overlapping area, the more we can reduce the differences at the tile boundaries, but the chances increase that we need to compensate intensities at the tile center which are not represented in the overlapping areas. In such cases, we need to extrapolate excessively.All five proposed methods have their own characteristics, from which we need to choose one that is flexible enough to correct the uneven intensities. For instance, when the inhomogeneities in our WSI system are caused by the stochasticity of the illumination, whose unevenness is linear and homogeneous throughout the field of views, the histogram matching method could be appropriate. However, if the strength of the inhomogeneities are different in parts of the field of view, we need to use a more adaptive approach. For some scanner configurations, it can be observed that the illuminating stochasticity is not completely random, only some FOVs are darker than others. In this case, using the iterative extension, we can eliminate these darker areas, since the local differences became visible to the algorithm considering wider neighborhood. Multi-focal extension can compensate the intensity differences between layers in multi-focus scanning, however, it should be noted that not all focus layers contain exactly the same tissue structure, they can be visually different, and its extent is depending on the sample thickness and distance between layers. If the layer distance is greater , there is an increased chance of compensating by independent tissue structures.The present study validates the field of views before and after the execution of compensation mostly by synthetic methods. There are two main reasons why we want to produce images with as little unevenness as possible: to make the doctor’s job easier and to improve the reliability of automatic and algorithmic evaluation. In both cases, the visible and detectable intensity differences decrease the accuracy. No matter how adaptive or intensity-independent is the processing, be it a traditional algorithm or even a more sophisticated convolutional neural network, if it is unaware that there can be intensity errors at a certain point of the slide, it will detect incorrectly. The effect of compensation on human and machine measurements and processing could be examined in a future study.Analyzing the results of the validation set, several ideas have been put forward to improve the proposed methods. One such optimization is when there is just a little information in an overlapping area, such as when it only contains background intensities. Then, there is a greater chance that the real tissue structure in the middle of the field of view can be compensated using a large-scale extrapolation. However, it can be detected, for example, if the intensities in the middle of FOV are considered by how much they deviate from the overlapping area. If the discrepancy is large, we need to reduce the strength of local compensation and take better account of the wider neighborhood in the iterative approach.Another improvement possibility is the dynamic determination of the number of iterations, which is currently defined and fixed at the beginning of the model execution. In case of a larger number of iterations, there is a chance to do the compensation by a more distant area that does not belong to the same tissue structure as the reference field-of-view, causing incorrect compensation. Recognition of different tissue structures is a well-studied research field in the literature, with many approaches and methods already available from traditional pattern recognition algorithms  to sophiIn addition to digital microscopy, the proposed methods can be adapted to other disciplines, in which images must be compensated to each other, improving the quality and consistency of images or samples. The compensated images may be side by side, such as in case of panoramic photography or satellite pictures, may follow one another in time, such as in a motion picture or in astronomical images, or even spatially behind each other, such as in optical coherence tomography or magnetic resonance imaging. The field of application can be very diverse, and the proposed methods are flexible enough to be transferred with little modifications.The use of tile compensation methods is critical in whole slide imaging (WSI). In this study we collect a set of digital slides with tile effect issues and after the implementation of five different compensation methods we have made a validation study to determine which models provides the best results.According to the characteristics of the collected validation set, the polynomial regression method gave the best results regarding the average compensation error validation, however considering the tile homogeneity, the iterative method can provide better results. In the case of multi-layer slides, the multi-focal optimization can enhance the quality of compensation.Finally, we can conclude that each approach can be useful for compensation of FOV in whole slide imaging systems, but in our validation study, we find the iterative approach most promising using the multi-focal extension for multi-layer slides.


The image deconvolution technique can recover potential sharp images from blurred images affected by aberrations. Obtaining the point spread function (PSF) of the imaging system accurately is a prerequisite for robust deconvolution. In this paper, a computational imaging method based on wavefront coding is proposed to reconstruct the wavefront aberration of a photographic system. Firstly, a group of images affected by local aberration is obtained by applying wavefront coding on the optical system’s spectral plane. Then, the PSF is recovered accurately by pupil function synthesis, and finally, the aberration-affected images are recovered by image deconvolution. After aberration correction, the image’s coefficient of variation and mean relative deviation are improved by 60% and 30%, respectively, and the image can reach the limit of resolution of the sensor, as proved by the resolution test board. Meanwhile, the method’s robust anti-noise capability is confirmed through simulation experiments. Through the conversion of the complexity of optical design to a post-processing algorithm, this method offers an economical and efficient strategy for obtaining high-resolution and high-quality images using a simple large-field lens. The performance of optical systems depends considerably on the design of the optical system, as aberration is a key obstacle for an optical system to reach the ideal diffraction-limited resolution. To obtain high-quality images, optical imaging systems designers must correct and balance aberrations by combining multiple lenses of different glass materials. Even if the final design of the optical systems meets the requirements, it will make optical systems cumbersome and expensive.Fortunately, the aberration correction problem can be reshaped into a computational problem to be solved after the acquisition of the image data. The acquired aberrated image is digitally post-processed using an image deconvolution algorithm to reconstruct an aberration-corrected high-quality image . This loIn recent years, a few methods have been proposed to correct optical aberrations and improve image quality by acquiring PSFs. The blind deconvolution algorithm is a widely used method for PSF acquisition ,6,7,8 thAnother way to mitigate optical aberrations operates by adding masks to the optical system to encode the wavefront, including amplitude masks ,13 and pAnother class of methods acquires the optical system’s PSF by reconstructing the wavefront. A simple method is to employ the Shack–Hartmann wavefront sensor , which cHere, we perform an optical aberration calibration and correction method of photographic systems based on wavefront encoding. To minimize the aberrations introduced by optical elements besides the given lens during the measurement process, we designed an experimental setup resembling the direct measurement method, as shown in Similar to the direct measurement method’s experimental setup, as shown in For an optical system with a large FOV, the aberration is spatially varied. We divide the full FOV into multiple small fields of view. Within a smaller field of view angle, it can be assumed that the spatially varying aberrations are invariant ,27,28. IC on the sample plane can be described byWe consider an unknown sample The wavefront transmits by Fresnel propagation to the front surface of The idealized thin lens that has a focal length The field then propagates with distance Subsequently, a mask The field distribution at the sensor plane isSetting Thus, after neglecting the constants and dropping coordinate scaling,In a small field of view range By scanning the Fourier spectrum of a sample, we can use the captured intensity images to synthesize the sample’s Fourier spectrum by the phase retrieval algorithm. In our work, the PSFs are the intensity images captured by the sensor, and the pupil function is the Fourier spectrum to be synthesized. Thus, we can use the phase retrieval algorithm to synthesize the pupil function using a series of acquired PSF intensities by scanning the optical system’s spectral plane. The aberrations in the image are then removed by deconvolution.n sets of intensity images n sub-aperture positions in the spectral plane, as shown in n PSF intensities n masks in the spectral plane; (2) we reconstruct the pupil function n PSF intensities As shown in The following three subsections present these steps in detail.n sets of intensity images n locations in the spectral plane. We then applied an image pair-based blur kernel estimation algorithm [To obtain the point spread function lgorithm  to detererration . We estierration .The update of The algorithm flow is depicted in n sub-apertures. We show the synthesis of the complete pupil function by the phase retrieval algorithm in the next section.With this algorithm, we can obtain the local PSF and the intensity information of the pupil function captured by the sensor plane under the modulation of n PSFs obtained by the local aberration recovery algorithm in the previous part are the intensity images used for reconstruction. To reconstruct the optical pupil function, we used an alternating minimization-based phase retrieval algorithm [By scanning the Fourier spectrum of the sample, the phase retrieval algorithm can synthesize the sample’s Fourier spectrum from a collection of intensity images captured by the sensor. In this paper, the pupil function is the Fourier spectrum we want to recover, and the lgorithm , and theWe aimed to solve the following problem:At each iteration (k), we performed the following three steps to update the estimate of the pupil function:(1): (2) We replaced the amplitude of (3) We updated the estimate of truction . This reBy using the algorithm in the previous section, we were able to obtain the pupil function In an optical system, the latent image rization  to obtaiTo certify the effectiveness of the proposed method, we conducted both simulated and real data experiments.0 = 50.8 mm, f0 = 500 mm, GCL-010611), an SLM, and the crude optical system. The focal length of the crude optical system was In this paper, we used the imaging simulation function of the CODEV software to investigate the effectiveness of the proposed method for reconstructing the PSF. Firstly, we built the optical system shown in Since the given optical system was not well-corrected, it had severe off-axis aberrations. The crude optical system was rotated 10° to reconstruct the PSF at the 10°field of view of the crude optical system. A series of sub-apertures were loaded on the SLM plane as shown in We used the 664 px × 664 px resolution chart in The availability of truth data in simulated experiments allowed us to further quantitatively evaluate image quality. The full reference indexes—the structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) ,35—were The quantitative evaluation results of the simulated experiments are shown in In summary, the proposed method was able to reconstruct the PSF of the optical system robustly. The aberration-corrected images obtained by the proposed method were able to remove the aberration well without the ringing effect. We could also verify that our method has good noise immunity and can accurately reconstruct the PSF of an optical system in the presence of severe noise.Our experimental setup was similar to the simulated optical system, as shown in In our experiments, the range of the scanned spectral plane was matched to the pupil diameter R of the optical system. The position of the sub-aperture was determined by the radius In our experiments, we moved the sub-aperture In the image plane, we placed a CCD sensor  and a pixel size uirement .To quantitatively verify the reliability of the results obtained after aberration correction, we first conducted experiments using a WT1005-62 resolution test target as a sample. In our experiments, we first imaged the resolution test target through a collimator to optical infinity. Subsequently, we rotated the crude optical system to collect data from multiple fields of view.Following the scheme mentioned in To further quantify the resolution improvement before and after aberration correction, the line profile of line sections before and after aberration correction is shown and compared in Two non-reference indexes—the coefficient of variation (CV) and the mean relative deviation (MRD) —were useTo further validate our method’s ability to correct aberrations when imaging complex scenes, In this paper, we propose a method for measuring PSF based on wavefront coding. By performing wavefront coding, local aberration recovery, and pupil function reconstruction, we can reconstruct the crude photographic system’s pupil function accurately. We designed simulated and real experiments for our experiments, and both qualitative and quantitative assessments confirmed that the proposed method can precisely reconstruct the given photographic system’s PSF across all fields of view under different noise conditions, which shows that our method has good universality. Finally, we can obtain high-quality and aberration-free images by performing deconvolution.Although our method works robustly on reconstructing the PSF, the method still suffers from the limitation of long measurement times. Thus, in the future, we will focus on reducing data acquisition times; for example, by using faster SLM and better scanning strategies.


As a novel infection with relatively high contagiousness, the coronavirus disease emerged as the most pertinent threat to the global community in the twenty-first century. Due to Covid-19’s severe economic impacts, the establishment of reliable determining factors can help to alleviate future pandemics. While a population density is often cited as a major determinant of infectious cases and mortality rates, there are both proponents and opponents to this claim. In this framework, the study seeks to assess the role of population density as a predictor of Covid-19 cases and deaths in Saudi Arabia and China during the Covid-19 pandemic. With high infectivity and mortality being a definitive characteristic of overpopulated regions, the authors propose that Henry Kissinger’s population reduction theory can be applied as a control measure to control future pandemics and alleviate social concerns. If high-density Chinese regions are more susceptible to Covid-19 than low-density Saudi cities, the authors argue that Neo-Malthusian models can be used as a basis for reducing the impacts of the coronavirus disease on the economic growth in countries with low population density. However, the performed correlation analysis and simple linear regression produced controversial results with no clear connection between the three studied variables. By assessing population density as a determinant of health crises associated with multiple socio-economic threats and epidemiological concerns, the authors seek to reinvigorate the scholarly interest in Neo-Malthusian models as a long-term solution intended to mitigate future disasters. The authors recommend that future studies should explore additional confounding factors influencing the course and severity of infectious diseases in states with different population densities. A novel infection with high contagiousness, the coronavirus disease Covid-19) emerged as the most pertinent threat to the global community in the twenty-first century. With a variety of transmission routes underlying the initial spread of the virus, globalization proved to be one of the key factors responsible for the unprecedented rise of infection cases across the world. The lack of evidence-based practice and empirical knowledge on the topic contributed to the ineffectiveness of certain preventive measures and delayed response in most states. As the Covid-19 pandemic paralyzed health systems and impaired national economies, most scholars focused on the short-term prospects of the encountered epidemiological crisis [9 emergedEmerging in late 2019, the novel coronavirus infection affected the entirety of the world’s population to a different degree. For the purposes of convenience and simplicity, both the infectious disease and the associated viral pathogen are addressed with the acronym Covid-19. With the first cases of Covid-19 being recorded in China’s Hubei province, the severity of the acute respiratory syndrome caused by the SARS-CoV-2 strain of coronaviruses was not immediately apparent . By JulyConsidering that countries at different stages of economic development have been reporting hundreds of thousands of Covid-19 cases and thousands of fatal outcomes on a daily basis, the situation is unlikely to be completely resolved in the upcoming months and years. By the time medical professionals started to identify instances of human-to-human transmission outside of China, the spread of Covid-19 was already out of control . The colAs evident from the presented overview, the severity of detrimental impacts associated with Covid-19 necessitates the development of comprehensive long-term solutions that would help in mitigating socio-economic consequences of future epidemiologic crises. To this end, we performed a comparative analysis of Saudi Arabia and China, two non-democratic countries that successfully curtailed the spread of the coronavirus disease thanks to strict lockdown measures. The choice is further justified by the contrast in their respective population densities—the average for Saudi Arabia is 15.6 inhabitants per sq. km, and for China it is 148 inhabitants per sq. km. Although population density is an established demographic parameter, we approached the investigation from the socio-economic standpoint by assessing the predictive value of population density for a country’s ability to cope with the pandemic [The current study begins with a literature review focusing on the development of the Covid-19 pandemic in Saudi Arabia and China which includes a detailed overview of the epidemiological, political, and economic circumstances in both countries. The subsequent subsection provides a background for using population density as a determinant of the epidemiological situations that can further inform the adoption of a Neo-Malthusian model to address further pandemics. Following the Methodology section that delineates the used quantitative techniques as well as the conceptual framework for qualitative analysis, the research continues with the Results and Discussion section. In the Conclusion section, the authors briefly outline the key findings of the study by reiterating the findings and commenting on the significance of the results in the context of the posed research question. Centered on the Arabian Peninsula, The Kingdom of Saudi Arabia is one of the largest and most economically prosperous Middle Eastern states. The population of more than 34 million is unevenly spread across the vast area of 2,149,690 square kilometers resulting in a relatively low population density of 15 inhabitants per sq. km. The Kingdom boasts a robust economy with the highest GDP figures among other Middle Eastern nations. The development of the Covid-19 crisis in Saudi Arabia began with the official report of the first confirmed case on 2 March by the Kingdom’s Ministry of Health . To curtThere were no indications grounded on concrete evidence that would have suggested that statistics presented by the Saudi government are inaccurate or misleading. Although the reports on Covid-related infection rates and mortality rates are unlikely to be fabricated, it does not mean that the data provide an accurate representation of the pandemic’s development. The key concern pertains to the fact that official reports account only for confirmed cases with the latter being closely correlated with the total number of performed PCR tests . By the From the economic perspective, the Covid-19 pandemic both directly and indirectly disrupted a variety of Saudi Arabia’s industries and sectors. While the nine-billion-dollar budget deficit in Q1 2020 was partially tied to disturbances in the global market for petroleum products, a sizable role was attributed to the coronavirus crisis . StrugglA country occupying the fourth largest landmass on the globe, PRC features a considerable population of 1.4 billion dispersed across sparsely populated rural regions and megacities such as Chongqing, Shanghai, and Beijing. The world’s most populated state is subdivided into twenty-eight administrative regions as well as four centrally-controlled metropolitan areas. Although the total population density is 145 citizens per sq. km, the density of urban centers such as Shanghai and Beijing is considerably higher and approaches 4200 and 1300 per sq. km, respectively. Due to the fact that the Republic of China and regions such as Hong Kong followed distinctly different approaches towards registering Covid-related incidents and mitigating economic consequences, the research focuses on PRC, with both PRC and China being used interchangeably throughout the paper. While China has been at the forefront of numerous major epidemics throughout the past millennia, the most recent incidents include the 1968 Hong Kong flu, the 1997 bird flu, and the 2003 SARS epidemic . The higHowever, the veracity of data and accuracy of reports presented by China’s Communist Party had been the subject of critique for the entire duration of the pandemic. Scholars and governments questioned not only China’s methods of containing the pandemic but also the validity of information on Covid-related morbidity and mortality ,21. The Although PRC’s anticipated economic growth in 2020 was estimated to be at 5.9%, Covid-19 had profound effects on all sectors including the transportation industry, tourism, banking, and others . The sinConsidering the decisive role of human-to-human transmission for spreading the coronavirus disease, population density emerges as one of the most apparent factors influencing infection rates. From this perspective, numerous experts emphasize that people living in densely populated areas are more susceptible to being infected as opposed to the inhabitants of low-density rural regions ,30. The Despite the extensive empirical evidence indicating the significance of population density as a determinant of Covid-19 spread, the findings vary from country to country and from region to region. The pandemic lasted for less than 12 months by the time of this study but a comprehensive body of knowledge started to form with regard to the interrelation of socio-demographic factors and lockdown policies . From thDistinctly from the theory outlined above, certain researchers revealed that the Covid-19 pandemic does not always follow the expected pattern of high infection rates in densely populated regions. For example, Hamidi et al.  performeDeclassified in the 1990s, Henry Kissinger’s 1974 report touched on the problem of worldwide population growth as a possible threat to the national interests of the US. The key idea of the document was that unchecked population growth in underdeveloped nations puts a strain on the global supply of food, minerals, and other resources. Whether due to possible civil disturbances or overconsumption of resources, the report recommended preemptively addressing the uncontrollable population growth by implementing population reduction strategies focused on promoting abortion, contraception, and family planning . After bThe ongoing pandemic exacerbated socio-economic challenges encountered by the populations of rapidly developing non-democratic countries such as China and Saudi Arabia. As both China and Saudi Arabia enacted strict lockdown measures, less affluent citizens lost access to critical services and products which contributed to the already-existing lack of food security. Unable to survive without stable income, foreign workers and poor citizens experienced the devastating social impacts of an epidemiological emergency first-hand ,47. In tWith high infectivity and mortality being arguably a definitive characteristic of overpopulated regions, the authors propose that a Neo-Malthusian model informed by Henry Kissinger’s population reduction theory can be applied as a control measure to control future pandemics . In thisUnder the scope of the current study, the principal goal is to investigate the possibility of a correlational relationship between population density in Chinese and Saudi Arabian regions and Covid-related cases of infections and deaths. If the authors confirm the existence of correlation as was theorized, the consequent stages would pertain to modeling the linear relationship between population density and Covid-related deaths and cases as an independent variable and dependent variables, respectively. With infection rates and mortality being perceived as a possible function of population density, the authors consider the latter’s significance for economic growth in the selected countries. Furthermore, the economic significance of the pandemic is analyzed through the lens of Kissinger’s Malthusian theory. At this stage, there is no cohesive theoretical framework for describing the spread of highly infectious viral diseases in areas with different population densities and their impact on economic growth. Moreover, conflicting views on the topic necessitate additional research presenting a comparative analysis of high-density Chinese regions and low-density Saudi regions. Considering dependent variables for the study, the authors collected data for both infection cases and deaths caused by Covid-19 per 100,000 people for both countries with regions being selected as a unit of analysis. Furthermore, the research relies on the up-to-date dataset containing cumulative confirmed cases and mortalities for each province until 4 January 2021. Although the pandemic started and reached its peak at different timeframes in the two countries, the researchers are interested in comparing the results after the pandemic has been contained to a reasonable degree which justifies the choice of cumulative records for daily infection rates/mortality rates. Different reference dates would have no impact on the results because the study is focused on static data accumulated throughout a prolonged period as opposed to dynamic records. In the case of China, the dataset for 30 provinces, municipal areas, and territories was extracted from the JHU Covid-19 Resource Center . To ensuAs the population density was chosen as the independent variable for the study, the authors calculated the demographic characteristic for each province by dividing the total number of residents by the land area of the respective province. With the province being the unit of analysis for the research, the population density represented with the number of citizens per sq. km was identified for China’s 30 provinces and Saudi Arabia’s 13 provinces. Specifically, the authors extracted the information regarding the total population for Chinese regions from the National Bureau of Statistics of China’s database, with figures being the estimates for 2017 based on national sample surveys . From thThe author’s primary goal is to identify a possible correlation between population density and Covid-related infection cases as well as mortality in administrative units of China and Saudi Arabia. After evaluating the existence and strength of correlation between variables, the following step includes modeling of simple linear regression data plots visualizing the linear relationship between the following variables: population density and Covid-19 cases in China; population density and Covid-19 mortality in China; population density and Covid-19 cases in Saudi Arabia, and; population density and Covid-19 mortality in Saudi Arabia. The obtained results are further reviewed and discussed through the lens of Kissinger’s Malthusianism theory and its importance for economic growth in low-density and high-density regions.We employed the hierarchical cluster method to facilitate the comparison of Saudi and Chinese provinces based on the infection rates and population density. The average-linkage approach to the hierarchical agglomerative cluster analysis was deemed suitable for grouping distinct administrative regions for each of the investigated countries. The chosen technique helps to identify the groups of provinces with highest and lowest infection rates with regard to their respective population density figures . The twop-value for each pair of variables [p < 0.05 is the threshold of statistical significance. The formula used for the correlation equation is presented below.The authors calculate Pearson’s two-tailed coefficient and R Square to establish nature as well as the strength of linear relationships between the two dependent variables and the independent variable. The computation is performed with the help of the standard formula for the product-moment correlation coefficient by separately using the datasets for China and Saudi Arabia. Furthermore, the null hypothesis test allows confirming the existence of a linear relationship in regard to the selected variables. After comparing the variance of variables with the help of Fisher’s transformation and establishing the standard error, it is also possible to calculate the ariables . For theApart from the main data set including all provinces and administrative regions of China and Saudi Arabia, correlation coefficients were additionally calculated for a separate model based on the average population density within both countries. In Saudi Arabia, population density is higher than the country-wide average of 15.6 people per sq. km in the following territories: Al Bahah, Al Qaseem, Jazan, Aseer, Ar Riyad, and Makkah al Mukarramah. The remaining geographic regions correspond to the vast swaths of sparsely inhabited land such as the Rub’ al-Khali dessert. The median density for the group of densely populated territories is 38.4 per sq. km, and for the group of sparsely populated it is 6.2 per sq. km. As for PRC, the Heihe-Tengchong Line was employed as an accepted approach for demarcating highly-populated and sparsely-populated provinces in China. To ensure consistency, we also included Yunnan and Jilin to the following list of regions with population density being below the country average of 148 people per sq. km: Tibet, Qinghai, Xinjiang, Inner Mongolia, Gansu, Heilongjiang, and Ningxia. The median density for the group of sparsely populated territories to the West of the Heihe-Tengchong Line is 57.41 per sq. km, and for the group of densely populated it is 373.2 per sq. km.Simple linear regression is performed to understand the underlying relationships between the chosen sets of variables for both countries. With the help of the regression line based on the standard formula for the method of least squares, the authors seek to illustrate the significance of population density as a determinant of Covid-19 cases and mortality over the observed period. The following mathematical equation underlies the process of modeling simple linear regression: Moreover, the simple linear regression provides an opportunity to model the aforementioned relationship with the intention of better understanding the role of the independent variable. To preserve the integrity of results, the authors ensure the following characteristics: normality, linearity, homoscedasticity, and independence of observations . For theThe authors rely on empirical analysis of primary and secondary data to explore the possibility of applying Neo-Malthusian models to alleviate the detrimental impacts of future pandemics. Due to the absence of scholarly consensus on the role of population density as a determinant of Covid-19 morbidity and mortality, the present research performs a comparative analysis focusing on China and Saudi Arabia, two non-democratic states that have implemented strict lockdown policies to curb the pandemic. In this study, pragmatism was the chosen philosophical paradigm that helped to structure the scholarly inquiry in regard to the data collection and analysis for the purposes of answering the posed research question. The grounded theory underlies the comparative analysis by considering both primary and secondary data to investigate the studied problem and reach a plausible solution. The authors performed a comparative analysis of Saudi Arabia and China while using the resulting findings as a basis for exploring the possibility of applying Neo-Malthusian models for addressing future pandemics. With Saudi Arabia being sparsely populated and not engaged in population reduction programs, China emerges as a densely populated country that continuously relies on policies informed by Neo-Malthusianism. Throughout the 1970s, numerous scholars revisited the ideas proposed by Malthus in his 1798 essay ,59. CulmBy the time of the research, Saudi MOH reported the following cumulative figures from the pandemic’s onset: 363,259 Covid-19 cases and 6265 deaths. In contrast, PRC confirmed 97,028 cumulative cases of Covid-19 infection and 4792 deaths by the same date. As China’s population at 1.3 billion in 2017 is substantially larger than that of Saudi Arabia with 31.8 million inhabitants, the Covid-related figures were converted to represent incidence rates per 100,000 people. In the case of Saudi Arabia, 13 emirates substantially differ in population density as well as the number of reported cases and deaths, with Jazan being the most densely populated region of the country. On the other hand, among China’s thirty administrative units selected for the research, the following cities feature the highest density: Shanghai, Tianjin, and Beijing. By employing the hierarchical agglomerative cluster method based on the average-linkage approach, we initially classified the regions of Saudi Arabia and China based on the infection rates. The division into groups facilitated the consequent clustering process based on the population density in each of the aforementioned regions. The study population of 30 administrative regions in China was divided into groups as displayed on the dendrogram see . The firThe preliminary analysis of data strongly suggests that Covid-19 infections in predominantly sparsely populated Saudi Arabia are not representative of population density figures. When excluding Hubei from the analysis, the opposite is evident in the case of China with high density being closely tied to the rise in Covid infections as seen in p-values for China’s Covid-19 infection rates and Saudi Arabia’s mortality rates to be smaller than 0.05 at the chosen significance level, allowing us to reject the null hypothesis. As the p-value for China’s mortality rates is 0.44 and the p-value for Saudi Arabia’s infection rates is 0.64, the null hypothesis cannot be rejected and results should be considered not significant as p is larger than 0.05. As evident in The following correlation coefficients are computed for independent and dependent variables for China’s administrative regions: 0.73 for Covid-19 cases indicating a moderate positive correlation and 0.14 for Covid-19 deaths indicating a weak positive correlation. Furthermore, the coefficients for Saudi Arabia are as follows: −0.15 for Covid-19 cases indicating a weak negative correlation and 0.63 for Covid-19 deaths indicating a moderate positive correlation which is depicted in p < 0.05. The sole exception pertained to Covid-19 cases in Chinese provinces to the east of the Heihe-Tengchong Line which indicated a relatively strong positive correlation with significant results at p < 0.05. The assessment of R Square values does not show considerable differences after separately analyzing the data for regions with different population density averages.At the R Square values of 0.54 (for China’s infection rates) and 0.4 , the results indicate that the independent variable can partially explain the incidence of Covid-19 cases in China and the mortality rates in Saudi Arabia, respectively . Low R SAs evidenced by the results, the chosen methodology and data are prone to certain limitations that include the rapid development of the Covid-19 situation in different countries, the lack of conclusive data, and a non-uniform approach to recording and reporting Covid-related events. In this framework, a longitude time-series research would be uniquely suited to further explore the topic and provide robust evidence regarding the role of population density as a determinant of Covid-19 infection and mortality rates. Moreover, as the pandemic peaked at different periods in Saudi Arabia and China, the two countries relied on dramatically different approaches for reporting new cases, treating Covid-19 patients, and implementing anti-pandemic measures. From this perspective, future studies should account for distinctions in health systems as well as for numerous confounding factors such as pre-existing medical conditions.The data analysis indicated that population density has no significant causal relationship with Covid-19 morbidity and mortality which suggests that other factors play a more significant role concerning the Covid-19 spread. The effectiveness of lockdown policies implemented by Saudi Arabia and China, two non-democratic regimes, may obscure the actual connection between the studied variables. In this framework, the existing evidence is not sufficient to conclude whether Covid-19 is a Malthusian event or not. Nevertheless, the concepts developed by Malthus and further explored in Kissinger’s report may inform the solution to the ongoing environmental, social, and epidemiological crises. As the population growth puts a strain on the global supply of resources, this includes the ability of countries to provide healthcare services to the ever-increasing population. By considering various population reduction strategies, governments would be able to align with the requirement of sustainable development. Despite the fact that Neo-Malthusianism remains a highly controversial theory, its implementation can indirectly contribute to the resolution of pertinent problems such as food shortages and epidemics . The latConsidering that the research did not provide uniform evidence that population density has a strong positive relationship with Covid-19 cases and deaths, population control informed by Neo-Malthusian models is likely to be rather ineffective as a direct Covid control measure. As seen in the example of densely populated regions of China, the ensuing economic recovery was primarily tied to centrally-planned policies and effective quarantine measures implemented by the government. From another perspective, Covid-19 events in low-density regions of Saudi Arabia are preeminently tied to other confounding factors that should be further explored in future studies. Furthermore, both global prices for crude oil and Covid-19 are responsible for Saudi Arabia’s economic decline in 2020. As China and Saudi Arabia managed to achieve meaningful economic recovery after containing the coronavirus, Kissinger’s Malthusian theory of population reduction is likely to be ineffective as a control measure for future pandemics. Population reduction approaches should be further explored after identifying other major determinants for Covid-19 infection and mortality along with population density.The Covid-19 pandemic emerged as the most significant crisis of the twenty-first century that continues challenging health experts and economists across the world to find suitable solutions to the ever-increasing number of epidemiological, social, and environmental concerns. Due to the infection’s rapid spread across the globe, the existing evidence and knowledge base are constrained by the lack of data. In this context, the pandemic’s economic impact remains largely understudied as many potential determinants of Covid-19 infection and mortality rates remain hidden. The role of population density as a factor contributing to the course of an emerging infectious disease is somewhat controversial with several authors presenting contrasting viewpoints on the topic. The current study did not uncover a meaningful and conclusive causal relationship between the studied variables suggesting that population density does not explain variations in Covid-related mortality and infection rates. Although Covid-19 does not appear to be a Malthusian event, the possibility of using Neo-Malthusian models to address future pandemics should be explored further. Furthermore, the findings indicate that the role of population density could be obscured by other factors such as statistical inaccuracies and lockdown policies. There is a need for additional studies on the importance of strict lockdown policies in non-democratic nations as an effective anti-pandemic measure. In the case of population control, Neo-Malthusianism remains a viable solution to the problems posed by the uncontrolled population growth that should be further investigated. Although a moderate positive correlation was observed between the independent variable and China’s infection rates and Saudi Arabia’s mortality rates, the results are inconclusive due to the potential impact of other factors. In the case of China and Saudi Arabia, the analysis of Covid-19 cases and deaths in comparison to the population density of different regions did not reveal any conclusive evidence that population control can be used as a Covid control measure. However, the analysis indicates that Neo-Malthusian models could be considered by non-democratic governments to implement long-term policies that would facilitate economic recovery in the post-pandemic period. The research reveals that the severity of health crises such as the Covid-19 pandemic is closely tied to an array of social and economic problems. To ensure food security and sustainability consistently threatened by overpopulation, the authors recommend to prioritize scholarly investigations of Neo-Malthusian models as possible solutions intended to mitigate future disasters and health crises. Future research is necessary to investigate population control policies as a possible approach to alleviating the economic impact of global pandemics.


We present a new architecture to address the challenges of speaker identification that arise in interaction of humans with social robots. Though deep learning systems have led to impressive performance in many speech applications, limited speech data at training stage and short utterances with background noise at test stage present challenges and are still open problems as no optimum solution has been reported to date. The proposed design employs a generative model namely the Gaussian mixture model (GMM) and a discriminative model—support vector machine (SVM) classifiers as well as prosodic features and short-term spectral features to concurrently classify a speaker’s gender and his/her identity. The proposed architecture works in a semi-sequential manner consisting of two stages: the first classifier exploits the prosodic features to determine the speaker’s gender which in turn is used with the short-term spectral features as inputs to the second classifier system in order to identify the speaker. The second classifier system employs two types of short-term spectral features; namely mel-frequency cepstral coefficients (MFCC) and gammatone frequency cepstral coefficients (GFCC) as well as gender information as inputs to two different classifiers (GMM and GMM supervector-based SVM) which in total leads to construction of four classifiers. The outputs from the second stage classifiers; namely GMM-MFCC maximum likelihood classifier (MLC), GMM-GFCC MLC, GMM-MFCC supervector SVM, and GMM-GFCC supervector SVM are fused at score level by the weighted Borda count approach. The weight factors are computed on the fly via Mamdani fuzzy inference system that its inputs are the signal to noise ratio and the length of utterance. Experimental evaluations suggest that the proposed architecture and the fusion framework are promising and can improve the recognition performance of the system in challenging environments where the signal-to-noise ratio is low, and the length of utterance is short; such scenarios often arise in social robot interactions with humans. Voice modality is central to most creatures, particularly in human interactions among themselves. It is envisaged that it is also sensible that social robots communicate with humans through voice. There are three related yet different challenges that researchers are currently engaged towards realization of voice as a medium through which robots and humans interact: Who is speaking? What is being said? Which language is being spoken? This paper is concerned with the first question. Humans are better in person recognition by face than voice . HoweverSpeaker recognition systems can be classified into text-dependent speaker recognition and text-independent speaker recognition. The former system is employed when the speaker is cooperative and willing to pronounce fixed utterances as an authentication password or prompted by the system to pronounce pre-defined phrases that have been already registered in the system during the enrolment process. This scenario is readily used in speaker authentication systems where they extract signature(s) from a fixed utterance pronounced by an unknown speaker and verify if this utterance corresponds to the claimed identity. Even though the text-independent speaker recognition can be used as a speaker verification system, the system demonstrates its advantages in other applications where the speaker needs to be identified from unconstrained utterances as in the case of identifying a speaker by a social robot in a social human–robot interaction scenario. Constrained utterances used in text-dependent speaker recognition systems limit the functionality of the system in some applications such as social robot, speaker diarisation, intelligent answering machine with personalized caller greeting, and forensic investigation of telephone conversations. A speaker identification system for social human–robot interaction, which is the main motivation of this research, should be able to extract a voice signature from unconstrained utterances while maintains its performance at challenging scenarios including various low signal-to-noise ratio and short length of utterance as well as different types of noise. However, most of the state-of-the-art speaker recognition systems, including those based on deep learning algorithms, have achieved significant performance on verification tasks which are not suitable for social human–robot interaction ,5,6. In The background speaker model is used in speaker recognition literature extensively as a way to enhance the robustness and computational efficiency of speaker recognition system . Unlike Motivated by the above challenges, particularly those related to social robot applications, and noting that there is no single speaker recognition model that is universally applicable in different types of noise, different signal-to-noise ratios, and variable utterance lengths scenarios, and the absence of a “crystal ball” for speaker modeling, normalization, and adaptation, we propose a sophisticated architecture of speaker identification system for social robots. The proposed architecture encapsulates heterogeneous classifiers that employ prosodic and short-term spectral features in a two-stage classifier in order to integrate the advantages of using complementary features and different classifier models (generative and discriminative) in developing a robust speaker recognition system for social robots. The architecture utilizes gender information to cluster the population in the dataset into two groups  as well as deriving a speaker model from the gender-dependent model to provide strong coupling and improve recognition rate of the base classifier. Additionally, the system exploits the knowledge based-system (IF-THEN fuzzy rules), which is derived from the performance of the trained base classifier on limited size development set, to overcome the problem of limited size speech dataset which contains short utterances distorted with different types of noise.The rest of the paper is organized as follows: in Humans have the ability to recognize another person’s voice seamlessly without conscious effort. It is understood that various aspects of a person’s voice characteristics are implicitly and explicitly involved in the recognition process including spectral characteristics, prosody , and conversation-level features (lexicon and language). Analogous to humans, automatic speaker recognition systems employ various voice proprieties to recognize a person from his/her voice. These can be categorized into (1) short-term spectral features; (2) voice-based and prosodic features; (3) high-level features. The short-term spectral features are computed based on short frames in the range of 10–20 ms and can be seen as descriptors of vocal tract characteristics. Since this category of features require a small amount of data to be extracted, they fit well with real-time applications as in the case of speaker identification in social settings . In socii-vector [In the last five years, deep learning methods have been demonstrated to outperform most of the classical speech and speaker recognition systems such as GMM-Universal Background Model (UBM), SVM, and i-vector ,33. Howei-vector . In addii-vector . Hence, i-vector ,35,36 hii-vector based system in short duration utterance and over perform it in very short utterances (less than 2 s) [Some research studies suggest that the performance of GMM-UBM system is close to han 2 s) . Anotherhan 2 s)  and synthan 2 s) ; voice ci-vector with a combination of the two prosodic i-vector representation. Some prosodic features, particularly pitch frequency F0, have demonstrated excellent performance in gender classification tasks [The high-level features use a speaker’s lexicon  to extract a signature that characterizes a speaker. Some research studies show that this category of features is robust against channel variation and background noise, but it requires substantial computational cost and is difficult to extract ,39. Addion tasks . Gender on tasks . Second,on tasks  demonstron tasks . Construon tasks . A combion tasks . AdoptinThe contribution of this study within this context is presentation of a novel architecture of a speaker identification system for social robots that employs prosodic features as well as two types of spectral-based features (MFCC and GFCC) in order to enhance the overall recognition accuracy of the system. The system works in two stages; in the first stage, a binary classifier exploits the superiority of prosodic features to infer gender information and reduce the size of gallery set by clustering the speaker population into two groups. In the second stage, the outputs of four classifiers are fused in a novel way to improve the overall performance of the system; particularly in the case of short duration utterances and low SNR which is a common condition in speaker identification in social settings.The proposed architecture of the speaker recognition system consists of two classifiers working in a quasi-parallel fashion. The overall architecture of the system is depicted in The proposed speaker identification system works in two stages: first, the prosodic feature vector is used to determine if an utterance originated from a male or a female speaker. A binary SVM is trained using a prosodic feature vector to classify the utterance into two classes . The proposed architecture incorporates the outcome of the first stage (gender classification) into the second stage where MFCCs and GFCCs feature vectors that are extracted from the same utterance are used to derive speaker-dependent GMM from a pre-trained gender-dependent GMM-UBM. The gender-dependent GMM-UBM is trained by utterances originating from specific gender groups of speakers . The speaker-dependent GMM derived from gender-dependent GMM-UBM shows excellent coupling to the gender-dependent GMM-UBM and requires low computational power and less time as compared with the model derived from gender-independent GMM-UBM . The resultant speaker-dependent GMMs are used to create GMM-supervectors by stacking the mean vectors of the speaker-dependent GMMs. As shown in https://en.wikipedia.org/wiki/Borda_count, accessed on 12 July 2021). In this study, the weighted Borda count method is a plausible choice as it does not require transforming all the scores of the base classifier into a common domain . Additionally, the fusion system exploits the classes ranking information of the base classifiers on the development set to compute the weights for the base classifiers . Then, for each combination of the length of utterance and the SNR, a respective rule is derived taking into consideration that each classifier is weighted by a factor proportional to its recognition rate. Additionally, the fuzzy rules consider the selected classifiers to be combined to complement each other from the perspectives of feature types and classifier model with priority to feature type.In the testing path, the feature extraction modules of all feature vectors used by the two classifiers  are initiated at the same time. Thus, there is no noticeable delay caused by this architecture  as one of its inputs in order to identify a speaker).The proposed system exploits two groups of voice-based features to identify a speaker, namely prosodic features and spectral features. The GFCC and MFCC features are adopted as spectral features. The modular representation of GFCCs and MFCCs feature extraction methods are depicted in MFCCs feature vector is determined in accordance with the notion that the frequency range of human spoken words is generally limited to 1000 Hz. Therefore, MFCCs employ linearly spaced filters at below this threshold frequency and logarithmic spaced filters at above this frequency. This implies that the filter-bank selected the most appropriate frequency range of interest, i.e., more filters in the narrow bandwidths below 1000 Hz and fewer number of filters at outside this range as shown in We then transform the resulting time-domain signal to frequency domain via the ubiquitous fast Fourier transform (FFT). As expected, the FFT process generates a fluctuating spectrum. Thus, in order to ensure that the FFT spectrum extracts the most relevant information for speaker recognition, we design the filter-bank based on the mel scale. The GFCCs feature vector is computed in the same way as MFCCs feature vector as shown in Since s. Moore  modeled The idea of ERB is adopted by Patterson et al. , to estiAs suggested by Moore , the cenThe ERB scale, which is approximately logarithmic, is defined as the number of ERBs below each frequency. This scale correlates the center frequencies with It has been documented that 90% of speaker recognition systems have employed short-term spectral features such as MFCC, LPCC, and GFCC which have been found to carry high power discriminant information for the speaker recognition task . While tIn this section, for the sake of completeness of the paper, we consider popular classification algorithms that are widely used in speech recognition and speaker identification.The well-known GMM approach was adopted to construct the first two classifiers; namely GMM-MFCC MLC and GMM-GFCC MLC. We extracted the feature vectors for the MFCC and GFCC from the speech data of all speakers in training data . For each of these feature vectors, two speaker-independent world models (a well-known UBM) were created; the first UBM is trained by the feature vector extracted from female speakers and the second UBM is trained by the feature vector that is extracted from male speakers. The UBM is estimated by training M-component GMM with the standard expectation–maximization (EM) algorithm . The UBMThe model is estimated by a weighted linear combination of D-variate Gaussian density function The training of UBM is to estimate the parameters of 256-component GMM, Assuming equal prior probabilities of a speaker, the terms One of the challenges of exploiting information in voice modality is that the utterances are manifested with varying time durations. The dimension of feature vectors depends on the time duration of these utterances; hence the resultant feature vectors from feature extraction modules  have variable dimensions. Since most of the discriminant classifiers including SVM require fixed length feature vector as input, the speaker recognition research community has discovered a way to represent these time-varying utterances as a fixed-length feature vectors. The method relies upon using the parameters of speaker-dependent GMM. A speaker can be modeled as M-component GMM either by adapting a specific speaker model from UBM-GMM using MAP or by training M-component GMM with EM algorithm independently from UBM-GMM. Deriving a speaker-dependent model by adaptation approach provides a good coupling between a speaker model and UBM-GMM. Since the UBM-GMM represents a distribution of all speakers in the galley set, this coupling is desirable. Additionally, the adaptation approach reduces the computational cost of building a speaker-dependent model and facilitates real-time response.In speaker recognition literature, supervector refers to a fixed length feature vector constructed by combining many smaller-dimensional vectors into a higher-dimensional vector. In this study, GMM supervector is constructed by concatenating d-dimensional mean vector of the M-component speaker-dependent model that is adapted from pre-trained UBM-GMM. The resultant GMM supervector with SVM is one of the most powerful discriminative classifiers with excellent generalization performance to classify any unseen data. Basically, SVM is a supervised binary classifier that aims to separate the two classes by modeling a decision boundary as hyperplane; hence, adopting SVM to solve speaker verification is sensible. In speaker verification, the task is to determine if a given utterance matches or does not match a target model (claimant identity). In the training stage, all training feature vectors that are extracted from the target speaker’s voice samples are represented as one class and the second class is represented by all training feature vectors that are extracted from the background “impostor” speaker’s voice samples. SVM maps the training vector to high-dimensional space and finds an optimum hyperplane that separates the two classes  with maximum margin. Since speaker identification is a multiclass classification problem, the well-known method One-Vs-All (OVA) SVM is adopted to extend the binary SVM to accommodate the multiclass classification task. Adopting the OVA approach, which requires constructing as many binary SVM classifier as the number of classes, fits our framework of integrating the outputs of various classifiers. The output of SVM should be represented as score vector that can be interpreted either as the degree of match between a given utterance and every speaker’s voice signature in gallery set or as the probability that a given utterance originates from every speaker in the gallery set. The outputs of multiclass SVM that are constructed by the OVA approach can be expressed as probabilistic outputs; hence, the OVA is adopted to construct multiclass SVM classifier. The probabilistic outputs are used to rank the classes and compute the Borda count value for each class.In the proposed speaker recognition system, there is no need to unify and transform the outputs of the base classifiers to common domain. However, the outputs of the base classifiers should be expressed as scores (represent the degree of support) that are used to rank all the classes in descending order. The outputs of SVM, which are mostly expressed as a label for the predicted class that a test sample is assigned to , are not compatible with our fusion system. Therefore, the method suggested by Platt  is used refer to , and to refer to ,61 for mA large number of biometric authentication systems have adopted fusion of information at the score level to improve the overall performance of authentication systems. These systems employ various biometric modalities, different classification architectures, and feature extraction approaches. The key is not only to generate a set of feature vectors that complement each other but also to develop classifiers with satisfactory performance in diverse and challenging conditions. Here, the scores of the four classifiers  are integrated using weighted Borda count method. It is worth noting that these classifiers represent both generative and discriminative approaches which intuitively can be seen as highlighting similarities and differences between classes, respectively. The weight factors are computed on the fly using a Fuzzy rule-based inference engine. The knowledge base of the fuzzy inference system is represented as IF-THEN fuzzy rules. These rules are derived by studying the recognition rate of the aforementioned classifiers as a function of SNR and the length of utterance.If the length of utterance is short and signal-to-noise ratio is very low , then the weight of GMM-MFCC MLC is high and the weight of GMM-GFCC MLC is very high and the weight of GMM-MFCC supervector SVM is very low and the weight of GMM-GFCC supervector SVM is low.If the length of utterance is medium and signal-to-noise ratio is very low , then the weight of GMM-MFCC MLC is very high and the weight of GMM-GFCC MLC is low and the weight of GMM-MFCC supervector SVM is high and the weight of GMM-GFCC supervector SVM is very low.If the length of utterance is short and signal-to-noise ratio is low , then the weight of GMM-MFCC MLC is low and the weight of GMM-GFCC MLC is very high and the weight of GMM-MFCC supervector SVM is very low and the weight of GMM-GFCC supervector SVM is high.In order to study the recognition behavior for individual classifiers as a function of SNR and the length of utterance, the TIDIGITS corpus  was diviConsidering five membership functions for input 2  and three membership functions for input 1 (length of utterance), 15 IF-THEN rules are derived for each type of noise. One of the shortcomings of this approach is that for each type of noise, a set of IF-THEN fuzzy rules need to be derived.The TIDIGITS database was used in this research study. TIDIGITS is a speech dataset which was originally collected at Texas Instruments, Inc., Dallas, TX, USA. The corpus was collected in a quiet environment and digitized at 20 kHz. The TIDIGITS corpus contains 326 speakers categorized into four groups  each pronouncing 77 digit sequences. Only men and women groups were used in the experiments. Speech signals of 40 speakers  out of 225 were randomly chosen in this study. The database was selected to emphasize that the proposed system was trained on limited speech data and that is not necessary to have phonetically rich utterances and at the same time is easy to pronounce. The main point is to facilitate collection of a similar database in social robot environments, such as robots working with speech impairment person or robots working with people with developmental disability as well as robots working in assisted living and hospitals ,63,64. AThe results suggest that the recognition rates of the base classifiers that are used within the proposed architecture  outperform that of the same classifiers when they are used independently (without the first-stage classifier). However, the recognition rates of these base classifiers and consequently the overall recognition rate of the proposed system is highly affected by the performance of the first classifier (gender classification). Additionally, the outcome of fusing all the base classifiers within the proposed architecture outperforms the performance of the best of the base classifiers at low SNR and matches the performance of the best of the base classifiers at high SNR. The proposed fusion system exploits the knowledge about the strengths and the weaknesses of the base classifiers in order to improve the overall performance of the system. The knowledge about the strengths and the weaknesses of each base classifier at different combinations of SNR and length of utterance is used to increase the contribution of the strong classifier and to reduce the contribution of the weak classifier, consequently improving the overall performance of the system. For instance, when the speech signals are distorted with white noise, the GMM-GFCC classifier outperforms all other base classifiers at low SNR and short utterance. On the other hand, at low SNR and long utterance the GMM-MFCC MLC classifier is superior to all other base classifiers. Thus, more weights are given to these classifiers when they encounter similar conditions at test time. The weight factors are governed by fuzzy rules that are derived relying upon the performance of the base classifiers as discussed in In the absence of a unique robust speaker identification system that demonstrates superior performance for applications where the system is expected to perform in challenging scenarios such as different types of environmental noise, at different levels of environmental noise (low SNR to high SNR), and with only access to short utterances (at test time), the plausible contention is to integrate the advantages of using a multi-feature speaker recognition system with a multi-classifier speaker recognition system. In this study, two types of speech-based features  and three powerful classifier systems  are incorporated within an elegant architecture to identify the speaker and his/her gender as a byproduct. Exploiting prosodic features to cluster the population into two groups reduces the population size and builds a strong coupling between the speaker-dependent model and the UBM. The reduction in population size as well as deriving the speaker-dependent model from the gender-dependent model, improves the recognition rates of the base classifiers and reduces the computational cost. The recognition rates of the base classifiers , were improved by maximum , , and  when evaluated on short utterances distorted with white noise, pink noise, and street noise, respectively. Moreover, combining the base classifiers at score level by assigning weights proportional to their performance at different conditions (combinations of SNR and length of utterance), improve the overall recognition rate of the proposed speaker recognition system particularly at low SNR and short utterance.


Goal: To present a framework for data sharing, curation, harmonization and federated data analytics to solve open issues in healthcare, such as, the development of robust disease prediction models. Methods: Data curation is applied to remove data inconsistencies. Lexical and semantic matching methods are used to align the structure of the heterogeneous, curated cohort data along with incremental learning algorithms including class imbalance handling and hyperparameter optimization to enable the development of disease prediction models. Results: The applicability of the framework is demonstrated in a case study of primary Sjögren's Syndrome, yielding harmonized data with increased quality and more than 85% agreement, along with lymphoma prediction models with more than 80% sensitivity and specificity. Conclusions: The framework provides data quality, harmonization and analytics workflows that can enhance the statistical power of heterogeneous clinical data and enables the development of robust models for disease prediction. I.Nowadays, there are several significant and challenging open issues in healthcare. Examples of such open issues include the sharing and interlinking of clinical data from different clinical databases Everything starts from data sharing. The data sharing process must take into account legal and ethical issues which are posed during sharing sensitive personal data with the platform according to the existing data protection regulations, such as, the General Data Protection Regulation (GDPR) in Europe Medical data harmonization overcomes the structural heterogeneities through the identification of lexically or conceptually similar terminologies between two or more heterogeneous clinical datasets. This is most commonly achieved by lexically and/or semantically matching the terms of the heterogeneous datasets using a reference model which serves as a gold standard Once the data are harmonized they need to be co-analyzed. The conventional approach is to integrate the harmonized data under a common database and apply machine learning to deal with the unmet needs in various medical domains, such as, the development of disease prediction models. A centralized database, however, is prone to privacy breach and computationally inefficient in the case of big data, where the memory and processing requirements are demanding In this work, we present a complete framework for medical data sharing, curation, harmonization and federated data analytics. The legal and ethical compliance of the data sharing process is ensured through a data sharing assessment mechanism. A medical data curation workflow is applied on the clinical data to deal with outliers, inconsistent fields and missing values. The curated data are then harmonized using lexical and semantic matching methods based on a disease-oriented ontology. The harmonized data are finally stored in private cloud spaces and co-analyzed using incremental learning algorithms to address a clinical unmet need related to lymphoma prediction in primary Sjögren's Syndrome (pSS) using clinical data from four European cohorts. The outcomes of the case study yield highly qualified and harmonized data with more than 85% agreement along with robust lymphoma prediction models with more than 80% accuracy, sensitivity and specificity.II.A.We acquired anonymized clinical data from four European cohorts on primary Sjögren's Syndrome (University of Athens (UoA); Harokopion University (HUA); University of Pisa (UNIPI); University of Udine (AOUD)). The cohort data (Supplementary Table I) were shared with the platform under the data protection agreement version 3.7 as of August 2018 according to the Article 35 (3) (b) of the GDPR fulfilling all the necessary ethical and legal requirements for data sharing. In brief, pSS is a chronic inflammatory autoimmune disease causing salivary gland dysfunction, affecting primarily women near the menopausal age B.The cohort data curation workflow  was applC.The cohort data harmonization workflow  was applTo demonstrate the consistency of the harmonized cohort data we applied Principal Component Analysis (PCA) on each harmonized cohort dataset, separately, as well as, on the integrated dataset and extracted the first two principal components (PCs) as those that describe the largest portion of variance within the data. The distributions of the two PCs from each harmonized cohort against those from the integrated cohort, are depicted in D.To deal with the high imbalance between the lymphoma and non-lymphoma groups (Supplementary Table I), the number of controls was set as twice the number of lymphoma cases per training cohort by downsampling the majority class so that a 1:2 ratio, with age and sex matched controls, is maintained between the case group and the control group (Supplementary Table IV). Due to the small population and in order to take advantage of the statistical power of the whole population on each cohort, the incremental learning process was repeated 5 times using different subsets of controls for the training process each time (Supplementary Table IV).The incremental learning workflow  was applThe performance evaluation measures include the accuracy, sensitivity, specificity and area under the curve (AUC) scores, which are depicted in Supplementary Table V, averaged across 5 runs. According to Supplementary Table V, the XGBoost algorithm  outperforms the rest of the incremental learning algorithms. The Multinomial Naïve Bayes has the second best performance along with the Multi-layer Perceptron. The performance of the Logistic regression and the Support Vector Machines was significantly low in terms of sensitivity. The ROC curves per incremental learning algorithm are depicted in To further enhance the clinical findings of the case study we have induced the decision tree from the XGBoost schema which includes the features that highly participated in the decision-making process . The feaIII.In this work, we present a computational framework that deals with: (i) the establishment of a GDPR compliant data sharing process, (ii) the enhancement of the quality of the clinical cohort data, (iii) the harmonization of the structurally heterogeneous curated cohort data, and (iv) the application of distributed data analytics workflows on the harmonized cohort data which are stored in private cloud spaces. The applicability of the framework was evaluated on a set of four European cohorts yielding highly-qualified and harmonized cohort data with more than 85% agreement between them along with the development of a lymphoma prediction model having more than 85% accuracy, sensitivity and specificity towards the precise identification of lymphoma cases across highly imbalanced cohort data on pSS. Detailed information regarding the methodology that was used for cohort data harmonization and distributed lymphoma prediction, as part of the data harmonization and distributed data analytics modules , respectThe establishment of the DCC ensures the legal and ethical compliance of the shared medical data through the in-depth exploitation of the data protection impact assessment documents and the signed consent forms, among others. The value of the data curation workflow was presented in a previous study The data harmonization strategy is semi-automated and requires a reference model as input. This strategy is in line with the majority of the state-of-the-art data harmonization tools, such as, the BiobankConnect software An incremental learning strategy is also presented, as part of the distributed data analytics module, which is able to sequentially build machine learning models across harmonized cohort data that are stored in private cloud spaces without the need to construct a centralized repository and thus overcomes data security issues and high-performance computing requirements. The value of the incremental learning strategy is demonstrated in the pSS domain, where there exists a significant lack of lymphoma prediction models due to the rareness of the disease, as well as, due to the increased imbalance between the lymphoma and non-lymphoma pSS patients, a fact that hampers the development of robust lymphoma prediction models for effective disease management. In this work, we extend the incremental learning process from a previous study According to Supplementary Table V, the logistic regression, Support Vector Machines, and Multilayer Perceptron are less robust due to the escalations that occur around the minimum values a fact that introduces large variations in the loss function and thus in the sensitivity of the final model. This issue, however, is not present in the case of the Multinomial Naïve Bayes, which uses the maximum a-posteriori (MAP) rule for solving the cost minimization problem, as well as, in the XGBoost trees, which deploy tree ensembles to solve a cost function with a regularization parameter that avoids overfitting.In addition, the fact that the number of lymphomas in the HUA cohort is limited affects mainly the training procedure, because of the large class imbalance that is introduced. Besides, in order to deal with this large class imbalance , random undersampling of the majority class was performed, in a ratio 1:2 with age and sex matched controls. Thus, due to the small number of lymphoma cases in that cohort, the ROC curves are steeper.The clinical outcomes are in line with previous findings IV.The proposed computational framework provides data quality, data harmonization and data analytics workflows that do not only enhance the statistical power of the heterogeneous clinical data through data curation and harmonization but also applies incremental learning to co-analyze the harmonized data towards the development of robust prediction models for the disease of interest. The framework can be generalized to any clinical domain, as long as, a reference ontology is provided as input towards the construction of a medical corpus which will in turn enable the application of lexical and semantic matching algorithms. As a future work, we are planning to include more harmonized cohort data on the private cloud spaces to further enhance the statistical power of the lymphoma prediction models in the pSS domain, as well as, interlink the corpus with other medical index repositories and include deep learning algorithms in the distributed data analytics module to enhance the robustness of the disease prediction models.V.A.The overall analysis workflow is depicted in The latter uses lexical and semantic matching algorithms to align lexically and conceptually similar terms between each cohort dataset and a disease-oriented ontology and consists of a six-step sequential process, including: (i) the construction of a reference model, (ii) the transformation of the reference model into a disease-oriented ontology, (iii) the extraction of terminologies from the disease-oriented ontology, (iv) the definition of a medical corpus, and (v) the application of lexical and semantic matching to align the terminologies between the input dataset and the corpus.Finally, federated data analytics methods are applied for the analysis of the harmonized cohort data in an incremental manner to deal with the unmet needs in the clinical domain of interest through the application of incremental learning algorithms which include additional functionalities for hyper parameter optimization and class imbalance handling. The outcomes of the framework include high quality clinical data, disease-oriented ontologies, harmonized clinical data, machine learning models for patient stratification and disease prediction.B.The data sharing assessment module is responsible for the evaluation of the regulatory compliance of the clinical data with the requirements which are posed by the GDPR for data sharing and data protection C.A medical data curation workflow, which was already described in a previous study D.Data harmonization depends on the disease under discussion in this work which is the primary Sjögren's Syndrome (pSS). The proposed strategy, however, can be extended to other medical domains as long as a reference model is available. The data harmonization process is a six-stage process  reference model construction, (ii) ontology construction, (iii) lexical matching, (iv) semantic matching, and (v) incremental learning which are part of the cohort data harmonization and distributed data analytics modules followed by related pseudocodes and results.