The environment of underground coal mines has challenging properties that makes this zone inadaptable for a stable communication system. Additionally, various deteriorating physical parameters strongly affect the performance of wireless networks, which leads to limited network coverage and poor quality of data communication. This study investigates the communication capability in underground coal mines by optimizing the wireless link to develop a stable network for an underground hazardous environment. A hybrid channel-modeling scheme is proposed to characterize the environment of underground mines for wireless communication by classifying the area of a mine into the main gallery and sub-galleries. The complex segments of mine are evaluated by categorizing the wireless links for the line-of-sight (LOS) zones and hybrid modeling is employed to examine the characteristics of electromagnetic signal propagation. For hybrid channel modeling, the multimode waveguide model and geometrical optic (GO) model are used for developing an optimal framework that improves the accessibility of the network in the critical time-varying environment of mines. Moreover, the influence of various deteriorating factors is analyzed using 2.4 GHz to 5 GHz frequency band to study its relationship with the vital constraints of an underground mine. The critical factors such as path loss, roughness loss, delay spread, and shadow fading are examined under detailed analysis with variation in link structure for the mine. Underground coal mines have a complex and unfavorable environment with inappropriate conditions to establish a stable communication system. Traditional communication techniques are less sophisticated for underground coal mines and conditions including uneven structure, rough walls, large mining equipment, non-uniform permittivity, and irregular geometry . The undOres inside the mines are fixed in the ground, but available in variable chunks over a long distance; hence, the mining activities are spread over the entire mine. Therefore, to cover the whole mine is a complex task for a communication system. The existing underground mine system provides limited monitoring of sites, and cables are the major source of information exchange . As wireConcerning the significance of underground communication, although a large body of research exists for safe communication, several problems require immediate attention. One of the primary concerns is a safe and efficient wireless communication system to realize fast-track flexible sensor access, and larger coverage to underground monitoring areas ,6. PhysiThis study investigates the influence of several deteriorating factors on a wireless link using the 2.4 GHz to 5 GHz frequency band in underground coal mines.The characteristics of electromagnetic signal propagation are investigated for complex segments of a mine using line-of-sight (LOS) links.A hybrid channel-modeling approach is proposed for communication in underground coal mines using a multimode waveguide model and geometrical optic (GO) model.Performance evaluation of the proposed approach is done using several critical factors such as path loss, received power loss, delay spread, and shadow fading.The rest of the paper is organized in the following manner. This section briefly describes the conventional communication techniques and potential challenges in underground mines that complicates the development of an efficient channel model for wireless communication.The underground communication network (UCN) consists of wireless or wired devices that are either placed in a bounded open space of tunnels or completely buried under dense soil . A briefIn the harsh environment of underground mines, a fixed infrastructure of wires can always be the source of problems for mine workers. Lacking a stable wireless communication framework, signal transmission is carried out with the aid of cables called through-the-wire (TW) communication . CoaxialIn some cases, hybrid systems are used that consist of leaky feeder cables that radiate signals for data reception . HoweverThe through-earth (TE) scheme is data transmission using the propagation of EM signals through the surface of the earth. These signals penetrate in depth through the multiple layers of underground mines; however, such propagation weakens the power of EM signals . Even soPath loss is one of the major concerns for propagated signals in underground mines. Communication channels share several characteristics in the critical environment of mines, which leads to difficulty in reliable operation. As the signal propagates away from the transmitter, extreme path loss is observed in the gallery of a mine due to irregular geometric effects . The linEnvironmental channel characteristics in mines change irregularly, which complicates the design of an efficient channel model. Frequent movement of convenient communication equipment along with variations in the prime channel of mines leads to the production of Doppler frequency shifts and fast signal strength fluctuations . Signal One of the main factors that causes interruption of communication signals is the eruption of flammable gases. A risk of gas blast increases when the concentration of such gases exceeds a particular threshold. Therefore, continuous evacuation is required to decrease the probability of any disaster. Moreover, the explosion or any disaster can affect the propagation characteristics of EM signals and leads to a complete link failure . LOS linThis section describes various analytical models for characterizing wireless propagation in underground mines. Existing models for channel analysis used in underground mines are briefly discussed, and a summary of their benefits and drawbacks is given in Tunnel structure in underground mines can guide EM waves throughout their path and this type of EM wave propagation has been modeled as a waveguide . In the For smaller-wavelength operations in underground mines, the GO model is useful to estimate the characteristics of the propagated signal in complex environments. Unlike the single-mode waveguide model, it can be used to investigate branches of mines for signal propagation. It can provide accurate results when the dimensions of the environment are larger than its wavelength. This means that it can satisfy the higher UHF-band frequencies for propagation loss in underground mines. The use of this model is based on optical ray theory, where EM waves used for signal transmission are considered to be optical rays . The disIn the environment of underground mines, the multimode model can be used to estimate the signal propagation characteristics for the near and far regions. This model is valid for multiple frequencies, and the operating frequency in this model can be higher than the cut-off frequencies of propagation by increasing communication frequency . The proThe full-wave model is an effective scheme to estimate channel characteristics for underground mines. This approach is considered valid for analyzing the degrading effects of reflection, diffraction, and refraction in tunnels with a complete solution for signal propagation in complex areas . It is cThis section describes the development of a hybrid channel-modeling scheme by distinctly using the multimode waveguide and GO model for underground mines. The complete flow structure of channel modeling is shown in This research categorizes the areas of the mine into different categories. The tunnel of the underground mine is considered to be the mine gallery, and its branches are represented as sub-galleries, as shown in It is very important to consider the structure of the mine before developing its mathematical model. Although mines consist of different portions, in this case the general cross-section area for signal propagation is in rectangular form . In thisThe transmission of EM signals in the mine can be observed in the form of a superposition of various modes with varied distributed field patterns and attenuation coefficients. The complete distributed EM field patterns of every mode can be derived by sorting Maxwell’s expression in the form of Eigen-function , as giveThe distribution field is calculated against any position ectively ,33. The The waveguide model declares the lower-order mode in the environment of underground mine as The entire distribution field on the excitation plane is obtained by summing up the contribution of the source and total reflected images. In a rectangular-shaped mine gallery, this geometry exhibits the protocol that vertical walls reflect the image p or q is even and a −ve sign for p or q represents the odd.Assume that the incident angle for the horizontal floor or ceiling is Equation , where aThe parameters The ray sum is reorganized from Equation  and spliAnd the function The parameters It is noteworthy to point out that every segment of Equation  reflectspression (18)∑p,qThe coefficient The Saddle-point methodology is used to obtain the result of integration, as the study is focused on the field Similarly, the sum of complex modes in Equation  can be oEquation  is the aEquation (22)Cmn=The field can be computed at a point within the mine gallery by substituting Equations (4), (7), (8) and (22) into Equation . Let Pt B and central frequency For wideband transmitted signals, distortion is caused by dispersion in the environment of underground mines which can be a source of inter-symbol interference (ISI). This work investigates the power delay profile (PDP) by characterizing the propagation channel. Let a signal f as described belowThe strength of each signal can be obtained by taking a Fourier transform of signal The mode intensity parameter t at any position As shown in Equation , the groThe interior area of a mine is usually large and irregular, which causes reflection for the transmitted signals. The impact of reflected signals from the vertical sidewalls is nominal but the reflection due to the ceiling/floor is significant. A specific approach is adopted in this study to model this structure as a planar air waveguide and the same procedure is used to compile a multimode model as used in the mine gallery .y-coordinate is affected, and in this case, the incident angle is observed to be 0°. The reflection coefficient is X-polarized and Y-polarized. If the transmitting node and reception node is placed at the height of y, respectively, then the EM field at the reception node can be computed asIt is observed that the scenario of images and reflected rays are still considerable here, as only the X-polarized modes is obtained from (42(4238](Equation (43)BERBIt can be observed that in the case of a mine sub-gallery, the typical pillars of irregular dimension and the random distribution of pillars is a source of reflection and diffraction for propagated signals. Each mode experiences independent and distributed shadow fading, as a single mode comprises a bunch of rays with an identical angle. Therefore, the net field at any location can be obtained by adding up the field of all modes , as giveBy compiling distance-dependent path loss from Equation  and the The shadow-fading parameters vary in different zones of the mine, and this variation shows a lognormal distribution. The cumulative distribution function (CDF) is drawn, and the probability of its deviation is shown in This study approaches the problem of efficient channel modeling for the adverse and erratic environment of underground coal mines. It reveals that different areas of an underground mine need to be characterized separately for electromagnetic signals. It is very important to efficiently model the entire environment of a mine for the development of a stable wireless network. The areas of a mine are classified into main gallery and sub-galleries, and a hybrid model scheme is used for channel modeling of complex mines. The EM wave propagation characteristics are defined by the multimode model, and field intensities are estimated with the aid of the GO model. The received power and various evaluating parameters are computed to examine the effectiveness of the suggested model. The complex segments and branches of the mine are classified by defining the line-of-sight, non-line-of-sight, and partial-line-of-sight zones for managing the optimized signal strength in every zone of an underground mine. The environment of the mine rapidly changes due to mining activities, and the materials that occupied the mine are a big source of deterioration for wireless communication. A detailed evaluation of these degrading elements is conducted in this research, which elaborates that the irregular structure of a mine, the roughness of its boundary walls, the dielectric medium of a mine, and equipment used for mining applications cause major disruption for signal propagation and network decay. Operating frequency is the most important parameter in a wireless network and, after a detailed examination of 2 GHz to 5 GHz band frequencies in different zones of the underground mine, it can be concluded that a lower-frequency signal in the UHF band attains low path loss, but captures high refraction loss and roughness loss. The overall performance of the 2.4 GHz frequency is much better than the 5 GHz frequency in the harsh environment of underground coal mines.


Background and Objectives: Aortic arch disease is still a high-risk surgical challenge despite major advances both in surgical and anesthesiological management. A combined surgical and endovascular approach has been proposed for aortic arch disease treatment to avoid hypothermia and circulatory arrest in high-risk patients. Materials and Methods: Between June 2004 and June 2021, 112 patients were referred to our department for aortic arch surgery; 38 (33.9%) patients underwent supra-aortic debranching and endovascular treatment. Of these, 21 (55%) patients underwent type I aortic arch hybrid debranching procedure and in 17 (45%) patients a type II aortic arch hybrid debranching procedure was performed. None of the patients were emergent. Results: No intra-operative deaths were recorded. In the type I aortic arch hybrid debranching patients’ group, one patient died at home waiting the endovascular step, one developed ascending aortic dissection and another one developed a pseudoaneurysm at the site of the debranching at follow-up. In the type II aortic arch hybrid debranching patients’ group, left carotid artery branch closure was detected at follow-up in one patient. Thirty day/in-hospital rates of adverse neurological events for both the surgical and endovascular procedures were 3% for minor stroke, with no permanent neurological deficit and 0% for permanent paraplegia/paraparesis. In 100% of the cases, the endovascular step succeeded and the type Ia endoleak rate was 0%. Conclusions: Hybrid arch surgery is a valuable option for aortic arch aneurysm treatment in patients with high surgical risk. The choice of aortic arch debranching between type I or type II is crucial and depends on anatomic and clinical patient characteristics. Further larger scale studies are needed to better define the advantages of these techniques. Aortic arch diseases are a hard entity to treat particularly in acute settings. In the last decade, the introduction of technical improvements and new surgical procedures, such as frozen elephant trunk, have improved the outcomes of open repair of the aortic arch and reduced mortality . HoweverFollowing the advent of transcatheter endovascular aortic repair (TEVAR) a combined vascular and endovascular approach has been proposed for aortic arch disease treatment to avoid hypothermia and circulatory arrest.The hybrid arch concept is essentially a supra-aortic debranching to create a proximal landing zone in Ishimaru  arch zonBased on Bavaria and co-authors’  classifiBetween June 2004 and June 2021, 112 patients came to our attention for aortic arch surgery. Conventional surgery was performed in 74 (66.1%) cases, while the hybrid approach was preferred in the remaining 38 (33.9%) because of cardiovascular, neurologic or pulmonary comorbidities causing high surgical risk. Patient characteristics are shown in Standard general anesthesia is always associated with non-invasive monitoring of cerebral oxygen saturation by near-infrared spectroscopy for continuous cerebral monitoring during the procedure. Usually a median sternotomy is performed, but also a mini-sternotomy at the third intercostal space as an inverted T shape allows good exposure and mobilization of the ascending aorta and supra-aortic vessels.For type I arch debranching, heparinization is started to reach a coagulation activated time longer than 250 s. After reaching a mean blood pressure of 80 mmHg by pharmacological therapy or by ventricular pacing, the ascending aorta is tangentially clamped with a side-biting clamp and the proximal part of a bi- or trifurcated Dacron vascular prosthesis  is sutured end-to-side to the aorta. Preoperative CT scan, transesophageal echocardiography and digital inspection are the tools to determine the position on the ascending aorta for proximal anastomosis, in order to avoid atherosclerotic plaques and to obtain an adequate landing zone for the TEVAR. Once the proximal anastomosis is completed, the first limb of the branched graft is usually anastomosed directly end-to-end to the proximal left subclavian artery when it is easily reached directly. Keeping a mean arterial pressure of over 100 mmHg for a better cerebrovascular perfusion, the second limb is then anastomosed end-to-end to the left common carotid artery and finally, the third limb of the prosthesis is anastomosed to the innominate artery, in the same fashion, .The procedure is mostly completed by reinforcing the proximal aorta, immediately after the origin of the main trunk of the new supra-aortic vessels. The reinforcement is generally 4 cm long and the very proximal part of the bi-trifurcated graft, that is not used for the debranching, is opened longitudinally and wrapped around the aorta ,6. This For type II arch debranching with ascending aorta replacement, patients are placed on cardiopulmonary bypass. The arterial cannulation is performed via the ascending aorta or via axillary artery; axillary artery cannulation is performed by interposition of a Dacron graft and is more often utilized. It allows the extension of the ascending aorta replacement more distally, just proximally to the origin of the brachiocephalic trunk and, moreover, if a brief circulatory arrest is mandatory, antegrade cerebral perfusion is guaranteed by right carotid and vertebral artery perfusion. The right atrium is cannulated for venous drainage. Once the aorta is clamped as distally as possible, the ascending aorta is replaced with a straight Dacron tube graft with a pre-anastomosed bi-tricfurcated vascular prosthesis on its very proximal part. If an open distal main graft body anastomosis is required, it is not critical that an aggressive hemiarch has to be performed, as it will be covered by the stent graft. In case of concomitant surgery, as when an aortic root or valve replacement or coronary bypass is needed, the concomitant surgery is performed as the first part of the procedure. Then the epiaortic vessel debranching is performed individually, starting with the left subclavian artery .The 2 to 4 limb graft origin should be right above the sinotubular junction to allow a longer landing zone and it is important that the branched graft portion sits antero-laterally, at around 10 o’clock, on the ascending aorta. This orientation allows the displacement of the limbs to the epiaortic vessels anterolaterally (between the aorta and the vena cava) to secure the grafts away from the sternum. To avoid any possible kinking the right pleura can be opened to allow more space for the branched grafts.The left subclavian artery can sometimes be difficult to reach due to aortic arch aneurysm lateral displacement and it is always prone to dissect or more fragile than the other epiaortic vessels. A preemptive elective carotid-to-subclavian bypass can be a good option in case of an unfavorable anatomy at preoperative CT scan. This procedure can be performed before the hybrid arch repair or immediately after the debranching in case of unexpected unfavorable anatomy. The proximal left subclavian artery is covered with the deployed stent graft in the aortic arch. It can also be proximally ligated during the hybrid arch procedure or coiled endovascularly to prevent a type II endoleak. Another surgical option is to tunnel the first limb to the mid-portion of the subclavian artery and an end-to-side anastomosis is performed through a 4-cm subclavicle incision. In some cases, the left subclavian artery can just be sacrificed without a carotid subclavian bypass, and the stent graft may provide an adequate seal without a type II endoleak. To better visualize the proximal and distal ends of the Dacron tube graft or the reinforcement of the ascending aorta (the landing zone of the future TEVAR) during the endovascular step, we mark each end with radio-opaque thread markers passed around and fixed on to the reinforcement. Another marker is placed around the origin of the branched grafts for a precise deployment of the stent graft immediately above it.The TEVAR implant is usually done as a second-stage procedure. At the beginning of our study the second endovascular step was delayed until after the full recovery of the patients, from fifteen to thirty days after the surgical stage; since the sudden death of a patient waiting for the second step, the TEVAR is usually implanted one or two days after surgery, when the neurological status and hemodynamic status are stable. In the case of poor peripheral artery access, the stent graft is deployed during the surgical step (single step procedure) in an antegrade fashion through a slave limb of the branched grafts on the ascending aorta. Occasionally, a slave conduit via the common iliac artery is used for an endograft implant. In the case of type I debranching, we always performed a control CT scan after the aorta bending in order to check the new landing zone diameter; TEVAR is usually over-sized by 10% to 20%. The endovascular step was performed under general anesthesia, in the angiographic room, after femoral artery exposure or slave conduit preparation. Ventricular pacing was used to obtain a temporary hypotension for a precise deployment of the endoprosthesis. A pre-curved stiff wire, also commonly used for TAVI, is placed in the left ventricle apex for a better stabilization of the endograft in the ascending aorta during its deployment. The radio-opaque markers positioned at the extremities of the aorta bending and around the origin of the new supra-aortic trunk helped to accurately identify the optimal landing zone, avoiding branching graft occlusion. An occlusion balloon was always used to expand and better adapt the proximal part of the TEVAR to the aortic anatomy. Aortic reinforcement also protects native aorta against rupture or dissection even in case of balloon overinflation. Cerebrospinal fluid drainage was positioned before the endovascular step in the presence of previously treated abdominal aortic aneurysm or non-transposed left subclavian artery. The first cases were treated with the Endofit stent graft ; subsequently we used the Gore TAG stent graft , the Talent stent graft and the Valiant endovascular stent graft . Patients underwent serial contrast CT scans before discharge at 3, 6 and 12 months, and annually thereafter. All the patients were affected by aortic arch disease involving zones 0–2 according to the Ishimaru arch map . Nine  was noted with bulging of the endograft, but without any endoleak; this patient was treated by implanting a new TEVAR (Gore) to cover the bulging part and prevent possible future endoleak .Aortic aneurysms are diagnosed more and more frequently due to better imaging and screening tools. Twelve percent of aortic aneurysm > 6 cm will rupture, without treatment, in one year and fifty percent of these patients will die within five years if they only receive medical treatment ,8. The gold standard of therapy for patients with extensive aorta arch pathology is still nowadays, surgical . Arch reThird generation thoracic endoprosthesis, more conformable to aortic anatomy and specifically designed for deployment on curvature, together with specific endovascular techniques such as appropriate C-arm angulation to avoid parallax, use of a stiff wire leaning on the aortic valve and short period of hypotension, reduce proximal endoleak. In our study none of the patients treated had a type IA endoleak immediately after TEVAR implantation or at follow-up. Unique anatomic (angulation and supra-aortic vessels origin) and hemodynamic features (high blood flow and considerable motion) of the aortic arch have made this portion of the aorta the Achille’s heel of TEVAR for a long time. In one patient, treated with first generation endoprosthesis, we found, at five year CT scan follow-up, a fracture of part of the stent endoprosthesis without fabric rupture, probably due to the above mentioned reasons. However, improvements in endograft technology, anatomical customization and embolic protection will expand the use of endovascular arch repair, and as we have seen with the transcatheter aortic valve, a similar endovascular revolution may soon come to the aortic arch. From the first clinical use of stent grafts for abdominal aorta aneurysm repair in 1991 , many stAt this point, given the fact that the aortic arch restitutio-ad-integrum is only possible with surgery and a minimally invasive approach is already feasible for high-risk patients, we wonder what could be the role of hybrid surgery. Surgery is the ideal solution, but it is onerous in terms of procedural risk. Although intriguing, the totally endovascular approach is not free from complications (primarily of embolic origin or related to possible retrograde dissection) and some doubts still remain, mainly concerning connective tissue disorders, the possible progression of aneurysmal disease at the level of the proximal landing zone with potentially fatal consequences and the long-term follow-up. For these reasons, type II debranching is preferred whenever the patient can afford a short cardiopulmonary bypass for ascending aorta replacement. In this regard, we should deepen the aspect of the greater pulsatility and flow of the aortic arch and its dynamic strain that could have some consequences in terms of stent patency, migration, kinking and fracture. For all these reasons, despite device improvement and technical progression, total endovascular treatment is nowadays, still limited to a small group of patients with prohibitive surgical risk. Obviously, as in all new techniques, this fact does not allow, at the moment, a fair comparison between open and endovascular surgery and large-scale studies are needed to fill the gap in evidence but we must keep in mind that minimally invasive approaches do not necessarily translate to minimal risk outcomes. In our experience the hybrid approach with debranching technique has proved to be safe and effective and early results are promising. Aorta reinforcement for type I debranching is a simple, quick and cost-effective trick. Obviously, a careful preoperative planning is mandatory. Left subclavian artery revascularization, metachronous procedures (debranching and TEVAR) maintaining high arterial pressure during and immediately after the surgical step and the cerebrofluid drainage in case of extended endoprosthesis, contribute to reduce the paraplegia/paraparesis risk. In our series, we have not experienced any paraplegia/paraparesis mainly because all the previous technical aspects were adopted in most of our procedures and only in case of antegrade TEVAR implant, the endovascular step was done simultaneously to the surgical one. Of course, stroke remains a major concern in any kind of open, hybrid or endovascular aortic arch treatment strategy and it can be seen as the major important challenge to address in the years to come. Despite careful attention during surgical and endovascular steps we experienced, in our group of patients, one minor stroke in the thirty-day postoperative period. Open, hybrid and endovascular approaches constitute, together, a spectrum of possibilities from the more anatomical to the less invasive, a continuum into which each patient should be inserted in the right place in order to find the solution that allows maximum benefits with minimum risk. This is possible only with a careful and customized evaluation, case by case, and with a deep knowledge of the different techniques, materials and innovations. Modern cardiac surgeons have to acquire and master interventional and vascular skills so that they can control the whole range of possibilities in the grey zone of the aortic arch pathology and offer the patient the best solution. At the moment, it is science fiction but who knows if in the future, with the improvement of technology and techniques, the surgical risk will be reduced to the point of considering type II debranching a routine option for young patients with ascending aorta aneurysm and initial aortic arch dilation.Hybrid surgery is still a valuable option for aortic arch aneurysm treatment in patients with high surgical risk. It allows the exclusion of the diseased aortic wall by means of endoprosthesis after the creation of a safe proximal landing zone, thus avoiding both the risk of cardio-circulatory arrest and the uncertainties related to the total endovascular approach. Further large-scale studies are needed to better define the application fields of each technique.


The early diagnosis of autism spectrum disorder (ASD) is highly desirable but remains a challenging task, which requires a set of cognitive tests and hours of clinical examinations. In addition, variations of such symptoms exist, which can make the identification of ASD even more difficult. Although diagnosis tests are largely developed by experts, they are still subject to human bias. In this respect, computer-assisted technologies can play a key role in supporting the screening process.This paper follows on the path of using eye tracking as an integrated part of screening assessment in ASD based on the characteristic elements of the eye gaze. This study adds to the mounting efforts in using eye tracking technology to support the process of ASD screeningThe proposed approach basically aims to integrate eye tracking with visualization and machine learning. A group of 59 school-aged participants took part in the study. The participants were invited to watch a set of age-appropriate photographs and videos related to social cognition. Initially, eye-tracking scanpaths were transformed into a visual representation as a set of images. Subsequently, a convolutional neural network was trained to perform the image classification task.The experimental results demonstrated that the visual representation could simplify the diagnostic task and also attained high accuracy. Specifically, the convolutional neural network model could achieve a promising classification accuracy. This largely suggests that visualizations could successfully encode the information of gaze motion and its underlying dynamics. Further, we explored possible correlations between the autism severity and the dynamics of eye movement based on the maximal information coefficient. The findings primarily show that the combination of eye tracking, visualization, and machine learning have strong potential in developing an objective tool to assist in the screening of ASD.Broadly speaking, the approach we propose could be transferable to screening for other disorders, particularly neurodevelopmental disorders. Autism spectrum disorder (ASD) has been described as a pervasive developmental disorder characterized by a set of impairments including social communication problems, difficulties with reciprocal social interactions, and unusual patterns of repetitive behaviors or interests . During Early diagnosis may lead to early intervention, which generally proves beneficial for both the child and the family. The diagnosis process usually involves a set of tests that can require hours of clinical examinations or is based on an interview with the parents. Furthermore, the variation of symptoms with regard to deficits in social communication and social interaction as well as the social communication impairments and restricted, repetitive patterns of behavior make the identification of ASD more complicated to decide. In this respect, computer-aided technologies have been embraced to provide helpful guidance through the course of examination and assessment. Examples include magnetic resonance imaging, electroencephalography , and eyeThis study provides a meeting point for eye tracking and machine learning (ML) for supporting the diagnosis of ASD. It is part of an interdisciplinary collaboration between research units of psychology and artificial intelligence at the University of Picardy Jules Verne in France. Our approach is distinctively based on the premise that visual representations of eye-tracking recordings can effectively serve as features for discriminating individuals diagnosed with ASD. At its core, the key idea is to compactly render eye movements into an image-based format while maintaining the dynamic characteristics of eye motion  using color gradients. In this manner, diagnostic-related tasks can be approached as a problem of image classification or analysis. The applicability of the proposed approach will be demonstrated based on the classification accuracy. Further, we will support our results with a statistical analysis that will explore possible correlations between the Childhood Autism Rating Scale (CARS)  and the Eye tracking has been used in numerous research studies. It can be described as the process of capturing, tracking, and measuring eye movements or the absolute point of gaze (POG), which refers to the point where the eye gaze is focused in the visual scene . The sigEye trackers aim to capture 3 basic categories of eye movements: (1) fixation, (2) saccade, and (3) blink. A fixation is the brief moment that occurs while pausing the gaze on an object so that the brain can perform the perception process. The average duration of fixation typically ranges from 150 ms to 300 ms . HoweverAbundant studies have sought to take advantage of eye-tracking applications for studying and analyzing eyes movements. For instance, a team of psychologists and neuroscientists recently showed that children with ASD have faster eye movements than do children with typical development, but these results depend on the visual task the children are asked to perform. If they are faster while remaining precise in prosaccade tasks with a gap paradigm, the same children are less accurate but faster than are children with typical development in another gap paradigm during short visual search. This means that children with ASD favor speed over accuracy and that they have shorter saccadic latencies . MoreoveML is subfield of computer science involved in providing computers the ability to learn without being explicitly programmed . In contThe coupling of eye tracking with ML is currently leveraging further capabilities for advancing ASD diagnosis and its applications. The literature includes several contributions in this context. For instance, Pusiol et al  worked oOther recent studies have focused on predicting the visual attention of children with ASD. For instance, Wei et al  proposedCompared to the literature, the main distinction of this paper is that it is purely reliant on the visual representation of eye-tracking scanpaths. The study aims to produce scanpath visualizations that can represent the spatial patterns of gaze behavior and its dynamics. In this way, the vision-based approach allows for approaching the diagnosis problem as a typical task of image classification and is a continuation of our earlier work ,27. Our A group of 59 children took part in this study. It was highly desirable to have participants at an early stage of development, as the principal goal was supporting the early detection and diagnosis of ASD. Specifically, all participants were school-aged children of a mean age of about 8 years. This somewhat advanced age was indispensable here because in our region there were not enough diagnosed children younger than 6 years, and the time it takes to consult a doctor to make a diagnosis can be as long as 2 years. For the group of typically developing children (non-ASD), parental reports of any possible concerns were carefully considered.The ASD diagnosis was confirmed by health professionals using standardized tools . However, we did not get permission to read the children’s medical files. ADI-R and ADOS-G scores were not analyzed in this study. The participants were broadly organized into 2 groups: (1) diagnosed with ASD or (2) non-ASD. Children diagnosed with ASD were examined in multidisciplinary ASD specialty clinics. The intensity of autism was estimated by psychologists using the French version of the CARS , while cAll the children’s parents or legal guardians were informed of the objectives of the study, the nature of the tasks that would be administered, and the fact that they could withdraw their agreement at any time. Their informed consent was received in writing in accordance with the Declaration of Helsinki of June 1964 . Moreover, all children gave their agreement to participate, and if they wished, parents could be present with their children in the experimental room. This study did not require authorization from an ethics committee based on the recommendations for psychological research in France and in agreement with the national and institutional guidelines.The SMI RED250 remote eye tracker  was the main instrument used to perform the eye-tracking function. The device belongs to the category of screen-based eye trackers. It can be conveniently placed at the bottom of the screen of a desktop PC or laptop. In our case, a 17-inch monitor with a 1280 x 1024 resolution was used.Further stimuli were presented from the SMI Experiment Center software. Stimuli represented multiple distinct types used in the eye gaze literature. Examples included static and dynamic naturalistic scenes with and without receptive language, joint attention stimuli, static face or objects, and cartoon stimuli. The average duration of eye-tracking experiments was about 10 minutes. Participants were mainly examined for the quality of eye contact with the presenter and the level of focus on other elements. A 5-point calibration scheme was used. The calibration routine was followed by a set of verification procedures.The participants were invited to watch a set of photographs and videos, which included scenarios tailored specifically to stimulate the eye movement across the screen area. Participants could be seated on their own or on their parents’ lap at an approximately 60-cm distance from the display screen. The experiments were conducted in a quiet room at the university premises. Physical white barriers were also used to reduce visual distraction.The scenarios varied in content and length in order to allow for analysis of the ocular activity of participants from different perspectives. In general, videos were designed to include visual elements that are especially attractive to children . Specifically, the stimuli presented are part of various psychological studies. One of these studies involves the presentation of 3 videos including a situation of joint attention initiation (duration of 58 seconds per video) and 18 photos from the same situation (5 seconds per photo). The scene presented in the video and corresponding photos started with an attention grabber . The woman in the video then said, “Hello, how are you?” to the child and looked, verbalized, pointed, and/or verbalized at a joint attention target present or absent to the children’s visual field. All conversations were performed in French as the native language of participants.The assessment of gaze following included 12 videos (4 seconds each in duration) of an actor with a neutral face first engaging in direct gaze and then shifting to 1 out of 3 objects. In 6 videos, the actor shifted his eyes and head to the target, and in 6 other videos, he only moved his eyes to the target. The same actors were engaged in another research protocol where their photo was shown for 5 seconds on half the screen next to an object. Other stimuli presented scenes with emotional valence extracted from cartoons in which the faces of the characters expressed an emotion that was either contingent or not contingent on the previous scene . Moreover, in all tests, the interstimulus interval lasted 2 seconds, during which a central crosshair was presented. The differences between the stimuli used included dynamic or static, human  or cartoon, and human or object. The counterbalancing of stimuli for participants and the number of participants included allowed the artificial intelligence and psychology teams to collaborate on the basis of this predefined research protocol. The results of these tests have been partly exploited, presented, and published -33.t, yt)x to , where t represents a point of time during the experiment. (2) The change in color across lines was used to visualize the movement dynamics. Through use of a grayscale spectrum, the color values were tuned based on the magnitude of velocity  with respect to time. (3) The images constructed were vertically mirrored since the origin was located at the bottom of the screen.The premise of this study is based on the learning of visual patterns included in eye-tracking scanpaths. Specifically, scanpaths are used as a means to compactly describe the gaze movements into a visual representation that can simplify the learning process. Further, the scanpaths were also used to visually encode the dynamics of eye motion using color gradients. To achieve this, we used the coordinates in eye-tracking records, which represented the participant's POG during the experiment runtime. Based on the change in POG over time, we were able to calculate the velocity of gaze movement. Subsequently, the scanpath and computed dynamics were transformed into images. For each participant, a set of images was constructed in 3 steps: (1) A line was drawn for each transition from  and a popular Matplotlib library . The visEye trackers can provide the POG coordinates on the screen. The coordinates were genuinely significant to implement our approach in terms of visualizing the gaze scanpath and computing its dynamics . The eye-tracking records describe the category of movement and the POG for both eyes over time. To simplify the learning process, a set of image processing techniques was applied as follows. First, the black background was cropped from images as much as possible. The cropping was implemented using the OpenCV library. Second, all images were consistently scaled down to 256 x 256 dimensions. Resizing the images helped to reduce the problem of dimensionality by decreasing the number of features under consideration. The impact of resizing was also examined in the initial ML experiments.Further, we applied image augmentation to produce variations of the scanpath images. Augmentation was recognized to generally improve the prediction accuracy in image classification applications ,36. The The ML work described here falls into the category of supervised learning. The basic goal was to develop a binary classifier that could predict the class of participant  based on the scanpath images. The classification model was implemented using an artificial neural network approach. Specifically, we designed a deep CNN.CNNs typically include 3 categories of layers including convolutional layers, pooling layers, and fully connected layers . The leaThe classification accuracy was analyzed based on the receiver operating characteristic (ROC) curve. The ROC curve plotted the relationship between the true-positive rate and the false-positive rate across a full range of possible thresholds. The model was implemented using the Keras library  with PytTo further examine the model performance, the training and test sets were split based on the participants. The data set was split into training and test sets based on a 3-fold cross-validation using 3 stepwise procedures. First, the group of 59 participants were randomly split into 2 independent sets , then the images were matched and loaded into the training and test sets based on the IDs of participants, and finally, these 2 steps were repeated for each round of the cross-validation process.The features were extracted using the convolutional layers in the CNN model. The learning process goes through a series of convolutions and pooling, which break down the input image into a set of features maps. Convolutional layers initially attempt to extract features from the image through applying a convolutional kernel all over the image. Subsequently, pooling layers work on reducing the dimensions of feature maps extracted. Eventually, the output of this process usually feeds into a fully connected layer structure to produce the final prediction. Expectedly, the model performance declined as shown in This section serves as an integral part that supports the experimental results gained by our approach. Through statistical analysis, we attempted to explore possible correlations between the CARS score and the dynamics of eye movement in the eye-tracking scanpaths. Initially, the average velocity magnitude was calculated per image. In this way, the CARS scores of participants could be considered multiple times with respect to velocity. This could help mitigate the effect of outliers in eye-tracking experiments.r) would not be useful in such a case. Instead, we made use of the maximal information coefficient (MIC) [R2). The MIC method has been embraced in a large number of studies to find correlations in complex data sets related to, for example, biology and genomics [The patterns largely revealed the nonlinearity of the relationship between CARS scores and velocity. Therefore, standard correlation tests  . The MICgenomics ,42. We ugenomics , which gA demo application was developed to serve as a practical illustration of our approach. The application links the 3 components of eye-tracking, visualization, and ML together to support the diagnosis process of ASD.The application goes through 3 steps as follows. First, the user is asked to upload the eye-tracking data. The data records should describe the coordinates of the viewer’s gaze into the screen along with the associated time. Second, the application produces a visualization of the eye-tracking scanpath. Eventually, the application calls the prediction web service, which returns the prediction from the trained classification model. Azure ML is employed to host the classification model and the Python implementation used to produce visualizations. The application can be accessed online by asking the authors for the URL link.This study demonstrated the strong potential of eye tracking as an objective tool for assisting ASD diagnosis. Indeed, abnormal eye gaze has been a hallmark characteristic of ASD ,7. Over The empirical results provided a set of implications to be considered. First, the ML experiments confirmed the core idea behind our approach, which hinges on the visual representation of scanpaths. The classification accuracy indicated that scanpath visualizations were able to successfully pack the information of gaze motion and its underlying dynamics. This evidently translates into the validity of employing such visual patterns in order to diagnose individuals with ASD.Equally important, the study brought further interesting insights into the features of autistic gaze. We provided a statistical analysis that revealed possible a correlation between the level of autism  and the dynamic characteristics of eye motion . The analysis can lend support to the findings of Vabalas and Freeth , which sEven though the results presented in this study are promising, the following set of limitations should be highlighted. The primary limitation was the relatively small number of participants. In a future study, a data augmentation method for an ASD data set may be considered . The intTo conclude, the combination of eye tracking, visualization, and ML may hold considerable potential for the development of an objective tool to assist the diagnosis of ASD. These results can be used and new data analyzed to create a screening tool for health professionals Further, features related to the dynamics of eye movement can also be considered as candidate features for developing predictive models, and recently published deep neural network methodologies can be adapted to our model . Eye-tra


In 2020, the world was affected by the COVID-19 pandemic, which remains a major challenge for most countries today. In Brazil, football clubs' youth academies have faced a disruption of their regular activities. In order to study how the learning cultures of a Brazilian professional football club youth academy have been changed, and the alternatives created by the club's staff within this context, this perspective article aims to analyze how they have structured the Under-15 (U15) team learning culture during social isolation due to the COVID-19 pandemic. Through document and thematic analysis on a Brazilian professional football club's youth academy program, we promoted a dialogue between the process of adaptation to remote theoretical-tactical teaching with the learning theory proposed by Hodkinson and collaborators. The main theme of analysis of this study was the remote structure of the theoretical-tactical learning and physical training. Challenged with the need to transpose face-to-face activities into a learning culture based on remote communication, the U15 team coaching staff created a process to prescribe physical training, and to teach and discuss football tactical issues with young players during the period of social isolation. This perspective article shows that it is possible for sports institutions to create programs for the development of young athletes within the social isolation/distancing context, considering both theoretical-tactical learning and physical training processes. The adaptation to remote environments as structures for the learning culture seems a challenge, but is also a good alternative for young players to develop their interpretation and perception of football theoretical-tactical issues. Since the beginning of 2020, the world has been facing a pandemic caused by COVID-19 , U13, U15, U17, and U20. In the state of São Paulo, the Paulista Football Federation (FPF) approved the return of professional competitions in July, 2020 FPF, . HoweverIn the specific case of Brazil, even before the pandemic, the country was already facing political and economic crises that mainly affected the poorest socioeconomic groups. As little is known about the characteristics of COVID-19 transmission in a context of great social inequality , but social and symbolic spaces with imprecise and overlapping boundaries, where people interact, constructing learning processes through interpersonal relations  consists of 27 players. In addition, there is a Department of Human Development (DHD) in this institution that offers psychological and social care for players.After the interruption of training and competitions due to the COVID-19 pandemic, and the returning of young athletes to their family homes (around 50% of players' families live in different towns than the club's headquarters), the club adjusted its procedures and ways of interaction with athletes to remote communication alternatives. At that time, several activities of psychological support from the DHD were proposed for athletes and their families, considering the new social context. In addition to this, the club's coaching staff, together with the head manager of the youth academy, proposed diversified physical and game activities for players training at home.Within the scenario of sports activities interruption, in the following we present and analyze the learning culture structure of this Brazilian professional football club youth academy U15 team, located in a countryside city in the state of São Paulo. For this, we took as main sources of data the action plan for remote activities created by the club's coaching staff  remote tactical learning activities; (b) physical conditioning exercises; (c) retrospective analysis on the players' trajectories before joining the club. The three major themes are: (1) Awareness and care; (2) Theoretical-tactical learning and physical training; (3) Career counseling and psychological support . AlthougAs the purpose of this article is to analyze how a Brazilian professional football club youth academy staff have structured its U15 team's learning culture during social isolation due to the COVID-19 pandemic, the authors' choice was to present and discuss the action plan related to the Theoretical-tactical learning and physical training.The U15's action plan and reports provided by the club's coach staff concerns ways to enable the continuity of the teaching specific football topics during COVID-19 pandemic. This document is part of the Training Curriculum of the youth academy, and aims to structure a remote learning culture considering both the social isolation conditions and the young players' sociocultural background, creating specific and particular horizons for learning . In this sense, multidisciplinary work among educators, managers, coaches, physiotherapists, psychologists, and social assistants was fundamental for the creation of a remote learning culture that could offer institutional support and suitable remote learning conditions.The support from the DHD was decisive for the coach staff planning the theoretical-tactical learning activities. Not all young players have had good access to the internet and to technological devices. Therefore, activities that depended on the players watching long lasting football game videos for theoretical-tactical reflection and collective discussion were not easily available for every player. Taking this information into account, some activities were adapted by the coach staff. As an example, young players had to read the information in the text messages shared into the group, and based on this they should draw, explain, and produce texts about the theoretical-tactical issues that have been trained before the pandemic. The main concern on this activities' adaptation was to vary the teaching and learning methods, in addition to diversifying the materials for the fulfillment of tasks, seeking to reduce the inequality of internet access among young players.Within this process, activities proposed for the theoretical-tactical learning were based on pre-pandemic lived situations during training and matches sessions. Based on videos and literature sent by coach staff, players studied and answered questionnaires related to tactical issues. Then, in remote meetings among players and coaches, theoretical tactical topics were discussed and analyzed in-depth (the Brazil is a country in a socioeconomic development process with a high level of social inequality (Graeff et al., The aim of this article was to analyze how a Brazilian professional football club youth academy staff have structured the U15 team learning culture during social isolation due to the COVID-19 pandemic. According to this, we presented how interventions and interactions between the club's staff and young players were planned and adapted into a specific learning culture during the social distancing period.Faced with the need to transpose in-person activities into remote communication, the club's coaching staff involved with the U15 team was obligated to plan and create a favorable environment to social interaction among all athletes. The work developed with the U15 football players through interactive meetings with coaches, and also the DHD interventions, offered an in-depth understanding about what were the conditions that each player was living at their homes during pandemic social isolation. In parallel to the perspective of learning as a cultural practice in the sports context, it is evident that the social structure offered by the club was strongly related to the practices performed before the pandemic. The remote activities sought to reconstruct the club's learning culture, but still shared a very close interaction among players and coach staff.Important problems caused by the COVID-19 pandemic are related to the social isolation, and consequently the transformation of the learning culture of this club from in-person relations to a remote environment. This seems to be one of the biggest challenges imposed over sports institutions during the pandemic context. However, such factors may present themselves as an opportunity for clubs to reflect on their teaching/training practices and investigate socioeconomic and sociocultural conditions lived by young players. A better understanding of contextual factors can help with the adaptation of teaching/training and support the expansion of the remote learning cultures possibilities by sport institutions.This perspective article shows that it is possible for sports clubs to create programs for the development of young athletes within the social isolation/distancing context, considering both theoretical-tactical learning and physical training processes. The adaptation to remote environments as structures for the learning culture seems a challenge, but is also a good alternative for young players developing their interpretation and perception of football theoretical-tactical issues. The possibility of creating a remote environment meets the need for social distancing imposed by the COVID-19, besides favoring new technologies that can be used as a teaching tool even in a post-pandemic world.The contribution of this work is specially related to the presentation of an adaptation process from a Brazilian professional football club youth academy to the conditions imposed by social distancing due the COVID-19 pandemic. However, it can also be considered as a first step for future studies on applications of remote activities to young players' training programs. It seems relevant that future research should focus on exploring the perspective of the social agents involved in sport programs, with special attention to young players.The original contributions presented in the study are included in the article/MM, AK, and RM: study design. BG and PM: critical friend. MM, AK, PM, BG, and RM: manuscript preparation. All authors contributed to the article and approved the submitted version.MM and AK are staff members of the club that was analysed in this case report. MM is coach of U15 team and AK is chief psychologist of the Department of Human Development. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Data-driven based rolling bearing fault diagnosis has been widely investigated in recent years. However, in real-world industry scenarios, the collected labeled samples are normally in a different data distribution. Moreover, the features of bearing fault in the early stages are extremely inconspicuous. Due to the above mentioned problems, it is difficult to diagnose the incipient fault under different scenarios by adopting the conventional data-driven methods. Therefore, in this paper a new unsupervised rolling bearing incipient fault diagnosis approach based on transfer learning is proposed, with a novel feature extraction method based on a statistical algorithm, wavelet scattering network, and a stacked auto-encoder network. Then, the geodesic flow kernel algorithm is adopted to align the feature vectors on the Grassmann manifold, and the k-nearest neighbor classifier is used for fault classification. The experiment is conducted based on two bearing datasets, the bearing fault dataset of Case Western Reserve University and the bearing fault dataset of Xi’an Jiaotong University. The experiment results illustrate the effectiveness of the proposed approach on solving the different data distribution and incipient bearing fault diagnosis issues. Currently, rotating machinery is becoming a critical component of modern industrialisation. With the continuous improvement of the safety and reliability requirements of mechanical equipment, the research on the fault diagnosis of rotating machinery has become indispensable, and rotating machinery fault diagnosis has become a major research direction in machinery fault diagnosis. In rotating machinery, rolling bearings are one of the most critical components, and bearing failures will affect the operation of the equipment and cause the failure of the rotating machinery. Wu et al.  pointed By combining different feature extraction and selection methods, the conventional machine learning based methods have achieved high classification accuracy, and various algorithms including support vector machine  and fastBoth conventional machine learning based methods and deep learning based methods require a major assumption that the training data and the testing data must follow the same data distribution. However, this assumption cannot be satisfied in the practical application. As a result, while the classifier performs well on datasets with the same statistical distribution, it struggles to achieve acceptable classification accuracy on datasets with different statistical distribution.Transfer learning (TL) based methods, in contrast to deep learning and conventional machine learning based methods, can reuse prior knowledge or a developed model to address new tasks of a target domain with different statistical distributions by minimizing the difference between different domains . As demoIn addition to statistical distribution requirements, quality and plenty of labeled training data are also required conditions for conventional machine learning and deep learning to train classifiers or models . HoweverIn this paper, an incipient fault diagnosis approach based on transfer learning is proposed to solve the above mentioned problems. The proposed approach employs the correlation alignment (CORAL) algorithm to deal with the data distribution problem in different domains, as well as a hybrid feature extraction algorithm based on statistical algorithms, stacked auto-encoder (SAE) network, and wavelet scattering network. The sigmoid entropy is applied on feature vectors after feature extraction to construct feature matrices of both domains in the proposed approach. The feature matrices are aligned on the manifold space by taking the geodesic flow kernel (GFK) method. The k-nearest neighbor (KNN) algorithm is used to classify the faulty data of the target domain for fault diagnosis. The proposed approach is an unsupervised method for rolling bearing incipient fault diagnosis, which does not require any labeled data of the target domain. The proposed approach only uses the labeled data of the source domain to train the classifier, which means the proposed approach is practical. To verify the effectiveness of the proposed methods in fault diagnosis, experiments are conducted on two rolling bearing datasets, the Case Western Reserve University (CWRU) dataset and the Xi’an Jiaotong University-Sumyoung (XJTU-SY) dataset. By comparison, the proposed approach has achieved an outstanding performance among several approaches.The main contributions of this paper are: (1) a transfer learning based unsupervised incipient fault diagnosis approach is proposed; (2) a novel hybrid feature extraction algorithm based on a statistical algorithm, SAE network and wavelet scattering network is proposed. The rest of this paper is organized as follows: To deal with the different data distribution issues associated with bearing incipient fault diagnosis, the structure of the proposed approach is illustrated in The domain adaptation method used in the proposed approach is the CORAL algorithm, which is unsupervised and can minimize the difference between domains. The CORAL is a simple yet effective method for unsupervised domain adaptation, and the effectiveness of the CORAL algorithm has been demonstrated by Sun . The CORx and u are d-dimensional, In the CORAL algorithm, it is supposed that the data ius norm .Algorithm 1 CORAL for Unsupervised Domain AdaptationInput: Data of Source Domain Output: Data of Adjusted Source The Algorithm 1 of CORAL is shown as below:The process of the CORAL algorithm is illustrated in In the fault diagnosis problem, the design of feature extraction plays a significant role in obtaining a high accuracy diagnosis result. In conventional machine learning, the feature extraction of signals is commonly based on the time domain, frequency domain and time-frequency domain . CompareIn data-driven rolling bearing fault diagnosis, one of the common strategies to extract process parameters is to analyze the mechanical vibration signals with statistical signal processing techniques ,22. The N is the number of samples for one signal and Nth root of unity [In the frequency-domain, the Fast Fourier Transform (FFT) is applied to extract the frequency-domain characteristic indicators. The FFT algorithm is an efficient method to transfer a time series into a frequency-domain representation. In this paper, four types of frequency-domain features are extracted: center-of-gravity frequency, power spectrum, frequency variance, and mean square frequency. The FFT of a signal can be computed using the following equations:of unity .In the time-frequency-domain, the wavelet packet decomposition (WPD) is used to extract the time-frequency domain features in this paper . WPD is With a wavelet convolution layer, non-linear layer, and pooling layer, the data is filtered using predefined wavelet and scaling filters in a wavelet scattering network . The strIn The auto-encoder network is an unsupervised learning method, which takes the backpropagation algorithm to ensure the output data is equal to the input data. The core idea of the auto-encoder network is trying to learn a function In this paper, the SAE network consists of two auto-encoders, and the structure of the SAE network is shown in L2 regularization term, and the sparsity regularization term, as shown below:L is the number of hidden layers, n is the number of examples, k is the number of variables in the training data, D is the number of neurons in the hidden layer, i.The cost function in training the auto-encoder network consists of three terms: the mean squared error term, the S and the target domain is transformed into point T on the manifold. The distribution of three example classes represented by a circle, triangle, and square can be adjusted through the measurement of geodesic flow, represented by the green curve in In this paper, all feature vectors of both domains are aligned on manifold with the GFK algorithm. The core idea of GFK is to construct a geodesic curve For geodesic flow, the subspace angle between the source domain and the target domain is computed first. It is supposed that (1) = PT  as(5)Φ and with statistical features only and the GFK approach. The comparison results of the five methods are provided in (a)Effect of Domain AdaptationThe experiment results in As (b)Effect of Feature ExtractionFrom the experiment results, the proposed hybrid feature extraction method combining the three feature extraction (95.56%) can increase the diagnosis accuracy by around 4% compared with that of the statistical feature only method (92.00%). This result shows that introducing deep features can improve the classifier performance and the proposed hybrid feature extraction method has significant influence on the incipient fault diagnosis result.(c)Effect of Sigmoid EntropyAccording to (d)Effect of Neighbor NumberAccording to In summary, the experiment results demonstrate that the proposed approach has achieved the best performance. Based on the results, the fundamentality of domain adaptation has been demonstrated, as both CORAL and GFK algorithms can help improve the fault diagnosis accuracy. The results also illustrate that a hybrid feature extraction method can achieve better classification performance in incipient fault diagnosis.In this paper, a rolling bearing incipient fault diagnosis method based on improved transfer learning with hybrid feature extraction is proposed. Firstly, the CORAL algorithm is adopted to achieve the domain adaption of source and target domains; secondly, the proposed approach extracts features using the statistical algorithms, the wavelet scattering network and SAE network; thirdly, the proposed approach applies sigmoid entropy on extracted features of both domains; lastly, the GFK algorithm is used to align the feature vectors of both domains on the Grassmann manifold and the K-NN classifier which is trained with the source domain and is adopted to classify samples of the target domain. The experiment results show that the proposed approach has achieved highest classification accuracy among different approaches. Therefore, the effectiveness of the proposed approach in rolling bearing incipient fault diagnosis with different data distribution problem has been verified via experiments. The possible future development of this proposed approach is to reduce the computational power as several methods are adopted to extract the domain features.


Video analytics and computer vision applications face challenges when using video sequences with low visibility. The visibility of a video sequence is degraded when the sequence is affected by atmospheric interference like rain. Many approaches have been proposed to remove rain streaks from video sequences. Some approaches are based on physical features, and some are based on data-driven  models. Although the physical features-based approaches have better rain interpretability, the challenges are extracting the appropriate features and fusing them for meaningful rain removal, as the rain streaks and moving objects have dynamic physical characteristics and are difficult to distinguish. Additionally, the outcome of the data-driven models mostly depends on variations relating to the training dataset. It is difficult to include datasets with all possible variations in model training. This paper addresses both issues and proposes a novel hybrid technique where we extract novel physical features and data-driven features and then combine them to create an effective rain-streak removal strategy. The performance of the proposed algorithm has been tested in comparison to several relevant and contemporary methods using benchmark datasets. The experimental result shows that the proposed method outperforms the other methods in terms of subjective, objective, and object detection comparisons for both synthetic and real rain scenarios by removing rain streaks and retaining the moving objects more effectively. Challenging atmospheric conditions such as rain and snow degrade the visibility of video sequences ,3,4,5,6.Rain-streaks removal research can be categorised into two cases, single image-based rain-streaks removal and video-based rain-streaks removal. The methodologies have been designed for both cases. In recent years, various methods have been proposed for rain-streaks removal for both cases: video and single image as the video sequence is a combination of sequential frames  ,21,22,23The approaches that have been proposed for Rain-Streaks Removal in Video (RSRV) can be categorised into two types. Conventional approaches mainly focus on the apposite implementation of the physical properties of the rain streaks and the prior knowledge of the background scene in the time domain and frequency domain. These methods are used to encode all the exploited features into an optimisation problem and propose algorithms to solve it rationally. In the rest of the paper, we will mention these approaches as a feature-based model. Recently, emerging approaches have been introduced based on data-driven processes, e.g., deep learning. They are mainly based on the specific network architecture using the knowledge of the previously processed relationship of images/videos with and without rain.Garg and Nayar  developeRecently some existing methods have often used a data-driven approach by designing specific network architectures and precollecting rain-free image/video pairs to learn network parameters to attain complex rain removal functions ,34. LateThis paper addresses the issues mentioned above, raised by both the approaches, data-driven and feature-based models. First, we consider a single but powerful novel feature, the temporal appearance (TA) of the rain streaks. We observed that normally rain streaks do not appear in a particular pixel position for more than a few adjacent frames. We exploited this property by defining the TA feature to distinguish rain streaks from moving object areas. However, some portions of the moving object areas have been missed out in the generated rain-free frame if we solely depend on the TA property for rain removal. The TA property mainly exploited the brief appearance of a rain streak in a pixel location among adjacent frames. Although the exploitation of this characteristic successfully distinguishes most of the rain streaks, unfortunately, it classifies the brief appearance portion of a moving object as rain. The preliminary idea is published in a conference paper .Mask R-CNN is a data-driven model which is used to segment and construct a pixelwise mask for each object in an image. The Mask R-CNN algorithm was introduced by He et al.  on top oWe introduce and formulate a novel hybrid technique to combine data-driven and feature-based models to overcome individual limitations.We develop a pixelwise segmentation strategy to distinguish between rain, moving objects and background pixels for fine-level accuracy to remove rain streaks by keeping the entire area of the moving objects.For better rain interpretability, we exploit outcomes from the deep-learning-based model with the physical-feature-based technique.We propose and formulate TA features of the rain streaks with an adaptive threshold to separate them from the moving objects irrespective of the frame rate.This paper proposes a novel hybrid technique to combine data-driven and feature-based models for better rain removal without sacrificing the quality of moving objects. The proposed algorithm fuses the prediction’s data from three sources: the Mask R-CNN model, an adaptive TA feature, and the background and foreground extraction in two separate steps. The three predicted data are combined to generate a rain-free frame. To make the proposed method effective at different frame rates, we modify the TA feature by an adaptive threshold to work in different frame rates as the duration of rain appearance in terms of frame number depends on the frame rate. The main contributions of the paper can be summarised as:The rest of the paper is organized as follows: We combined the feature-based and data-driven models to remove rain streaks from a video sequence and generate a rain-free video. ‘Fold’ refers to the foldup of each column of a matrix into the corresponding frame matrix of a tensor. The superscript T indicates the transpose of a matrix. Here, B is the data matrix, and d and n are the column and row of the images, respectively.Many dynamic background modelling approaches ,43,44 arAt each frame, we generate a background frame. We use the background frame to find rain streaks and the moving objects, in order to generate the rain-free video in the proposed method.nF is a foreground binary image of the nth frame, nI is the input nth frame and nB is the background frame at the nth frame. Here, we use an intensity threshold value of 20 to eliminate the effect of other light or illumination interference from the generated foreground binary image [Initially, we generate the foreground by subtracting the background from the input frame, and then we use an intensity threshold to generate the binary image of the foreground.ry image ,54,55. TAfter applying the background extraction, we separate the background and binary foreground of the current frame. This binary foreground contains rain streaks and other moving objects, where both are dynamic and have different characteristics. We exploited the TA feature of rain streaks to separate the rain streaks and other moving objects. It was observed that the rain streaks appear in a location discreetly, while the movement of moving objects is smooth and consistent. The duration of the appearance of rain streaks in a location is only for a few frames based on the frame rate. We developed the TA feature-based model to separate the moving objects and rain streaks based on this characteristic of rain streaks.F for each frame against its adjacent previous m number of frames to model rain streaks and analyse the temporal feature. In the binary image, ‘1’ represents the binary foreground comprising rain and other moving objects, and ‘0’ represents the background. TA object mask is predicted based on the following equations:M represents a mask of the nth frame, which contains the foreground’s appearance value, F represents the binary foreground of adjacent frames (generated by equation (2)), and m is the number of adjacent previous frames considered. The previous m number of frames is used to make the decision contemporary, as the scene may be changing significantly enough. In Equation (4), TAObj is the predicted object mask of the TA model, thD denotes duration threshold, c is duration threshold coefficient and rf denotes frame rate of the video sequence. The mask may not be relevant to represent the recent changes. We consider every pixel location’s appearance value in the mask M. If the appearance value is more than a certain duration threshold in terms of the frame rate of the video, it is considered as the part of the object area and any value more than zero and up to that duration threshold is considered as the rain area; otherwise, it is considered as part of the background area. We use the duration threshold coefficient c = 0.25 or 25% of the frame rate to classify the rain, object, and background areas. The appearance duration varies with the capturing frame rates (see explanation below). In We apply the TA characteristic of rain streaks in a modified form to separate rain streaks from the moving objects of the binary foreground. A mask is generated using the binary image thD mostly depends on the frame rate, because if the capture device is operating at a high frame rate, a rain streak will appear in a greater number of frames. Equation (4) shows the linear property of thD to the frame rate rf. This is why for better rain removal, we need to make sure the proposed method is adaptive to the frame rates, or in other words, the proposed method should be applicable for different frame rates. Thus, the duration threshold thD we have used against mask M to predict TA object mask TAObj is a function of the frame rates so that the threshold can be adaptive to the frame rate for the success of rain removal. The duration threshold The Mask R-CNN model is a Deep Neural Network (DNN)-based model. Mask R-CNN is developed on top of the previous object detection model, Faster R-CNN. Faster R-CNN is a region-based convolutional neural network . Mask R-The TA and Mask R-CNN models detect objects using a different technique. The TA model separates moving objects and rain streaks from the binary foreground. We have obtained separated TA object masks in an individual binary image.The mask R-CNN model is a deep-learning technique that makes each object’s pixelwise mask (dynamic and static). This mask enables us to segment the object area from the current frame.We fuse the predicted object’s pixels from the TA object mask and Mask R-CNN object mask to detect moving objects and eliminate static objects, as the static objects are already in the background frame. Here, pixels of the predicted object in the Mask R-CNN object mask are distributed into a separate group of pixels, and pixels of the predicted objects in the TA objects mask are distributed in a single group of pixels. In The binary foreground extracted from the background and foreground extraction step contains a moving object area and rain streaks typically. We can recover the distorted object area from the binary foreground information. We fuse these two data in such a way that it can generate a less distorted object mask. We divide all the connected pixels of the binary foreground into a different group of pixels. Each group includes connected pixels. Here, we apply the fusion strategy explained in the previous section, where the binary foreground data is model 1, and the predicted object mask from the previous step is model 2. Then, we integrate both the obtained results, applying the logical “or” operation. ith background and the ith frame to generate the rain-free frame for the current ith frame. We identify each pixel as a background, rain or moving object through the processes mentioned earlier. The corresponding pixel intensity is taken from the background frame if the pixel is identified as a background or as rain for a rain-free frame. If the pixel is identified as a moving object, then the corresponding pixel intensity is taken from the current frame. After applying all those extracted features, we have generated an object mask for the current processing frame. We have used both the generated background frame at the current frame position and the current frame to generate a rain-free video frame. For example, we use both the We have conducted experiments using video sequences with real rain to compare the performance of the proposed method and other contemporary and relevant methods. This comparison provides a subjective quality assessment, as there is no ground truth for the rain-free real videos. We also compare the performance using video sequences with synthetic rain to understand subjective and objective measurements, as the synthetic video sequences have ground truth. We have tested the proposed method on a benchmark dataset called the CDNET dataset , which iThe first row of The second and third rows compare the video sequences “Wall” and “Yard” experimental results. They do not include moving objects. The proposed method performs better than the other methods. It has removed more rain streaks than the other methods. The rectangle and circle marked areas clearly show that the proposed method can remove more rain streaks.The first row of The second row of One of the main purposes of rain removal techniques is to detect and recognise different objects, as sometimes object detection/recognition is problematic in a rainy video. Thus, to understand the strength of the proposed rain removal method, we evaluate its performance in terms of object detection/recognition using its rain-free images. Then, we calculate the precision and recall using the equation below.The precision and recall values show that the proposed method outperforms the state-of-the-art methods for every frame.In this paper, we combined a data-driven model and a feature-based model to address their individual drawbacks. For this, we applied a hybrid technique to combine both models and fuse the models’ predictions. To verify the superiority of the proposed method, we used video sequences with both real and synthetic rains, and compared the performances against four contemporary and relevant methods. The experimental results confirm that the proposed method outperforms those methods by providing better rain-free video and better-quality moving regions. The better rain-free video is demonstrated in terms of better objective and subjective quality comparisons as well as accurate object detection/recognition evidence. Furthermore, as the proposed method used the physical property of the rain and moving objects, it has better interpretability compared to the solely data-driven, i.e., deep-learning, approaches.