Trauma is the leading cause of morbidity and mortality for those under 45 years of age in the United States with half of the deaths in trauma being attributed to hemorrhagic shock. The use of enhanced care teams (ECTs) that include physicians in selective prehospital settings has allowed the delivery of advanced critical care interventions in the field. We present a unique case where a young driver involved in a motor vehicle accident was trapped under the weight of his vehicle, causing extended extrication time. An ECT from the closest trauma center was able to deliver massive transfusion and definitive airway care while the patient was being extricated. While previous literature regarding the benefit of ECTs has been debated, this case suggests a unique niche where rapid deployment of an ECT to the scene made a pronounced difference in survival of the patient. In the United States (U.S.), non-accidental injury is the third leading cause of death overall, remaining as the leading cause of mortality in those below 45 years of age . Among dOver the last decade, advances in civilian trauma care have accelerated due to the integration of experiences from trauma management during the war in Afghanistan . During In different regions across the world, the implementation of trauma care differs depending on the development of the region and resources available. In the majority of regions in the U.S., prehospital trauma care is usually provided by emergency medical service (EMS) personnel, such as emergency medical technicians (EMT) or paramedics . These rHere, we present a case where a victim of a motor vehicle accident was entrapped for a prolonged amount of time, and an appropriate ECT deployment in coordination with a local prehospital EMS team provided the early stages of trauma resuscitation in a prehospital setting.Prehospital courseAt approximately 0445 on 11/9/2019, a 24-year-old restrained male driver, who allegedly fell asleep traveling on a major highway at high speeds, lost the control of his vehicle, rolled his vehicle over the center divider into oncoming traffic, resulting in a collision with a 29-year-old male driving in the opposite direction. Shortly after, the closest rescue team from the city of Colton Fire Department was able to extract, stabilize, and transport the 29-year-old male to the closest regional trauma center, Arrowhead Regional Medical Center (ARMC). While the trauma team attended to the first victim at the trauma center, another EMS crew was trying to extricate the 24-year-old male from his entangled vehicle.Based on the initial evaluation of the scene by the incident commander, a prolonged extrication time was estimated due to the entrapment of the driver and possible need for field amputation of limbs in order to free the patient. Given the estimated prolonged on-site time, the Hospital Emergency Response Team (HERT), a variation of ECT in our region, was activated by the request of the incident commander from the regional dispatch. Due to the after-hour time of the activation, the ECT in this case consisted of a trauma surgeon, emergency medicine resident, and a trauma nurse. After arriving on the scene, the ECT developed the initial plan to perform an advanced airway procedure, apply tourniquets to the mangled limbs, amputate limbs if indicated to extricate the victim, and transport the patient back to the trauma center for further assessment and definitive care. Brief assessment revealed that the patient was trapped at the level of the pelvis and the left proximal humerus under the weight of his entangled vehicle, making field amputation inadvisable Figure .At the time of the ECT’s arrival, patient’s vitals included blood pressure of 130/74 mmHg, heart rate of 130 beats per minute, and Glasgow Coma Score (GCS) of 15, and the patient was able to speak in full sentences but was in severe distress. Intravenous access was established using a 16-gauge needle in the right antecubital fossa. After the initial assessment, the main concerns for the ECT team remained to be a prolonged extraction time, the patient’s declining ability to protect his airway, continuing hemorrhage from his right lower extremity, and a concern for exsanguination once pressure was relieved from the tamponaded extremities during extraction. The weight of the vehicle surpassed the capability of the local rescue team equipment, and a large capacity crane was requested from the county fire department and local private contractors. This need for additional heavy equipment drastically increased victim entrapment time.Based on the circumstances and proximity of the trauma center to the scene of the accident, the team initiated massive transfusion protocol from the field with the coordination of ARMC’s base station. During the initial resuscitation, tourniquets were loosely applied to accessible limbs, and four units of packed red blood cells and one gram of tranexamic acid (TXA) were administered. Afterwards, a critical care transport team was requested from the regional ambulance transporting agency. Gradually the patient’s level of consciousness began to decline, and the decision was made to secure his airway utilizing an endotracheal tube with rapid sequence intubation. The patient was then induced with 30 milligrams (mg) intravenous etomidate and 100 mg intravenous rocuronium, followed by intubation using a bougie with an 8.0-millimeter cuffed endotracheal tube (ETT). Given concern for possible ETT dislodgement during the extraction process, the decision was made to place the ETT in the right mainstem bronchus. After successful intubation, the patient was sedated using 5 mg intravenous midazolam and, analgesic control was achieved using 100 micrograms intravenous fentanyl.An hour after the ECT’s arrival, the heavy crane arrived to lift the vehicle. Once successfully extricated, the patient was secured on a backboard, and double tourniquets were applied on both lower extremities and the left upper extremity. The patient was loaded into the critical care ambulance and was transported to ARMC.Trauma center resuscitationThe patient arrived in ARMC’s trauma bay approximately two and a half hours after the motor vehicle accident. Upon arrival, the primary survey revealed decreased breath sounds of the left chest with subsequent ultrasound confirming decreased sliding for the left lung. A left chest tube and a left subclavian cordis line were placed. It was also noted that the right lower extremity below the knee sustained severe crush injury and would require surgical amputation. Repeat vital signs were noted to be improving, and the patient underwent imaging with CT. A CT scan of the chest, abdomen, and pelvis revealed a mild right lung contusion and right comminuted inferior pubic ramus fracture with minimal diastasis of the sacroiliac joint. There was no evidence of gross organ injury of the chest, abdomen, or pelvis and no evidence of spinal injuries. A CT of the lower extremities with an accompanying vascular study revealed bilateral compound fractures of the tibia and fibula with complete transection of the right anterior tibial artery and tibial peroneal trunk Figure . The patHospital courseOn initial surgical intervention, an above the knee (ATK) amputation of the right lower extremity was performed, and external fixation of the left distal tibia was placed. On hospital day 2, the patient’s total creatine kinase, secondary to his prolonged entrapment, peaked at 7,000 units/liter . On hospital day 3, irrigation and debridement of the left lower extremity and revision of the right ATK amputation were performed. On hospital day 5, the patient had a repeat operating room washout of the left lower extremity. Due to insurance preferences, the patient was transferred to another medical facility on hospital day 10. At one-month follow-up, the patient admitted to continuing palsy secondary to brachial plexus injury of the left hand. He was in the process of obtaining a right lower extremity prosthesis and was three weeks postoperative of the internal fixation of the left lower extremity. He had been working with physical and occupational therapy and was able to return home to his family.Motor vehicle accidents often result in severe trauma, which places the victim at high risk for death due to airway complications, chest injuries, and hemorrhagic shock . OptimalWhile an ECT may provide advanced resuscitation, the degree of resuscitation to be performed in a prehospital setting is controversial. Prehospital blood transfusion is generally considered acceptable and is supported by a previous study of U.S. soldiers in Afghanistan that associated the time to initial transfusion with a decreased 24-hour mortality . While bThe benefits of ECTs for prehospital resuscitation have been widely debated, as emergency response teams in the U.S. generally deploy without accompanying physicians . ResearcWhile many of these studies were unable to associate ECTs with reduced mortality, one reason for this perceived lack of benefit is that ECTs are generally deployed to more severe cases with victims often presenting with a higher ISS or in a more unstable condition . Each trIn the U.S, ECTs are uncommonly deployed to scenarios of massive trauma and allow advanced prehospital resuscitation. In situations where transportation to a trauma center is delayed, a trained on-site physician may be able to make a real-time difference in survival of victims prior to definite care at a trauma center. Further research into the benefit of ECTs in situations with prolonged entrapment and transportation time may support increased usage of physician-supported response teams.


Green tide, which is a serious water pollution problem, is caused by the complex relationships of various factors, such as flow rate, several water quality indicators, and weather. Because the existing methods are not suitable for identifying these relationships and making accurate predictions, a new system and algorithm is required to predict the green tide phenomenon and also minimize the related damage before the green tide occurs. For this purpose, we consider a new network model using smart sensor-based federated learning which is able to use distributed observation data with geologically separated local models. Moreover, we design an optimal scheduler which is beneficial to use real-time big data arrivals to make the overall network system efficient. The proposed scheduling algorithm is effective in terms of (1) data usage and (2) the performance of green tide occurrence prediction models. The advantages of the proposed algorithm is verified via data-intensive experiments with real water quality big-data. Environmental pollution has been one of the most serious problems for a long time and the situation is getting worse due to the development of modern industrial societies. Human beings have suffered from a series of problems, such as diseases, ecosystem destruction, and natural disasters caused by the polluted environment ,5,6,7,8.ery year . FurtherEfforts to detect changes in water quality and predict the patterns of water quality have changed to prevent pollution, and they are modeled and estimated in various forms using Internet of Things (IoT)-based smart sensors ,15,16,17As discussed in , there aFor this reason, this paper proposes a new water quality monitoring system which is designed for achieving federated learning (FL) functionalities. The system is fundamentally based on distributed network architectures, and it consists of smart IoT sensors, edge servers, and a central cloud. The edge server gathers IoT sensing data, learns a ML/DL-based prediction model using real-time real-world sensing data and communicates with the centralized cloud, and the cloud builds and updates a global prediction model for green tide occurrence, which can be applied to a wide range of regions.Instead of FL computation, which is for conducting ML/DL learning via distributed computing platforms and then aggregating the results in a centralized platform to build a global model, centralized learning is optimal in terms of learning accuracy ,21. HoweIn this system, a number of smart sensors are distributed in large areas to collect various water quality-related indicators in the rivers and lakes of South Korea. The edges that are scattered around the sensors in order to receive data from the smart sensors are matched by the scheduling algorithm between edges and smart sensors. The transmitted data from various locations are used to learn/train the green tide prediction model. According to the progress of FL computation, the performance of local models that participate in aggregation affects the performance of the global model generated in the central cloud. We decided that only edge servers that guarantee the model’s accuracy can participate in the aggregation, and their parameter information is utilized. In this process, maximizing the number of edge servers that have models that guarantee the accuracy of the green tide occurrence forecast is important. Therefore, we design a fairly optimal scheduling algorithm for this purpose in order to maximize the number of local edge learning initiations. When multiple edge servers receive data from distributed smart sensors, the proposed scheduler lets as many edges as possible have sufficient data for learning without significant gaps in the amount of data supplied between each edge. It is meaningful in an environment where there is a difference in the degree of dispersion of smart sensors around the actual position of edges or where certain edges can monopolize the matched sensor data. In addition, setting a baseline  for appropriate learning data can prevent some cases that affect the performance of the green tide occurrence prediction models, such as overfitting and underfitting caused by the insufficient amounts of data.The existing research results that use only limited case information at specific times and places are not suitable for the accurate prediction of green tide phenomenon. The general central computing method is not suitable to handle massive real-time water quality data from the several smart sensors that are located over the river irregularly. A new network model with a distributed structure, which is based on FL is suggested. Based on the network model, we can utilize real-time data generated at a certain frequency for a general prediction global model rather than for a certain point.D).This paper presents an optimal fair scheduling algorithm for efficient data transmission between edge servers and smart sensors. The algorithm prevents some edges to receive excessive data from the smart sensors and allows as many edge servers to perform local model training by maximizing the number of learning edges where the learning edge is defined as the edge that has sufficient data to initiate prediction model leaning/training  communications are used for establishing massive connections between sensors and edge servers in large-scale networks . As discThe water quality observation data generated by smart sensors are used as learning data to create a green tide prediction model in each local edge server. The smart sensor means a new IoT-based networked sensor that incorporates various sensors for green tide indication, i.e., temperature, pH, dissolved oxygen (DO), total organic carbon (TOC), total nitrogen (TN), Chlorophyll-a (Chl-a), and turbidity.In this section, we describe the motivation to design the optimal fair scheduling and the details of the proposed algorithm.According to the scheduling policies for the communication matching between edge servers and smart sensors, the numbers of associated sensors in edge servers can be varied. In general, after matching is complete, the edge servers perform their own local training for green tide prediction based on real-world sensing data. If some edge servers are with an insufficient number of water quality sensing data, the performance will be poor due to overfitting. Eventually, this result is definitely harmful for the learning accuracy performance of our ultimate global prediction model because the model will be built by the aggregation of local prediction models through the FL model aggregation procedure as illustrated in Algorithm 1 Fair Federated Learning Computation at Edge Server i.1:Input: Sensing Data: 2:Output: Local Learning Parameters: 3:Local_Training. It is obvious that the training initiation threshold can be varied depending on the required target performance of our green tide prediction model. Our proposed optimally fair scheduling algorithm for FL computation in distributed network is able to avoid training data imbalance among edge servers. Furthermore, the algorithm is able to control certain edges to proceed with local learning with a quantity of data that cannot satisfy the target performance. Through this solution, the performance degradation by overfitting or underfitting in some edge servers can be avoided.t where the set of edges is denoted as i, set of sensors, the amount of transmitted data from sensor i at t, and scheduling vector between edge j and sensor i at t where it can be 1 it they are scheduled , respectively. The size of the queue is independent to the training initiation threshold D, and has an infinite size. These features allow the edge to store all data in a queue even if it receives data larger than the specified D size in one iteration and to use the stored data for learning on the edge. The queue is controlled by a scheduling vector D , The proposed fair FL scheduling algorithm for green tide prediction with water quality sensing training data is presented as follows. Note that the proposed algorithm individually operates in each river, and the overall network architecture is as illustrated in As mathematically described in this optimization formulation –(5), ourThis section verifies the novelty of the proposed algorithm with real-world data obtained Keum Rivers in D, it waits for the next scheduling to meet the size of D and if the data size is equal to value of D or higher, it uses all received data for local model training.The performance of the proposed optimally fair FL scheduling algorithm is evaluated via data-intensive simulations. The simulator is designed with cvxpy . We set http://www.koreawqi.go.kr, http://water.nier.go.kr, http://www.wamis.go.kr) (accessed on 1 January 2021). Each site provides information on real-time water quality and green tide outbreak in Korea. The disclosed data from the sites differ from each other in terms of inclusive water quality indicators and measurement time or coordinates. Generally, seven sensing data  are used as water quality monitoring indicators. For these reasons, we collect the seven kinds of cumulative data over 5 years that are provided from different sites and refine the obtained water quality records by pre-processing, considering the missing data, normalization of variables and class imbalance. Through this process, all the essential data are formulated to form of one row in the dataset based on the same date (time) and location, such as in We collect the water quality data for the simulation from the State-led managed sites  and 1 output parameters (due to binary decision). The model includes 4 hidden layers and each layer is followed by ReLU and sigmoid activation functions . For theIn conventional deep neural network computation procedures, we have three phases: (1) model setup , (2) training , and (3) inference . For the first phase , we already introduced our model, as illustrated in In this subsection, we describe the simulation results in several respects. The simulation is processed under the simulation settings, already mentioned above.D from D becomes larger, the accuracy of the local model, which is trained on one edge is increased. As we can see in D size is between 20 and 50. When D size is over 50, the accuracy is slightly higher and is converging to D is 100. Because the frequency and number of smart sensors generating water quality observation data in the Keum river are fixed, the total amount of data that exists within the entire system is also defined. Therefore, it is necessary to set the appropriate value of D in advance. Based on the result in First, we derive the local model performance for each edge of the river, increasing the size of I. The value of I indicates whether I edges have enough data and are able to learn the local models in the system. As we set the condition in the proposed scheduling algorithm’s equation, only when each edge gets more than D amount of data, I becomes 1 and the size of D guarantees the corresponding local model accuracy. For these reasons, the results represents that the higher the value of I, the more edges that have sufficient learning data through the matching methods used in the experiment, and each edge can train the green tide prediction model, which can participate in the global model generation. We proceed with the simulation during 50 iterations and compare the proposed optimally fair scheduling algorithm performance results. Baseline matching , which ignores filling D, is used as a comparative group.We evaluate the scheduling performance based on the number of D, are distinguished with different marker types and line colors in D size, there are performance differences depending on the value of D. In D is set 20, the proposed scheduling algorithms always match the baseline performance. The larger the setting for the D value, the greater the amount of data that the edge must receive from the sensors to meet the conditions. The size of sampling data produced by the sensors in every iteration is always constant. For these reasons, the number of edges, which satisfy D for each period, decreases compared to the results with small D. However, even in this situation, the proposed algorithm matches the sensors and the edges in an optimally fair direction and produces as many federated learning participants  as possible.This evaluation has the performance results, as shown in the D and the actual data size of the edges over the 50 iteration. D, as tested using 5 edges located in Backjae barrage and Gongju barrage. The random matching, which is represented with gray color, results in all edges being able to start local learning with the received data no matter the size. It also has the most variety in the numbers of data distribution. If D value according to D data. Even if D on average, and less than D condition. For D, and in some cases have more than D. If We also measure the number of edges that satisfies the training threshold D. We derive CDF graph for the performance of the local model generated by each edge. In D sizes are distinguished by the color of the line and the shape of the marker. One marker represents the cumulative distribution of model accuracy corresponding to the x-axis. Both Backje barrage and Gongju barrage have a concentration between 90 and 100 accuracy at Model accuracy is also tested using real data received by the scheduling results above D has value of 50, the largest number of edges during all iterations satisfy target performance. When D has a small value such as 20, the proposed scheduling with D is too large, high performance models can always be obtained. However, in this case, each edge should receive more than the threshold value D and it is impossible to get many edges that can perform learning.Consequently, the results of the proposed scheduling algorithm show that the case of Nowadays, various deep learning algorithms are widely used for major environmental problems, and this paper designs a novel green tide phenomenon prediction model which is one of major environmental problems. In order to gather the green tide related indicator information in real-time, smart sensors are used and the gathered data will be delivered to intermediate edge servers  due to the limitations of transmission distances and the burdens of massive accesses. Then the edge servers conduct their own local training, and the trained parameters are delivered to the centralized platform for building our desired global prediction model via federated learning computation. In order to improve the performance of federated learning, gathered data should be fairly distributed over the edges’ servers for avoiding overfitting. Therefore, this paper proposed a novel federate learning scheduling algorithm in order to fairly schedule sensing data over edge servers. Our data-intensive simulation results verify that the proposed algorithm presents remarkably effective performance due to balanced data gathering. As future research directions, various factors, such as the positions of network components , distribution ratios, and communication channel qualities can be considered. In addition, experiments can be extended using various real-world data for a wider range of regions.


Emotion recognition has drawn consistent attention from researchers recently. Although gesture modality plays an important role in expressing emotion, it is seldom considered in the field of emotion recognition. A key reason is the scarcity of labeled data containing 3D skeleton data. Some studies in action recognition have applied graph-based neural networks to explicitly model the spatial connection between joints. However, this method has not been considered in the field of gesture-based emotion recognition, so far. In this work, we applied a pose estimation based method to extract 3D skeleton coordinates for IEMOCAP database. We propose a self-attention enhanced spatial temporal graph convolutional network for skeleton-based emotion recognition, in which the spatial convolutional part models the skeletal structure of the body as a static graph, and the self-attention part dynamically constructs more connections between the joints and provides supplementary information. Our experiment demonstrates that the proposed model significantly outperforms other models and that the features of the extracted skeleton data improve the performance of multimodal emotion recognition. Multimodal emotion recognition has attracted a lot of attention due to its wide range of application scenarios. Many previous research efforts on emotion recognition process the information from different modalities and use multimodal clues to infer the emotional states. Body gestures are an integral part of nonverbal communication and deliver extremely important supplementary information when expressing emotions . A robotThe dynamic human skeleton is the most intuitional and natural method for depicting human actions, and it does so by structuring the body movement as a sequence of joint positions. The scarcity of labeled data containing 3D skeleton data is one of the significant reasons the body gesture modality has seldom received attention in emotion recognition tasks. Although several multimodal emotional databases include skeleton coordinates, e.g., the multimodal database created by Sapiński et al. 560 samples)  and the 0 samplesSince deep learning has revolutionized many fields and had an outstanding performance, some recent research has utilized neural networks on gesture-based emotion recognition ,8,9, whiIn the field of action recognition, the Spatial Temporal Graph Convolutional Network (ST-GCN)  abstractTo solve these problems, we make the following contributions: (i) we extract 3D skeleton movement data from raw video based on pose estimation and the method can be used to expand existing databases to alleviate the lack of labeled data. (ii) We propose a self-attention enhanced spatial temporal graph convolutional network for skeleton-based emotion recognition, in which the self-attention part of the spatial graph convolutional layer dynamically constructs connections between the nodes of the entire graph and provides supplementary information for the spatial graph convolution. The model outperforms baselines by a significant margin. (iii) We fuse the representations extracted by the proposed model with the audio and text high-level features, to use audio, text, and skeleton modalities simultaneously. The performance significantly exceeds that of the bimodal model using only audio and text information, which shows the effectiveness of the extracted modality.Emotion not only plays a crucial role in interaction, decision making, and cognitive processes , but alsThere have been some studies about gesture-based emotion recognition over the past few years . For exaMost of these studies used open-source multimodal datasets or created new datasets without a large amount of data. Since labeled emotional gesture data is scarcer than speech and facial data, these studies have been limited. Moreover, single-person action performance instead of natural dialogue-based interactive data has been used in these studies, which makes learning the natural expression of human emotions in interactive scenes difficult. Besides, most of the existing deep learning methods for gesture emotion recognition are based on RNNs or CNNs, which cannot represent the spatial connections of joints.Graph is a data structure composed of a series of nodes and edges. As non-Euclidean data, graph data have no regular spatial structure, and their complexity poses a great challenge to existing machine learning algorithms . RecentlEncouraged by the success of CNNs in the computer vision domain, convolutional graph networks (GCNs) that generalize convolutions to graph data have attracted an increasing interest . These aWe extract dynamic body skeleton data from raw video of a large existing open-source emotional database to add a new modality representing gestures. To do this, we use a human pose estimation method to draw a temporal sequence of joint positions in the form of 3D coordinates. Then some prepossessing steps are applied for the data before feeding it into the emotion recognition model.Human pose estimation is used to localize the human joints in images or videos, obtain 2D or 3D coordinates of the joints, and reconstruct the coordinate representations of the human body. With the increase of computing resources and the amount of data, the pose estimation method using learning has gradually matured and had considerable success. We first apply AlphaPose ,37,38, aIn the 2D pose estimation results, there is high-frequency noise because of the image quality and the estimation error, which causes instability of the 3D coordinates. We add a low-pass filter after 2D joint position detection to filter out the noise and obtain clean skeleton data.Since the lower halves of the actors’ bodies are not visible in the video clips most of the time, the reliability of the estimation results of this part of the body is much lower than that of the upper body. Therefore, the lower body part is removed from the skeleton graph and only 10 joints of the upper body are considered in our research.G composed of vertexes V and edges E, represented as The extracted body skeleton in each frame is represented in the form of 3D coordinate vectors of human joints. The dynamic motion data of each sample is a time sequence with different timesteps because of the variable lengths of the video clips. Following the work of , we conscurately ,41. FiguAs described in The structure of our self-attention enhanced spatial graph convolutional layer is illustrated in K subsets, i.e., K subsets, i.e., Graph convolutional part: For the spatial graph convolutional part, the structure of the graph is predefined by the the adjacency matrix T is the temporal length, V is the number of joints, and Self-attention part: The self-attention part applies a multi-head scaled dot product attention on the graph. In the field of Natural Language Processing, Vaswani et al. proposed multi-head self-attention, in which the queries and the corresponding keys are used to compute the weight and the weighted sum of the values is calculated as the output . The seli and the features of key j is calculated by the scaled dot product for each head:For each timestep in one sample, we represent the feature vector of each joint as H is the number of attention heads, and The weighted sum of values for each head is r to adjust the weight of the self-attention part, which is formulated as:r is initialized as 1.Gating mechanism: Considering that the contributions of the two parts may be different for updating representations of nodes, we use a trainable coefficient The basic block of the self-attention enhanced spatial temporal graph convolutional network (S-STGCN) is composed of a self-attention enhanced spatial graph convolutional layer, a temporal convolutional layer, and several functional layers See . The selAs illustrated in In some previous works of action recognition ,43, in aSimilar to the work of , we defiThe architecture of the two-stream self-attention enhanced spatial temporal graph convolutional network (2s-S-STGCN) is illustrated in i represents the timestep) are fed into a gated recurrent unit (GRU) for each timestep, and the final hidden state We construct a skeleton enhanced emotion recognition network (SERN), which integrates text and audio information with the features extracted by the self-attention enhanced spatial temporal graph convolutional network See . The mulWe propose a two-phase hierarchical network to simultaneously fuse the audio, text, and gesture information. In the first phase, the uni-modal features are fed into the ARE, the TRE, and the self-attention enhanced spatial temporal graph convolutional network, respectively, to obtain the audio, text, and gesture representations. In the second phase, these representations are passed through the fully connected layers and then concatenated to pass to the output layer to predict emotion.We usde the IEMOCAP database  in the eTo keep consistent with previous work, we merged the samples with excitement labels into the happiness subset and used four emotion classes, i.e., neutral, happy, sad, and angry. The dataset used in the experiment contained a total of 5492 samples in those four classes . Some examples of the dataset are shown in Following the work of , we usedThe performance of the proposed models and baseline models is shown in For skeletal movement, ResNet18  and ST-GWe also compared the trimodal SERN with the MDRE  that wasIn the data preprocessing See , we applr was set to constant 1 in the ablation experiment. Thus Equation (In Equation  was tranThe result is shown in r of each layer in We also visualized the coefficient As introduced in The experimental result is shown in From the confusion matrix in Overall, both of the two-stream models obtained better performance for most of the classes. The two-stream model using the summation of the scores also frequently incorrectly classified the samples of angry, happy, and sad classes as the neutral class. The other two-stream model that fed concatenation of the scores into the fully connected layer alleviated this problem to a certain extent and performed better on most of the categories.Different modalities showed different characteristics in the ability of emotion prediction. In this paper, we extracted body skeleton data using a pose estimation based approach from the videos of the IEMOCAP database. We proposed a novel two-stream self-attention enhanced spatial temporal graph convolutional network for emotion recognition based on the skeleton data. It models the body skeleton as a graph and constructs both static and flexible connections between the joints to update the representations. Through our experiment, we demonstrated that the performance of the proposed model is superior to that of the ST-GCN on emotion recognition tasks and that the two-stream architecture further improves the performance. Besides, we integrated information from audio signals, text transcripts, and skeletal movement, showing that our method outperforms the bimodal model, which indicates that the extracted skeleton data can provide important supplementary information for emotion detection. Since the multimodal fusion strategy that directly concatenates different modalities may not be optimal, we will focus on exploring more multimodal fusion strategies. We also plan to generate emotional gestures using the extracted skeleton data in our future work.


Given the pressure on healthcare authorities to assess whether hospital capacity allows properly responding to outbreaks such as COVID-19, there is a need for simple, data-driven methods that may provide accurate forecasts of hospital bed demand. This study applies growth models to forecast the demand for Intensive Care Unit admissions in Italy during COVID-19. We show that, with only some mild assumptions on the functional form and using short time-series, the model fits past data well and can accurately forecast demand fourteen days ahead (the mean absolute percentage error (MAPE) of the cumulative fourteen days forecasts is 7.64). The model is then applied to derive regional-level forecasts by adopting hierarchical methods that ensure the consistency between national and regional level forecasts. Predictions are compared with current hospital capacity in the different Italian regions, with the aim to evaluate the adequacy of the expansion in the number of beds implemented during the COVID-19 crisis. By giving rise to sudden surges in hospital admissions, pandemics place a heavy load on health systems . The curResearchers in the medical field typically use epidemic models of disease spread to gain insight into the transmission dynamics. These models hold two sources of uncertainty, the first linked to observational error and limited resolution and detail from available data, and the second stemming from uncertainty in the model structure, especially when confronting outbreaks triggered by a novel virus. Furthermore, some studies have underlined the sensitivity of key epidemiological parameters to model assumptions .st to April 30th, 2020 (61 Days) with a training sample of about 60 days (30th June). Ribeiro et al. build forecasts of cumulative confirmed cases over ten Brazilian states with three forecasting horizons  [Several very recent studies have tackled the problem of predicting COVID-19 related health outcomes using data-driven models. Petropoulos and Makridakis predicted cumulative level of confirmed cases, deaths and recoveries at worldwide level using models from the exponential smoothing family . They uss-ahead) . A trains-ahead) . Their os-ahead) . Their ss-ahead) .Phenomenological models , which are used extensively in modelling and forecasting the diffusion of innovations, new technologies or new products in marketing and operations management , have reAnother important issue when forecasting ICU demand lies in the fact that the spread of the disease has been highly variable within a single country. To illustrate, the Italian national health system’s capacity to match the demand for ICU beds has dramatically been put under stress especially in those areas of the country that were hit first and foremost , but lesThis study applies three growth models and hierarchical forecasting methods to predict the demand of ICU beds at national and regional level in Italy during the most severe period of the COVID-19 outbreak. We show that, with only mild assumptions on the functional form, one model fits past data on bed demand well and can be used to forecast demand in the days ahead.With respect to previous contributions, this study innovates by: i) proposing the use of Harvey model , which rThe relevance of the study lies in the provision of a decision support tool that can be straightforwardly implemented in countries that are being hit by the COVID-19 outbreak. Forecasts obtained can be compared with current ICU capacity, with the aim to estimate the need to either expand the number of beds or to organize the transfer of patients to neighbouring areas.The article is organised as follows: Section 2 introduces the methodologies adopted, Section 3 provides a brief overview of the data used, Section 4 presents model fit and forecasts, while Section 5 concludes with a discussion of the results.susceptible, infectious, recovered), which allows an exact evaluation of the epidemiological diffusion of the disease in the long term [The most frequently used approach in epidemiology for forecasting the evolution of an infectious disease adopts mechanistic models relying on assumptions about the transmission mechanisms of the disease in an infected population, which are embedded in the model parameters . This approach uses different variants of the so-called SIR model  as starting model parameters. In fact, comparability is severely hindered by the fact that the infection rate in the population and the mortality rate exhibit significant cross-country differences due to varying national public health practices, data collection and political containment decisions. This limitation may extend to comparisons of infection rates among regions within a country, depending on the degree of autonomy of regional healthcare bodies.Past epidemics have taught that it is important to balance the need for detailed models against the limitations stemming from the parameter information they require. Knowing the growth rate of cases and when they will peak is critical for healthcare providers to organize an adequate response. For this reason, phenomenological models  that use a fully data-driven approach are also used in forecasting the diffusion of an epidemic, in alternative to or in conjunction with SIR models . In spitYt is the cumulative level of the variable, t denotes time, Θ1, Θ2 and Θ3 are the parameters to be estimated using nonlinear least squares and εt is the error term. The curve is symmetric about its points of inflection .Many functional forms have been used in epidemic forecasting research, as documented by other contributions in this journal. The logistic model is the baseline model. It is defined by the equation:The Gompertz curve is another widely applied model. Using the same notation as in , the equUnlike the logistic, this curve is asymmetric about its points of inflection.Flexible functional forms have been proposed with respect to the baseline logistic model , 26, 27.yt is the change of Yt (that is yt = Yt − Yt-1), β are the parameters to be estimated and εt is the error term. Thus, this rate-of-change model does incorporate the variable time (t), allowing for flexibility in skew and, as discussed by Harvey and Young, it includes the logistic, Gompertz and modified exponential as special cases [In Harvey’s model, al cases , 26. Theal cases –18, we aAs suggested by previous studies, epidemic curves are characterized by different spatial and temporal patterns , with thh-step-ahead forecasts for all the regional series ph,j are the regional proportions, and are obtained as:The top-down approach discussed by Athanasopoulos et al. is based on disaggregating the national forecasts according to forecast proportions of the regional series . First, Therefore, the regional forecasts are not used directly, but they drive the decomposition from the top to the bottom level, generating regional forecasts coherent with the national one.We are interested in comparing the forecast accuracy of models introduced in section 2.1–2.2. When choosing models, it is common practice to identify a training sample, where the training data is used to estimate any parameters of a forecasting method, and a test sample, that is used to evaluate its accuracy.Yt is the observation at time t. Two commonly used scale-dependent measures are the mean absolute error (MAE) and the root mean squared error (RMSE) [Using the same notation of the previous sections, we denote cumulative forecast with r (RMSE) :MAE=meaIt is possible to define percentage errors that are unit-free . The mosAccording to Lewis, MAPE values suggest accurate forecast when <10, good forecast when they are in the interval 10–20, and reasonable forecast when they are in the interval 20–50 . HyndmanThe Italian healthcare organizational context and experience with COVID-19 appears to be well suited to illustrate the benefits of the hierarchical forecasting approach. Since the 1990s, the Italian National Healthcare System has been organized on a regional base and exhibits significant geographical variations in health services provision and organization , 31. A dhttp://www.protezionecivile.gov.it/attivita-rischi/rischio-sanitario/emergenze/coronavirus), making Italy one of the most severely affected countries by the virus. After the start of the infection in Italy in late February, the Italian Government was among the first to implement extraordinary community containment measures on March 8, in the attempt to limit the spread of contagion. However, during the months of March and April the hospital system was under heavy stress. This was especially true for ICUs, which accounted for about 14% of patients admitted to hospitals. During this critical period, due to the heterogeneous diffusion of the virus across the country, ICU demand in some regions stayed well below ICU bed capacity (e.g. in Southern regions), while in other regions capacity saturated quickly (e.g. Lombardy). Therefore, the epidemic put ICUs to test in an asymmetric manner, with some regions experiencing a sudden shortage of beds, equipment, and medical staff to respond to the COVID-19 emergency.As of June 15, the number of patients infected in Italy had reached 237,290 confirmed cases and 34,371 deaths according to data provided by the Dipartimento della Protezione Civile . However, given that the bed shortage was most severe for ICU and that this type of hospital department captures a high consumption of resources, the analysis was restricted to officially published data concerning net daily admissions (admissions minus discharges) to ICU with a COVID-19 diagnosis, by region. Admissions were monitored starting from February 25 (Day 1 of the outbreak) until April 3 (Day 39 of the outbreak). We consider this time interval significant, because it includes the peak of the daily admissions to ICU. After April 4 the daily variation of ICU admissions becomes negative at national level and demand pressure on hospitals reduces significantly.Official statistics for Italy provide information on the total number of hospitalized patients and of ICU patients , while using the remaining data as test sample. This choice of training sample allows testing the ability of our model to make accurate predictions with a short time-period. Additionally, it allows testing the goodness of fit of the model in the initial period of the outbreak, when there was high uncertainty concerning the success of containment measures introduced by the Italian government on March 8.This section presents evidence of model fit and forecasts of ICU bed demand using the methods described in Section 2: growth models and hierarchical forecasting . Figs 1 In order to evaluate the accuracy of these forecasts, the mean absolute error (MAE), the mean absolute percentage error (MAPE) and the root mean squared error (RMSE)  considerFigs Predictions obtained can be compared with ICU capacity in the different Italian regions, with the aim to evaluate the adequacy of the expansion in the number of beds during the COVID-19 crisis. Columns (8) and (9) report the forecasts of total ICU capacity (including COVID and non-COVID patients) and COVID-19 only capacity respectively. The figures have been created adding to the model forecasts a buffer of 15% (i.e. an occupation rate of 85% is assumed). The total number of ICU beds resulting from this calculation is very similar to the actual number of ICU beds created . However, the distribution of beds across the country and especially across the three different macro-areas of Italy  is significantly different, with forecasts from our model assigning the greatest portion of capacity increase to northern regions. Column (10) reports the difference between forecast capacity and actual ICU admissions on April 3, showing that the allocation of new capacity proposed by the model would have avoided the shortage of ICU beds experienced by Lombardy . Overall, our model creates fewer shortages , although these are spread over several regions.The first wave of the COVID-19 epidemic that has occurred in the first semester of 2020 has shown the difficulty of reliably predicting the number of patients requiring ICU hospitalization. Healthcare managers and policy makers at regional level critically need this information in order to rapidly adjust capacity or to organise the transfer of patients from one region to another.This study has made two contributions to the problem of managing ICU capacity during the COVID-19 outbreak. The first contribution is the identification of a flexible functional form for predicting ICU admissions using past data that allows fitting short time series and correctly predicts the peak of ICU admissions. Results are especially relevant because the dates at which forecasts are made range from one to two weeks after the start of the lockdown in Italy, i.e. when there was high uncertainty on the effects of the community containment measures undertaken. This model has been compared to two of the most frequently used growth models, showing better performance. The second contribution concerns the decomposition of national-level forecasts of ICU admissions into regional-level predictions, with the aim of producing consistent and accurate forecasts at both levels. Forecasts of ICU capacity expansion substantially align with realized capacity expansion at national level. However, the decomposition proposed by our model assigns a greater portion of beds to Northern Italy, the area of the country that was ravaged by the virus, at the expense of creating some shortages in the South. Although our model forecasts would have avoided creating the major ICU capacity shortage in Lombardy that was unfortunately observed, we realize that such planning would have required centralised healthcare planning at national level, which is at odds with the current regional structure of the Italian healthcare system.Limitations of the study include the fact that forecasts are based on three growth models while it would be of interest to compare forecasts from several different functional forms including other methods. Another limitation arises from the reliability of data concerning ICU demand, given that during the peak of the crisis some regions decided not to admit to hospitals patients whose conditions were too severe and who were unlikely to survive. Finally, the exact rate of occupancy of ICU beds for non-COVID inpatients is not available at local level, and this has limited ability to exactly assess the capacity gap. Despite these limitations, we are confident that the study can be useful to forecast ICU capacity needs, which may support healthcare decisions in countries that are currently experiencing COVID-19.


High altitude aerial surveys have the potential to improve disturbance-free data collection in wildlife research, but previously, bird species were not recognizable in high-altitude orthophotos. This method of aerial surveying is effective and can be repeated frequently due to its low cost; it also has the additional advantage of being able to monitor the status of protected areas. In the case of waterbirds, due to the low vegetation coverage, aerial remote sensing is an exceptionally effective technique for surveying populations and detecting nests. Aerial surveys made at low altitudes can cause serious stress for birds. The method we developed and employed is unlikely to be detected by either ground-based or nesting birds but is far more reliable compared to the low-resolution imaging methods and to the evaluation of non-georeferenced photo series. The modern sensors and photogrammetric procedures enable the use of the present method worldwide; furthermore, the large-scale ortho image-derived information has become obtainable more frequently. Direct georeferencing makes the field geodetic survey unnecessary. Orthophotos with a 0.7 cm spatial resolution allow us to reliably identify even the individuals of smaller species, and by the use of oblique images, they can be tracked from two or four different directions. From the perspective of wildlife conservation, it is very important to develop such monitoring systems that are able to reveal the changes in the environment. Wetland systems are vulnerable to changes in the quantity and quality of their water supply, and specific restoration and management plans will require examinations by habitats .The success of reliable and frequently sampled monitoring procedures are based on working with indicators that can be credibly tracked in space and time ,3,4. In In the field, we can deduce the number of nesting pairs at the heron colonies at dusk or at dawn, when they are flying above several square kilometers of dense reeds or areas with muddy soil. On an impenetrable terrain, when counting from the ground, in smaller sub-areas after counting the occupied nests and determining the proportion of nesting species, the results can be projected to the entire area of the colony. We can count the nests within the 5 m bands marked on large colonies and summarize them . In thesDuring the latter half of the 20th century, aerial-based surveys were being utilized to observe and study breeding sites, but due to the limitations of the camera equipment and particularly the resolution, it was necessary to fly close to the nests, causing disruption.Due to the physical approach, these aerial surveys have also disturbed the colonies. Therefore, it is worth examining how to perform high-spatial-resolution data acquisition from higher altitudes using airplanes and UAVs .2-large areas [During the period of 2006–2014, we developed a method with a large spatial resolution that allows us to create a contiguous orthophoto map with subcentimeter spatial resolution from a high altitude 450–800 m) of a few km00 m of aTherefore, we would like to introduce a long-term monitoring procedure, undertaking at least three monitoring surveys a year for each breeding site, with the aim of introducing a completely disturbance-free procedure.Egretta alba), Little Egret (Egretta garzetta), Squacco Heron , Black-crowned Night Heron (Nycticorax nycticorax), Grey Heron (Ardea cinerea), Eurasian Spoonbill , and Pygmy Cormorant . Sparsely breeding species consist of Purple Heron (Ardea purpurea), Common Little Bittern (Ixobrychus minutus), and Eurasian Bittern (Botaurus stellaris). The area of Lake Kolon at Kiskunság National Park is the largest natural wetland of Kiskunság. The 1200 ha large reedbed lake is bordered by sand dunes, meadows, and swamp woodlands, which have high natural values. The area is also part of the Ramsar sites, which are protected wetlands. Colony species that are breeding there today consist of Great White Egret  is installed correctly, calibrated, and operating well at all altitudes.AHRS (Attitude and Heading Reference System) can be combined with air data computers to form an air data, attitude, and heading reference system (ADAHRS), which provide additional information, such as airspeed, altitude, and outside air temperature. ADAHRS installation location should be a rigid surface within 12 feet longitudinally (front-to-back) and 6 feet laterally (side-to-side) of the aircraft’s center-of-gravity. Typically, the accuracy expected is within ±0.5 degrees. ADAHRS in case of roll accuracy is under ±0.02 degrees between −20 to 70 °C in linear flight. State sensor technology delivers roll hold accuracy to within 2 degrees in all experienced weather conditions.Recording speeds with redout speed (minimum of four frames per second) and 4.87 microns CMOS  allow for extremely high spatial resolution images . Given ax-axis), phi (the rotation about the y-axis), and kappa (the rotation about the z-axis). The coordinates, the altitude, and the omega, phi, and kappa angles are collectively referred to as the six parameters of external orientation [The system records the position and tilt of the camera during the exposure, which means the x, y, and z location of the camera’s projection center point and the yaw, pitch, and roll of the camera axis. The orientation of the camera is given by omega  and 3D photogrammetric surface reconstruction accuracy must be considered because the best combination of grid shapes and acquisition grid geometric parameters determine the desired accuracy for the required GSD (Ground Sample Distance) .Geometric verification was performed on a land-based test field in order to determine the geometric properties of the method . This eaOrthophotos and the oblique camera-axis images are evaluated manually in GIS software in triplicate by different analysts. The final result is a composite of the evaluation. Individuals and nests are interpreted in a point-type vector-graphic map layer.Due to the geometrical accuracy of high-resolution technology, no motionless individual can disappear when the images are overlapped. However, orthophotos are also analyzed without mosaicking as individuals can be seen from multiple angles, and the movement of flying individuals can also be determined. Moving individuals, especially flying ones, require manual inspection at the linking points of the images. The high flight speed and the rapid return between the flight rows are not important during the counting of nests but are important throughout the counting of colonies and those individuals that are surrounding them.The reliability of the results from aerial surveys can depend on the experience and training of the observers, as well as the variation in the detectability of different species ,24,25.The feet of the Little Egret is yellow, which allows us to distinguish it from the Great White Egret during its flight, whose legs are dark brown or black. The pastern of the Little Egret is black, while the Great White Egret’s pastern is yellow—these can’t be seen from a top view. The beak of the Great White Egret is yellow beside the breeding period when it is black, but its facial skin is green at all times. By contrast, the Little Egret’s beak is always black.The Great White Egret builds its nest in areas where the reeds are densest, forming sites with its companion species. Its nests are made of dry reeds; for this purpose, the Great White Egret uses the surrounding reeds. The Great White Egret builds its nest densely, but only the middle of the nest is more robust, where it is lined with dry reeds. Occasionally, it puts the nest on willow bushes, which can be found in swamps; in this case, the nest is smaller, and it consists of branches.Plegadis falcinellus). Its nest is made of willow branches and can be found in willows or bushes. Its nest is small and very loose compared to the bird’s size. Later, during the breeding and nestling rearing period, the Little Egret lines the nest with fresh, leafy branches.The Little Egret breeds in mixed heron colonies. Typically, it nests with Squacco Herons, Black-crowned Night Herons, and Glossy Ibises  range, we identified six major coherent breeding sites at Lake Kolon  in a 55 The distinction of White Herons from other species was feasible from a 2.5 cm spatial resolution, and heron species could be distinguished at a 0.7 cm large spatial resolution .The 0.7 cm spatial resolution orthoimages of heron colonies at Lake Kolon and the interpretation of the individuals and the nests are shown in The eggs that were not in the coverage could be counted in those orthophotos whose resolution was less than one cm detailed . In the If we considered the 0.7 cm GSD orthophoto made simultaneously with the oblique camera technique to be a reliable source of data, only 11.7% of the individuals in the 7–10 cm GSD orthophoto were identifiable, while the 1.5 cm GSD orthophoto detected only 21% of the individuals.In the 7–10 cm GSD orthophoto, we could identify 80% of the nests, which improved to 81% by increasing the spatial resolution to 1.5 cm.If we did not consider the Pygmy Cormorants and concentrate exclusively on herons, the 7–10 cm spatial resolution surveys detected 20.9%, while the 1.5 cm spatial resolution surveys detected 37.5% of those individuals that were identified with 0.7 cm spatial resolution ortho-oblique technology . The 0.7Since the accuracy of the analysis of the highest spatial resolution orthophoto and oblique imagery was not verifiable, a sub-area analysis was repeated with field validation. In June 2019, hail caused serious damage to the colonies of Lake Kolon. Since we had to check the colony after the hail, we could test the reliability of the high-resolution aerial survey method in one of the smaller colonies with the help of field inspection, and it was possible to prove the spatial resolution dependence of the accuracy. At that time, no adult herons were in the vicinity of the nests because of the field inspection, and the adult birds left the colony. The overflights were also carried out during the field survey for comparison. In the case of that separate Great White Egret breeding colony, the interpretation was performed with eight different analysts from small spatial resolution to large. Examining the subjective effect of the evaluators, it was found that orthophotos with 0.7 cm spatial resolution were already sufficiently reliable to analyze the number of colonies .The experiments pointed out that a reliable counting procedure in the case of heron colonies could consist of the elaboration of an orthophoto map with 0.7 cm spatial resolution, with 90% overlap in line and at least 40% overlap between flight lines, made from high altitudes (450–800 m) and followed by a GIS (geographic information system)-based evaluation of the colony.The remote sensing platform can be a large-sized fixed-wing UAV or a pilot-driven aircraft. The most important requirements for the flying platform are to fly steadily and minimize roll (max. 1–2 degrees). It should carry securely a 6–25 kg  aerial camera system. Depending on the size of the breeding site, the flying groundspeed needs to be at least 160–270 km/h to prevent birds from significantly shifting until the aircraft returns to the next flight line. Ground speed is very important because more than 50 breeding sites, each are more hectares, need to be surveyed in a given time period per day. Depending on the species and habitat, it must fly at least 500–800 m above the ground so that the frequent surveys will not disturb the birds. It should be able to work in special weather conditions, as we need to survey a large number of colonies in a specific time period. As a result, those lenses that can be used in non-ideal lighting conditions, wide-aperture telephoto lens, and lenses that are able to make sharp and detailed images even from large distances from the object make up 80% of the equipment weight. The most important requirements for the camera system are its fast readout and recording speed and high sensitivity sensor with low noise levels, which provides dynamic and well-exposed images with short shutter speed. Care should be taken to ensure the proper plane alignment of the sensor and the quality of the minimally distorted optical system. It is very important to use direct orientation because there is no possibility for the measurement of field ground control points near the breeding sites. The direct georeferencing with GNSS  post-processing enables the production of orthophotos accurate to the cm, while without correction, the extent of the accuracy is half a meter without carrying out field measurements.For the production of orthophotos, any modern photogrammetric software that is operating on visual SfM  principle is recommended. The visual interpretation, which is the geoinformatic evaluation of the orthophotos, can be performed with any GIS software, but we primarily recommend software that enables the changing of the histograms of the images during the evaluation. This offers a great deal of support because the dynamics of the photographs allow us to bring hidden information, for example, by darkening the image to sharpen the yellow spot on the back of a Eurasian Spoonbill.Interspect IS 5 digital aerial cameras were used with 300 mm and 500 mm focal length lenses, but experiments were conducted with a number of small and medium format cameras that could continuously shoot at exposures intervals of less than half a second . Leading camera manufacturers  offer such camera bodies available in any country, as well as high-quality lenses .In our experience, a camera body with a higher resolution sensor (40+ MP)—a smaller focal length (300–400 mm) lens is preferable because longer focal length lenses with a few degree fields of view make the surveys extremely sensitive to aircraft roll. Images were processed with an Agisoft Photoscan photogrammetric software and analyzed in Quantum GIS software.If oblique camera axis images were made from two directions, the identification of individuals increased by 4.55%. This was primarily due to the fact that the identification of 14% of those individuals that were classified as “other species” (that cannot be determined exactly) during the analysis of a simple orthophoto was becoming feasible by the simultaneous analysis of orthophotos and oblique camera axis images. Through the use of this technique, the isolation of Great White Egrets and Eurasian Spoonbills was 28% more reliable. The definition of White Herons improved by 31%, but the identification of Grey Herons was only more accurate by 0.5%. This survey that was made using the oblique photogrammetric technique was an essential element in the counting of individuals .In case of the monitoring the nests of Lake Kolon, the simultaneous application of images with a vertical and bi-directional oblique axis resulted in an improvement of 8.41% in reliability.The greatest improvement (15%) in the detection of Eurasian Spoonbill nests was due to the simultaneous application of oblique and vertical axis orthophoto technology. The total number of White Heron nests could be determined 21% more precisely with the mentioned method. The identification of Great White Egret nests improved by 6%. It is worth mentioning that the Purple Heron nests could only be detected through the use of this complex method; with a simple orthophoto-based analysis, we could not distinguish Purple Heron individuals from Grey Herons.In addition to a one-time investment, it does not create any significant increase in costs to make captures with an oblique camera axis while taking high-resolution digital photographs with a vertical camera axis. Considering the minimum increase in costs and the impact of the method on the reliability of the survey, we recommend applying this technology for the monitoring of heron colonies.The presented method allows non-disturbing bird colony-mapping in case of nesting areas where the canopy level does not cover the nests. An inspection of the number of individuals, repeated three times per year, and an annual inspection of the number of nests could play an important indicator role in the area of nature and landscape protection. The method could also contribute to the protection of the adjacent areas and be important because the number of populations does not only depend on the quality of the direct breeding sites.Previously, the creation of orthophotos with sufficient overlap and ultra-detailed spatial resolution was not accessible at high flight speed and at high ground speed. In the present work, we developed a practical environmental application of the method that we introduced in 2014 . With ouThe simultaneous use of the subcentimeter spatial resolution orthophoto mapping and oblique mapping method followed by a GIS-based evaluation improved the precision of individual counting by at least 60%, made the analysis of the nest numbers more accurate, and even made it feasible to classify the nests. Compared to those field and aerial surveys that were made in recent years, the survey reliability of Lake Kolon’s heron colonies improved considerably, and the survey provided far more parameters for Nature Conservation Information Systems. The non-disturbing, high-speed process from an altitude of 450–800 m was not only more reliable than previous methods but provided more detailed information without disturbing the animals.We recommend three aerial surveys per year in the case of important breeding sites, which are remarkable from the perspective of nature conservation and which are good environmental indicators in the period from May to June because with disturbance-free methods, it is possible to track the status of the populations without risking the living conditions of birds and by minimizing the amount of stress. It would be worthwhile to investigate the possibilities of evaluating orthophotos based on an automated or semi-automated image analysis algorithm in independent research. It is also worth adding to the procedure with thermal imagers, although due to the high flight altitude, we do not expect such good results as we see in the thermal counting works ,27.An algorithm, which allows distinct small signals and features, has great potential in accelerating evaluation, irrespective of the position, size, and rotation of the object, as well as robust affinity transformations  and lighting conditions. There are methods for Spatio-temporal analysis of image series for the exploration of relationships , so autoDuring rapid recording, 6 to 10 consecutive shots also record the movement and change of position of the individual birds as the aircraft passes. This allows the use of action classification algorithms too.


Recently, Cu-Ag nanocomposites (NCs) have been extensively used as medical implants and surgical instruments due to their antibacterial properties. Consequently, mechanical behavior analysis of these NCs is of crucial importance with emphasis on their plastic deformation mechanisms. From the materials science perspective, dislocations slip at the room temperature and high strain rates conditions is hindered. However, copper and silver, as two metals with low stacking fault energy are prone to twin formation. Since microstructural changes in these nanostructured composites occur at the atomic scale, molecular dynamics (MD) simulation is undoubtedly a great tool to use. Accordingly, in the present research, first, the deformation mechanism of perfect copper-silver NCs under uniaxial compression is deeply analyzed employing MD. This is followed by inspection of the voids effect on their plastic deformation process. The results show that twinning is the dominant mechanism governing their deformation under uniaxial compressive loading conditions. It is revealed that twins are created by the conversion of internal stacking faults to their external counterparts. Also, investigation of the microstructural evolution demonstrates that the presence of voids within NC samples provides new sites for nucleation of Shockley dislocations in addition to the interface zone. Finally, to address the effect of interfacial coherency on the results, copper-based NCs infused with gold and nickel nanoparticles are also thoroughly examined. Among these, copper-silver NCs can significantly improve the performance of biomedical implants due to their antibacterial effects3. It is worth mentioning that the use of nanocomposites for this goal may face several challenges. For example, an implant as a replacement of the damaged tissue should possess mechanical properties similar to those of the bone structure. Otherwise, under external loading conditions, deformation occurring on the implant is really different from that of the bone leading to the so-called implant loosening6. Accordingly, the analysis of their mechanical behavior and deformation mechanism is a focal point of research. From the materials science perspective, at the atomic scale, plastic deformation of nanocrystalline materials is achieved through twinning and/or slip of dislocations7. Although both of these mechanisms change the mechanical properties of the sample, however, their influence on the microstructural evolution can be completely different. To be more specific, dislocations slip creates local deformations, whereas twinning reorients the crystal structure with the cooperative movement of atoms. However, from the microscale point of view, since deformation by dislocations takes place in the whole material, plastic deformation occurs mainly through this mechanism8. One of the other factors limiting the twinning-based deformation is the fact that slip normally occurs in the discrete multiples of the atomic spacing. Meanwhile, the atom movements are much less than the mentioned parameter during the deformation twinning9.In recent years, metal matrix nanocomposites (MMNCs) have been widely used in health, transportation, electronics, and aerospace industries10. It also depends on the type of crystal structure. In BCC and HCP structures, due to the lack of sufficient active slip systems, deformation occurs by the twinning process. Similar behavior has also been observed in the low stacking fault energy (SFE) metals having the FCC structure such as copper, silver, and aluminum, which is ascribed to the inverse relationship between the SFE and twinning behavior13. Additionally, it has been demonstrated that twinning-based deformation is favored at higher strain rates and lower temperature conditions15.Several factors affect the type of plastic deformation mechanism that occurred within the nanocrystalline metals. For example, in the case of metals with nano-sized grains, twinning is the governing mechanism, whereas with increasing the grain size, dislocation slip is more likely to happen16. Carrying out molecular dynamics (MD) simulations, Weng et al.17 have also shown that the interfacial area is one of the most important sites for the nucleation of dislocations and therefore, can noticeably affect the mechanical behavior of nanocomposites. Regarding this issue, Pogorelko et al.18 demonstrated that copper nanoparticles form a semi-coherent interface with the aluminum matrix and as a result, efficient load transfer cannot be achieved from the matrix to the embedded copper nanofillers. However, the interfacial region can play a different role in the plastic deformation behavior of MMNCs. For example, in the study of Buehler et al.19, it has been observed that the niobium/copper interface would stop the slip of dislocations leading to the enhanced yield strength and morphological stability of the nanocomposite samples at the higher temperature conditions.Concerning the plastic deformation behavior of copper/silver nanocomposites, a key question naturally arises: While a good consistency is predicted for slip planes and slip directions of these two metals owing to their similar structure, why their composites exhibit a different behavior? Finding the possible answer to this question is the main aim of the present work. In this context, most previous studies have been devoted to analyze the plastic deformation behavior of MMNCs in the presence of the second phase. Among these, it has been demonstrated that the formation of stacking fault regions within the bulk matrix has a direct correlation with the geometrical features of the additive phase and its distribution20. However, the formation of these porosities is considered to have negative effects on the mechanical properties. This is attributed to the creation of partial dislocations in the interfacial region as depicted by Zhao et al.21 during the analysis of the defected single crystalline copper sample under tensile loading conditions. This arises because voids can resemble the dislocation generation sources due to the locally induced stress concentrations in their vicinity21. This issue was previously addressed by Zhan et al.22 for copper nanowires analyzed under uniaxial tensile test. It was concluded that porosities can act as new sites for the creation of dislocations, which in turn deteriorates the mechanical characteristics of these nanostructures.An important issue regarding the biomedical applications of these NCs is the adhesion of body cells to the implants, which improves the strength of the implant/tissue interface. It is worth mentioning that this phenomenon could be drastically enhanced through the formation of nanovoids within the bulk matrix25., in the present work, this special shape of Ag NPs has been considered in our models. This paper is organized as follows. The simulation details and sampling techniques are denoted in Section 2 along with introducing the crystal structure analysis tools including Common Neighbor Analysis (CNA), Centrosymmetry Parameter (CSP), and Dislocation Extraction Algorithm (DXA). The most important findings of this work are presented in Section 3 with emphasis on the mechanisms governing the plastic deformation of MMNCs under compression. Section 4 contains the concluding remarks.In recent years, extensive research has been conducted to examine the role of interfacial area, type and location of voids, and geometrical features of the second phase on the mechanical behavior of MMNCs under tension loading conditions. However, the influence of the above-mentioned phenomena on the mechanical response of these nanocomposites to the other loading conditions has rarely been considered using atomic-scale simulation techniques. In particular, to our knowledge, no study has been devoted to examine the mechanical characteristics of defected Cu/Ag nanocomposites under compression. Therefore, in this study, we utilize MD simulation to see the effects of voids content on the compression behavior of copper-based nanocomposites embedded with Ag NPs. Additionally, the underlying plastic deformation mechanisms are introduced and compared to those activated in the tensile loadings. It is worth mentioning that since spherical Ag nanoparticles have more antimicrobial effects compared to the other geometrical features such as nanoplatelets, nanorods, and etcAs mentioned before, this paper is aimed to assess the nanovoids effect on the mechanisms governing the plastic deformation of Cu/Ag nanocomposites subjected to uniaxial compressive loadings. In the following, MD parameters, samples preparation method, and the most common techniques implemented to analyze the crystal structure evolutions within the samples are introduced.https://lammps.sandia.gov)26. In line with the previous studies of various copper-silver systems29, the EAM potential was employed as the interaction model. This many-body potential has been successfully utilized to analyze the dislocations and stacking fault regions in metals, alloys, and MMNCs1. It should be pointed out that in the present work, the EAM potential developed by Williams et al.30 was utilized to mimic the interactions within the Cu-Ag samples. This version of EAM has been successfully employed to reproduce the lattice parameter, cohesive energy, elastic constants, lattice-defect energies, phonon frequencies, thermal expansion, and stacking faults energy of the Cu–Ag compounds. Additionally, comparing the simulated data with the experimental measurements, it has been shown that the Cu-Ag phase diagram can also be correctly predicted utilizing this potential, which demonstrates the validity of the EAM potential used in the current study. For each simulation, at first, it is required to relax the structure so that the energy of the whole system stabilizes at the lowest possible value. For this purpose, setting the temperature at 300 K, analysis of 200 ps at NVT ensemble was initially performed for all samples. Based on our previous study31, the value of 2 fs was considered as the time-step, which has proven to provide the accurate data at a reasonable computational cost for these systems. For controlling the temperature at the desired value, the Nose-Hoover thermostat was employed. After assigning the initial atomic velocities by the Maxwell-Boltzmann distribution, the well-known velocity Verlet scheme was implemented to advance the system variables32. Following the relaxation, to apply the uniaxial compression load on each sample, the left part was kept fixed  was utilized35.Simulations were carried out using LAMMPS  have been illustrated in Fig. The similar zigzag trend was found in the plastic region of copper-silver nanocomposites under tensile loading conditionszed Fig. –c. As di49. Porter et al.50 showed that for the semi-coherent interface, in the absence of external loading conditions, a cohesive strain is created at the interface with a direction opposite to that of the misfit strain to cancel out its effect. This is not the case happened after yielding in which, the matrix and nanoparticles are deformed resulting in the increase of the discussed atomic misfit. Accordingly, the cohesive strain would not be capable of preserving the continuity of the interface. Thus, this area would be a preferred site for dislocation nucleation. This issue has been experimentally confirmed for Cu-Ag systems in the literature51. For example, Zheng et al.52 introduced the Cu/Ag interfacial zone as the hetero-twin (non-coherent) interface, which is not capable of defects movement across the region  increases to facilitate the process See Fig. . TherefoSee Fig. . As seencompressive loadings. Nucleation of Shockley partial dislocations with concurrent formation of stacking fault regions between them has been enumerated as the main reason for the occurrence of the twinning morphology in the copper-based systems57. Our findings are in line with the experimental observations of Beyerlein et al.58, which confirm that the twinning phenomenon in the copper-silver nanocomposite starts from the interface. It should be pointed out that in addition to the interfacial region, the SFE is also an important parameter affecting the plastic deformation mechanism of these NCs. Accordingly, it can be concluded that if we change the role of the constituents in the discussed composite systems, the plastic deformation will be intensified through the enhanced twinning formation. This is attributed to the lower intrinsic SFE of Ag compared to that of Cu as predicted by Hunter et al.59.Therefore, based on the above discussions, it is clear that the twinning formation is main mechanism dominating the plastic deformation of copper-silver nanocomposite systems subjected to 60. It has been proven that this type of interface is formed for each pair of elements having more than 5% lattice mismatch50. In contrast, for the Cu/Ni sample, a coherent-like interface is formed due to the only 3.1% atomic mismatch between them60. This difference in the interfacial coherency would manifest itself in the emission of partial dislocations nucleated in the interface zone of these composite systems  were also examined under the prescribed compressive loading conditions. It is noted that similar to the Cu/Ag sample, the former resembles a case with the semi-coherent interface owing to 12.5% difference between the atomic radius of Cu and AuSee Fig. . As illu61. To shed more light on this issue, we also approached to explore the role of SFE on the plastic deformation behavior of the introduced samples. Figure 62), the sample embedded with Ni nanoparticle has the lowest number of HCP atoms. In this case, not only a high density of stacking faults cannot be formed within the Ni inclusion, but also, more importantly, their expansion through the interfacial area and finally, the bulk copper is hard to occur. Contrary to the former case, significant regions having HCP atoms were observed for Cu/Ag and Cu/Au nanocomposites, which is attributed to the low values of SFE reported for silver and gold NPs . Accordingly, compared to Ag NPs, the presence of Au inclusion could promote the plastic deformation of the bulk copper through the enhanced twinning formation. However, this mechanism would not be effective in Cu/Ni samples, presumably owing to the lack of sufficient partial dislocations and stacking faults in their interfacial area as the main resources needed for twinning formation.Another important factor leading to the different behavior of these MMNCs is the stacking fault energy. As discussed in the introduction, it has been confirmed that there is a reverse correlation between the SFE and the extent of stacking faults in the metals and metallic systemset al.63. As previously discussed, due to the presence of the hetero-twin interface in these systems, this region would be potentially a preferred site for enhanced deformation twinning. Accordingly, as seen in Fig. 64 demonstrated that this phenomenon could increase the stress required for motions of twins in the nanotwinned metals.As observed in the previous discussion, the type of inclusions is very important in determining the mechanism of MMNCs deformation. But, a vital question that should be answered is as follows: What is the size effect of the interfacial region on the deformation twinning? For this purpose, in addition to the introduced sample infused with the 2 nm diameter Ag NP, we further examined the copper systems embedded with Ag nanoparticles of 1.5 and 2.5 nm diameter having the VF of 1.98 and 3.25%, respectively. Figure mentclasspt{minimatensile loading conditions32.After examining the perfect Cu/Ag NC sample, the subject was thoroughly inspected in the case of defected samples to find out the role of topological voids on the creation of dislocations, stacking fault regions, twins and in general, mechanical features of the NC sample. For this purpose, three samples containing different voids content were subjected to uniaxial compression loading. Geometrical characteristics of the samples have been listed in Table et al.65. For a more detailed investigation of this issue, the cross-section of the nanocomposite sample containing 2.5% porosity was studied using the dislocation extraction algorithm (See Fig. et al.66 showed that the Cu/Ag interface possesses similar misfit dislocations to those of reported by our MD simulations in Figs. et al.49 analyzed the defect-interface interactions in the irradiated Cu/Ag NCs including some voids attached to the interface. TEM image shown in Fig. quantitative tool supporting this idea, the total length of Shockley partial dislocations and the number of stacking fault atoms were obtained for all of the three introduced samples. Figure Considering the proposed mechanism highlighting the role of twinning formation on the plastic deformation of these nanocomposites, it seems that the presence of voids would affect the creation of Shockley partial dislocations and stacking fault regions as the resources of twinning. This was attributed to the stress concentration around the voids as numerically confirmed by Traiviratana See Fig. –d. As seSee Fig. , Zheng eIn summary, several MD simulations were conducted to explore the dominant mechanisms affecting the plastic deformation of perfect and defected copper-silver NCs under compressive loadings. It was deduced that at the room temperature and high strain rate conditions, twinning formation is the governing mechanism. It was found that twins are created from the conversion of intrinsic to extrinsic stacking faults. We further proceeded to examine the proposed mechanism in other copper-based composite systems embedded with Au and Ni nanoparticles. The main aim was to assess the effect of interfacial coherency on the nucleation of partial dislocations and the formation of stacking faults as the main resources of twinning-based plastic deformation in MMNCs. Moreover, the results revealed that the presence of voids within the bulk samples provides more active sites for nucleation of Shockley partial dislocations. This is followed by the conversion of stacking faults to twinning, which leads to a significant drop in the mechanical features of the NC samples. It is worth mentioning that to support the main findings of the present research, several experimental-based observations were also provided throughout the article. Since various factors affecting the plastic deformation of MMNCs under compressive loadings have been thoroughly studied in this work, the introduced mechanisms can be used to tailor the mechanical performance of these NCs as medical implants.