Power lines are cables that carry electrical power from a power plant to an electrical substation. They must be connected between the tower structures in such a way that ensures minimum tension and sufficient clearance from the ground. Power lines can stretch and sag with the changing weather, eventually exceeding the planned tolerances. The excessive sags can then cause serious accidents, while hindering the durability of the power lines. We used photogrammetric techniques with a low-cost drone to achieve efficient 3D mapping of power lines that are often difficult to approach. Unlike the conventional image-to-object space approach, we used the object-to-image space approach using cubic grid points. We processed four strips of aerial images to automatically extract the power line points in the object space. Experimental results showed that the approach could successfully extract the positions of the power line points for power line generation and sag measurement with the elevation accuracy of a few centimeters. Power line cables supported by transmission towers carry electrical power from a power plant to an electrical substation. Tower structures have been widely used to transmit high voltage current and more than 40,000 towers have been constructed in South Korea . These tThe periodic 3D mapping of power lines is critical for power line maintenance. Power lines are mostly fabricated of ACSR  and are constructed with a proper dip by loosening the cable. The power line dip is defined as the difference in level between the points of support and the lowest point on the line. Maintaining the proper dip is important because, while a large dip decreases the tension for better safety, it also decreases the clearance from the ground. When power lines stretch and sag with the changing weather, they can eventually exceed the planned tolerances. Excessive sagging can cause serious accidents and degrade the durability of the power lines. In addition, the sagging cables can sway in strong winds, touching adjacent topographic features.Many remotely-sensed data products, such as SAR (synthetic aperture radar), thermal sensor, LiDAR (light detection and ranging), land-based mobile mapping data, and UAV , have been studied for power line surveys . LimitinRegarding the automated power line extraction in 2D aerial image space, Li et al.  presenteSome studies utilized 3D point cloud extraction of power lines from multiple aerial images. Yan et al.  used aerIn this study, we proposed the object-to-image space approach using cubic grid points for 3D power line mapping. The aim of the study was to derive 3D power line point cloud from multiple aerial images acquired using a low-cost drone. In the method, each aerial image provides 2D power line primitives that are used to filter the dense 3D cubic grid points generated around the power line on the ground. The relation between the image and object spaces is established by performing accurate bundle adjustment.The paper is structured as follows: In A flowchart of the proposed method is presented in Even low-cost drones provide functions for the autonomous flight mission. The flight plan including the flying height above ground and overlaps can be easily set up in the mobile devices. For power lines mapping, the safety and GSD (ground sampling distance) of images are taken into account for planning the flying height. The GSD can be estimated using the flying height, the focal length, and the pixel pitch in Equation (1). The information about the camera and sensor can be found in the specification from manufacturers. Typical overlaps of the aerial image acquisition range from 60% to 80% but higher overlaps are preferred for the facility mapping in a small region because they provide greater redundancy.The acquired images are tagged with the position and attitude information from installed GNSS  and IMU . However, the reliability of the system in a low-cost drone is not high so that the images need to be processed with the bundle adjustment for accurate IOPs and EOPs estimation .We carried out the power line pixel extraction from each aerial image using a simple template as shown in Applying the filter enhances the power lines that have strong responses and suppresses the backgrounds. Thus, the candidate pixels of power lines are extracted.The candidate pixels after the extraction are connected as line segments using connected-component labeling . In thisThe cubic grid points are generated around the targeted power lines in the 3D object space as in Given a cubic grid point, the grid point is projected to the multiple aerial images. In coordinates of an image point in the photo coordinate system .The object to image projection of a grid point is carried out using the well-known collinearity equation (Equation (2)). The equation includes EOPs and IOPs. EOPs are the positions and attitudes of the camera at the moment of exposure. IOPs are the parameters of the camera, such as the focal length and distortions.The rotation matrix After the power line points are selected from the cubic grid points, we generate the power line by interpolating the point cloud using the parabola equation. This equation requires the points of supports at the tower structures, which can be manually extracted as shown in The test site is in Gwangju City of Korea, as shown in We used a low-cost drone, the Phantom 4 (DJI), the specifications of which are shown in During the flight mission, four strips of images crossing the power lines were produced, as shown in Though the diameter of all ASCR power lines is around 1.5 cm, the power lines are shown at 2–3 pixels because of their linear pattern and the contrast with the backgrounds, as shown in We used Pix4Dmapper Pro for the bundle adjustment. The software uses the position and attitude information from installed GNSS and IMU sensors for the initial approximation. For the bundle adjustment, over 20,000 tie points per image and a total of 12 GCPs are utilized. The interior and exterior orientation parameters were adjusted together, i.e., on-the-fly self-calibration. We used a 3 × 9 horizontal line extraction filter, since most power lines run along the horizontal direction of the images. This filter generated noisy line components over the image space and the connectivity analysis was then used to label the components. To suppress noise, we selected only those line components with a length greater than 100 pixels. We set a region around the power lines in a map to generate cubic grid points. The red box in In this experiment, the cubic grid points were generated using two cases: less dense and dense. For the less dense case, we used intervals of 1 m, 5 cm, and 5 cm for along the power lines, across the power lines, and the elevation directions, respectively. For the dense case, we used intervals of 0.3 m, 2 cm, and 2 cm, for along the power lines, across the power lines, and the elevation directions, respectively. We then iterated the projection of the cubic grid points into each image strip which consists of five aerial images. In each strip, the number of images is counted when the projected location is on a line segment. If the number of images is larger than the established threshold, the point is classified in the power line point group. We carried out experiments for two threshold cases of four and five images. We interpolated the power line points using the parabola equation, connecting the points of supports at the tower structures. The resultant accuracy is acceptable for power line monitoring, which typical requires the elevation accuracy of 20 cm. We cannot directly compare the results to that of other studies because the system, the operation condition, and sites are different. But the result is decent that LiDAR systems show elevation accuracy within ±15 cm and previous studies on using aerial images reported the 3D power line extraction accuracy of 0.5 m referring to the ground truth data .We used photogrammetric techniques with a low-cost drone to achieve efficient 3D mapping of power lines that are often difficult to approach. Unlike the conventional image-to-object space approach that extracts image features, performs image matching, and reconstructs 3D coordinates, we used the object-to-image space approach using cubic grid points. We processed four strips of aerial images for automatic extraction of the power line points in the object space. We observed that changing the cubic grid point density did not significantly affect the sag measurement of the power lines. When we increase the threshold from four to five images, the population of extracted power line points was reduced but the line interpolation precision increased. Finally, experimental results showed that the approach successfully extracted power line points for the power line generation and the sag measurement at the accuracy of a few centimeters. For future study, more experiments are required for power lines in a variety of environments and backgrounds, such as snow. It will also include the 3D topographic mapping and analysis for near-zones of electric-power facilities using visible and multi-spectral sensors to extract hazardous topography.


Interobserver agreement was analyzed quantitatively by Cohen's kappa using the free QSEQ5 software program. Once we had confirmed the reliability of the data, these were analyzed by polar coordinate analysis, which is a powerful data reduction technique that provides a vector representation of relationships between categories. The results show significant relationships between the therapist and (1) the activation of turn-taking by the participants and the co-therapist and silence and (2) conversation-facilitating interventions and interventions designed to improve mentalization abilities. Detailed analysis of questions demonstrating interest in others showed how the communication changed from radial interactions stemming from the therapist at the beginning of therapy to circular interactions half way through. Repetition was found to be a powerful conversation facilitator. The results also illustrate the role of the therapist, who (1) did not facilitate interventions by all participants equally, (2) encouraged turn-taking from more inhibited members of the group, (3) stimulated conversation from the early stages of therapy, and (4) favored mentalization toward the end. Despite its complexity, polar coordinate analysis produces easy-to-interpret results in the form of vector maps.Group psychotherapy is a useful clinical practice for adolescents with mental health issues. Groups typically consist of young people of similar ages but with different personalities, and this results in a complex communication network. The goal of group psychoanalytic psychotherapy is to improve participants' mentalization abilities, facilitating interactions between peers and their therapist in a safe, containing environment. The main aim of this study was to analyze conversation turn-taking between a lead therapist, a co-therapist, and six adolescents over the course of 24 treatment sessions divided into four blocks over 8 months. We employed a mixed-methods design based on systematic observation, which we consider to be a mixed method itself, as the qualitative data collected in the initial observation phase is transformed into quantitative data and subsequently interpreted qualitatively with the aid of clinical vignettes. The observational methodology design was nomothetic, follow-up, and multidimensional. The choice of methodology is justified as we used an  Peer groups are a natural setting for young people Erikson, . In the Little has been published on group therapy in children or adolescents. Most of the studies conducted to date have reported on brief cognitive-behavioral interventions with specified diagnostic populations  facilitate conversation and (b) promote mentalization, i.e., stimulate thought, reflection, and understanding about oneself and one's relationships with others.ad-hoc observation instrument used in the study, we labeled this first group of interventions DYN, as they have a dynamic, stimulating function. They are interventions in the form os a request or question in which the emitter (the therapist or participants) show interest in the life of the receiver. Demonstrating interest in others by asking questions, allowing them to intervene, and showing curiosity in their answers is considered to be a specific benefit of group therapy as opposed to individual therapy  and intrasessional analyses , and multidimensional because the ad-hoc observation instrument contained various dimensions selected on the basis of the theoretical framework and our experience.There are eight possible study designs in observational methodology , the co-therapist (coT), and six adolescents . The adolescents (four boys and two girls) had requested support at the Center for Child and Adolescent Mental Health of the The inclusion criteria were (a) an age of 12–15 years and (b) recommendation for group therapy following diagnostic evaluation at the Mental Health Center. The exclusion criteria were (a) anticipated difficulty attending all the therapy sessions and (b) contraindication for group therapy.The group was led by an expert therapist, assisted by a co-therapist who participated as an observer. Both were clinical psychologists trained in group psychoanalytic psychotherapy.In accordance with the principles of the Declaration of Helsinki and the Ethical Code of the General Council of the Official College of Psychologists of Spain, the participants were informed that they were being filmed. They were shown the location of the video cameras, which were positioned discretely to minimize reactivity bias. Informed consent was also obtained from the parents of the minors.In systematic observation Anguera, , a distiThe group sessions were recorded using two video cameras, two microphones, two video units, and two screens comprising a closed-circuit television system. The dataset was built in the software program GSEQ5, v.5.1  and other behaviors . Associations between pairs of behaviors are represented graphically by vectors. Polar coordinate analysis requires a prior stage consisting of lag sequential analysis  and n is the number of lags considered. The Zsum is calculated for each target behavior for both positive lags (prospective Zsum) and negative lags (retrospective Zsum). The technique thus yields a statistical relationship between the given behavior and each of the target behaviors, which is reflected by a prospective and a retrospective Zsum value, as proposed by Sackett  when it exceeds 1.96 The angle of the vector is calculated as follows: φ = arc sen Polar coordinate analysis uses the prospective and retrospective ZThe meanings of the different quadrants are shown in Figure Quadrants I and III are symmetrical in terms of the relationship they depict between the focal behavior and the different conditional behaviors they contain. Quadrant I (++) indicates mutual activation while quadrant III (−) indicates mutual inhibition. Quadrants II and IV, in turn, depict asymmetrical relationships. Quadrant II (−+) indicates that the focal behavior inhibits but at the same time is activated by the conditional behaviors, while quadrant IV (+−) indicates the opposite .z values in addition to analytical parameters and polar coordinate maps. The analysis was conducted by exporting the data file from GSEQ5 to HOISAN.The polar coordinate analysis for this study was performed in HOISAN v. 1.6.3.2  and the conditional behaviors were interventions by the participants , interventions by the co-therapist (coT), and silence (Q) in the four blocks of sessions spanning 8 weeks.As shown in Table The graphs in Figure Again, the focal behavior was intervention by the therapist (T) and the conditional behaviors were the DYN categories FF, FO, RP, RT, QA, QC, and QV and the MNT category.The majority of results in this case were also significant Table .The graphs in Figure Below we discuss the significance of the relationships detected by polar coordinate analysis in five sections. We also illustrate our findings with clinical vignettes containing coded transcripts of the interventions.All the significant results are located in two opposing quadrants, indicating two clearly differentiated types of relationship: mutual activation and mutual inhibition. The therapist always facilitates intervention by Fred, the participant with the greatest difficulty relating to others, and in the early phases of therapy, she also encourages interaction from Danny, John M, and Meg. Her interventions never activate those of the two impulsive participants, Gabriel and Lucy. This does not mean that she excludes these participants, simply that they intervene on their own initiative. The changes detected in Danny, John M, and Meg are an indication of the progress they make over the therapy. Block 1 is characterized by radial communication between the therapist and all the participants. Vignette 1 shows an example of an interaction between the therapist and Danny Table .However, not all interactions are the same. Gabriel and Lucy, for example, spontaneously take turns in these early sessions Table .Lucy raises conflicts about herself that interest everyone Table .John M is a reserved person with anxiety problems. He has difficulty intervening and when he does, he often mumbles, says very little, and adheres to what has just been said Table .Haen and Weil  have higThe co-therapist and the therapist was mutually activated (quadrant 1). The co-therapist's interventions reflect her role of interfering as little as possible in the group dynamics. They complement those of the main therapist. Together, they form a team and create and maintain a safe environment .The examples below show how the adolescents fall silent when faced with difficult issues, such as verbalizing why they are in the group or talking about their relationship with their parents or their concerns about sexuality Tables –12.The different strategies for facilitating conversation  showed varying patterns of change over the course of therapy but converged at the end.Repetition (RP) was the most powerful strategy, as it activated conversation from the start of the therapy program. The next most powerful strategies were phatic function (FF) and greetings (FO). The transcripts of the sessions show that in the early sessions, it was the therapist who verbally greeted the adolescents (by saying hello and goodbye). However, few of them responded and the others returned the greeting or made a non-verbal gesture. This behavior changes after the first block, indicating an increase in reciprocity between the therapist and the participants.The appearance of QA (questions directed at others) in the second half of the therapy is, in our opinion, a highly significant indicator of the group process. It tells us that the communication is no longer radial and that the adolescents have achieved one of the most important benefits of group therapy, which is showing interest in others Yalom,  in the pIt is also interesting to see how QV (repetition of a previous utterance in the form of a question) changes from being mutually inhibitory to being mutually activating. We think that this strategy initially surprised the adolescents but was then gradually adopted by them. The same was not observed for QC (clarifying questions), which were used only by the therapist when the adolescents were “doing their own thing” and she was “excluded” from the group. Examples of what she said were: “I'm not quite following you now…maybe I'm being a bit dense, can you help me understand what's going on?” This strategy is similar to the attitude of respectful curiosity shown by therapists in the Adolescent Mentalization-Based Integrative Treatment (AMBIT) approach , which is designed to help put a halt to non-mentalization mental states  and suggesting looking at this in greater depth was only done by the therapist.At the end of therapy, all the categories in the DYN dimension except RT are located in the mutually activating quadrant. This supports the idea that the communication strategies used by the therapist were adopted by the participants, enabling them to talk more autonomously and facilitating their personal growth Yalom, .The changes observed in the MNT category, which corresponds to interventions aimed at improving the adolescents' mentalization abilities, also reflect interesting aspects of the group process. The MNT category changed from inhibitory (quadrant III) to partially inhibitory (quadrant II) and finally to mutually activating (quadrant I). The changes also show that the therapist's role changed over time, as mentalization strategies were only used by her. We can deduce that the participants gradually overcame their early inhibitions and dependence and acquired more sophisticated mentalizing abilities, helping them to become more aware of themselves and of others. This result is consistent with the concept known as the interpretative function of the therapist within the theories of Foulkes  and TorrPolar coordinate analysis provides a new approach for gaining insights into dialogue in group pyschotherapy. The results show that the technique provides a novel means of analyzing the role of the therapist and describing her conversational style. The therapist proved to be an expert in creating a communicative environment that allowed the adolescents to grow. She employed four core strategies: (1) she did not facilitate communication equally for all participants, (2) she encouraged turn-taking by the more inhibited members of the group, (3) she stimulated conversation from the early stages of therapy, and (4) she promoted mentalization toward the end of therapy.We were particularly pleased to see that the use of repetition (RP) facilitated communication flows from the beginning. The positive results indicate that rather than simply acting as an echo or a loudspeaker, this strategy produces a mirroring effect similar to that described in the social biofeedback theory of parental affect-mirring (Gergely and Watson, Observational methodology and polar coordinate analysis could prove to be of great value for detecting changes in psychotherapy models based on spoken conversation.EA developed the project. MA performed the method section and polar coordinate analysis. Both authors have participated in the writing of the article.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


This study aims to explore the effect of the neighborhood scale when estimating public services inequality based on the aggregation of social, environmental, and health-related indicators. Inequality analyses were carried out at three neighborhood scales: the original census blocks and two aggregated neighborhood units generated by the spatial “k”luster analysis by the tree edge removal (SKATER) algorithm and the self-organizing map (SOM) algorithm. Then, we combined a set of health-related public services indicators with the geographically weighted principal components analyses (GWPCA) and the principal components analyses (PCA) to measure the public services inequality across all multi-scale neighborhood units. Finally, a statistical test was applied to evaluate the scale effects in inequality measurements by combining all available field survey data. We chose Quito as the case study area. All of the aggregated neighborhood units performed better than the original census blocks in terms of the social indicators extracted from a field survey. The SKATER and SOM algorithms can help to define the neighborhoods in inequality analyses. Moreover, GWPCA performs better than PCA in multivariate spatial inequality estimation. Understanding the scale effects is essential to sustain a social neighborhood organization, which, in turn, positively affects social determinants of public health and public quality of life. Analyzing the relationship between the state of the public social services and environmental health has become a major issue for quality of life analysis ,2. PubliTwo key aspects of research on the impact of public services have been identified . The firThe second focus of public service inequality is concerned with the identification of geographical areas with public health service inequalities and exposure to potential hazards to human health ,21. VariTraditional statistics , Rank Sum Ratio, Delphi) have been used to estimate the contributions of multiple accessibility variables in describing the characteristics of public services ,36,37. NSome previous studies on public services characterized spatial, environmental, and social indicators on only a certain scale, and they used multiple accessibility variables to evaluate inequality ,44,45 wiA key question regarding the MAUP is how to apply user-controlled neighborhood analysis and flexible geographical units by developing sensible and appropriate zoning systems for particular purposes ,61,62,63The self-organizing map (SOM) algorithm ,73, as aIn this study, we aim to explore the neighborhood effects on measuring inequality in public services by considering these services as opportunities to define positive socially determined health outcomes and inequality as the level of access to these services. Based on the analysis of the above-mentioned literature on public service analysis, we selected the multiple accessibility variables that properly represent the public services and are available from the statistical institutes. Firstly, we established multi-scale neighborhood units based on two regionalization approaches: the SOM and SKATER algorithms. These regionalization approaches were based on a composite index of deprivation. Secondly, we applied the PCA and GWPCA to describe the inequalities within these different multi-scale neighborhood systems by analyzing area-based indicators of access to public services. Then, we validated our inequality analyses using individual-based social indicators extracted from a field survey. Is it appropriate to use SKATER and SOM models to define the socially meaningful neighborhood? What are their sensitivities, their strengths and limitations in social public service inequality analysis?What are the advantages of the GWPCA in measuring inequality in public services within multi-scale neighborhoods compared with the PCA?In this study, we discuss the following research questions:Our study area is the city of Quito, the capital of Ecuador . It is hIn Quito, residential land use can be found throughout the city, but it is particularly concentrated in the northern and suburban zones of the city. In the south, land use primarily consists of industrial and commercial zones and low/medium income residential areas. The indicators used to characterize public service inequality are shown in Additionally, analyzing geographical accessibility to urban resources  allows the identification of spatial and social inequalities with important implications for health planning ,85. Thisthe total number of households in a specific census block and the number of households that do not have access to a specific public service. These indicators are expressed as ratios  and include (1) no access to drinking water (NonDri); (2) no access to the sewerage system (NonSew); (3) no access to the public electricity grid (NonEle); and (4) no access to a garbage collection service (NonCol). The variables used to calculate these indicators were extracted from the 2010 Ecuadorian Population and Housing Census. Four indicators were calculated using variables of the 2010 Ecuadorian Population and Housing Census and all indicators are based on census blocks (dissemination areas). Four indicators were estimated based on the parameters of Two indicators concerning the accessibility to health services and to educational services were calculated. The geographical accessibility was considered, which, in the case of this study, was calculated as the Euclidean distance to the nearest healthcare service  from the centroid of each of the census blocks. This kind of distance has been proven to be a useful measure of spatial accessibility and a proper proxy of travel time ,88. The Finally, the indicator limited access to green areas was calculated, which is also considered important for the understanding of urban inequalities and their impact on health . Remote The field survey conducted as part of this study was carried out in the months of July, August, and October 2014 and encompassed different questions related to quality of life, health, and social dimensions. The survey was conducted throughout the entire study area using a two-stage sampling strategy. In the first stage, ARCGIS tool  was applied to divide the study area into sampling areas (hexagons). Eighteen hexagons were randomly chosen based on the time and financial resources available for the survey. In the second stage, pseudo-random interviews were carried out in each sample area using a door-to-door interview petition to obtain answers in households where people were willing to participate. The number of interviews also depended on the population density and land use of each hexagon. In areas with a higher population and urban cover, more pseudo-random interviews were conducted. The survey included questions regarding individual health conditions, social cohesion of the neighborhood, and neighborhood security. The answers to these questions were rated using a 1–5 Likert scale, whereby, for example, in the case of the self-perceived quality of life, a value of 5 indicates that the interviewed person considers themselves to have a very good quality of life. The same logic was applied to the other variables. Four variables were obtained: self-perceived health status (HS), the perceived social cohesion of the neighborhood (NC), the perceived security of the neighborhood (NS), and the self-perceived quality of life (QoL).The response rate of the survey was 61%. A total of 489 responses was obtained. The random hexagon strategy resulted in obtaining responses in socio-economically distinct neighborhoods. The survey is statistically reliable: 54.6% of the interviewees were female and the sampling margin of error obtained was ±4 (95% level of confidence). As shown in The second stage of the study was the evaluation of public services inequality. We applied the GWPCA and PCA separately to integrate the seven indicators of public services inequality  into the factor analysis to estimate the weights of these indicators in order to construct measures of public services inequality. These measures were calculated within the different zoning systems. The final step was the assessment of MAUP effects on the analysis of public services inequality. This assessment is related to the evaluation of uncertainties in the analysis of inequality. Increasing evidence suggests that the urban poverty situation is a key determinant of population health . We thusThe SOM method has several advantages: independence from external evaluation functions, recognition of the most meaningful features within vector space, and auto stability of the network . This meAll of the original objects were treated as Kohonen neurons, whereby each neuron has its weight vector indexed with the same attribute dimension, in this case, the deprivation index. Hexagonal grids are the preferred choice for presenting neurons then each node then has six immediate neighbors. After initializing all neurons’ weight vectors, we randomly chose a data point from the training data for the SOM. We then determined its best matching unit (BMU) on the map by measuring the Euclidean distance between the weight vector for each of the Kohonen neurons. The closest Kohonen neuron  is the chosen neuron. The weights of this chosen neuron and the neurons included within the neighborhood of the BMU are adjusted by the neighborhood function. This learning process consists of selection and adaptation of the synaptic weights until the Kohonen neurons converge to a unique limit. This method has been fully described by Kohonen .SKATER is an efficient regionalization technique that uses minimum spanning trees (MST), which consist of a connected tree with no circuits. It transforms the regionalization problem into an optimal graph partitioning problem . The minIn our research, we created an original MST by establishing a connectivity graph with a set of vertices (V) and a set of edges (L) for the original regions to capture the adjacency relations between objects. The original MST was defined by a Euclidean distance cost d with the edge  between the neighborhood objects i and j using their attribute vectors xi and xj. In our MST, the deprivation index represents an unequal attribute to measure the dissimilarity between objects. This index is measured in standard deviation units in order to have comparable scales. Based on the original MST, we used a hierarchical division strategy to prune the MST and obtained a set of spatial clusters. Each resulting cluster will be a tree with all vertices connected and no circuits. The pruning produces a reduced graph that defines a smaller class of possible partitions whose edges join similar categorical attribute areas. In order to maintain consistent levels of the geographical units with the regionalization SOM, we defined the initial node as 40 for the hierarchical division procedure. This procedure has been fully described by AssunÇão et al. .Our public service inequality analysis began with the identification of multiple accessibility variables linked to data structures or potentially linked to the data structure. The PCA and GWPCA were applied separately to organize these variables. PCA was applied to identify the correlation between the public services indicators and to convert a set of possibly correlated accessibility variables into a set of values of linearly uncorrelated variables called principle components. GWPCA was also used to identify the correlation of the datasets based on their spatial heterogeneity and spatial autocorrelation information. PCA and GWPCA provide the inputs for the factor analysis, which will determine the weights of selected factors in the linear regression model of the public services inequality analysis. PCA is one of the most popular dimensionality reduction methods; it groups together individual collinear indicators to form a composite indicator that captures as much of the information as possible . The maiThe standard PCA does not consider spatial effects. As an adaptation of the PCA approach, the GWPCA takes the spatial autocorrelation in the spatial process into account ,93,94. IDepending on the correlation structure of the indicator set, it is possible to identify a certain number of latent factors  to represent the indicator set through a factor analysis . In the Two measures of public service inequality were calculated by adding the weighted normalized indicators: one measure is based on the weights extracted from the PCA and the other measure is based on the weights extracted from the GWPCA. In order to validate the measures of public services inequality, data extracted from a field survey, including the variables of individual self-perceived health status (HS), the perceived social cohesion of the neighborhood (NC), the perceived security of the neighborhood (NS), and the individual self-perceived quality of life (QoL), are applied and treated as the ground truth data.t-test, and can be used when two independent groups are ordinally scaled [p-values of the “matched-pair”  were used to determine whether there were statistically significant differences between the underlying distributions of each paired sample. The Wilcoxon signed-rank test is a nonparametric test, analogous to the independent samples y scaled . This teIn the GWPCA, a specific localized PCA is estimated at a target location, permitting a local identification of any change in structure of a multivariate data set. Harris et al.  integratIt is important to emphasize that GWPCA reveals local spatial structures in a given multivariate data set when compared with global PCA. Using the global PCA, it was possible to ascertain that the first component loading of seven multiple accessibility indicators encompassed 67.46% and 65.29% of the percentage of the total variances (PTVs) for the SKATER-based and the SOM-based zoning system, respectively, and that these values are higher than the 42.34% PTVs found in the original census blocks. However, it is still difficult to gain insights into the spatial distribution of each composite variable and capture their dimensionality variations in the multivariate dataset. In the GWPCA, these investigations and interpretations all take place locally in each neighborhood unit, that is, the PTVs for the first component in two aggregated zoning systems and in the original census blocks are all summarized at each data location in the study area. As shown in The spatial patterns in the PTVs vary significantly between different neighborhood units and different scales. This is especially true for the north of the city where spatial patterns are very sensitive to the scale of the geographical units, ranging from 80.58% to 42.78%. The highest PTVs in the three zoning systems are generally located in the south of the city, while the lowest PTVs are located in the city center. It is also possible to identify the number of components to retain, given some pre-specified threshold of total variance preserved. For example, for a threshold of 85%, only the first principal component is required for the southern regions of Quito in all zoning systems. In many regions of central and northern Quito, the first two (or more) components are required to represent the significant spatial variation of public services indicators values.PCA and GWPCA analysis are used separately to group seven accessibility variables  to capture as much of the multivariate information as possible. After weighting the score of each principle component in the PCA- and GWPCA-based groups through factor analysis, these two groups of intermediate principle components are collinearly combined to estimate the final inequality results. This inequality workflow is applied to the original census block system in a, the SKIt is worth noting that there are similar patterns of public services inequality based on the PCA- and the GWPCA-based measurements for all three zoning systems shown in . NeverthThe definition of neighborhood also has a significant influence on the public service inequality estimation. As shown in We used the field survey indicators, namely self-perceive health status (HS), the perceived social cohesion of the neighborhood (NC), the perceived security of the neighborhood (NS), and the self-perceived quality of life (QoL) to validate our public service inequality assessment. As shown in In the case of the GWPCA-based measures of public services inequality, levels of public service inequality vary significantly between the different neighbourhoods and zoning systems. Therefore, the neighborhood units based on two computer-aggregated zoning systems correspond well to the neighborhood cohesion and neighborhood security; the neighborhood units in the SKATER zoning system especially coincide with the human perception of health-status and quality of life.To highlight the spatial heterogeneity of the set of public services accessibility indicators, we chose two representative areas of Quito to demonstrate the local outlier of their multivariate structure separately, namely the historical downtown or old city center, and the modern downtown or new city center. Multivariate structure analysis was applied to all the public services accessibility indicators: no access to drinking water (NonDri), no access to the sewerage system (NonSew), no access to the public electricity grid (NonEle), no access to the garbage collection service (NonCol), limited access to health care services (Dist_H), limited access to educational services (Dist_E), and limited access to green areas (Green).The spatial heterogeneities in the structure of the indicators within the three zoning systems are summarized in In the original census blocks , the locWe thus conclude that the spatial heterogeneity of the dataset derived by GWPCA allows us to evaluate the uncertainty of the MAUP effect in accessibility indicators analysis to some extent, and the aggregated zoning procedures based on the SOM and the SKATER methods perform well in clustering neighborhoods with similar public service capabilities. The exploration of functional neighborhood units for the optimization of health-related public services is the potential of this study. The two aggregated zoning systems created in this study enable delineating functional neighborhoods that are in accordance with the reality of the social and economic characteristics of Quito. The statistical test of these three zoning systems confirms the previous study and the conclusion that, if properly carried out, regionalization approaches can produce spatial entities that are more useful for spatial pattern analyses than the original data set . FurtherWe have shown that the social and health-related indicators used to calculate public services inequality represent a degree of spatial heterogeneity across multi-scale geographical units, not considering the MAUP effects in spatial dataset analyses may generate misleading results for decision-makers. Our results show that even though the PCA is a statistical technique widely used in many studies, it ignores spatial effects and spatial characteristics in multivariate data sets, and the PCA-based inequality results do not perform better than the GWPCA-based inequality results in relation to the field survey data. The GWPCA-based inequality results clearly show the spatial heterogeneity of data structures and regional differences in public services analysis. The GWPCA also enables full use to be made of geographically weighted correlation coefficients in a certain observation location to identify the MAUP effect. For example, the old city center of Quito does not show the obvious differences in the structure of the multivariate data set with its neighborhood at a large scale, meaning that the public inequality analysis at this site is not likely to be biased by scale and zoning effects at a large scale. In other words, the GWPCA is valuable for investigating the correlations of each indicator with the locally derived components and showing the indicators that differ most significantly compared to those observed in the surrounding neighborhoods.self-perceived quality of life, the self-perceived health-status, and the perceived neighborhood cohesion and security was possible in the two zoning systems created. Nevertheless, different clustering procedures largely depend on the structural features of the original blocks and on the capability of different regionalization methods. For example, SKATER can only produce a good-quality partition in a highly efficient manner if the original area has high homogeneity characteristics and is sensitive to the choice of the partition trees, which significantly influence the quality of the resulting partitions. SOM is an unsupervised learning algorithm and works well to improve spatial correlations of the data set, but lack of data or extraneous data will create randomness in the groupings. A better identification of changes in the variables The resolution and the quality of the data are also important criteria when designing new zoning systems because the regionalization approaches largely depend on the characteristics of the aggregated data set. How these approaches best integrate a set of social, economic, or environmental indicators into homogeneous contiguous regions is a question that needs to be further discussed and investigated in future research.This study focused on indicators, which were based on census variables and distances to services. Such indicators can be obtained for most regions of the world. Therefore, the methodology developed in this study can be applied to other case studies while different methods and techniques can be used for the construction of other, locally relevant indicators. For instance, in the case of accessibility to health care services, indicators based on gravity models and services catchment areas may be used. The Euclidean distances used can easily be adapted to local needs or other measures of spatial accessibility. However, we consider it important to also conceive accessibility as a multidimensional measure, which includes not only geographical variables, but also social, cultural, and economic variables. Future research may apply our methodology using multi-dimensional measures of accessibility. We are also aware that the threshold for the partition in the regionalization algorithms is at risk of being subjective. The use of thresholds is only suitable to optimize a particular statistic or model, thus providing a map-based visualization of the interaction between data, model or statistic, and the level of aggregation. We recognize that there is no decisive way to define the most suitable scale because the definition of geographical unit scale or neighborhood scale depends on the combination of characteristics in many different disciplines. Therefore, the complex urban system, including environmental conditions, neighborhoods, and residential effects, would result in varying public services based on the scales of different geographical units. The multi-scale zoning systems provided in this study offer the possibility but not the solution to explore the MAUP effect in the evaluation of public services inequality. Generally, the principal difficulty concerning zone-design is to choose the most suitable design for a particular purpose. The two zoning systems created for our study are homogenous scenarios in terms of deprivation, in which measures of public services inequality were calculated and then evaluated from a MAUP perspective. However, we consider this study as a first step in order to evaluate suitable scales to analyze inequality. The explanation of inequality related to access public services or any other kind of inequality requires different models and approaches such as regression analyses. Areas located in Quito´s periphery are areas with high deprivation ,80. Our We proposed a framework to analyze public services inequality considering seven indicators and applying PCA and GWPCA. These analyses were performed in three zoning systems: one was the original census block system of the study area, and the other two zoning systems were based on the homogenization of deprivation values using the SOM and SKATER algorithms. The variations of the indicators across multi-scale neighborhoods allowed a better understanding and interpretation of public services inequality considering a geographical perspective.This methodology can be used to better monitor inequalities in a given territory and to partially overcome the scale issue related to data aggregation. It is also an attempt to explore the spatial representativeness of a data set: the spatial heterogeneities and spatial autocorrelations may have important implications for results and conclusions used by decision makers and planners in the functional delineation of neighborhoods. Our study also reflects the well-recognized difficulty of defining a neighborhood, both at the conceptual and the operational level. We propose the consideration of additional variables in future studies, e.g., socioeconomic indicators to represent potential threats that can enhance social inequality, to further explore the implications of neighborhood on the basis on this methodology. Our results encourage further research to develop concepts and methods for analyzing social and environmental factors by considering different spatial and non-spatial hierarchical levels. Urban regionalization can be viewed as a dynamic process: subjective boundaries mainly depend on the location of the residents and characteristics of public services. We need neighborhood definitions that relate to public services and reflect social determinants of public health, but also serve as measures of residential segregation—localities that share similar sentiments, traditions, and history. The concept of neighborhood and its definition are, therefore, central to residential segregation and residential location choice analysis. The combination of urban regionalization methods with the analysis of social variables within multi-scale neighborhood zoning systems is crucial to supporting actions in reducing social inequalities and to supporting a local needs-centered urban planning.


Examples are, e.g., collision avoidance, navigation and route planning, active sensing of its environment, or the interaction and non-verbal communication with people in the extended reach space. Here, we focus on the recognition of the action of a human agent based on a biologically inspired visual architecture of analyzing articulated movements. The proposed processing architecture builds upon coarsely segregated streams of sensory processing along different pathways which separately process form and motion information (Layher et al.,  Analyzing and understanding the actions of humans is one of the major challenges for future technical systems aiming at visual sensory behavior analysis. Acquiring knowledge about what a person is doing is of importance and sometimes even crucial in a variety of scenarios. In the context of automated surveillance systems, action analysis is an essential ability, allowing to identify potential threads emanating from an individual or a group of persons. In Human-Computer-Interaction (HCI), action analysis helps in understanding the objectives and intentions of a user and increases the potential of a system to adapt to the specific context of an interaction and appropriately support, guide or protect the user. Moreover, recognizing actions in the surrounding area is an integral part of interpreting the own situative context and environment, and thus is in particular crucial for mobile robotic systems which may find themselves embedded in a variety of different situations.feature-driven key pose selection method is proposed, which is driven by combining two features in the biological motion input, namely extrema in the temporal motion energy signal and the relative extent of a subject's pose. Such temporally defined features (from the motion stream) help to automatically select key pose representations. The use of these dynamic features has been motivated by psychophysical investigations . In a detection task, subjects were capable of recognizing a walking motion within about 200 ms . STS has been suggested to be a point of convergence of the separate dorsal “where” and the ventral “what” pathways  and medial superior temporal (MST) areas of the dorsal stream (implied motion), although there is no motion present in the input signal , and local representations, which describe image and motion characteristics in a spatial or spatio-temporal local neighborhood. Prominent examples for the use of whole body representations are motion history images (MHI)   Yamato et al., dynamic time warping (DTW) Chaaraoui et al., In computer vision, the term vision-based action recognition summarizes approaches to assign an action label to each frame or a collection of frames of an image sequence. Over the last decades, numerous vision-based action recognition approaches have been developed and different taxonomies have been proposed to classify them by different aspects of their processing principles. In Poppe , action global representations together with a direct classification scheme. Key pose frames are considered as temporal events within an action sequence. This kind of action representation and classification is inherently invariant against variations in (recording and execution) speed. We do not argue that modeling temporal relationships between such events is not necessary in general. The very simple temporal integration scheme was chosen to focus on an analysis of the importance of key poses in the context of action representation and recognition. Because of the relevance to the presented approach, we will briefly compare specifically key pose base action recognition approaches in the following.The proposed approach uses motion and form based feature properties to extract key pose frames. The identified key pose frames are used to learn class specific key pose representations using a deep convolutional neural network (DCNN). Classification is either performed framewise or by temporal integration through majority voting. Thus, following the taxonomy of Poppe , the appfeature-driven events occur within an observed action sequence, neither during the establishment of the key pose representations during training, nor while trying to recognize an action sequence. Others regard key pose selection as the result of a statistical analysis, favoring poses which are easy to separate among different classes or maximally capture the characteristics of an action sequence. The majority of approaches rely on such statistical properties and either consider the intra- or the inter-class distribution of image-based pose descriptors to identify key poses in action sequences.Key pose based action recognition approaches differ in their understanding of the concept of key poses. Some take a phenomenological perspective and define key poses as events which possess a specific feature characteristic giving rise to their peculiarity. There is no a priori knowledge available about whether, when and how often such intra-class properties of the feature distributions regard key poses as the most representative poses of an action and measures of centrality are exploited on agglomerations in pose feature spaces to identify the poses which are most common to an action sequence. In Chaaraoui et al.  is applied to account for different temporal scales. The class of the closest matching temporal sequence of key poses from the training set is used as the final recognition result. Based on histograms of oriented gradients (HOG) and histograms of weighted optical flow (HOWOF) descriptors, Cao et al.  and thus key poses are dominated by frequent poses of an action. Because they are part of transitions between others, frequent poses tend to occur in different classes and thus do not help in separating them. Infrequent poses, on the other hand, are not captured very well, but are intuitively more likely to be discriminative. The authors' are not aware of an intra-class distribution based method which tries to identify key poses based on their infrequency or abnormality .In all, key pose selection based on an inter-class distribution, on the other hand, consider highly discriminative poses as key poses to separate different action appearances. Discriminability is here defined as resulting in either the best classification performance or in maximum dissimilarities between the extracted pose descriptors of different classes. To maximize the classification performance, Weinland and Boyer  to selectively learn representations of key poses which are assigned to different action classes. In the preprocessing phase, optical flow is calculated on the input sequences and key pose frames are selected in an unsupervised manner. Form and motion information is calculated for each key pose frame. The concatenated form and motion information is then used as the input to the DCNN. In the following, detailed information about the image preprocessing, the key pose selection automatism and the structure and functionality of the DCNN are presented. All simulations were carried out using a neuromorphic computing paradigm and mapped to the IBM TrueNorth hardware platform , the optical flow u = , v) at timestep t and position x =  is estimated in a local neighborhood N(x) by minimizingThe key pose selection process operates upon two parameters, namely (I) local temporal extrema in the motion energy and (II) the extendedness of a subject at a given timestep. Optical flow is calculated using a differential method, as suggested in the W(x − y) increases the influence of the optical flow constraints within the center of the local neighborhood )Lucas-Kanade algorithm is not a hard prerequisite for the proposed approach. Other types of optical flow estimators might be applied as well  framework  in the spatial and (f) in the feature domain and, wxyfj the respective synaptic weights. In the following, we will use the linear index i to denote locations in the (xyf) space-feature cube. Normalizing the weighted sum over a set of input samples  allows to accelerate the training of the network by standardizing sj as defined through:where j the mean and σj the standard deviation of s calculated over the number of training examples within a batch , and ϵ guarantees numerical stability. The output activation of the artificial neuron is calculated by applying an activation function on the standardized filter response:with C given the current and the desired response of the output layer cell activations, as defined by:Weight adaptation is performed through gradient descent by applying error backpropagation with momentum  of a cell j with activation rj. A softmax function is employed as activation function in the output layer, as defined through:with E(t) = C is then propagated backwards through the network and the synaptic weight adaptation is calculated for all cells in the output and hidden layers by applying the chain rule. The strength of weight adaptation Δwij is given through:The cross entropy error k is the index of cells in the layer succeeding cell j, t describes the current training step, or iteration, and η denotes the learning rate. The momentum factor 0 ≤ α ≤ 1 helps the network to handle local minima and flat plateaus on the error surface. After the backward pass, weights are finally adapted by:which includes a momentum term for smoothing instantaneous weight changes. Here, To ensure the compatibility to neuromorphic processing principles, a binary activation function wij of a cell j are shared over multiple input fields, which are arranged as a regular grid in the source layer. The calculation of the weighted sum during the forward, as well as the integration of the error derivative during the backward pass can be formulated as a convolution with the input from the source, or the error signal from the succeeding layer. The weights wij act as the filter (or convolution) kernel, rj as the output of an artificial cell. The size and stride of a filter allow to adjust the size and the overlap of the input fields to a filter in the source layer. A small stride results in an increased overlap and thus a large number of output values. The number of features defines how many different filters are employed in a layer. The weight matrices of the cells within a layer can be separated into groups of filters, which define the set of input features from the source layer covered by a filterWithin a convolutional layer, weights Multilayer Perceptrons/MLPs) on top for classification purposes. In contrast, the proposed network follows the strategy of global average pooling (gap) proposed in Lin et al. (vj and propagated back through the network (according to Equation 16). Networks using parameter-free global average pooling layers in combination with softmax are less prone to overfitting (compared to MLPs) and increase the robustness to spatial translations  fully connected layers  perform a standard convolution operation, pooling layers (POOL) reduce the spatial dimensions by applying a convolution with a large stride  are realized by convolutional layers with a size of 1x1 and a stride of 1 and act as cross channel integration layers . For the backpropagation of the error signal, the derivative of the binary activation is approximated linearly in the range of , as given through:and ternary synaptic weights  and preprocessed as described in Section 3.1. The resulting input map Iinput is fed into the DCNN and the class label c associated to the cell population in layer 15 with the maximum average output Iinput. The value of For framewise classification, a key pose frame is identified in an input image sequence n[frames], which are shifted over the input sequence I. The class with the most frequent occurrence of key pose frames determines the class predicted for the window (majority voting). At the moment, we do not use the confidence In sliding window based classification, the predicted class labels for key pose frames are collected within temporal windows of size Full sequence classification follows the same principle as sliding window based classification, but collects all key pose frames within a sequence. Thus, the amount of temporal information integrated in the voting process might differ substantially from sequence to sequence.uulm multiperspective action dataset (uulmMAD). In addition, we analyzed the performance on the widely used Weizmann dataset to allow a comparison to other approaches and to perform a cross-dataset evaluation of overlapping classes. In the following, we will briefly describe the main characteristics of the two datasets.The proposed action recognition approach was evaluated using two different action datasets. Due to the higher number of subjects and actions, we focused our analysis on the uulm multiperspective action datasetThe At the time we carried out the simulations, silhouette representations were available for all sequences of 22 subjects. Since the silhouettes are used to calculate an estimate of the horizontal and vertical extent of a pose, only the frontal recordings of this subset of subjects were used within the evaluation. Some action pairs  in the dataset are deliberately intended to appear visually similar and thus be difficult to separate. In total, the sequences used for evaluation contain 381, 194 frames, of which 28, 902 are selected by the key pose selection procedure.Weizmann datasetHz from a frontoparallel perspective in front of a uniform background.To allow a comparison with different action recognition approaches, simulations were additionally carried out using a well established action dataset. The Silhouettes are available for all subjects and sequences. In total, the sequences contain 5, 594 frames, 1, 873 of which are identified as key pose frames by using the procedure described in Section 3.1.Several simulations were carried out to evaluate the performance of the proposed key pose based action recognition approach. The simulations were intended to address questions related to (I) the overall performance of the approach on different datasets using a framewise, as well as windowed and full sequence majority voting recognition schemes, (II) a comparison to other action recognition methods, (III) a juxtaposition of key pose based and full sequence learning, and (IV) cross-dataset evaluation. Since action recognition datasets—in particular, in case of framewise recognition—are often highly imbalanced, we provide different types of performance measures, as well as classwise performance values for the most essential results. Since the nomenclature and definition of performance measures vary largely in the pattern recognition and machine learning community we will briefly define and describe the reported measures to allow a better comparability. For a comprehensive discussion on performance measures, we refer to Sokolova and Lapalme  and the N classes tpi (true positives) are commonly defined as the number of correct acceptances (hits) for a class Ci , fni as the number of false rejections (misses), tni as the number of correct rejections of samples of different classes Cj≠i and fni  as the number of false acceptances . Together, these four counts constitute the confusion matrix and allow to derive a variety of measures describing the performance of a trained classification system. The ones used for the evaluation of the presented results are listed alongside with an abbreviation and their definition in Table In a multiclass classification problem with  Ci , M, often referred to as (average) recognition rate or somewhat misleading as (classification) accuracy, might be the performance measurement most frequently used in the action recognition literature and describes the average percentage of correctly identified positive examples per class. InfM reflects how informed the decision of a classifier is in comparison to chance, whereas MarkM follows the inverse concept by describing how likely the prediction variable is marked by the true variable , since using micro averaging, classes with a large number of examples would dominate the averaging. Rec Powers, . Note, tLeave-one-subject-out cross-validation (LOSO) was performed in all test scenarios and the resulting average performance measures are reported together with the corresponding standard deviations. In the following, rates are either reported in a range of  or  (due to limited space).8). Figure M of 0.887, std 0.057). Some of the confusions between classes might be explained by similar visual appearances of the key poses . Accumulating the classified key poses over a sequence by majority voting increases the classification performance  is shown. Table M of 67.56 (std 6.06) indicates, that the learned key pose representations are still rich enough to classify a majority of the frames correctly. Note, that the relative number of correct classifications clearly exceeds the percentage of key pose frames in the dataset  was used to train the network on the two datasets described in Section 4. In case of the uulmMAD dataset, 28, 902 key pose frames  were selected and used as the training input. 576 cells in the last convolutional layer (layer 15) of the CNN were assigned to each of the 14 classes in the dataset. The network was trained in 150, 000 iterations. Testing was performed using the preselected key pose frames of the test subject as input. The average population activation of the cells assigned to each class was used to infer the final classification decision , the lack of a sufficient number of descriptive key poses, or a combination hereof. A direct relationship of the classwise performance and the per class number of key pose frames available for training cannot be observed. Even though the least number of key pose frames was extracted for the class bend, the second best recall value was achieved. As for the uulmMAD dataset, performance measures are reported for different single frame and temporal integration schemes in Table M = 77.15, std 6.46). Table skip is excluded) and the evaluation strategy is listed together with classification performance and speed. Evaluation strategies are either leave-one-subject-out (LOSO), leave-one-action-out (LOAO) or leave-one-out  cross-validation.The model was additionally trained using the Weizmann dataset . If the task at hand is to separate an a priori defined set of actions, this seems to be the superior way of defining key poses for the establishment of temporally sparse representations of actions. On the other hand such poses always describe differences based on comparisons and do not necessarily capture characteristic poses of an action.On a sequence level, the classification performance of the proposed approach is on par with almost all other key pose based methods. Only Liu et al.  achievedfeature-driven, pose centered key pose selection mechanism is capable of achieving the same level of performance, without loosing generality.The presented approach follows a different principle. Certain properties of image or skeleton based pose features are assumed to co-occur with characteristic body configurations and thus are used to identify key pose frames. The feature characteristic indicating a key pose and the representations/descriptors used for the recognition of a pose do not necessarily have a close relationship. In doing so, we accept the fact that the selected poses are not guaranteed to be very distinctive and some even may occur in more than one action in exactly the same way. Key poses are assumed to be the most representative poses of a particular action, not in comparison, but in general. Nevertheless, the presented results demonstrate that a M of 82.15 for key pose evaluation and 77.15 for the classification of all single frames, compare Table M = 77.15 when evaluated over all frames, although in case of the Weizmann dataset only a per class average of 33.84% (std 8.63%) of all frames is used for training.Most key pose based approaches in the literature try to assign single frames of an image sequence to key pose frames with a high similarity, temporally integrate the result  and perform a classification of the action on a sequence level. The result of single frame action recognition based on the extracted key poses (directly linking key poses to actions) is rarely reported. Single frame based approaches .In the third block of Table To address the question whether and how the proposed classification architecture might benefit from using all frames (as opposed to only key pose frames) during training, we performed exactly the same training and testing procedure twice on the uulmMAD dataset. First, only key pose frames were presented during training, while second, all frames were provided during the training phase. Likewise, testing was performed just on the preselected key pose frames, as well as the full set of frames. Table In both cases, training and testing on key pose frames achieves the highest performance. However, the observed differences between the two training conditions could not shown to be significant, neither when testing on key poses nor on the full set of frames. Nevertheless, having a closer look at the activation patterns of the network reveals some insights on the effectiveness of the two variants of trained representations. Figure one-handed wave (ED1 and wave1), which is available in both datasets. The second class was formed by combining the visually similar classes SP2/SP6 and jack/wave2 during evaluation into one joint class raising two hands. Training was performed on the full set of classes in both cases. Thus, for one-handed wave a random guess classifier would achieve a recall of either 7.14 (uulmMAD) or 10.00 (Weizmann). In case of the combined class raising two hands, the recall chance level increases to 14.29 (uulmMAD) and 20.00 (Weizmann), respectively. Table one-handed wave for the two testing (row) and training (column) setups alongside with exemplary pictures of the classes from both datasets. When training was performed on the Weizmann dataset, the recall performance for examples from the uulmMAD dataset is still considerable (loss of 24.07). Training on the uulmMAD and testing on the Weizmann dataset results in an increased performance loss, but still achieves a recall of 53.03.Learning to classify input samples and the associated representations is conducted with the aim to robustly predict future outputs and, thus, generalize for new input data. Here, we evaluate such network capability by evaluating the classification of the trained network using input data across different datasets. More precisely, cross-dataset evaluation was performed to evaluate how the learned representations generalize over different datasets. The preselected key pose frames of the uulmMAD and the Weizmann dataset were used for both training and testing constellations. Performance is reported for two classes, one being raising two hands, the performance loss is below 30 for both training and testing configurations. Table jumping jack is recognized almost without any loss of performance. Vice versa, SP2 is often confused with wave2 when training was performed on the Weizmann dataset. This may be explained by the large visual similarities between the classes.In case of the combined class feature-driven selection of the key pose frames.The proposed approach shows promising generalization capabilities, which might partially be explained by the class-independent, feature-driven key pose selection mechanism is proposed, which builds upon evidences about human action perception. The selection mechanism does not utilize any information about the inter- or intra-class distribution of the key poses (or key pose descriptors) to optimize the classification accuracy. It is demonstrated, that the classification accuracy is on par with state-of-the-art key pose based action recognition approaches, while only motion and form related feature characteristics are used to select a key pose frame. Second, we propose a biologically inspired architecture combining form and motion information to learn hierarchical representations of key pose frames. We expect such hierarchical feature representations to make the recognition more robust against clutter and partial occlusions, in comparison to holistic shape representations of the full body configurations used in previous approaches. Form and motion pattern representations are established employing a neuromorphic deep convolutional neural network. The trained network is mapped onto the IBM Neurosynaptic System platform, which enables a computationally and energy efficient execution.The presented work consists of two main contributions. First, a The presented results demonstrate, that classifying actions using a minimal amount of temporal information is in principle possible. This is in line with results from other action recognition approaches. For example, Schindler and van Gool  reportedinter-class distinctiveness or classification performance of the selected key poses. The feature-driven selection criterion proposed in this work combines form and motion information and allows the identification of key poses without any knowledge about other classes. It extends a previous proposal utilizing local temporal extrema in the motion energy as a function of time  and that the associated temporal events contain a high proportion of the information about the main components of the action sequence.The investigation reported in this work adds an important piece to the debate of how representations for action sequence analysis might be organized. Some previous approaches have utilized motion and form information for the classification of action categories. For example, Giese and Poggio  proposedenergy-efficient deep neuromorphic networks (Eedn) framework  in an action sequence. Thus, the reversed, or scrambled presentation of images  of a sequence would result in an assignment to an action class, although, the visual appearance of the sequence is totally different. A modeling or learning of the temporal relationships between the key pose frames, e.g., their temporal order, would help in reducing ambiguities and thus increase sequence-wide or windowed classification rates. In case of the proposed approach, this could be achieved by employing, e.g., We also did not apply a weighted majority voting scheme using the confidences of the frame-wise predictions or apply thresholding on the predictions. Both strategies might further increase the classification performance but again would weaken the focus on the analysis of key pose base representations of action sequences.The proposed architecture of a deep convolutional neural network (DCNN) as depicted in Figure IBM Neurosynaptic System . HN has been supported by the collaborative project “SenseEmotion” funded by the German Federal Ministry of Education and Research (BMBF).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


This technique is demonstrated for the Bridgman-type crystal growth enabling remote and direct measurements of growth parameters crucial for process optimization. The location and shape of the interface between liquid and solid phases are monitored in real-time, concurrently with the measurement of elemental distribution within the growth volume and with the identification of structural features with a ~100 μm spatial resolution. Such diagnostics can substantially reduce the development time between exploratory small scale growth of new materials and their subsequent commercial production. This technique is widely applicable and is not limited to crystal growth processes.Energy-resolved neutron imaging is investigated as a real-time diagnostic tool for visualization and  In situ diagnostics combined with computer simulation could substantially decrease the process development time and provide guidance for real-time feedback control and optimization of crystal growth. However, such diagnostics are difficult to implement. For example, the Bridgman crystal growth process145Single crystals are core elements in electronic, optical, microwave, and numerous other devices in domains as diverse as medical imaging, security screening, high energy physics, space exploration and many others. Key to any viable development of these applications are yield, reproducibility and final cost of the material production. The optimization of the latter for large size industrial scale has always been challenging. A new crystal material and its growth technique almost always require specific time consuming optimization as limited knowledge can be inferred from previous developments. in situ characterization of the crystal growth process. It also provides support for the validation of simulations. Neutrons can penetrate the thermal insulation and structural materials used in most pullers, i.e. Bridgman furnaces, and probe the growth volume through neutron absorption and scattering. The attenuation cross section for neutrons in general is not proportional to the atomic number of a particular element, as in the case of X-rays, because neutrons interact, except for magnetic scattering, with the nucleus of the atom and not with its electrons. Therefore, many materials opaque to conventional X-ray techniques can still be interrogated with neutrons. This approach enables multimodality provided by the different nature of neutron interactions that can occur within the material: the elemental composition within the growth volume can be studied through the neutron resonance absorption789101113141618Real-time neutron imaging technique appears to be a unique tool for energy-resolved neutron imaging implemented at a pulsed neutron source to provide quantitative information on growth conditions and some crystal properties in real time  and in situ. The measurements reported in this paper were performed during two solidification runs of the scintillator BaBrCl:Eu2+ (0.5 and 5 mole % Eu) in the Vertical Gradient Freeze (sub-type of the Bridgman) configuration.In this study, we demonstrate the ability of Measurement of neutron transmission spectra in a wide range of energies, including epithermal, thermal, and cold ranges, allows for a quantitative analysis of the elemental composition within the sample through the resonance absorption typically appearing at the epithermal neutron energies. At certain energies specific to a particular element  neutron attenuation increases sharply, leading to sharp dips observed in the spectrum measured in the area where such element is present. These are the resonance features in the transmission spectrum  to convex as the growth proceeded.The shape and location of the liquid/solid interface are among the important diagnostic parameters, which can be used for process optimizationin situ and in real time. A higher transmission of the boundary layer is observed as the growth rate is increased  and the energy of this resonance should be less than ~103 eV. These two parameters and the thickness of the sample determine the limits of elemental concentration which can be mapped by energy-resolved neutron imaging. Consequently, the possibility of elemental mapping for a particular element in a particular material can be accurately estimated from the tabulated cross sections10The presence of sharp variation of neutron attenuation at energies specific to a particular element  can be used for the spatially-resolved analysis of elemental composition of the sample, as demonstrated in this section. The sensitivity of this technique obviously depends on the resonance attenuation cross sections, which for many elements exist in the epithermal range of neutron energies (~eV to ~keV neutrons), which matches the accessible range of energies for the present energy-resolved neutron imaging facilities. However, there are also elements which exhibit resonances only at higher energies (~MeV) which cannot be resolved with good spatial resolution, which makes their quantification nearly impossible for the methods described in this section. Therefore there are two parameters crucial for the applicability of this method: the value of resonance cross section should be sufficiently large , and variation of elemental composition, can also be visualized in three dimensions through tomographic reconstruction, which typically requires measuring a large number of transmission images while the sample is rotated along the axis perpendicular to the incoming neutron beam. In principle, that measurement can be performed during in situ diagnostics technique demonstrated in this paper provide a new tool for an efficient guidance for the development of crystal growth processes for new materials. Obviously, the limited number of experimental facilities make the technique described in this paper unavailable for industrial scale crystal production. However, once computer simulations are fully validated by such experiment, scale-up of a particular crystal growth process, especially for new materials, can be performed from data acquired at the beamline. The applications of this technique are not limited to crystal growth of scintillator materials; it can easily be used toward development and optimization of many other material processes.The The results of the presented study have been made possible by the use of i) the recently developed unique 2D fast neutron counting detector2527282931in situ diagnostics of crystal growth in a Bridgman furnace were conducted at a pulsed neutron beamline FP52 active area252 pixel within an energy range of ~0.005–100 eV. Two charges of BaBrCl:Eu with 0.5 mole % and 5 mole % Eu doping were sealed under vacuum before the experiment into ~12.3 mm ∅ silica glass ampules. This materialThe proof-of-principle experiments on in situ diagnostics, rather than growing best quality crystals. In particular, the initial polycrystalline charge was not melted to the bottom of the ampoule. This did not allow proper nucleation of the crystals and growth of single crystals, but was necessary to test the in situ imaging starting on a defined stabilized interface. All images presented in this paper were normalized by the images taken with no sample present in the beam to eliminate the non-uniformities of the neutron beam, detector response and the image features produced by the attenuation by the furnace. An accurate selection of materials with low neutron attenuation cross section was performed before the furnace was built in order to minimize neutron attenuation in the furnace and crucible. Most heat insulating ceramics and quartz crucibles have very low neutron attenuation and the most opaque elements of our experimental setup were thin wires of the heating elements. All the transmission spectra were also corrected pixel by pixel for the spectrum of the incoming neutron beam, background contribution, and for the transmission of the furnace and the crucible.The temperature gradient within the furnace was calibrated prior to the measurements . The BaBHow to cite this article: Tremsin, A. S. et al. Real-time Crystal Growth Visualization and Quantification by Energy-Resolved Neutron Imaging. Sci. Rep.7, 46275; doi: 10.1038/srep46275 (2017).Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


Nanomaterials have been actively employed in various applications for energy and sustainability, such as biosensing, gas sensing, solar thermal energy conversion, passive radiative cooling, etc. Understanding thermal transports inside such nanomaterials is crucial for optimizing their performance for different applications. In order to probe the thermal transport inside nanomaterials or nanostructures, tip-based nanoscale thermometry has often been employed. It has been well known that phonon transport in nanometer scale is fundamentally different from that occurred in macroscale. Therefore, Fourier’s law that relies on the diffusion approximation is not ideally suitable for describing the phonon transport occurred in nanostructures and/or through nanoscale contact. In the present study, the gray Boltzmann transport equation (BTE) is numerically solved using finite volume method. Based on the gray BTE, phonon transport through the constriction formed by a probe itself as well as the nanoscale contact between the probe tip and the specimen is investigated. The interaction of a probe and a specimen  is explored qualitatively by analyzing the temperature variation in the tip-substrate configuration. Besides, each contribution of a probe tip, tip-substrate interface, and a substrate to the thermal resistance are analyzed for wide ranges of the constriction ratio of the probe. Recently, we are witnessing extreme miniaturization of thermal devices and other components ,2,3. BesDevelopment and continuous evolution of a tip-based thermal microscopy are revolutionizing our perception about thermal transport at sub-micron scale. In a thermal microscope system, a nanometer-sized probe is often employed to interact with a specimen for measuring local temperature. Such tip-substrate configuration also has enormous practical applications, such as in data writing , chemicaIt is known that the thermal conductivity of nanostructures is significantly lower than its bulk counterpart because of the phonon boundary scattering. Recent research also suggested that it is possible to alter the transport regime and thus the transport property by tailoring the geometry of nanostructures ,13. BesiAnother area of our interest is in the quantification of thermal resistance for a tip-based thermal analysis system in which a probe with a nanoscale tip is in contact with a substrate. For macroscale thermal transport, the thermal contact resistance across the interface has been subjects of multiple studies ,16. As fIn light of the preceding discussion, the objective of present work is twofold: (i) fundamental understanding of phonon transport between a probe with nanoscale tip and a specimen  and (ii) quantification of thermal resistance of such geometry. These will be achieved by solving the phonon Boltzmann transport equation. In a recent study , the autBeing aware of the complexity of the configuration under consideration, following assumptions are made. First, we assumed that perfect contact exists between the probe tip and the substrate; that is, there is no boundary scattering and/or resampling at the tip-substrate interface. Secondly, it is assumed that experiments occur in the vacuum; thus, convection loss through the air is absent.The schematic of geometric configuration considered in the present study is shown in At steady state, the gray Boltzmann transport equation (BTE) can be written for an axisymmetric system with the relaxation time approximation as :(1)μr∂ model of the BTE may differently predict when compared with the axisymmetric model. Here, thermal transport in the tip-substrate system has been analyzed using both the axisymmetric model and the 2-D model. The Gray BTE code utilized for the 2-D model is same as used previously by the authors . Figure In summary, we have investigated the phonon transport through the constriction formed by a probe itself as well as the nanoscale contact between the probe tip and the specimen based on the gray Boltzmann transport equation. We observed that constriction of the probe tip could play a dominating role in governing the phonon transport especially when the degree of constriction is high. It was also found that the combined thermal resistance of the probe tip and the tip-substrate interface predicted by the BTE is much greater than the total resistance predicted by Fourier’s law and this difference increases as the degree of constriction increases. Lastly, the tip-substrate interface is found to offer significantly higher resistance to phonons in the axisymmetric model as compared to the 2-D model. The insights obtained from the present study may be applied to improve our understanding of a tip-based device.