Infrared and visible image fusion (IVIF) aims to provide informative images by combining complementary information from different sensors. Existing IVIF methods based on deep learning focus on strengthening the network with increasing depth but often ignore the importance of transmission characteristics, resulting in the degradation of important information. In addition, while many methods use various loss functions or fusion rules to retain complementary features of both modes, the fusion results often retain redundant or even invalid information.In order to accurately extract the effective information from both infrared images and visible light images without omission or redundancy, and to better serve downstream tasks such as target detection with the fused image, we propose a multi-level structure search attention fusion network based on semantic information guidance, which realizes the fusion of infrared and visible images in an end-to-end way. Our network has two main contributions: the use of neural architecture search (NAS) and the newly designed multilevel adaptive attention module (MAAB). These methods enable our network to retain the typical characteristics of the two modes while removing useless information for the detection task in the fusion results. In addition, our loss function and joint training method can establish a reliable relationship between the fusion network and subsequent detection tasks. Extensive experiments on the new dataset (M3FD) show that our fusion method has achieved advanced performance in both subjective and objective evaluations, and the mAP in the object detection task is improved by 0.5% compared to the second-best method (FusionGAN). A single sensor has its limitations, and it is challenging to create a thorough, credible, and accurate description of a multitude of scenarios involving people, vehicles, roads, traffic lights, and so on. This has emerged as the biggest obstacle to the ability of intelligent systems to carry out various complex tasks. The core capabilities of the entire intelligent system, such as information gathering and intelligent cognition, have advanced in recent years thanks to the rapid development of multi-mode sensors . Among tInfrared and visible images have very different imaging principles and feature representations. While visible images can more effectively present the scene texture details and retain the illumination intensity information, infrared images aim to highlight the overall contour characteristics of the object. However, due to hardware and environment factors, blur and halos may appear in infrared images. Therefore, it is crucial to understand how to fully exploit their benefits and combine infrared and visible images.Additionally, the demands for multimodal fusion and subsequent downstream tasks, such as object detection and semantic segmentation, are growing rapidly with the vigorous development of video surveillance and automated driving; the majority of currently used approaches focus on creating networks or models to improve the visual impact of the fused image, but they ignore the fact that the fused image only matches the human vision, making it hard to meet the perceptual requirements of subsequent tasks, such as object detection and segmentation.The approach suggested in this paper aims to address the existing practical issues. In order to effectively address the underlying visual issues in challenging environments, this method employs two techniques: multi-layer semantic information guidance and a neural network architecture search. These techniques work together to enhance the visual quality and effects of fused images as well as the performance of downstream tasks.In order to reduce feature redundancy and preserve complementary information, we designed a multi-level adaptive attention block (MAAB) in the network, which allows our network learning to retain rich features at different scales, and more efficiently and effectively integrate high-level semantic information.To discard the limitations of the existing manually constructed neural network structure, we introduce a neural architecture search (NAS) in the construction of the overall network structure, so as to adaptively search the network structure that is suitable for the current fusion task.Aiming to fully utilize semantic information and conduct target detection-oriented fusion research, this paper employs a neural network search scheme and attention mechanism to process various downstream tasks. The main research work and contribution of this paper can be divided into two main parts:This paper will be divided into the following sections.In In In this section, we review the current state of research on visible and infrared image fusion methods, specifically divided into two parts: traditional fusion methods and deep learning fusion methods. Moreover, we introduce the work on a neural architecture search.IVIF algorithms based on traditional approaches are divided into five categories: the fusion method based on multi-scale transform, sparse representation, subspace, saliency, and other traditional theories.Due to their unique information extraction tactics, multi-scale transform approaches have had a lot of success in the image fusion field. The multiscale transform acquires the multiscale representation of the input image, then constructs the fused multiscale coefficient using fusion rules. Finally, to acquire the fusion result, the coefficient is inversely converted. Although multi-scale transformation is a well-known and effective technique, it has issues with halo and ghosting when reconstructing images.Several researchers are altering the multi-scale transformation method to address these issues. To accomplish greater noise reduction and more efficiently extract rich information, Nencini et al.  used theSparse representation methods utilize an over-complete dictionary, which is trained on a large set of high-quality natural images, to sparsely represent the input images. This is in contrast to multi-scale-based fusion methods. This compact formulation is more practical for infrared and visible image fusion processing since it only requires a small number of atoms from the dictionary to describe the image’s information.Creating an appropriate over-complete lexicon is a highly researched area. Liang et al.  providedA subspace is a region of space whose dimensions are less than or equal to the total dimensions of the space. High-dimensional input image dimensions are reduced by the subspace-based fusion technique so that they can be projected onto a low-dimensional subspace. In order to fuse infrared and visible light, it is necessary to remove some redundant information from the source images. By utilizing the dimensionality reduction representation approach, it is possible to capture the valuable internal structure of the original image while minimizing the execution time and memory costs.Li et al.  used priSaliency detection is an intelligent algorithm that simulates the perceptual features of the human visual system and extracts the key elements of human perception from a picture. This bottom-up attention process is generated by the pixel and its neighbors, and researchers aim to preserve the target’s position in the salient zone.Saliency can inevitably be used to extract a considerable target region from a source image. For instance, Liu et al.  used a sFurthermore, a fusion method uses importance to calculate the weight. In this process, the source image is first multi-scale transformed into the overall target layer and the detail information layer. Afterward, the saliency map is extracted using the saliency model, and transformed into the weight map to create the fused two layers; finally, the fused image is recreated using multi-scale inverse transformation.Gan et al. , for exaIn addition to the fusion processes mentioned above, many other fusion methods exist that offer novel concepts for infrared and visible light fusion. Researchers have presented fuzzy theory and practice as a means of addressing the challenge of image blurring caused by the fusion of high-quality source images. For example, Rajkumar et al.  proposedThe amount of information conveyed by entropy is an indicator of its richness. It can be used to evaluate the extent to which data from the source image is transmitted to the fused image. Zhao et al.  developeThe primary concept of morphology is to extract the form or feature in an input image for subsequent operation and processing using unique structural components. Bai et al.  suggesteThe development of deep learning has led to outstanding achievements being made in many fields of computer vision ,26,27,28This kind of method transforms the image feature extraction operation from the manually designed mathematical operations to the pre-trained deep neural network. It uses the multi-channel feature map generated by the neural network to achieve rich information representation, and reconstructs the feature map through the fusion strategy to generate results. The key to this kind of method lies in the design of the neural networks and fusion rules.In the image classification task, there are very mature, large-scale visible light databases with multiple scenes and targets, and they have been used to train neural networks for classification. Common ones include the linear multilevel VGG convolutional neural network , ResNet However, such methods have obvious limitations. First, the pre-trained network parameters limit the infrared and visible light fusion since these networks are trained for image classification. Secondly, the deep neural network suffers from the degradation phenomenon. When using its deep-seated features for fusion, it will inevitably be disturbed by the lost and redundant/useless features. How to reduce this impact in applications also needs special consideration ,35,36.In the network structure based on the autoencoder, the main parts are the encoder and the decoder. The overall process is that the source image pair is input into the encoder, transformed into a feature map representation, and is fused at the feature level through the set fusion strategy; the fused feature map is then reconstructed by the decoder to generate the fusion result map.The training strategy for the self-coding network is flexible. Encoders and decoders with different network structures are designed and trained on large-scale datasets to obtain network parameters with good generalization performance. After the training, in the test phase, the learned parameters of the network are fixed, and the fusion strategy is added between the encoder and the decoder to perform the fusion operation.Compared with the fusion algorithm of the pre-training network, the encoder and decoder have good scalability. Li et al.  took theThis method also has some disadvantages. First, the manually set fusion strategy cannot reduce the difficulty and time cost of the algorithm in the operation. Second, this method fails to address the challenge posed by the lack of multimodal images in the training data, which ultimately limits the network’s ability to adapt to a wider range of parameters.The end-to-end model attempts to integrate the multi-step model into a single model to directly realize feature extraction, fusion, and image reconstruction in the algorithm. Li et al.  improvedAlthough the end-to-end model is widely used, it also has some issues. Compared to the step-by-step multi-module model, the specific process in the end-to-end model has poor interpretability, making it difficult to determine the contribution of each part to the fusion result.The generation adversarial network (GAN)  is compoMa et al.  introducJust as the encoders in the self-encoder structure can design their own encoders for different modes of source images to better extract features, Ma et al.  split thLiu et al.  designedOn the premise of good training, this type of method can fit the distribution of training samples well and generate more realistic fusion images. Additionally, its loss function design is relatively simple. However, its disadvantage is that the training process can be unstable, and the training data samples at this stage may be limited and not diverse enough.Lee et al.  introducAs mentioned in the introduction, the goal of IVIF is to retain complementary information and remove redundant information from the two modal images. At present, most fusion methods achieve high-quality fusion results by increasing the network depth or manually designing various loss functions. We anticipate that the foreground object in the fused image will be nearer to the infrared image, consistent with the top-down attention mechanism of human vision, which directs our focus toward the highlighted regions of the infrared image. In addition, saliency representation has achieved great success in the field of computer vision. It extracts salient features in images by similar means. For infrared and visible image pairs, the internal specific contrast can distinguish between the foreground target and background details using significance representation.Moreover, it is well known that high-level semantic information can effectively guide subsequent downstream tasks. When performing infrared-visible image fusion and subsequent target detection simultaneously, incorporating such rich semantic information is expected to enhance feature representation and improve target detection results.Finally, considering that manually designed network structures may have difficulty adapting to the introduction of high-level semantic information, we propose using neural architecture search with a high upper limit and good effectiveness to determine sub-operations. Thus, we propose a multi-level attention-guided search fusion network based on semantic information. By incorporating saliency information constraints into the loss function, designing multi-level attention modules, and introducing NAS, we expect that this method can generate visual fusion results with clear thermal targets and rich real details.The network structure is composed of three parts: a high-level semantic network, a multi-layer adaptive attention block, and a searched residual network. The overall structure of the whole network is shown in In this paper, we use the pre-trained VGG16 network as the basic framework of the high-level semantic network As shown in To adapt the image features of the two modes to their semantic information at different scales, this paper proposes a multi-layer adaptive attention block (MAAB) ,48.i denotes the number of downsampling operations applied. As the feature map size continuously decreases, we reduce it to the order of As shown in i times, the module can fully contain local and global feature information. Jump links are added at the corresponding convolution layer to enrich the amount of information and make up for some losses. Then these intermediate features, containing spatially significant information, are sent to the channel attention module introduced below to obtain richer and more effective deep-seated semantic information  before being fed to the network.In the search process of the neural architecture search, we used Adam  as the oIn object detection, we used 4200 pairs of images labeled in M3FD for network training. We used YOLOv5 as the object detection network and we used the pre-trained YOLOv5s model. In object detection training, the image size was set to 320 × 320, and other parameters were carried out according to the original data provided by the official.Our approach was implemented on PyTorch with an NVIDIA Tesla V100 GPU. The tuning parameters First, as shown in We evaluated the fusion performance of our method by comparing it with seven state-of-the-art methods: DenseFuse , FusionG(1)Qualitative Comparisons on TNO DatasetsThe intuitive qualitative results on two typical image pairs from the TNO dataset are shown in The first pair of famous images depicts a residential area under night surveillance. The infrared image clearly shows useful information that is difficult to distinguish in visible images, such as human figures, clouds, shrubs, and wire circles. As shown in the red boxes, our method effectively preserves the contrasted information of the image, and the fused image indicates the shrubs as background with an even distribution of pixels, while other methods, such as FusionGAN and GANMcC, generate blurred backgrounds. As shown in the green boxes, our approach outlines target objects that are more apparent, while other methods, such as RFN-Nest, are greatly affected by the visible images, and the outline of the target object is blurred and difficult to identify.The second pair of images depicts a daytime surveillance scene. The human figure in the visible image blends into the background, while the target object is evident in the infrared image. The outline of the street light pole in the infrared image is difficult to identify, and the outline is more apparent in the visible image. As shown in the red box, our method sharpened the contours of the target object compared to the visible image and effectively absorbed the valuable information from the infrared light image, while other methods, such as RFN-Nest and U2Fusion, generated blurred low-brightness target objects. As shown in the green box, our method effectively preserves the gradient information and clearly shows the outline of the street light pole as a detailed background, while other methods, such as FusionGAN, generate fused images with uneven distribution of pixels in the background, making it difficult to distinguish the outlined details.(2)Qualitative Comparisons on Roadscene DatasetsThe intuitive qualitative results of two typical image pairs from the Roadscene dataset are shown in Two classic pairs of images depict street scenes at night. In contrast to the previous pairs, both pairs include artificial light sources, setting a unique barrier to image fusion.In the first group of images, the target object in the visible image is not identifiable, while the background details in the infrared image have a chaotic pixel distribution. In the second group of images, the vehicle in the visible image blends in with the blackness, while the overexposure caused by the artificial light source blurs the background details in the infrared images. Our method shows stronger contrast and richer color details compared to FusionGAN and GANMcC, as shown in the green box in the first set and the red box in the second set. It also avoids overexposure that causes ghosting compared to MFEIF. Moreover, as shown in the red box in the first group and the green box in the second group, our method effectively enhances the contours of the target object.(3)Qualitative Comparisons of M3FD DatasetsThe intuitive qualitative results of two typical image pairs from the M3FD dataset are shown in These two sets of images evaluate the effectiveness of different fusion approaches under typical and unique barriers.The first set of images shows a night street scene. The visible image has overexposure due to artificial light sources, making it difficult to recognize the human silhouette. The infrared image cannot capture the artificial light sources and loses text information on the signboard. As shown in the green box, our method generates the fused image with the brightest and most visible target object compared to other methods. As shown in the red box, our method produces an image with the highest contrast, good distribution of background details, and precise text information on the signboard.The second set of images depicts a mountain scene during the daytime. The presence of smoke in the visible image makes it impossible to identify the target object. Still, the infrared image can capture the target object information through the smoke. As shown by the green box, our method fully absorbs the knowledge of the target object in the infrared image, with clear contour edge information and good brightness. As shown by the red box, the fused image generated by our method has high contrast, good pixel distribution, and rich color details compared to other methods in representing background details.In order to make a more comprehensive quantitative comparison, we selected six common fusion indicators in the field of image fusion as the evaluation benchmark, including three referenced indexes: mutual information (MI) , structuThe quantity of information transmitted from the source image to the fused image is measured using MI. A larger MI indicates that more information from the source image pair is maintained in the fused image.The human visual system is sensitive to picture loss and distortion, and this is modeled using SSIM. It has a good correlation with fusion performance.SCD displays the correlation between the source and fused images. A larger SCD indicates a higher fusion performance.EN measures the information in the fused image. A higher EN typically denotes improved fusion performance.The contrast and pixel distribution of the fused image are reflected by SD. A larger SD typically denotes a more aesthetically pleasing fused image.SF represents the overall gradient distribution of the image in the spatial domain. The texture and edges become richer as the SF becomes larger.The specific results are shown in In the TNO dataset, as shown in In the Roadscene dataset, as shown in In the ensuing target detection section, we conducted the following actions to ensure that the experiment was fair: 300 epochs were retrained based on the pre-trained YOLOv5s model for each fusion method, and their corresponding detection network parameters were obtained before detection. That is, a pre-trained YOLOv5s model was used and fine-tuned on the fused images.In the dataset used in this paper, only the M3FD dataset has object annotation, so relevant experiments were carried out on this dataset. Different from our previous work, this time we enabled a new training set and test set. The 4200 images in the dataset are divided into a test set and a training set according to the 800/3400 images and scenes. Each scene in the test set has not appeared in the training set.The quantitative results of target detection in the M3FD dataset are shown in We study our method’s model architecture and further validate the effectiveness of several individual components as shown in The initial single-layer search convolution layer is replaced with ConV 3 × 3 (stride = 1) operation when NAS is not used. When MMAB is not used, a combination of ConV 3 × 3 (stride = 1) + BN + ReLU is used instead. Without MAAB, the utilization of high-level semantic information has some adaption issues, and the overall contrast and color are poor. Without NAS, it is impossible to make greater use of semantic information with the original features, resulting in a significant loss of detail. These two components have favorable effects on the ultimate fusion result.The relevance of MAAB and NAS in the network can also be seen in the quantitative comparison presented in the table. Our overall model’s final findings came in second place. The M1 model creates too much noisy pixel distributions, which interferes with the findings, thus ’en’ is not the greatest value.We discuss the impact of different loss functions on our method. The resulting image is substantially deteriorated without SSIM limitations, as seen in Overall, the loss functions we investigated do their jobs and have a significant impact on the training process.Overall, the paper presents a valuable contribution to the field of IVIF and object detection by proposing a multi-level structure search attention fusion network based on semantic information guidance. Future research could expand on these findings to enhance the accuracy and robustness of object detection algorithms in fused images under challenging conditions.In this paper, a multi-level structure-search attention fusion network based on semantic information guidance is proposed. The multi-level adaptive attention block (MAAB) is designed in the network to reduce feature redundancy and preserve complementary information. The neural architecture search (NAS) is introduced in constructing the overall network structure to eliminate the limitations of the existing manually constructed neural network structure. Extensive experiments on the new dataset (M3FD) show that our fusion method has achieved advanced performance in both subjective and objective evaluations, and the mAP in the target detection task is improved by 0.5%.


Variations with respect to perspective, lighting, weather, and interference from dynamic objects may all have an impact on the accuracy of the entire system during autonomous positioning and during the navigation of mobile visual simultaneous localization and mapping (SLAM) robots. As it is an essential element of visual SLAM systems, loop closure detection plays a vital role in eradicating front-end-induced accumulated errors and guaranteeing the map’s general consistency. Presently, deep-learning-based loop closure detection techniques place more emphasis on enhancing the robustness of image descriptors while neglecting similarity calculations or the connections within the internal regions of the image. In response to this issue, this article proposes a loop closure detection method based on similarity differences between image blocks. Firstly, image descriptors are extracted using a lightweight convolutional neural network (CNN) model with effective loop closure detection. Subsequently, the image pairs with the greatest degree of similarity are evenly divided into blocks, and the level of similarity among the blocks is used to recalculate the degree of the overall similarity of the image pairs. The block similarity calculation module can effectively reduce the similarity of incorrect loop closure image pairs, which makes it easier to identify the correct loopback. Finally, the approach proposed in this article is compared with loop closure detection methods based on four distinct CNN models with a recall rate of 100% accuracy; said approach performs significantly superiorly. The application of the block similarity calculation module proposed in this article to the aforementioned four CNN models can increase the recall rate’s accuracy to 100%; this proves that the proposed method can successfully improve the loop closure detection effect, and the similarity calculation module in the algorithm has a certain degree of universality. Mobile robots are capable of determining their own motion trajectories in uncharted territory utilizing simultaneous localization and mapping (SLAM) ,2, whichThe current loop closure detection algorithm uses appearance information to address the issue of data association between images, and it is primarily based on the similarity matching method of the image and its data. The robot’s trajectory can vary in practical applications due to the interference of dynamic objects, which causes visual bias in the collected images , furtherCompared with the artificial features used in the traditional bag-of-words model, deep learning can extract more abundant image information  and is mTherefore, we redesigned the loop closure detection algorithm in SLAM by combining MobileNet_v3 and block similarity calculation. The main work of this paper is as follows. (1) In comparing the precision–recall curves of existing excellent CNN models, the global descriptor of the image sequences is extracted using the pretrained lightweight neural network model MobileNet_v3 as the feature extractor in combination with the inverse residual structure in the network. (2) A principal component analysis (PCA) and whitening are used to improve the computational efficiency. (3) A block similarity calculation module is introduced to extract the local information of image block descriptors from the previously determined possible loop closure detection similarity pairs in fixed blocks and to re-judge the loop through a similarity calculation in order to improve the method’s loop detection accuracy. Finally, in order to verify the feasibility of the proposed method, a loop closure detection experiment is designed, and the results are analyzed. Experimental results show that the proposed method is effective and robust.The structure of this paper is as follows: In The method mainly consists of the extraction of image descriptors, the reduction of the dimensionality of image descriptors, and block similarity calculation. The overall framework of the method is shown in Firstly, the pretrained CNN model is used to extract image descriptors and gather all the descriptors extracted from the image sequence. Then, the descriptors are subjected to a principal component analysis (PCA) and whitening to reduce some relevant dimensions, thereby reducing the subsequent computational complexity and preserving the main information. Finally, the cosine similarity of these descriptors is calculated, and a similarity matrix is generated. The similarity matrix is a symmetric matrix, each row of which can be regarded as a sequence of the current query image; each column can be regarded as a loop closure candidate. The query image is only compared with the image before the current time, and the image with the highest similarity to the query image can be found in this matrix. The two images with the most similarity are found, and the overall similarity is recalculated using the block similarity calculation module. The recalculated similarity is used to determine whether loop closure has occurred.Directly calculating similarity from image data requires a significant amount of computation, and the results are frequently unreliable due to variables such as changing lighting, shifting viewpoints, and dynamic environments. An image descriptor is a vector used to represent an image, and representing the image as a vector is a necessary process for loop closure detection. Pretrained CNN models typically have good generalization performance while also reducing the time cost of retraining the network. Using a pretrained CNN to extract image features means richer image information can be used than with manually designed features. Visual Geometry Group 16 (VGG16) , AlexNetIn order to meet the size requirements of the input image of the CNN model, the image size of the dataset should be adjusted to the input picture size of MobileNet_v3. To reduce the gradient value during the training, the model converges smoothly, and the image data are naturalized; the grayscale range of the image element is mapped from 0 to 255, in proportion, to between 0 and 1. After the naturalization process, the image is consistent with the original image, and the image’s information is unchanged.At the same time, with standardized processing, the grayscale value of the image pixel point is mapped from −1 to 1, and decentralization is achieved; it is then easier for the image data to be subsequently generalized. Images are naturalized and standardized when training CNN models, so the input image is processed in the same way in order to achieve the extraction of characteristics. The calculation formula for image standardization is as follows:x representing the original image matrices, In this case, The fully connected layer 1280-dimensional output vector of MobileNet_v3 is employed as the descriptor of the input image by the algorithm in order to verify the universality of the block similarity calculation method developed in the method. Other CNN models in the experiment also used the fully connected layer as the descriptor for the input image.Due to the high number of sub-dimensions of the extracted image description, dimensionality reduction processing is needed to increase the speed of the subsequent cosine similarity calculation. Principal component analysis (PCA) can help vectors better represent images by reducing their dimensionality while also preserving the essential information in the vectors.m images in the image sequence, the descriptor sub-dimension extracted from each image is n, and these image descriptors are combined together in rows to generate a generator matrix Suppose there are The specific calculation process of PCA is as follows.(1)The mean is calculated for each column.(2)The corresponding mean is subtracted from each column of (3)The covariance matrix (4)Covariance matrix (5)The first n to k, and the main information is extracted. In order to reduce the correlation between various dimensions, whitening is usually performed after PCA dimensionality reduction, and the calculation process is as follows.After the aforementioned PCA dimensionality reduction and whitening processing, the image descriptor subcombination matrix n-dimensional vector space. As the cosine similarity increases and the vector angle decreases, the similarity between the images increases.The reduced dimensionality image descriptors can better represent the image and facilitate calculation. The cosine similarity between these descriptors can be calculated directly to produce a similarity matrix. Through calculating the cosine value between two description vectors, which represents the angular distance between the two description vectors, the cosine similarity is obtained. The calculation equation is as follows:The images are read in chronological order, and the similarity between the images closest to the current query image and the query image is relatively high, so images near the query image are not detected. Of the remaining images, the image with the highest similarity to the query image is used for subsequent block similarity calculation.i, j) represents the similarity between the i-th image and the j-th image in the dataset. Therefore, the value on the diagonal is 1. The darker the color in The similarity matrix is used to measure the similarity between the query image and the loop closure candidate image. This matrix is a symmetric matrix. The value at  represents the similarity between the i-th and j-th image blocks in the image. It can be seen from subgraph (e) and subgraph (f) that the elements on the main diagonal of the matrix similarity of the correct loop closure image pair are darker and more similar than the elements on the matrix similarity of the wrong loop closure image pair, which indicates that the overall similarity between the loop closure image pairs can be expressed by the elements of the main diagonal of the matrix similarity, to a certain extent. However, in order to avoid the false negative results caused by direct calculation, it is necessary to further use the connection between image blocks, thereby connecting the local information of the image with the overall information. Comparing subgraphs (d) with (e) and (f), it can be found that the correct loop closure image is more similar to the matrix similarity as a whole, while the error loop closure image is more different from the matrix similarity as a whole. Therefore, the similarity of image pairs is recalculated based on the similarity difference between the image blocks mentioned above, further distinguishing between correct and incorrect loops.SM1, SM2, and SM3 are all 9 i-th image block of the loop closure candidate image.To more intuitively represent the difference between the correct loop closure image pair and the incorrect loop closure image pair, the matrices SM1 and SM2 are subtracted and taken as absolute values, and the matrices SM1 and SM3 are subtracted and taken as absolute values. The calculation process is as follows, and the results are shown in In i, j); they represent the difference in similarity between the i-th and j-th image blocks in the image pair. The lighter the color, the smaller the similarity difference, while the darker the color, the greater the similarity difference. The correct loop closure image has a lighter color representing the similarity difference matrix compared with the incorrect loop closure image, indicating that the loop closure image has a smaller overall difference.SM_ d1 and SM_ d2 both are 9 Each line of SM_d1 except the elements on the main diagonal is added to obtain the similarity difference value of nine image blocks: K_1 = . Each line of SM_d2 except the elements on the main diagonal is added to obtain the similarity difference value of nine image blocks: K_2 = . Each element in K_1 and K_2 represents the overall similarity difference between the corresponding image blocks in the correct looping image pair and the incorrect looping image pair; the smaller the value, the smaller the difference. Through more intuitive data comparison, it can be found that individual values in K_2 are not significantly different from those in K_1, but overall, K_2 has a larger value than K_1. Therefore, the similarity difference value can be used to recalculate the similarity between image pairs, thereby reducing the similarity of error loops to a greater extent.i-th image block of the loop closure candidate image. The term k is an adjustment parameter used to indicate the degree to which similarity differences , subgraph (e), and subgraph (f) in Sim, and the similarity between the corresponding image blocks after the query image and the loop closure candidate image are segmented is denoted as The overall similarity between the query image and the loop closure matching image is denoted as Sim. The smaller From Equation (12), the closer To visualize the calculations, a set of images are separately extracted from the New College and City Center datasets. Take the New College dataset, for example, which includes an image to be queried, serial number 793; a correct loop closure image, serial number 580; and four images with similarities to images to be interrogated, with serial numbers 570, 574, 650, and 653, respectively. The City Center dataset images are extracted using the same method, as shown in The adjustment parameter From the comparison data, it can be observed that images with a higher similarity but non-loop closure have higher differential values of similarity compared to the loop closure image pair, and after the calculation of the similarity of the block, similarity decreases more, which is more conducive to judging the correct loop closure.The experiment used two publicly available datasets, namely, the New College and City Center datasets. New College and City Center are datasets provided by the Mobile Robotics Group of the University of Oxford . During k is added. The algorithm can adjust for various environments via altering the value of k. As a result, the parameter k impacts the extent to which the algorithm performs in terms of time and loop closure detection. The effectiveness of the loop closure detection is expressed through accuracy and recall. Accuracy is defined as the ratio of the correct loop closure detected (true positive (TP)) to all loop closure, as determined by the algorithm (true positive (TP) and false positive (FP)). The recall rate is defined as the ratio of detected correct loop closure (true positive (TP)) to all loop closure (true positive (TP) and false negative (FN)) in the dataset. Due to the occurrence of false positives leading to errors in backend optimization algorithms, the loop closure detection effect is represented by a recall (%) with 100% accuracy. The time performance is represented by the average query time t (ms) of each image in the algorithm. The experimental results are shown in Within the block similarity calculation module, an adjustment parameter k = k increases, the loop closure detection performance of this algorithm first increases and then decreases. Within a certain range, the recall rate at 100% accuracy is higher than the red dot; outside of this range, it will be lower than this point. This is mainly because as parameter k increases, the similarity of error loop closure image pairs with similar appearances gradually increases compared to many correct loop closure image pairs with less similar appearances, resulting in a decrease in recall rate at 100% accuracy.In k = k value.The average detection time of the method consists of three parts: image descriptor extraction, image descriptor reduction, and block similarity calculation. On the New College dataset, this algorithm performs best in loop closure detection when We undertook a comparison of the present method with the loop closure detection algorithms of four CNN models: GPR, COVFast-LCD, VGG-NetVLAD, MobileNet_v3, VGG16, AlexNet, and ResNet18; the recall rates at 100% accuracy on two datasets are shown in As depicted in The block similarity calculation module proposed in the present method is applied to three CNN models: VGG16, AlexNet, and ResNet18. The experimental results are shown in k = k, resulting in difficulties in real-time performance.In This article presents the use of the MobileNet_v3 neural network model to extract image descriptors and presents a block similarity calculation module used to reduce the similarity of error loop closure image pairs. We successfully improved the recall rate of the visual SLAM loop closure detection method at 100% accuracy, while meeting real-time requirements. Regarding the New College and City Center datasets, this method increased the recall rate at 100% accuracy by 8%, 31%, 26%, and 43% and by 34%, 57%, 42%, and 63%, respectively, compared to four CNN models based on MobileNet_v3, VGG16, AlexNet, and ResNet18. The three CNN models VGG16, AlexNet, and ResNet18 show improved recall rates at 100% accuracy after the block similarity calculation module is applied, demonstrating the method’s universality.


The COVID-19 pandemic has hit urban areas particularly hard, yet there is a lack of research on the hypothesis that living in more compact cities can provide better support for work and social conditions during the pandemic. This study addresses this gap by examining whether city compactness can mitigate the negative impact of the pandemic on the work and social life of urban residents in Jakarta, Indonesia. The study uses a household phone survey combined with publicly available urban form data. Ordinary Least Squares (OLS) regression, supplemented with a matching technique to address potential selection bias, is employed. The results suggest that living in more compact locations can reduce the disruption to work and social life associated with COVID-19 in urban communities. This positive effect is particularly experienced by males, non-migrants, and individuals from wealthy families. The COVID-19 pandemic has forced people to change their work habits and social interactions  regressions, with the urban form composite index as the variable of interest. We recognize the potential endogeneity of urban form and social life or work. For example, if social interaction increases with compactness, a particularly sociable person may intentionally choose to live in a more compact area to find a community that aligns with their outgoing personality  are often used as explanatory variables for outcomes of virus spread  that are characteristic of Indonesian society, have been either eliminated or significantly reduced. This is because people are either concerned about the spread of the virus or are following lockdown measures, which has resulted in them keeping their distance from others.The disruption of livelihoods and reduced intensity of neighbour interactions can potentially lead to mental health problems. Therefore, it is essential to identify potential mediators to better understand the underlying mechanisms. Despite the same harmful shock exposure, the intrinsic factors possessed by the individual can differentiate the stringency of the disruption felt by the individual. Factors such as individual characteristics , family characteristics, and environmental characteristics may play a role in this differentiation.This study investigates how environmental factors, particularly the compactness of urban living, can impact livelihood and social life during the COVID-19 pandemic. Research suggests that compact living environments may lead to less social life disruption and better mental health outcomes due to the social capital they foster  classified as urban, and one administrative regency classified as rural. This study analyses the compactness of the 42 urban subdistrict areas in Jakarta. Moreover, Jakarta was the site of the first reported case of COVID-19 in Indonesia and became the epicenter of the virus spread within the country.For our case study, we focus on Jakarta, the capital city of Indonesia with a population of over 10 million people , and j in area i. The higher the index means the more diverse land use the area is.The entropy index of area type measures land use diversity in Jakarta's subdistricts, as presented in Eq. . We diffmentclass2pt{minimStreet accessibility is the third scale and is determined by the ratio of road length to area size, as shown in Eq. . Higher We present the statistics for the mean and standard deviation of the three urban form indicators in the 42 subdistricts of Jakarta in Table The average road length in our sample is 24.8 km per square km, which is slightly higher than that of Vancouver, the densest city in Canada, with 23.2 km per square km  is the lowest residential density value, and MAX is the highest residential density value in the subdistricts in the dataset.MIN(street_accessibility) is the lowest street accessibility value, and MAX(street_accessibility) is the highest street accessibility value in the subdistricts in the dataset.To calculate the urban form index, we employ the aggregation method by adding the three indicators. Residential density and street accessibility are standardized to ensure comparability, as shown in Eqs.  and 5)::5):4\docThe urban form index is the average of the standardized residential density, standardized street accessibility, and land use diversity indicators, assuming equal contribution from each of the three indicators. This index ranges from zero to one, where a higher value indicates a more compact area. The distribution of the urban form index is shown in Fig. The second source of data was collected through a phone survey conducted between 1 May and 30 June 2020, in collaboration between the Faculty of Economics and Business at the University of Indonesia (FEB-UI) and the Australian National University (ANU). Undergraduate students primarily from Universitas Indonesia (UI) and Universitas Negeri Jakarta (UNJ) were employed as enumerators for the survey. The study was approved by the ethics committee of the Australian National University.The sampling frame and sample weights for the phone survey were collected from the statistical agency of Indonesia (BPS), using the 2018 Flood Survey sampling frame and the Micro and Small Enterprise sampling frame of the 2016 Economic Census. A random sample of 4,600 respondents was selected to be interviewed by phone, but 3,559 could not be reached, and 699 declined to participate. Ultimately, only 342 respondents agreed to be interviewed, resulting in a response rate of approximately 7%. This response rate is comparable to other phone surveys conducted in Indonesia during the COVID-19 pandemic  of Indonesia 2018. Table This study addresses the potential for systematic selection bias in the dataset before estimating the impact of urban form on work and social life disruptions, specifically the bias caused by sorting mechanisms. Sorting occurs when individual characteristics drive them to live in more compact or less compact subdistricts, which can influence the likelihood of individuals experiencing work and social life issues.For example, if older individuals tend to live in less compact areas and have lower work-social life resilience due to their age, the estimates of the urban form effect on work-social life disruptions could be understated. If this issue is not appropriately addressed, it can lead to biased estimates . The variables significantly associated with the urban form were subjected to the outlier removal process.To address the sorting issue, we employed propensity score-based matching using the PSMATCH2 command in STATA software developed by Leuven and Sianesi . The aimThe selection into urban form measure is specified as follows:Here, The vector i is an index for observation and j is an index for subdistricts:The primary specification for examining the impact of urban form on work and social life disruptions is presented in Eq. , where iβ indicates that urban form is a protective factor for work and social life conditions during the COVID-19 pandemic.The outcome variable entclass1pt{minimaThe first post-regression analysis in this study aims to test the robustness of the treatment effect coefficient . The marginal effect of urban compactness in this sensitivity analysis is estimated using OLS and logit estimators to observe whether they indicate a similar effect to the results from the main estimate.Secondly, we use the ordered logit model to evaluate the marginal effect of urban form on work and social life, which has an ordinal scale. Thirdly, we also examine the sensitivity of the urban form variable by transforming it into a binary variable indicating compact or not compact areas. This transformation is done because the main estimation uses post-matching results where the matching process is based on binary transformed urban form variables.The final robustness test is related to the possibility of spatial dependencies because we study the effect of urban form from a geographical standpoint. Therefore, we also examine the stability of the urban form effect using the spatial error model and spatial lag model. To make the spatial model estimation work, we expand the data based on the weight of the matching results. The spatial weighting matrix is created based on rook and queen contiguity.We conducted additional analyses to investigate which component of urban form is most relevant to work-social disruption. Specifically, we used Eq.  but replIn this section, we assess the possibility of data sorting based on residing in a compact urban environment. The analysis indicates that four variables are significantly associated with the urban form variable, suggesting the presence of sorting issues. Two variables are at the individual level, including age and family size, and the other two variables are at the community level, including access to community health centres and other health facilities. The results in Table We examined the presence of outliers in the observational data of the four variables significantly associated with the urban form variable. Outliers were found in the family size, community health centre access, and other health access variables, except for the age variable. A total of 26 outliers were identified, resulting in 314 observations remaining in the dataset. Next, we re-evaluated the association between urban form and these four variables using the excluded outlier data. The results showed that the associations of the three variables were no longer significant. However, the age variable still had a significant association with weaker significance  as a factor affecting work and social life during the COVID-19 outbreak. If the outcome variable of our study can also be viewed as a potential proxy for well-being, a possible comparison can be made with the work of Huerta and Utomo  who analOutside the COVID-19 pandemic, other relevant studies that created an urban form index using Ewing’s approach include Garrido-Cumbrera et al.  and SturThe stability of the urban form's effect on work and social life disruption is examined through several robustness tests, including different standard error formulas, coefficient stability testing, different matching methods, and alternative operational definitions of the outcome and interest variables. The matched data are used in conducting these tests and subsequent analysis. Firstly, the bootstrapping confidence interval is performed using the framework developed by Roodman et al. . The resSecond, another robustness test was conducted using the coefficient stability framework developed by Oster  to addreThird, we conducted a robustness test using different matching methods . The comparison of the coefficients using the OLS model and spatial model can be seen in Table The first additional analysis aims to identify which component of urban compactness is relevant to work and social life disruption. The results are presented in Table The second extended analysis examines the heterogeneous effects of urban compactness on work and social life disruption by performing regressions on subgroups based on gender, migrant status, and family expenditures. The results show that the effect of urban form is significant for males, non-migrants, and high family expenditure groups. However, the effect is insignificant for females, migrants, and low-family expenditure groups  are often used as indicators of social capital. However, these activities tend to be dominated by males, which may explain why the effect of compact living environments is more strongly felt by males than females. Additionally, non-migrants may experience a stronger effect of urban form on work and social life disruption because their social cohesiveness is more affected by the environment in which they have lived. On the other hand, the urban form effect is less influential in low-expenditure families, as they may not have the same level of participation in social and community activities as high-expenditure families.These assumptions are based on the theoretical relationship between compact urban form, social capital, and work and social life disruption outlined by Evans . HoweverThis paper explores the role of urban form in moderating the negative effects of the COVID-19 pandemic on work and social life. Our findings suggest that urban form can serve as a protective factor in preventing these disruptions among individuals living in cities. Specifically, our estimates indicate a non-trivial premium of living in a more compact location, with a less reduction in work and social life disruptions by approximately 13%. Our conclusion is supported by several robustness tests that address potential selection bias, omitted variable bias, and spatial interdependency.Our research supports the implementation of pro-compact city policies that prioritize balanced and diverse land use factors, as well as accessibility. Compactness can provide a protective layer for work and social life during times of crisis, such as the COVID-19 pandemic. Policymakers should focus on promoting the advantages of compactness by creating policies that prioritize these factors. By doing so, they can help build resilient communities and reduce the negative impacts of future crises.Our research highlights the importance of land use diversity and road access in providing protection to communities against work and social life disruption during the pandemic. However, we do not deny the role of higher residential density in providing some protection in this matter as well. This research emphasizes the crucial role of good urban design in responding to urbanization and promoting the well-being of urban residents.Our research also highlights the unequal distribution of benefits of compactness during pandemics. We found that certain subpopulations, such as males, non-migrants, and individuals from high-expenditure families, are more likely to benefit from living in compact environments during pandemics. This suggests the need for policymakers to consider targeted policy interventions that address these disparities. Specifically, we recommend that policymakers develop policies that prioritize the needs of females, migrants, and individuals from low-expenditure families. By doing so, they can help reduce the disparities in the benefits of compactness and build more inclusive communities that are better equipped to handle future crises. We believe that prioritizing equity and inclusion in urban policy can lead to stronger and more resilient communities that are better equipped to tackle future challenges.In a broader context, our research emphasizes the positive impacts of compact city design on the economy, environmental quality, and social and health issues. Specifically, our findings suggest that compactness can help reduce work and social life disruptions, which may have potential benefits for mental health among individuals. This highlights the importance of prioritizing compact urban form in building stronger and more resilient communities that are better equipped to handle future crises.While our research does not provide conclusive evidence on the mechanisms through which urban compactness protects against the negative effects of pandemics, we believe that stronger social capital may play a key role. Several studies have suggested that urban compactness is associated with stronger social capital (Glaeser and Sacerdote Overall, this paper provides important insights into the potential benefits of compact urban form in promoting resilience during crises. By knowing the role of urban form in mitigating the negative effects of pandemics, policymakers can develop more effective strategies for building strong and resilient communities. However, further research is needed to understand the mechanisms underlying these findings and to explore the potential trade-offs associated with promoting compact urban form.


Dissociation is a generally recognized phenomenon in psychology and psychiatry; however, questions are still not fully resolved about the difference between pathological and normal dissociation, as well as the role of dissociation, depending on its aetiology, in the formation of clinical manifestations of mental disorders.To complement the existing data about the significance of dissociation in non-psychotic mental disorders.Using the Dissociative Experience Scale (DES), we screened 62 patients  from the Non-Psychotic Conditions Inpatient Department of the Udmurt Republican Clinical Psychiatric Hospital . Nineteen of the patients had mental disorders of organic aetiology and 43 patients had mental disorders of psychogenic aetiology.Dissociation at the pathological level was detected in 12.9% of the patients, all of them female. Among patients with psychogenic disorders, the proportion of patients with pathological dissociation was more than three times that of patients with organic disorders. Among the particular dissociative phenomena, absorption had the highest average severity, both in the general sample and in each aetiological group of patients, while dissociative amnesia had the lowest average severity. The highest levels of dissociation were found in young female patients who had never been married. In patients with psychogenic disorders, the average dissociation severity was significantly higher than in the general population, while in patients with organic disorders it was significantly lower.The dissociation phenomenon may play a significant symptom-forming role in young women suffering from non-psychotic mental disorders of psychogenic aetiology. In the case of organic mental disorders, the severity of dissociative manifestations decreases even below the conditionally normal level, which may indirectly indicate the destruction of dissociative physiological mechanisms by an organic brain process. Dissociation is a psychic phenomenon consisting in the detachment of certain experiences  from a person’s consciousness. In a particular individual, this phenomenon may be manifest as the alienation of his own psychic acts or even his own “self”, the alienation of psychotraumatic events experienced by him, or the disintegration of memories of these events . The terThe predominant concept of the origin of dissociative disorders in the world today is the so-called traumatic concept, which considers these disorders a result of childhood mental trauma . In thisabsorption — an affective absorbing of attention by a bright external object, up to complete mental fusion with it; distraction — uncontrolled shifting of attention from surrounding reality to internal experiences ; depersonalization — a feeling of one’s emotional change or loss of a sense of reality; dissociative identity violation — a transient or persistent experience of splitting or alienation of one’s “self ”; and dissociative amnesia — obscurity or total loss of memories about psycho-traumatic events.Several authors note the existence of normal dissociation, which is part of a healthy psyche and is a mechanism of psychological defense, along with other similar mechanisms . The difSeveral scales have been developed to assess the severity of dissociative manifestations, including scales for large-scale epidemiological studies . There aThe objective of the study is to complement the existing data about the significance of the dissociation phenomenon in non-psychotic mental disorders.Tasks of the study:To determine the prevalence of pathological dissociation among patients with non-psychotic mental disorders;To explore dissociation severity in non-psychotic mental disorders with different aetiology;To explore dissociation severity in non-psychotic mental disorders at different demographic groups;To compare dissociation severity in non-psychotic mental disorders of different aetiology with its severity in the general population.Table 1. Patients with non-psychotic mental disorders from two blocks of the ICD-10 — “Organic, including symptomatic, mental disorders” (F0) and “Neurotic, stress-related and somatoform disorders” (F4) — were included in the sample. Nineteen patients were diagnosed with a mental disorder from block F0, and 43 patients from block F4. In other words, the mental disorders of 19 patients had an organic aetiology and those of 43 patients had a psychogenic one.The study was conducted with patients of the Inpatient Non-Psychotic Conditions Department of the Udmurt Republican Clinical Psychiatric Hospital . All adult patients being treated in that department for the three-month period (from 1 October 2019 to 31 December 2019) who agreed to participate in the study were screened, for a total of 62 patients, 13 male and 49 female . The age of the patients ranged from 21 to 74; the average was 50.4 ± 3.6 years. The diagnostic composition of the sample is presented in An experimental-psychological method was used to perform the study, the Dissociative Experience Scale (DES) developed by Dissociation exceeding the normative level of the DES was detected in 8 patients (12.90% of the sample). All the patients with pathological severity of dissociation were female, and 16.33% of all female patients had a pathological level of dissociation. In patients with psychogenic disorders, dissociation at the pathological level was observed in 7 patients (16.28%), while in patients with organic mental disorders that level was found only in 1 patient (5.26%). Thus, among the patients with psychogenic disorders, the proportion of persons with pathological dissociation was more than three times that of the patients with organic disorders.Table 2). Among the particular dissociative phenomena, absorption, on average, was the most severe in the sample, followed (in descending order of average severity) by distraction, depersonalization, dissociative identity violation, and dissociative amnesia. It is particularly significant that the average level of dissociation in the patients with psychogenic disorders was almost twice that of the patients with organic mental disorders (p < 0.05). The average severity of each particular dissociative phenomenon in the former group was also higher.The average general dissociation severity in the sample according to the DES was 11.36 points (see Table 3) shows that in women its average was slightly higher than in men (p < 0.05). The average severity of each dissociative phenomenon was also higher among female patients. In conjunction with the higher prevalence of pathological dissociation among the women, this result indicates that dissociative mechanisms in patients suffering from non-psychotic mental disorders are more developed specifically among the women.A comparison of general dissociation severity in patients by gender (see Table 4). The same pattern was observed for each particular dissociative phenomenon. The data suggests that dissociative psychological mechanisms in patients with non-psychotic mental disorders weaken with age.For studying dissociation severity in the different age groups of the patients, we used the World Health Organization’s (WHO) age periodization. The highest rate of general dissociation was observed in young patients (ages 18–44). In older age groups, that rate gradually decreased (see Table 5). Married and divorced patients had medium values according to the considered indicators. It is likely that the described pattern is related to differences in the average age of the patients, as younger people are predominant in the “have never been married” group, while older people are predominant in the “widow and widowers” group.An analysis of the dependence of dissociation severity on the marital status of the patients showed that the highest rates of both general dissociation and particular dissociative phenomena were observed in patients who had never been married, and the lowest rates were observed in widow and widowers . In patients with psychogenic disorders, the average dissociation compared to the corresponding value for the general population was even higher than in the study sample as a whole. This result allows us to hypothesize that in the case of psychogenic mental disorders, dissociative processes are indeed stronger and probably play a significant role in the formation of clinical manifestations of these disorders.In this study, the average level of dissociation in the sample was also compared with the level of dissociation in the general population. The data about the level of dissociation in the general population were taken from the literature . We founThe average dissociation severity in patients suffering from organic mental disorders, by contrast, was significantly lower than in the general population. This leads us to conclude that in organic mental disorders, unlike mental disorders of psychogenic aetiology, dissociative processes in most cases do not intensify, but rather weaken.The degree of scientific novelty of the results obtained is ambiguous. In particular, the results obtained in the present study, showing greater severity of dissociative processes in women suffering from non-psychotic mental disorders compared to men, confirm the data of several previous studies . As a poAn interesting result is that dissociative processes in patients with non-psychotic mental disorders weaken with age. Several hypotheses can be suggested to explain this phenomenon. First, organic brain changes that occur with age, particularly of vascular or atrophic origin, may contribute to the weakening of dissociation . Second, our results could be explained by the physiological age-related reduction of psychical mobility . Third, An interpretation of possible causes for the influence of a person’s marital status on the severity of his or her dissociative reactions has already been given above. At the same time, it should be noted that the inverse relationship is also possible: Higher levels of average dissociation in persons who have never been married or in divorced persons may not be a consequence, but a cause of their dysfunctional marital status. This means that mental health problems related to the increased dissociation can lead to difficulties in starting a family or to issues in family relationships that might result in a divorce. Clarification of this question also requires further research.The analysis of the severity of various particular dissociative phenomena in non-psychotic mental disorders showed that in the studied patients these phenomena appear unevenly. As already indicated, absorption reaches the highest level in the sample, followed by distraction, depersonalization, identity violation, and dissociative amnesia. Possibly this result is related to the different degrees of the adaptation violation connected with each of these phenomena and to the different degree of deviation of the corresponding mental functions. Obviously, such dissociative phenomena as absorption or distraction are less severe and disadapting than identity violation or dissociative amnesia. Apparently these dissociative phenomena, typically observed among mentally healthy people, constitute a certain level of so-called normal dissociation. These phenomena are amplified in non-psychotic mental disorders, so their severity becomes significantly higher. Such phenomena as identity violation or dissociative amnesia are not common in a healthy population; they occur, mainly, in cases of mental illness, and require more serious reasons for their formation. Probably they become intense only among a small percentage of non-psychotic patients with sufficiently severe mental disorders. Accordingly, the average rate of identity violation and dissociative amnesia in the sample is not so high in general. Depersonalization occupies a middle position among all the dissociative phenomena by the severity of its disadaptation and therefore its level in the sample is in the medium range.An important finding of this study requiring a separate discussion is the weakening of dissociation in organic mental disorders that are accompanied by decreasing dissociative reactions even below the conditionally normal level. In our opinion this phenomenon may be caused by the destructive effect of an organic brain process on the physiological bases of dissociation, appearing to be a quite complicated and delicate mechanism in terms of brain physiology and requiring a sufficiently intact condition of its anatomical structures.In this study, dissociation at the pathological level was found in about one-eighth of all the patients with non-psychotic mental disorders. It can be assumed that dissociation plays a significant role in the formation of clinical manifestations of mental illnesses among those patients. Dissociation makes an especially major contribution in the cohort of young females who have never been married. According to the results obtained, the increase of dissociative mechanisms is characteristic mainly of mental disorders of psychogenic origin. In the case of organic mental disorders, on the contrary, these mechanisms weaken, possibly due to the destructive effect of the organic brain process on their physiological bases.Generalization of the results of this study is limited by the small sample size.


This study analyzes the explanatory power of the latent factor conditional asset pricing model for the Korean stock market using an autoencoder. The autoencoder is a type of neural network in machine learning that can extract latent factors. Specifically, we apply the conditional autoencoder (CA) model that estimates factor exposure as a flexible nonlinear function of covariates. Our main findings are as follows. The CA model showed excellent explanatory power not only in the entire sample but also in several subsamples in the Korean market. Also, because of this explanatory power, it can better explain market anomalies compared to the traditional asset pricing models. As a result of examining investment strategies using pricing error, the CA model measures the expected return of stocks better than the traditional asset pricing model. In addition, the CA model indicates that the firm characteristic variables are important in asset pricing conditional on macro-financial states, such as the global financial crisis and the coronavirus disease 2019 pandemic. The result shows that the major variables considered in the explanation of stock returns through the CA model may vary depending on the time. This is expected to provide a broader perspective on asset pricing through the CA model in the future. Various risk factors have been introduced in the extant literature to explain cross-sectional stock returns or market anomalies that traditional asset pricing models fail to elucidate –3. Theseet al. . Subsequently, we rank the standardized value to check the importance order of the variables in the CA model. This case has a problem in that the importance of another variable may converge to zero when the extent of change in the R2 of a specific variable is large. Here, we identify the top 10 most important firm characteristics ex-post. To simplify, we set K = 5 in the CA model. We rank the importance of the characteristics by estimating the reduction in the total RAdditionally, The importance of firm characteristic variables in each model is similar. Particularly, the top five high-importance variables and the bottom five low-importance variables are similar in each model.When the characteristic importance is calculated through the CA models, different results may be obtained depending on the process of model fitting. We fit several parameters because the autoencoder is a type of neural network, and the model may overfit or underfit depending on hyperparameters. Nevertheless, the characteristic importance in the autoencoder model is useful in assessing the latent factors. For instance, the CA model has important implications for identifying significant firm characteristic variables in asset pricing. By employing these, we can identify the factors that affect asset prices when the market changes rapidly, such as in a financial crisis. Therefore, we now examine the change in the importance of firm characteristics in the subsample through the CA models.We set the global financial crisis as the period from July 2007 to June 2009 and the COVID-19 pandemic from January 2020 to December 2020. The results in et al. [We applied machine learning-based CA asset pricing models to the Korean stock market. The autoencoder—one of the popular machine learning methods—was employed to extract latent factors, following Gu et al. . The aut2 and stability under various settings including KOSDAQ, small stocks, penny stocks, illiquid stocks and irrational investors’ stocks. This result shows that the CA model can provide generalized explanatory power in markets other than the US market, which is widely used in asset price model studies. Also, as a result of subsample analysis, the existing asset price model has a difference in explanatory power depending on the sample, whereas the CA model shows excellent explanatory power for several subsamples. This indicates that the CA model sufficiently supplements the limitations of the existing asset price model, which lacks explanatory power in a specific subsample. This indirectly shows the structural advantage of deep learning in which the importance of input data dynamically changes through a hidden layer when estimating the latent factor of the CA model.We examined the explanatory power of the CA model for the Korean market. Subsequently, we compared the CA model with the traditional asset pricing model. Our results demonstrated that the CA model dominates the traditional models  in terms of OOS RThe CA model can explain several market anomalies that the FF models are unable to clarify. In other words, it shows that the latent factor has a common market risk factor that has not been considered in the existing asset price model. In addition, by using the pricing error of the asset pricing model, strategies based on overvalued stocks or undervalued stocks were compared. The comparison reveals that the performance of the CA model was excellent. This means that the CA model can accurately determine whether a stock is overvalued or undervalued compared to traditional asset pricing models. Thus, the CA model explains the expected returns of stocks well. Lastly, the CA model also revealed the firm characteristics that are important in asset pricing and how their importance varies with macro-financial states. This is the advantage of being able to identify variables that had a significant impact in the entire sample period or a specific period through the CA model. In addition, it shows that the importance of each variable is changed over time, which is a major feature of the CA model. This is a big difference from the model in which the factor is fixed and defined in advance like the FF model. Due to these characteristics, it can be inferred that the CA model shows superior explanatory power compared to the existing asset pricing model.This study provides several research possibilities. First, trading strategies can be devised using the CA model. Second, more international studies beyond the Korean and U.S. markets are needed. Third, another machine learning-based dimension reduction technique can be compared with the CA model. Fourth, while we focus on an equity market, fixed income and other asset classes can also be analyzed.S1 Appendix(DOCX)Click here for additional data file.


An end-to-end approach to autonomous navigation that is based on deep reinforcement learning (DRL) with a survival penalty function is proposed in this paper. Two actor–critic (AC) frameworks, namely, deep deterministic policy gradient (DDPG) and twin-delayed DDPG (TD3), are employed to enable a nonholonomic wheeled mobile robot (WMR) to perform navigation in dynamic environments containing obstacles and for which no maps are available. A comprehensive reward based on the survival penalty function is introduced; this approach effectively solves the sparse reward problem and enables the WMR to move toward its target. Consecutive episodes are connected to increase the cumulative penalty for scenarios involving obstacles; this method prevents training failure and enables the WMR to plan a collision-free path. Simulations are conducted for four scenarios—movement in an obstacle-free space, in a parking lot, at an intersection without and with a central obstacle, and in a multiple obstacle space—to demonstrate the efficiency and operational safety of our method. For the same navigation environment, compared with the DDPG algorithm, the TD3 algorithm exhibits faster numerical convergence and higher stability in the training phase, as well as a higher task execution success rate in the evaluation phase. Autonomous intelligent mobile robots  are suitOn the basis of their principles and era of development, robot planning methods can be divided into traditional and learning-based methods. Traditional methods include graph search algorithms, sample-based algorithms, interpolating curve algorithms, and reaction-based algorithms . The DijReinforcement learning (RL) ,17,18 isAn RL agent learns by directly interacting with the environment without supervision and without a model of the environment. At the end of an episode, the agent generally receives positive rewards through environmental feedback. For long-distance navigation in environments containing obstacles, robots have difficulty obtaining the final positive reward signal. The sparse reward problem  leads toAC-based deep RL (DRL) approaches—including the deep deterministic policy gradient (DDPG) , twin-deIn an environment exposed to multiple obstacles, the agent learns how to avoid obstacles and reach its destination without collision. The DWA  is a traTo address the severe problem of reward sparsity in complex environments, survival penalties can be applied as rewards to ensure successful learning. A survival penalty-based approach is used within an AC framework to solve a motion-planning problem in which the target position and orientation are aligned simultaneously. We design a comprehensive reward function and conduct it in three navigation scenarios to ensure that the goal can be approached while avoiding obstacles dynamically.The remainder of this paper is organized as follows. A path is a route from one point in space to another. The path-planning problem is closely related to the collision avoidance problem. Although most studies have aimed to solve the path-planning problem in a flexible and efficient manner, they have considered only motion planning involving position transitions, without considering orientation-related tasks. In this paper, we propose a planning approach in which orientation and localization tasks are tackled simultaneously for further investigating the potential of WMRs. We address attitude adjustment in the final goal of the WMR’s task. As illustrated in As depicted in The attitude of the WMR is described by the following generalized coordinate vector with five components: Let  the WMR  can be eThe line velocity and angular velocity of the WMR at point Equation (3) is substituted into (2) to obtain the ordinary form of a WMR with two actuated wheels, which is expressed as follows:As shown in The objective function in the DDPG algorithm is maximization of the expected cumulative reward t + 1; In the DDPG approach, Monte Carlo-based updating is converted into a temporal difference (TD) method. The actual reward is compared with the critic’s estimate to obtain the TD error, which is used to adjust the parameters of the critic network and thus obtain a more accurate estimate. When Monte Carlo methods are used, the actual reward cannot be determined until the end of the RL episode. In the TD method, only the next interaction state For the TD method, the objective function of the critic network that minimizes the mean square error (MSE) between the target value The parameter Conversely, the parameter The gradient is computed using the chain rule as the expected value of the product between the gradient of the critic network with respect to the state–action and the gradient of the actor network with respect to its action. The actor is updated to make the output action be more favorably evaluated by the critic.In the DDPG algorithm, a replay buffer is used to break the Markov nature of sampled data, and two target networks are employed to ensure the stability of the training process. The TD method is used to redefine the objective functions of the critic network and actor network as follows:The parameters Q function beginning to overestimate the Q values substantially. Errors of the Q function are introduced into the training of the actor network, which leads to policy violation. The TD3 algorithm [j represents the two target critics, and the clip function constrains future actions within the lower and upper bounds specified for the controls. As an output, the clip function returns its argument unless this argument violates the bound, in which case the argument is set to be equal to the bound. Each critic computes the MSE of the TD error A common mode of failure for the DDPG algorithm involves the learned lgorithm , which iUsing a small critical value of the target and regressing to that value helps to prevent overestimation of the Q function. The parameters of the target actor and target critic are updated less frequently in the TD3 algorithm than in the DDPG algorithm.The motion planning required to perform a given WMR task can be fundamentally divided into navigation and obstacle avoidance. The navigation module obtains information on the relative attitude of the WMR with respect to the target, whereas the anti-collision planning module receives raw LiDAR sensor data that contain information on the unknown and dynamic environment in which the WMR is moving . The WMRectively a. To speAs the WMR successfully navigates itself to the target, all elements in the pose state vector ith laser sensor. The input state information To plan an obstacle-free path, the robot’s system must extract information from the environment by using sensor data and thereby generate commands that ensure obstacle avoidance. The action to be taken by the WMR is the set of the linear forward velocity When the reward function is used as a training metric to encourage or discourage action, this function has a strong influence on learning performance. In many studies ,26,49, tPath planning is performed at regular intervals. When the WMR is traveling at a forward velocity of 1 and 2 intervals per time step, the agent receives a cumulative reward of 55 and 30, respectively, after completing episodic work. The goal in DRL is to guide an agent that is attempting to maximize the cumulative reward from the initial state distribution. A potential drawback of only implementing positive rewards is that the agent employs a slow movement policy to perform the situational task and thereby obtains a relatively high cumulative reward. In study , auxiliaThe survival penalty function in which a negative relative distance As displayed in To guide the WMR precisely into the target area and simultaneously achieve the correct position and orientation, the intersecting position a. The smThe constant negative bias in the first term of each penalty equation results in the WMR being primarily steered in the correct direction, with the lock angle being reduced and the punishment lowered as the WMR approaches the target.The second case we consider is that in which the WMR is located in the first or fourth quadrant. The objective of the reward function in this case is to guide the WMR away from the dangerous passageway adjacent to the target area and into the rear of the target. When the alignment angle The survival penalty functions expressed in (25) and (26) indicate that negative rewards (punishments) are always awarded as the WMR transitions from a high-penalty state to a low-penalty state. An additional reward is provided at the end of each episode: 5 and −10 for successful completion of the navigation task and for a failed mission, respectively.During the training phase, when navigation paths are being explored, scenarios involving collisions between the WMR and the environment or obstacles are easily generated. In many studies ,49, schoWe introduce a simple method that involves exploiting the survival penalty function to manage obstacle avoidance. The TD3 and DDPG algorithms are off-policy algorithms that create experience replay buffers to store historical experiences, randomly sample transitions from these experiences, and then employ these sample data to update the actor and critic networks. The existence of the experience replay buffer helps the agent to learn from previous experiences and improve the efficiency of sample utilization. Q value. The input layer of the critic network is the same as that of the actor network. The output of the critic network’s first fully connected layer is concatenated with the output of the actor network. The critic network, the inputs to which are state–action pairs, finally generates a continuous value through a linear activation function.γ is set as 0.99, the learning rate of the actor and critic networks is set as 0.00001, and τ = 0.01 for the soft updating of the target networks. The size of the replay buffer is 40,000, and the batch size of updating is 128. Moreover, the maximum number of steps in each episode is set as 300. Simulations are performed on a computer with an Intel i7-11 CPU with 32 GB of memory and an Nvidia RTX 4090 GPU. Python 3.10 is used as the project interpreter. The deep-learning framework PyTorch 2.0 is employed for training the networks in a Windows system.To prove the effectiveness and superiority of the proposed survival penalty function, simulations are performed that involve the DDPG and TD3 algorithms with the same reward functions and hyperparameters. Navigation tasks in four virtual environments are investigated. In this scenario, we construct a virtual navigation environment in which the initial and target yaw angles have the same magnitude but opposite signs. The distance between the initial and target points is set as 300. The navigation task is accomplished using a survival penalty function. h. Liang  imposed h. Liang  proposedTo demonstrate the general ability of the proposed method to adapt to other environments, we consider a parking scenario. The virtual environment that is simulated consists of two parallel rows of parking spaces in a space with upper and lower boundaries , and parA virtual environment is designed that simulates a general intersection, as illustrated in The virtual environment is designed to demonstrate the adaptability of our approach to scenarios with multiple obstacles. The scenario shown in In order to further verify the actual performance of our method, we conducted comparative simulations using these two algorithms in four virtual environments. A common failure mode of the DDPG algorithm  is that In our simulations, the DDPG agent easily forgets previously learned knowledge. During the training process of the obstacle-free scenes, the agent sequentially learns the optimal paths from the configurations of yaw angles of 0°, 20°, …, 180°. When the overall configurations are performed, the autonomous path of the 0° yaw angle configuration becomes poor. The 0° yaw angle configuration needs to be retrained for improving the path quality. Backpropagation, which performs a backward pass to adjust neural network model parameters, may refresh or update previously learned weights. As shown in The reward function is an incentive mechanism that tells the agent what is correct and what is wrong through rewards and penalties. RL applications can be transformed into sequential decision-making problems, and states, actions, and rewards can be established. The agent’s goal is to maximize the total reward. Designing and implementing a reward function that is consistent with an application is challenging. For example, the application of RL in electric vehicle battery-charging management  uses posTD3 and DDPG are model-free RL off-policy RL algorithms. Model-free algorithms learn directly from experience and can operate without complete knowledge of environmental dynamics or transition models. Simulated environments are the most common method of training agents. The kinematic model of WMR is based on a two-dimensional framework in the x- and y-directions. When a vehicle travels on uneven terrain, the vehicle will generate significant vertical vibrations. Movements in the z-direction may affect the precise positioning of the vehicle, which is a key requirement for autonomous navigation. Bikes, tripod cars, and four-wheel drive cars are real vehicles with different motion structures. The simulation environment could use an accurate vehicle model to generate relevant experience, but this would require a lot of computer time to solve the nonlinear vehicle equations. The agent relies on sampling to estimate the value function, resulting in noisy estimates and slower convergence, but model-free algorithms can be used in large, complex environments. Our approach can be extended to complex 3D environment applications such as UAVs or robotic manipulators. The goal of path-planning techniques is not only to discover optimal and collision-free paths, but also to minimize various issues such as path length, travel time, and energy consumption that require further research. Using multiple robots to conduct collaborative multi-observation of multiple moving targets is a future research direction.(1)Appropriate reward functions are designed by considering human driving experiences and using a survival penalty method so that a wheeled mobile robot (WMR) can receive negative reward feedback at every step. The reward function varies continually on the basis of observation and action. Dense reward signals are found to improve convergence during training, and this finding verifies the feasibility of solving the sparse reward problem encountered in autonomous driving tasks for WMRs.(2)Two consecutive RL episodes are connected to increase the cumulative penalty. This process is performed when the WMR collides with an obstacle. The agent is prevented from selecting the wrong path, learns a path in which obstacles are avoided, and successfully and safely reaches the target.(3)The effectiveness and robustness of the proposed method is evaluated for three navigation scenarios. The simulation results indicate that the proposed method can effectively solve the problem of planning a path along which a WMR can drive to its destination in a complex environment. The results also indicate that the proposed method is robust and self-adaptive.(4)By using the DDPG and TD3 algorithms with the same AC framework, the WMR can safely and quickly reach its target under the simultaneous alignment of position and orientation. For the same environment and standard, compared with the DDPG algorithm, the TD3 algorithm requires considerably fewer computations and has higher real-time performance.(5)The path-planning method has the ability to handle orientation and positioning simultaneously. In an environment without map information, the intermediate waypoints between the departure point and the destination do not need to be pre-determined segmentally, and an end-to-end obstacle-free autonomous U-turn path can be realized. End-to-end vehicle navigation has the potential to be used in high-end industrial robot applications such as electric vehicle parking and forklift picking and placing.The main contributions of this study are summarized as follows:For the autonomous navigation of a WMR, the target location as well as a target orientation are considered task constraints. Traditional random-action exploration leads to high-probability sparse reward systems. After each action in our method, the agent acquires information regarding the relative position and orientation of the target state, and this information is suitable for providing a dense reward signal, with the provision of a certain immediate reward being a considerably better option than the provision of no reward at all. The survival penalty function utilizes negative rewards to guide the agent to achieve the goal as quickly as possible. The concept underlying the design of the survival penalty function is similar to that underlying human driving behavior, and the negative reward provides a useful means of exploration for the training of agents. Training paradigms based on different task goals can enable the state update policy to be iteratively improved on the basis of the dense feedback signal provided by the environment during the interaction process. Every action taken by the WMR should be rewarded so that it can most effectively assess the quality of its actions and alleviate training difficulty.The DDPG and TD3 algorithms are deterministic policy gradient algorithms based on the AC framework. In the proposed method, these two algorithms are incorporated with high-precision LiDAR range findings. A process in which two consecutive RL episodes in the experience pool are connected; this process helps the WMR to avoid collisions and enables paths to be planned for environments containing obstacles. The optimal policy can be regarded as a mapping from state to action, which enables the WMR to perform the optimally rewarded actions given the current state of the environment. We conduct simulations for three scenarios to verify the performance of the proposed approach: scenarios involving obstacle-free spaces, a parking lot, and intersections without and with obstacles. The actor network relies heavily on the critic network; thus, the performance of the DDPG algorithm is highly sensitive to critic learning. The DDPG algorithm is unstable because of its sensitivity to its hyperparameters and tends to converge to very poor solutions or even diverge. The TD3 algorithm, which has an auxiliary critic network, more accurately estimates cumulative rewards than does the DDPG algorithm and prevents high levels of cumulative reward overestimation. The simulations conducted in the three scenarios reveal that compared with the DDPG algorithm, the TD3 algorithm converges considerably faster, has a superior convergence effect, and exhibits higher efficiency and adaptability in complex dynamic environments. The TD3 algorithm results in the effective avoidance of obstacles in environments and a high rate of successful task completion.