Carrier phase measurements currently play a crucial role in achieving rapid and highly accurate positioning of global navigation satellite systems (GNSS). Resolving the integer ambiguity correctly is one of the key steps in this process. To address the inefficiency and slow search problem during ambiguity solving, we propose a single-frequency GNSS integer ambiguity solving based on an adaptive genetic particle swarm optimization (AGPSO) algorithm. Initially, we solve for the floating-point solution and its corresponding covariance matrix using the carrier-phase double difference equation. Subsequently, we decorrelate it using the inverse integer Cholesky algorithm. Furthermore, we introduce an improved fitness function to enhance convergence and search performance. Finally, we combine a particle swarm optimization algorithm with adaptive weights to conduct an integer ambiguity search, where each generation selectively undergoes half-random crossover and mutation operations to facilitate escaping local optima. Comparative studies against traditional algorithms and other intelligent algorithms demonstrate that the AGPSO algorithm exhibits faster convergence rates, improved stability in integer ambiguity search results, and in practical experiments the baseline accuracy of the solution is within 0.02 m, which has some application value in the practical situation of short baselines. In the Global Navigation Satellite System (GNSS), achieving highly accurate positioning results heavily relies on utilizing carrier phase observations to calculate the receiver-to-satellite distance. However, during carrier phase observation, the receiver can only measure the non-integer portion of the carrier phase, and each measurement introduces an unknown constant referred to as the integer ambiguity . ConsequOver the decades, a variety of algorithms have emerged for solving integer ambiguity problems, including the Least Squares Ambiguity Search Method (LSAST) , Fast Amg et al.  proposedg et al.  improvedg et al.  introducg et al.  proposedg et al.  proposedg et al.  investigWith the advancement of artificial intelligence, various algorithms such as the artificial fish swarm algorithm , simulatThe application of the optimization algorithm to solving integer ambiguity enhances search efficiency compared to classical algorithms. However, these optimization algorithms possess inherent issues such as slow convergence speed, susceptibility to local optima, and other instabilities that result in unreliable ambiguity-solving outcomes. To address these concerns, this paper proposes a single-frequency GNSS integer ambiguity solving based on an adaptive genetic particle swarm optimization (AGPSO) algorithm. By leveraging the strengths of various optimization algorithms, this approach improves the stability of ambiguity solving and facilitates escape from local optima. Firstly, the carrier-phase double difference equation is employed to solve for floating-point solutions and their corresponding covariance matrix for integer ambiguity. Subsequently, an inverse integer Cholesky algorithm is utilized to reduce correlation while an improved fitness function is proposed to improve the convergence and search performance of the algorithm. Finally, the particle swarm optimization algorithm with adaptive weights is combined to perform the integer ambiguity search. Additionally, a selection mechanism is devised wherein the best-performing half of each generation is chosen based on fitness function values to proceed to the next generation. Furthermore, a random crossover operation is applied to the underperforming group by introducing a randomly determined crossover location. Moreover, a mutation factor is introduced to provide a certain probability of mutation for the underperforming group, thereby enhancing the algorithm’s ability to escape local optima. Comparative studies with traditional algorithms and other intelligent algorithms show that the AGPSO algorithm has a faster convergence speed, improves the stability of the integer ambiguity search results, and in practical experiments the baseline accuracy of the solution is within 0.02 m, which has some application value in the practical situation of short baselines.u, r and the satellite j at the moment t; u, r and the satellite j at the moment t; u, r at the moment t; t; and t.In high-precision positioning of GNSS, the term “single difference” refers to the disparity between carrier phase observations acquired from different stations while synchronously observing the same satellite. By employing a single difference, it becomes possible to eliminate satellite clock discrepancies, and, simultaneously, if the user and reference station are near equal altitudes, approximate zero values can be achieved for the ionospheric delay and tropospheric delay . Consequu, r and the satellites j, k at moment t; u, r and the satellites j, k at moment t; t. In estimating the ambiguity floating-point solution, The single-difference observation model can only partially mitigate the error parameter. Therefore, based on this model, an additional reference satellite is selected to calculate the observation difference between satellites, thereby obtaining the double difference observation equation. The double difference further eliminates the receiver clock difference in observations . Consequy is the carrier phase observation; A and B are both design matrices; a and b are the integer ambiguity vector and the baseline vector between the two base stations, respectively; and a and b in Equation (3) can be solved by least squares parameter estimation [It is known from the double difference observation model that a double difference carrier phase observation equation has three unknowns and each additional co-viewing satellite will have one more double difference integer ambiguity, whereas the least-squares estimation can be easily extended to the case of multi-parameter estimation by solving an optimization problem to estimate all the parameters. For carrier phase-based GNSS positioning models, the mathematical models can all be reduced to the following linearized model :(3)y=Aa+timation , therefoThe ambiguity fixation process in Equation (7) is the process of searching for the integer vector with the shortest distance The first thing to be determined in the process of searching for integer ambiguity is the search space. All correct solutions must be encompassed within this search space. In practice, the baseline is usually the parameter vector that needs to be solved. For real-time localization of short baselines, the use of baseline length as a constraint can effectively reduce the scope of the search space and thus improve the efficiency of the algorithm, when the exact value of the baseline vectors is obtained beforehand using precision measurements. Assuming a given baseline length denoted as The ambiguous floating-point solution and its covariance matrix obtained by least squares estimation have a high correlation, in which case the search space is a narrow ellipsoid, which leads to inefficient search. Therefore, an ambiguity decorrelation operation should be performed before the integer of ambiguity search.Among the ambiguous decorrelation algorithms, the integer Gaussian transform , LLL algThe ambiguity search is essentially defined as the minimum value of the equation within the search space. To enhance both the efficiency and stability of this search method, we propose in this paper to employ a genetic particle swarm optimization algorithm for conducting ambiguity searches.The particle swarm optimization (PSO) algorithm was introduced by Kennedy and Eberhart in 1995 AD, inspired by the study of birds’ flocking behavior. Its fundamental concept is to achieve optimal solutions through collaboration and information sharing among individuals within a group . In the The inertia weights ω play a crucial role in optimizing the performance of the PSO algorithm by effectively balancing the global exploration and local exploitation capabilities of the swarm. To enhance the global exploration capability during the early evolutionary stages, it is expected that larger values of particle velocities will be employed. Conversely, as the PSO algorithm progresses towards its later stages, where a strong local detection capability is required, smaller velocity values are anticipated. To address the issue of falling into local optima commonly encountered in standard particle swarm optimization algorithms, this study incorporates Hu’s  proposedThe genetic algorithm (GA), initially proposed by John Holland in the 1970s, is a computational model that simulates the process of biological evolution, incorporating natural selection and genetic mechanisms from Darwin’s theory. By employing mathematical operations and computer simulations, it transforms the problem-solving process into a series of chromosome gene crossovers, mutations, and other evolutionary processes. Essentially, GA is an efficient parallel global search method that autonomously acquires and accumulates knowledge about the search space while adaptively controlling the search process to identify optimal solutions.The genetic algorithms encompass three fundamental operators: selection, crossover, and mutation. Selection functions as a mechanism for “choosing the best from the worst”. Crossover, in conjunction with selection rules, facilitates the preservation of valuable information while discarding unfavorable information. Mutation can produce new varieties that are indeed substantially different . It is tDue to the direct utilization of the PSO algorithm for resolving the integer ambiguity, it often suffers from premature convergence and gets trapped in local optimal solutions, significantly compromising the accuracy of the integer ambiguity solution . In cont(1)The selection operation is responsible for identifying the dominant individuals within the current population. In this study, we adopt a meritocratic approach combined with half selection to identify the superior individuals in the population. Firstly, we calculate and rank their fitness levels, where higher fitness corresponds to higher ranking and increased probability of selection;(2)Crossover operations are responsible for generating novel individuals, achieved by exchanging segments of their chromosomes to produce two offspring chromosomes. The position of crossover is determined randomly, increasing the likelihood of escaping local optima. In this study, the crossover was randomly performed on two selected individuals at two specific crossover points to create new individuals by combining information from the parent’s mating population;(3)The mutation operation is responsible for facilitating the algorithm to escape from local optima, and a smaller value is generally chosen as the mutation probability. A higher mutation probability may lead to the destruction of optimal solutions. After conducting numerous experiments, a mutation probability of 0.1 was adopted in this paper.To address the aforementioned issues, this paper proposes an adaptive genetic particle swarm optimization (AGPSO) algorithm for single-frequency GNSS integer ambiguity-solving. The algorithm incorporates selection, crossover, and mutation operations from the genetic algorithm into the PSO algorithm, with specific enhancements as follows:N undergoes a small change, the function values are still relatively stable, which makes the search inefficient, so the objective function (16) is improved by proposing the following fitness function:The fitness function of AGPSO utilizes the principle of least squares to fit a fixed solution with an ambiguous floating-point solution after reduced correlation so that the floating-point solution reaches the integer optimum under the least squares criterion. The AGPSO population is optimized according to the size of the fitness. The objective function of the integer least squares estimation of integer ambiguity can be obtained according to Equation (7):eunissen  gave a p(1)Initialize the velocity and position of the particles, the maximum and minimum values of the weights, the acceleration constant, the population size, the Mutation factor, the maximum number of iterations, and the minimum error for the termination of the algorithm;(2)The initial adaptation value of each particle is calculated, the population is divided into two groups with good and poor adaptation, and the group with good adaptation is selected to enter the next generation;(3)A random crossover position is generated, and a crossover operation is performed on the poorly adapted set;(4)Introducing a mutation factor that randomly mutates the poorly adapted group when the random number is smaller than the mutation factor;(5)After the group with poor fitness undergoes the cross-mutation operation, the particle fitness is recalculated, combined with the group with good fitness initially, and reclassified into two groups of good and poor fitness. The re-grouped group with good fitness and the initial group with good fitness are taken to form a new particle selection pool;(6)Use the best-adapted value in the new particle pool as the global best optimum, and use the position corresponding to this adapted value as the global optimum position of the particle;(7)Update the particle velocity by the previous formula and limit the flight speed so that it cannot exceed the maximum flight speed;(8)Update the particle position by the previous formula and compare whether the adapted value of each particle is better than the historical optimal value; if yes, then replace it;(9)Calculate whether the adapted value of the particle’s global optimum is better than the historical optimum, and if so, replace it;(10)Repeat 2–9 until the set minimum error is met or the maximum number of iterations is reached;(11)Output the global optimal value of the optimal particle and its corresponding position as well as the local optimal value and corresponding position of each particle.The flowchart of the AGPSO algorithm is shown in To assess the feasibility and performance of the adaptive genetic particle swarm optimization (AGPSO) algorithm proposed in this study for solving ambiguity problems, a comparative analysis was conducted between classical algorithms, particle swarm optimization algorithms, and genetic particle swarm optimization algorithms.In this study, three classical intelligent algorithms, namely Simulated Annealing Algorithm (SA), Genetic Algorithm (GA), and Particle Swarm Optimization Algorithm (PSO), were selected as comparison algorithms. The reference index used to evaluate their performance is the ability to converge to the optimal solution quickly within a specified number of iterations. To ensure consistent experimental conditions, we set the dimension by Jonge . Figure From the comparison results in From As can be seen from the curves in It can be seen from From the principle of the algorithm, the adaptive weights of AGPSO gradually decrease during the iteration process, which reduces the search range of the algorithm when the individual population tends to mature. In addition, the optimal half-taking strategy, the stochastic crossover strategy, and the stochastic mutation strategy all enable the algorithm to jump out of the local optimum quickly.To verify the applicability and effectiveness of the algorithm in practical situations. In the new playground of Nanjing Forestry University, two satellite receivers are used to form a short baseline of 1.42 m. The test scene is shown in As shown in As can be seen in In response to these problems, the test scenario was reconstructed, while the baseline length was increased to 263.52 m. The test scenario is shown in As shown in As can be seen from If the fixed integer ambiguity vector is inaccurate, not only will it not improve the localization accuracy, but also reduce the localization accuracy. Therefore, when the fixed integer ambiguities vector is fixed, it also needs to be examined, i.e., the ambiguities are confirmed. On the basis of systematic study of the theory of ambiguities solution, Verhagen et al.  proposedThe results in (1)In the integer ambiguity search, the particle swarm optimization algorithm (PSO) converges faster than the SA and GA algorithms, and the AGPSO algorithm can reach the optimal solution faster compared with the PSO algorithm. Through high-dimensional data simulation, it is verified that the proposed AGPSO algorithm can effectively solve the problem that the PSO algorithm is easy to fall into the local optimum and improves the efficiency of the integer ambiguity search;(2)To eliminate the chance of the test, the PSO algorithm and the AGPSO algorithm were used to search the three-dimensional and twelve-dimensional integer ambiguity 100 times consecutively. The results show that the AGPSO algorithm approximately doubles the convergence speed of the PSO algorithm, and the AGPSO algorithm jumps out of the local optimum more easily than the PSO algorithm, which significantly improves the stability of the results of the integer ambiguity solution;(3)For GPS L1 single-frequency signal, the AGPSO algorithm is used to search the integer ambiguity after double difference and carry out the baseline solving, the solving result shows that the baseline error is within 0.02 m, which verifies the applicability and validity of the AGPSO algorithm in the practical application. The AGPSO algorithm can be a very good solution to the short baseline solving of the integer ambiguity of searching the problem of inefficiency and instability.In this paper, the important problem of integer ambiguity solving is investigated. Aiming at the existing ambiguity solving algorithms, which still have shortcomings such as low convergence speed and instability, the AGPSO algorithm is proposed to find the optimal solution of the integer ambiguity, which introduces the optimal half-taking, random crossover, and mutation operations to make the algorithm jump out of the local optimal solution more easily, and the improved fitness function to improve the convergence and search performance of the algorithm, and to improve the stability of the ambiguity solving results. Finally, experiments have been carried out using simulation data and measured data, as can be seen from the experimental results:


Goal: The modern way of living has significantly influenced the daily diet. The ever-increasing number of people with obesity, diabetes and cardiovascular diseases stresses the need to find tools that could help in the daily intake of the necessary nutrients. Methods: In this paper, we present an automated image-based dietary assessment system of Mediterranean food, based on: 1) an image dataset of Mediterranean foods, 2) on a pre-trained Convolutional Neural Network (CNN) for food image classification, and 3) on stereo vision techniques for the volume and nutrition estimation of the food. We use a pre-trained CNN in the Food-101 dataset to train a deep learning classification model employing our dataset Mediterranean Greek Food (MedGRFood). Based on the EfficientNet family of CNNs, we use the EfficientNetB2 both for the pre-trained model and its weights evaluation, as well as for classifying food images in the MedGRFood dataset. Next, we estimate the volume of the food, through 3D food reconstruction of two images taken by a smartphone camera. The proposed volume estimation subsystem uses stereo vision techniques and algorithms, and needs the input of two food images to reconstruct the point cloud of the food and to compute its quantity. Results: The classification accuracy where true class matches with the most probable class predicted by the model (Top-1 accuracy) is 83.8%, while the accuracy where true class matches with any one of the 5 most probable classes predicted by the model (Top-5 accuracy) is 97.6%, for the food classification subsystem. The food volume estimation subsystem achieves an overall mean absolute percentage error 10.5% for 148 different food dishes. Conclusions: The proposed automated image-based dietary assessment system provides the capability of continuous recording of health data in real time. I.A daily healthy diet and a daily intake of essential nutrients can significantly affect the modern lifestyle as they prevent malnutrition in all its forms, but also a wide range of nutrition-related diseases. Chronic noninfective diseases, such as obesity, diabetes and cardiovascular diseases may be affected either positively or negatively by an individual's diet. Diabetes mellitus is considered a group of metabolic disorders characterized by a high blood sugar level (glucose) and has become a common disease of the modern way of life Nowadays, with Artificial Intelligence (AI), the Internet of Things (IoT) and computer vision becoming omnipresent technologies, individuals are allowed to use food applications to monitor and record their daily diet The food image dataset is the key to creating a highly accurate model for a dietary assessment system. These datasets can be characterized by the number of food classes Image classification, which is a subset of AI and computer vision, is the process of defining and labeling groups of pixels or vectors within an image based on precise rules Food classification systems are responsible for the identification of the food class to which a food image belongs. Existing food image classification approaches can be divided into two main categories: (1) traditional machine learning approaches The final step in a food dietary assessment system comprises the estimation of foods’ quantity and the analysis of their nutritional composition, such as carbohydrates, proteins, fats and total calories. Accurate estimation of the amount of food, assumes that the previous step of food classification has been accomplished correctly. The existing approaches in food image volume estimation, can be divided into five main categories: stereo-based approaches In this study, we present a fully automated image-based dietary assessment system. We report the characteristics of the food image dataset and the methods used for classifying the content of food images and estimating the food volume. First, we present a new food image dataset representing the Mediterranean cuisine which enables the development and evaluation of both computational modelling subsystems. This is the first study in this field concerning Mediterranean foods and, in addition to this, the number of foods used to evaluate the food volume estimation system is greater than those reported in the relevant literature This study is a continuation of our recent work in this field The primary intended use of the proposed system lies in the context of digital-based dietary assessment interventions, e.g., as integrated into a dietary assessment application aiming at improving a person's eating habits or into an electronic health record system aiming at monitoring and prescribing the diet of patients. In addition, the output of the proposed system, i.e., macronutrients, can make up the input of predictive models of health outcomes affected by one individual's diet, e.g., blood glucose prediction models.II.The proposed dietary assessment system needs two food images to calculate the nutritional composition of the food. Initially, using a smartphone camera, the images are taken with a shooting angle of about 45 degrees from the vertical axis of food, and at a distance of about 40−50 cm. A reference card placed next to the dish helps to estimate the scale of the captured images. Then, the food classification subsystem, using deep learning techniques, recognizes the class of the food. At the same time, the food volume estimation subsystem, using stereo vision techniques creates a 3D reconstruction of the food and estimates its quantity. An appropriate food image database was created for the training and evaluation of the two subsystems. The image database consists of two sub-datasets: (a) an image dataset appropriate for deep learning classification models, and (b) an image dataset appropriate for food volume estimation systems based on stereo vision techniques. Finally, knowing the class of food and its volume, using the USDA database A.The Mediterranean diet is a well-known healthy dietary pattern which is based on the traditional cuisines of Greece, Italy, Spain and other countries that border the Mediterranean Sea. Plant-based foods, such as vegetables, legumes, fruits, nuts and seeds, are the foundation of the diet. Olive oil is the main source of added fat. Moreover, fish, seafood, dairy  and poultry are included in moderation while red meat and sweets are eaten only occasionally B.To create the food recognition model we used as base model an existing CNN classification model, that belongs to the EfficientNet 1)EfficientNet is a convolutional neural network architecture and scaling method that uniformly scales all dimensions of depth, width and resolution using a compound coefficient. In contrast to the conventional practice of arbitrarily scaling these factors, the EfficientNet scaling method uniformly scales network depth, width and resolution with a set of fixed scaling coefficients. The depth corresponds to the number of layers in the CNN. The width is related to the number of neurons in a layer, or more relevantly to the number of filters in a convolutional layer. The resolution is the height and width of the input image. Increasing the depth, by adding more convolutional layers, allows the network to learn more complicated features. However, deeper networks become difficult to train and require more computational power. Scaling the width of the CNNs allows layers to learn more detailed features. Higher image resolution provides a greater detail of image details, improving the model's ability to detect and analyze relatively smaller objects and export finer patterns 2)The architecture of food classification subsystem consists of seven CNN block layers based on EfficientNet-B2 deep learning model and three blocks of fully connected activation, batch normalization and drop-out layers. Each of the seven blocks have a varying number of sub-blocks. The total number of layers is 349 for the classification model. The architecture of the baseline network uses a mobile inverted bottleneck convolution similar to MobileNet V2 3)Deep learning algorithms, so far, have been designed to work individually. These algorithms are trained to solve specific tasks and must be rebuilt from scratch as soon as the distribution of feature-space changes. Transfer learning is the concept of overcoming the isolated learning paradigm and leveraging the knowledge gained for a task to solve related ones. Moreover, it is considered a popular method in machine learning because it allows the user to build accurate models in a timesaving way Fine-tuning involves unfreezing some layers of the CNN model for feature extraction and jointly training the newly added part of the model and these top layers. We use the fine-tuning technique, in which the first layers of the recognition model are frozen, transferring to these layers the weights of the pre-trained model mentioned above, while the last layers of model are unfreezing and re-trained on the new data with a very low learning rate. This can potentially achieve meaningful improvements by incrementally adapting the pre-trained features to the new data. Initially, in the proposed CNN model, we unfreeze the last two blocks of network layers. Then, we add three blocks of fully connected, activation and drop-out layers on the base EfficientNet-B2. Finally, the unfreezing layers and the additional layers are trained in MedGRFood dataset. These added layers further increase the performance of the proposed model and prevent overfitting problem.4)Data augmentation is based on techniques used to enhance the volume and the diversity of training data, by applying random but realistic transformations of existing data. Data augmentation acts as a regularizer, makes the model more robust to slight variations, helps to reduce overfitting and improves the performance when training a CNN model C.Initially, for the estimation of food volume and for its 3D reconstruction, some assumptions need to be made: i) the smartphone's camera is calibrated and we know the intrinsic camera matrix, ii) the food volume estimation subsystem needs two images of food to calculate its volume, iii) the two images are taken with a slight shifting of the camera to the right when capturing the second image, iv) the images are taken at a distance of about 40−50 cm from the dish, and with a shooting angle of about 45 degrees from the vertical axis of food, v) the type of dish used should be shallow, and vi) a reference card of known dimensions (8.5 × 5.5 cm) is placed next to the dish. Fig. 1)The first step of the proposed methodology is to improve the quality of the input images by applying the CLAHE histogram equalization, and at the same time the input images are resized to 600x800 pixels Fig. . Improvi2)inliers1[i] ↔ inliers2[i] and the fundamental matrix F from the corresponding inliers, it computes the corrected correspondences newInliers1[i] ↔ newInliers2[i] that minimize the geometric error 2+d2d  is the geometric distance between inliers a and b), subject to the epipolar constraint T*F*newInliers1 = 0newInliers2  is referred to the position and orientation of a camera in relation to the world coordinate system, known as the reference coordinate system. To achieve this, we first filter the inliers from the outliers matching points by calculating the homographic matrix using the RANSAC-based robust method ])2 Fig. , Refd be the real distance of reference card and Estd be its size as obtained by estimation in 3D, then the scale factor is computed using the ratio Ref/dEstd.In order to estimate the scale factor of the image, a reference card with known dimensions is placed next to the dish. To find the scale factor, we first detect the reference card in the image followed by its 3D reconstruction to obtain the corresponding 3D coordinates. For reference card detection we use a pattern of the card and we compute the homography between the food image and the pattern Fig. . Let dRe4)To estimate the dense depth map between the two images, first the image pair is rectified, so that we can search for corresponding pixels between them Fig. . Stereo 5)estV is the estimated volume of food, while USDAW and USDAV are the weight and volume of food, respectively, in the USDA food database. Finally, we use the Greek Food Composition Dataset to calculate macronutrients, such as carbohydrates, proteins, and fats through  for the food dishes, and have made five different estimates for each dish from the MedGRFood dataset. Equation  is used m we use . In totaThe overall MAPE of the proposed food volume estimation subsystem is 10.5%. Table IV.In this paper we presented an automated image-based dietary assessment system which consists of three major parts: (i) the MedGRFood image dataset, (ii) the food classification subsystem and (iii) the food volume estimation subsystem.The use of deep learning techniques to classify food images requires the development of datasets with a big amount of imaging data. However, the existing food image datasets are limited to the number of food classes, depending on the dataset constructor's eating habits and the type of cuisine one wants to cover. In addition, image quality has positive or negative effect on the performance of classification systems, so it is equally important that the food image dataset contains high-resolution images. High quality images allow the classification model to be trained with higher-resolution images, in which more details of the food are discernible, creating models having better accuracy. The MedGRFood image dataset is a first step towards the creation of a complete dataset of food images representing the Mediterranean cuisine and diet, according to the Greek Food Composition Dataset by the Hellenic Health Foundation. Expanding the dataset by adding images from missing foods classes and categories, such as fruits, is the next step in completing it, which will provide a comprehensive food dataset with high-resolution images available to the scientific community for research purposes.By using the EfficientNetB2 pre-trained in Food-101 image dataset as base classification model and by applying fine tuning, transfer learning and data augmentation techniques, we were able to achieve 83.8% top-1 accuracy and 97.6% top-5 accuracy in the MedGRFood food image dataset. By achieving better performance with the proposed model in the Food-101 when compared to the pre-trained models in the ImageNet, we proved the significancy of pre-trained classification model in a relevant dataset. Our methodology presupposes the identification of the appropriate dataset and the training of the base model in the Food-101 dataset. While most food classification models are either trained in a dataset from scratch or use a pre-trained model in the ImageNet dataset, we used a pre-trained model on a food dataset. Therefore, the proposed methodology for image classification, can also be used to classify images from different datasets. Knowing that basic image processing tasks (such as edge detection) are performed in the first layers of a CNN classification model, we used the pre-trained weights of a related dataset to significantly improve all classification indices. We chose the EfficientNet as base model because it achieves both higher accuracy and better efficiency compared to previous CNNs image classification models. Furthermore, the computational cost, the parameters size in addition to the time required to train the model based on EfficientNet are substantially smaller.Finally, we described and analyzed the procedure for estimating the volume of the food using a stereo vision approach. The proposed volume estimation subsystem presupposes two captured images to calculate the amount of food contained in a shallow dish. The volume estimation pipeline involves 3D reconstruction of food using stereo vision techniques, such as feature detection and extraction, feature matching, pose estimation, scale determination, stereo matching and point cloud generation. The overall MAPE of the proposed methodology is 10.5% for 148 different dishes from the MedGRFood image dataset. The dish with the lowest MAPE is the beefsteak while the dish with the highest MAPE is the fish roe dip, with MAPE values 5.1% and 21.3%, respectively.Comparing our proposed dietary assessment system to the related literature, we observe that we have created a unique Mediterranean image dataset suitable for both food image classification systems and food volume estimation systems. Moreover, the proposed food classification methodology gives a very good top-1 accuracy, an excellent top-5 accuracy, and a very low loss index. By counting the number of food classes we want to classify and by comparing the classification results with other food image classification models, we observe that our results are among the best in literature. Finally, the proposed volume estimation subsystem calculates the MAPE for the largest number of dishes in the relevant literature.The main advantage of the proposed methodology is that it is a fully automated process to calculate the nutritional composition capturing food images through a smartphone camera. This is what today is used and it is easy to be used by most of the people and of all ages to capture photos and more specifically food images, which provide the capability of continuous recording of health data in real time. The use of mobile devices and cloud technology to monitor health data and sharing it with physicians, can lead to faster and lower misdiagnosis of diseases, such as diabetes, obesity and CVDs. By creating a new food image dataset and using methods and techniques from computer vision and image processing, we offer a possible solution to the problem of calculating food nutrition through images. A limitation of the proposed methodology is that it estimates the nutritional composition of foods contained only in shallow dishes which must be improved to include different types of dishes .V.The Supplementary Materials includes food images of the MedGRFood dataset, learning rate schedule, higher resolution food point cloud reconstruction images and tables that contain the top-5 dishes with higher and lower MAPE index.


With the introduction of various loads and dispersed production units to the system in recent years, the significance of precise forecasting for short, long, and medium loads have already been recognized. It is important to analyze the power system’s performance in real-time and the appropriate response to changes in the electric load to make the best use of energy systems. Electric load forecasting for a long period in the time domain enables energy producers to increase grid stability, reduce equipment failures and production unit outages, and guarantee the dependability of electricity output. In this study, SqueezeNet is first used to obtain the required power demand forecast at the user end. The structure of the SqueezeNet is then enhanced using a customized version of the Sewing Training-Based Optimizer. A comparison between the results of the suggested method and those of some other published techniques is then implemented after the method has been applied to a typical case study with three different types of demands-short, long, and medium-term. A time window has been set up to collect the objective and input data from the customer at intervals of 20 min, allowing for highly effective neural network training. The results showed that the proposed method with 0.48, 0.49, and 0.53 MSE for Forecasting the short-term, medium-term, and long-term electricity provided the best results with the highest accuracy. The outcomes show that employing the suggested technique is a viable option for energy consumption forecasting. On the other hand, due to the non-renewability of fossil fuel sources, especially oil and gas sources, it has been several decades that the issue of replacing these types of energy and saving and optimal consumption of energy has been seriously considered in the economy of advanced countries, and very effective measures have been taken to optimize energy consumption . As a reWith the establishment of the electricity market in the power grid, electricity companies on the one hand, and distribution companies, which are considered energy buyers, on the other hand, need to be able to predict and present the hour-by-hour consumption of the entire network covered by them in the coming days. In addition, power plant production and operation management companies may also need load forecasting to forecast and offer prices . In a deShort-term and medium-term load forecasting is generally done for periodic planning and is not very complicated . But lonThe literature shows that short-term load forecasting has attracted more attention than medium-term load forecasting . Of courRecently, there has been a significant shift away from purely theoretical research and toward applied research to tackle issues that have no solution or cannot be handled easily . Given tIn the last decade, different analytical or numerical techniques have been proposed in the field of forecasting science, in such a way that instead of presenting a mathematical model, these methods try to use a free model obtained from a series of experimental and practical data . These t1.2With the increasing use of various loads and dispersed production units in the power system, the importance of accurate load forecasting has been widely recognized. It is crucial to analyze the system’s performance in real-time and respond appropriately to changes in the electric load to maximize the energy system’s efficiency. Long-term electric load forecasting enables energy producers to increase grid stability, reduce equipment failures and production unit outages, and ensure the reliability of electricity output.1.3Several methods have been proposed for electric load forecasting, including statistical models, machine learning techniques, and deep learning algorithms. However, most of these methods have limitations in terms of accuracy, computational efficiency, and scalability. For example, Romera et al.  studied Huang et al.  presenteChen et al.  presenteGhanbari et al.  suggesteAhmad et al.  examinedThe limitations of the research were not explicitly stated, but possible limitations could include the generalizability of the findings to other energy demand forecasting scenarios, as the study focused specifically on predicting the energy demand of water source heat pumps. Additionally, the study may have relied on assumptions and simplifications that may not accurately reflect the complex and dynamic nature of energy demand, and it was unclear whether the machine learning-based models used in the study can be easily implemented in practical settings. The study also did not provide a detailed analysis of the limitations of the different machine learning models used, and it was unclear whether the findings can be replicated using different models or datasets.Su et al.  investigThe limitations of the research were not explicitly stated, but possible limitations could include the generalizability of the findings to other natural gas demand forecasting scenarios and the applicability of the proposed hybrid gas consumption approach to other types of energy forecasting. Additionally, the study may have relied on assumptions and simplifications that may not accurately reflect the complex and dynamic nature of energy demand, and it is unclear whether the deep-RNN model and wavelet transform-based approach used in the study can be easily implemented in practical settings. The study also did not provide a detailed analysis of the limitations of the different components of the proposed hybrid approach, and it was unclear whether the findings can be replicated using different models or datasets.Ruiz et al.  investigThe limitations of the research work were not explicitly stated, but possible limitations could include the generalizability of the findings to other types of buildings or energy forecasting scenarios, as the study focused specifically on public buildings. Additionally, the study may have relied on assumptions and simplifications that may not accurately reflect the complex and dynamic nature of energy demand in buildings, and it was unclear whether the Elman neural network-based approach used in the study can be easily implemented in practical settings. The study also did not provide a detailed analysis of the limitations of the genetic algorithm used to improve the weight of the models, and it is unclear whether the findings can be replicated using different models or datasets.In the literature, many approaches for predicting energy consumption were proposed. It is a randomness-based model for picking either the user data in the modeling or the NN training. When it comes to predicting energy use, researchers have routinely used artificial neural networks. When a neural network cannot execute on its own, optimization strategies such as metaheuristic algorithms must be utilized to improve training outcomes. When compared to a regular neural network, the optimized neural network model produces better results.1.4Electric load forecasting poses several challenges, including uncertainty in load demand, variability in renewable energy generation, and the difficulty of modeling complex energy systems accurately. The motivation behind this study is to enhance the accuracy of long-term electric load forecasting by developing a new method that can overcome these challenges.1.5−The study uses SqueezeNet to forecast power demand at the user end.−The structure of SqueezeNet is enhanced using a customized version of the Sewing Training-Based Optimizer.−The proposed method is compared with other published techniques using a typical case study with short, long, and medium-term demands.−Objective and input data are collected at intervals of 20 min.The recommended approach involves modifying the weights throughout a neural network training session to get a faster convergence rate with excellent accuracy. Because of the aforementioned difficulties, we are motivated to use NN-based optimization for demand-side management in the smart network. Highly adaptable new electricity transmission networks must be constructed to satisfy current needs and enhance consumer profitability. To avoid abuse of the energy system by destroying electrical equipment, the model must estimate client demand. It must also be suitable for the design and construction of new power generation facilities. Therefore, the main contributions of this study can be highlighted by the following:1.6The study first uses SqueezeNet to obtain power demand forecasts at the user end. SqueezeNet is a deep learning architecture that has been previously used for image classification tasks. The structure of SqueezeNet is then enhanced using a customized version of the Sewing Training-Based Optimizer. This optimizer has been previously used in various deep-learning tasks and has shown improved performance compared to standard SGD.Next, the proposed method is compared with other published techniques using a typical case study with short, long, and medium-term demands. The previous research studies that are used for comparison are cited in the paper.To collect objective and input data from customers, a time window is set up at intervals of 20 min, allowing for highly effective neural network training. The authors provide a detailed description of the data collection process in the paper.Finally, the proposed method is evaluated and the results are presented, showing that it achieved the best results with the highest accuracy, with 0.48, 0.49, and 0.53 MSE for short-term, medium-term, and long-term electricity forecasting.In conclusion, the authors have presented a detailed description of the methods used in the study and have cited relevant previous research studies where necessary, which would help readers replicate their results.22.1The introduction and mathematical simulation of the optimization algorithm based on sewing training (STBO) are presented below:2.1.1The process of training novice tailors by the tailoring trainer is the central idea and source of motivation for the STBO Algorithm. First, a novice selects a sewing trainer to enhance her sewing abilities. Then sewing strategies are taught by the trainer. In the second stage, the trainer’s abilities are imitated by novice tailors and they attempt to increase their abilities to the level of their trainer. In the last stage, novice tailors increase their abilities by doing exercises. The process described as sewing training is an intelligent process that by mathematical simulation of this process, the suggested optimizer is inspired.2.1.2Novice tailors and tailor trainers are individuals who constitute the population of the suggested STBO algorithm. The value of the decision variables is suggested through any individuals of the algorithm which indicates a candidate solution to the problem. Consequently, a vector and a matrix can express any individual and population of the STBO algorithm mathematically. The following equation represents the population of STBO as a matrix  Eqs. (1)lgorithm :(2)zi,j=Any STBO individual illustrates a candidate solution to the determined issue. Hence, the fitness function of the problem can be assessed based on the defined values through any candidate solution. The computed values for the fitness function are simulated based on the placement of candidate solutions in the problem parameters as follows:Stage 1: Instruction (exploration). The first stage of renewing STBO individuals is based on modeling the approach of choosing a sewing trainer and attaining sewing ability by novice tailors.For any STBO individual as a novice tailor, whole different individuals with a more suitable value for the fitness function are assumed sewing trainers for that individual. The grouping of whole candidate individuals as the set of probable sewing trainers for any STBO individual  follows :(4)CSIi=Eventually, if this novel location enhanced the fitness function value, it changes that individual’s former location. The following equation represents the condition of updating:s− an assortment of the Stage 2: Emulation of the trainer’s abilities (exploration). The second stage of renewing STBO individuals is based on modeling novice tailors attempting to imitate the abilities of trainers. It is assumed that novice tailors want to increase their ability as much as possible and bring it to the level of a trainer. According to any optimization individual is a vector of the dimension m and any component illustrates a decision variable therefore, in this stage of the optimization, it is presumed that any decision variable indicates a sewing ability. Any member of STB optimization emulate The novel location for any STBO individual is computed based on the modeling of emulating these trainer abilities, utilizing the following formula :(8)zi,jpStage 3: Assiduity in exercise (exploitation). The third stage of renewing STBO individuals is based on modeling novice tailoring exercises for enhancing sewing abilities. In this stage of the optimization process, a local search is executed near candidate individuals to discover the finest probable solutions around these candidate individuals. This stage of the STBO illustrates the exploitation sufficiency of the suggested algorithm in local search. For the mathematical expression of this step, a novel location near any individual of the STBO is created through the following equation.2.1.3By renewing all candidate solutions based on the first to third stages, the first iteration in the optimization process is performed. After that, the renewing approach is continued up to the final iteration of the algorithm, according to Eqs. 2.2Despite having several advantages for solving optimization problems, the Sewing Training-Based Optimization Algorithm occasionally runs into problems. Every algorithm encounters a situation like this on occasion. In some challenging problems, this algorithm has the benefit of being able to escape the local minimum. Additionally, it can occasionally have a weak and raw convergence speed. Different approaches are used to solve this problem. Utilizing the benefits of another algorithm in addition to the current work is a useful method for this purpose. To produce even better results, the strengths of the Sewing Training-Based Optimization Algorithm and the particle swarm optimization (PSO) algorithm are combined in this study. Following the PSO algorithm , each in follows :(13)Zi,jAccordingly, the final updating equation can be rewritten by the following equation:2.3sewing training-based optimization algorithm should be fairly analyzed to determine the effectiveness of the methods by applying them to some common benchmark functions and comparing the results with those obtained using other state-of-the-art methods [The proposed Combined  methods . In thisTo demonstrate the method’s effectiveness, it should be compared to some cutting-edge techniques. In this case, three metaheuristic algorithms, including Wildebeest herd optimization (WHO) , Archime33.1Due to their high effectiveness, convolutional neural networks (CNNs), as a popular deep learning technique, are now widely used in a variety of applications. For the artificial processing and forward classification of multidimensional data, such as images, convolutional neural networks were developed . To bettSimilar to other artificial neural networks, convolutional neural networks are composed of neurons that store biases and weights and contain a decision-making process in the following layers . The hidConvolutional neural networks have a lot of parameters, which rises memory usage. SqueezeNet is an architecture that is identical to the AlexNet architecture but has fifty times fewer parameters, making it ideal for gadgets like mobile phones. An activation, convolution, and pooling function are all combined to form a convolution layer.Global average pooling is a technique used in the SqueezeNet in place of their customary fully connected layer. In contrast to networks that add a fully connected layer for classification, this method creates a feature map for each class after the final mlpconv layer rather than adding a full layer as is done in networks that use the fully connected layer. They are interconnected on top of feature maps, gaining from them, and the results of feature vector analysis are fed directly into Softmax. Due to the lack of optimization parameters, one advantage of this layer is that it does not cause more trouble than it solves. Alternatively, the Softmax is extra resilient to local variations and more in line with the concept of convolution networks.The SqueezeNet is composed of various fire modules, each of which has a squeeze convolution layer and an expansion layer. The SqueezeNet Fire Module is illustrated in The squeeze convolution layer’s findings are passed on to the following expand layer in the fire modules, as seen in 3.2CSTBO algorithm are outlined below.As mentioned before, when training SqueezeNet, the CSTBO algorithm is employed. This method can offer and improve the potential resolution to the specified optimization problems. The phases of the It is necessary to initialize the algorithmic restrictions and solutions before using them to generate the optimum solution. The stated aim function is to identify the MSE’s ideal solution, such that :(15)Fit=The goal of this idea is to minimize the fitness function using the CSTBO algorithm, in other words:4The data used in the simulation was taken from the 2022 report “Uk Energy In Brief 2022” . ConfirmBecause it better satisfies the SqueezeNet model, a day-in-advance prediction with a 30-min precision is employed. Input, goal parameters, and invisible layer neurons are assigned before the experiment. Input variables include day, month, and year. By identifying and employing a hidden layer, the experiment aims to improve the accuracy and correctness of network training. The variable about power usage serves as a goal variable for the duration of the experiment.The findings are divided into three categories of daily energy consumption pattern, consumer short-term, and long-term forecasting.4.1It is evident from Summer vacations show a trend in energy consumption that is a little lower than working days. The prolonged sunny day also contributes to the negligibly good results that green power systems produce.4.2Findings of 55.1Experiments are performed by Matlab R2018b on a laptop with a core i7-6700HQ Asus N552 processor and 12 GB RAM. It is assumed that there will be a maximum of 200 iterations and that there will be 60 individuals. Energy forecast performance is enhanced by using the suggested SqueezeNet method. To meet consumer energy demand and to better understand how customers use energy, the predicted electricity under the proposed SqueezeNet method is used. The offered optimized SqueezeNet/CSTBO method is compared to some other techniques in the literature, such as the LSTM network , Elman n5.2The study has made predictions about energy demand and compared them with the actual energy demand. Assuming that the authors' results provide the best results means that the authors have used the most accurate and reliable methods for predicting energy demand and have obtained the most accurate and reliable data for actual energy demand. Therefore, the results presented in 5.3The expected and actual monthly kilowatt usage for energy is shown in The statement is describing the results obtained from an experiment or study related to energy consumption forecasting using the SqueezeNet/CSTBO technique. The results are referring to 5.4The projected and actual yearly kilowatt usage is shown in 5.5For both short- and long-term power forecasting, the suggested optimization approach’s convergence values were compared to those of the other second-best method, i.e., FNGB. The results are shown in The research mentioned above for the example under discussion demonstrates that using the proposed SqueezeNet/CSTBO method results in cost values for short-term energy prediction that are significantly lower than those obtained using the FNGB. So, it could be said that the suggested SqueezeNet/CSTBO and FNGB algorithms result in better convergence outcomes for both short-term power forecasting and long-term energy predicting.As a result, the suggested SqueezeNet/CSTBO may be the main consideration for processes of medium-term power prediction and long-term energy prediction. As a result, the suggested SqueezeNet/CSTBO and FNGB algorithms for altering the NN method weight worked well and produced better results for demand forecasting.According to the results, the SqueezeNet/CSTBO procedure has yielded the best results when compared to other prediction methods that have been evaluated, and the results are truly satisfying. The evaluation has been performed based on the Mean Squared Error (MSE) metric, which is a commonly used metric in regression problems. MSE measures the average squared difference between the predicted and actual values. The lower the value of MSE, the better the model’s performance. The results show that the SqueezeNet/CSTBO procedure has the lowest MSE across all prediction types. This implies that the model is performing well in predicting various types of data. The statement suggests that the results are truly satisfying, indicating that the model has performed exceptionally well.6−The proposed technique was demonstrated as a viable option for energy consumption forecasting−The study showed that the proposed SqueezeNet/CSTBO method outperformed other published techniques for short-term load forecasting−The proposed SqueezeNet/CSTBO method had a competitive performance for medium-term and long-term load forecasting compared to other published techniques.Accurate forecasting of the consumption curve for network load management and dispatching has become crucial for distribution businesses with the emergence of smart networks. Distribution firms face significant difficulty in the development and selection of time series models to select the best and most useful model for load forecasting. The objective of this article was to develop an optimal load forecasting model utilizing machine learning on short, long, and medium loads an optimized version of SqueezeNet based on an improved metaheuristic, called a customized version of the Sewing Training-Based Optimizer (CSTBO) with the forecasting models' parameters for load forecasting based on distribution network information. Here, the proposed SqueezeNet/CSTBO method and those of some other published techniques were then implemented after the method to a case study with three different types of demands-short, long, and medium-term. The results demonstrated that employing the suggested technique was a viable option for energy consumption forecasting. In conclusion, the proposed method offers several advantages over existing techniques in electric load forecasting. Its high accuracy, ability to handle different types of demands, and real-time analysis make it a viable option for energy consumption forecasting. However, the potential for data bias and the need for real-time data should be taken into consideration when implementing the method in practice. The main results of this study are as follows:The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper


To address the problem of accurate recognition and localization of multiple clusters of green persimmons with similar color to the background under natural environment, this study proposes a multi-cluster green persimmon identification method based on improved Faster RCNN was proposed by using the self-built green persimmon dataset. The feature extractor DetNet is used as the backbone feature extraction network, and the model detection attention is focused on the target object itself by adding the weighted ECA channel attention mechanism to the three effective feature layers in the backbone, and the detection accuracy of the algorithm is improved. By maximizing the pooling of the lower layer features with the added attention mechanism, the high and low dimensions and magnitudes are made the same. The processed feature layers are combined with multi-scale features using a serial layer-hopping connection structure to enhance the robustness of feature information, effectively copes with the problem of target detection of objects with obscured near scenery in complex environments and accelerates the detection speed through feature complementarity between different feature layers. In this study, the K-means clustering algorithm is used to group and anchor the bounding boxes so that they converge to the actual bounding boxes, The average mean accuracy (mAP) of the improved Faster RCNN model reaches 98.4%, which was 11.8% higher than that of traditional Faster RCNN model, which also increases the accuracy of object detection during regression prediction. and the average detection time of a single image is improved by 0.54s. The algorithm is significantly improved in terms of accuracy and speed, which provides a basis for green fruit growth state monitoring and intelligent yield estimation in real scenarios. Digitalization, information technology, and intelligent management technology of orchards can collect fruit growth information in real time, quickly, and objectively, which will serve as reference and a basis for the scale of orchard production . China iArtificial intelligence technology has advanced the development of intelligent and informative agriculture . ConvoluAlthough the above research can identify fruits, there are still some drawbacks including a high computational cost, a poor recognition accuracy, a single environmental factor, and a lack of research on multiple clusters of green fruits in complex backgrounds. At present, convolutional neural networks have more mature applications not only in handwritten character recognition  and vehi22.1In this study, the Faster RCNN model for multi-cluster green persimmon detection is developed based on the PyTorch framework. Faster RCNN is a typical representative of two-stage detection models . It is aThe RPN is a lightweight region generation module that produces a better “Proposal” of the proposed frame. It uses a sliding window to slide over the feature map generated by the pre-trained network model, producing a one-dimensional vector for each sliding position, which is then convolved with a 3 × 3 convolution kernel to further extract the frame features. The multi-task binary classification loss function of RPN is represented as follows:where the classification loss function is:ip being the probability that the i-th anchor is predicted to be the target; ip is 1 for positive samples and 0 for negative samples. The bounding box regression loss function is:with it is the regression parameter of the bounding box of the i-th anchor and i-th anchor.where The Fast RCNN algorithm uses the selective search (ss) algorithm to generate 1k-2k candidate regions in an image, and the entire image is then input into the CNN to obtain the corresponding feature maps. The ss algorithm projects the candidate regions generated on the original image onto the feature map to obtain the corresponding feature matrices, and each feature matrix is finally scaled to a uniform size by ROI Pooling layer, followed by spreading the feature map through a series of fully connected layers to predict the class to which the target belongs and the bounding box regression parameters. The loss function of Fast RCNN can be expressed as:p is the SoftMax probability distribution predicted by the classifier, u the real class label of the target, ut represents the regression parameters of the corresponding class u predicted by the bounding box regressor, and v represents the bounding box regression parameters of the real target.where In this study, the joint training method of RPN Loss and Fast RCNN Loss is used to train the network by backpropagating the losses of these two parts. The architecture of the Faster RCNN network is shown in 2.2Target detection requires not only identifying the class of objects, but also spatially locating their bounding box. Most conventional backbone networks use classical convolutional neural network models, such as ResNet , AlexNetk in order to determine the coverage of local cross-channel interactions. The channel-attentive feature map 1×1×C is then combined with the original input feature map H×W×C by channel-by-channel multiplication for channel attention, and the feature map with channel attention is output. This attention mechanism significantly reduces the model’s complexity while performance is maintained. To avoid cross-validation, the value of k needs to be optimized. This value can be adaptively determined as a function of the total number of channels C:In their natural environment, green persimmons often experience fruit overlap issues as well as branch and leaf occlusion, which reduces the model detection accuracy. In this study, the channel attention module (ECA) is introduced in the DetNet network structure to enhance the extracted feature representation capability and high resolution of the feature map, as shown in C is the total number of channels, and |χ|odd represents the nearest odd number.where κ is the number of neighbors per channel, The feature utilization of the three convolutional blocks of shallow (DetNet-2), middle (DetNet-4) and deep (DetNet-6) layers is enhanced by the ECA mechanism to focus on the relevant information corresponding to the specific dimension extracted in the channel during the three attention calculations. The attention distribution is first computed over all input information. To select the color, texture, and semantic feature information corresponding to each of the three attention mechanisms from several different feature dimensions, one type of relevant feature information is introduced for each, and the correlation between the input image feature dimensions and the relatively introduced feature information is calculated by the scoring function:a is the attention distribution, h is the feature dimension of the input image, q represents the specific feature information parameters, and:where s is the scoring function, w,u, and v– are learnable feature network parameters, and r represents channel location parameters.where The weighted average of the input information is then computed based on the attention distribution. The soft attention mechanism is used to obtain the results after the query, and the weighted average of the feature dimension information across multiple channels in the image is obtained for each channel of the feature map to determine the Attention value:C is the total number of channels and r is the channel position parameter.where After three weightings, the channel attention mechanism is used to enhance the target attention faster and directly, which improves the dense target detection as well as the anti-background interference ability. To improve the recognition accuracy of multiple clusters of green persimmons in natural environments, high weights are assigned to increase the attention to detail information such as texture and edges of green persimmons, as well as to overlapping and occluded targets that are more difficult to recognize. whereas, low weights are assigned to suppress useless information.2.3The backbone in Faster-RCNN is used to extract the target features through layer-by-layer abstraction, and the RPN performs feature extraction on the last layer of convolutional response obtained by the underlying CNN. Convolutional neural networks have higher resolution in the process of feature extraction with shallow layer features and more accurate target locations. In this study, the shallow layer tends to extract persimmon color features, and with the increase in convolutional layers, further features such as persimmon edge shape and texture are extracted. The deep layer features have stronger semantic information after multiple convolution operations and can more effectively convey the image information. This study extracts more abstract features in the green persimmon image in the deep layer and ignores the more detailed information in the image. The results of feature visualization in DetNet with different convolutional layers are shown in The amount of feature information that can be gleaned from an image is constrained when the target is set against a complex background that is comparable to it and is affected by various lighting conditions. Feature extraction from only one layer cannot adequately detect and localize the target object. Therefore, this study uses a serial layer-hopping connection structure to combine features from several abstraction levels, and then trains a predictor on the combined features to achieve DetNet-based multi-scale feature fusion. To use multiscale features generated by multiple scale network layers, the dimensionality (feature map size and number of channels) and the magnitude must be considered. In this study, the shallow DetNet-2 layer’s features are maximally pooled to make them consistent with the scale of the middle and deep layer feature maps. The maximum pooling formula for its output feature map is:where, kernel_size is the max pooling window size, stride is the step size for moving the max pooling window, padding is the number of layers of zeros appended to the input, and dilation is a parameter that controls the step of elements in the window.Maximum pooling can obtain local information and better preserve the features on the texture while reducing the feature map size. The size of the feature map output from DetNet-2 is transformed to 14 × 14 with 256 channels by maximum pooling. The three feature vectors with 256 channels of the feature map are then fused early to form the candidate box region features to detect green persimmon in complex environments. The improved Faster RCNN multi-cluster green persimmon recognition algorithm is shown in 2.4a priori frame sizes to improve the overlap rate between the predicted bounding boxes and the actual targets, thereby improving the algorithm’s detection performance and hastening its convergence.The anchor frame scheme in Faster RCNN is established for the PASCALVOC 2012 dataset, and the aspect ratio of anchor frames includes three ratios of 1:2, 1:1 and 2:1 and three sizes, so each object has 9 fixed ratios of anchor frames, and the default anchor frame ratios cannot be accurately identified due to the different aspect ratios of different datasets. Therefore, this work uses the K-means clustering algorithm to determine the anchor frame generation scheme, and groups the sizes of the real frames in the training samples to determine the most appropriate Anchor box generation method: First, nine samples are randomly selected as the initial cluster centers, and the Euclidean distance is computed between each sample and the center of each cluster. Each sample is then assigned to the cluster center closest to it and the cluster centers are updated. The Euclidean distance is calculated as follows:iX is the number of samples in the dataset and kC is the set of initial cluster centers.where, The mean value of all samples in each cluster is then calculated as the new cluster center until the cluster center no longer changes or the cluster center changes very little to satisfy a given termination condition. The anchor frames suitable for the dataset are regenerated by clustering, and the proportions and size of the nine anchor frames are obtained by training. The generated clustering results are shown in 33.1Persimmon images were collected at the National Persimmon Germplasm Resource Nursery in Hefei, Anhui Province, China, from July 10 to August 31, 2022. The experimental sample database consists of 9300 images collected under natural light, with different light intensities at multiple angles and distances for various scenarios, including single fruit, multiple fruit, branch leaf shading, and multiple clusters of fruit overlapping, with an image resolution of 4608 × 3456. The shooting distance is within the range of 10cm~50cm, the number of persimmon fruits included in a single picture is about 2~12, and the number of persimmon fruits included in a single picture is about 12~40 when the shooting distance is greater than 50cm.The quality of the dataset determines the quality of model training and the accuracy of predictions. In the early process of identification and detection of green persimmons, the factor of fruit occlusion seriously affects the accuracy of model recognition, and the data set is divided into three levels according to the severity of persimmon fruit occlusion, which improves the complexity of the dataset and the generalization ability of the model, so as to facilitate the detailed and accurate test results, which is more conducive to the analysis and solution of subsequent problems. As shown in 3.2The specific operating environment parameters of this study are shown in 3.3In this study, evaluation metrics commonly used for target detection models, average precision (AP), and mean average precision (mAP) are used to evaluate the model’s performance. The larger the value, the better the performance of the modelis.Precision (P) is the ratio of the number of correct objects detected to the number of correct objects in the sample, which measures the accuracy of the model detection. Recall (R) is the ratio of the number of correct objects detected to the number of objects in the sample, which measures the percentage of positive samples obtained by the model in the prediction process. Precision and recall can be respectively expressed as:where TP is the number of correctly identified positive samples, FP is the number of incorrectly identified positive samples, and FN is the number of missed positive samples.AP refers to the area of the P-R curve plotted by P and R. The area under the curve is the average of all accuracies in different recall values, which measures the accuracy of the model in the defined categories:i is the number of thresholds j represents the categories.where mAP refers to the mean value of AP for each category, which measures the accuracy of the trained model on all categories:n represents the total number of categories and j represents the class.Where 3.4To test the accuracy and efficiency of the multi-cluster green persimmon recognition and detection in complex natural environments based on the improved Faster RCNN developed in this study, backbone network performance optimization experiments and improved Faster RCNN performance detection experiments were conducted.3.4.1The feature extraction network model is the core part of Faster RCNN, and the good or bad features extracted by the convolutional neural network affect the quality of subsequent training. This paper conducts performance comparison tests between the improved ECA-DetNet network and three classification networks, namely ResNet, AlexNet, and VGGl6, to evaluate the effectiveness of the proposed backbone network improvement.The same image dataset was used for target recognition and detection experiments. In this study, persimmon fruit recognition is a binary object detection model, so persimmon fruit samples are set as positive samples and the rest of the objects are negative samples. To evaluate the overall effectiveness of the examined classifier, a P-R plot is used to highlight the tradeoff between accuracy and recall.3.4.2To evaluate the effectiveness of the improved Faster RCNN method as proposed in this study, its performance is contrasted with the three networks of Faster RCNN, SSD, and yolov3-spp for the identification of green multi-cluster persimmons under different shading degrees, lighting conditions, and shooting intervals.The experiments replace the feature extraction network of Faster RCNN with ECA-added DetNet and perform multi-scale feature fusion as the final candidate frame region features, while using k-means clustering to determine the anchor frame size. The test set is a randomly selected corresponding image from the entire test set samples, and the test is divided into different occlusion degrees, different illumination, and different shooting intervals as variables.Target detection with different occlusion degrees: under the condition of controlling the lighting and shooting interval, three groups of heavy, medium, and light occlusion were set up and respectively detected by the three algorithms. Each group was repeated three times, the precision rate, recall rate, and detection time were recorded, and the average of the three trials was recorded as the valid value. The outcomes of the four algorithms are compared to verify if the algorithm improvement proposed in this study can accurately identify the targets under different occlusion situations.Target detection test under different light conditions: in the complex natural environment of the actual collection, the accuracy of fruit recognition might be affected by issues like exposure on sunny days or low light on cloudy days. The test was set up in three groups: sunny day smooth light, sunny day backlight, and cloudy day backlight. Each group was divided into three types of photos with heavy, medium, and light shading, and the four groups of twelve types of photos were detected by four algorithms. Each group was repeated three times, the average precision (AP) and the mean average precision (mAP) were recorded, and the average of the three times was taken as the valid value. The results of the three algorithms are compared to verify whether the improved algorithm proposed in this study can accurately identify the targets under different lighting conditions.Target detection test with different shooting intervals: the accuracy of the green persimmon detection model is tested by randomly alternating recognition at different shooting distances, such as near, medium, and far distances. The test is divided into three groups: near, medium, and far distances, with the shooting interval of 10cm~20cm set as near distance, 30cm~50cm as medium distance, and more than 50cm as far distance. Each group was divided into three types of photos: heavy, moderate, and light obscuration. Comparing the results of the four detection algorithms, it has been tested that the improvement of the algorithm used in this study can improve the detection performance of small distant targets.44.1The P-R plots and detection times of the test set image recognition using four different backbone networks are shown in As can be seen from 4.24.2.1Model training and testing were performed under the same conditions using AP with different occlusion degrees, mAP of combined samples and single image detection time as evaluation metrics. As shown in 4.2.2As can be seen in As can be seen from The test set samples of sunny and cloudy sky with smooth light and backlight conditions were mixed to increase the robustness of the model, and evaluated with two evaluation indices, AP and mAP, and the results are shown in 4.2.3According to the testing of the dataset with different occlusion levels at 3 shooting distances, the detection pairs derived from the three algorithms are shown in As can be seen in 4.3It is difficult to detect the target object by the color and outline of the target alone when the target is illuminated by different lights. Based on this issue in different light detection tests, the three target detection algorithms were compared under paralight conditions, and it can be seen in After the improved algorithm combines intuitive and abstract features, the multi-scale feature fusion can detect the target object more accurately. Faced with target objects at different shooting distances, the target object to be detected is large for close range shooting and small for long range shooting. Therefore, we must also take into account the accuracy of small target identification in addition to the issue of the target being obscured. The improved algorithm in Early fusion is used in the Faster RCNN algorithm to enhance the utilization of information from different feature layers, i.e., multi-layer features are fused first, and the predictor is trained on the fused features afterwards. This approach can effectively cope with the problems faced by target detection of obscured objects in complex environments with near views. After improving the algorithm, the computational effort brought on by different dimensions is solved in advance in the multi-scale feature fusion so that the fused feature layers have the same number of channels and are directly connected afterwards to improve the detection accuracy by the complementarity of features in different layers. The anchor frame size is re-determined using k-means clustering method to make it closer to the actual edge of the target. The accuracy of target detection is likewise increased in regression prediction, and the enhanced algorithm in this study improves detection accuracy while also ensuring detection speed. It is expected to be applied to persimmon fruit growth information monitoring in the future, and also provides technical support for subsequent automated harvesting.5In this paper, we design a green persimmon recognition model with good robustness and accuracy against background interference, fruit occlusion, and other factors by improving the Faster RCNN algorithm, which resolves issues like difficulty in recognizing near scenery occlusion in complex environment. By replacing the DetNet backbone feature extraction network, adding ECA attention mechanism, and adjusting and optimizing the model’s structure and parameters, the feature layers of different depths are combined with multi-scale features to improve the performance of algorithm retrieval robustness, speed, and accuracy.Through backbone network performance optimization experiments, the ECA-DetNet algorithm refined in this study has been significantly improved in terms of speed, accuracy, and robustness, which proves the effectiveness of this study on algorithm optimization. The improved Faster RCNN performance detection test is compared with the conventional Faster RCNN, SSD, and yolov3-spp target algorithms from multiple perspectives of different occlusion degree, different illumination, and different shooting intervals. The experimental findings demonstrate that the improved Faster RCNN has significantly better detection speed and accuracy compared to other algorithms for heavily occluded targets, targets with backlight conditions on cloudy days, and small targets at long distances. The effectiveness of the algorithm optimization in this study is demonstrated, and the mAP value of the improved Faster RCNN algorithm has improved to 11.8 percentage points compared to the Faster R-CNN. The near-view, complex environment interference resistant multi-cluster green persimmon detection and identification algorithm in this study can rapidly identify immature green persimmons in natural environments and enable growth state monitoring, which provides effective technical support for the development of smart orchards. The method proposed in this study can be applied to the detection of green fruits in unstructured orchard environment, it can be applied to the detection of green fruits in unstructured orchard environment, not only limited to the identification research of persimmons, but also applied to the information monitoring of different green growth periods of fruits, and at the same time provides theoretical and technical support for the subsequent visual system of automatic inspection and picking mechanization of orchard operations, and provides effective technical support for the intelligent and digital development of smart orchards.The datasets presented in this article are not readily available because privately owned. Requests to access the datasets should be directed to m17634935951@163.com.LY: conceptualization, methodology, supervision, funding acquisition, and project administration. RH: methodology, software, validation, investigation, and writing - original draft. ZZ: investigation, formal analysis, and writing - review & editing. MF: supervision, visualization, and project administration. ZP: data curation. WL: validation. FR: resources. All authors contributed to the article and approved the submitted version.