With the development of artificial intelligence technology, the behavior decision-making of an intelligent smart marine autonomous surface ship (SMASS) has become particularly important. This research proposed local path planning and a behavior decision-making approach based on improved Proximal Policy Optimization (PPO), which could drive an unmanned SMASS to the target without requiring any human experiences. In addition, a generalized advantage estimation was added to the loss function of the PPO algorithm, which allowed baselines in PPO algorithms to be self-adjusted. At first, the SMASS was modeled with the Nomoto model in a simulation waterway. Then, distances, obstacles, and prohibited areas were regularized as rewards or punishments, which were used to judge the performance and manipulation decisions of the vessel Subsequently, improved PPO was introduced to learn the action–reward model, and the neural network model after training was used to manipulate the SMASS’s movement. To achieve higher reward values, the SMASS could find an appropriate path or navigation strategy by itself. After a sufficient number of rounds of training, a convincing path and manipulation strategies would likely be produced. Compared with the proposed approach of the existing methods, this approach is more effective in self-learning and continuous optimization and thus closer to human manipulation. Since the 1970s, the combination of robot technologies and vehicles has led to the emergence of drones, unmanned vehicles, and unmanned ships  or . Two ships should alter course port through passing the starboard side of another ship. The head-on situation is displayed in (2) OvertakingThe overtaking situation means that the speed of the rear ship is greater than that of the front ship. When the own ship chases the target ship in a certain direction 22.5 degrees behind the target ship, the target ship is a stand-on ship, and the own ship should give way to the target ship. The overtaking situation is displayed in (3) Crossing give-wayWhen two ships meet and there is a risk of collision, the relative position of the target ship and the own ship is in . In this case, the own ship should give way to the target ship. According to COLREGs, the own ship should alter her course to starboard to avoid a collision. The crossing give-way situation is displayed in (4) Crossing stand-onWhen two ships meet and the relative position of the target ship and the own ship is in , there is a risk of collision. In this case, the ship is stand-on, and the target ship should give way to the own ship. If the target ship does not take avoidance action timely, the own ship should take action to avoid the collision. The crossing stand-on situation is displayed in At present, artificial intelligence technologies have developed rapidly; especially after AlphaGo defeated Lee Sedol, the nine-stage chess player, reinforcement learning has risen rapidly to provide new possibilities for intelligent ship path planning. The Q-learning algorithm could obtain the best behavior decision-making through the optimal action-value function . HoweverThe development of deep reinforcement learning is greatly accelerated by neural networks . With thReinforcement learning based on value function is represented by Deep Q-Learning (DQN), and the problem of correlation and non-static distribution could be solved by the experience replay method. The current Q value is generated by the evaluation network, and the target Q value is generated by the target network . The expm of DQN . AlthougThe Actor-Critic (AC) algorithm is representative of strategy learning. There are two neural networks that exist in the AC algorithm. One is used to interact with the environment to select actions, and the other is used to evaluate the quality of actions, and the network parameters are updated by gradient descent. The AC algorithm is good but difficult to converge. Compared with random strategies, deterministic strategies adopt different action probabilities at the same state when solving continuous action problems, but the maximum probability is only one. Double actor neural networks and double critic neural networks were used in the Deep Deterministic Policy Gradient (DDPG) algorithm to improve the convergence of neural networks . The algThe current and previous strategy networks were used by the traditional PPO algorithm to improve the uncertainty of the learning rate, but they still had a large variance. A generalized advantage estimate was proposed by John Schulman et al. to improve the TRPO algorithm , which cFirst, the application of baseline in strategy learning should be understood. The baseline could be regarded as a function Although the gradient is not affected by the value of At this time, the Monte Carlo approximation of Because the value We can define Therefore, the generalized advantage estimation can be obtained. The formula is as follows:The loss function of the PPO algorithm is:In this equation, State information is input by the actor network and critic network in the PPO algorithm. Two-dimensional plane coordinates of the ship  Goal approach rewardThe primary task to solve the intelligent SMASS path planning was to make the SMASS reach the target position. The goal approach reward value was set as follows:(2) Yaw angle rewardWhen the SMASS is planning the path, the heading angle should be taken as an important indicator. As shown in (3) Target point rewardIn order to get the SMASS to the target point, it is necessary to set a reward at the target point position. At the same time, the SMASS should also receive a negative reward when it collides with obstacles during navigation. The reward value is set as follows:(4) Obstacle avoidance rewardThe laser radar detection range of the SMASS is a circle, launching 24 detection lines from the center of the circle; (5) COLREGs rewardIn order to make the trained SMASS behavior satisfy COLREGs, a COLREGs reward function was introduced. The distance between SMASS and the target point was designed in the COLREGs reward.While SMASS needs to keep heading, the rudder angle should be 0. In addition, when SMASS needs to avoid obstacles or target ships, she should alter her course to starboard. These are defined as satisfying COLREGs. Otherwise, SMASS should alter her course to port or hold heading after encountering obstacles or target ships, which is considered to be a violation of COLREGs. When the SMASS operations comply with COLREGs, the SMASS would obtain positive rewards. However, when SMASS violates COLREGs, it will be punished. Hence, the reward function can be set as follows:Therefore, the calculation process of the total reward function is shown in The training environment is necessary for the intelligent SMASS deep reinforcement learning. A designed unmanned ship training environment can quickly test algorithms . Hence, Some restrictions were attached to the SMASS model. SMASS cannot slow down her speed and can only alert her course during the voyage. The SMASS inertia was appropriately increased to simulate the real motion state of the SMASS. In the SMASS steering phase, with the increase of the rudder angle, the rudder transverse force and rudder force turn the SMASS moment. In the transition stage, the transverse velocity and angular velocity were generated under the action of transverse force and rudder force transfer torque, and the increasingly obvious oblique shipping motion made the ship enter the accelerated rotation state. When the SMASS moved in a fixed-length cycle, the steering force transfer torque, drift angle hydrodynamic transfer torque, and resistance transfer torque were balanced. The acceleration of the rotational angular is zero, and the rotational angular velocity was the largest and most stable at this value. This experiment assumed that the SMASS navigated in still water.Experimental parameter settings are shown in In the experiment, the initial position of the SMASS in the simulation environment was . There were six static obstacles in the simulation environment, and the coordinates of these six static obstacles were , , , , , and . There were two target points, and the coordinates were  and , as shown in After 1000 training times, SMASS could avoid obstacle 1 and obstacle 2. When the SMASS sailed on the port side of obstacle 2, the course remained unchanged. When encountering obstacle 2, the SMASS took two consecutive port alters of 25° and moved towards the upper right under obstacle 2. When it was 0.6 miles from obstacle 1, her course to port was altered to 45°, along with obstacle 1 upward obliquely. The SMASS continuously steered port and starboard and changed course during movement, but the SMASS could not reach the target point and collided with the environmental framework during the wandering process. SMASS collision avoidance obstacles 1, 2, and 3 are shown in After training about 1200 times, the SMASS successfully reached the first target point. Subsequently, the SMASS continuously altered her course to port 45° and sailed to the target point 2. When the SMASS passed under obstacle 3 and navigated towards obstacle 4, her course was altered to starboard 25°, then port and starboard rudder were altered continuously to ensure heading stability.After training 1500 times, the SMASS could maintain her course and sail to the target point. The SMASS first altered her course to port 25° close to the upper starboard of obstacle 6, and then turned starboard by 25° twice in succession, passing over Obstacle 6, successfully reaching the target point 2. The collision avoidance process is shown in There were five obstacles in the second simulation environment, and the coordinates of these five obstacles were , , , , and . The coordinates of the two target points were  and , as shown in After training about 1200 times, the SMASS frequently operated the rudder and reached the first target point. In the process of sailing to the second target point, the SMASS chose to sail around obstacle 1 from above, as shown in After training 1400 times, the SMASS almost did not collide with five obstacles or enter the minimum distance To verify the effectiveness of the improved PPO algorithm, this paper compared the improved PPO algorithm with the other classic strategy-based reinforcement learning algorithms . As shown in The Generalized Advantage Estimation Algorithm directly affects the convergence speed and convergence quality of the PPO algorithm. In this experiment, four groups of comparison experiments were conducted to prove the influence of differences in the generalized advantage estimation on the PPO algorithm. Taking the training environment as an example, four The convergence of actor and critic networks when The convergence of actor and critic networks when The convergence of actor and critic networks when The convergence of actor and critic networks when Generalizability refers to the ability of trained models to apply to new data and make accurate predictions. When the training is insufficient, the fitting ability of the decision-making system is not obvious. The disturbance of training data is insufficient to make the decision-making system change significantly. With the increase of training times, the fitting ability of the decision-making system is gradually enhanced. The disturbance can be detected by the decision-making system. A model is often trained too well on training data, that is, overfitting, so that it cannot be generalized. In order to prove the generalization of the proposed SMASS intelligent obstacle avoidance model in this paper, several different simulation environments were constructed to verify the generalizability of the trained SMASS obstacle avoidance network.The eight representative simulation environments were extracted and displayed as shown in In addition, the avoidance simulations of sailing target ships were carried out to verify the trained SMASS obstacle avoidance capability. Taking the No. 9 environment as an example, these sailing target ships met the trained SMASS under the different collision encounter situations, and the trained SMASS could avoid them accurately and safely according to COLREGs.As shown in The improved PPO algorithm is superior to other traditional model-free reinforcement learning algorithms based on strategy learning in solving ship decision-making and local path planning problems. The improved PPO algorithm has the advantages of fast convergence and low loss value.The improved PPO algorithm has a strong self-learning ability and strong generalization, which could be used to solve the SMASS local path planning and collision avoidance decision-making simultaneously in different complex navigation environments.An improved PPO algorithm for path planning and obstacle avoidance in different complex waters was presented in this paper. SMASS can perform complex local path planning and obstacle avoidance operations when external information is not fully accepted. In this experiment, five factors were considered in the design of the reward function, namely, the relationship between target position, angle, and distance, COLREGs, the reward for safety obstacle avoidance, and whether to reach the target point. This algorithm also performed well in complex waters composed of different numbers of obstacles. The contributions of this experiment are as follows:Some works should be explored in the future. In the experiment, there are some limitations in setting obstacles into cylinders and squares. Actual obstacles such as islands and navigable areas are not suitable to be set into base shapes. The design of complex obstacles is one of the directions in the future study. In addition, the rudder angle output in this study was the command rudder angle, which has a certain deviation from the execution rudder angle. This is also an important factor to be considered in future studies.


Inguinal hernia repairs are commonly performed procedures. The surgical techniques vary from open procedures to minimally invasive and robotic-assisted surgeries and include totally extra-peritoneal hernia repairs (TEP) and robotic transabdominal pre-peritoneal hernia repairs (rTAPP). So far, there is no randomized and blinded clinical trial comparing these two surgical approaches. Our objective is to investigate whether rTAPP is associated with a decreased postoperative level of pain.This is a prospective, single center, randomized and blinded clinical trial. Patients will receive either rTAPP or TEP for uni- or bilateral inguinal hernias. All patients and assessors of the study are blinded to the randomization. The perioperative setting is standardized, and all surgeons will perform both rTAPP and TEP to eliminate surgeons` bias. Primary endpoint is the assessment of pain while coughing 24 hours after surgery using the numeric rating scale (NRS). Secondary endpoints include the assessment of multiple pain and quality of life questionnaires at several defined times according to the study schedule. Furthermore, intra- and postoperative complications, duration until discharge, procedure time, duration of postoperative sick leave and the recurrence rate will be evaluated.ClinicalTrials.gov under the registry number NCT05216276.The trial has been registered at Trial comparing robotic and conventional minimal-invasive inguinal hernia repairRandomized and patient/assessor blinded trialEarly postoperative pain as primary outcome (24 hours)Secondary patient outcomes include pain and quality of life scores up to one yearFurther secondary outcomes: complications, costs, surgeon’s stress level Inguinal hernias are the most common type of hernias and the reported annual incidence varies from 13 to 34/10,000 13. The onThe applied surgical techniques for inguinal hernia repair have evolved over the past decades . For a l78Robotic-assisted hernia repair has previously been described as a “natural progression” . RoboticIn the literature, comparison of rTAPP and TEP shows controversial results. Some studies report lower pain scores and a lower incidence of complications for rTAPP compared to TEP 13. Other14We hypothesize that rTAPP will lead to a reduction of acute postoperative pain compared to TEP and that pain reduction will later translate into earlier recovery. We presume a 20% reduction of the numeric rating scale (NRS) while coughing 24 hours after surgery. Accordingly, the objective of the current randomized controlled trial is to investigate whether rTAPP is associated with a decreased level of pain shortly after surgery compared to conventional TEP.This is a prospective, single center, randomized and blinded clinical trial. The trial will be led by and conducted at the Clarunis, University Center for Gastrointestinal and Liver Diseases in the St. Clara Hospital in Basel, Switzerland.ClinicalTrials.gov under the registry number NCT05216276.This trial has been registered at Patients with a unilateral or bilateral inguinal hernia, older than 18 years of age and able to give their informed consent are included in the study. Exclusion criteria are recurrent hernias, previous open abdominal surgery at or below the umbilicus, liver disease defined by the presence of ascites, end-stage renal disease requiring dialysis, pregnancy, the inability to give informed consent, and the requirement of emergency surgery. Furthermore, patients with the need of an open surgery, due to their preference, the inability to undergo general anesthesia or to tolerate a pneumoperitoneum will not be included. A flowchart of participant recruitment, randomization and follow-up is shown in All patients seen during consultation for inguinal hernia will be registered. This primary assessment will be performed for every patient, including patients that withhold consent or fulfill any of the mentioned exclusion criteria, hereby ensuring a thorough documentation and a complete dataset.A numeric rating scale (NRS) of 4.37+/–1.66 at coughing 24 hours after surgery is assumed in the control group. This estimate takes two previous studies into consideration: the double-blinded randomized controlled trial by A. Mughal comparing TEP with and without peripheral nerve block  and the 2). This ensures that the groups will be similar regarding these important covariates. The randomization is performed by a case manager that is not involved in the treatment of the patients. The randomization is performed software-based using the REDCap randomization module. All patients and assessors of the study variables are blinded to the randomization. The surgeon is not blinded. The surgeon will not assess information for the endpoints. The exact surgery is not mentioned in the primary report before surgery nor in the surgery protocol given to the patient or to the postop personnel. The exact report is published only after 7 days. In addition, case manager, anesthesia-, OR-, wake-up-, ward-  personnel are informed about the blinded study. Given the fact that for rTAPP the incisions are made in a horizontal line at the level of the umbilicus and for TEP the incisions are made in a vertical line between the umbilicus and the symphysis, all patients will receive the same standardized opaque wound dressing, covering all potential incision sites. This ensures the blinding of the patients and the assessors of the study variables postoperatively. Patients will be unblinded 7 days after the surgery. Any violation of the blinding will be noted in the case reporting form.Patients are allocated to rTAPP or TEP by randomization. The randomization is performed as a block randomization and stratified for unilateral or bilateral hernias and for the body mass index , TME, mesh-rectopexy and TARUP. The operation scheduling will equally distribute rTAPP and TEP for each surgeon, to ensure an elimination of surgeons’ bias.Due to Swiss regulations, surgical treatment of a unilateral hernia is performed in an outpatient setting, whereas patients with bilateral hernias are hospitalized for at least one night. All patients will receive an intravenous antibiotic prophylaxis with 2 g of Cefazolin 30 minutes before the incision. The patients will receive a standardized general anesthesia with an endotracheal intubation, and intraoperative pain management will follow a standard based on fentanyl. Postoperatively, all patients will receive Celecoxib 200 mg twice a day for 5 days and Metamizole 500 mg 4 times a day for 7 days.The Xi da Vinci patient cart is placed on either side of the operating table with integrated table motion. The abdomen is insufflated with carbon dioxide using a Veress needle at Palmer’s point. The first 8 mm robotic port is placed above the umbilicus and the AirSeal® insufflator is connected hereto. After a brief laparoscopy, two further 8 mm robotic ports are placed on each side of the first port, with a distance of 6–8 cm. After placing the patient in a slight Trendelenburg position, the Xi robot is docked. The monopolar scissors are inserted in the right arm, the fenestrated bipolar grasper in the left arm. The center port holds the 30° endoscope. The peritoneum is incised 4–5 cm above the internal ring, from the median umbilical ligament in direction of the anterior superior iliac spine and gradually detached from the transversalis fascia. Medially the space of Retzius is entered and the Cooper ligament is exposed. Laterally the dissection is completed to the psoas muscle. The hernial sac is separated from the spermatic cord in males or the round ligament of the uterus in females and is then reduced. The peritoneum is then further dissected in cephalad direction for at least 5 cm to create adequate space for the mesh. The mesh with a size of at least 12 × 15 cm is positioned to cover the entire myopectineal orifice including the site of direct, indirect, and femoral hernias. The mesh is sutured to the Cooper ligament and in case of a larger medial hernia additionally to the ventral abdominal wall, using an absorbable Vicryl suture. The triangle of pain is meticulously spared. The initially opened peritoneum is readapted with an absorbable 3-0 V-Loc™. In the end, the ports are removed, the pneumoperitoneum is desufflated, and the skin is closed with a resorbable intracutaneous suture.The first incision is made at the umbilicus and access to the layer between the rectal muscle and the posterior lamina of the rectus sheath is gained. Here, a dissecting balloon is inserted to create a preperitoneal space under direct camera visualization and then a 12 mm port is placed. Two further 5 mm ports are installed along the median line below the umbilicus. The patient is placed in a slight Trendelenburg position and tilted towards the surgeon. Equivalently to the rTAPP procedure, the hernial sac is separated from the spermatic cord or the round ligament of the uterus and then reduced. The peritoneum is further dissected to create sufficient space for the mesh which is then inserted and positioned. The mesh is secured with glue or, in case of a M3 hernia according to the European Hernia Society (EHS) classification , with reThe primary endpoint is the assessment of pain while coughing 24 hours after surgery, measured on a numeric rating scale (NRS). The endpoint was determined considering that pain after inguinal hernia repair is a major issue as it may compromise the quality of life and acute postoperative pain may be seen as a surrogate for the development of chronic pain 2021.20Secondary endpoints include the analysis of the NRS score 2 hours, 7 and 30 days as well as 6 and 12 months postoperatively and the assessment of the inguinal pain questionnaire (sf-IPQ), the European Quality of Life 5 Dimension 5 Level questionnaire (EQ-5D-5L), the ICEpop CAPability measure for Older people (ICECAP-O), the Short-Form Six-Dimension (SF-6D), the 12-item Short Form Survey (SF-12) and the Carolina Comfort Scale (CCS) at multiple defined times within 1 year after the surgery. Furthermore, secondary endpoints include the evaluation of intra- and postoperative complications, according to the Clavien-Dindo classification and the Comprehensive Complication index (CCI), and the assessment of the recurrence rate 6 and 12 months postoperatively. The procedure time, duration until discharge, amount of intraoperative and postoperative pain medication, costs of the treatment, duration of the sick leave as well as the costs of the sick leave and the type of labor that can be conducted will be evaluated. Additionally, the ergonomics of the surgeons measured by the National Aeronautics and Space Administration Task Load Index (NASA TLX) and the Dutch Musculoskeletal questionnaire will be examined.Data is collected by a dedicated study nurse and the operating surgeon face-to-face or via telephone and online survey, after thorough explanation of the different scores and questionnaires according to the schedule displayed in Analysis will be performed on an intention-to-treat basis with all patients in the group to which they were allocated. Statistical analyses will be performed using IBM SPSS Statistics software , depicting continuous data as mean or median and categorical data as counts and percentage of total. The Mann-Whitney U-test will be used to compare ordinal data and the Fisher’s exact test for the analysis of nominal data. Multiple group comparisons of normally distributed continuous variables will be performed by analysis of variance (ANOVA). Pearson’s r-correlation will be used to correlate the primary and secondary outcomes with surgical experience as well as specific patient characteristics.


Robust, fault tolerant, and available systems are fundamental for the adoption of Internet of Things (IoT) in critical domains, such as finance, health, and safety. The IoT infrastructure is often used to collect a large amount of data to meet the business demands of Smart Cities, Industry 4.0, and Smart Home, but there is a opportunity to use these data to intrinsically monitor an IoT system in an autonomous way. A Test Driven Development (TDD) approach for automatic module assessment for ESP32 and ESP8266 IoT development devices based on unsupervised Machine Learning (ML) is proposed to monitor IoT device status. A framework consisting of business drivers, non-functional requirements, engineering view, dynamic system evaluation, and recommendations phases is proposed to be used with the TDD development tool. The proposal is evaluated in academic and smart home study cases with 25 devices, consisting of 15 different firmware versions collected in one week, with a total of over 550,000 IoT status readings. The K-Means algorithm was applied to free memory available, internal temperature, and Wi-Fi level metrics to automatically monitor the IoT devices under development to identify device constraints violation and provide insights for monitoring frequency configuration of different firmware versions. To the best of the authors’ knowledge, it is the first TDD approach for IoT module automatic assessment which uses machine learning based on the real testbed data. The IoT status monitoring and the Python scripts for model training and inference with K-Means algorithm are available under a Creative Commons license. The Internet of Things (IoT) is a critical infrastructure necessary to the deployment of large projects with huge potential to change whole business sectors. Things present in IoT environments are responsible for measuring the variables of the physical world and share it across the network. Either a local network or the Internet can be used to connect things and generate a connected environment that can be measured and controlled. Sensors embedded in these things enable the measurement of temperature, pressure, oxygen levels, presence, and others that are applied to projects related to Smart Cities, Industry 4.0, and Big Data.Consider an example of IoT application in the remote operation field. Workers operating remotely on hazardous situations and dangerous scenarios, with reduced costs and safety guaranteed by the distance and the real time monitoring . IoT couMoreover, critical applications, such as health, finance, and safety, require higher levels of quality. These areas are good examples of critical areas that require robust and fault tolerant systems with high availability, as they are used in fields that are crucial for the function of society, and unavailability events in these areas can be dangerous and expensive. As analyzed by IBM in 1998 , the unaConsidering an IoT architecture organized in layers as described in the reference architecture IoT-A , the appHowever, the deployment of traditional approaches to the IoT domain is not trivial. The IoT integrates constrained devices with energy, processing, memory, and communication limitations, and it also employs heterogeneous communication protocols, devices, and architectures, such as those that are edge and cloud-based. A motivation for this work is relative to the application of a similar protocol to the Simple Network Management Protocol (SNMP)  to managIn this article, we consider the following research question: “is it possible to assess if non-functional requirements are being supported by an IoT system in real time in an autonomous way?”. The objective is to monitor IoT modules with the goal of increasing its Testability. Testability is considered a sub-characteristic of the Maintainability non-functional characteristic as described in ISO25010 .The problem considered within the scope of this paper is the lack of automated testing and monitoring of IoT devices under development. Such low Testability hinders the quality aspects of the resulting IoT system. For example, if an IoT device violates its resource utilization in the development phase, the resulting IoT product may present unavailability events when deployed.Our hypothesis considers that the RM-ODP  standardA Test-Driven Development (TDD) tool built with K-Means clustering algorithm with real testbed data to enhance the Testability non-functional requirement of IoT systems under development;A framework that considers business drivers and non-functional requirements to support the IoT solution development process;Validation of a proposed framework in smart home and remote lab study cases to obtain deployment conditions and monitoring frequency of different IoT device firmware versions.The main contributions of this work are:This article is an extended version of our paper published in Intercloud and IoT at 8th International Conference on Future Internet of Things and Cloud (FiCloud 2021), considering the historical data collection, the addition of the remote lab case study, and the proposition of a generalized framework considering RM-ODP besides the TDD.To the best of the authors’ knowledge, it is the first TDD approach for IoT module automatic assessment which uses machine learning based on real testbed data. Some of the related work carried out experiments in testbeds with real hardware devices, but none of them used machine learning, which motivated the novel TDD approach validated with two study cases.https://github.com/vthayashi/TDD-IoT, (accessed on 30 November 2022).The open source materials related to this article are available in the following repository: This document is organized as follows: The related work  offers aThe researchers in  presenteDevelopers in  demonstrResearchers in  compare Researchers in  proposedThe researchers in  proposedIn , a routiThe proposal in  is to usResearchers in  presenteIn , the resA mathematical model to represent a system in aggressive cyberspace is presented in , and it The authors of  propose Regarding IoT systems development, some application fields are vehicular networks  and smaThis section presents the IoT architecture considered in this work, the ISO25010 standard to understand the non-functional requirements used to specify quality levels the resulting IoT system must adhere. The RM-ODP method is described as a structured approach to consider different views during a system specification and modeling. The microservices architecture is described considering its strong and weak points. Finally, the TDD methodology is summarized as a guideline we used in the proposition of the automated monitoring based on automated tests.Standards and reference architectures (RA) contribute to the reduction of the complexity of IoT systems and the architectural structures help to build IoT systems that can support and execute necessary functions . It is iIoT architecture presents challenges in the most diverse scenarios. These challenges may be in architectural layers (inter-architecture or intra-architecture), entity-based , technological (specific IoT or related technologies), or feature-based challenges  . Then, tIoT heterogeneity can be observed in  with an Therefore, the diversity of IoT architectures result in its complexity. Reference architectures are proposed to mitigate this diversity. Another issue must be taken into account, the heterogeneity of physical and software components to be integrated in these architectures. These components directly influence the complexity of the IoT design and the choice of architecture.In the proposed TDD framework, we propose the automated monitoring of black-box connected devices and software modules that compose IoT architectures. As long as the architecture uses the open standard MQTT (Message Queuing Telemetry Transport)  protocolThere are development guidelines to be followed in order to ensure that the IoT system delivers the needed level of trust and reliability, with non-functional requirements to be fulfilled. By definition, the non-functional requirements of a system are what the system is, not what the system does. The non-functional requirements must be evaluated under a quality model that assures the quality in a degree that the system can satisfy the needs and demands of its stakeholders. The ISO/IEC model defines the standard of non-functional requirements and guarantees that each of the stakeholders needs are covered.The ISO25010 standard defines a quality model composed of eight quality characteristics: functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, and portability. The performance efficiency considers the time behavior, resource utilization and capacity of a system. The maintainability characteristic has the modularity, reusability, analysability, modifiability, and testability characteristics.The development process must be planned in order to be compliant with the ISO/IEC 25010  from theThe proposed framework aims to enhance the testability, which is a characteristic of the maintainability non-functional requirement of ISO25010 standard. The study cases reported in the next sections investigated the automated monitoring of the resource utilization, characteristic of the performance efficiency non-functional requirement.The RM-ODP method  providesConsidering the trade-off involving high capacity in the IoT scenario in terms of processing from cloud resources, and limited capacity of vulnerable sensors and actuators, these views from RM-ODP contribute to identify architectural decisions that must be carefully created, tested, and monitored through all the development and operation life cycle to support quality aspects, such as fault tolerance, availability, precision, and traceability.Simulations regarding trade-off analysis in IoT architectures are presented in the literature. For example, the authors of  use PetrThe case studies presented in the following sections use the RM-ODP views to organize the documentation of the automated monitoring tests.A monolith structure means high coupling in terms of data, functionality, and technology. Low development scale, low productivity in tests and limited evolution are some characteristics in opposition with an agile development process. There are non-functional requirements as the foundation to adopt microservices ,30 strucSensors precision, availability, and resulting data integrity considering vulnerable devices. In this scenario, fault tolerance is achieved by microservices which deal with the low robustness of devices, so that the fragility of specific parts of the system do not damage the rest of the IoT application (if well isolated).The fault tolerance aspect needs redundancies of hardware, software, and data algorithms with high cohesion and low coupling structuring components. Small parts command alternative functions that are sufficiently complete to avoid system unavailability because of these smart redundancies provided by independent services.Modifiability would be one of the leading quality aspects of successful microservices adoption. The modification and test spaces are limited to pieces that bring agility, velocity for changes, and business evolution due to the flexible architectural solution.An IoT system requires a balanced architecture to support agility to evolve, perform, monitor, and respond considering suitable functions and the large scale of a parallel development team. All of these aspects are present in the microservices-based architecture. Additional benefits are to be achieved if the agile development process, automation tests, and continuous integration are combined with the microservices architecture.The manifest of agile software development main point is to focus more on what makes the solution work than in rules and writing about the solution . The goaTest programs may create conditions with high simultaneous access then measure the response time to assess good performance levels;Similar to precision, using simulated fault of sensor readings to validate the employed redundancies to support precise responses;In case of simultaneous access, the first item would be sufficient to create a quality gate to approve modifications;Availability can be measured by test cases that create fault parts of the system and the resultant behaviors.The TDD seems easy to write but difficult to execute. In a microservices-based system, the TDD is so crucial as it is related to properly testing components (code and assets) of the IoT system. If an automation test module is responsible for approving modified parts, then an automated quality gate is working. Additional software process challenges arise with non-functional requirements; response time, precision, simultaneous access, and fault-tolerance are examples of quality aspects that might require automated verification:As the development process follows the test driven methodology, an opportunity arises: the technology itself could be used to ensure the system compliance levels. In this work, we consider the automation of the test cases based on data collected by deployed modules ensures that these tests are close enough to the real-world scenario.This section presents the proposed framework for IoT automated monitoring. First, the proposed method based on TDD and RM-ODP is described. The automated test approach detailing how the unsupervised machine learning is used on real testbed data follows. Finally, an IoT status example for the ESP32 IoT Development Module is presented.IoT developer: the technical staff, individual, or company that is held responsible by the implementation of the IoT solution. It must have necessary hardware, software and architectural expertise to develop and deploy the IoT solution;Business specialist: usually the client, a team, or individual that wants to deploy IoT solutions to reduce costs and create value in Smart City, Industry 4.0, Smart Home, and other application fields. It has a deep understanding of the business needs related to the IoT solution.The proposed method considered in the framework for automated monitoring in IoT is illustrated in the 1.Business drivers: related to business view from the RM-ODP. Business needs must be specified by the business specialist to the IoT developer;2.Non-functional requirements: according to the ISO25010, specify the non-functional requirements and prioritize these requirements considering the business drivers described in the first step;3.Engineering view: related to the engineering view present in RM-ODP. The IoT developer must present the architectural solution that support the non-functional requirements monitoring;4.Dynamic system evaluation: real time monitoring and data collection from testbeds that enable automated tests and analysis with unsupervised machine learning mechanisms. The automated tests are used by the IoT developer in a TDD methodology;5.Recommendations: the IoT developer must provide suggestions based on the analysis performed to the business specialist, considering how the business drivers are modeled in non-functional requirements. The suggestions may be related to the architecture .We consider that the IoT developer and the business specialist are individuals or teams that participate in an IoT project in a critical field. The quality aspects must be controlled to make sure that the resulting technical solution adheres to the non-functional requirements modeled considering the business drivers. The method may be applied in the start of an IoT project to enhance the resulting IoT solution quality attributes. It is comprised of the following steps:The proposed automated test approach is illustrated in ESP32 modules with different firmware versions (and even for different applications) send their data frequently to an open MQTT broker using the integrated Wi-Fi communication (1). A script is responsible for data collection from the open MQTT broker deployed in a cloud computing environment (2). The data collected are used to train the K-Means model with another script deployed in Google Colab cloud environment, with the cluster labeling  performed by the developer (3), as detailed in As illustrated in The metrics free memory available, processor’s temperature and Wi-Fi level may be collected with the ESP32. We validate our approach with a Proof of Concept (PoC) with ESP32 and ESP8266 (its predecessor) Wemos IoT Development modules. The libraries used for the IoT module monitoring script are Wi-FiClient and Wi-Fi for Wi-Fi connection, and PubSubClient for MQTT connection.This section presents two study cases that show how the proposed method could be applied in the wild with different MQTT broker implementations, SQL and NoSQL databases, and in private and public cloud environments.The first study case is a remote lab whose IoT devices are responsible for temperature and humidity monitoring, and air conditioner control by infra-red signals. As illustrated in the The data collection procedure is detailed in the The TDD procedure is described in the We collected a total of 1178 readings from two ESP32 devices with two different firmware versions from 9 July 2021 to 13 July 2021. The metrics of the ESP32 devices are freeheap in bytes , internal_temperature in Celsius  and wifi_level in decibels . A summary of the collected metrics of these two devices is presented in the The correlation matrix of the The negative correlation between wifi_level and freeheap is confirmed by the clusters visualization in The clustering results are summarized in With the trained model, it was possible to classify the IoT device under development status described in  in the b1.Business drivers: the motivation is to scale the air conditioner automation solution to the entire campus, considering the different communication conditions, and the possible additional functionalities that may be required by certain labs;2.Non-functional requirements: the performance efficiency requirement is prioritized, considering the resource utilization. Another relevant requirement is the maintainability, considering the testability characteristic.3.Engineering view: the architectural solution was illustrated in the 4.Dynamic system evaluation: initial descriptions are described in 5.Recommendations: considering that the deployed modules presented Wi-Fi levels from −83 to −65 see , it is dConsidering the proposed framework, we describe the five steps considered:The second study case is a smart home whose IoT devices are responsible for energy consumption monitoring, and appliances control by smart plugs.As illustrated in the The data collection procedure is detailed in the The TDD procedure is described in the We collected a total of 595,554 readings from 22 ESP8266 devices with 13 different firmware versions from 10 July 2021 to 17 July 2021. The metrics of the ESP8266 devices are freeheap in bytes , wifi_level in decibels , fs_usedbytes  and max_loop_wdt_cnt .The readings count by each of the 13 different firmware versions is presented in A summary of the collected metrics of the 22 ESP8266 devices installed in the smart home is presented in the The correlation matrix of the The K-Means clustering algorithm was performed using the aforementioned four metrics with the total number of cluster of two corresponding to a good and bad status clusters. After the clustering, the cluster label is performed by the IoT developer based on the visual representation, such as the one represented in The clustering results are summarized in With the trained model, it was possible to classify the IoT device under development status described in  in the g1.Business drivers: the motivation is to scale the energy efficiency solution to smart homes in the same region, considering the different communication conditions;2.Non-functional requirements: similar to the remote lab scenario, the performance efficiency requirement is prioritized, considering the resource utilization. Another relevant requirement is the maintainability, considering the testability characteristic.3.Engineering view: the architectural solution was illustrated in the 4.Dynamic system evaluation: initial descriptions are described in 5.Recommendations: considering that the deployed modules have the filesystem usage presented in Considering the proposed framework, we describe the five steps considered:The proposed framework and the TDD tool built with the K-Means algorithm could provide a rationale for the remote lab and smart home study cases, considering an end-to-end view. The RM-ODP standard was essential to provide a link between business drivers and non-functional requirements, and an approach to use the automated monitoring tool with the objective to analyze and assist in the IoT solution development process.Some benefits found in the study cases: in the remote lab scenario the deployment and free memory conditions for devices with extended functionality were elucidated, and in the smart home scenario the status of 13 different firmware versions could be evaluated in an automated way, and provide insights for the monitoring frequency configuration.The different conditions of the study cases show that our approach is not limited to specific scenarios or proprietary technologies. The study cases cover private and cloud computing environments in Brazil and USA, HiveMQ and Mosquitto MQTT brokers, SQL and No-SQL databases, and ESP8266 and ESP32 devices. The proof of concept was performed with these IoT development boards, but any board that supports the MQTT protocol might be integrated with the proposed TDD framework.Another contribution is related to the applicability of the proposed method, considering that the proof of concept is open source for facilitated replication, and the communication is based on the open source standard MQTT . The scrThe validation of the proposed method with real devices might contribute to close the gap between proposed TDD methodologies and their effective application. We also expect that the proof of concept contributes as an example of using machine learning to generate automated tests for IoT.However, a limitation of the proposed solution is that the automated tool must be trained in each different deployment scenarios for accurate assessment results. For example, in the remote lab the Internet connection could provide a remote clock sync in a faster way than in the smart home scenario, so an IoT device developed for the remote lab may not perform well in the smart home scenario.Considering the most relevant related work found in the literature regarding IoT testing,  shows anThe architecture proposed in  providesThe MAPE-K framework , proposed in  requiresResearchers in  presenteThe Open-Multinet (OMN) framework  is basedSimulations and its underlying mathematical apparatus ,20 are aTo the best of the authors’ knowledge, it is the first TDD approach for IoT module automatic assessment which uses machine learning based on real testbed data.Considering the challenge of quality assessment in IoT systems, this work addressed the topic of automated non-functional requirements evaluation. The objective is to enhance the Testability non-functional requirement of IoT systems under development.The proposal is evaluated in academic and smart home study cases with 25 devices, consisting of 15 different firmware versions collected in one week, with a total of over 550,000 IoT status readings. K-Means algorithm was applied to free memory available, internal temperature and Wi-Fi level metrics to automatically monitor the IoT devices under development to identify device constraints violation and provide insights for monitoring frequency configuration of different firmware versions. The clustering results enabled to assess if new data of a specific IoT device fits into bad or good status. In the remote lab, the bad status accounted for around 60% of the total readings.The proposed framework guides the IoT system designer from the business domain to the non-functional requirement dynamic evaluation, based on RM-ODP  standardTo the best of the authors’ knowledge, it is the first TDD approach for IoT module automatic assessment which uses machine learning based on real testbed data. Some of the related work carried out experiments in testbeds with real hardware devices but none of them used machine learning, which motivated the novel TDD approach validated with two study cases.As future work, the comparison of K-Means with other unsupervised learning algorithms could be performed. Other possibility is to test different deployment conditions with synthetic firmware versions with known resource misuses to investigate how the number of clusters may affect the proposed automated tool performance. Supervised algorithms for time series prediction and firmware version classification tasks also might be applied to the collected data to expand the TDD approach to the security domain, focusing on anomaly detection.A limitation of the proposed framework and the associated TDD tool based on machine learning is the need to know the total number of clusters beforehand, because the tool uses the K-Means algorithm. Such a limitation may be investigated in future work by testing other machine learning models. For example, graph-based deep learning approaches have been proposed as a promising solution for relevant problems (e.g., anomaly detection in communication networks ,35, intr


Recently, content-based image retrieval (CBIR) based on bag-of-visual-words (BoVW) model has been one of the most promising and increasingly active research areas. In this paper, we propose a new CBIR framework based on the visual words fusion of multiple feature descriptors to achieve an improved retrieval performance, where interest points are separately extracted from an image using features from accelerated segment test (FAST) and speeded-up robust features (SURF). The extracted keypoints are then fused together in a single keypoint feature vector and the improved RootSIFT algorithm is applied to describe the region surrounding each keypoint. Afterward, the FeatureWiz algorithm is employed to reduce features and select the best features for the BoVW learning model. To create the codebook, K-means clustering is applied to quantize visual features into a smaller set of visual words. Finally, the feature vectors extracted from the BoVW model are fed into a support vector machines (SVMs) classifier for image retrieval. An inverted index technique based on cosine distance metric is applied to sort the retrieved images to the similarity of the query image. Experiments on three benchmark datasets  show that the presented CBIR technique can deliver comparable results to other state-of-the-art techniques, by achieving average accuracies of 92.94%, 98.40% and 84.94% on these datasets, respectively. An image is more than thousands of words and is the cornerstone of visual media representation. With the tremendous development in information technology, computer engineering, and smartphone, the number of shared images over networks has increased rapidly ,2. Thus,Computer interpretation of the contents of an image is less trivial. A machine’s vision is an extensive matrix of numbers, and it has no idea of the thoughts, knowledge, or meaning of the image it tries to convey. To understand the contents of an image, apply image classification and retrieval that utilize machine learning algorithms’ tasks to extract meaning from an image. This process could be as simple as assigning a label to what the image contains. Searching for an image based on its content or example is known as CBIR or Query By Image Content (QBIC). Many modern applications in hospitals, factories, geographical information systems, remote sensing, cultural heritage, mining, education, home entertainment, journalism and advertising, fashion and interior design, crime prevention, military, medical diagnosis, web searching, etc., use CBIR models to make their systems more intelligent and robust .The first search by an image system was built by IBM . After tFeature-level fusion is a critical stage in feature preparation. The advantage of feature fusion lies in two aspects: first, it can derive the most discriminatory information from the original multiple feature sets involved in fusion; second, it can eliminate redundant information resulting from feature set correlation, allowing for real-time decision-making. In other words, feature fusion can derive and obtain the most effective and low-dimensional feature vector sets that will help with the final conclusion. Although the merits and importance of feature fusion in the CBIR applications, it has some challenges where it must run some experiments on the dataset to select the most appropriate features to fuse between them. Moreover, it should tune the appropriate percentage of features which will be used in the learning stage to avoid the overfitting of data in the learning stage. The feature fusion between the KFDs introduced in this paper is more effective than the recent fusion techniques between the FDs, where the extracted features at the fusion between KFDs are less than in the case of the fusion between FDs, which decreases the probability of overfitting and presenting more accurate results. Furthermore, the KFDs fusion has less computational complexity than the FDs fusion.There are many techniques for CBIR. However, the outcomes techniques obtained using a single feature type are unsatisfactory. Multiple combined feature descriptors are frequently used to achieve better outcomes, but quickly and effectively searching a database for a relevant query image is still challenging. Moreover, the computational complexity is higher for the existing CBIR systems. To overcome the challenges mentioned above, besides giving better results; this paper introduced four main contributions to improving the CBIR systems, first, comparing the strongest nine KFDs parallel with the seven robust FDs. The comparison is to select the best two KFDs with the most convenient FDs. This contribution produces effective low-dimensional Hybrid Keypoint Features Vector (HKFV) that reduce the next steps’ complexity, cost, and time. The second contribution is using a new feature selection algorithm  to select the powerful features from the extracted features for the next clustering step. This contribution determines the overall powerful features extracted from the dataset that reduces the time complexity and cost for the clustering stage. Furthermore, it decreases the data overfitting probability at the learning stage. The third contribution is applying a feature selection algorithm on the BoVW to select adequate visual words in the learning and sorting stage, called BoVW FeatureWiz Selection (BFS). This contribution improves the learning stage and decreases the data overfitting probability. It also contributes to reducing the time complexity and cost of similarity matching and sorting. The fourth contribution is the new Keypoint Feature Fusion Visual Words (KFFVW) framework. The rest of the paper is structured as follows: CBIR has been an active growing research area for the last three decades . FeatureGhahremani et al.  proposedThe BoVW is a common effective technique in image retrieval tasks. Yousuf et al.  proposedKhan et al.  proposedLastly, CBIR has many techniques, but each has its specific process. This work focused on the recent techniques that depend on BoVW. The latest fusion techniques in BoVW considered the fusion between feature descriptors. The suggested method improves the extracted features by fusing the keypoint features, and it is then described by feature descriptor algorithms rather than the direct fusion between the feature descriptors.The proposed method depends on extracting the FAST and the SURF keypoints separately from each image. Then, these keypoints are fused in a single keypoint feature vector. The RootSIFT algorithm is applied to describe the region surrounding each keypoint. A large number of feature vectors characterizes each image. Afterwards, a feature selection algorithm is used to select the appropriate feature vectors from all images as a sample for the clustering step. A k-means clustering algorithm is applied to formulate feature vectors into clusters. Each cluster is a group of feature vectors whose center represents a visual world. The combination of the visual words constructs the codebook. Then, vector quantization is applied to this codebook to describe each image in one dimension feature vector. The output of the vector quantization is a separate feature vector for each image in the training dataset.The query process requires comparing the query features to every feature vector in the dataset. It is a linear operation O(N). As the size of the dataset increases, the time it takes to perform a query will increase. Building an inverted index of the BoVW to optimize the data structures of the indexed features breaks down the query process to sub-linear O(log(N)). Before the feature extraction phase, a preprocessing step is performed for computational efficiency. The images in the collection are first converted to HSV-color format and then down-sampled to a fixed resolution of 320 × 320 pixels. Feature extraction in CBIR is the process of representing a broad set of images by applying an adequate image descriptor to the keypoints of images to extract features from each image in the dataset. One of the most influencing factors of the BoVW is the feature extraction process, which contributes to constructing powerful visual words. Keypoints detectors fusion in this paper contributes to the success of the feature extraction process for a better BoVW. The feature extraction stage is the baseline of the adaptive image classification, and its success determines the success of the overall classification performance . After rBay et al.  providedx and scale For adapting to any scale, the image is filtered by a Gaussian kernel. The Hessian matrix H . These approximative second-order Gaussian derivatives can be calculated using an integral image regardless of size at a much lower computational cost, which is a part of why SURF is fast. Applying box filters of various sizes in SURF allows for scale-space implementation. Therefore, rather than iteratively shrinking the image size, the scale space is utilized by upscaling the filter size. A 9 × 9 size filter with scale s = 1.2  technique in a 3 × 3 × 3 neighbourhood is used to localize interest points in the image. The Brown et al. approach  is then 9 as Bay  suggesteThe FAST idea is based on the corner detector known as the Smallest Univalue Segment Assimilating Nucleus (SUSAN). The brighter and darker pixels next to each other are determined using the center of a circular area. Only the pixels on the discretized circle that characterize the segment, not the entire circle, are evaluated. Sixteen pixels must be compared to the nucleus value for a full accelerated segment test as shown in As shown in Firstly, the scientific explanation. In many computer vision tasks, the FAST is a corner-detection technique that might extract feature points and then track and map objects. The FAST corner detector’s high computing efficiency may be its most advantageous feature. Furthermore, using machine learning techniques, faster and more efficient computations may be made by computer vision tasks using FAST detectors. The FAST detector is quicker than the difference of Gaussians (DoG), employed by the SIFT, SUSAN, Harris detectors, and many other popular feature extraction techniques. In addition, FAST extracts more keypoints than other detectors. Additionally, when machine learning techniques are used, the outcomes are better.The SURF method is a quick and reliable approach for local, similarity-invariant representation and comparison of images. The SURF approach’s primary appeal is its ability to compute operators quickly using box filters, opening the door to real-time applications, including image matching, object identification, image registration, and classification. The SIFT feature detector stage served as some of the inspiration for SURF authors. The authors of SURF say that the standard version is more resistant to various image alterations than SIFT and is many times quicker than SIFT.On the objective of feature detection, ORB outperforms SIFT and is better than SURF while being nearly two orders of magnitude quicker. The well-known FAST keypoint detector and the BRIEF description are the foundation for ORB. Both methods are appealing due to their high effectiveness and low expense.Secondly, the experimental results. The experimental comparisons of the most vital nine Keypoint Feature Detectors (KFDs) parallel with the seven robust Feature Descriptors (FDs) are implemented based on the BOVW technique, as shown in Based on the previous scientific explanation and the experimental results, it was very interesting to find results of the best combination between SIFT, FAST, SURF, and ORB to produce the HKFV, but the experimental results shown in i and j are the numbers of those keypoints. To form the unique keypoints Compute the SIFT descriptors.(ii)L1-normalize each SIFT vector.(iii)Take the square root of each element in the SIFT vector.In our work, we decided on the RootSIFT feature descriptor for this task. This selection was based on experimental reasons. The experimental comparisons of the most vital nine KFDs parallel with the seven robust FDs shown in C is the value of the percentage , which identifies a pair of variables beyond an externally passed correlation threshold, so they are highly correlated. After determining the pairs, the Mutual Information Score (MIS), which gauges the relationship between two variables, is calculated for each pair. The pair of variables with the lowest correlation and the highest MIS scores are then considered. The variables chosen from SULOV are recursively sent through XGboost in the second stage (Recursive XGBoost), which aids in identifying the best features by skipping the most similar data. One of the optional FeatureWiz parameters is the percentage of the output features from the total input features. The FeatureWiz Percentage Parameters (FPP) will be tuned based on various experiments to select the feature percentage that gives the best retrieval results. The chosen features percentage for the clustering could be calculated asIt is common to group features in an image to give them meaning. K-Means clustering is the most common clustering algorithm for this grouping. Clustering is splitting a large number of data points into a small number of clusters. Data points are assigned to the closest centroid to build a cluster. Therefore, data points in one cluster are easier to compare with data points in other clusters. The center of each cluster is called a visual word. The combination of visual words constructs the vocabulary. The following F and W is calculated as follows:To quantize feature vectors of an image to a single BoVW histogram vector, the Cosine distance matrix D between the total extracted features The Cosine distances matrix D for Then apply image mapping by forming the histogram that describes the image using A scaling technique called normalization shifts and rescales values so that they fall between the ranges of 0 and 1. They are additionally called Min-Max scaling. The normalization step reduces the overfitting probability. The normalization process could be calculated by:The Bag-of-Words (BoW) model represents text data at modelling text using machine learning algorithms. The BoW model is a well-known model for feature extraction in natural language processing. The BoW is a representation of text that describes the occurrence of words within a document. Selecting powerful words from a text is a crucial step for the success of the BoW model. In Visual retrieval, there is a similar model, which is the BoVW model, and it operates as same as the BoW model technique. In our work, we added a step for selecting powerful visual words after clustering. This step is built based on the FeatureWiz selection technique. FeatureWiz is used for the feature selection before clustering and to choose the most powerful visual words after the normalization step.The prediction model is applied using the state-of-the-art SVMs on the BoVW. The SVMs are initially proposed by Vapnik . The SVMIf If If If If The mathematical description of SVM has several approaches, such as optimal separating hyperplanes, linearly separable, linearly non-separable, and nonlinear SVM. In the proposed work, it is recommended to use linear SVM duo to many features as it is more likely that the data are linearly separable in high dimensional space. In Linear SVM, we want only to tune one parameter (c), which controls the amount of regularization applied to the data. A smaller value of C allows the SVM to be a soft classifier. As the value grows, the SVM becomes more and more strict, allowing for fewer misclassifications; however, if it grows too large, it could easily overfit the training data.This section illustrates the outcomes of the experiments and the discussion of the proposed KFFVW framework. The performance and accuracy are evaluated on the three standard datasets . The experiments were focused on the Corel-1000 dataset to select the best two keypoints detectors to apply fusion between them to form robust HKFV for the features’ extraction stage. The FAST and SURF keypoint feature detectors gave the best performance. The performances of each standalone FAST and SURF keypoint feature detector based on traditional CBIR were evaluated respectively on various vocabulary sizes compared to the performance of the proposed HKFV with the BFS for the proposed KFFVW framework. Every experiment was implemented five times based on K-fold Cross-Validation (KCV) to achieve the average performance. Furthermore, the performance of the proposed KFFVW was compared to the recent state-of-the-art methods on the specified datasets.The CBIR parameters are a primary factor for evaluating any CBIR system’s performance and running time. The parameters of the different experiments for the proposed framework are detailed below.The extracted features were used to construct the vocabulary. Using all the extracted features as input to the clustering step is not recommended. Datta et al.  reportedThe vocabulary size is the number of feature clusters. The vocabulary size is a primary key for evaluating any CBIR System. More extensive size vocabulary increases the system’s performance, but the massive vocabulary size tends to overfit and the processing cost and vice versa . VariousThe experimented datasets are designed in two groups; one is for testing, and the other is for training. The training group is selected based on five-fold cross-validation. The dataset is divided into 80% for the training process and 20% for the testing stage.The KCV is a simple approach for selecting the best model and determining an estimator’s accuracy . The priAccuracy: Accuracy relates to how close a measurement is to the true or recognized value. That means it is the probability that the CBIR framework test retrieves the correct images. It is one of the most intuitive and widely used performance measures.Precision (P): Precision relates to the capability to detect the true class among all predicted valid classes accurately. Moreover, it is expressed as a ratio of the correctly predicted valid classes to all anticipated true classes. High precision is always associated with a low false positive rate, as shown by the equation below.Mean Average Precision (MAP): Often known as the average precision (AP), it is a well indicator for assessing how well models execute tasks such as document/information retrieval and object recognition. The MAP of a set of queries is defined byRecall (R): Recall, also known as sensitivity, hit rate, or true positive rate, can be broadly considered as the ability of the model to identify all positive cases so that it can be expressed asIt should be noted that the above formula shows that high recalls are certainly associated with low false negative rates.F1-score (F): F1-score score is not as intuitive as accuracy, but it is beneficial when measuring the robustness and accuracy of a CBIR system. The F1-score, which serves as a single measure of test performance that describes both recall and precision, is usually calculated as a weighted average of recall and precision.The Corel-1000 dataset  is a subFor selecting the best keypoint detectors with the best features descriptor, each keypoint detector of the , DoG, SURF, CenSurE, Maximally Stable Extremal Regions (MSER), Binary Robust Invariant Scalable Keypoint (BRISK), and Oriented FAST and Rotated Binary robust independent elementary features (ORB)) with each features descriptor of the , and Binary Robust Independent Elementary Features (BRIEF) have experimented.The experiments were on the Corel-1000 dataset, where 2048 clusters were used to form the vocabulary set. Each experiment was tested five times with a different collection of images for the training (80%) and the rest for testing. Then, the average of all results was calculated. The performance comparison in The FAST and SURF keypoints were detected separately from each image in the dataset. Then, they were concatenated uniquely to prevent any point from duplication. Then the RootSIFT describes the detected keypoints. If an inquiry example image is given, its quantized features histogram will be predicted by the trained SVMs model. Moreover, it will be compared with the quantized features histograms of the images in the target dataset based on the Cosine distance similarity metrics. The result of the Top-20 retrieved images in response to the queries belonging to the Elephants class is shown in In The Caltech-10 dataset consists of 1000 images of 10 semantic classes, each containing 100 images. It is a subgroup of the Caltech-101 dataset, which includes 101 classes. In our work, the classes Faces, Leopards, Motorbikes, Airplanes, Bonsai, Brain, Car side, Grand piano, Ketch, and Watch were selected from the Caltech-101 dataset to construct the Caltech-10 dataset. To investigate the effect of the combination of BFS with the vocabulary size on the performance of the proposed framework, we have designed a combination between BFS and vocabulary size as mentioned in We computed the performance via various experiments of the proposed framework on the Oxford Flower-17 dataset on different combinations of BFS and vocabulary sizes. The results in In this experiment, we evaluated how well our framework performed using various numbers of images it returned. The classes of Sunflower, Pansy, Coltsfoot, Windflower, Tulip, Lilyvalley, Iris, Fritillary, and Buttercup yielded the highest results. In contrast, the classes Crocus, Bluebell, Cowslip, Daffodil, Dandelion, Snowdrop, Tigerlily, and Daisy yielded the lowest results because they have similar features and less dissimilarity over other semantic classes. The proposed framework attained a maximum mean average precision of 84.94% on the Oxford Flower-17 dataset. The comparison between the proposed framework and the recent CBIR approaches ,51,52 onIt was interesting to find the best combination between SIFT, FAST, SURF, and ORB to produce the HKFV. The performance and execution time comparison results between HKFV(SIFT+ ORB)+BFS, HKFV(SIFT+ORB+ FAST+SURF)+BFS, and HKFV(FAST+ SURF)+BFS were included as The tests and computations were performed on an AMD Ryzen 5 3400G with Radeon Vega 11 graphics (4 CPUs) 3.7 GHz, RAM 8 GB PC running Ubuntu 18.04 LTS 64bit.This paper has presented a novel CBIR framework based on visual word fusion of multiple feature descriptors to improve retrieval performance. The interest points have been separately extracted from an image using FAST and SURF. The retrieved keypoints have been combined into one keypoint feature vector, and the enhanced RootSIFT technique has been used to describe the area around each keypoint. The FeatureWiz selection technique has been applied to reduce and select the robust features for the BoVW learning model. K-means clustering has been used to quantify visual features into a more manageable set of visual words to produce the codebook. Finally, The SVM classifier for image retrieval has been fed by the feature vectors generated from the BoVW model. The obtained images have been sorted according to how closely they resemble the query image using an inverted index technique based on the cosine distance metric. By achieving average accuracies of 92.94%, 98.40%, and 84.94% on these benchmark datasets , experiments on the three benchmark datasets have demonstrated that the presented CBIR technique could deliver results that were comparable to other state-of-the-art methods. Nevertheless, a possible limitation of this study is that the validation process needs wider datasets. The future work will be in two main directions. The first one is applying advanced deep learning techniques for better retrieval similarity and accuracy as an adaptive version of the framework. The second direction will focus on performing more experiments for testing and evaluating our method for addressing content-based image retrieval systems and presenting it as an online API.


A new plasmonic configuration is proposed for application in a sensor and demonstrated for the detection of variations in the bulk refractive index of solutions. The configuration consists of monitoring two diffracted orders resulting from the interaction of a TM-polarized optical beam incident on a grating coupler, operating based on an effect termed the “optical switch”. The two monitored diffracted orders enable differential measurements which cancel the drift and perturbations common to both, leading to an improved detection limit, as demonstrated experimentally. The measured switch pattern associated with the grating coupler is in good agreement with theory. Bulk sensing is demonstrated under intensity interrogation via the sequential injection of solutions comprised of glycerol in water into a fluidic cell. A limit of detection of about  Biosensors and their performance characteristics have been the subject of vigorous global research for several decades. However, the COVID-19 pandemic highlighted and increased the need for portable and cost-effective biosensors to address challenges in large-scale testing and screening. Surface plasmon resonance (SPR) biosensors  and theiPortable and miniaturized surface plasmon (SP) sensors have been investigated and developed using, for example, configurations based on gratings ,10 and wRecently, a new surface plasmon detection configuration termed the “optical switch” ,24,25 waIn this paper, we propose a new, cost-effective, easy-to-implement plasmonic sensor based on an optical switch configuration, offering high detection performance, good miniaturization potential, and the ability to cancel common noise by a differential measurement. Our approach of producing a grating master then applying a replication process to produce the sensors is aligned with low-cost manufacturing.Acetone HPLC grade The grating masters were produced by laser interference lithography (LIL) . First,  Grating .LIL is a fast and cheap method to produce large-area periodic nanostructures. However, the profile of the photoresist produced by this technique becomes modified and damaged after a prolonged exposure to fluids encountered in biosensing, which compromises the reliability and the repeatability of the measurements. Thus, the master gratings produced in photoresist by LIL were replicated using a nanoimprinting process adapted from . FirstlyTo form the final grating couplers, chromium and gold layers were deposited sequentially on the amonil replicas, as shown in To perform the measurements, a grating was placed in a custom fluidic cell, as illustrated in The sensor sketched in To demonstrate the optical switch configuration for the detection of small RI variations, three solutions . The switching operation requires that the incident beam is simultaneously coupled to two SP modes, one propagating co-directionally and the other contra-directionally from the incident beam.The Ewald circles sketched in momentum a. The SP of −2KG b. The SPThe computed angular response of the optical switch is given in s method , by modeExperimental switch patterns were obtained using the set-up sketched in The theoretical response, plotted as the yellow curves, was also obtained with the software MC Grating, by modelling one period of the fabricated grating. The modelled period is highlighted by the yellow box in The slight mismatch between the three measured switch responses and the computed response can be explained in several ways. First, the software used to obtain the theoretical switch pattern assumes a perfectly uniform grating, infinitely periodic of the period extracted from the AFM profile. However, even though the grating is rather homogeneous, its period varies slightly over its length, as observed in SPs are sensitive to the cover medium bounding the metal surface; therefore, a change in the refractive index of this medium induces a change in the optical switch pattern, as shown in To demonstrate the optical switch as a piece of equipment to detect small RI variations, four solutions of different RI were sequentially injected into the fluidic cell. These solutions were mixtures with different proportions of water and glycerol, with a RI increment of 2 ×The power carried by the 0th and The angle of incidence was then adjusted to The differential voltage should vary linearly about each working point with the RI of the solutions  introduce common noise into the system, the interest in monitoring two orders and in differential detection becomes clear. Indeed, A new plasmonic sensor configuration based on the optical switch effect was successfully implemented and demonstrated by detecting variations in the bulk refractive index of solutions. The measured switch patterns of gold-coated replicated gratings agree very well with theory. The grating-based devices have the ability to cancel noise and drift due to common fluctuations by monitoring two complementary diffracted orders and taking their difference. An LOD in the range of


All-carbon systems have proven to present interesting transport properties and are often used in electronic devices. Motivated by recent resonant responses measured on graphene/fullerene junction, we propose coupled nanoribbons/carbon-nanotube heterostructures for use as charge filters and to allow tuned transport. These hybrid systems are engineered as a four-terminal device, and we explore multiple combinations of source and collector leads. The armchair-edge configuration results in midgap states when the transport is carried through top/bottom terminals. Such states are robust against the lack of perfect order on the tube and are revealed as sharp steps in the characteristic current curves when a bias potential is turned on. The zigzag-edge systems exhibit differential negative resistance, with features determined by the details of the hybrid structures. Molecular transport has been addressed recently as a valuable possibility for driving electrons in convenient device architectures ,2,3. FurComplex nanostructured materials have been proposed based on the coupling of graphene nanoribbons and other carbon allotropes ,14,15,16Previous theoretical works describing finite carbon nanotubes on a nanoribbon of the same width revealed Fano antiresonances and tuning conductance  when varHere, we adopt the scattering formalism to obtain transport transmission coefficients, assuming perfect matching between two nanoribbons (top/bottom) and a carbon nanotube inside them. Both zigzag and armchair edge geometries are considered. We follow a tight-binding approximation to describe the device. A multiple-mode approach is used in the case of armchair systems to analytically calculate the transmission function for the hybrid system. Standard real-space renormalization techniques are employed to obtain the system’s Green functions and electronic properties. Both results coincide perfectly well. The conductance for a finite CNT embedded into two graphene nanoribbons presents different behavior depending on the type of transport through the heterostructure. For in-plane transport, the conductance of the systems with armchair and zigzag graphene nanoribbons (AGNR and ZGNR) is similar to the case studied in our previous work . The conThe system is separated into two major parts, depicted in The central hybrid system and the leads are connected via a coupling Hamiltonian,  2.75 eV . A small 2.75 eV , t′=0.2tStandard numerical procedures based on real-space renormalization techniques are adopted to obtain the Green functions of the system recursively ,28,29,30Finally, to perform the calculation of the electronic conductance from the top left to bottom right leads, we use the energy-dependent transmission provided byConsidering two leads at different chemical potentials A and B sublattices is provided byt being the same hopping energy mentioned above and p=top and bottom. Taking into account the translation symmetry along the isolated ribbons, the electronic wave function may be written as q being defined by the lateral ribbon confinement [j in the A(B) nanoribbon sublattice. Using the above and considering a coupled finite carbon nanotube  between the ribbons of the same extension and chirality of the armchair GNRs, we obtainj are the junction sites at the top and bottom ribbons. For all other combinations, M, corresponding to a  tube.Alternatively, in the case of armchair nanostructures, we adopt multiple-mode approximation  for the finement , and ΦjAEquivalent equations can be written for the tube:Applying the boundary conditions for the tube atom positions closest to the up and bottom nanoribbons, In the following, we discuss the transport properties of such nanoscaled devices, considering both zigzag and armchair geometries. We focus first on the zigzag-edged systems. n = 10 and 12. Similarly to the zigzag systems, there is an important correspondence between the conductance depressions occurring in the top to top movement at the energies where localized states within the conductance gap are manifested in the top to bottom transport.Results for the conductance of metallic armchair hybrid systems, composed of 5-AGNR/CNT/5-AGNR, are shown in To highlight the origin of the antiresonances and resonances shown in In the following, we address current versus bias calculations for both zigzag and armchair configurations when the leads are submitted to a finite bias potential. First, we consider the zigzag-edged hybrid systems. Similarly to what was previously found in the case of a hybrid system composed of a nanotube on a single nanoribbon, negative differential resistances (NDRs) are verified at particular ranges of bias voltage. Depicted in the figure is an example of top/bottom current for q is a complex number with real and imaginary parts. Here, the Fano antiresonances appear due to the quantum interference between the continuum states in the graphene nanoribbons and the discrete states in the CNT.We explored the dependence of the current dip as a function of the bias voltage for different tube radii, fixing the nanoribbon width as 6-ZGNR and the size of the central conductor as As expected, the negative differential resistance is evidenced in different bias potential ranges depending on the details of the nanostructured systems  and on the local temperature . As the n = 10 and two different conductor spacers, Results for the current when the leads are submitted to a finite bias are shown here for the AGNR/CNT/AGNR devices for a (3p + 2) metallic family (with p being an integer). While the top to top current presents the same ohmic-like behavior due to the central plateau found for low energies, the top to bottom results exhibit a sequence of steps at bias voltages In fact, for the top/bottom transport corresponding to the first miniband, the transmission can be written as a sum of Breit–Wigner line shapes around the resonant energies By integrating over the energy range From the above equation, we can conclude that in the vertical transport through the carbon molecule, the transmission channels are provided by the eigenvalues of the finite armchair CNT and do not depend on the central conductor spacer provided by the To emphasize the difference between the T/T and T/B transport features, we plot in the inset of noribbon , and is Taking into account the fact that defects and disorder are in general unavoidable in practical materials, a further investigation of the transport effects due to breaking the perfect symmetry of the discussed armchair hybrid devices was performed as well. Isolated impurities or a set of substitutional impurities were considered through the nanotube walls by assuming different on-site energy values. In addition to a discrete lack of electron-hole symmetry and quite small energy shifts in the conductance results, this type of disorder does not reveal important changes in the step-like features of the current for such hybrid systems.In this study, we have addressed the electronic transport properties of an all-carbon structure composed of a CNT molecule sandwiched between two graphene nanoribbons. We have found that the T/T (or B/B) transport response for zigzag-edged junctions exhibits similar general behaviors to those found when the tube is on a single nanoribbon. The T/T (or B/B) transport reveals robust negative differential resistance at a given range of voltage values that may be controlled, as previously verified for simpler hybrid systems. The tube radii, nanoribbon width, and conductor spacers provide characteristic negative differential resistance details. The NDR is found in the T/B current in the zigzag geometry. In the case of metallic armchair nanoribbons, the transport from top to bottom (and vice versa) through the carbon molecule was proven to exhibit interesting features, such as the formation of midgap states, typical of a resonant-tunneling device. The positions of the conductance resonances for the vertical transport correspond to the CNT eigenenergies. The related characteristic I/V curves are marked by a sequence of plateaus occurring at the half value of the emergent resonant energies in the gap. Our findings indicate that the main responses are robust against disorder in the nanotube walls. The obtained fine-tuning of electronic transmission through the studied hybrid heterostructure allows us to predict interesting routes of tunneling transport, taking into account such all-carbon nanodevices.