BMC Medicine 16:112, 2018) found significant associations between depression and all-cause and cause-specific mortality. However, as the authors clearly argued, these results do not prove causation. The next logical step is to study the potential effect of depression and other mental disorders by placing emphasis on temporality (associations over time) and specificity (unique associations) of associations between a comprehensive set of mental disorders and somatic diseases. A data-driven approach in large samples could uncover disease development trajectories to provide a route for researchers and clinicians to improve medical outcomes in vulnerable patient groups.The question of whether depression is a causal factor in somatic diseases remains unanswered despite decades of research. In an extensive umbrella review of systematic reviews and meta-analyses, Machado et al. ( Depression is a major health problem, being among the top five causes of burden of disease in Europe and North America according to the Global Burden of Disease study . This hiBMC Medicine, Machado et al. ).Other approaches that can be taken to examine the likelihood of causality is to assess the specificity of associations and temporal associations between diseases . MachadoAnother important factor is the temporality, or order, of the associations. Depression predicts the development of somatic diseases, but these somatic diseases in turn also predict depression. Associations between depression and potential mediating mechanisms, e.g., obesity  and physResearch on temporality and specificity should now progress by taking multiple somatic diseases and mental disorders into account instead of limiting analyses to one outcome such as cancer or heart disease. A data-driven approach could be used to examine temporal and specific associations between depression, other mental disorders, somatic diseases, and mortality. For example, a study that applied this approach by examining temporal associations between disease diagnoses (mainly somatic) retrieved from electronic health registries covering large numbers of individuals found that gout plays a central role in the progression of cardiovascular diseases and highlighted the important role of retinal disorders as a marker of disease progression in individuals with diabetes . DiseaseIt is time to move forward from studying the association between one mental disorder and one somatic disease. These studies are unlikely to provide definitive answers since reality is much more complex. Instead, the temporal order of multiple mental disorders and somatic diseases, ideally including the mediating mechanisms, should be examined in concert. Findings from these studies could demonstrate linkages between specific diseases for which previous results have been inconclusive. Furthermore, trajectories that may be suggestive of specific disease development patterns could be uncovered. However, these studies in themselves are not capable of proving causality per se, but may provide persuasive results on the actual burden of disease resulting from depression and other mental disorders as well as important suggestions on when and how to intervene to reduce this burden.


Predictively successful list chunk combinations are selectively enhanced or suppressed via reinforcement learning and incentive motivational learning. Expected vs. unexpected event disconfirmations regulate these enhancement and suppressive processes. Adaptively timed learning enables attention and action to match task constraints. Social cognitive joint attention enables imitation learning of skills by learners who observe teachers from different spatial vantage points.This article develops a model of how reactive and planned behaviors interact in real time. Controllers for both animals and animats need reactive mechanisms for exploration, and learned plans to efficiently reach goal objects once an environment becomes familiar. The SOVEREIGN model embodied these capabilities, and was tested in a 3D virtual reality environment. Neural models have characterized important adaptive and intelligent processes that were not included in SOVEREIGN. A major research program is summarized herein by which to consistently incorporate them into an enhanced model called SOVEREIGN2. Key new perceptual, cognitive, cognitive-emotional, and navigational processes require feedback networks which regulate resonant brain states that support conscious experiences of seeing, feeling, and knowing. Also included are computationally complementary processes of the mammalian neocortical What and Where processing streams, and homologous mechanisms for spatial navigation and arm movement control. These include: Unpredictably moving targets are tracked using coordinated smooth pursuit and saccadic movements. Estimates of target and present position are computed in the Where stream, and can activate approach movements. Motion cues can elicit orienting movements to bring new targets into view. Cumulative movement estimates are derived from visual and vestibular cues. Arbitrary navigational routes are incrementally learned as a labeled graph of angles turned and distances traveled between turns. Noisy and incomplete visual sensor data are transformed into representations of visual form and motion. Invariant recognition categories are learned in the What stream. Sequences of invariant object categories are stored in a cognitive working memory, whereas sequences of movement positions and directions are stored in a spatial working memory. Stored sequences trigger learning of cognitive and spatial/motor sequence categories or plans, also called  An autonomous neural system for incrementally learning planned action sequences to navigate towards a rewarded goal.This article contributes to an emerging scientific and computational revolution aimed at understanding and designing increasingly autonomous adaptive intelligent algorithms and mobile agents. In particular, it summarizes an emerging neural architecture that is capable of visually searching and navigating an unfamiliar environment while it autonomously learns to recognize, plan, and efficiently navigate toward and acquire valued goal objects. This article accordingly reviews, and outlines how to extend, the SOVEREIGN architecture of Self-Organizing emphasizes that SOVEREIGN’s learning is carried out autonomously and incrementally in real time, using unconstrained combinations of unsupervised or supervised learning. Expectation refers to the fact that key learning processes in SOVEREIGN learn expectations that match incoming data, or predict future outcomes. Good enough matches focus attention upon expected combinations of critical features, while mismatches drive memory searches to learn better representations of an environment. Recognition acknowledges that SOVEREIGN learns object categories, or “chunks,” whereby to recognize objects and events. Emotion denotes that SOVEREIGN carries out reinforcement learning whereby unfamiliar objects can learn to become conditioned reinforcers, as well as sources of incentive motivation that can maintain attention upon valued goals, while actions to acquire those goals are carried out. Reinforcement learning also supports the learning of value categories that can recognize valued combinations of homeostatic drive inputs. Intelligent means that SOVEREIGN includes processes whereby sequences, or lists, of objects and positions may be temporarily stored in cognitive and spatial working memories as they are experienced in real time. Stored sequences trigger learning of sequence categories or plans, also called list chunks, that recognize particular sequential contexts and learn to predict the most likely future outcomes as they are modulated by reinforcement learning and incentive motivational learning. Goal-oriented navigation means that SOVEREIGN includes circuits for controlling exploratory and planned movements while navigating unfamiliar and familiar environments.The architecture was called SOVEREIGN because it describes how Self-Organizing, Vision, Expectation, Recognition, Emotion, Intelligent, and Goal-oriented Navigation processes interact during adaptive mobile behaviors. The term SOVEREIGN used these capabilities to simulate how an animal, or animat, can autonomously learn to reach valued goal objects through planned sequences of navigational movements within a virtual reality environment. Learning was simulated in a cross maze  that wasSOVEREIGN incrementally learned how to navigate to a rewarded goal object in this cross maze, which is the perhaps the simplest environment that requires all of the SOVEREIGN designs to explore an unfamiliar visual environment  while lelabeled graph of angles turned and distances that are traveled between turns. The angular and linear velocity signals that are experienced at such times are used in the model to learn the angles that a navigator turns, and the distances that are traveled in a straight path before the next turn.One key SOVEREIGN accomplishment is worthy of mention now because it illustrates how SOVEREIGN goes beyond reactive navigation to autonomously learn the most efficient routes whereby to acquire a valued goal, while rejecting less efficient routes that were taken early in the exploratory process. SOVEREIGN explains how arbitrary navigational trajectories can be incrementally learned as sequences of turns and linear movements until the next turn. In other words, the model explains how route-based navigation can learn a The prediction that a labeled graph is learned during route navigation has recently received strong experimental support in SOVEREIGN did not include various brain processes and psychological functions of humans that are needed to realize a more sophisticated level of autonomous adaptive intelligence. This article summarizes some of the neural models that have been developed to explain these functions, and that can be consistently incorporated into an enhanced architecture called SOVEREIGN2. These processes have been rigorously modeled and parametrically simulated over a 40-year period, culminating in recent syntheses such as what goes on in each of our brains when we consciously see, hear, feel, or know something; where it is going on; and why evolution may have been driven to discover conscious states of mind.The most important new perceptual, cognitive, and navigational properties emerge within feedback networks that regulate one or another kind of attention as part of resonant brain states that support conscious experiences of seeing, feeling, and knowing. These resonant states are modeled as part of Adaptive Resonance Theory, or ART. Additional processes in SOVEREIGN2 include circuits for target tracking with smooth pursuit and saccadic eye or camera movements (see section 3.2); visual form and motion perception in response to noisy and incomplete sensor signals (see section 4.13); incremental unsupervised view-, size-, and position-specific object category learning and hypothesis testing in real time in response to arbitrarily large non-stationary databases that may include unexpected events ; incremental unsupervised learning of view-, size-, and position- invariant object categories during free scanning of a scene with eye or camera movements ; selective storage in working memory of task-relevant object, spatial, or motor event sequences ; unsupervised learning of cognitive and motor plans based upon working memory storage of event sequences in real time, and Where’s Waldo search for currently valued goal objects (see sections 6.10 and 7); unsupervised learning of reaching behaviors that automatically supports accurate tool manipulation in space (see section 5.4); unsupervised learning of present position in space using path integration during spatial navigation (see sections 6.11 and 8); platform-independent navigational control using either leg or wheel movements (see section 5.6); unsupervised learning of adaptively timed actions and maintenance of motivated attention while these actions are executed (see sections 6.7 and 6.8); and social cognitive capabilities like joint attention and imitation learning whereby a classroom of robots can learn spatial skills by each observing a teacher from its own unique spatial perspective (see section 5.5).laws or equations-such as the laws for short-term memory or STM, medium-term memory or MTM, and long-term memory or LTM-and a somewhat larger set of characteristic microcircuits that embody useful combinations of functional properties-such as properties of cognitive and cognitive-emotional learning and memory, decision-making, prediction, and action. Just as in physics, only a few basic equations are used to explain and predict many facts about mind and brain, when they are embodied in a somewhat larger number of microcircuits that may be thought of as the “atoms” or “molecules” of intelligence. Specializations of these laws and microcircuits are then combined into larger systems that are called modal architectures, where the word “modal” stands for different modalities of intelligence, such as vision, speech, cognition, emotion, and action. Modal architectures are less general than a general-purpose von Neumann computer, but far more general than a traditional algorithm from AI.ART architectures embody key design principles that are found in advanced brains, and which enable general-purpose autonomous adaptive intelligence to work. These designs have enabled biological neural networks to offer unified principled explanations of large psychological and neurobiological databases . This essay illustrates that we do not need to start over.Perhaps these problems are why Geoffrey Hinton said in an 15, 2017  that he Section 17 in The original SOVEREIGN architecture contributed models of three basic design themes about how advanced brains work. The first theme concerns how brains learn to balance between reactive and planned behaviors. During initial exploration of a novel environment, many reactive movements may occur in response to unfamiliar and unexpected environmental cues . These mOne way that SOVEREIGN realizes a flexible balance between reactive and planned behaviors is its organization into parallel streams for computing visual form and motion. In fovea, with resolution decreasing with distance from the fovea  region. The MSTv cells are the ones that control the movement commands whereby the eyes pursue the moving target. When the eyes catch up to the target, they can maintain accurate foveation even in the absence of retinal slip signals, because background motion signals compensate for the reduced retina speed of the target, and can thus be used to accurately move the eyes in the desired direction at the target speed  cortical processing stream for spatial representation and action . Key proand direct appropriate goal-oriented actions toward them.One of several basic reasons for this particular kind of complementarity is that the cortical What stream learns object recognition categories that become substantially invariant under changes in an object’s view, size, and position at higher cortical processing stages, such as at the anterior inferotemporal cortex (ITa) and beyond . These iThe original SOVEREIGN model explained simple properties of how such invariant categories are learned as an animal, or animat, explores a novel environment. It used log-polar preprocessing of input images, followed by coarse-coding and algorithmic shift operations, to generate size-invariant and position-invariant input images. These preprocessed images were then input to a Fuzzy ART classifier  for learRecognition and Where’s Waldo search capabilities, which will be further discussed in sections 6.1 and 6.4, can also be incorporated into SOVEREIGN2 instead of the bottom two category learning processes in Since SOVEREIGN was published, the 3D ARTSCAN SEARCH model was developed to explain how humans and other primates may accomplish incremental unsupervised learning of view-, position-, and size-invariant categories, without any algorithmic shortcuts, and how these invariant categories can be used to trigger a cognitively or motivationally driven Where’s Waldo search for a desired object in a cluttered scene . These iThe ART in the Fuzzy ART algorithm abbreviates Adaptive Resonance Theory, which was introduced in 1976 ,b and dethought experiment whose simple assumptions are familiar to us all as facts that we experience ubiquitously in our daily lives. These facts embody environmental constraints which, taken together, define a multiple constraint problem that evolution has solved in order to enable humans and other higher animals to be able to autonomously learn to attend, recognize, and predict their unique and changing worlds. Such a competence is essential in autonomous adaptive mobile agents, which is why some ART algorithms were already algorithmically implemented in SOVEREIGN.ART’s significance is highlighted by the fact that its design principles and mechanisms can be derived from a stability-plasticity dilemma. This dilemma asks how can our brains learn quickly without being forced to forget previously learned, but still useful, memories just as quickly?One of the critical properties of ART that enable it to support open-ended incremental autonomous learning is that resonant states can trigger rapid learning about a changing world while solving the catastrophic forgetting problem was stated , and one-to-many maps .ART hereby predicts that interactions among CLEARS mechanisms solve the stability-plasticity dilemma. That is why humans and other higher animals are As will be explained in greater detail below, the link between Consciousness, Learning, and Resonance is a particularly important one for understanding both characteristically human experiences and how future machine learning algorithms may embody them.selectively process only those data that are predicted to be relevant in any given situation. Because of the CLEARS relationships, such selective attentive processing also solves the stability-plasticity dilemma.ART solves the stability-plasticity dilemma by using learned expectations and attentional focusing to top-down, modulatory on-center, off-surround network. This network is said to obey the ART Matching Rule.For this to work, the correct laws of object attention need to be used. ART has predicted how object attention is realized in human and other advanced primate brains e.g., , 1991. Imodulatory because the off-surround network also inhibits the on-center cells, and these excitatory and inhibitory inputs are approximately balanced (“one-against-one”). When a bottom-up input pattern and a top-down expectation are both active, cells that receive both bottom-up excitatory inputs and top-down excitatory priming signals can fire (“two-against-one”), while other cells are inhibited. In this way, only cells can fire whose features are “expected” by the top-down expectation, and an attentional focus starts to form at these cells. As a result only attended feature patterns are learned. The system wherein category learning takes place is thus called an attentional system.In such a network, when a bottom-up input pattern is received at a processing stage, it can activate its target cells, if nothing else is happening. When a top-down expectation pattern is received at this stage, it can provide excitatory modulatory, or priming, signals to cells in its on-center, and driving inhibitory signals to cells in its off-surround, if nothing else is happening. The on-center is The property of the ART Matching Rule that bottom-up sensory activity may be enhanced when matched by top-down signals is in accord with an extensive neurophysiological literature showing the facilitatory effect of attentional feedback e.g., . This prtop-down, adaptive, and specific event that activates its target cells during a match within the attentional system. “Adaptive” means that the top-down pathways contain adaptive weights that can learn to encode a prototype of the recognition category that activates it. “Specific” means that each top-down expectation reads out its learned prototype pattern. One psychophysiological marker of such a resonant match is the processing negativity, or PN, event-related potential ; medical database diagnosis and prediction; remote sensing and geospatial mapping and classification; multidimensional data fusion; classification of data from artificial sensors with high noise and dynamic range ; speaker-normalized speech recognition; sonar classification; music analysis; automatic rule extraction and hierarchical knowledge discovery; machine vision and image understanding; mobile robot controllers; satellite remote sensing image classification; electrocardiogram wave recognition; prediction of protein secondary structure; strength prediction for concrete mixes; tool failure monitoring; chemical analysis from ultraviolet and infrared spectra; design of electromagnetic systems; face recognition; familiarity discrimination; and power transmission line fault diagnosis. Some of these applications are listed at It is because the good learning properties of ART have been mathematically proved and tested with comparative computer simulation benchmarks that ART has been used with confidence in these applications e.g., , 1998.These theorems prove how ART can rapidly learn, from arbitrary combinations of unsupervised and supervised trials, to categorize complex, and arbitrarily large, non-stationary databases, dynamically stabilize their learned memories, directly access the globally best matching categories with no search during recognition, and use these categories to predict the most likely outcomes in a given situation.In particular, ART provably solved the catastrophic forgetting problem that other approaches to machine learning have failed to solve.1).ART offers a solution of another problem that other researchers in machine learning and AI are still seeking. The learned weights of the fuzzy ARTMAP algorithm  translatIn addition, ART can self-organize hierarchical knowledge structures from masses of incomplete and partially incompatible data taken from multiple observers who do not communicate with each other, and who may use different combinations of object names and sensors to incrementally collect their data at different times, locations, and scales . If swarIntelligent and Goal-oriented navigation processing whereby cognitive working memories  are manifestly complementary.Boundary completion and surface filling-in processes illustrate one of the best known examples of complementary computing : Boundarwhat happens in each brain and how and where it happens as it learns to consciously see, hear, feel, or know something, as well as for why evolution was driven to discover conscious states in the first place .A third design theme that is realized by the SOVEREIGN model is that advanced brains use homologous circuits to compute arm movements during reaching behaviors, and body movements during spatial navigation. In particular, both navigational movements and arm movements are controlled by circuits which share a similar mismatch learning law—called a Vector Associative Map, or VAM (target position vector T) and subtracts from it an outflow representation of where the arm is now . The resulting difference vector D between target position T and present position P represents the direction and distance that the arm needs to move to reach its goal position. Basal ganglia (BG) volitional signals of various kinds, such as a GO signal G, transform the difference vector D into a motor trajectory that can move with variable speed by multiplying D with G, before this product is integrated by P. Because P integrates the product DG, DG represents the commanded outflow movement speed. Then P moves at a speed that increases with G, other things being equal. As P approaches T, D approaches zero, along with the outflow speed DG, so the movement terminates at the desired target position.Neural models of arm movement trajectory control, such as the Vector Integration to Endpoint, or VITE, model  Figure   control of direction and distance that is gated by a GO signal can be used in either case.To help guide the development of a legged robot, neural network models have shown how leg movements can be performed with different gaits, such as walk or run in bipeds, and walk, trot, pace, and gallop in quadrupeds, as the GO signal size increases .An example of DV-GO control in a wheeled mobile robot was developed by In summary, both navigational movements in the world and movements of limbs with respect to the body use a difference vector computational strategy.Sections 6–8 provide a deeper and broader conceptual and mechanistic insight into the themes that the earlier sections have introduced.In vivo, view-, position-, and size-specific visual percepts in the striate and prestriate visual cortices V1, V2, and V4 are transformed into view-, position-, and size-specific object recognition categories in the posterior inferotemporal cortex (ITp) via feature-category resonances , with the PN and N200 ERPs illustrating these complementary properties. The orienting system can detect that an error has occurred, but does know what category prediction caused it. The attentional system knows what categories are active, but not if these categories adequately represent current inputs. By interacting, these systems can determine what the error is and discover and learn a new category to correct it. Complementary computing hereby accomplishes incremental learning and autonomous error correction of a large non-stationary database, without incurring the risk of catastrophic forgetting.In our brains, as ITp categories are learned using feature-category resonances, they create the substrate for learning view-, position-, and size-invariant object recognition categories within the ventral What cortical processing stream, notably in the anterior inferotemporal cortex, or ITa. The 3D ARTSCAN Search model has been incrementally developed to explain in detail how our brains learn to solve the invariant pattern recognition problem during active vision, a problem that is just as important for human survival as it is for designing machine learning algorithms that can autonomously learn in the real world . When itTo carry out effective invariant category learning, the model needed to solve a basic View-to-Object Binding Problem, which concerns how our brains automatically know, without external supervision or prior learning, which views of a novel scene belong to the same object-and thus can be associated with the same invariant category-and which do not-so should not be associated. As a result, the model can learn invariant object categories in response to arbitrary combinations of unsupervised and supervised learning trials as the eyes freely scan a complex scene.As ITp categories are learned using feature-category resonances , they arAn active surface-shroud resonance embodies the brain state that maintains spatial attention upon the object that is being learned about. While the object is attended, its shroud also inhibits category reset cells in SPL . While tWhen spatial attention shifts from the object, its shroud collapses, thereby disinhibiting the reset cells in SPL. A transient burst of inhibition from these SPL cells resets the active invariant object category in ITa . As the After invariant categories are learned, the system can also solve the Where’s Waldo Problem; that is, it can search a scene for a desired goal object within it. Such a search requires What-to-Where stream interactions.cognitive-emotional resonance that is triggered when a conditioned stimulus, such as a buzzer sound, activates its invariant object category while an unconditioned stimulus, or primary reward such as presentation of food to a hungry animal, activates its value category.Invariant object categories in ITa , and OFC that supports the cognitive-emotional resonance. This kind of resonance focuses motivated attention upon valued objects, while triggering context-appropriate actions toward them.gated dipoles  enables conditioning to bridge temporal gaps using a type of adaptively timed learning . Thus, Item-Order-Rank working memories can store sequences of events with repeats while satisfying the LTM Invariance Principle. They do so by preserving the relative activities of stored items as new items in a sequence are stored, even while the total activity of all stored items can change greatly through time.Even if a novel longer list like MYSELF could overcome competition from its familiar subwords, what would prevent its new learning from forcing catastrophic forgetting of the list chunks of its familiar subwords? A solution of this problem is said to obey the LTM Invariance Principle. Item-Order-Rank working memories solve the LTM Invariance Principle . They stall working memories, whether linguistic, motor, or spatial, were predicted to be realized by a similar kind of circuit. This circuit was shown to be a specialized version of a type of circuit that is ubiquitous in the brain; namely, a recurrent shunting on-center off-surround network, thereby clarifying how such a seemingly sophisticated design as a working memory could be discovered during evolution. Masking Fields are also recurrent shunting on-center off-surround networks, and thus are also working memories, albeit working memories that also represent list chunks.Because all working memories need to satisfy the LTM Invariance Principle, item-list resonance that dynamically stabilizes the bottom-up list chunk learning and the learning of the top-down expectations that enable list chunks to activate sequences of events in working memory for skilled performance. Item-list resonances hereby illustrate how ART dynamics solve the stability-plasticity dilemma in the temporal domain, and include predictions about the oscillatory dynamics, including gamma and beta oscillations, that occur during these resonances in primate brains.Feedback interactions between an Item-Order-Rank working memory and a Masking Field solve the Temporal Chunking Problem, and can do so under unsupervised learning conditions. These feedback interactions trigger an All of the predicted properties of Item-Order-Rank working memories have been supported by subsequent psychological data e.g., , 2007 anIn SOVEREIGN2, with item-list feedback signals implemented, each learned list chunk, or plan, can be selectively activated by motivationally salient sequences of previously experienced objects and positions/directions, and can then read out context-sensitive predictions of the objects and positions/directions that should be acquired next, thereby generalizing the SOVEREIGN interactions in entorhinal-hippocampal resonance that supports learning and stable memory of entorhinal grid cells and hippocampal place cells during spatial navigation that were mentioned in section 5.2. This kind of resonance will be discussed in section 8. It illustrates the claim that, although “all conscious states are resonant states,” the converse statement is not true. In order for a resonant state to become conscious, it is necessary for it to include either representations of external sensory cues, such as visual or auditory cues, or internal sensory cues, such as emotional cues.Yet another kind of resonance may be incorporated into SOVEREIGN2. This is the The kind of adaptive mobile intelligence that is exhibited by humans and other primates required a major expansion of the PFC to enable its working memory and planning networks to flexibly interact with multiple other brain systems, notably cognitive-emotional systems. The predictive ART, or pART, model  Grossbe has clarpART model explanations more fully embody and extend many of the processes that were included in SOVEREIGN, including how the value of visual objects and events is computed, which objects and events cause desired consequences and which may be ignored as predictively irrelevant, and how to plan and act to realize these consequences. To achieve this properties, pART includes reinforcement learning and incentive motivational learning; object and spatial working memory dynamics; and category learning, including the learning of object categories, value categories, object-value categories, and sequence categories, or list chunks. pART also explains properties that go beyond SOVEREIGN and other neural models, such as how to selectively filter expected vs. unexpected events to determine which events get stored in working memory, and how such filtering controls movements toward, and conscious perception of, expected events.Incorporating this level of sophistication in SOVEREIGN2 will require a coordinated research program. Here primarily the new competences will be reviewed of how events can be selectively filtered before being stored in working memory, and how that ability alters the understanding of how a top-down cognitive prime from the PFC can bias object attention in the What cortical stream to anticipate expected objects and events, while it also focuses spatial attention in the Where cortical stream to trigger actions that acquire currently valued objects .minimal anatomy that could carry out this function, while also simulating challenging reaction time (RT) data about visual search for target objects (As explained in greater detail in the pART model , after W objects . Such a  objects . HoweverIn the minimal anatomy of ARTSCENE Search, winning VLPFC activities send a top-down attentional prime to ITa using a circuit that obeys the ART Matching Rule. In order to transform the primed ITa cells into firing cells, an additional input must converge on ITa. This kind of signal is regulated by the BG (cf. BG in Multiple experiments show that selective working memory storage in the PFC does occur. The pART model offers an explanation of how this is predicted to work . For exabefore they reach the working memory, a mechanistically distinct processes enables all the items that get through the filter to be stably stored after they reach the working memory; namely, keeping an SNr gate open to enable the recurrent excitatory connections within PFC to maintain working memory storage. Closing this SNr gate can rapidly reset, or delete, the entire stored sequence from working memory when there is an attention shift to do a different task.In addition to this kind of task-sensitive filtering of individual items The property of selective working memory storage clarifies the functional role of neurophysiological data about the role of VPA as “a source for feature-based attention” , p. 832,pART proposes the following mechanistic and functional explanation of how these cells interact together to enable matched objects to be selectively processed and stored by PFC : Both ITActive VLPFC top-down signals project to both VPS and VPA, and learn modulatory top-down expectations when VPS and VPA cells are also active. In pART, these expectations obey the ART Matching Rule that is realized by a top-down, modulatory on-center, off-surround network.VPA cells that receive a previously learned VLPFC-to-VPA prime are enhanced when an extrafoveal object matches its target features, and are suppressed when the object mismatches them, properties that are consistent with the ART Matching Rule. This enhanced VPA activity is sufficient to trigger an output signal to FEF at the corresponding FEF positional representation in FEF. This property is supported by A similar match-mismatch dichotomy regulates the activity of VPS cells when they receive an active VLPFC-to-VPS prime. Their activity is enhanced when an ITa invariant object category matches their receptive field, and are suppressed by a mismatch, again consistent with the ART Matching Rule. When a match occurs, a synchronous VPS-ITa resonance develops that enables the category’s temporal order to be stored in VLPFC. This resonance can also propagate top-down through multiple cortical areas (e.g., ITa-ITp-V4-V2-V1 in Section 5.2 noted that a representation of an animal’s Present Position Vector, or NET, as it navigates in space is derived in SOVEREIGN from an algorithm that computes a head/body turn angle as well the length of the next straight distance that is navigated. That section also noted that, in order for NET to be computed without algorithmic short cuts, an animal or animat needs to learn a representation of its present position in space as it navigates in different environments. The GridPlaceMap model of spatial navigation  proposesThe GridPlaceMap model and its variants have explained and simulated many behavioral and neurobiological data about spatial navigation and how its circuits learn and remember e.g., . A comprThe model responds to realistic rat navigational trajectories by learning both grid cells with hexagonal grid firing fields of multiple spatial scales, and place cells with one or more firing fields, that match neurophysiological data about their development in juvenile rats. The fact that individual grid cells can fire at positions on a hexagonal lattice when rats navigate in an open field is one of the most remarkable facts in contemporary neuroscience . The Grisame self-organizing map model equations can learn both grid cells and place cells. The different response properties seem to arise entirely due to their different stages of processing in a hierarchy of self-organizing maps (least common multiple of the grid cell scales that input to it. Thus grid cells that respond on a centimeter scale can support learning of place cells that can represent spaces that are many meters in size.Other properties of the GridPlaceMap model are also worth summarizing both because they are so parsimonious and data-predictive, and because they will simplify their embodiment in SOVEREIGN 2. For example, the ing maps . In thising maps . Place cring attractor networks are used to convert vestibular angular velocity signals into responses of head direction cells, and linear velocity signals into responses of stripe cells (medial entorhinal cortex to HIPPO, with the largest grid and place cell spatial scales occurring at ventral positions. Spatial learning hereby converts slower cell response rates into larger learned spatial scales. Temporal learning along the dorsoventral axis passes through the lateral entorhinal cortex to HIPPO, with the longest time intervals spanned at the most ventral positions in this gradient. Temporal learning uses spectrally timed conditioning with cells in the spectrum responding more slowly at more ventral positions (Parsimonious properties also occur at the earliest stages of the GridPlaceMap model. For example, similar pe cells . Both spositions .spectral spacing to match the term spectral timing. The computational homology between them is called neural relativity.This computational homology provides a harmonious explanation of why both spatial and temporal representations occur in the entorhinal-hippocampal system. Many challenging neurophysiological data are explained by this homology between spatial learning in the medial entorhinal-hippocampal system and adaptively timed temporal learning in the lateral entorhinal-hippocampal system e.g., . When coThe top-down hippocampus-to-entorhinal attentional network that stabilizes map learning uses the same ART Matching Rule that stabilizes learning of all ART circuits, including object categories learned via a feature-category resonance , 9. In tThis article summarizes basic design principles, networks, and functional capabilities of the SOVEREIGN architecture  and outlSOVEREIGN was designed to serve as an autonomous neural system for incrementally learning planned action sequences to navigate toward a rewarded goal. SOVEREIGN also illustrates how brains may, at several different organizational levels, regulate the balance between reactive and planned behaviors, and proposes how homologous circuit designs regulate spatial navigation and reaching behaviors. These capabilities were demonstrated by learning efficient routes whereby to navigate to a valued goal in a virtual reality environment.Some of the designs in SOVEREIGN were realized algorithmically, and can be realized dynamically in SOVEREIGN2. Other processes that are needed to achieve a more comprehensive autonomous adaptive intelligence in an embodied mobile system were not included at all. This article summarizes neural models of important missing capabilities with enough detail to define a research program that that can consistently incorporate them into SOVEREIGN2. Missing designs occur across both the What and Where processing streams of SOVEREIGN e.g., .In order to include these missing designs, SOVEREIGN2 embodies foundational brain design principles such as complementary computing, hierarchical resolution of uncertainty, and adaptive resonance that enable biological brains to realize their autonomous adaptive intelligence. Some of the missing designs in the What stream occur at early processing stages, such as visual boundary completion and surface filling-in. These processes require hierarchical resolution of uncertainty to be completed. How this occurs sheds light on deep computational reasons for how and why animals like humans and other primates become conscious in order to generate effective actions.Other missing What stream processes occur at higher processing stages, such as autonomous learning of view-, position-, and size-invariant recognition categories. Such invariant learning requires modulatory interactions from parietal regions of the Where cortical stream to inferotemporal regions of the What cortical stream in order to ensure that only views of a single object get bound together by associative learning in a single invariant object category. The surface-shroud resonances that support invariant category learning also play a role in enabling social cognitive skills such as joint attention and imitation learning to occur between a teacher and a student who experience the world through different spatial perspectives.Still higher levels of processing have parallel object and spatial processing systems in both the What and Where cortical streams. For example, prefrontal object and spatial working memories need to be able to selectively filter targets from distractors before storing them and their target positions in working memory. The filtering machinery that does this also allows attention to be paid to salient targets, and to use those targets to drive orienting movements toward them.Cognitive-emotional circuits are needed to enhance predictions and actions that lead to valued outcomes, and to attenuate those that do not. In order to do this effectively, cognitive-emotional learning needs to be able to associate sensory and rewarding cues that are separated in time. Spectral timing circuits in the HIPPO help to support cognitive-emotional learning in inferotemporal-amygdala/hypothalamus-orbitofrontal circuits. These circuits, in turn, amplify or suppress cognitive and spatial working memory circuits and plans according to whether they generate successful goal-oriented actions or not.Although looking and reaching behaviors can use target position and present position estimates that can both be directly computed from either external sensory cues or internally generated movement commands, navigational movements need more sophisticated networks to learn a navigator’s present position in space. Entorhinal grid cells and hippocampal place cells interact to incrementally learn place cells that can represent spatial scales that are sufficiently large to support navigation in ecologically relevant spaces. These learned spatial categories are dynamically stabilized using the same Adaptive Resonance Theory, or ART, Matching Rule that is found in the resonant dynamics of many of the missing competences from which SOVEREIGN2 can benefit.These resonances include feature-category resonances, surface-shroud resonances, cognitive-emotional resonances, entorhinal-hippocampal resonances, and item-list resonances. All of these resonances help to dynamically stabilize the learned memories of their respective networks, and thereby enable them to successfully operate in open-ended non-stationary environments without experiencing the learning and forgetting problems, notably catastrophic forgetting, that plagues all algorithms of back propagation type, including the currently popular and useful Deep Learning algorithms, and Bayesian Explaining Away algorithms, among others.The author confirms being the sole contributor of this work and has approved it for publication.The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Immunotherapy holds tremendous promise as a strategy for eradicating solid tumors. However, poor T cell infiltration and persistence within most solid tumor microenvironments, as well as mechanisms of adaptive resistance, continue to severely limit the accessibility of most immunotherapies to a broad patient population. This limitation perpetuates the demand for allied therapeutic strategies. Among such strategies is focused ultrasound (FUS), a non-invasive, non-ionizing technique for precisely targeted acoustic energy deposition into tissues. FUS has gained remarkable attention over recent years as a modality for elicitation of immune mechanisms in cancer and other pathologies. In 2017, we published a comprehensive review paper detailing existing evidence for immune modulation and therapy with FUS, as well as impending challenges and opportunities of consideration for the field. Over the last two years, a multitude of clinical trials have come online to explore safety, feasibility, and efficacy of FUS for cancers of the brain and periphery - including the first clinical trial to combine FUS with immunotherapy. Moreover, the last two years have seen a surge in FUS immunotherapy presentations at therapeutic ultrasound scientific meetings. Given the burst of activity in this field, we submit that an update on FUS immunotherapy progress is timely. In this review, we offer an updated overview and perspectives on scientific and clinical development in the FUS immunotherapy domain. Because immunity plays such a pivotal role in tumor evolution, it holds that immunotherapies have tremendous promise as strategies for tumor eradication. Recent decades have seen substantial advancement in immunotherapy interventions capable of eliciting unparalleled therapeutic outcomes; such strategies include checkpoint blockade (e.g. anti-PD1) Theranostics that covered the novelty and significance of FUS for immunotherapy as well as the state of the field, with a focus on challenges and opportunities in the central nervous system (CNS) In 2017, we (NDS and RJP) served as co-authors on a comprehensive review article in th International Symposium on Focused Ultrasound (October 2018) and the most recent International Society for Therapeutic Ultrasound Annual Symposia . Despite these meetings being relatively small, ~50 abstracts on FUS immunotherapy were presented. In this brief review, we will offer updated perspectives on progress within the field, with an emphasis on studies performed since our 2017 review in Theranostics.Since the time of our last review, a handful of findings have emerged in the literature that will be discussed herein. However, it is worth emphasizing what is not immediately evident from the literature; while many groups are working in this space presently, publications from this new wave of research are just now beginning to emerge. FUS immunotherapy has gained notable traction as evidenced by the number of oral and poster presentations at international meetings including the 6oC, leading to nearly instantaneous onset of coagulative necrosis in the focal zone and thermal stresses in the periablative zone FUS is a non-invasive, non-ionizing technique for high-density acoustic energy deposition. It has gained legitimacy and momentum as an approach for targeted tumor disruption via thermal and/or mechanical mechanisms, which are comprehensively detailed in other reviews to which interested readers are herein referred The concept of FUS immunotherapy finds its roots in the supposition that these regimens are distinctly capable of stimulating anti-tumor immune mechanisms that include tumor-specific inflammation, tumor-associated antigen and alarmin liberation, cytokine modulation, leukocyte infiltration and activation, and/or curbing of immunological tolerance. Figure Transcranial FUS has been explored as an intervention for a variety of neurological pathologies including neurodegenerative diseases, primary and metastatic brain tumors, neuropathic pain, and psychiatric disorders Ultrasound treatment has been demonstrated to confer clearance of amyloid plaques in transgenic models of Alzheimer's disease, independent of drug or gene delivery These studies come at a time when perspectives framing the CNS as a site of “immune privilege” are rapidly evolving into ones that instead consider the CNS to be “immunologically distinct.” With the continued emergence of discoveries that connect the CNS and immune system, the tractability of leveraging FUS for immunotherapy in the brain is clearly growing.Since the benefits of immunotherapy have been realized more extensively for extracranial tumors, new and ongoing clinical trials utilizing FUS in the treatment of neoplasms outside of the CNS will also be vastly informative to the development of FUS immunotherapy. Beginning in 2014, a clinical trial at the University of Virginia investigated the safety and feasibility of thermally ablating benign breast fibroadenomas using the CE-marked Theraclion Echopulse system (ultrasound-guided FUS device) (NCT02078011) Owing to this previously established use of Theraclion's Echopulse system at the University of Virginia, a FUS immunotherapy milestone was achieved in 2017 with the initiation of a clinical trial evaluating the combination of FUS thermal ablation with pembrolizumab in metastatic breast cancer patients NCT03237572). Although the trial has not yet fully accrued, several women have been treated thus far. The design of the trial is shown in Figure 237572. AA handful of recent studies have probed relationships between various mechanical FUS parameters for BBB opening and the elicitation of local inflammatory responses. As the field of FUS-immunotherapy advances into the treatment of CNS neoplasms, these studies will likely provide valuable insight into how mechanical FUS may be tuned to possibly synergize with immunotherapies.In 2016, it was reported that FUS-mediated BBB opening is capable of eliciting sterile inflammation in naïve brain During and after the publication of these sterile inflammation studies, other investigators published studies emphasizing how these effects are dependent on sonication parameters (e.g. acoustic pressure), as well as other parameters such as MB formulation, choice of anesthesia, carrier gas In particular, MB dose is known to be a critical parameter for instigation of neuroinflammatory effects in the brain. In a study comparing the recommended MB dose for clinical imaging with a dose 10x greater, FUS BBB permeabilization of rat dorsal hippocampal microvessels with the higher MB dose acutely induced elevated transcription of proinflammatory cytokine genes  - all of which was mostly resolved by 24 hrs. EC activation and astrocyte activation were noted via Sele and Gfap gene expression, respectively. Interestingly, there was no elevation in gene expression for endothelial adhesion molecules Vcam1 or Icam1 - ligands key to leukocyte extravasation out of the vasculature - at the time points evaluated Taken together, these studies provide evidence that certain signatures of immune activation can occur in the brain with FUS. Factors such as sonication parameters, MB dose, anesthesia, model, MB formulation, and timing of onboard therapies relative to treatment are thus important considerations for those conducting FUS immunotherapy studies. Despite the promise of FUS to lift barriers to effective cancer immunotherapy applications in the brain, there remains an unresolved contention between maintaining safety to healthy tissue and causing immunogenic “damage” to malignant tissue. It is possible that with appropriately tuned exposure conditions, acoustic energy can be leveraged to better enable immunotherapies not only via improved delivery, but also via favorable modes of immune modulation. It is of note, however, that the capacity of FUS to elicit adaptive resistance mechanisms or modulate immunosuppressive cell populations remains poorly understood across tumor types Beyond the brain, a handful of notable strides have been made to advance the combination of FUS with immunotherapies in the periphery. For example, the combination of toll-like receptor agonist CpG, anti-PD1 and FUS thermal ablation in murine B16 melanoma was shown to dramatically increase local and systemic immunity compared to monotherapies Sonodynamic therapy (SDT), an emerging technique for exerting tumor cell death through combination of low intensity ultrasound with chemical sonosensitizers, is under investigation as a strategy for cancer vaccination. A combinatorial therapeutic paradigm employing SDT with sonosensitizers/imiquimod-loaded liposomes and anti-PD-L1 has recently been demonstrated to exert efficacy against 4T1 mammary and CT26 colorectal carcinomas Lastly, a couple of recent investigations into the immunological impact of mechanical acoustic energy deposition have emerged over the last two years. Boiling hisotripsy (BH) has been demonstrated to promote transient elevations of plasma and intrarenal TNF, high-mobility group box 1 (HMGB1), IL-10, and IL-6 in a rat model of renal cell carcinoma The above highlighted studies represent some of the earliest published efforts to combine FUS with immunotherapies such as checkpoint blockade across a diverse array of physical regimens. Publications emerging in the field have called immunological priming to attention as a means towards achieving effective FUS immunotherapy. Notably, these studies arrived at priming as an important consideration when coincident immunotherapy protocols did not yield similar success To date, no studies have evidenced the capacity of FUS monotherapy to achieve immunological tumor control independent of adjuvant therapy. Through appropriate tuning of exposure conditions and comprehensive immunological characterization, the prospect of unmasking the utility of FUS as an immunomodulatory monotherapy may be attainable. Since FUS immunotherapy is in its earliest stages, it is paramount to the field that such formative variables as immunotherapy timing, FUS exposure conditions, choice of immunoadjuvant, and differential response across tumor models be resolved over the coming years. We are optimistic that with the continuation of pre-clinical development and clinical translation at the current pace, FUS will surface as a transformative modality for combination with immunotherapies.


Big Five personality traits are assumed to be linked with attitudes toward own aging (ATOA). Both constructs have central importance for the aging process, it is thus important to comprehensively address their mutual connection over time. We used data from the ILSE study, a longitudinal study with four measurement occasions, spanning 20 years and including two participant cohorts . Dual latent change score models showed that personality was longitudinally related to change in ATOA: Lower Neuroticism, higher Conscientiousness, and higher Openness predicted more positive attitudes; the effect for Extraversion varied by time. Furthermore, the role of personality seems to be confined to certain sensitive periods in midlife and early old age. ATOA had only marginal longitudinal impact on personality. Our results shed light on the developmental co-dynamics of personality and subjective perceptions of aging across the second half of life.


As a synergistic integration between spectroscopy and imaging technologies, spectral imaging modalities have been emerged to tackle quality evaluation dilemmas by proposing different designs with effective and practical applications in food and agriculture. With the advantage of acquiring spatio-spectral data across a wide range of the electromagnetic spectrum, the state-of-the-art multispectral imaging in tandem with different multivariate chemometric analysis scenarios has been successfully implemented not only for food quality and safety control purposes, but also in dealing with critical research challenges in seed science and technology. This paper will shed some light on the fundamental configuration of the systems and give a birds-eye view of all recent approaches in the acquisition, processing and reproduction of multispectral images for various applications in seed quality assessment and seed phenotyping issues. This review article continues from where earlier review papers stopped but it only focused on fully-operated multispectral imaging systems for quality assessment of different sorts of seeds. Thence, the review comprehensively highlights research attempts devoted to real implementations of only fully-operated multispectral imaging systems and does not consider those ones that just utilized some key wavelengths extracted from hyperspectral data analyses without building independent multispectral imaging systems. This makes this article the first attempt in briefing all published papers in multispectral imaging applications in seed phenotyping and quality monitoring by providing some examples and research results in characterizing physicochemical quality traits, predicting physiological parameters, detection of defect, pest infestation and seed health. The last couple of decades have witnessed growing concerns from governmental agencies to enhance seed quality parameters because without good seeds investments in fertilization programs, integrated pest management programs, water and other inputs won’t be worthwhile. Providing high quality seeds is very important for agriculture and seedling progression because seed quality is strongly related to the resistance to biotic and abiotic stresses, to the germination rate and to the plant performance . DespiteIn most quality control programs, the evaluation of seed quality employing efficient and reliable methods for continuously checking fresh and stored seeds to explore their physicochemical properties, purity, germination, vigour, freedom from debris, dead or broken seeds and presence of pests, moulds or any other deterioration symptoms is an essential step. However, these continuous checking processes are not an easy task, particularly in the case of insects that are living and may be concealed inside the seeds with no external symptoms or signs of infestation. Suppliers tend to adopt reliable testing scenarios to provide trustful information about seed quality parameters. Some of this information is very easy to achieve, while other information related to viability, damage, vigor and health is very difficult to obtain directly due to time and technological constraints. Seed testing is generally carried out by detecting various physical, biochemical, physiological biomolecular markers which correlate well with the known seed constituents . For impMost of the methods employed in seed quality evaluation provide information about the overall quality of batches or bulks of seeds without referencing variation among single seeds that could be considered very important in plant breeding programs. For instance, a high vigor seed lot should hypothetically be homogeneous and produce a very narrow range of physicochemical and physiological properties, but under the effect of varied storage conditions, pathogen contamination or pest infestation, the overall quality of individual seeds within a seed lot will not be affected to the same extent, resulting in a wide spectrum of quality within this lot. Detecting the variations among seed lots and among individual seeds within a seed lot has great importance for both the seed industry and markets. Regrettably, the physiological and biochemical variation existing among individual seeds can’t be figured out from pooled seed samples, especially when some of seeds in the lots are dormant . ActuallRecently, imaging techniques are gaining great attention in the seed industry for quality assessment and monitoring . They weThe ideal seed inspection protocols should include methods for seed measurements, field inspection, testing and retesting as well as seed certification. Seed testing is the process during which the essential characteristics of a seed lot in terms of varietal purity, moisture, germination, vigour are determined to enable the farming community to get high quality seeds. The ordinary practices of seed quality evaluation are usually performed by germination tests and can be followed by vigor tests and seedling growth characteristic measurement, especially because germination test might overestimate the seed potential . The meaFor increasing the reliability and reproducibility, accurate and faster quality assessment methods with minimum human intervention are basically required to provide the best levels of seed quality for production and trade purposes ,19. TherOn the other hand, spectrometric technologies have been employed in determining various seed quality traits due to their practicality and ability to explore internal properties of biological materials. Depending on the frequency or the wavelengths used, the electromagnetic waves may be reflected, absorbed or transmitted through such materials, offering potential quantifications of many quality traits simultaneously . This kiRecently, optical sensing techniques, including spectroscopy and imaging systems, have been widely utilized for quality and safety evaluation of seeds. Systems based on spectroscopy provide spectral data either by spectroradiometry or spectrophotometry measurements. In spectrophotometry, the light reflected from a sample is related to the incident light as a measure of the spectral reflectance for the sample. In the case of spectrophotometers, a light source is also required, most commonly tungsten-halogen or xenon lamps . In receHowever, the spectral information extracted by this method is presumably limited to only a little portion of a sample where the measuring probe is positioned, without considering the spatial information. Thus, the sample being analyzed should be reasonably homogeneous in order to extract representative spectral fingerprint of the whole sample. This disadvantage of conventional spectroscopy can be easily alleviated by combining both spectral and spatial information by using hyperspectral (HIS) and/or multispectral (MIS) imaging techniques . In addiThe unbridled desire of seed service centers to provide seed markets with high quality criteria seeds requires more concerted efforts and cooperative actions to develop advanced and innovative technological systems to provide unique quality that can satisfy all stakeholders’ expectations. The use of high speed digital image detection and computerized data processing to improve traditional seed quality analysis is increasing. The decreasing cost and increasing power of computer hardware and software packages for image processing have made image analysis systems more attractive in automatic inspection of different aspects of quality evaluation and sorting processes . The impAlthough conventional colour imaging systems can provide a number of essential quality traits for each individual seed within a seed lot such as area, perimeter, length and width, shape, surface colour, such a system is unable to provide detailed information about the chemical composition, structure and other invisible features of the seeds. On the other hand, spectral imaging systems in the visible to near infrared spectral range 400–2500 nm) have facilitated the development of imaging platforms having the capability for rapid quality assessment of grains and seeds based not only on the visual features of the seeds, but also on their intrinsic physicochemical characteristics. The correlation of these parameters with data obtained by traditional methods may also indicate the potential of these computerized imaging systems to generate new seed germination and vigour markers, suitable also for purity analysis and sorting 00 nm hav. Owing tBuilding an integrated computer-aided image analysis system for overall quality evaluation processes of seeds could be realized by combining colour imaging and spectral imaging in one system . The cola and chlorophyll b simply look green in RGB images although they are distinctly different in terms of their spectral patterns. Metameric colours (metamers) represent a severe problem in many applications. For instance, in seed health diagnosis, the traditional trichromatic camera or the human eye are unable to discern the healthy and diseased tissues when spectral variations between these two classes are small and the appearance of both tissues looks the same under certain illuminating conditions. There have been numerous research attempts to utilize color as the only parameter for distinguishing seeds of different quality grades [Similar to the human eye, traditional colour imaging known also as RGB imaging uses three broadband colour channels  to produce a single color value for each pixel in the image . In the y grades ,45. Basediscrete’ wavelength bands spaced over an extended spectral range from ultraviolet (UV) to near infrared (NIR) region  region . The resanalyzed . BesidesWhen spectral images are acquired for very narrow spectral bands rather than discrete bands, the system is called hyperspectral imaging. The main difference between multispectral imaging and hyperspectral imaging techniques is number of wavebands at which the spectral image is acquired. In a hyperspectral imaging scenario, the spectral image is recorded by utilizing a great number of wavebands leading to a continuous spectral range. On the contrary, the spectral image is recorded for a few discrete bands in the case of multispectral imaging . Thus, hDue to the fact that all spectral imaging systems  are required to gather information from the target objects in the spatial and spectral domains, it is very important to supply such systems with an acquisition mechanism to scan the sample in both directions simultaneously and to make the light sensor records light intensity at every pixels at every wavelengths. Based on the applications and the available optical devices, there are three methods of acquiring spectral imaging data sequentially, namely point-scanning (whiskbroom scanning), line-scanning (push-broom scanning) and wavelength-scanning (area or plane scanning) methods. The first acquisition method depends on acquiring the data point by point which required to systematically scanning the target seed samples in both X and Y directions as shown in data cube’ or ‘spectral cube’ x, y) and one spectral dimension (λ) as shown in i index corresponds to the first spatial direction, the j index corresponds to the second spatial dimension, and the k index represents the spectral dimension. For example, a scene with 1024 × 1024 spatial samples and 20 wavelength bands in the visible range (380–780 nm) will constitute a 1024 × 1024 × 20 data cube, meanwhile an RGB image of the same scene is a 1024 × 1024 × 3 data cube revealing that a great deal the spectral information of the scene being images is lost in case of the later one.Generally, at the end of ‘one scan’, the point scanning method provides a full spectrum for only ‘one point’ of the target sample d and theThe acquired spectral image has plethora of data and the formed data cube could be viewed either in the spatial or the spectral domains . As showBased on the relative position of the light source to the detector , spectral images could be acquired in reflectance, transmittance or interactance modes based on the way by which the light beam is received by the detector after interaction with the examined sample . Reflectdiscrete’ wavelength bands. Instead of using filter wheel or tunable filters, the approach of using various illuminants of different spectral power distribution depends on ‘sequentially’ illuminating the scene with different narrow-band wavelength light sources, which can reduce or eliminate light scattering caused by non-flat or irregular surfaces. As each discrete wavelength light source illuminates the sample, a 2D greyscale image is acquired using a camera with a uniform wavelength response. Because the number of wavelengths is limited, this approach is only suitable for acquiring multispectral images. In this approach, instead of using tungsten- halogen lamps, multispectral images could be acquired by illuminating the sample with narrow-wavelength-range light emitting diodes (LEDs) of known spectra. Contrary to the single broadband light source of fixed emission characteristics , using strobed LED illumination at multiple specific wavelengths allows the illumination conditions for each different sample type to be optimized. By taking greyscale images under the illumination of these LEDs in turn, each at a successively longer wavelength, one can build up a set of registered images that contain more information than a simple RGB image under white-light illumination. During operation, the object is lit with a set of distinct LED illuminations and a synchronized monochromatic or RGB camera captures the corresponding images. Most multispectral LED imaging systems, with a few exceptions, often use monochromatic cameras owing to its high sensitivity compared to the colour camera. By activating the illuminations one by one and concatenating the acquired imaging synchronously, a multispectral image of the scene is obtained. As the cost of such a system is presumably low; producing commercial LEDs at different wavelengths throughout the electromagnetic spectrum has been gradually increased over the last 20 years. In a very short time, LED lighting has widespread in agricultural, industrial and household settings, which opens new venues of real applications of high-power narrow-waveband LED devices. In fact, it is easier to create an illumination unit having fast changing spectra than a camera having changing spectral sensitivity. Thus, compared with the optical filter configuration that requires longer acquisition time for changing the filters in the filter wheel or in the tunable filter device, multispectral imaging design supported with different LED illuminants requires shorter acquisition times and best suited for moving objects [In fact, the lighting is considered as significant as the optical parts, because it has the essential role in conveying the primary information of the inspected specimens. Hence, changing light geometry may lead to different images even when using the same camera. A well-designed illumination system helps in improving the accuracy of the imaging system, increasing contrast in the acquired scenes, reducing the time and complexity of the image analysis steps, leading to successful image analysis, and decreasing the cost of the image processing routines . Althoug objects .mnI can be modeled as a radiometric transfer process measured at a pixel  in the mth channel and nth illumination and is given by the following equation:S(λ) is the spectral reflectance of the scene point, mc(λ) is the spectral response of the camera at mth color channel, and np(λ) is the spectral power distribution of the nth illumination. If a monochromatic camera was used with only one channel (m = 1), a multispectral image with n sub-images would be obtained and when an RGB camera was used with three color channels (m = 3) and n illuminations, then by stacking the n images together, a multiplexed multispectral image with 3 × n sub-images would be obtained. The spectral reflectance S(λ) of a point in the scene is determined by the imaged material itself that characterizes the property of the object. If the scene’s spectral reflectance is given, it is possible to calculate the imaging result under different combination of any type of camera and light source [fluorophores  such as chlorophyll or fluorescent pigments in seed samples [From mathematics point of view, a multispectral signal at any point in a scene t source . Therefot source . More in samples ,66.In recent years, great attention has been paid to this type of LED illumination-based multispectral imaging system due to its robustness, rapid controlled switching ability and cost effectiveness ,74. The seen’ otherwise. Analyzing complex spectra having relatively weak or overlapping bands poses a great challenge [chemometric data analysis can underpin the dominant patterns in large data matrices in a fast and robust manner. The term ‘chemometrics’ is originally applied in chemistry, where huge data matrices are generated by developing mathematical and statistical methods to extract useful information related to the chemical characteristics of the analyzed seeds. Although the multispectral images contains very useful information about absorption bands related to overtone and combination vibrations of C-H, N-H and O-H functional groups of the major chemical constituents of the tested seeds, dealing with all of this information is a challenge itself [The acquired multispectral images are very abundant in information in the spatial and spectral domains, so that bias-free methods for data mining from the acquired images are very crucial, especially if one desires to analysis several components simultaneously . These mhallenge . When mae itself . Data exBased on the purpose of analyses, there are many calibration algorithms and pattern recognition methods that are widely known to perform well for qualitative (classification) and quantitative (prediction/regression) analyses of seed quality. The classification modeling method could be also classified into supervised or unsupervised methods. The supervised techniques are used to build classification models to classify new unknown samples to predefined known classes on the basis of their spectral signatures; meanwhile, the unsupervised methods do not require a prior knowledge about the grouping of the samples . DespiteBecause the images collected by the multispectral imaging system are in a three-dimensional form, they could be used to analyze minor and subtle variations in the physicochemical properties of tested seeds. This technology has shown promising results in determining different seed quality features. By the way, applying this technology in seed quality evaluation is based basically on the rigorous understanding of the technology fundamentals and how to link seed quality parameters with the information available in the images . BecauseAlthough a lot of articles reviewing the potential usages of hyperspectral imaging, RGB imaging, X-ray imaging systems for seed sorting and testing are available ,13,16, tDiscrimination among seed varieties and checking the seed purity plays a key great role in plant breeding programs, seed production, genebank management and in the general trading of seeds. Testing seed purity involves checking the presence of plant debris, foreign materials, weed seeds, contaminating species and broken and damaged seeds. Separating varieties and determining their critical properties in terms of distinctness, uniformity and stability standards are extremely significant for variety registration, intellectual property rights of plant breeders, as well as for developing new varieties in the markets . MeasureRecent studies have demonstrated the ability of multispectral imaging in tandem with multivariate analyses for evaluating seed purity and varietal discrimination. The potential of using this technology in identifying and characterizing seed variety on the basis of differences in spectral signature and the ordinary morphological features was investigated in ,77,81,94By using illumination-based multispectral imaging system in the visible and near infrared region (375–970 nm) for varietal identification and discrimination tomato seeds, Shrestha et al.  examinedWith the same technique, Hansen et al.  collectek-nearest neighbor (k-NN) based model was built to discriminate different wheat varieties. The overall results on varieties separation using the proposed method revealed that Agostino variety was classified with accuracy of 97.4%and the overall classification accuracy for the rest of varieties was below 67%. They reported that, discrimination accuracy could be greatly improved up to 95.8% in case of grouping varieties together according to visual similarity.For differentiation of wheat varieties and triticale varieties, Vrešak et al.  used a VBacillus thuringiensis transgenic rice seeds from non-transgenic ones. In another study, the same authors [L*, a*, b* colour values, area, length, width and roundness) produced better predictive accuracies and helped in improving the discrimination for all examined multivariate analysis algorithms.The feasibility of using LED illumination-based multispectral imaging (405–970 nm) associated with chemometric data analysis was examined by Liu et al.  for nond authors  followedBecause the sorting process of seeds based on certain genetic features is a very time-consuming process that can take thousands of hours of labour to complete for a single average-sized breeding programmer, De La Fuente et al.  evaluateWhile hyperspectral imaging systems with relevant multivariate analyses are able to identify kernel damage, multispectral systems are much simpler and less expensive. In this regard, Shahin et al.  analyzedIn another study, Sumriddetchkajorn et al.  used a mw/w) adventitious contamination of durum wheat (T. durum) with other common wheat (T. aestivum), and any malpractice during manufacturing by adding a higher percentage is considered a kind of adulteration that should be instantly prohibited. In a study conducted for seed authentication, Wilkes et al. [T. durum and adulterant common wheat T. aestivum cultivars based on the spectral signature of each grain species. The performance of the system was evaluated on the basis of its ability to correctly assign the percentage of adulteration level  of the wheat test samples. Each grain was represented by thousands of pixels that can be scored based on their closeness to the spectral signature of T. aestivum or T. durum. The normalised-Canonical Discriminant Analysis model (nCDA) algorithm was then applied to calculate the likelihood of each grain being T. aestivum or T. durum based on composite spectral signature of the pixels within each grain. The resulting classification images, shown in For manufacturing pasta, the current EU law permits up to 3%  separating wheat kernels based on their average protein content to three classes i.e., high, medium, and low.A fully mechanized LED-pulsed multispectral system was designed and developed by Pearson et al.  using thBy recording sequences of images by changing the spectral conditions, Novales and Bertrand  acquiredIn practical applications from farmers’ point of view, seed companies and seed warranty determination, it is very crucial to have some knowledge about seed viability status before sowing to have information for yield prediction . Seed deRicinus cummunis L.) seeds. The data of mean intensity for each individual seed were analyzed by nCDA transformation to correlate these data with viability values obtained by the ordinary tetrazolium test for every individual seed. In viability scoring by the tetrazolium test, seeds that developed red colouring of the embryo indicates enzyme activity and were therefore scored as viable seeds and the remaining unstained seeds were considered dead (unviable) seeds. The results revealed that viable seeds were distinguished from dead seeds with accuracy of 92 and 96% in the calibration and validation data sets, respectively. As shown in In this application, multispectral imaging systems employed in the visible and NIR regions (375–970 nm) with 19 wavelength bands has been tested by Olesen et al.  in viabiShetty et al.  investigSeed health is directly affected by any damage occurred to the seeds as well as the invasion of disease and insect infestation. Seed damage may result from poor harvesting and storage conditions; from rough handling during cleaning and processing or even from disease and/or insect infestations. Such damages may be external, such as cracks in the seed coat or splitting of the seed which may also lead to internal damages and sometimes germinating and vigor losses . Beside Fusarium strains . They used canonical discriminant analysis (CDA) for separation of image pixels based on their mean intensity and Jefferies-Matusita (JM) distance for classification and modeling spectral data which gave a separation accuracy of 26-88% between uninfected and Fusarium spp. infected seeds.As seed health tests are time consuming processes and require substantial training for characterization of pathogenic fungi on seed, multispectral imaging system (395–970 nm) was used by Olesen et al.  for idenTriticum aestivum L.) varieties and nine triticale  varieties in terms of their resistance to fungal infection. The same method outlined by Olesen et al. [Alternania sp. appeared in orange and parts infected with Fusarium sp. appeared as light blue; meanwhile, the uninfected healthy parts appeared in dark blue.Vrešak et al.  utilizedn et al.  was follFusarium or Diplodia; pinked white or yellow maize, broken kernels; and foreign matters . Moreover, partial least squares discriminant analysis (PLS-DA) models were used for distinguishing between sound maize and undesirable materials. The PLS-DA model had good classification accuracies ranging from 83% to 100%. In another study for detecting different defects in maize kernels, Sendin et al.  used a mThe study also concluded that wavelengths related to red, pink and blue carotenoid and anthocyanin pigments , and NIR wavelengths 890, 940 nm (associated with fat) and 970 nm (associated with water) are considered as relevant wavelengths for discrimination. Fusarium head blight infection in wheat kernels. For each sample, eight RGB images with a spatial resolution of 524 × 702 pixels each were acquired and merged into an “image-cube” dimensioned 524 × 702 × 24. Spectral data were extracted from every single kernel and then analyzed via PCA and stepwise multilinear regression (SMLR) methods. The developed SMLR model was then applied to estimate the degree of contamination of each pixel in the image to produce ’predicted’ classification images with high values indicating a high local contamination. The PCA results of the data indicated the possibility of separating the ‘susceptible’ and ‘resistant’ genotypes of wheat kernels (For the detection of fungal contamination, Jaillais et al.  designed kernels a,b. Simi kernels c,d.Because mycotoxins, the toxic metabolites of certain filamentous fungi, cause very severe health problems and are considered as potent carcinogens causing many human deaths per annum, Kalkan et al.  applied Pests inflict their damage on seeds mainly by feeding on the endosperm leading to sever losses in both seed weight and quality. When pest species feeds on the germ, it results in poor seed germination and less seed viability . In addiMa et al.  tested aThe main challenge of seed industry in the recent century is to acquire consistent information about overall seed quality and safety during the production, processing, handling, marketing and distribution chains, to satisfy consumer demands and the requirement for market expansion. During the last couple of decades, the advancement in image acquisition protocols along with rapid image processing algorithms has permitted quantifications of many morphological and physico-chemical features of the inspected seeds. Conventional methods for measuring seed quality traits by visual inspection of seed intactness and checking germination capacity, seed health and freedom from disease infestation are criticized as being costly and tedious analytical procedures. In this context, imaging techniques, especially spectral imaging methods, have been proved to be efficient techniques for rapid monitoring of numerous seed quality traits. Thence, it was very critical for seed community to explore the potential of this technique in real-time implementations . Becauseoptimal’ wavelength is considered the most important step in developing and establishing an a high-throughput system that has the capability of performing the inspection and quality control decision in real time [In general, the real-time or near real-time assessment of quality attributes of seeds and/or any relevant bio-product in harsh environment requires a better understanding of the critical quality attributes of the seeds being analyzed. Also, a balance between the cost of equipment including hardware and software, the detection speed of the imaging system, the detection accuracy of the model, and the service life of the equipment must be all considered during building a comprehensive real-time spectral imaging system to move the operations the from the laboratory setting to online systems in the industry . Becauseeal time .Indeed, the real-time operations are very complicated processes that require acquisition of images at several bands and process such images simultaneously. The procedure of selecting the ideal wavelengths can either be performed before the modeling or even during the modeling process by discarding the unnecessary wavelengths that add complexity of the model and reduce its efficiency in classification or prediction. These unwanted wavelengths make model interpretation difficult and affect the classification and prediction efficiency of the model . There aThe potential of spectral imaging technique in providing rich and valuable information about seed quality traits and phenotyping parameters is powered from the robust integration among spatial imaging, spectroscopy and chemometrics tools that makes this technique an ideal tool in studying various morphological, physicochemical and physiological properties of seeds. This extraordinary capability has enticed researchers to exert cooperative efforts in developing fast, accurate and low-cost spectral systems to be installed in seed and grain industry. With the advantage of acquiring three-dimensional data across a wide range of the electromagnetic spectrum, state-of-the-art multispectral imaging along with relevant multivariate chemometric analysis has been successfully implemented, not only for food quality and safety control purposes but also in dealing with critical research challenges in seed science and technology. This paper provided an overview of the previous research activities employed for quality evaluation and safety analysis of different seed cultivars using multispectral imaging techniques. The structure of all possible designs and image acquisitions modes of these systems were reviewed and the overall merits and demerits of such configurations for specific usability and applicability were also discussed. This review continues from where previous reviews stopped and focuses only on real implementations of fully-operated multispectral imaging systems for quality assessment of different sorts of seeds. Thence, this review comprehensively reviewed research attempts only devoted to fully-operated multispectral imaging systems and does not consider those ones that just dedicated to selections of key wavelengths from hyperspectral data analyses without building up independent multispectral imaging systems. This makes this review article the first attempt in summarizing all published papers in multispectral imaging applications for seed phenotyping and quality monitoring. The review has emphasized the capability of the multispectral imaging with different chemometrics algorithms in characterizing physicochemical quality traits, predicting physiological parameters, variety identification and classification, detection of damages, defect, pest infestation and seed health. By careful consideration of all limitations and challenges faced by this technology, it is anticipated that the multispectral imaging technique can be moved from laboratories to practical applications in the form of real-time seed monitoring systems that meet the requirements of the modern industrial control and sorting systems. The decreasing cost and increasing speed and capability of computer hardware and artificial intelligence will push this mission forward and make this technology more attractive for prospective usages in quality control and automatic inspection of seeds.