Single Nucleotide Polymorphisms (SNPs) are among the most important types of genetic variations influencing common diseases and phenotypes. Recently, some corpora and methods have been developed with the purpose of extracting mutations and diseases from texts. However, there is no available corpus, for extracting associations from texts, that is annotated with linguistic-based negation, modality markers, neutral candidates, and confidence level of associations.In this research, different steps were presented so as to produce the SNPPhenA corpus. They include automatic Named Entity Recognition (NER) followed by the manual annotation of SNP and phenotype names, annotation of the SNP-phenotype associations and their level of confidence, as well as modality markers. Moreover, the produced corpus was annotated with negation scopes and cues as well as neutral candidates that play crucial role as far as negation and the modality phenomenon in relation to extraction tasks.http://nil.fdi.ucm.es/?q=node/639.The agreement between annotators was measured by Cohen’s Kappa coefficient where the resulting scores indicated the reliability of the corpus. The Kappa score was 0.79 for annotating the associations and 0.80 for the confidence degree of associations. Further presented were the basic statistics of the annotated features of the corpus in addition to the results of our first experiments related to the extraction of ranked SNP-Phenotype associations. The prepared guideline documents render the corpus more convenient and facile to use. The corpus, guidelines and inter-annotator agreement analysis are available on the website of the corpus: Specifying the confidence degree of SNP-phenotype associations from articles helps identify the strength of associations that could in turn assist genomics scientists in determining phenotypic plasticity and the importance of environmental factors. What is more, our first experiments with the corpus show that linguistic-based confidence alongside other non-linguistic features can be utilized in order to estimate the strength of the observed SNP-phenotype associations. Trial Registration: Not ApplicableThe online version of this article (doi:10.1186/s13326-017-0116-2) contains supplementary material, which is available to authorized users. An SNP is a single base mutation occurring at the DNA level. Variations in DNA sequences can affect how humans develop diseases and respond to pathogens, chemicals, drugs, and other agents . There eIn genetic epidemiology, GWA study refers to the process of examining several common genetic variants in different people so as to discover a possible correlation between a variant and a phenotype trait. A phenotype is an organism’s recognizable characteristics or traits such as its development, biochemical or physiological properties, behavior, and the concomitant products of that behavior . The larIn this paper, we described and discussed the process of constructing ranked SNP-phenotype association corpus (SNPPhenA), inter-annotator agreement analyses and the results of some utilized baseline methods during an initial experiment. In most cases, implementing a biomedical text-mining system is a difficult task as the basic scientific communication components — i.e. journals and databases — are designed to be read by humans, not machines or computers. In order to address this problem, xml was selected as the main format for the produced corpus. Furthermore, biomedical Natural Language Processing (BioNLP) systems (e.g. relation extraction) have been mostly applied to abstracts as, though concise, they are more readily available. Also, abstracts are deemed as good targets for information extraction (IE) because they are a succinct and summarized version of an article , hence tp-values and study size.Several named entities have been investigated during the biomedical relation extraction task, few of which are suitable candidates for annotating with confidence degrees, which is the major aim of the research when identifying the strength (severity) of associations or interactions. The reason for this is that there are no adequate biomedical agreements. For instance, Drug-drug Interactions (DDI) or Protein-protein Interactions (PPI) are two biomedical relations discussed by a myriad of researchers. However, it is difficult even for a human expert to reliably classify the strength or severity of DDIs or PPIs according to confidence level, a problem existing due to the variation in the types of related experiments and the paucity associated with the methods of quantifying and estimating the significance of both the research method and the association. Most GWA studies that report SNP-phenotype associations are generally based on case-control researches  initiallBoth preparing a reliable corpus annotated with confidence level in associations and developing an automated tool for this purpose are evidently more difficult for a host of other biomedical named entities that may require different models of study . For insConsequently, insofar as NLP, ranked SNP-phenotype association extraction based on confidence level is considered to be a more feasible task in comparison with many other biomedical association extraction tasks. Additionally, it is worth mentioning that specifying neutral candidates and the effects of negation annotated in the corpus is influenced by measured confidence level of association between two entities, elaborated in the following sections. This shows how crucial it is to have reliable annotations for confidence level in associations as well as an automated method for identifying them.Yet another objective of the present was to identify the association of such phenotypes as quantitative traits instead of diseases with SNP’s, variously studied by researchers. Such extension is significant because many phenotypes can be detected during the sub-clinical phase of a disease history, hence determining their association with an SNP entails a more early diagnosis and treatment of the disease. Certain phenotypes, it should be noted, are important risk factors for the disease.negation. According to linguistics [One of the linguistic-based phenomena discussed in this paper is guistics , negatiomodality. Generally, modal expressions are words that state modality which is the expression of the subjective attitudes and opinions of the presenter about a possible fact or to control a probable action including intentions, possibility, probability, necessity, obligation [The other linguistically-driven phenomenon employed here is linguistic ligation . In thisNER) is the first step towards extracting associations and relations as well as making related corpora within biomedical texts [BioNLP, investigated in a myriad of studies such as [EMU is another mutation tagger effective in reducing the annotation time of articles candidate for mutation related associations [Named Entity Recognition  are annotated for negation, yet no linguistic cues are annotated. In fact, the objective of the BioScope corpus is to approach this language phenomenon in a general, task-independent, and linguistically-oriented manner. It can further automatically recognize negation scopes and cues in sentences.In general, two negation detection methods have been developed to annotate the employed corpora: A linguistic-based approach and an event-oriented approach. Among other negation annotated corpora, one may refer to the two most well-known: the linguistically-focused, scope-based BioScope  and the Genia . In BioSNegDDI-DrugBank is another corpus that was annotated by the authors of the previous work with scopes of negation and negation cues [ion cues . The aution cues . Adding In this section, the steps followed in the construction of the SNPPhenA corpus are explained. The entire process consists of three major steps of collecting documents, automatically and manually recognizing the SNP and phenotypes, and annotating the associations and the related information Fig. . The lasIn order to have consistent annotations, all annotators were given the same instruction which includes a pellucid definition of the entities and their relationships, rules and conventions of annotating the confidence level of associations and complete examples for each type of tags. The annotation guideline also contains rules for tackling linguistic phenomena such as negation cues and modality markers. Moreover, this document presents different types questions raised and retorted by the annotators during the annotation process. The annotation guideline can be found on the website of the corpus.In the end, 360 XML files were generated comprised of the abstract texts, SNPs, Phenotypes, and the SNP-phenotype associations in the selected sentences. The Phenotypes, SNP names and the association candidates were annotated as xml element tags for each nominated sentence in the abstract. Next, the annotations and the final product were manually checked. The produced SNPPhenA corpus is available for public use http://www.gopubmed.org/” search engine was used to collect genome-wide association abstracts. GoPubMed is a webserver allowing users to explore PubMed search results with Gene Ontology [http://www.snpedia.com/”website; the extracted list of abstracts was shortened via selecting those comprised of popular disease names. The list was finally truncated again through choosing those that have candidate sentences consisting of both types of entities. We collected a total of 360 abstracts (including 2625 sentences) with at least one candidate sentence with an SNP and a phenotype name. There were 483 key sentences containing at least one SNP and one phenotype name that were annotated with the xml element “SENTENCE”. The total number of SNP names annotated in the SNPPhenA corpus was 875. It is worth mentioning the SNPPhenA is a sentence-level corpus and sentences merely including SNP or Phenotype were not annotated.Information provided by the “Ontology . Twenty The next step was to perform an automatic Named Entity Recognition, followed by a manual checking of sentences with candidate relations for SNPs and phenotype names, as explained in the section below.An essential part of biomedical NLP is to detect biomedical named entities . During A phenotype is the appearance of an organism in terms of its morphology, development, physiology, behavior and its concomitant products . AlthougIn this regard, a dictionary-based NER task was implemented by combing two more complete and pertinent databases. The prepared dictionary includes a list from the Comparative Toxicogenomics Database (CTD) for disease names . Also inThe phenotype names were initially recognized automatically by the prepared dataset. Manual checks were subsequently made by two experts in order to identify missed or inexact phenotypes.A short list of the most frequent phenotypes is shown in Table rsID numbers, while utilizing the Variant Name Mapper(VNM) tool [The inconsistent description of biological data elements renders the relation extraction tasks challenging. Names associated with polymorphism are particularly problematic because historical or common names are, more often than not, employed instead of standard nomenclature , specifiNM) tool . The VNMNM) tool , SNPediaNM) tool , pharmGKNM) tool . The datSimilar to the phenotype NER process, SNP name annotations were initially checked manually by two biology experts and verified by a third professional annotator. A short list of the most frequent SNPs is shown in Table This section deals with the process of annotating the associated candidates which includes the annotation of the SNP-phenotype associations, the confidence level of associated candidates, modality markers, and negation scopes and cues in the negated sentences.Following the collection of abstracts and the determination of the SNP and phenotype candidate names, the associations between SNP and phenotype were manually annotated by three gurus in genetics Fig. . The SNPAs Fig. In more precise terms, an SNP-Phenotype association candidate is identified as neutral if:(i) The absence or presence of association between SNP-phenotype cannot be specified from the sentence (or container clause) with a confidence level of more than zero.(ii) The status of presence or lack of association between the SNP and the phenotype does not change from positive to negative or vice versa if the sentence (or container clause) is negated and SNP and phenotype names are located in the scope of the negation.(iii) The confidence level of association between SNP and the phenotype does not change if a modal marker is utilized in the sentence and both entities are located in the scope of modality.The association in Fig. It is worth mentioning that in most relation extraction corpora, neutral candidates were considered to be part of the negative (non-positive) class. Considering them as a separate class of associations allows researchers to conduct different types of experiments. More details as to the role of neutral candidates in biomedical relation extraction tasks can be found in the author’s other study .Similar to the previous steps, the manual checking was initially performed by two experts, and in order to sort out the issue of contradictory confidence levels, the verdict of a third expert annotator was taken into account.In spite of the fact that genetic components have the instructions for the growth and development of each individual, a person’s phenotype is influenced by environmental factors during embryonic development and throughout life. Environmental factors can stem from a variety of influences such as diet, climate, illness and level of stress. For instance, the capability to taste food is a phenotype estimated, by scientists, to be 85% influenced by genetic inheritance . Neverth“Phenotypic plasticity” is the ability of a genotype to generate more than one phenotype due to various environments . The plaDiffering phenotypic plasticities alongside possible unknown genetic components are the two reasons why GWA study uses confidence level in order to describe the strength of association. The linguistic-based confidence level of the reported association ultimately yields informative data leading to the determination of phenotypic plasticity.However, there is no available data source or automated method for extracting confidence level from the obtained results. This is when the presence of such a tool and data source is critical and conducive to reviewing literatures.For this purpose, the confidence levels of positive association candidates in the corpus were annotated by a guru in human genetics. Based on the strength of the linguistic correlation between each individual phenotype and the relevant SNP mentioned in the abstract, the confidence level of associations was categorized into weak, moderate, and strong. Moreover, when the association is neutral , the degree of confidence is set to “zero”. The confidence levels were assorted considering modality, adverbs and the reported statistical results . Detailed information about the annotation guidelines can be seen in the guidelines document, available on the website of the corpus. The process, all the same, is demonstrated here via some samples.The sentence shown in Fig. The sample mentioned in Fig. The annotation of confidence level was carried out by two biology experts both of whom had the same opinion regarding 86% of the association candidates in the whole corpus. In order to sort out the issue of contradictory confidence levels (14%), the opinion of a third guru annotator was considered.Identifying negative statements is essential in order to obtain accurate information from the text data. The sentence in Fig. A rule-based system, proposed by , was iniIn addition to the negation cue and scopes, modality markers were annotated during the annotating process. The employed modality markers obtained from the list were already provided in , which iIn this section, inter-annotator agreement analyses and the calculated scores are initially presented; then some of the basic statistics of the produced corpus will be demonstrated; and finally, the results obtained from our first experiment using the corpus are presented.In order to evaluate the quality of the corpus and the reliability of the annotations, the inter-annotator agreement score was measured for the task of classifying candidate sentences into positive, negative and neutral classes, and also for the task of determining the confidence level of the association. As was mentioned before, two annotators had independently tagged the corpus. In the case of disagreement between two tags, a third annotator was asked to decide on the correct one. For the task of classifying candidate sentences, inter-annotator agreement was 91%, which means that in 91% of cases, the two annotators agreed. Additionally, we computed Cohen’s Kappa coefficient  for the o is the relative observed agreement among annotators, and pe is the hypothetical probability of chance agreement. The Kappa value was 0.79 for the two annotators. In general, κ = 1 indicates a complete agreement. Furthermore, κ < 0 shows that there is no agreement between annotators other than what would be expected by chance (as given by pe).Where PAs far as the task of annotating the confidence level of the association with four categories , annotators agreed in 87% of the occasions; yet the Kappa value was 0.80 which is satisfactory.This section provides detailed statistics as to the linguistic and nonlinguistic properties of the corpus. The basic properties of the corpus are presented in Table Table Additionally, as mentioned earlier, the key negated sentences in the corpus were annotated with scopes of negation and negation cues. As Table As illustrated in Table Additionally, as Table The results of our first experiments with the corpus are presented in this subsection. Although several mutation-related association extraction methods have recently been developed, automatically measuring the confidence level in an association is a novel task. Consequently, our first experiments were evaluated via certain baseline kernel methods for the two subtasks.In order to categorize the associations, we employed the two kernel methods that have been expansively made use of in the relation extraction task; the local context kernel  and sub-Table The results of the confidence level prediction of associations are presented in Table The lower performance of identifying the confidence level of association in comparison with the association extraction method demonstrates that the simple features used in the binary BOW may not have enough information to surmount the task and more linguistic features are required. Moreover, the difficulty of the task might be precipitated by the fact that during the annotation process, the annotators employed the mentioned p-value number as a complementary factor for identifying the confidence category, which was the case with 20% of the candidate sentences. It can, accordingly, be concluded that accurately identifying ranked association from biomedical articles requires more linguistic features including dependency parsing, lemmatizing and features related to identifying the significance degree of the biomedical statistical tests.A simple version of the baseline method can be found online All the kernel method experiments were carried out by a support vector machine with SMO  implemenIn this research, a SNPPhenA corpus was developed in order to extract the ranked associations of SNPs and phenotypes from GWA studies. The process entailed collecting relevant abstracts, Named Entity Recognition, and annotating the associations, negation, modality markers, and the confidence level of the associations.As opposed to the previous biomedical relation extraction corpora containing true and false types of relations, the annotated associations in the corpus were divided into three classes: positive, negative and neutral candidates. The neutral candidates were those SNP-phenotype candidates that showed no clear evidence as to the presence or lack of association between the SNPs and phenotypes. Identifying neutral candidates is critical for the negation process as the status of such candidates and their corresponding level of confidence do not change when they are located in the scope of negation terms; the status of distinguished association candidates, on the other hand, change in such cases. Similarly, the confidence level, certainty or uncertainty of a neutral candidate, does not change if it is located in the scope of a speculation or modality term. Hence, determining the effect of negation as well as modality terms requires the identification of neutral candidates.Not to be forgotten is the fact that the SNPPhenA corpus must be considered as an initial step in extracting graded associations from literature, which could result in the idea of a fuzzy relation extraction task that can be employed so as to construct better biomedical ontologies.Furthermore, it is important for future researches to employ more linguistic-based and non-linguistic-based factors that could be utilized to determine the confidence of the reported associations. Credibility of the genotyping techniques (such as MLPA or RFLP) and the validity of the research through graph-based network analyses can be employed in the process of identifying the overall confidence level of the reported associations.


This paper presents an out-of-plane electrostatic micro-power generator (MPG). Electret-based continuous MPGs with different gaps and masses are fabricated to demonstrate the merits of this topology. Experimental results of the MPG demonstrate output power of 1 mW for a base acceleration amplitude and frequency of 0.08 g and 86 Hz. The MPGs also demonstrate a wideband harvesting bandwidth reaching up to 9 Hz. A free-flight and an impact mode model of electrostatic MPGs are also derived and validated by comparison to experimental results. Advances in low-power integrated circuits have led to the realization of embedded sensors with low power requirements. Combining these sensors with sustainable energy sources will allow the deployment of autonomous sensor arrays. Energy harvesting of ambient vibrations is a promising direction to satisfy this need. Vibrations are widely available in domestic and industrial environments. Vibration-based micro-power generators (MPGs) have a demonstrated ability to generate electric power in the micro- to milli-watt range .In addition to the amount of generated power, a low cost implementation is a requirement for using MPGs in autonomous sensors. Among vibration harvesting techniques, electrostatic MPGs have the lowest implementation cost since no specialty materials are needed, such as magnetic materials used in electromagnetic MPGs and ceramic materials used in piezoelectric MPGs. On the other hand, the need for a charge source and switching losses adds limitations to electrostatic MPGs.Continuous mode electrostatic MPGs are designed to eliminate the need for switched circuits. Many of them use electrets, quasi-permanently-charged dielectrics, to induce charges on the capacitor plates and eliminate the need for an initial charge source. Due to these advantages, significant efforts have been devoted to developing electret-based electrostatic MPGs. Boland et al.  miniaturn et al.  proposedTsutsumino and co-workers ,6 develoEdamoto et al.  construcectrodes .Bartsch et al.  fabricatectively . Hoffmanectively  realizedMahmoud et al.  proposedtions as . Tao et tions as  realizedTao et al.  developeIn most reported literature, in-plane configurations are used to implement the capacitive transducer due to the low damping of these motions. However, such implementations require small vertical feature sizes, as well as high aspect supports to protect against out-of-plane disturbances.In this paper, we compare in-plane and out-of-plane transducers and report on an out-of-plane electret-based electrostatic MPG. Test results demonstrate 1 mW of output power at less than 0.08 g excitation acceleration. Moreover, the prototype exhibits a wideband operation that reaches 9 Hz. We also report on two models for free-flight and impact MPGs. The simulation results obtained from these models show good agreement with the experimental results.In R. When vibrations are allowed to change the relative positions of the capacitor plates, capacitance varies over time Electrostatic energy harvesters  consist First, we compare the energy harvesting capabilities of the two most prevalent variable capacitor implementation: in-plane comb-finger, C) curve to determine the amount of electrical energy harvested per conversion cycle. The curves are obtained by substituting Equation =x∘siEquation  for the Equation  for the Equation  and inteComparing the two figures, it is clear that for the same normalized stroke (input vibrations), the singularity in out-of-plane parallel-plate transducers, as motion size becomes comparable to the gap m to capture base excitations To study the full system dynamics, the electrical model of the transducer, Equation , is augmitten as :(6)mx¨=A are air viscosity and the electrodes’ surface area, respectively.Squeeze-film damping depends on the gap between the electrodes. It can be represented by :(8)csq=Substituting Equations  and 8)  8) into This nonlinear system of differential equations represents a lumped-mass model of the energy harvester in free-flight. It does not account for the possibility of the inertial mass coming into contact with the fixed electrode, which is addressed later.The prototype used to demonstrate the out-of-plane electrostatic energy harvester was fabricated using precision machining. It consists of a bottom fixed electrode and an upper movable electrode supporting a steel inertial mass n, E, b, h and L are the number, Young’s modulus, width, thickness and length of the beams, respectively. Using Equation (The linear spring constant of guided beams can be estimated using the Equation :10)k11k11(10)k1To account for the non-idealized geometry and configuration of the beams, nonlinear finite element analysis (FEA) is carried out on the prototype using COMSOL. The linear spring constant is then extracted from the relationship between the static force and displacement to be 10,987 N/m. The first and second mode shapes of the harvester prototype are also found using COMSOL and shown in A schematic of the experimental setup is shown in The gap between the electrodes is initially set to a nominal value of 250 The lower natural frequencies obtained experimentally are due to support flexibility , unaccounted for in the FE model. Using the effective mass calculated from FEA and the natural frequency of the second mode obtained experimentally, the linear spring constant is calculated as g is the acceleration of gravity. The static deflection of the prototype is calculated as 25 In order to compare the experimental results with the model, the actual gap The total damping ratio can be extracted from the experimental results using the half power bandwidth Using the results in Equation  is integUsing this procedure, we found the value of the linear damping coefficient to be Equation  for the C curve is found experimentally and predicted using the model  based on this energy harvester is introduced and studied.Realization of a practical electrostatic energy harvester requires the substitution of the DC voltage source with a portable charging source. A charged dielectric embedded within the structure of the transducer will be used to induce charges on the capacitor electrodes. A permanently-charged dielectric, an electret , constitAn electret film is integrated into the energy harvester prototype introduced in The nominal capacitor gap is initially set equal to The frequency-response curves in Nonlinear response is seen at acceleration amplitude of The MPG bandwidth increases as the amplitude of base acceleration increases, and the response region changes from linear to nonlinear to impact;The up-sweep and down-sweep bandwidth are equal for linear and nonlinear responses while the up-sweep bandwidth is wider than the down-sweep bandwidth for impact response, in agreement with Soliman et al. .Defining the MPG bandwidth as the half-power bandwidth for linear response and the distance between the response knees for nonlinear and impact responses, we observe the following:Specifically, and MPGs  that canTo test the effect of variation in the electrostatic field strength on the MPG performance, we repeat the experiment shown in The results show that the MPG output power and bandwidth depend on the interaction between three factors: the strength of the electrostatic field, squeeze-film damping and impact. A small gap and, thus, a stronger electrostatic field for a given electret voltage allow the MPG to extract more output power from lower excitation levels than MPGs with weaker electrostatic fields. This can be seen by comparing the output power of the MPG at A smaller gap leads to low impact losses since it means that impact will occur at a lower velocity. This can be seen in the figures by comparing the up-sweep bandwidth for n et al.  show thaFor environments where small amplitude vibrations are available, a small capacitor gap should be used to increase the strength of the electrostatic field and obtain higher nominal capacitance For environments where large amplitude vibrations are available in a narrow frequency band, a large capacitor gap combined with a strong electrostatic field generated by a large source voltage will lead to high efficiency energy extraction by maintaining the response near the linear response region while guaranteeing a larger output power saturation level by allowing for a larger stroke.For environments where large amplitude vibrations are available in a wide frequency band, a large capacitor gap will lead to larger output power and wider MPG bandwidth by operating in the impact region.We conclude that it is crucial to balance these three factors in the design of out-of-plane electrostatic MPGs. The proper balance depends on the harvesting environmentOperating the MPG in the impact region creates a wideband MPG with up to 9 Hz in the up-sweep and 7 Hz in the down-sweep bandwidth. Therefore, it is important to develop an MPG model valid for the impact region for use in performance prediction and optimization. In this section, a modified system model is developed and verified by comparison to experimental results.Manual assembly of the electret on the bottom electrode creates bumps on the electret surface. To account for this, the electret film position is elevated, thereby reducing the effective gap to The restoring force Using Equations  and 14)14), a moRealization of the impact model requires estimation of the system parameters. The linear m FEA fb=1 Hz to eThe effective gap The linear stiffness of the contact spring We examine the potential to design better MPGs that can capture more kinetic energy from the environment by testing the effect of stronger electrostatic fields and larger inertial mass. The strength of the electrostatic field is increased using a high-voltage electret at a similar gap to the previous experiment to minimize the effects of variation in squeeze-film damping and impact losses on the MPG performance. The surface voltage of the charged electret film was measured upon receipt from the manufacturer at −1700 V. At the time of the previous experiment, the surface voltage had degraded and was measured at −410 V. The surface voltage is known to drop in open air due to charge recombination under the influence of humidity . For thiWe note that while increasing the inertial mass allows the MPG to capture more kinetics, it changes the optimal load resistance as shown in The results and analysis presented above demonstrate the feasibility and advantages of electret-based out-of-plane continuous MPGs. In fact, the improved MPG proves this potential by realizing almost 1 mW of output power in We find that the capacitor gap should be set to match the requirements of the harvesting environment. On the other hand, increasing the electrostatic field strength by using a high DC voltage source is always advantageous leading to more sensitive MPGs that can collect energy at lower excitation levels, larger optimum output power and wider MPG bandwidth. Larger nominal capacitance, and thus capacitor area, and inertial mass allow for a larger optimum output power; however, they change the nature of the electromechanical coupling in the MPG and require a search for the optimal load resistance at a particular configuration.Finally, the use of an electret layer as a DC charging source does not only make the MPG portable and low cost, but also helps to isolate the two capacitive electrodes during impact. However, since negatively-charged electrets degrade with exposure to humidity, the implementation of a good sealing is a must to preserve the electret charge over the lifetime of the MPG.


The impact behavior between the charge and lifter has significant effect to address the mill processing, and is affected by various factors including mill speed, mill filling, lifter height and media shape. To investigate the multi-body impact load behavior, a series of experiments and Discrete Element Method (DEM) simulations were performed on a laboratory-scale mill, in order to improve the grinding efficiency and prolong the life of the lifter. DEM simulation hitherto has been extensively applied as a leading tool to describe diverse issues in granular processes. The research results shown as follows: The semi-empirical power draw of Bond model in this paper does not apply very satisfactorily for the ball mills, while the power draw determined by DEM simulation show a good approximation for the measured power draw. Besides, the impact force on the lifter was affected by mill speed, grinding media filling, lifter height and iron ore particle. The maximum percent of the impact force between 600 and 1400 N is at 70–80% of critical speed. The impact force can be only above 1400 N at the grinding media filling of 20%, and the maximum percent of impact force between 200 and 1400 N is obtained at the grinding media filling of 20%. The percent of impact force ranging from 0 to 200 N decreases with the increase of lifter height. However, this perfect will increase above 200 N. The impact force will decrease when the iron ore particles are added. Additionally, for the 80% of critical speed, the measured power draw has a maximum value. Increasing the grinding media filling increases the power draw and increasing the lifter height does not lead to any variation in power draw.  A ball mill  is the kFor ball mills, it is dependent on the rotation of the mill shell for lifting the grinding media to impact the particle, and the mill shell, grinding media, particle and lifter are contents of the grinding environment ,6. The gThe charge consists of particle and grinding media. Charge parameters include powder filling, grinding media filling and charge size. The particles are grinding material, which are used to further comminute to obtain a suitable size distribution. The grinding media is used for impact-breakage and abrasion. Moreover, the size and filling of grinding media has a strict influence on the milling process ,18,19,20Currently, there are two methods to measure the impact force on the lifter. Method 1 changes in accordance to the drop height of charge in the mill, but transforms into measuring the single charge impact on the lifter material in free fall ,34,35,36To investigate the multi-body impact breakage behavior, we designed a drop ball tester and laboratory-scale mill to measure the power draw and impact force on the lifter. In this paper, we primarily studied the effects of mill speed , grinding media filling (5–30%), lifter height (10–20 mm) and grinding media shape  on impact breakage behavior. This improved understanding of the impact process in the ball mills and also improved mill grinding efficiency.The DEM is a numerical method for simulating and analyzing the bulk performance of granular materials and many geomaterials such as coal, ores and rocks. This method was designed to simulate dynamic systems of particles where each element is considered to be an independent and non-deformable entity that interacts with other particles by the laws of contact mechanics and moves following Newton’s second law. This method applies solid-rigid mechanics at the particle level, and the discrete elements are considered to be rigid, non-deformable elements. The constitutive model or behavior of the material is established at the contact areas between particles.DEMP is the power in watt; cm is the effective mass of charge; g is the acceleration of gravity; x is the distance between the center of the mill and the center of the gravity of load; and w is the rotational angular velocity of ball mill. This method has proven to be a useful tool in the milling industry. According to DEM theory, the power draws of ball mills can be calculated as follows:BondPis the power in watt; β is the mill coefficient (β = 1.08); D is mill diameter; φ is mill filling; Ψ is mill speed; and d is particle diameter in meter. In the last century, excluding the DEM model of mill power prediction, many models applied to predict the power draw of tumbling mills were very similar . Bond 3 applied In the ball mill, iron ore particles are broken by the grinding media acting on high-speed crushing and grinding. Reduction of the size of iron ore particles is caused by a cumulative process of single breakage behavior with a longer period of time. To understand the iron ore particle size reduction process, the single breakage behavior must be investigated.To measure the single breakage behavior, a drop ball tester was designed, as shown in To carefully collect the load behavior on the lifter, a laboratory-scale mill was designed. In 2O3 67.46%) pellet. Since grinding fineness has a great effect on iron ore floatation, appropriate grinding fineness can not only guarantee good monomer dissociation, but also help to avoid excessive grinding [grinding ,40; we cThe impact force plotted against the time in this research is characterized using DH5981 data acquisition equipment collected from the drop ball test performed on lifter. The drop test is performed using different drop height and using three different ball sizes, 20, 30 and 40 mm. The results presented in Experimental results indicate that each breakage event depends on the steel ball size and drop height, and increasing the steel ball size and drop height increases the gravitational potential energy. Hence, increasing the steel ball size and drop height can increase the impact force in ball mills.In the experiment, the mill fitted with a transparent end, and a high-speed digital camera is used to record steel ball trajectories. DEM software provides an accurate prediction of the steel ball trajectory in the ball mill. However, the above discussion is dependent on observing the steel ball trajectory in photographs. Simply observing the trajectory is not enough to describe what is occurring inside the ball mill. Hence, an impact force sensor was mounted in the lifter to investigate impact load behavior.From the above results, we can conclude that mill speed has a significant influence on load behavior. The impact force of the lifter for investigating multi-body load behavior is a useful method. At lower speeds , steel balls from the toe position to shoulder position cascade down along the charge surface, so the load behavior is mainly in a grinding process. At higher speeds , the steel balls are projected to fall down and impact the toe position, so the load behavior is mainly in a crushing process. However, an excessive speed result in steel balls impacting space position, and increases the lifter wear.In the DEM simulation and experiment, the power draw increases with mill speed up to a peak value of 80% of critical speed. This means that the optimum grinding efficiency occurs at mill speeds of near 80% of critical speed , becauseFrom the above results and discussions, the mill speed crucially affects the charge motion, impact force and power draws. In order to obtain an efficient grinding, the mill speed between 70% and 80% can be beneficial to ball mill operation. Furthermore, the ball mill with one transparent end and a photographed trajectory of charge can offer accurate input parameters for DEM simulation. Increasing the filling not only increases the impact load but also results in more wearing and breakage of the lifter. Hence, a suitable filling level is critical to improving efficiency. From the above investigation, increasing the grinding media filling does not increase the impact force on the lifter. Due to the lifter profile and rotation speed at the same conditions, the number of impacts on the lifter does not change when the filling level up to a suitable level. Therefore, further increases the filling level results in more steel balls impacting the space region and energy consumption.From the above results, this Bond formula is only suitable for a rough estimation of the mill power draw, and the Bond semi-empirical power draw model is less accurate than the DEM simulation. For the Bond semi-empirical power model, the main difference is that the calculation result does not consider the effect of the lifter. For the DEM simulation, this difference can be explained by the effect of the contact parameter. Still increasing the grinding media filling could not increase the milling efficiency according the impact force on the lifter. As a result, a suitable grinding media filling has a beneficial to ball mill grinding process. In the milling process, due to wear and impact, the lifter height gradually decreases until it is too low and must be replaced. This condition causes the steel ball not to reach a sufficient location, and the number of high–speed steel balls decreases. Finally, if the lifter height is too low, the working efficiency of the ball mill decreases. Therefore, the lifter height is a significant factor influences the steel ball trajectories.From the above results, it can be concluded that lifter height has a significant influence on working efficiency. Since the steel balls are lifted much higher when the lifter height increases, many high-speed steel balls impact the lifter. Therefore, effectively changing the lifter height helps to improve the milling efficiency.From the above results, the Bond semi-empirical power draw model is only suitable for estimating the power draw, and difference is influenced by the lifter. Besides, the difference between experiments and DEM simulation results are determined by the distance between the center of the mill and the center of gravity of the load. Thus, the DEM simulation can predict the load behavior at different lifter heights when the input parameters are accurate.From the above results and discussions, the lifter height has a significant effect on charge motion and impact force, but little effect on power draw. Increasing the lifter height increases the lifting ability, and the lifter height is related to the charge diameter. Thus, it is important to choose a suitable lifter height for a determined ball mill. The charge, the particle and lifter which collide together form a complex environment in the mill shell. To investigate the real grinding process, a simulation was performed to obtain the influence of particles and different media shapes on the impact breakage behavior. The results are shown in From the above results, it can be determined that media shape has a significant influence on the impact force in the lifter. The results occur because of the media shape of the steel ball, cylinder and square is point, line and surface contact, respectively. Moreover, adding iron ore particles buffers the impact force on the lifter, which decreases the impact force. To further compare the effect of adding iron ore particles on the grinding process, the mill power and particle size distribution are investigated. As shown in To study the difference of media shape on the milling process, an iron ore particle size distribution analysis is the most direct way to explain the effect of multi-body impact breakage behavior of different media shapes. As shown in From the above results and discussions, it can be concluded that different media shapes have a different influence on grinding efficiency. Besides, adding the particle in the ball mill can influence the milling process. Therefore, choosing a suitable media shape based on the milling environment is an effective method for improving grinding efficiency.In this paper, a laboratory-scale ball mill is employed in investigating the impact of different operating variables on the impact behavior of lifter. The determinations of ball trajectory, impact force and power draw are used to characterize the impact behavior between the charge and lifter. The main research conclusions are summarized as follows.The magnitude of impact force has strong sensitive to ball size and drop height. The impact force increases with the increase of ball size and drop height.Mill speed has profound effect on the ball trajectories, impact force and power draw. The mill can reach the best performance at the mill speed ranging from 70% to 80% of critical speed, and correspondingly the maximum percent of the impact force between 600 and 1400 N is obtained. For the 80% of critical speed, the measured power draw has a maximum value.Commonly, increasing the grinding media filling not only increases the fraction flying in the cataracting zone but also increases the rate of ball on ball contact per unit. In this research, it is interesting to find that increasing the grinding media filling does not the number of ball on ball contact. The impact force can be only above 1400 N at the grinding media filling of 20%, and the maximum percent of impact force between 200 and 1400 N is obtained at the grinding media filling of 20%. Besides, increasing the grinding media filling increases the power draw.Lifter height has a significantly influence on the load behavior. Increasing the lifter height increases the shoulder position and lifts the steel balls to the higher position,. The percent of impact force ranging from 0 to 200 N decreases with the increase of lifter height. However, this perfect will increase above 200 N. In general, increasing the lifter height does not lead to any variation in power draw.The impact force will decrease and the power draw will increase when the iron ore particles are added. The mass fraction passing of cylinder is greater than that of ball and square. The percent of −0.074 mm particle is 8.08% (cylinder), 6.42%  and 5.14% (square), respectively.The semi-empirical power draw of Bond model in this paper does not apply very satisfactorily for the ball mills, while the power draw determined by DEM simulation show a good approximation for the measured power draw.


Wearable electronics are rapidly expanding, especially in applications like health monitoring through medical sensors and body area networks (BANs). Thermoelectric generators (TEGs) have been the main candidate among the different types of energy harvesting methods for body-mounted or even implantable sensors. Introducing new semiconductor materials like organic thermoelectric materials and advancing manufacturing techniques are paving the way to overcome the barriers associated with the bulky and inflexible nature of the common TEGs and are making it possible to fabricate flexible and biocompatible modules. Yet, the lower efficiency of these materials in comparison with bulk-inorganic counterparts as well as applying them mostly in the form of thin layers on flexible substrates limits their applications. This research aims to improve the functionality of thin and flexible organic thermoelectric generators (OTEs) by utilizing a novel design concept inspired by origami. The effects of critical geometric parameters are investigated using COMSOL Multiphysics to further prove the concept of printing and folding as an approach for the system level optimization of printed thin film TEGs. There is a considerable amount of energy generated by the human body in the form of biomechanical and thermal energies, but most of it is usually wasted into the surrounding environment. There are significant potentials to harvest some of these wasted energies by means of devices, which benefits from the physical effects such as piezoelectricity , triboelIn this work, an origami-like concept is used to design a planar flexible TEG in order to increase the thermal gradient caused by the body warmth through the module cross plane. Then, the proposed configuration is modeled in COMSOL Multiphysics, version 5.3  based on the state of the art characteristics of organic thermoelectric materials. The module configuration is optimized with regard to the thermoelectric active material geometry.n pairs of p- and n- type legs with the Seebeck coefficient of pS and nS for p- and n- type legs, respectively, and thermal gradient of ΔT between the hot and cold sides of the legs, which can be calculated asThe voltage on a typical thermocouple consists of ΔT at the highest possible amount. There are many parasitic thermal losses in the way of the heat transfer from the body core to the ambient, which makes it difficult to obtain the highest amount of the thermal gradient. To effectively harvest body warmth, it is necessary to keep this thermal gradient On the other hand, utilizing a typical inorganic thermoelectric material like bismuth telluride for low-grade energy harvesting such as human body warmth forces us to use a heat exchanger since these materials have relatively high thermal conductivity, which makes it difficult to maintain the temperature gradient across the device. Consequently, adding a heat exchanger increases the total size and cost of the device. Organic thermoelectris can be an alternative, as the lower thermal conductivity in these conducting polymer–based materials results in a uniform thermal gradient distribution across the device. Besides, these materials are cheap and solution processable, which makes them suitable for fabricating the TEGs cheaply and quickly. The possibility of tailoring the state-of-the-art manufacturing method such as the additive manufacturing also brings the opportunity of doing the miniaturization and mass production at the same time. Despite all these advantages, organic materials have a significantly lower Seebeck coefficient compared with the inorganics counterpart and are normally processed in thin film architecture, which leads to lower efficiency and requires it to be retrieved through the device design and architecture optimization. In this study, we tried to address the low thermal gradient in the printed thin film devices through device geometry modification, which can be considered as a part of the TEGs’ system level optimization.All utilized additive manufacturing methods deposited the thermoelectric active material with a thickness of tens of microns to fabricate the thermocouples. This limitation makes the use of in-plane thermal gradient  and lateAs shown in l, L, and W are the thermoelectric legs thickness, length, and width, respectively, and other geometrical parameters of the devices are shown in T, and voltage, V, are calculated using the default governing equation in this section. The following are the differential equations utilized to model the TEG:s is Seebeck coefficient, K is the thermal conductivity, and σ is the electrical conductivity, respectively. In this work, the conducting polymer (PEDOT: Tos + TDAE) [The multi-physics simulation in this work is carried out through the thermoelectric section of the COMSOL software package, version 5.3 . Both the temperature variation,  + TDAE)  is used 2-K, which is calculated based on the ANSI/ASHRAE Standard 55 [ndard 55  and as aIn this section, it is shown that the proposed device geometry can be further optimized by altering the different parameters and a finite element method (FEM) tool like COMSOL Multiphysics version 5.3—supported by COMSOL A/S, Lyngby, Denmark. The comparison between the alternative situations is carried out based on the produced maximum output power in the form of voltage-current-power curves. This maximum power is basically a function of the TEG’s length, width, thickness, and the amount of the total thermal gradient across the device. It normally occurs when the device electrical resistance matches the external load resistance, based on the maximum power transfer theorem. DV is the produced electrical potential by the device, which makes the electrical current of DI flows through the external load of externalR. tegR is the overall electrical resistance of the TEG, and OV is the open circuit voltage. The produced power is calculated as follows:Where nr and pr are the electrical resistivity of the n- and p-type elements, A is the cross-sectional area, and L is the length of each leg, respectively. However, in the case of TEGs, a nonlinear behavior, because of the Joule heating and Peltier effect, needs to be considered to find the maximum amount of power. The combination of these alternative situations and parameters are simulated by means of the COMSOL version 5.3  thermoelectric model and will be discussed in this section (tegR. According to the maximum power transfer theorem and Equation (1), the maximum power could be reformulated asFor further investigation of the geometric parameters, the thermoelectric widths are altered in two different scenarios. First, it is assumed that the total number of thermocouples is fixed, and then in the second case, the number of thermocouples increases to fill the gap of decreasing the widths (4 legs with width of 3 mm and 12 legs with width of 1 mm). A new concept for system level optimization and manufacturing of printed organic thin film thermoelectric generators is introduced. It is shown that folding the flexible substrate after printing the thermoelectric materials can retrieve the small amount of thermal gradient for the lateral device configurations. It is also indicated that the concepts can be more customized by means of an FEM tool like COMSOL Multiphysics version 5.3  in terms of the device geometry in order to achieve the best performance with regard to the maximum feasible electrical power. At the end, it is also worth mentioning that a prototype demonstration made by printed thermoelectric legs and interconnects on a kapton substrate is already underway to further proof the concept. Modeling of the different thermal and electrical parasitics will also be a part of the future optimization.


Bidirectional brain-machine interfaces (BMIs) establish a two-way direct communication link between the brain and the external world. A decoder translates recorded neural activity into motor commands and an encoder delivers sensory information collected from the environment directly to the brain creating a closed-loop system. These two modules are typically integrated in bulky external devices. However, the clinical support of patients with severe motor and sensory deficits requires compact, low-power, and fully implantable systems that can decode neural signals to control external devices. As a first step toward this goal, we developed a modular bidirectional BMI setup that uses a compact neuromorphic processor as a decoder. On this chip we implemented a network of spiking neurons built using its ultra-low-power mixed-signal analog/digital circuits. On-chip on-line spike-timing-dependent plasticity synapse circuits enabled the network to learn to decode neural signals recorded from the brain into motor outputs controlling the movements of an external device. The modularity of the BMI allowed us to tune the individual components of the setup without modifying the whole system. In this paper, we present the features of this modular BMI and describe how we configured the network of spiking neuron circuits to implement the decoder and to coordinate it with the encoder in an experimental BMI paradigm that connects bidirectionally the brain of an anesthetized rat with an external object. We show that the chip learned the decoding task correctly, allowing the interfaced brain to control the object's trajectories robustly. Based on our demonstration, we propose that neuromorphic technology is mature enough for the development of BMI modules that are sufficiently low-power and compact, while being highly computationally powerful and adaptive. The possibility of controlling a prosthetic device through a direct interface with the central nervous system represents a promising solution for restoring sensory-motor functionalities in patients with limb amputations or peripheral and neurological deficits due to spinal cord injury, amyotrophic lateral sclerosis, or stroke. In the last two decades, a fast-growing worldwide scientific community has developed several brain-machine or brain-computer interfaces  toward the clinical application of these devices. Such interfaces offer also a powerful tool for exploring the sensory-motor mechanisms of control, adaptation, and learning that are employed by the central nervous system. This research has been assisted both by progress in our understanding of the underlying neural processes that take place in the brain, and by technological advances that have dramatically improved the quality of the signals recorded from the brain and the possibility of managing and processing large amount of data in real-time  fully implantable for long term use and therefore miniaturizable; (b) able to reliably process neural signals with a limited power budget; (c) powerful enough to implement non-trivial computational tasks involved in a BMI system. Additionally, the decoding algorithms need to be (d) sufficiently flexible to be implemented with different types of hardware components, and (e) able to dynamically adapt to changes in the neural activity due to the interaction with the artificial device . The task of the decoder consists in extracting from each evoked neural response a resulting force, calculated as a weighted sum of the four calibration forces defining the force field. In particular, the decoder needs to extract the four coefficients corresponding to the contribution of each of the four calibration forces to the decoded force.The calibration force corresponding to each region was defined by a vector pointing from the region's centroid to the target  anesthetized for the entire duration of the experimental sessions by means of Xylazine (5 mg/kg) and a mixture of Tiletamine and Zolazepam (30 mg/kg). Two craniotomies were performed above the somatosensory (S1) and the motor (M1) cortex representing the whiskers on the same hemisphere. The stimulation microwire array (Tucker Davis Technologies—TDT) was lowered perpendicular to the somatosensory cortex 300–500 μm under the surface . The recording array was placed at depth 900–1100 μm below the pia  using a hydraulic microdrive. These locations have been chosen for the presence of several cortico-cortical connections between the two regions that can be connected to satellite modules, each dedicated to specific tasks as decoding the neural signal, controlling the movement of an external device, and encoding the information collected from the external environment to provide sensory feedback. The MU does not require any information about the specific implementation of each module, which can be a software running on general purpose processing units, a dedicated programmable hardware such as Field Programmable Gate Arrays (FPGA) or a neuromorphic chip. This modularity ensures a fast and flexible prototyping phase required during research and development, whereby different software modules can allow testing the algorithms to be implemented on custom low-power, miniaturized implantable hardware.In this implementation, we connected five different satellite modules to the MU realizing the functionalities required by a bidirectional BMI: Acquisition Unit, Stimulation Unit, Decoder, Encoder and Dynamical System, as shown in Figure mbBMI algorithm is in charge of reading the spiking neural data coming from the Acquisition System module and communicating them to the decoder. Once the decoder generates an output signal, the algorithm transforms it into motor commands usable by the Dynamical System. To close the loop on the brain, the algorithm acquires the current position reached from the external device and communicates it to the encoder that returns the next stimulus to be communicated to the Stimulation System module.The main algorithm running on the MU named TM equipped with a Xilinx Zynq®-7000 All Programmable System On Chip (SoC). The Zynq®-7000 family integrates a feature-rich dual-core ARM CortexTM-A9 based processing system (PS) and 28 nm Xilinx programmable logic (PL) in a single device. In our implementation, the PL runs a custom module that can interface with neuromorphic chips and implements two software modules named NeuElab and Dynamical System Controller. The NeuElab module acquires the pre-processed brain signals from the mbBMI algorithm and routes them to the decoder and vice versa, via its hardware interface (Zynq2Neuro described in Section 2.3.1).We implemented the Managing Unit by using the development board ZedBoardFigure 5) and then forwarded to the neuromorphic chip. The MU communicates with the decoder using the native neuromorphic asynchronous communication protocol, known as Address Event Representation (AER) protocol  of the emitting electrode. At the end of each recording period, spike trains are generated from the recorded spike time-stamps according to the decoder's requirements  that set the operating point of the analog circuits. NeuElab can be used, in principle, for interfacing the BMI with any neuromorphic chip that uses the AER communication protocol. In this implementation, the output spiking activity of the neuromorphic chip is translated into a bidimensional force applied to the Dynamical System by means of a pair of Pulse Width Modulated (PWM) analog signals generated by the ZedBoard that drive the external object.The decoder that transforms the recorded brain activity into motor commands is implemented on a neuromorphic chip. In the following, we describe the chip and the printed circuit board (PCB) that we developed to connect the chip with the rest of the system.2 and has approximately 12.2 million transistors. It comprises 256 adaptive exponential integrate-and-fire neurons implemented in a mixed signal analog/digital circuit design.The Reconfigurable On-line Learning Spiking (ROLLS) Neuromorphic Processor is a general-purpose spiking neural network chip . The Z2N  PCB that can host up to two daughterboards (DTB) that mount neuromorphic chips. The Z2N connects the neuromorphic chips to the FNC connector of the ZedBoard, supplies power to the chips and supports the AER communication and the chip configuration signals. Analog biases that configure the parameters of the silicon neural and synaptic models on the neuromorphic chip can be set either by means of external digital to analog converters (DAC), or by on-chip programmable bias generators (BG) .We configured the ROLLS chip to implement a feed-forward spiking neural network that exploits the spike-timing dependent plasticity of the chip's LTP synapses to learn how to extract the pattern of four calibration forces that should result in the net desired force, from the recorded neural activity. Each of the output neurons of the network was trained to act as a binary classifier by re-weighting the features of its input that were distributed across its synapses, so as to ultimately yield, via its activation function, a higher output spike rate for one, positive class of input compared to the other three, negative classes. Neurons were grouped into four ensembles, each corresponding to one of the four stimuli. The spike counts output by the four ensembles during the presentation of the recordings to the network were directly used as the coefficients that weight the contributions of the four component forces acting on the BMI's end effector.The spike-based learning algorithm implemented on the chip is based on the model proposed in Brader et al. . Using tTo reconcile the characteristics of the data with the network requirements we mapped uncorrelated sub-samples of the spike sequence to different synapses of the classifier neuron, using a mean-rate encoding. Specifically, we binned the recorded spike trains in time bins of 1 ms Figure  and assoUnder the constraint of a finite number (256) of available synapses per neuron, there was a trade-off among the number of recording channels, the duration of the recording patterns, and the temporal precision desired. The first 200–300 ms of each recorded pattern included significant differences across the four different classes Figure , that woThe aim of the BMI is to best approximate the desired force field over the duration of the experimental session, through weighting the four force components. To achieve this aim, there are two criteria based on which the decoder has to simultaneously optimize its learning. Firstly, it needs to learn to classify the patterns, i.e., to correctly output the single class to which each presented recording truly belongs, as expressed by the “winning”  ensemble of output neurons. Secondly, the decoder also needs to prevent the other three “losing” ensembles from biasing the force field toward particular directions on average over the trajectory of the end effector. That is, it needs to classify the recordings under the constraint of learning to equalize the average outputs of “losing” ensembles. Thus, despite the similarities to a classifier, classification of individual recordings is only partly the decoder's task.The decoder had to address certain additional characteristics of the recordings to achieve its goal of approximating the desired force field over the experiment's course. Specifically, different classes of recordings differed in number of recorded spikes on average, and this difference in the input energy could be reflected as a bias in the chip's output and consequently in the direction of the decoded force in each trial. Moreover, even though spike timing was the principal difference between recordings of different classes, some spike timings were common between classes. This increased the difficulty in distinguishing between different classes. That is, the different classes had a certain level of overlap between their features, which could increase classification errors. Additionally, this overlap was not of the same extent for all pairs of classes, i.e., some classes were more similar to some than to others in terms of common spike timings Figure . This asTo address these points, we used the “stop learning” feature of the ROLLS chip learning circuits . During the training procedure were randomly interleaved Figure  and presOnce the decoding and encoding interfaces were properly calibrated, in order to test the system we ran the BMI by decoding from each neural trial a bidimensional force and by encoding each position of the controlled object through an ICMS pattern. We used a test dataset of neural recordings acquired by 10 repetitions of each of the four stimulation patterns , which were unseen by the BMI during its training. We selected eight different equispaced and equidistant positions as starting points in which the dynamical system was initialized and we ran the BMI 100 times starting from each initial position by obtaining 800 trajectories. We tested the system under two conditions: under normal operation (encoder-ON condition), each test recording was selected according to the dynamical system's current position. An alternate condition (encoder-OFF) was used to test the bidirectionality of the BMI and the learned coordination between the encoder and decoder modules. In the encoder-OFF condition, each test trial was randomly selected among all 40 test recordings.within-trajectory variance (abbreviated to wtv). In particular, each trajectory's wtv is defined as Cx and Cy is the covariance of the distribution of the per-step displacement along the x and the y axis, respectively. We obtained the mean wtv by averaging the wtv computed for each set of trajectories that started from one initial position.To assess the repeatability, the speed and the optimality of the generated trajectories we measured the number of steps required to converge to the target and the mean To assess the decoder we used test datasets, which were previously unseen by the decoder, as described in Section 2.6. For each decoded pattern, the output spikes produced by each neuronal ensemble Figure  were couFigure 8B).In addition, as a result of the introduction of “stop learning” to the silicon neurons average spike counts were relatively uniform across the other three ensembles despite the biases in pairwise similarities between input classes (see Section 2.5.4). The chip learned to suppress this bias, and, consequently, decoded resultant forces for each stimulus were, as originally intended, most similar to one of the four forces used during the calibration phase  showing a decrease of 99% in the distance from the target and demonstrating that closing the loop in the proposed BMI is crucial in order to correctly drive the dynamical system toward a target.In order to assess the BMI performance, we performed two different testing sessions: during the first session we set the maximum number of steps to 100 as stopping rule for the obtained trajectories Figure . The BMIIn the second testing session, we simulated a real experiment in order to generate motor commands that drive a mobile cart from predefined initial positions toward a target position represented by a slot in the glass that allows the rat to get the reward  with respect to the encoder-OFF case  in the case of encoder-ON and slightly negative (divergent from the target) when the encoder is turned OFF . In the OFF condition, this can be attributed to the randomness of the motion. In the ON condition, combined with the increased DT force, this is an indication of successful decoding.Finally, for each force produced by the decoding process, we measured the magnitude of two components: the component of the force that points toward the target point, named Figures In this paper, we showed the applicability of neuromorphic hardware in a brain-machine interface system, in the first demonstration of this kind. In particular, the decoder module of the BMI was implemented by a spiking neural network on a mixed-signal analog/digital neuromorphic processor, the ROLLS, that learned to perform on-line the decoding of the neural recordings into commands that addressed the brain-controlled device.The analog neuromorphic circuits of the ROLLS neuromorphic processor emulate functions of biological neurons and synapses by replacing biophysical properties with analogous properties of the sub-threshold physics of transistors. The resulting spiking neural networks operate on a power-efficient and compact system for applications of pattern recognition such as a BMI decoder's task. On the other hand, because of these underlying principles of operation, analog neuromorphic circuits like the ones found on the ROLLS are imprecise and variable, similar to biological neural elements, in sharp contrast to simulations of spiking neurons and synapses on digital neuromorphic or general-purpose processors. The neuromorphic decoding task was further complicated by the variability in the recorded data, and by the overlap in spike-timings between the to-be-discriminated classes.Further difficulty arose by the fact that the decoder's task was not a standard classification task, as the BMI required the decoder to output a contribution of all potential classes of recorded activity simultaneously, while preventing the average chip output from being biased toward any pair of classes, even though the pair-wise similarities between classes were biased.Despite these particularities, the spiking network we designed successfully learned the decoding task, enabling the BMI to perform at similar levels of a previous non-neuromorphic version of the bidirectional BMI. This was achieved by exploiting two key characteristics of the ROLLS chip: variability between silicon synapses and neurons deployed into an ensemble learning technique that aggregated multiple weak classifiers into a powerful one, and the heterosynaptic competition through the “stop-learning” feature of synapses on the ROLLS chip, which enabled the network to focus on the discriminative features of the input, thus both improving classification performance and reducing the reflection of biased similarities in the input onto the output of the trained network. A key feature of the decoder is that the spiking output of the neuromorphic chip is directly used to compute the force controlling the end-effector. The components of the force were weighted by the spike counts of the chip's output, an important step toward using neuromorphic hardware not only as a decoder, but also as prosthetic controller.The set-up we propose has been designed as an initial proof of concept prototype to evaluate the potential of neuromorphic hardware computing in BMIs, and to determine its limitations; within this context, this work shows that, even at this level, integration of neuromorphic hardware in set-ups characterized by the intricacy of a bidirectional BMI is technically possible. Our results show that, despite the low precision, low resolution, and noisy (but compact and low-power) analog electronic circuits in the neuromorphic chip, the system built in this way can recognize multi-dimensional input patterns. In particular, the results demonstrate how this neuromorphic hardware can be configured to produce the correct average forces over the controlled object's trajectory Figure , despiteThe simplicity of the single-layer feed-forward network of only 252 neurons that was employed for this particular application demonstrates the limitations and computational power of physical instantiations of spiking neural networks and suggests that further development of analog neuromorphic hardware and spike-based algorithms may yield a computationally powerful, yet low-power consuming alternative to software and conventional processors for a broad spectrum of tasks. With respect to the neuromorphic BMI decoder in particular, further work could enable two specific improvements and additions.Firstly, the present implementation addresses the complex temporal dynamics of the recordings with a processing step introduced between the neural recording and the output layer of the neural network, and performed off-chip, which transforms the temporal dynamics of the recordings to a spatial pattern input to the chip. While the method proposed is suitable for the presented system, we have been investigating alternative algorithms and spiking neural network architectures that can potentially decode and recognize these types of spatio-temporal patterns entirely on the chip. In this way, the chip could directly receive the recorded spike train, and operate on it with no need for an intermediate off-chip storage step. This would be possible because of the ROLLS' real-time operation, with time constants that match those of real neurons. To this direction, Corradi and Indiveri  perform On a separate but related note, here the BMI operated in discrete time steps. This permitted us to insert the processing step that inputs the recorded spike timings as rate-coded patterns into the ROLLS chip, without loss the system's continuity. Nonetheless, this will be a crucial obstacle for the decoding module's integration in future continuously operating BMIs. On the other hand, the limitation does not originate in the ROLLS chip itself. The chip does not have an internal clock that must be synchronized with the chosen time points. It rather recognizes inputs in which time represents itself in the spike train's statistics. This implies that removing any off-chip transformation that intermediates the input would also enable the on-line use of the chip in continuous-time BMI set-ups.As a further future improvement, the fact that the network learns on line could be used to allow the decoder to adapt to changes in the neural responses with time. Specifically, in the current implementation, the decoder updates itself incrementally after the presentation of each training pattern. Training inputs are combined with a teacher signal that biases different neurons to strengthen or weaken their connections to different features of the input, through imposing different levels of output firing during the presentation of different input classes. After training, we use the chip to decode new recordings of brain activity. The on-line learning feature is not crucial for demonstrating the performance of the BMI in its current instantiation, but can become useful in future chronically implanted setups, that have to adapt to continuous slow changes in the nature of the signals being recorded. In such a future implementation, learning could continue during the chip's use as a trained decoder. As the trained silicon neurons respond with high firing rates to their corresponding input classes, and with lower rates to the other classes, the neurons could bias themselves to continue correctly adapting their synapses to the input patterns in the absence of an externally provided teacher signal. This would be made possible after tuning the parameters of STDP synaptic dynamics of the ROLLS to enable potentiation and depression in the ranges of firing rate that the trained neurons output when decoding the input.As technological and scientific progress accelerates, it brings new opportunities for improving the quality of life of millions of people. The interdisciplinary field of brain-machine interfaces largely relies on the rapid evolution in the diverse fields that are involved (Nicolas-Alonso and Gomez-Gil, http://www.sicode.eu/results/software.The modularity allows to develop different parts of the BMI in different labs and assemble the complete system by plugging in these parts as modules. This structure makes easier and more reliable both the implementation of the single module and its integration in the complete system. Parallel development of components could also accelerate the ultimate realization of a device compact and powerful enough to be used as clinical tool able to transfer data between the brain and external devices wirelessly through an implanted interface (Azin et al., The relevance of neuromorphic technology in the design of brain-machine interfaces is demonstrated by the flourishing work in this domain (see Dethier et al., As in Vato et al. , we closFB, VD, FD, and CB designed, built, and debugged the hardware and software infrastructure for the mbBMI. TM and GI implemented the spike-based learning and decoding algorithm on the ROLLS neuromorphic processor. FB and AV performed the experiments and collected the neural data. FB, VD, and TM performed all the analysis presented in the paper. All the authors contributed in writing and editing the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


We present a pattern recognition framework for semantic segmentation of visual structures, that is, multi-class labelling at pixel level, and apply it to the task of segmenting organs in the eviscerated viscera from slaughtered poultry in RGB-D images. This is a step towards replacing the current strenuous manual inspection at poultry processing plants. Features are extracted from feature maps such as activation maps from a convolutional neural network (CNN). A random forest classifier assigns class probabilities, which are further refined by utilizing context in a conditional random field. The presented method is compatible with both 2D and 3D features, which allows us to explore the value of adding 3D and CNN-derived features. The dataset consists of 604 RGB-D images showing 151 unique sets of eviscerated viscera from four different perspectives. A mean Jaccard index of  Poultry is both the most popular and the fastest rising source of meat in the USA . The numThe current process for inspecting the viscera involves strenuous manual labour, which limits the throughput of the processing plant. The operators must inspect approximately three chickens per second. In addition to inspecting the viscera and corresponding carcasses, the birds that fail the inspection must be manually removed from the production line. Manual inspection is inherently slow, expensive and inconsistent; all of which represent a challenge when attempting to comply with the increasing demands from consumers.Machine vision has previously been explored for inspection and grading of poultry. A great introduction to the problem and existing work on vision-based poultry inspection is given in , where ample, in , the segWith this work, we exploit the recent advances in computer vision and pattern recognition by applying a modern visual perception framework to an unusual dataset in a problem domain that has seen limited progress in the last decade. We make use of a convolutional neural network (CNN) and 3D-derived features and demonstrate their contribution to the overall performance of the semantic segmentation of viscera. This paper is an extension of the work in our original conference paper . The extThis work can be summarized as: (1) using CNN activation maps as superpixel features; (2) quantifying the value of CNN- and 3D-derived features for semantic segmentation; (3) proposing a method for visual localization of organs enabling automated quality control.Quality control of organic material is based largely on hyper-spectral imaging (HSI). HSI makes it possible to capture nuances in colors that are normally not visible with RGB cameras ,11,12,13More complex approaches with segmentation of multiple organs are less explored. An example is found in , where RMost existing research in semantic scene segmentation is focused on scenes with man-made objects. Contrary to the organs in our viscera dataset, man-made objects comprise, to a large extent, straight lines and clean edges. Two widely used datasets with these types of objects are NYU-D 1  and NYU-Deep-learning architectures for semantic pixel-wise segmentation have, as in many other areas, raised the bar significantly from the former approaches that relied on handcrafted features and classic classifiers. The VGG CNN architecture  has proveference  proposeseference . They coeference , the preeference , the autA dataset consisting of 151 sets of chicken viscera was collected for our previous work . Each seData is captured using the RealSense F200 3D camera and the RealSense SDK  from IntA ground-truth (GT) is needed to train and evaluate supervised learning algorithms. For this dataset, the GT is obtained by manually annotating pixels belonging to the three classes of interest, namely heart, liver and lungs, and an additional fourth category encompassing miscellaneous organs and tissue. An ignore region is established around each organ, as labelling data is an ill-posed problem where the assignment of labels near borders is ambiguous. Unlike in our preliminary work , where oIn this paper we apply a framework for semantic segmentation inspired by the work of Müller  and WolfBoth the RF and the CRF are supervised classification methods that require training on labelled data. The dataset of 151 viscera sets are split into an RF training set of 91 sets, a CRF training set of 30 sets, while the remaining 30 sets are reserved for testing.Oversegmentation is often used as a pre-processing step, where similar pixels are grouped into superpixels in the 2D case , or supeIn this framework the voxel cloud connectivity segmentation (VCCS)  supervoxFeature maps are obtained in preparation for the feature extraction and include the color channels from the LAB space, depth, normal magnitude for each of the three dimensions, and activation maps from across different layers of a CNN. Examples of each type of feature map are shown in The features used for classification are extracted for each superpixel. The feature responses for each feature map are averaged for all pixels in a superpixel. Two sets of features are extracted. Unary features are extracted for the RF, and secondly, pairwise features are found for the CRF.Unary features are used in the RF to assign probabilistic labels to each superpixel. These probabilities serve as the unary component of the optimization performed later by the CRF. The features that are useful here are good at discriminating between the classes. The unary features used in the RF are listed in Edge features are used to measure the similarity of neighboring superpixels and serve as the pairwise component of the CRF. The features that are used here must be good at describing the similarity of two superpixels. The edge features for the CRF are listed in The unary potentials are used as an initial label estimate for the superpixels. Because the classification of the unary features is based entirely on local features, and since the neighborhood is taken into account at a later stage, these labels are assigned with a probability instead of one-hot encoding. Thereby, the later optimization can take into account the uncertainty of the initial classification. In this work the RF classifier  is used The RF is implemented using the scikit-learn library . The datThe random forest prediction based on local features can be further improved by taking neighborhoods into account. This is done using a CRF , where tThe optimization that the CRF performs is based on the minimization of the energy function consisting of unary potentials and pairwise potentials. The unary potentials originate from the class label probabilities of the RF classifier. In this work the pairwise features consist of difference in position and color information, and normals in the 3D case, all learned from the separate training set. Note that each class is weighted according to the inverse class frequency. The CRF is implemented using the freely available PyStruct . It is sThe value of adding 3D and CNN features is documented by progressively adding the different feature types to the semantic segmentation pipeline and evaluating the performance. The evaluation is based on 120 images displaying 30 unique viscera from different angles. Evaluation is performed at the pixel level. The regions with background and the 2-pixel-wide boundary area around classes are excluded.The segmentation is scored by the Jaccard index shown in , measuriThe Jaccard index is found for each class and averaged, without weighting according to class occurrence, for a combined score. For the results based exclusively on 2D features, the features that are related to 3D are disabled. This is impacting the clustering into superpixels, as the geometric similarity is an important feature, as well as the RF classification from the lack of depth and the CRF from the lack of normal features. The additional information of both 3D and CNN is shown to improve the segmentation as seen in The use of superpixels results in an upper limit on overall accuracy, which depends on the object boundary adherence of the oversegmentation. The impact of the oversegmentation is measured using an upper bound, namely the achievable segmentation accuracy (ASA)  defined The confusion matrices of The examples shown in We presented a framework for automatic visual inspection of poultry viscera, a domain with a weak spatial arrangement without sharp edges and with organs often occluded by fat. We show how segmentation algorithms previously used on man-made objects are able to function on these deformable objects, scoring a mean Jaccard index of


It has been shown that increased muscular activity along the trunk is likely to impair body balance, but there is little knowledge about its consequences on more dynamic tasks. The purpose of this study was to determine the effect of unilateral and bilateral increases of muscular tension along the trunk on the sit-to-stand task (STS) performance and associated anticipatory postural adjustments (APAs). Twelve healthy females , free of any neurological or musculoskeletal disorders, performed six trials of the STS at maximum speed, in seven experimental conditions varying the muscular tension along each side of the trunk, using a specific bimanual compressive load paradigm. A six-channel force plate was used to calculate the coordinates of the center of pressure (CP) along the anterior-posterior and medial-lateral axes, and the kinematics of the head, spine and pelvis, were estimated using three pairs of uni-axial accelerometers. The postural and focal components of the task were assessed using three biomechanical parameters calculated from CP signals: the duration and magnitude of APAs, and the duration of focal movement (dFM). Results showed that beyond a given level, higher muscular tension along the trunk results in longer APAs, but with a stable duration of the focal movement. In addition, no significant variation of APAs and FM parameters was found between bilateral and unilateral increases of muscular tension. It was suggested that restricted mobility due to higher muscular tension along the trunk requires an adaptation of the programming of APAs to keep the same level of performance in the STS task. These findings may have implications in treatment strategies aimed at preserving functional autonomy in pathologies including a rise of muscular tension. According to the seminal work of Bouisset and Zattara , voluntaAccording to Bouisset and Le Bozec , posturaTo date, a number of experiments have explored the effect of passive restrictions of postural chain mobility on voluntary movements. For example, a lower performance was shown to be associated with restricted pelvis mobility in a pointing task , free of any neurological or musculoskeletal disease took part in this study. To avoid any variation of muscular strength and spine mobility related to gender differences, we did not include male participants.Twelve healthy female subjects , which collected the ground reaction forces and moments applied at its top surface was used to calculate the coordinates of the center of pressure (CP) along the anterior-posterior (Xp) and medial-lateral (Yp) axes, with the following formulas: Xp = My/Rz Yp = Mx/Rz .A stool  with four legs and a round wooden top (diameter = 30 cm) was screwed on to the force plate and used for the experiments Figure . To keepA pressure sensor was placed under the left front leg of the stool to determine the “seat off” instant.Three pairs of mono-axial accelerometers , were used to assess the local accelerations of the pelvis and trunk. Each pair was screwed on to a customized cube (length = 2 cm) with the two active axes located along the anterior-posterior and the vertical axes. The cubes were adhered to the skin with double-sided tape at the level of the first sacral vertebra and first thoracic vertebra  were used to set the active muscular tension at different levels alongside the torso when performing a bimanual compressive load paradigm , controlled by a custom code written in Labview software .The participants sat on the platform with hips and knees approximately flexed to 90°, barefoot and feet apart. Adhesive tape was put around the feet outline in order to keep the same positioning for every trial. The load cells were positioned against hands palms and applied on the sides of the pelvis against the lateral surface of the trochanter major, with shoulders abducted to 35° Figure . SubjectTwo 3-s trials in bilateral maximum compressive load were first performed to determine the MVC. Subjects had next to achieve the STS transfer while performing a compressive load in seven conditions varying the load level  and side . It must be noted that for a compressive load at 0% (no force applied) the experimental instructions were the same for left, right and bilateral loads, and only one condition was considered. For each condition, the participants had to reach the requested compressive load parameters before starting the STS transfer and to keep them constant during the task, using the feedback screened on the visual field. This way, the variations of the muscular pattern that may be associated with the task completion were minimized.The “STS” paradigm consisted in rising from the stool to reach the standing position as fast as possible, in response to a “Go” signal. A training period was used to familiarize the subjects with the paradigm before recording. Six 3-s runs were performed in every condition, with six additional runs in 0% for subsequent statistical analysis. The rest period was 30 s between runs and 1 min between series.The order of the experimental condition was randomly assigned to avoid any order effect.A careful visual inspection of CP curves along the anterior-posterior and medial-lateral axes was first performed for qualitative assessment . It was followed by a quantitative analysis of the APAs and FM of the STS. These two distinct phases were considered to be separated by the seat-off instant, for which the curve of the pressure sensor starts its fall towards zero. Three parameters were calculated from CP signals along the anterior- posterior axis:–dAPAs): delay between the instant of seat-off and the first inflexion of the CP curve.Duration of APAs (–Xp): difference between the maximum and the minimum values of CP during the APAs phase.Magnitude of APAs (Δ–dFM): delay between the seat-off instant and the stabilization of the CP curve (beginning of the plateau region).Duration of the focal movement .p < 0.05.Statistical analysis was performed using the Statistical Package for Social Sciences (SPSS) software V22 . Normality of data distribution and the condition of sphericity were first checked using Shapiro-Wilk and Mauchly tests, respectively. A two-way repeated measures analysis of variance (ANOVA) was conducted for each dependent variable, with compressive load intensity  and laterality  as within-subjects factors. Values for the unilateral level were calculated from the mean between the left and right trials, as this study did not focus on specific left and right variations that might have resulted from handedness. When statistical significance was reached for compressive load intensity, the ANOVA was followed by a within-subjects analysis of contrasts (difference contrast) to compare the three levels of the independent variable. The level of statistical significance was set at The CP time course along the anterior-posterior axis revealed a typical shape which always starts with an almost linear backward displacement (beginning of the APAs), followed by a forward displacement with a steeper slope, and which ends with a less regular stabilization phase preceding the plateau  in unilateral conditions.p < 0.05). Subsequent contrast analysis showed a significant variation between a compressive load at 40% (comp40) and at the two other levels, namely 0% (comp0) and 20% (comp20), but no substantial difference between comp0 and comp20 conditions (Table Xp) and focal movement duration (dFM) were not sensitive to the level of the applied load . This variation may be interpreted as a reorganization of the motor pattern, aimed at compensating for the lower capacity to generate counter-disturbing movements due to higher active muscular tension. It has already been assumed that postural chain muscular stiffening may slow down the counter-perturbing movements necessary to keep the body stable, and make them less efficient  and central nervous  systems, the results from this study are worth being interpreted in a clinical perspective. It can first be assumed that a pathological increase of trunk muscular tension, when it exceeds a given level, may require an adaptation of the APAs pattern in order to keep the same level of performance in the STS task. It can also be considered that a limited asymmetry of trunk muscular tension may induce a negligible effect on the STS task performance. However, as increased tension was experimentally induced on healthy participants in this study, it is necessary to take into account the possible interaction with other symptoms in a pathological context, which may lead to more prominent effects. This might be the case in Parkinson’s disease or stroke, which are both associated with other deficits and have proved to impair STS task performance (Inkster and Eng, There were some methodological limitations of the study which need to be acknowledged. First, we used a model of increased muscular tension based on voluntary contractions performed in healthy subjects, which intrinsically differ from pathological variations related to deficits of the nervous or musculo-skeletal systems. Second, the STS task was performed at maximum velocity in young adults, whereas slower movements are frequently observed in a pathological population, with possible variations of the focal and postural components of the task. Third, we only used participants with a normal BMI, whereas the STS task requires the generation of high propulsive forces that may lead to different strategies in subjects presenting high BMI. For these reasons, future experiments are needed to confirm our hypotheses in a more realistic context, including different pathologies and classes of participants.The findings of this study support the view that experimentally increased muscular tension along the trunk requires an adaptation of the APAs, which become longer, to maintain the same level of performance in the STS. This effect, which might be more prominent in a clinical context where it is bound be associated with other symptoms, may have some implications in treatment strategies intended to improve functional autonomy.AH contributed with project creation, data analysis, and drafted the manuscript. CA-R contributed with project creation, data collection and data analysis. AH and CA-R discussed the results and revised the manuscript.This work was funded by University JF Champollion and Conseil Régional Occitanie, France.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The handling Editor currently co-hosts a Research Topic with one of the authors AH, and confirms the absence of any other collaborations. He states that the process met the standards of a fair and objective review.