First coined by Howe in 2006, the field of crowdsourcing has grown exponentially. Despite its growth and its transcendence across many fields, the definition of crowdsourcing has still not been agreed upon, and examples are poorly indexed in peer–reviewed literature. Many examples of crowdsourcing have not been scaled–up past the pilot phase. In spite of this, crowdsourcing has great potential, especially in global health where resources are lacking. This narrative review seeks to review both indexed and grey crowdsourcing literature broadly in order to explore the current state of the field.This is a review of reviews of crowdsourcing. Semantic searches were conducted using Google Scholar rather than indexed databases due to poor indexing of the topic. 996 articles were retrieved, of which 69 were initially identified as being reviews or theoretically–based. 21 of these were found to be irrelevant and 48 articles were reviewed.This narrative review focuses on defining crowdsourcing, taxonomies of crowdsourcing, who constitutes the crowd, research that is amenable to crowdsourcing, regulatory and ethical aspects of crowdsourcing and some notable examples of crowdsourcing.Crowdsourcing has the potential to be hugely promising, especially in global health, due to its ability to collect information rapidly, inexpensively and accurately. Rigorous ethical and regulatory controls are needed to ensure data are collected and analysed appropriately and crowdsourcing should be considered complementary to traditional research methods. Wired magazine article  all knowledge resides in humanity; digitalisation and communication technologies must become central in this coordination of far flung genius” . Althoug article . In the  article  and he a article . Since H article ,7, use o article . Althoug article ,9–11.Wired article, Louis van Ahn introduced the idea of human computing, where humans are used to solve complex problems that computers are not capable of [As crowdsourcing requires, depending on the definition, ‘outsourcing’ a task or tasks to a large crowd , advancepable of . While mpable of . In addipable of ,18. Cellpable of . Crowdsopable of .As individuals are biased towards the correct answer, Buecheler et al. estimate that if a million individuals were to contribute towards answering a problem via crowdsourcing, there would be a 97.7% likelihood that the crowd would arrive at the correct answer . While pDespite the interest in the area of crowdsourcing exploding in the past decade, many authors do not agree on its definition or on what counts as crowdsourcing, with some academics considering Wikipedia a “classic” example of crowdsourcing, for example, while others insist it is not crowdsourcing . Text anIn addition to there being many definitions of crowdsourcing, many authors have offered different taxonomies of crowdsourcing, some focusing on types of crowdsourcing while others focus on its production model. Furthermore, there are debates on who participates in crowdsourcing – whether it is laypersons, amateurs, professionals, experts, or a combination.Although crowdsourcing has existed for decades, it is agreed upon that technology has facilitated its growth. Platforms such as Amazon Mechanical Turk and Crowdflower enable companies to hire workers to perform crowdsources exercises for extremely low prices. Other crowdsourcing platforms, such as Innocentive or Crowdmed, offer a competitive winner–takes–all model. Sensors in wearable technology have also facilitated the ability to collect mass amounts of information.Crowdsourcing can increase the accuracy of computer automated tasks, lower costs, increase the scale of research, transcend boundaries and borders, produce novel discoveries and increase the speed of research progression, among other benefits. However, there are concerns with the generalisability of the samples, as the crowd is self–selected, security and data protection issues of sensitive data, and the possibility of malicious workers. Some studies have added quality protection measures to weed out malicious workers, such as adding cut–offs for scores on previous tasks and screening questions. Additional regulation is needed for ethical issues, such as obtaining informed consent and data use policies.Crowdsourcing has considerable benefits in research, as it has the potential to substantially lower costs while massively increasing the sample size and researchers can receive the data in real–time ,19,27–29As previous authors had reported few search results in indexed journals ,27,39 anBox 1“Crowdsourcing”Up to 25 pages“Crowdsourcing” and “Health”Up to 15 pages“Crowdsourcing” and “Immunology”Up to 5 pages“Crowdsourcing” and “Genetics”Up to 9 pages“Crowdsourcing” and “Public Health”Up to 20 pages“Crowdsourcing” and “Disease”Up to 25 pages“Crowdsourcing” and “Surveillance”To 20 pages“Crowdsourcing” and “Diagnosis”Up to page 14Results were organised within Endnote into categories, including reviews, theory of crowdsourcing, health, public planning, GPS–related, translation, robotics, visual perception, logistics of crowdsourcing, which was broken down into motivations, quality, reliability, stability, and others. This review reports on the papers reporting on reviews and theory as well as a portion of the health–related papers, as there were 285 health papers and many of their interventions overlapped. Further reviews can be conducted with the results of the search and organised Endnote library, but are outside the scope of the current review.The reviews and theoretical papers generally covered the varying definitions of crowdsourcing, taxonomies of crowdsourcing, participants, modes of participation, when research is suitable for crowdsourcing, benefits and concerns with crowdsourcing, recommendations for regulation and quality control, including ethical regulations and examples of crowdsourcing.The definition of crowdsourcing as well as some ‘traditional’ examples of crowdsourcing, such as Wikipedia, are highly debated; this is likely due to both the relative newness of the term and the flexibility and adaptability of the method ,20,40–43Three terms, specifically, are used abundantly in literature and often interchangeably with crowdsourcing: health 2.0, wisdom of the crowds, and citizen science. While applications of crowdsourcing are often a combination of these, especially in the field of health, there are important distinctions between them ,8,11.Swan defines citizen science as non–professionals conducting science–related activities . Non–proHealth 2.0 is defined, also by Swan, as active participation in one’s health care using web 2.0 technologies . This coknowledge of a large crowd of people and also requires an intelligent crowd. This also differs, slightly, from the term crowdsourcing, as not all crowdsourcing tasks require knowledge or intelligence. Unlike citizen science and health 2.0, all ‘wisdom of the crowds’ tasks are forms of crowdsourcing, but not all crowdsourcing are necessarily applications of a ‘wisdom of the crowds.’ An example of a task requiring intelligence would be using a crowd to diagnose malaria cells in blood smears. In this, each participant needs to use their knowledge or intelligence to consider which blood smears contain or do not contain malaria parasites. Some, perhaps arguable, examples of crowdsourcing that would not be considered requiring knowledge could be RECAPTCHA, passive surveillance such as environmental surveillance using ubiquitous computing and mobile phones, reporting systems, or text mining. In his book, Surowiecki lists four requirements for an intelligent crowd that are particularly important for crowdsourcing tasks that require knowledge . They are: (i) diversity, which adds perspectives that would otherwise be absent; (ii) independence, limiting the influence of one person’s opinions on other’s; (iii) decentralisation, to develop tacit, specialised knowledge; and (iv) aggregation, to combine the diverse, independent, knowledgeable opinions of the crowd [“Wisdom of the crowds” is another related term. This refers to the use of he crowd .In addition to these three terms, crowdsourcing is often contrasted to open sourcing or outsourcing. Although some authors believe that crowdsourcing is a special form of outsourcing , many auCrowdsourcing has other key features including a clear, open call for participants and a large crowd. Since there are many different definitions, Estelles–Arolas et al. reviewed definitions of crowdsourcing and developed an integrative definition using Tatarkiewicz’s approach, which is based on developing a global definition of the concept of art. In their review, the authors found 8 key qualities of a crowdsourcing definition, namely: a) who forms the crowd; b) what the crowd has to do; c) how the crowd is reimbursed; d) who initiates the crowdsourcing process; e) what the product of crowdsourcing is; f) what type of process is used; g) what type of call is used; and h) by what medium the call is made .The integrative definition that the authors devise from their review is :“Crowdsourcing is a type of participative online activity in which an individual, and institution, a non–profit organisation, or company proposes to a group of individuals of varying knowledge, heterogeneity, and number, via a flexible open call, the voluntary undertaking of a task. The undertaking of the task, of variable complexity and modularity, and in which the crowd should participate bringing their work, money, knowledge and/or experience, always entails mutual benefit. The user will receive the satisfaction of a given type of need, be it economic, social recognition, self–esteem, or the development of individual skills, while the crowdsourcer will obtain and utilize to their advantage what the user has brought to the venture, whose form will depend on the type of activity undertaken.”Although he describes it as a taxonomy rather than a definition, the features of Geiger et al.’s description of crowdsourcing is similar to Estelles–Arolas et al.’s integrative definition. The key features Geiger describes are: (i) pre–selection of contributors ; (ii) accessibility of peer contributors (whether they can access each other’s contributions); (iii) aggregation (to what extent the input is used); and (iv) remuneration  ,44.Each feature of Estelles–Arolas et al.’s integrative definition and Geiger et al.’s taxonomy is discussed below.The majority of authors reviewed by Estelles–Arolas et al. did not provide a distinct definition for their crowds, instead describing a crowd as a large group of people or individuals, consumers, or volunteers . The autIn contrast, Brabham specifically examined how authors refer to crowds and found that the majority of articles refer to crowds as being composed of amateurs . HoweverWith regards to the demographics of the crowd, Ranard et al.’s review found that few articles reported on demographics and for those that did, the level of demographics reported varied . HoweverGeiger et al. aim to classify different types of crowdsourcing processes, and in doing so describe the ‘openness’ of their calls. The authors found that most crowdsourcing processes have a completely open call but some restrict contributions from participants by using either qualification–based  or context–based  limitations .Estelles–Arolas et al.’s review came across a dichotomy regarding the purpose of the crowd; one group of authors believed that the purpose of the crowd was to complete tasks and the other, to solve problems . Some auwhat crowdsourcing should aim to do. These are found in In this review, various authors attempted to make classifications of Many of the authors in Estelles–Arolas et al.’s review identified reimbursement as monetary reimbursement. The range of monetary reimbursement is large, varying from US$ 0.01 for each human intelligence task (HIT) performed on the AMT platform to millions of dollars for the successful solution chosen from InnoCentive’s competitions . Geiger Some authors reviewed mentioned inequities regarding crowd contributions. Parvanta et al. describe a 90%/9%/1% rule for participation, in which 90% of the crowd observes, 9% participates from time–to–time and 1% participates regularly . This brGenerally, an institution or organisation initiates the crowdsourcing process with an open call . HoweverMany authors reviewed by Estelles–Arolas et al. felt that the initiator receives the result sought for the task advertised, which was usually the result for a given problem. Others believed the product was either knowledge, ideas, or some type of added value . The exaEstelles–Arolas et al.’s review found many authors who identified crowdsourcing as an outsourcing process, specifically referring to AMT while others referred to it as a problem–solving process or a production model . As descThe majority of authors reviewed by Estelles–Arolas et al. refer to an open call as the form of call that must be made in order to satisfy a crowdsourcing criterion. However, Estelles–Arolas et al. disagree and use the term ‘flexible open call’ meaning that participation is non–discriminatory but the call is tailored to the specific initiative and thus, can be limited to a community where there is specific knowledge or expertise (but anyone in this community can answer) .Estelles–Arolas et al. state that of the authors they reviewed, the medium the call was made through was unanimously agreed upon to be the Internet and Estelles–Arolas et al. agreed . HoweverGeiger et al. discuss the degree of which the crowd is able to access each other’s contributions to the product as a feature of the crowdsourcing process and have four categories: none, view, assess or modify . In someselective crowdsourcing companies, which choose the best solution or design to a particular problem. Crowdsourcing projects run on AMT often aggregate or combine solutions from the crowd as a whole.Aggregation refers to how the responses of the crowd are used by the crowdsourcer. The two major ways the responses can be used are to be combined or to be selected . InnoCenThe definition from Estelles–Arolas et al. excludes Wikipedia, YouTube and Flickr. Wikipedia is excluded on the grounds that there is no initiator (crowdsourcing organisation), that the authors do not feel that the initiator receives benefit, and that there is no open call. YouTube is excluded on the grounds that there is no clear goal, that the crowdsourcer’s benefit is not clear, which is arguable as YouTube ‘stars’ receive compensation for views, that there is no clear initiator, the initiator’s benefit is not clearly defined, the crowdsourcing process is not participative and there is no open call. Finally, Flickr, which is a photo sharing website, also fails due to lack of a clear goal, lack of clear benefit to the crowd, lack of clear benefit to the initiator, not being participative and not using an open call .the classic examples of crowdsourcing [Despite Estelles–Arolas et al.’s integrative definition, some authors strongly believe websites such as Wikipedia are not only examples of crowdsourcing, but are sourcing . Indeed,sourcing ,10,20. Osourcing .Many authors reviewed discussed situations that were amenable to crowdsourcing (see Box 2Tasks that require humans (i.e. where technology either cannot complete the task or where people can do it better) ,6Crowds are better than individuals or experts Firm expertise is low Likelihood of crowd expertise is high Firm expertise is distant from solution not tacit, immobile, unique or complex [Problem is  complex Relevant experience is diverse Problem is modular Expertise is tacit, immobile, unique or diverse IP for problem is protect, problem is not legally protected ,45No problems with ownership or usage of solution Problem does not contain sensitive information Problem divisible into lower–level tasks ,48Low cognitive load Problem is fast to complete Task requires little expertise Solution is objective and verifiable Low barriers to entry ,49Low interaction required As opposed to focusing on characteristics of projects amenable to crowdsourcing, Buecheler and colleagues describe the characteristics of a principal investigator who would be amenable to taking on a crowdsourcing project. They state that the career age, job satisfaction, cosmopolitan scale, tenure, funding, apparatus and time must be considered; however, the authors do not give an estimate of which features within these characteristics are ideal for crowdsourcing .Other authors gave specific tasks that they felt crowdsourcing was most suitable for, such as solving problems, completing tasks, being creative, developing products or ideas . CastillFinally, some authors reviewed gave tips for using crowdsourcing in research. Most importantly, selecting a clear and appropriate research question was emphasised ,45,49. HBenefits identified in the literature review are divided into process–based benefits and results–based benefits, and are displayed in In spite of its benefits, crowdsourcing is still subject to numerous challenges, regulatory, and ethical issues that need to be addressed, considered, and anticipated prior to designing a crowdsourcing study or intervention.Quality assurance issues were the most commonly identified by the articles reviewed. In instances where a crowd is asked to answer questions where there is no ‘right’ answer, it becomes difficult to verify if responses are true and not malicious ,48. AddiRegarding sampling, the denominator is rarely known in crowdsourcing tasks and this can pose problems for analysis . SamplinOther authors cited concerns for security, citing potential loss of data due to a rise of cyber–attacks  or mishaDespite Thawrani et al.’s and other’s concerns that crowdsourcing could compromise anonymity, other authors were concerned that the anonymity of crowdsourcing could raise ethical concerns . WilliamBrabham reports that, while currently it is difficult for crowds to organise themselves against unfair labour practices, “crowdslapping” does happen . This isWhile not considered crowdsourcing by the working definition in this article, text/data mining has unique ethical issues, especially regarding consent, anonymity and researchers planning to use this method must consider this, through community engagement or other methods.A second paper  will revThe UK newspaper, the Guardian, utilised crowdsourcing and freedom of information request to have the crowd comb through Members of Parliament’s (MP’s) expense claims to look for fraudulent claims. There were over 500 000 expense claims uploaded and over 170 000 documents were analysed within 80 hours alone . As a reUshahidi is a SMS– and web–based platform that was created after the Kenyan election in 2007 to report on election violence . It is aGalaxyZoo is a crowdsourcing project that uses volunteers from around the globe to classify galaxies visually. As of 2013, it had successfully classified nearly 900 000 galaxies using hundreds of thousands of volunteers .Transcribe Bentham is a project which aims to transcribe works of Jeremy Bentham, a famous utilitarian philosopher, in order for them to be available to all. There were over 12 000 un–transcribed manuscripts and the project is based at University College London (UCL) .Captcha stands for “Completely Automated Public Turing test to tell Computers and Humans Apart.” Louis van Ahn, the father of human computing, extended CAPTCHA, adding an additional word so people would need to translate two words; the first was a known ‘anti–bot’ word but the second was from an archive that needed to be digitalised . In 2009Crowdsourcing is a field that is relatively nascent, yet blossoming. Because of its infancy, researchers have not yet agreed on its definition or what does or does not constitute its practice. Despite this, several key qualities have emerged. In order to be considered crowdsourcing, a task must be distributed by an organisation via a flexible open call for the purpose of obtaining some knowledge, idea or added value, through a medium that’s similar but not an outsourced model. Usually, crowdsourcing employs the Internet, though this is not necessary. A crowd can be formed by both experts and amateurs, and the crowd can be rewarded monetarily or through recognition or skill–development. Sometimes the results are aggregated, but in other exercises, the best solution is chosen. In this way, applications of crowdsourcing are themselves very diverse and it is not surprising that authors have struggled to provide an all–encompassing definition.Despite the difficulties defining it, crowdsourcing is beneficial both in the process and in the results. It is often low–cost, rapid, and has the possibility to transcend fields, borders, can coordinate with machine–learning, raise public awareness and produce novel discoveries. Crowdsourcing could be hugely promising in global health where resources are low and there is a paucity of data if a concerted effort is made to bring it to scale, especially through marrying the global health community with crowdsourcing and computer science researchers.


The school environment can enhance children’s skills, knowledge and behaviours in relation to healthy eating. However, in many countries, unhealthy foods are commonly available in schools, and children can be exposed to aggressive marketing by the food industry. Taking the perspective of policymakers, this study aimed to identify barriers and enablers to effective school food policy development and implementation in the Philippines.In May 2016, semi-structured interviews were conducted with 21 policymakers and stakeholders involved in school food policymaking and implementation in the Philippines. The Health Policy Analysis Triangle was used to identify interview questions and to guide the thematic analysis. These included the political and socio-environmental context, strengths and limitations of existing policy content, roles and behaviours of actors, implementation processes, policy outcomes, and opportunities to improve policy coherence.The Department of Education’s policy ‘Orders’ represented a relatively strong policy framework for the education sector of the Philippines. However, a lack of human and financial resources for implementation, planning, and policy enforcement limited the impact of the policy on the healthiness of school food provision. Ambiguity in policy wording allowed a wide interpretation of the foods eligible to be provided in schools, and led to difficulties in effective monitoring and enforcement. Food companies used existing relationships with schools to promote their brands and compromise the establishment of a stronger food policy agenda. We found a motivated group of actors engaging in policy-oriented learning and advocating for a stronger policy alternative so as to improve the school food environment.The adoption of policy mechanisms being used to promote healthy dietary practices in the school setting will be strengthened by more robust implementation planning processes, and resources to support implementation and enforcement. Policymakers should ensure policy language clearly and unequivocally promotes healthier food and beverage options. Steps should be taken to achieve policy coherence by ensuring the objectives of one agency or institution are not undermining that of any others. Where there is reliance on the private sector for school resources, safeguards should be established to protect against conflicts of interest. Unhealthy diets are one of the main risk factors for disability and deaths globally , 2. For (2016) recommends establishing standards for meal provision to meet nutrition guidelines, eliminating unhealthy foods from the school environment and establishing mechanisms to safeguard public health from conflicts of interest ? There could be the sanitary inspectors or barangay health workers … or community health workers, barangay nutrition scholars, but then that would add to their roles…They have so many functions to attend to already.Interview 30, Nutrition Official.But if you’d look at the data, the number of under nourished children is still higher compared to the overweight ones. So, in terms of a priority I think this is not much of a priority [for Barangays].Interview 24, Nutrition Official.The Philippines Government has decentralised most governance responsibilities to approximately 1490 local government units and municipalities across 81 provinces. At the base of governance arrangements are small village-like administrative zones called ‘Barangays’. To improve community reach in implementing the Plan of Action on Nutrition, a voluntary network of over 19,000 community based nutrition workers called ‘Barangay Nutrition Scholars’ are trained and supervised to undertake nutrition assessment and counselling in communities across the country . Some inAccording to informants at the Department of Education, ‘most’ schools have a canteen or store, and ‘most’ offer a school feeding program to the most vulnerable (‘at-risk’) children. Many children also bring food to school, reportedly dominated by highly processed products (including hot dogs and fruit juice). Public school canteens and stores are most often run by canteen workers, parents or other volunteers, and in some cases by teacher’s cooperatives. In the latter scenario, the income of teachers is directly impacted by canteen profitability.All interviewed policy officials and principals were committed to deliver school feeding programs for undernourished children. According to the policy, 35% of all profits generated from the canteen are directed towards the school feeding program for undernourished children . For somSmall sari-sari stores, that is the big problem, mostly, in the public schools because they know that children really love those junk foods….Interviewee 29, Policy maker.The high density of stalls selling unhealthy foods around the school perimeter was noted as a major challenge to ensuring children consume healthy food during school hours. School workers and policy makers both noted the influence of ‘sari-sari’ stores  on children’s exposure and consumption of unhealthy foods. One stallholder outside a school described that school children most commonly purchased biscuits, sugar-sweetened powder mixes for drinks, juice, cola, fudgy bars and biscuits.Many interviewees felt it would be difficult to put in place restrictions on ‘sari-sari’ store sellers due to the potentially negative impact on sellers’ livelihoods, and that it would be difficult to ‘police’ food sellers.In terms of the affordability, well, it’s still a big factor because if you buy the chips, you can spend only two pesos [USD 0.04]. Unlike if you buy fresh fruit, it could be five times or as much, 10 pesos [USD 0.2].Interview 30, Education Official.Affordability was also cited as having a significant baring on children’s purchasing decisions in and around the school environment, with interview participants reporting that highly-processed foods (e.g. extruded snacks) were “cheap” compared with healthy locally prepared alternatives .And we understand that there are some conflicts of interest within the school system…school equipment and some of the materials needed in school are lacking. So, the school actually resort[s] to some fundraising and partnership.Interview 22, Nutrition Official.The Philippines Department of Education instituted the ‘Adopt-a-School’ Program in order address resource shortages to the education sector , 49. TheThere is an engagement with partners through the Adopt-A-School Program…you are not allowed to advertise but you can promote the program. So, if there is a partnership, you can see the logo…but they promote the program, not their product.Interview 21, Education Official.Companies that partner with the government as part of the Adopt-a-School Program are reportedly not allowed to advertise their products in schools, but guidelines outlining criteria for participation in the program do not specifically reference restrictions on brand visibility or marketing in schools .There iI think that’s [regulating marketing in schools] really…difficult because we have this Adopt-a-School Program. We encourage companies to partner with us in adopting a school, maybe they provide a school building in exchange of tax incentive. So, for example, the Coca-Cola Foundation – they are donating classrooms in exchange of tax incentives.Interview 20, Education Official.Policymakers were concerned that the Adopt-a-School Program might provide a barrier to imposing restrictions on marketing with schools.root crops, noodles, rice and corn products in native preparation”, fruit and vegetables, and “fortified food products labelled rich in protein energy, vitamins and minerals”. It prohibits the sale of carbonated drinks and “sugar-based synthetically or artificially flavoured juices” and “junk foods” . However, it does not specify the frequency that specific foods should be provided and it does not have nutrient-based or food-based criteria to classify foods.At the time of these interviews, Department of Education Order Number 8 (2007) was recognised by all relevant interviewees as the prevailing policy mechanism governing food provision and sales in the school setting. The Order stated that foods provided must be “nutrient rich”, then lists “The foods explicitly promoted in Department of Education Order Number 8 (2007) are consistent with the Government’s Philippines Food-based Dietary Guidelines (2012), but they do not promote the full range of food groups recommended by the Guidelines, for instance fish, lean meat, poultry, eggs, dried beans, nuts, milks, and fish. The Order does not specifically restrict intake from salty, fried, fatty and sugar-rich foods, as recommended by the Philippines Food-based Dietary Guidelines (2012).We do not sell junk-food, only those foods with Sangkap Pinoy which are acceptable for kids.Interview 14, Principal.Currently, the Department Order includes also fortified foods, but these fortified foods are mostly chips. They may be high in vitamin A, but the rest is just calories-.Interviewee 30, Health Official.All products bearing the words ‘Sangkap Pinoy’ are allowed under the Order number 8 (2007). This is a government ‘seal’ signifying the fortification of a product with nutrients as per Republic Act 8974 . InterviAt the time of these interviews, there were no national marketing controls in the Philippines. Within the school setting, another Department of Education policy, Order 37 (2010), is a generic marketing control that prohibits “the use and or display of school signage showing commercial advertisements, sponsorships and endorsements”.The ‘Philippines Pledge’ lists global food subsidiaries including companies Coca-Cola Far East Ltd, Mars Philippines, Kraft Foods Philippines Inc, Nestle Philippines Inc, Unilever Philippines Inc and Pepsico International (2010) as signatories, and pledges not to provide any commercial material to schools unless “agreed with school administration for educational or information purpose”. The pledge commits to restricting advertising of products that do not fulfil “specific nutrition criteria…based on accepted scientific evidence and/or applicable national and international guidelines relevant to children” . There iWell, actually there are a lot of studies but it’s international. And our legislator would say, “Oh, but that’s international, we don’t have local evidence.”Interview 22, Nutrition Official.Three key issues emerged as barriers to adopting comprehensive healthy school food provision and marketing policies. In the first, interviewees reported that policymakers who had tried to bring attention to the contribution of food provision and marketing to unhealthy eating habits were faced with requests from politicians for stronger evidence of the linkage between sugar consumption and poor health. For example, a House Bill to tax sugar sweetened beverages, introduced for the first time in 1999 had been opposed because policymakers could not provide evidence that sugary beverages posed a “social cost” . AdditioInterviewees indicated that the types of evidence called for included ‘scientific data’ proving unhealthy foods should not be provided to children; data on local consumption patterns among children within and outside school; clarity around which foods exceed levels of sugar deemed detrimental to Filipino children; clinical evidence of the impact of childhood obesity in the Philippines, linkages between sugar intake, obesity and health outcomes; and cost-benefit analyses of marketing restrictions. They also called for nutrient profiling systems to be developed using local data.Yeah, it’s very difficult to say that the product is unhealthy. It may kill the business industry. It’s like saying you should not buy their product, to stop doing business.Interview 17, Food Regulations Officer.The second barrier identified by interviewees was the widespread concern amongst policy makers that restrictions would interrupt the profitability and growth of food companies. The country draws significant income from primary products (like rice and sugar), and food manufacturers contribute to income, employment, economic growth.We expect more opposition coming from the food manufacturers…We don’t know how they can influence our legislators. Just the way when an amendment to the milk code was being proposed, there was such a strong lobby among the legislators.Interview 30, Health Official.Food companies are big companies, you know, they have lots of money...we will not expect them just sitting down, they will do something to counter our efforts.Interview 24, Nutrition Official.Thirdly, interviewees in the health and education arms of government indicated that they anticipated that the food industry would vigorously oppose any efforts to reduce marketing or restrict the sale of unhealthy foods in and outside of schools, and that it would have a strong influence on political commitment to this agenda.The Pepsi and Coca Cola they work together, can you imagine that? They came to the [Dep Ed] office, Pepsi, Coca Cola…They are the Beverage Association of the Philippines.Interview 23, Education Official.There’s a very strong lobby of the soft drinks industry. Already I learned that the association of food, beverages are already asking the Dep Ed to lift that ban [on sugar-sweetened drinks in schools]. The same way that there was a proposed … There was a bill on increasing the tax for soft drinks. There was a lot of opposition in that. We expect more opposition coming from the food manufacturers.Interview 30, Health Official.Informants cited examples of food companies deliberately obstructing the public health agenda by coercing policymakers and engaging in political advocacy against health legislation. Policy makers at both national and local government levels reported being directly approached by food industry executives and pressured to lift policy initiatives aimed at improving food in schools.Schools are always asking for information on what is healthy and not healthy? Can you provide us a list of foods that are healthy and unhealthy? We asked our Division Health Department for a list of such foods but they did not have one.Interview 18, Division Education Officer.Well this one [Dep Ed Order Number 8] is – is not you know, it’s not – cause it’s not very clear…this is subject to adaption by the schools. So, I don’t think this a strong enough tool to be used.Interview 24, Nutrition Official.At the national level, there was no evidence of implementation planning for the execution of (Department of Education Order Number 8 (2007), no resources specifically dedicated to implementation, and limited capacity building opportunities to imbed and sustain policy adherence. Most implementing actors  reported they had been trying to interpret the policy without adequate tools, training or support to guide them. There was a call for clearer guidance and capacity building opportunities to better prepare school workers to provide healthier foods and beverages.If you’re in the school there’s so many policies that goes down but there’s only one person that’s supposed to do it, it’s the principal. So, if prioritisation is  from the principal. And I feel that the school has a lot of needs yet for curriculum, for infrastructures, health will be third of the priority or at least last. So in monitoring of how the school canteen will be managed will be their least priority. They will always prioritise curriculum, content, teacher capacity first.Interview 23, Education Official.The motivation and capacity of the principals to promote Dep Ed Order Number 8 was viewed as crucial for sustaining implementation. It was noted by several informants that principals were overwhelmed by the sheer number of ‘priorities’ handed down to them, compromising their ability to govern school food policies.Most of the pupils go to the fence of the school and the ambulant [mobile] vendors just at the other side of the fence and they exchange goods…So, what I did is I put a barrier, one metre apart [from the fence]…and then I made it a vertical [vegetable] garden.Interview 14, School Principal.Amongst school principals interviewed, there were diverse views on the importance of nutrition relative to other considerations. For example, one principal had gone to great lengths to promote a healthy food environment in the school, with initiatives including a healthy school canteen, nutrition education, and a large ‘Gulayan sa Paaralan’ (vegetable program) to feed undernourished children. He had also built a garden wall around fences to prevent vendors selling ‘junk’ food through the fence to the children.Number one [challenge] is who’s going to do the monitoring? Because it’s another agency, so it’s Dep Ed and here is the health department monitoring the Dep Ed.Interview 12, Local Government Official.What will be the tool that should be used during the monitoring? For us, we just check. And then we look at, you know, even the hygiene of those who are preparing the foods in the canteens.Interview 12, Local Government Official.Department of Education Order Number 8 (2007) delineates responsibility for monitoring and enforcement to Divisional and Regional Offices of the education department. However, other government agencies were also being engaged in school food monitoring. Interviewees identified that a lack of adequate monitoring and enforcement was a key factor impairing the effectiveness of the policy. This related to a lack of human and financial resources at the national, subnational and local government levels , the lacThere are no sanctions actually aligned with the policies.Interview 21, Education Official.I know that they have issued this order, but they have not really monitored the implementation of this. There’s no conscious effort, really, to monitor the implementation. But if it is a law…they have an oversight function. They can actually investigate if it is really being implemented and they can require reports from agencies on how they have implemented these things.Interview 22, Nutrition Official.The 2007 policy does not list the nutritional quality of food among its other reporting requirements. While the Order provides a grievance mechanism and introduces sanctions for violations, in the case of “simple violations”, a warning is provided to principals. When questioned about enforcement of school food policies, no stakeholder raised the possibility of sanctions, and there was an inference amongst interviewees that the Orders would only be enforceable (and thus adhered to) should they become legislated.In general, Dep Ed orders are not very definite …implementation’s also a problem even though there’s a ban [on sale of sugar-sweetened beverages]. We have banned carbonated drinks but if we go to the school -why are you still (selling) Coca Cola?Interview 23, Education Official.Dep Ed has a policy on sugary drink but they’re still selling sugary drink, I don’t know what happened...We have already done monitoring…We asked the school canteens the available foods like for example any processed food or biscuits. And we found out that’s all, you know, sugary, it’s all sweet.Interview 12, Local Government Official.We received some feedback that there are some violations on the Dep Ed order. Some of the parents…they still see some unhealthy foods in the canteens.Interview 20, Education Official.There were no routine measures on which to gauge the effectiveness of the policies in place, however most interviewees alluded to the fact that existing policy mechanisms were not sufficiently protecting children from being exposed to the marketing and sale of unhealthy foods and drinks in and around schools.Biscuits are good, but you know it’s the spread [that can be problematic], but if it’s a plain biscuit, probably it’s good. It’s a healthy food.Interview 12, Local Government Official.I allowed Nestlé and Milo and milk because in my own opinion, using my own criteria, they are healthy for me. I don’t know if they’ll pass in this [nutrition] criteria.Interview 14, School Principal.Interviewees described commonly provided foods in schools as including bread, noodles, juice drinks, cupcakes, biscuits, fruits, packaged snacks and sweetened rice. One Division-level education official spoke of promoting white bread, homemade cakes and biscuits as healthy foods because they were allowable under the Orders. Fortified versions of sodas and sweetened juices, sweet and savoury biscuits, chocolate malt, chips and extruded snacks baring the Sangkap seal were also being provided. There was also variability in the nutrition literacy amongst the principals interviewed.According to the Department of Education, some schools have pursued unhealthy food bans under their own initiatives, and the degree of support for healthy food promotion is based on the perspective of school administrators with regard to the importance of healthy consumption to children and their development.There are Coca Cola red schools, there are McDonalds little red high school libraries!Interview 23, Education Official.Usually Nestlé company conducts parents’ education on nutrition…[Nestle?]…Nestle, that’s right... they give fliers to parents. Here in school, but sometimes I can see that it’s very limited.Interview 14, School Principal.We partnered also with one school and they’re promoting healthy lifestyle so their gym, actually the flooring has Pepsi [logo] on it…and of course the refrigerators and everything, it’s either Coca Cola or Pepsi!Interview 22, Education Official.Interviewees provided numerous examples of food company sponsorship and promotion in and around the school setting. These included ‘nutrition lectures’ provided to parents by Nestle, Jollibee scholarships, Milo sports days, Pepsi gymnasium floors, Coca-Cola red painted school buildings, and branded soft drink bottles used in school gardening projects. Two principals referred to the Department of Education Orders restricting commercial activity, but provided examples of commercial activity when prompted for examples of support and donations from private companies. One principal repeatedly referred to the Orders but proudly promoted his extensive cola-bottle vegetable garden.A scan of websites belonging to the Philippines arm of selected food companies revealed that food companies support schools in the Philippines by providing feeding programs, scholarships, resources including textbooks and computers, through infrastructure and maintenance, and by offering scholarships, training and learner support.The scan also found numerous images depicting prominent company branding in school settings Fig. . For insThis study found that, despite a relatively strong policy mechanism for healthy school food provision and marketing , the lack of human and financial resources for implementation, monitoring and policy enforcement restricted its impact. The study also found that clearer policy wording would likely improve both interpretation of the policy while better enabling accountability. Additionally, external actors, particularly from large food companies, were found to be compromising policy processes and agenda-setting. However, we did find a motivated group of policy-entrepreneurs advocating food policies to protect children.This research identified several opportunities for strengthening implementation of healthy school food provision and marketing policies in the Philippines, outlined below. By considering the findings of this study through the lens of theories of policy learning , our disThe healthy school food provision policies analysed here enabled a broad interpretation of their contents, demonstrating the need for countries to address ambiguity in policy language and ensure they unequivocally promote healthier food and beverage options. The success of school food policies may, at least in part, be predicated on the way in which ‘healthy’ foods are identified, categorised and subsequently managed, because differences can result in significant inconsistencies in interpretation, ‘strictness’ –60, and In relation to the industry-led marketing controls, we found that the Philippines Pledge was a weak set of standards which lacked in transparency and accountability. This is consistent with voluntary food industry initiatives adopted elsewhere , 66–68, The absence of goals, targets and implementation plans is a common limitation of nutrition policies globally , and pooThe food environment external to the school provided easy access to non-policy compliant foods. Local level factors have similarly been found to compromise school nutrition policy implementation in other settings, including the proximity of stores selling unhealthy alternatives and the prohibitive costs associated with policy-compliant foods . OngoingWeak accountability mechanisms further limited the success of policies examined here. Routine monitoring and accountability mechanisms are reportedly critical to the successful adoption and maintenance of school food policies , 77, 81.The Philippines Government is highly reliant on private partnerships to address resource challenges in the education sector . The priResources provided through industry partnership approaches such as the Adopt-a-School program create a potential conflict for policymakers and school workers seeking to implement healthy school food provision and marketing policies. For example, if the Department of Education was to enforce marketing restrictions in schools, it may be at the expense of infrastructure and other resources being provided to them by food companies. According to WHO, food company sponsorship and participation in schools presents a high risk for conflict of interest , 67. In Low resource settings are likely more vulnerable to this conflict because there is an increased reliance on private finance, and the benefit from engagement is comparably high . StrategAfter this research was undertaken in 2016, the Department of Education in the Philippines initiated a rapid process to develop and adopt a new Dep Ed Order to surpass Order Number 8. This process was led by officials who were interviewed as part of this study and actively participated in regional WHO-led meetings advocating similar policies, indicating that policy officials were engaging in policy-oriented learning . The newAn opportunity for stronger policy that still remains, is the adoption of regulations to comprehensively protect children from marketing of unhealthy food and beverages, which is a priority item on the legislative agenda of the country’s multi sectorial nutrition agency, the National Nutrition Council. This study provides evidence that the influence of the food industry presents a risk to the progress of this agenda in the Philippines, and contributes to the lack of policy action in this area. The co-opting of political leaders by food and tobacco companies, as well as other corporate political activity, has the potential to undermine health policy making . Policy The key strength of this research was that it directly involved policymakers from multiple levels of governance in the Philippines, and drew on the in-depth insights and perspectives they offered. Due to the extensive and well-organised policy material of the Department of Education, we were able to triangulate key information about sector partnerships, operations and policy processes against government policy materials. The policy analysis frameworks used have delivered an enriched understanding of the key influences on policy effectiveness, based on established policy science theory.Nevertheless, the study had several limitations. Only a small proportion of policy makers involved in this policy landscape were included in the study. Moreover, while National, Division and Municipal level informants were interviewed, we were not able to interview any Provincial-level policymakers. Thus, the insights expressed by participants may not reflect the full range of views from relevant decision-makers. However, where possible, information provided by informants was cross-checked against the extensive and well-organised policy archives maintained by the Department of Education.Without standardised and collated monitoring systems there was no data available on school food provision. Therefore, assertions on outcomes were based on qualitative contributions from the small range of interview participants. This was supplemented by the review of food company websites, but this was limited in that it could not date the activity depicted in the images found.This study found that in the Philippines, the adoption of tight regulations to protect children from the provision and marketing of unhealthy foods and beverages in the school setting was limited by the lack of resources and capacities for implementation planning, support and enforcement, a vague delineation of actor responsibilities, and potential conflicts of interest with food companies. This was compounded by unclear policy parameters, and ambiguous categorisation of foods to be restricted or promoted. These findings indicate that countries should facilitate implementation planning processes during policy development to establish goals, targets and activities, and measures for accountability and resource allocation. They should also carefully consider how food guidelines and policies will be interpreted, promoted and monitored. Subsequent efforts by policy makers in the Philippines after the study was conducted, have addressed this somewhat through clearer delineation of duties and responsibilities across government, although the availability of resources may yet constrain effective policy implementation.Countries and organisations disseminating their lessons on policy making through global networks promote transnational policy learning . Specifi


FP2020's overarching goal is framed around the new metric of “additional users.” This measure inherently captures population-level change but has been conflated with other ambiguous metrics, such as “new users.” Therefore, we propose a standard set of terms to provide more consistent measurement. Although commonly used service-level metrics cannot be directly compared to the population-level metric of additional users, we describe 2 modeling approaches that can allow service-level data to inform estimates of additional users. FP2020's overarching goal is framed around the new metric of “additional users.” This measure inherently captures population-level change but has been conflated with other ambiguous metrics, such as “new users.” Therefore, we propose a standard set of terms to provide more consistent measurement. Although commonly used service-level metrics cannot be directly compared to the population-level metric of additional users, we describe 2 modeling approaches that can allow service-level data to inform estimates of additional users. At that time, it was estimated that 222 million women in the developing world had an unmet need for modern contraception.Nearly 5 years later, the widely recognized “120 by 20” goal supported by the Family Planning 2020 (FP2020) global partnership can be credited for galvanizing renewed commitment to family planning. However, the new metric of “additional users”—an aggregate metric that estimates how many more modern contraceptive users there are now compared with the estimated 2012 baseline number—has created confusion about the definition and meaning of several other related family planning metrics, including “new users,” “acceptors,” “first-time users,” and “adopters.” It has also raised the question of how service-level metrics collected by programs can be linked to the aggregate concept of “additional users” to assess progress of individual programs toward population changes in contraceptive use at the country level. In this article, which follows from a panel discussion among the 4 coauthors held during the 2016 International Conference on Family Planning, we outline several of the metrics currently used to measure family planning program progress and propose a preferred set of service-level metrics to inform contributions to the FP2020 aggregate-level goal of reaching “additional users.” We also describe 2 approaches—Track20's Family Planning Estimation Tool (FPET) and Marie Stopes International's Impact 2 model—for bridging the gap between service-level measures available in programs' routine service statistics and the aggregate metric of additional users. Finally, we draw attention to the need for more robust data collection systems that allow for the collection of harmonized routine longitudinal metrics rather than focusing solely on visit-based service statistics or cross-sectional household surveys.Metrics, especially when used by donors and governments to set goals and measure performance, can drive how family planning programs are designed and implemented. Using the right metrics ensures that program growth translates into additional impact, gives credit for ensuring current family planning clients have continued access to services, and links programmatic increases in contraceptive use with expanded national coverage, not just attracting clients from other providers.Not all current data management systems enable effective monitoring of family planning program performance. A key dimension to the success of any family planning program that is not fully captured with service statistics, for example, is the effective, voluntary use of a preferred contraceptive method by each program beneficiary over time. Reasons for method-specific discontinuation rates are critical to understanding this longitudinal perspective but this information is often not captured with service statistics. Researchers have instead created models that use available service statistics to estimate program performance.Several types of indicators are collected to measure family planning program performance, and in particular contraceptive use. Some of these indicators, such as “commodities distributed” and “first-time users,” are collected through routine client-level service statistics or client exit interviews, whereas others, such as “users” and “additional users,” are estimated in models or collected from population-level surveys. To ensure clarity of terminology used throughout this article, we summarize these definitions in the BOXCommonly Collected Routine Service Statistics DataClient visits: The number of times clients interacted with a provider for contraceptive services. In most cases, the same client is counted multiple times because the client comes for multiple visits . Most health management information systems (HMISs) count client visits.Clients served: The number of clients who received contraceptive services in a given time period, often 1 year. This is often counted using a client-based HMIS and thus is not very common as few systems have the means to track a uniquely identified client across multiple visits .Commodities distributed/services provided: The number of contraceptive commodities distributed or services provided to clients . In some cases, this may be captured at the client level , while in other cases they might be counted further back in the supply chain . These counts are often aggregated into the couple-years of protection (CYPs) measure.Family Planning Client Characteristics Data Captured Routinely or via Client SurveysFirst-time user: A person who starts using modern contraception for the first time in her life.Lapsed user: A person who has used modern contraception at any time in the past, but is not currently using a modern method.Adopter: A client who was not using a modern contraceptive method at the time of her visit, which includes first-time users and lapsed users. The definition of “time of her visit” can vary, for example, today, last month, or last 3 months.Provider-continuer: A client who, at the time of her visit, was already using a modern contraceptive method that she received from the same service provider (or same network) and comes back for another family planning service . The definition of “time of her visit” can vary, for example, today, last month, or last 3 months.Provider-changer: A client who, at the time of her visit, was already using modern contraception and comes for another family planning service, but who had previously received her family planning from a different provider. The definition of “time of her visit” can vary, for example, today, last month, or last 3 months.Note: The 3 terms adopter, provider-continuer, and provide-changer are mutually exclusive groups: all clients served fall into only 1 of these 3 categories. Collectively, these 3 terms are often referred to as the “client-use profile.”Population-Level Data (not directly captured in routine data)User: A person who is currently using contraception, regardless of when the method was received. This is not directly comparable with the number of clients served in a year, because it includes women still using long-acting or permanent methods received previously . This can be estimated through population-based surveys, such as Demographic and Health Surveys, Multiple Indicator Cluster Surveys, or Performance Monitoring and Accountability 2020 surveys, or through modeled estimates of the contraceptive prevalence rate and the number of women of reproductive age  or modeling from service provision data . Note that “currently using” can be interpreted differently by women asked about current use in survey questionnaires.Additional users: The net number of current contraception users above a specified baseline; in the case of FP2020, the baseline is the number of current contraception users in 2012 in the world's 69 poorest countries. Note that this concept does not apply to an individual but rather to an aggregate population.Terms We Suggest DroppingNew user: A term that has multiple definitions including first-time user, new to the provider , new to the method , not recently using a method , and even additional user.Acceptor: A term that has multiple definitions including first-time user, new to the provider , new to the method , not recently using a method , using a method after an abortion or birth, and even additional user.Because of the ambiguity with the terms “new user” and “acceptor” and because the concepts are adequately captured in other clearer terms, we suggest the family planning community drops these 2 terms from our list of terminology.aggregate metric indicating how many more modern contraceptive users there are across the 69 poorest countries now compared with the estimated 2012 baseline number of modern contraceptive users in the same countries. Each year, the FP2020 Secretariat publishes its annual progress report showing progress toward the “120 by 20” goal—FP2020's first core indicator.,“Additional users” is inherently an The estimate of current mCPR (FP2020's second Core Indicator) is a model-based estimate, informed by nationally representative household surveys, such as the Demographic and Health Surveys (DHS), Multiple Indicator Cluster Surveys (MICS), and Performance Monitoring and Accountability 2020 (PMA2020) surveys, as well as service statistics (in select countries) and historic regional and global patterns of change.For FP2020, the goal of reaching 120 million additional users can only be achieved if family planning programs (1) continue to sustain services to more than 270 million women,,To date, national and global-level understanding of progress in family planning has typically been informed by household surveys, including DHS, MICS, and PMA2020, as well as other national and cross-national survey programs . Such surveys are invaluable in providing a cross-sectional insight into contraceptive use, typically from questions such as “Are you currently doing something or using any method to delay or avoid getting pregnant?” However, surveys have limited ability to capture the dynamic longitudinal nature of contraceptive use, including first-time use, discontinuation, switching of methods, resumption of use, and so on. An exception is the calendar section of the DHS questionnaire, which captures a woman's retrospective self-reported contraceptive status (and method), pregnancies, births, breastfeeding status, and method terminations every calendar month for the 5 years prior to interview. Although these data do not suffer from problems of loss to follow-up, the calendar data are vulnerable to selection bias as only women surviving to interview can report, and there are likely to also be memory recall issues. Besides DHS's calendar method, there are some recent examples of electronic client information systemsPerformance monitoring of program outputs and trends in family planning services occurs below the aggregate national and global levels, and presents different measurement challenges. Examples include government monitoring of provision of family planning services through the public sector or a private service delivery organization monitoring provision throughout its delivery network. In either case, program monitoring generally relies on routine data from an HMIS. These data originate from registers kept at family planning service delivery points, in which key programmatic elements are recorded on a daily basis, such as date of visit, age and gender of the client, type of family planning service provided, and so on. Yes, information available from HMIS and other routine systems are notoriously compromised by data quality issues, but they are valuable for indicating basic details of service delivery.At the same time, programs are increasingly interested in understanding how program-level outputs contribute to population-level changes in contraceptive use in the country. More specifically, they want to know whether their services are contributing to an increase in the number of additional users nationally. To answer this question, one must connect outputs measured at an individual level  to an aggregate change in contraceptive use nationally . As an attempt to bridge this gap, some programs capture information on recipients of services. The most commonly collected client attributes are “adopters” and “first-time users.” These metrics have their merits and can inform aspects of increasing access to contraception. However, they are not comparable with each other and are not the same as population-level “additional users.”To build a bridge between service and population-level measures, we first need to select the most appropriate service-level metrics that can help inform a program's contribution to population-level contraceptive use. We then need a way to account for both uptake and discontinuation of contraception as women move in and out of contraceptive use throughout their lifespan.At the individual level, a “first-time user” is a woman who initiates contraception having had no previous experience with contraceptive use. A client can be a first-time user only once during her lifetime. In contrast, an “adopter” is someone who starts using family planning who was not currently using modern contraception at the time of her visit, but may have used modern contraception in the past. Thus, a woman could be an adopter several times during her life if she stops and starts using contraception. As adopters include both first-time users and lapsed users, in the context of program-level monitoring, the number of first-time users will always be lower than the number of adopters.When assessing a program's contribution to increasing contraceptive use nationally, adopters is a more useful metric and preferred over first-time users, since it is a more inclusive measure of adding women into current contraceptive use. Ideally family planning providers or organizations would capture a full suite of indicators including both adopters and first-time users. In reality, they often have to choose a handful of indicators from a longer list to make routine data collection manageable for service providers; hence, the need to prioritize which indicators to include. It is important to note, however, that capturing the number of adopters is not sufficient alone to estimate how national levels of contraceptive use are changing, as will be discussed below.The terms “new user” and “acceptor” have been frequently used and misused in multiple contexts to refer to a first-time user, an adopter, and even an additional user. In recent years, “new user” has often been used incorrectly to refer to measuring contributions toward FP2020's goal of 120 million additional users. Not only is the “new user” term ambiguous and confusing—“New” to contraception or “new” to the provider? “New” as differentiated from the 2012 baseline of modern method users?—these measures of “new” do not adequately capture important concepts associated with measuring additional users. We suggest, in the interest of clarity, to drop the terms “new user” and “acceptor” from the family planning metrics language and replace them with the clearly defined term of first-time user, adopter, or additional user, as appropriate.,Maintaining the 2012 baseline number of modern contraceptive users in the world's 69 poorest countries does not mean that the same 270 million women will continue to use contraception year after year. Women will move in and out of contraceptive use over time, as their needs and situations change. There are many reasons a woman might discontinue use of contraception, such as ageing out of her reproductive years, mortality, method-related reasons when still in need , method failure, or no longer needing contraception due to not being sexually active, wanting to get pregnant, partner separation/dissolution, or menopause.However, tracking indicators that are captured at the service level  focuses on one side of the equation (gains) without measuring the other side (losses), and so does not allow us to see the full dynamic of continuation, discontinuation, and net change. One has to account for levels of continuation and discontinuation to know how many of the adopters are “replacing” women who moved out of current use and how many are “adding” to total current use. Without seeing the full equation, it is not possible to know the extent to which the number of adopters are offsetting declines due to women dropping out from the baseline . In factWhen available, client-level data can be used to inform national level changes in mCPR and estimate contributions by particular organizations to national-level changes using existing tools and models, namely, FPET and the Impact 2 model.FPET combines survey data and service statistics to inform trends in mCPR growth: Track20's FPET,Impact 2 model uses service statistics and client-use profile data to estimate contributions to additional users: The Impact 2 model, developed by Marie Stopes International, allows organizations to estimate their contribution to national-level additional users, based on program-level and other input data.,supplement.For more information about FPET and the Impact 2 model, see the We call on government ministries, service providers, and donors to align how they define and track their contributions to FP2020, so that across the sector we are all aiming for the same goal and measuring the same concept. Doing so will strengthen the field's ability to respond to important measurement challenges as we transition from the FP2020 goals to those of universal access to reproductive health and the 2030 Sustainable Development Goals that follow.There is much focus on the goal of ensuring that an additional 120 million women use contraception by 2020. What has been less visible from this goal is the 270 million women in the world's poorest 69 countries who were estimated to be already using contraception in 2012.Service-level outputs  play an important role in monitoring family planning programs, and thus their tracking should not be undervalued. As HMISs improve with technological advances in electronic data collection and analysis, enabling more robust collection of longitudinal metrics that track individual clients' first-time use, discontinuation, method switching, resumption of use, and so on, even more value can be obtained from routine tracking. However, due to the limitations described in this article, these program-level measures will never be directly comparable with population-level changes in contraceptive use such as additional users.Finding ways to bridge the gap between program and population measures is important. Using validated models allows implementing organizations, governments, and donors to translate service statistics into meaningful estimates of national-level changes in continued contraceptive use. For example, from experience in the social franchising space, the Metrics Working Group has defined the metric of choice for measuring “additionality” in contraceptive use to be the “contribution to additional users as modeled by the Impact 2 model.”Focus on “additional users” as an indication of sustained growth in contraceptive use at the national levelDrop “new users” and “acceptors” from the family planning measurement agenda as these ambiguous terms create confusionFocus on capturing data on “adopters” as the preferable client characteristic to better understand who is being reached by family planning programs and to inform modeled estimates of additional usersUse output measures such as client visits, services provided, and CYPs not only to inform program monitoring but also to inform modeled estimates of additional users and to better understand the link between the service and population levelsWe have outlined how national-level estimates of additional users can be informed by routine data and how individual organizations can estimate their contribution to additional users. Therefore, we call on donor agencies, governments, implementing organizations, and other partners to:Collectively, these steps will ensure that across the family planning sector we are using a standardized and comparable terminology and approach to define and measure progress. More importantly, they will ensure that growth is measured in a way that takes into account both growth beyond a baseline level of use and provision of services to women who were not previously protected by contraceptives. If the additionality metric is widely adopted, programs will be able to immediately see their contributions to the global goal of enabling women to access modern contraception.


Interviews were conducted with participants at farmworker housing during non-work time. Qualitative data were analyzed using ATLAS.ti. Farmworkers recognized pesticide protective behaviors as helping them to not get sick and stay healthy. Farmworkers perceived work experience as facilitating protective behaviors. Wetness in the field was the most commonly cited barrier to protective behavior use. To overcome this barrier, farmworkers suggested use of water-resistant outerwear, as well as packing a change of clothes for mid-day, with space and time to change provided by employers. Examination of the efficacy and feasibility of farmworkers’ suggestions for addressing barriers is warranted. Training and behavior modeling by experienced peers may improve behavior adoption and perceived control. Pesticide exposure is associated with deleterious health effects. Prior studies suggest Latino farmworkers perceive little control over their occupational health. Using the Health Belief Model as a theoretical guide, we explored the perceptions of Latino farmworkers working in tobacco in North Carolina ( Migrant and seasonal farmworkers perceive little control over their occupational health and safety, including pesticide exposure ; howeverSeveral studies of self-reported protective behaviors among farmworkers have found limited adherence to these behaviors ,10. EpidPersonal factors, such as time to do PPBs and comfort, have been reported as barriers ,11. ProtOne study reported that farmworkers’ motivation to engage in protective behaviors directly corresponded to the availability of the supplies needed to conduct the behaviors; the same percentage of farmworkers who reported hand washing before eating (42%) reported the availability of hand washing water . The lacFarmworkers’ perceptions of the benefits of protection against pesticides and other farm chemicals have also been explored, although to a lesser extent. Farmworkers reported believing that protecting oneself from farm chemicals leads to a healthier life, that it is important to protect children from farm chemicals, and that protective clothing is effective against chemical exposure . A variety of theoretical frameworks have been used to explain the likelihood that farmworkers will adopt protective behaviors. One of these is the Health Belief Model (HBM) . At its The University of Utah IRB determined this study to be exempt on 17 March 2014 (IRB # 00071986). After obtaining Institutional Review Board exemption, seven farm owners/operators from one of the three largest tobacco production counties in North Carolina were approached about participation in this study. These individuals had previously voluntarily participated in a safety intervention called Certified Safe Farm (CSF) to reduce agricultural injuries, demonstrating their orientation toward safety. Three of the seven farm owners/operators agreed to provide access to their farms so that the researchers could approach the farmworkers working for them. Two of the participating farms employed fewer than 10 workers. At the time of data collection, workers on one of those farms were engaged in weeding shortly after the transplantation of tobacco into the field, and on the second farm, workers were engaged in topping and suckering (the routine crop maintenance procedure in which the tobacco flower is removed) prior to harvest. On a third farm, which employed the greatest number of workers, some topping and suckering was done early during data collection, but the majority of the farmworkers were engaged in harvesting of the tobacco and barning activities. Farm owners/operators did not participate in recruitment of their farmworker employees, and farmworkers were informed that participation would not impact their employment. Seventy-two farmworkers were approached, agreed to participate, and provided informed consent. Inclusion criteria for farmworkers were being age 18 years or older, having the ability to speak Spanish or English, and self-identifying as Hispanic/Latino ethnicity. Each participant was given a study identification number, which was different from the participant numbers assigned in this paper, and the requirement of written consent was waived because it was the only source linking the farmworkers’ names to data. All data were collected between May and October 2014. A quasi-qualitative approach was selected to examine farmworkers’ perceived PPB benefits, barriers, facilitators, and strategies to mitigate barriers. Data were collected through structured interviews which were audio recorded. The structured interviews were conducted using an interview guide and a demographic questionnaire. The structured interviews took an average of 25 min per participant with a range from 10 to 65 min. All but three interviews were conducted by a native Spanish speaker. The first author observed all interviews and conducted three (in Spanish) when the interviewer was unavailable. The interviews were conducted on non-work time (Sunday afternoons) just outside of the homes of the workers (either employer-supplied or farmworker-rented mobile homes) and, in the case of four workers, at the farm on non-work time per their request to avoid having the research team come to their homes. For participation in the study, farmworkers were given a $25 Walmart gift card and a hat donated by a local farmworker service organization. The investigators developed an interview guide to assess farmworker demographic characteristics and perceived benefits and barriers of PPB as described in the HBM, as well as strategies to minimize those barriers and facilitators of PPB. To ensure that respondents focused on pesticide-related behaviors mandated to be taught by the WPS, those behaviors were described for respondents. Open ended questions (16) included items related to perceived benefits , barriers , facilitators , and strategies to minimize barriers . While the questions were the focus of analysis for each of these domains, sometimes relevant comments emerged in other sections of the interviews and were also included in the analysis. Two independent Spanish translations of the interview guide were made, and the versions were compared. The interview guide was pilot tested with two farmworkers not participating in the study; they were asked which questions, if any, caused confusion, so questions could be refined prior to the study. The interviews were transcribed and translated by the interviewer, a native Spanish speaker from Mexico whose parents were agricultural workers and who grew up close to the farms in the study. The selection of this team member to do translation and transcription was purposeful, as the literature suggests that a translator’s natal language and dialect should be similar to that of the participants to minimize threats to validity . The intDemographic data were collected and managed using REDCap electronic data capture tools ; the REDThe first author content analyzed the interviews for factors that were perceived as benefits, barriers, facilitators, and strategies to minimize barriers to PPBs . A priorWhile the first author coded, she listened to each audiotape again for context and as a form of quality assurance. When there were discrepancies between the audio recording and the transcript, the audio-recorded version was used. Seven of the transcripts (10% of the data) were selected using a random number generator for the first author and a second coder to code independently. The first author/coder had extensive knowledge of the study and its participants, and the second coder had none. For every response for which the two coders did not agree in their initial codes, a negotiated code was determined through discussion and transcript review . The intThe 72 participating farmworkers were predominantly male (96%), were from Mexico (97%), and had an average age of 33 years (range 18–68). The majority (90%) were in the United States on a work contract (H2A visa) and had completed no more than a middle-school education (89%). Additional demographic information, including living accommodations and years in agriculture, can be found in n = 24, 33%) or, framed positively, staying healthy . Several farmworkers also talked about not getting cancer specifically (n = 3): “Protect yourself against illnesses, cancer and any illnesses” (P62). Others described not getting “poisoned” or “intoxicated” from pesticides (n = 7): “It works to not get intoxicated and not get illnesses, for our own health” (P99). Some mentioned avoiding getting skin problems (n = 6): “One is protected from the small bumps that one gets with the pesticides” (P101). Also mentioned was not getting dizzy (n = 4); for example, “They protect you from not getting full of liquid or for it to penetrate and then you get dizzy” (P31). When farmworkers were asked about the benefits of pesticide protective behaviors, the most commonly shared response was not getting sick . One participant stated, “They [PPBs] are really good because they protect you, they protect your eyes, your nose, your respiration. With any contact with pesticides you have to protect yourself” (P25). Another emphasized illness prevention through exposure reduction, “The more you cover yourself, the less risk you run of getting sick because one has more protection” (P76). Closely related is the concept of protection from exposure in addition to illness : “We already know that when they apply the pesticides somewhere, we do not get close” (P17). Relatedly, another benefit included making the pesticides impenetrable (n = 3). Avoidance of pesticides altogether was described as a benefit of PPBs. Avoidance of pesticides usually centered on staying out of restricted areas (Two farmworkers perceived PPBs as a way to prevent green tobacco sickness (GTS), a form of nicotine poisoning that occurs by dermal absorption of nicotine from tobacco leaves when the plants are wet. One farmworker discussed the use of water-resistant outerwear in the context of nicotine exposure: “It [PPB] covers many things. If you enter without a poncho, the water from the tobacco can get you ill. But with the poncho, it covers you and not a lot falls on you” (P34). n = 19, 26%). The video method was discussed in particular: “I don’t know much, but the five years, five seasons that I have come, I have learned from the videos they play for us” (P101). The role the farm owner/operator plays in the context of the videos was also described: “We watch videos and we have to go by those rules, protect ourselves, and the boss tells us to protect ourselves from the pesticides. He showed us [the video] so we can know the rules and protect” (P11).Farmworkers were asked which PPBs were easy to do and why they were easy, so as to understand what facilitates these behaviors. Facilitators were also mentioned in response to other questions, and the most common facilitator that emerged across questions was training , as exemplified by this quotation in which the belief was gained from personal experience: “Use long sleeve[s], gloves, boots, plastic, long pants, and a nylon apron that we have. It is easier because the one time that I worked without it, I was sick in the afternoon” (P51). Farmworkers also mentioned behaviors that they perceived as being effective : “Gloves because they protect us more, the poncho because it protects us from the morning dew, and the rubber boots because it protects our feet” (P59). PPBs were also described as easy to do : “It is easy because I feel it is easy; it is an everyday thing” (P47).Farmworkers also reported the belief that protective behaviors were effective . For example, one farmworker stated, “I have years working and I look at the people. The more we observe, the better ideas we have for ourselves” (P5). Farmworkers also described bringing their experiences from their home countries to the workplace as well: “And all those precautions, we have brought them from Mexico, from the same job, agriculture. You have almost your whole life working in it. It would be very illogical not knowing what you need in order to protect yourself; it’s very dangerous” (P13).Several farmworkers talked about their own agricultural experience and specifically watching others in the field as facilitating their behaviors : “Our boss provides latex or cotton gloves, mouth covering, and glasses. It would be my mistake not putting the gloves, mouth covering, or glasses on” (P13). Also shared as facilitators were obeying orders (n = 9), making PPBs habitual (n = 6), making one’s health his/her first priority (n = 4), the fact that PPBs take no time (n = 3), and mechanization that makes working safer than it used to be (n = 3). Farmworkers also reported that their having what was needed to do the behaviors made them easy , but barriers arose in response to questions about which PPBs were hard. Wetness in the field was the most commonly described barrier to PPBs . Wetness was described as coming from rain, morning dew, the tobacco itself, and sweat. Farmworkers understood that moisture from the tobacco could make them sick: “The water from the tobacco can get you ill” (P34). Another farmworker explained, “When the morning dew falls onto my lips, I feel my tongue numbing, but nothing else” (P73).When asked directly, “What gets in your way of doing PPBs?” the vast majority of farmworkers said “nothing” initially  said, “When one starts to sweat, the pores open and the anxiety begins. If you are not careful to bathe or wash your hands before eating something, since you have your pores open, you get dizzy or vomiting because as a worker, you have your pores open because of the sweat.” Sweating was also described as a reason for not wearing long-sleeved, water-resistant outerwear: “When I put that [protective clothing] on, I sweat more, and if I use that, I am going to sweat even more. That is why I don’t use the long-sleeve one” (P59). Protective clothing that had become wet externally from dew or internally from sweating was understood to be less effective. One farmworker stated, “The shirt starts getting wet and that’s how one starts having contact with the pesticide” (P98).n = 24, 33%) not to use all recommended protective clothing. One farmworker reported, “When the tobacco has a lot of water, one puts on the nylon. Once the sun is heating up, we take it off because we cannot bear it” (P8). Another worker described lifting his protective glasses because of the heat: “Sometimes when the sun is really strong, a lot of the times, the glass of the glasses heats up our vision and a lot of the times you pull them up a little, but it is important to have the glasses on in case the wind brings a pesticide back and it can land in your eyes” (P13). Another participant described health and protective clothing use in this way, “I do not put on a mouth covering because the heat is unbearable being there, so one feels like they are suffocating” (P1).Heat was a major reason . It was usually described in combination with recognition that PPBs are not hard to do: “[I]t [PPB] is nothing one cannot do” (P76). Time was expressed as a barrier by only a small number of farmworkers (n = 3). Farmworkers described ‘flojera’, translated as ‘weakness’ or ‘lethargy’ as a common barrier  responded initially that either there were no strategies they could suggest or they did not know what to suggest. Additionally, a couple of participants stated that they did not feel authorized to make suggestions: “Well, no, I say that one cannot have an opinion on that or say what to do. One is not capable of that” (P8). However, a number of valuable suggestions emerged from other participants.Perhaps most importantly, the participants were asked to identify strategies to improve PPBs. The majority of farmworkers . Having supplies on hand was described as “tak[ing to the field] the necessary equipment, everything of rubber and everything necessary to do those jobs” (P66). More farmworkers spoke to employer provision of supplies as a strategy than spoke of lack of it as a barrier. For example, one participant had this message for employers, “Give us mouth coverings and ponchos; in case one rips, we can change” (P58).The most cited strategy by farmworkers was to have the necessary supplies on hand  both to decrease pesticide exposure and to minimize the reported barrier of wetness. One respondent said, “If one feels bad with the dirty clothes, change against an illness since there are some that get hives and are scratching and scratching all the time. Take a change of clothes to change over there [at the work site]” (P66). Changing clothes at mid-day was specifically mentioned: “Change at noon. I change my shirt and undershirt, and put clean gloves on, shoes” (P57). One person mentioned that the time when transferring from one field to another  was a good time to change clothes as well: “Change clothes, because when you change fields, you have more time to change your shirt or something” (P91). Farmworkers described preparing a bag with a change of clothes and supplies . “I have an apron, rubber gloves in store and when one needs them, they warn us. We take a bag with clothes, with the gloves and everything one needs in a bag, and there is always a bathroom close by to wash our hands. So one washes and uses what one brought” (P25). Multiple farmworkers described efforts to change clothing throughout the day; however, keeping up with the associated washing of clothing was described as a challenge: “The hardest is that we have to wash our clothes every day, and sometimes in the morning one has to change at mid-day because in the morning one gets wet and then we get full of dirt and we have to change again” (P8). Participants suggested changing one’s clothes during the day  was also mentioned as a strategy: “When they’re spraying and pouring liquid, we try to tell one another, ‘Not in that field.’ One takes care of another” (P94). Only one participant spoke of warnings coming from a supervisor: “Talk to people. Tell them they can’t harvest, or if it’s fruit, not touch it until they tell you. There is someone that gives orders of what can and cannot be touched” (P102). Communication of warnings among workers (n = 8) to counter the adverse effects of pesticides, including skin rashes: “I put itch medication on every day” (P49). Also mentioned were using less or no pesticide (n = 5), using “suero” (n = 4), and using milk (n = 4). “Suero” was the word used to describe a powdered form of electrolytes that was frequently observed by the researchers in use in the field. One respondent explained: “You saw that we carry small packs of ‘suero’ in order to withstand the temperatures, so we won’t dehydrate. With that, our temperature drops and so does the heat. We bring it from Mexico and we also use it over there. Once you drink it, fifteen minutes pass and you are comfortable. It comes out being like a Gatorade. It is a powder that we put in our water. We cut the tobacco here with machines, but over there, it is with the hands, and if you don’t drink it, you’ll just be vomiting. We look for ways to help ourselves and move forward” (P18). Several participants described drinking milk to prevent effects of pesticides: “I drink milk. They say that milk helps to cut the intoxication that is what they say. I drink milk in the morning and in the afternoon as well. Also, at night when I can’t sleep, I drink a glass of milk” (P51). Other strategies that were suggested by participants could not be linked to a specific barrier. For instance, farmworkers suggested using medications or ointments  the symptoms of which are similar to pesticide poisoning and heat illness . Other rn = 28, 39%) reported having learned from the video training or referred to the videos as a source of information. Very few farmworkers  in this study reported receiving pesticide training in the form of a presentation/discussion.Training emerged as the most common facilitator of PPBs identified by farmworkers, and while documented problems with video training include lack of time for questions, the ability to look away and not engage with the video, and not meeting the needs of non-Spanish speakers , many faPersonal experience working in agriculture was described as a facilitator of PPBs, and less-experienced workers reported looking to those with more experience for guidance. However, other studies have reported that farmworkers who have been working in agriculture for a long time (8–10 years or longer) are less likely to use PPBs . BehaviBased on previous studies, we expected availability of supplies (or lack thereof) ,13,14,15Farmworkers were much more likely to mention protective clothing than washing behaviors when discussing PPBs. When asked about PPBs, they identified clothing behaviors before washing behaviors, and sometimes did not comment on washing behaviors at all. In a complementary observational study, we found far greater adherence with protective clothing behaviors among farmworkers than with washing behaviors before eating and drinking in the field . FindingFarmworkers identified changing clothing, with space and time to change provided by employers, as an important strategy to address wetness in the field, and they described utilizing techniques to overcome the obstacle of lack of clothing-change breaks, such as changing on the lunch break and in the bus between fields, as well as packing a change of clothes in backpacks left on the bus. Changing clothes at mid-day likely represents not only a reduction in wetness but also in pesticide and nicotine exposure ,41.During a complementary observational study, farmworker participants were observed wearing trash bags, thin ponchos, or light-weight rain jackets over their clothes while working in the fields . In thisFarmworkers described medications and ointments as being beneficial in minimizing exposures. This finding was concerning because medications and ointments are not protective and at best treat symptoms that arise as a result of exposure. The prophylactic use of anti-itch medication and “suero,” powdered electrolytes brought from Mexico intended to treat dehydration caused by diarrhea, might actually be harmful if not properly utilized. More study on “suero,” its components, and its effects is needed. Additionally, drinking milk prophylactically—a strategy farmworkers sometimes use to feel better after exposure to nicotine  and as pThe most significant limitation of this study is its inclusion of only farms that had previously participated in the voluntary Certified Safe Farm program. Certified Safe Farms were selected for participant recruitment because there was a complementary observational component to this study. In order to gain access to the private properties on which these farms were located and on which farmworkers were working, it was critical to obtain farm owner/operator consent so that observations could be conducted. While it might be expected that individuals operating Certified Safe Farms might be most likely to consent to observation on their farms, it is noteworthy that fewer than 50% of those approached for study participation consented to participate. Certified Safe Farms, by their nature, are likely to be more safety-oriented and to have a higher probability of providing resources and training to their workers than an average farm. Despite the fact that these farms likely represent a ‘best case scenario’ in supply provision and training, opportunities to improve farmworker safety and health on these farms still emerged. Other limitations to the generalizability of this study are its small number of farms, single crop, and single geographic location. The small number of farms and the variability in number of workers employed at each made it impossible to view the farms as independent units for analysis. The use of a structured interview guide and trained interviewer limited deeper exploration of some emerging beliefs and practices. Furthermore, the HBM has been criticized for failing to perform well with rural, underserved populations like this one ; other mDespite its limitations, this study was novel in its exploration of the modifying factors influencing PPBs. While barriers to and benefits of protective behaviors have been explored with farmworkers previously, this was the first examination of facilitators of PPBs and elicitation of strategies to improve PPBs. Farmworkers offered their perspectives on several novel protective behaviors that they had adopted. It was apparent that they were aware of short-term health risks associated with pesticide exposure and had adopted a variety of strategies, both those addressed in the WPS and some that are not, to mitigate exposure. However, they did not emphasize washing behaviors as playing an important role in PPBs. Within the context of the HBM, this analysis demonstrated that perceived benefits and perceived barriers of PPBs were insufficient to understand farmworkers' beliefs about PPBs and the PPBs practiced in the field. Likewise, an examination of perceived threat (severity and susceptibility) that we undertook was insufficient to predict PPBs . In bothFuture research should focus on determining the efficacy and feasibility of farmworker-identified strategies in minimizing exposure to pesticides. Resulting data should be shared with policy makers, who may reconsider PPBs currently mandated to be taught and how and by whom protective clothing and supplies should be provided. In light of the fact that recent changes to the WPS are currently being implemented and new training materials are under development across the country, resulting data should also be shared with those who are revising and creating educational materials to reflect the updated WPS requirements. Based on the findings of the present study, new training materials should focus on farmworker understanding of short- vs. long-term risks posed by agricultural work, washing behaviors, and the benefits of protective behaviors to both mitigate nicotine and pesticide exposures. Another consideration for pesticide education developers is the establishment of programs to prepare moderately experienced farmworkers to become trainers who might support increased adherence to recommended behaviors and promote peer support in the field.