All interviews were recorded, transcribed and analysed using qualitative data analysis software. Thematic analysis was performed. Our findings suggest that the PBF programme improved the perceived availability of EMs in three regions in Cameroon. The change in availability of EMs experienced by stakeholders resulted from several pathways, including the greater autonomy of facilities, the enforced regulation from the district medical team, the greater accountability of the pharmacy attendant and supply system liberalization. However, a sequence of challenges, including delays in PBF payments, limited autonomy, lack of leadership and contextual factors such as remoteness or difficulty in access, was perceived to hinder the capacity to yield optimal changes, resulting in heterogeneity in performance between health facilities. The participants raised concerns regarding the quality control of drugs, the inequalities between facilities and the fragmentation of the drug management system. The study highlights that some specific dimensions of PBF, such as pharmacy autonomy and the liberalization of drugs supply systems, need to be supported by equity interventions, reinforced regulation and measures to ensure the quality of drugs at all levels.Performance-based financing (PBF) is being implemented across low- and middle-income countries to improve the availability and quality of health services, including medicines. Although a few studies have examined the effects of PBF on the availability of essential medicines (EMs) in low- and middle-income countries, there is limited knowledge of the mechanisms underlying these effects. Our research aimed to explore how PBF in Cameroon influenced the availability of EMs, and to understand the pathways leading to the experiential dimension related with the observed changes. The design was an exploratory qualitative study. Data were collected through in-depth interviews, using semi-structured questionnaires. Key informants were selected using purposive sampling. The respondents ( Key MessagesPerformance-based financing (PBF) led to an improvement of the perceived availability of essential medicines in Cameroon through several pathways: more autonomy of health facilities (HFs), enforced regulation from the district medical team, greater accountability of the pharmacy attendant and supply system liberalization;Many challenges, including delays in PBF payments, limited autonomy, leadership and contextual factors, hindered the capacity to yield optimal changes;Attention should be paid to reinforce regulation and ensure the quality of drugs once the supply system is open to private wholesalers;PBF interventions should promote equity interventions when granting pharmacy autonomy to HFs to avoid worsening pre-existing inequalities. Access to safe, effective, quality and affordable medicines is one of the specific objectives included in the Sustainable Development Goals . This acThe availability of EMs in low- and middle-income countries is influenced by a variety of complex, underlying factors such as inadequate financing, weak facility management, regulatory issues, lengthy procurement processes and poor logistics , covering a total population of 3 million inhabitants. Alongside the implementation of the project, the World Bank conducted an impact evaluation (IE) in three out of the four regions   and the Regional Supply Pharmaceuticals Centres.In 2008, the government launched its PBF scheme. The beneficiaries were public, private for-profit and faith-based facilities. The PBF intervention entailed a shift from an input-based to an output-based financing of healthcare services by introducing performance contracts between the performance purchasing agency (PPA) and single facilities. These contracts focused on the complementary package of activities for hospitals , and the minimum package of activities for health centres . Performance contracts regulated both performance payments from the PPA to the facilities and performance bonuses that facilities could distribute to their staff, which could not be >60% of the total PBF income. The PPA signed quarterly contracts with health districts both to evaluate the technical quality  of health centres and also, with the Regional Delegation of Public Health, to (1) organize peer evaluation of hospitals, and (2) carry out enhanced supervision of the health districts’ activities. Moreover, the PPA signed quarterly contracts with community-based organizations to conduct evaluations at the community level. Contracts targeted priority services , mostly related to maternal and child health, and operated according to a fee-for-service basis with additional adjustment made for quality of service delivery . FacilitThe opening of the drug procurement system to private actors was a new practice in Cameroon. Although the Regional Supply Pharmaceuticals Centres and the RFHP were the sole organization mandated to supply and distribute drugs to public facilities at the regional level, the PBF system brought other players into this drug supply system. In addition, for the sake of the sustainability, ownership and integration of the PBF intervention into the health system, the RFHP in each region became the PPA (The intervention theory on EM is presented in PBF in Cameroon placed an emphasis on four areas: (1) the purchase of services; (2) services delivery; (3) drugs and medical consumables; and (4) governance. The ‘purchase of services’ refers to the financial incentives that the purchasing agency gave to facilities in exchange for the services provided and the ‘service delivery’ refers to the actual healthcare services that the providers gave to patients. These areas of intervention were supported by two functions: the strengthening of the supervision of HFs and their management autonomy. For the EMs-related management autonomy, PBF provided facility managers total control over pharmacy management . PBF supported the four above-mentioned areas with the coaching and supervision of health personnel, payments for health services and personnel tied to qualitative and quantitative performances, the opening of drug procurement systems to private actors, and the separation of functions .PBF was expected to improve EM availability in HFs due to changes applied at three levels: supply system, regulation and/or facility management . Within The PPA was expected to sign performance contracts with the Regional Public Health Delegation for, among other things, the maintenance of the quality of the EMs and the drawing up of the list of accredited wholesalers. HFs could then use the list to select a supplier of choice. With the liberalization, competition between different actors was expected to result in a more balanced sale price from suppliers and an increased the availability and quantity of EMs. HFs were expected to use the additional funding from the PBF programme to strengthen the infrastructures and logistics  that could support the purchase, collection, transportation and storage of EMs. The intervention explicitly mentioned the need for regulators to focus more on their core functions, and performance indicators were defined in relation to more formative supervisions and coaching for staff, and more quality assurance of EMs .We employed an exploratory qualitative study . ExploraCameroon is divided into a three-tier administrative system including 10 regions, 58 divisions and 360 sub-divisions. The population of Cameroon was estimated to be 22 218 000 inhabitants in 2015. The health system in Cameroon is decentralized and organized into central, regional and district levels.2, the East is the largest region in Cameroon and the most sparsely populated with a total estimated population of 1 110 000 inhabitants in 2015. The region is divided into 14 health districts, all implementing PBF. The region was governed through the French direct rule administration until the early 1960s. Although the East region is blessed with natural resources, its population remains one of the poorest in the country, and the region’s rurality is also one of the most important in Cameroon.With 109 002 km2 and a population of 3 355 000 as of 2015. The region has 19 health districts, of which four were implementing PBF. The region was governed through the French direct rule administration until the early 1960s. The Littoral region is the wealthiest region in the country, with the largest sea port and the main industrial area of Central Africa. It also has large plantations of banana and cocoa.The Littoral region, located at the seacoast, has a surface area of 20 248 km2 and an estimated total population of 2 716 000 inhabitants for 2015. The population is unevenly distributed among 19 health districts, of which four were implementing PBF. The Northwest constituted one of the two English-speaking regions of Cameroon. Up to the early 1960s, the region was under the British indirect rule administration.Located in the western highlands of Cameroon, the Northwest region has a surface area of 17 812 km2 and its population was estimated at 1 853 000 inhabitants as of 2015. Of the 18 health districts of the region, four were implementing PBF.The Southwest region is 25 410 kmAt the time of the study, the PBF programme was operational in 26 health districts located in four regions  covering a total of 661 HFs , and 180 were in the control Group .As we wanted to deepen the mechanisms PBF put in place to facilitate the availability of EMs and how these mechanisms interacted with the overall context of the health system to produce or hinder the availability of EMs, we restricted our study to the intervention group.r) in the performance payment received between the start (Pi) and the end of the randomization phase (Pf) of the intervention, i.e. Tr = [(Pf – Pi)/Pi]×100. This categorization was further validated by key informants of the central level of the Ministry of Health (MoH). We selected contrasting HFs (low and high performance) and also took into consideration the location  and the type (public vs private). Public HFs are directly supported by the Government and manage their expenses with Government budgets. Conversely, Private-for-profit and Confessional HFs manage their expenses from their own investment or self-generated income. Consequently, the costs of services are theoretically high in these two last categories of HFs compared with those of the public group. All three types of HFs provided the same basic package of activities, although some particularities existed. For example, private HFs were less keen to offer preventive services. As for the location of the HF , the main characteristics relied on their population catchment area, frequentation and geographical accessibility. HFs in rural areas tended to have less population and patients, and to be also less accessible.In the pre-assessment phase, we categorized intervention facilities according to their levels of performance between 1 July 2012 (start of the programme) and 30 December 2014. The performance was calculated as the relative increase , including 3 privates for profit, 3 confessional and 18 public HFs, were retained for the study. Among the 24 HFs, five were located in urban areas. These different locations, settings, as well as owners of the facilities, provided a range of relevant experiences to our subject of interest see .We interviewed a total of 55 participants. The selection of participants was purposive. This approach is often used in qualitative research to identify actors who have some characteristics or are in situations pertinent to the object being studied , selectiMonnier and Spenlehaur (1992) describe three major roles that define the different actors of the decision triangle: the role of legitimation for qualified decision-makers, those who formalize decisions (i.e. the legitimizers), the role of action held by the implementers, those who operationalize those decisions (i.e. the actants), and finally that of the reaction to the policy envisaged or ongoing (i.e. the reactants). Legitimizers have an institutional power that does not stem solely from their status but also from their roles and capacities of integration between the different actors. They have no operational responsibility. In our study, the legitimizers included policy-makers of the central and regional levels of the MoH. Actants are actors who have views that in theory reflect the mandate they hold; however, the mandates are very often defined in an imprecise way, leaving them important room to manoeuvre. These actors included health workers, private wholesalers, and technical and financial partners of the PPA. Reactants mainly include the disparate set of civil society actors subject to or beneficiaries of the policy (i.e. users of HFs and members of the community) .As a first step, during a meeting organized by the MoH in March 2015 in Yaoundé, we conducted interviews with four regional level PBF managers and the national coordinator of the PBF programme, who played a role in formulating and implementing the PBF programme. In interviewing these key stakeholders, we also asked them about prospective participants to reduce selection bias. Each prospective participant was contacted by phone either by the first author or by a trained research assistant. Between April 2015 and May 2015, we conducted interviews with 50 participants, including 2 policy-makers at the central level, 21 health workers, 6 actors in the public supply drug system, 6 private wholesalers, 1 actor from international organization and 10 members of community-based organizations and health committees (see Table 2 for more details). The participants did not receive any incentive nor compensation in exchange for their participation in this study.n = 55). In the qualitative approach, interviews are frequently seen as one of the best techniques to ascertain the other person’s perspective (n = 47) in an interviewee’s chosen place and time or via videoconference (n = 8) by the first author and trained research assistants and lasted about 40 min each. The interviews were conducted in English (n = 32) and French (n = 23), audio-recorded and supplemented by note-taking. These interviews aimed to elicit the emic perspective (Data were collected through in-depth interviews (spective . The intspective  of some spective . To minispective  on (1) tspective . All intIt should be noted that we conducted our data collection (March to May 2015) before the IE was over (April to June 2015) and hence our study was not a follow-up to the quantitative IE.Transcripts were recorded verbatim and entered into QDA Miner Lite  for analysis. Each transcript had a unique identifier comprising date, region code, participant type, level of work, HF’s code, allowing exploration of data by subgroup status (e.g. HF performance). Analysis followed the four practical stages suggested by Coding into themes was carried out in a 2-fold manner. The second author developed the initial coding scheme, based on the literature and on the themes included in the interview guide, then discussed it with the research team and came to an agreement on the coding and categorization. The second phase of coding was during transcription, where prominent issues were marked and further discussed within the research team.Two researchers independently coded the transcripts of all the interviews and field notes using the agreed coding and categorization. We applied the predefined codes to the transcribed material, but allowed new codes to emerge where coding categories were directly derived from the text data  as we prn = 3) and regional (n = 4) levels of the MoH, allowing us to validate or refine our final interpretation  and the WHO Research Ethics Review Committee (RBF2014–395). Administrative authorization was granted by the Cameroonian Ministry of Health (D30-298 L/MINSANTE/SG/DROS/CRSPE/CEA2) prior to data collection. All respondents provided verbal or written informed consent before interviews. To ensure the confidentiality and anonymity of the data, we replaced participants’ names with codes.It was clear for almost all interviewees that drug availability was a challenge that existed before PBF. The choices of drugs in the pharmacies were limited and affected by frequent stock-outs. Patients sometimes had to travel far to buy drugs prescribed in facilities. According to respondents, the PBF intervention had positive effects on the availability and variety of drugs. Respondents reported that these effects progressively minimized stock-outs. Key findings are displayed to (1) present the causal pathways that influenced the availability of EMs and present the possible mediating factors affecting the ability of the programme to produce expected changes, and (2) highlight concerns about the PBF intervention in relation to the supply system of EMs. In addition, findings are also presented to highlight differences between high- and low-performing HFs .The findings highlight the possible pathways through which the PBF intervention influenced the availability of EMs in facilities. More specifically, the results suggest that the availability of EMs was linked to supply system liberalization, the increased autonomy of facilities, the enforced regulation from the district medical team (DMT) and the greater accountability of pharmacy attendants. The PBF intervention changed the existing institutional configuration by bringing new actors into the healthcare system. More specifically, the principle of liberalization led to the introduction of new accredited actors in the provision of drugs. Key stakeholders explained that opening the public national EMs supply system in Cameroon market to the private sector was one of the noticeable actions of the programme. As one informant put it:In fact, we go to ‘the lowest bid’. It's business […] L-PHARMA does not offer generics. Only specialties. […] But those ones, we get them from P-PHARMA, they are cheaper to buy, for the same quality .Respondents believed that the liberalization brought changes such as the possibility of having more drugs available as well as having more choices of drugs at the wholesalers’ level. However, the influence of the liberalization of drug prices remained limited. Respondents said that most private wholesalers were subject to heavy taxes that the national essential drugs and disposables procurement centre did not have because the latter benefited from numerous fiscal advantages. Consequently, PBF did not considerably influence the prices of drugs sold by private wholesalers.There were noticeable differences between the participants’ understanding of the principle of liberalization. Managers of facilities who understood the principle of liberalization seemed to be remarkably motivated and were keener to do more to improve the performance of their pharmacy compared with those with a lesser understanding of the concept. It also appeared that managers of top performing facilities had good understanding of the principle of liberalization compared with health workers in poorer performing facilities. These two respondents in top- and poor-performing facilities illustrate this:Liberalization pertains to our freedom and rights to buy our essential medicines where there are advantages for us, in term for example of price and type of the product. You will notice that our pharmacy has changed a lot since we have the possibility to buy essential medicines from various suppliers .Euh… I once heard from a colleague in a nearby facility that the ministry will put in place many organisations to supply facilities with EMs because the RFHP that is currently doing the job is overwhelmed and not performing well. That said, we are still buying our drugs only from the RFHP and it is what we were doing before that we are doing now .There were diverse views about the reasons for the poor understanding of the liberalization in poor-performing facilities. Although some facility managers pointed out the communication gap between the district and facilities, others highlighted their limited comprehension of their power over the pharmacy management.Although managers of the poor-performing facilities felt that they were informed about the EM supply system liberalization, they pointed out inadequate communication in the planning and implementation phase as a source of much frustration, as highlighted by the following quote:In the launching meeting in Bamenda, we were told that over the course of the program, we will be granted the authorization to buy drugs from other suppliers. However, things were not so clear. We did not receive any formal text nor directives to do so. After a year of implementation, we were informed during a coordination meeting that it is up to everyone to implement the mechanism. You see, they decided and just want us to act without any legal text signed .All managers of the poor performing facilities said their workload increased significantly with the PBF programme and that this increase seriously compromised their daily practice and their management capacity. They expressed their view that the personal and professional impact of the PBF programme was negative. About two-thirds of them disagreed with the statement that ‘liberalizing the EM supply system increases the pharmacy income and the personal financial incentive’. In expressing negative views, the health personnel of these facilities appeared to draw a valuable difference between policy objectives and its implementation. For example, they pointed out the lack of support strategies to optimize and to standardize implementation with the aim that the intervention is as uniform as possible.Respondents from the DMT further described how managers of the poor-performing facilities were not keen to move out of their comfort zone as they continued to manage their facilities without any change in their practices, as shown by the following extract, raising therefore questions regarding the quality of the supervision from the district:Since the beginning of the program, we have noticed that some chiefs of health facilities don’t want to change. We have explained to everyone that the essential drugs supply system is now open to many accredited providers, but health facilities that you observe with poor performance have still not changed their practices despite multiple supervisions .When asked about what role they played in the implementation of the drug supply system liberalization, facility managers reported several strategies they put in place. Their opinion was that these strategies helped to improve the performance of their pharmacy. In particular, these strategies  were only experienced in high-performing facilities. For example, two facility managers claimed that they leveraged loans with low interest among their personnel to buy the initial stock of drugs after the withdrawal of the RFHP from the pharmacy management. The following excerpt is from one of these participants:When we started to manage the pharmacy, we did not have funds to buy drugs. During a monthly meeting, one of our colleagues raised the idea that we could use our PBF incentives and put it as investment shares, so each month we could receive investment return as profit. We did it and for the past four months that we started it, we have been receiving at least 15, 000 FCFA [30 $US] per month, depending on the initial amount of investment. In addition, our pharmacy has now enough drugs, that is a win-win business .More broadly, respondents suggested that without local support from the personnel, they would not be able to deal with the EM liberalization, pointing to the likely role of staff cohesion as a resource for implementation.Before PBF, the centralized management system did not recognize any form of autonomy in the management of pharmacies in facilities. Staff at the operational level had no access to EMs revenues. Although these actors were previously considered to be executors in the distribution chain, PBF made them partners. The status of simple executors in a situation where actors felt that the benefit was far below their expectations favoured illegal practices such as the embezzlement of drugs and funds, insofar as actors resorted to parallel circuits other than the RFHP. Interviews with front-line health workers provided more insight:Public facilities are forbidden to get supplied elsewhere when the RFHP does not have all the drugs. It creates a black market. […] .Under the monopolistic regime, the distribution of EMs was described as a ‘push system’ associated with a central agency (RFHP) supplying facilities . FacilitSeveral managers of facilities felt that this situation was worsened by the fact that pharmacies were out of their control. With the liberalization, the new configuration of the distribution system put facilities in the vanguard. They became responsible for evaluating their needs and became also more responsive to stocking up their pharmacy in case of drug shortages. The new approach was seen as a ‘pull system’ coupled with liberalization, where facilities directly purchased drugs from accredited sellers. One informant explained:Our platform is the monthly meeting during which we evaluate our needs and also assess the previous month. During the month, if the facility faces a problem with drugs availability, the chief has the power to find solutions and then report it to the committee later on during the monthly meeting .According to facility managers, the autonomy gave them the flexibility and freedom to manage their own resources . Similarly, several facility managers mentioned having full control over the management of the benefits generated by the sale of medicines. Informants explained how surplus resources allowed them to improve the storage space and the working environment:We are now making considerable profit from the drugs and we are able to use it to construct more storage space. […] .Autonomy seemed to be much appreciated even by other members of staff as well as members of health committees at the community level. According to them, it had become easier to react quickly to any stock-out problems. From their perspective, it was really when facilities were granted more autonomy that the shortage of drugs decreased, as one community member expressed:To speak frankly, the issue of stock-out was very common in our facility. After one year of the program implementation, our facility started to be completely autonomous. From that period, there is no more shortage of drugs in our facility .However, results showed that the principle of autonomy stood against the interests of certain national essential drugs and disposables procurement centre and RFHP actors, in the sense that it might lead them to lose hidden advantages such as baksheesh. The preservation of their interests is an element that weighed a lot in their behaviour when implementing the principle of autonomy. One informant recalled:You know it’s not everyone who will agree with the autonomy principle. For some, it makes a kind of hollow in the pocket if they don’t sell their drugs .Although the advantages of autonomy were clear for most actors at the operational level, it faced an organizational culture of procedures and inaction that characterize public organizations in Cameroon . This leIn the PBF when you are in charge, you manage all your resources including drugs. So, the facility decides or not where to go for drugs, and is no more accountable to RFHP. […] The chief of the facility manages with his staff. [It’s]not that you sell and somebody in the regional city tells you that it is up to the RFHP to manage your drugs .The views and experiences of participants regarding autonomy varied across facilities with noticeable differences between health workers from good- and poor-performing facilities. For example, facilities that autonomously recruited and managed their pharmacy attendants acknowledged the highest performance in the three regions, whilst managers from low performing facilities tended to believe that autonomy would never be a reality because it is very complex to manage. Some facility managers from high- and low-performing facilities recalled:Autonomy is really a good thing. We have recruited additional staff for the pharmacy, and this is something that would has never happened without PBF .The image that people have outside is that we are not doing anything, yet to us, it’s not deliberate, it’s just that the business is very complex. We have been trained on how to manage our resources, though it is not easy for us to apply everything .At the same time, the vast majority of respondents made it clear that the autonomy process remained partial. They had concerns regarding the ability and willingness of the RFHPs to fully transfer the management of pharmacies to facilities. From their point of view, some RFHP actors even attempted to block the process, refusing e.g. to carry-out the initial inventory of the pharmacies of health centres, a preliminary step toward the autonomy.The most outstanding changes experienced by interviewees regarding external supervision of the pharmacy were its regularity and intensity. Most respondents perceived an increase in the availability of EMs in their facility as a consequence of more supervisions and coaching from the DMT. DMT was reported to visit facilities on a monthly basis and used a structured supervisory checklist. In addition, on a quarterly basis, they assessed structural quality components  and treatment protocols. Given that the scores that pharmacies received influenced bonus amounts, pharmacy attendants and facility managers considered this process to be vital to their achievement in the programme. They felt continuously inspected but were happy about the supportive nature of the enhanced supervision and coaching. The improved performance resulted from close formative supervision, coupled to the projection of earning more money, as one informant expressed:The DMT comes here at least once a month for the supervision and each time, they spend half day with the chief and me to review the management of the pharmacy. When they notice a problem, they tell us how to do and to avoid it in the future .The monitoring and evaluation indicators that were included in the quarterly contract and business plan, and that contained indicators regarding EMs, were used to assess the achievement of the facilities’ objectives. In case of a breach of contract, facilities incurred sanctions ranging from temporal to permanent suspension of the contract. A member from the community explained:Our health centre had been suspended for three months. […], well … I was told that something was wrong with the management of the pharmacy. […] .When we asked the district health authorities what they thought about their role in influencing the availability of EMs, they responded that they considered themselves to be at the centre of the intervention. They received funding from the PBF programme to improve their supervisory and regulatory roles. They said that in their own contract with the PPA, they were evaluated based on the number of formative supervisions and on the coaching of the management of pharmacies. In addition, payments to the DMTs were adjusted based on the availability of tracer drugs, the storage of EMs stocks, the absence of expired drugs and the financial management related to the sale of EMs. They also insisted on their mandate to enforce the utilization of protocols and guidelines by health workers to improve rational prescription in facilities as well as their mandate to penalize facilities that could not meet policy norms.Comparisons between participants from high- and poor-performing facilities regarding the regularity and intensity of the supervision, as well as its content, revealed similar patterns.Although the majority of respondents described positive changes in enforcing the regulation, many of them also mentioned how implementation-related challenges attenuated these changes. These challenges were linked to human and financial constraints. The most cited example was the delays in payment from the PPA, leading to the demotivation of the DMTs. They reported that in such situation, they had to cancel supervision missions or just carry it out very superficially.Before PBF, pharmacies were extensions of the RFHPs, with no direct link with the facilities’ management teams. With the advent of PBF’s principle of autonomy, this aspect was changed, and pharmacies became the property of their corresponding facility. Pharmacy attendants became supervised at the local level. Moreover, for almost all the cases, the pharmacy attendants were recruited locally and were not civil servants. They could be fired for not achieving objectives. A respondent stressed this, saying:Last year our first pharmacy attendant was fired after only six months of work. […] .However, some pharmacy attendants expressed concern over their fragile position as some facility managers could abuse their power. One respondent said:Everyday, I am just afraid that the manager [chief of the facility] can end my contract overnight. He once told me that he is the boss and that he can even stop my contract without any reason .In general, all pharmacy attendants supported the idea that PBF had made them more professional in their duty. First, they acknowledged that they benefited from onsite trainings and supportive supervisions from DMTs allowing them to improve their skill on stocks management, reporting and planning. These trainings were key to identify priorities in service and areas in need of improvement, as well as developing solutions. Secondly, PBF brought more clarity regarding their duties by elucidating their term of references and explicitly linking it to their performance salary. Finally, about a third of pharmacy attendants believed that they had been given more responsibilities, making them proud and confident.Both implementers and policy-makers highlighted the perceived improvement of transparency of the pharmacy management. Facility managers became more proactive in reporting the pharmacy’s financial accounts. Numerous intervention practices were important to this effect, including the continuous performance monitoring and feedback, coupled with quality evaluation, enhanced supervision and coaching by both the PPA and the DMT. A respondent stressed this by saying:The implementation of the business plan states how we should report our financial accounting, be it the one of the pharmacy, the laboratory, or any other service. Each month, we display our financial statement and the product of the indice tool on the wall […] .In high-performing facilities, PBF improved the pharmacy attendants’ perceived ability to do their job. Pharmacy attendants also voiced their appreciation for how the programme helped them develop their skills. They reported how the improved managerial environment resulting from the intervention made them feel more effective in translating their skills into practice and become more accountable. As one pharmacy attendant stated:Let me take an example. With the PBF program, we established monthly coordination meetings during which each unit must present detailed monthly managerial strategies, activities, statement of accounts and perspectives of the months forward. This is how we are on a continuous learning process and implementing what we are learning. I can assure you that I am now feeling more comfortable doing my job .Many health workers further described how they felt that their work teams had improved service delivery by lowering drug shortages and increasing responsiveness.In low-performing facilities, however, perceptions about the improvement of the transparency of the pharmacy management remained much more mixed. Although respondents generally agreed that the intervention could improve the management of the pharmacy, they were not able to support their statement with concrete facts.The regional health delegation’s mandate of regulation included drug quality control. Private wholesalers had to get accreditation before any drug was supplied to facilities. However, this accreditation was likely based only on administrative elements and not technical ones . In fact, the regional delegations in charge of giving these accreditations did not have the expertise to conduct quality control. Without a real quality control mechanism at the regional level, many of the interviewed actors expressed their worries concerning the deterioration of drug quality. A respondent who played a key role in implementing the programme gave us more insight:I must acknowledge that our quality control remains very poor. The quality of a drug is not who the manufacturer is but how it is handled. […] .According to regional stakeholders, monitoring and quality control should be facilitated and reinforced because of the various sources of drugs.According to informants, the principle of solidarity was an advantage of the supply system implemented prior to PBF. The drug supply system management was centralized at the level of the RFHP, which had the mandate to supply all public HFs, regardless of their size and location, through a push system. For example, respondents explained that during supervision missions prior to PBF, the RFHP picked out drugs with a close expiry date in one facility and repositioned them in another facility that had huge volume of activities or shortages of these drugs. Risks and administrative costs were, therefore, shared between facilities, as pointed out in one interview:It is because of the high income of the people in the urban area that we can sustain the rural area pharmacy. Actually, we've calculated that a pharmacy that does not make sales of >2 million CFA francs [$4 000 USA] a year cannot comfortably pay a pharmacy attendant, pay the transportation, […]. So, if the solidarity system was to stop, then I think those pharmacies will not be sustainable .Participants argued that with the application of the PBF’s principle of autonomy, small facilities that were not very active or that were hard to reach experienced financial difficulties and more drug stock-outs. These facilities faced low utilization and consequently low-cost recovery, especially for EMs. With PBF, these facilities started to buy small quantities of drugs on a regular basis because they did not have enough funds to acquire large quantities as they used to with the RFHP. As a result, fixed costs exploded, and funds to restock the pharmacies dwindled, leaving room for frequent stock-outs. The low resource mobilization faced by small facilities and those hard to reach was not only challenging for the facilities’ budgets, and subsequently the work environments but also for the pharmacy attendants’ enthusiasm to be present. One informant explained:Incentives are good. Unfortunately, we do not have them […]. They are good because they improve on one’s motivation. Let me give you an example of the maternity ward. For the last three months, we have received less than ten deliveries, so our recovery cost is very small and so does our PBF income. At the end of the day, our revenue is not enough to replenish our pharmacy stock and to incentive the staff .One of the PBF programme’s objectives was to improve the health information system , includiDuring our supervision in the field, some facilities were using separate drug cupboards for the ‘PBF drugs’ […]. There are some specific tracer drugs that the PBF qualitative evaluation checks. Facilities tend to put more emphasis on those tracer drugs .This study makes a unique contribution to the literature by exploring how a PBF intervention can influence the availability of EMs at the primary healthcare level. The Cameroon PBF case placed the drug supply system at the centre of its design, with the RFHP identified to take over the performance purchasing role from the early stage of the programme in order to facilitate its sustainability and ownership in the long term . The findings of this study confirm other studies’ evidence about unintended consequences that a programme such as PBF can cause beyond the targeted objectives of the intervention , and each coder must be consistent over time  and the WHO Research Ethics Review Committee (RBF2014–395). Administrative authorization was granted by the Cameroonian Ministry of Health (D30-298 L/MINSANTE/SG/DROS/CRSPE/CEA2) prior to data collection. All respondents provided verbal or written informed consent before interviews. To ensure the confidentiality and anonymity of the data, we replaced participants’ names with codes.czz084_Supplementary_AppendixClick here for additional data file.


Single-cell RNA-seq (scRNAseq) is a powerful tool to study heterogeneity of cells. Recently, several clustering based methods have been proposed to identify distinct cell populations. These methods are based on different statistical models and usually require to perform several additional steps, such as preprocessing or dimension reduction, before applying the clustering algorithm. Individual steps are often controlled by method-specific parameters, permitting the method to be used in different modes on the same datasets, depending on the user choices. The large number of possibilities that these methods provide can intimidate non-expert users, since the available choices are not always clearly documented. In addition, to date, no large studies have invistigated the role and the impact that these choices can have in different experimental contexts. This work aims to provide new insights into the advantages and drawbacks of scRNAseq clustering methods and describe the ranges of possibilities that are offered to users. In particular, we provide an extensive evaluation of several methods with respect to different modes of usage and parameter settings by applying them to real and simulated datasets that vary in terms of dimensionality, number of cell populations or levels of noise. Remarkably, the results presented here show that great variability in the performance of the models is strongly attributed to the choice of the user-specific parameter settings. We describe several tendencies in the performance attributed to their modes of usage and different types of datasets, and identify which methods are strongly affected by data dimensionality in terms of computational time. Finally, we highlight some open challenges in scRNAseq data clustering, such as those related to the identification of the number of clusters. Single-cell RNA sequencing (scRNAseq) has emerged as an important technology that allows profiling gene expression at single-cell resolution, giving new insights into cellular development , dynamicOver the last few years, a number of computational algorithms have been proposed to analyze scRNAseq data, focusing on different aspects . In partAnother great difference is given by the specific clustering techniques implemented in each method. Some of the methods use partitioning algorithms  in orderAll of the mentioned variations across clustering pipelines affect the performance of the methods. Currently, there is a limited amount of studies that infer clustering performance and robustness under various data-driven scenarios . The maiFor that purpose, we first described different modes of usage and parameter settings of 13 among the most widely used scRNAseq clustering methods implemented in R, then we applied them on a large set of real scRNAseq and simulated datasets. In order to fully understand the potential of each method, we tested them varying a wide range of available parameter settings which greatly expands the number of possible results. Through the analysis pipeline, we evaluated the performance of the methods with respect to several factors. First, we divided the real datasets into two groups, those that were expressed in the raw counts and those expressed on normalized fragments per kilobase of transcript per million mapped reads (FPKM) or reads per kilobase of transcript per million mapped reads (RPKM) counts. On the first group, we evaluated the performance of the methods on three data basic preprocessing types . On the second group, we evaluated the performance of the methods depending on a various number of dimensions supplied to dimension reduction techniques prior to clustering. Synthetic datasets were used to prove the capacity of each method in handling varying dataset dimensions that can additionally be diverse in the number of simulated cell groups and the type of group balance. In the simulation, we also accessed the accuracy of the methods in recovering cell population structure in the presence of noise. The type of noise that we simulated were dropouts and overlapping cell populations which are key features of scRNAseq datasets. In all cases, we evaluated the performance of the methods in terms of i) Adjusted Rand Index (ARI) index, ii) accuracy of methods in estimating the correct number of clusters, iii) running time.Overall, this work aims to provide new insights into the advantages and drawbacks of several scRNAseq clustering methods, by describing the ranges of possibilities that are offered to users and the impact that these choices can have on the final results. We also tried to identify some open challenges for future research that still need to be faced when doing the population inference.https://hemberg-lab.github.io/scRNA.seq.datasets/.In order to evaluate the performance of the clustering methods considered in this study we used 17 real scRNAseq datasets popular in the literature and listed in The selected scRNAseq datasets vary in terms of organisms, tissues under study and experimental protocols. As illustrated in We evaluated methods performance also on synthetic datasets. The simulation study was performed using Splatter package . SplatteIn the first simulation setup   that defIn this study, we evaluated 13 different methods aimed to identify cell populations from scRNAseq data. Most of the methods  considered in this study can be applied by setting different parameter combinations, thus providing potentially different results. Such combinations of parameters allow the user to tune different modes of usage, such as including or not an additional preprocessing step, including or not a dimension reduction procedure, using different criteria for choosing the suitable data dimension, applying different clustering algorithms within the same method, setting or estimating the number of clusters. As shown in Methods can be also customized by the clustering techniques they apply. Some of them are based on a fixed clustering technique (herein denoted fixed), others propose multiple choices in this step . The group of methods with multiple clustering options include: monocle3 that offers two types of clustering techniques, RaceID3 that utilizes two partitioning algorithms and a hierarchical clustering algorithm, sincell and sscClust which provide more clustering options. Depending on the clustering technique, methods either require to set the number of clusters by the user  or provide an internal functionality to estimate it . For more details about specifications, see methods descriptions in Finally, we stress that all 13 methods  can be applied to non-preprocessed or filtered Raw counts as well as simulated datasets see Figure 2To quantify the agreement between the partition obtained from the considered method and the true partition, we used a well-known and widely used measure, the Adjusted Rand Index (ARI), implemented in the R package mclust . The valTo evaluate the accuracy of methods in estimating the correct number of clusters, we used symmetric log-modulus transformation defined as follows:x is the difference between the estimated number of clusters and the true number of cell populations in a given dataset. The positive values of log-modulus transformation mean that the number of estimated clusters was higher than the number of true cell populations. Negative values indicate that methods underestimate the number of clusters whereas zero values denote the equality between the number of estimated clusters and the number of true cell populations.where To identify significant differences in methods performance (ARI Index) when applied after different basic preprocessing types, we used hypothesis testing procedures implemented in stats R package . The KruSys.time function from R that allows reporting time when the method starts and finishes the script. The difference between those time points constituted the computational time of the method in running dataset analysis. Note that computational times have been reported in the unit of minutes followed by log(t+1) transformation, where t is the running time in minutes, and include all the steps that the method needs to cluster a dataset (except data basic preprocessing) together with the loading of the required packages and package dependencies.Finally, to measure the computational time required by each method to complete its task, we used https://github.com/mkrzak/Benchmarking_Clustering_Methods_scRNAseq. The repository stores codes for data import, processing, and analysis as well as the information about system requirements and packages to be installed. When performing the analysis, additional HTML reports are produced with a detailed description of data analysis steps. Note that apart from the required methods, the analysis scripts call for other R packages used in plotting and managing R objects. The scripts have been tested on R version 3.5.1 and machine with specifications—Intel Core i7, 4.00 GHz × 8 and 24 GB RAM which are the minimum system requirements for the analysis.This clustering benchmark study was implemented in the R programming language and scripts necessary for the reproducibility were deposited at the time of publication on the GitHub page: https://github.com/DataStorageForReproducibility/Real_data_for_benchmark_reproducibility and https://github.com/DataStorageForReproducibility/Simulated_data_for_benchmark_reproducibility. Both directories include. RData files as SingleCellExperiment class objects that store the count matrices and the corresponding cell group annotations.Moreover, for the sake of completeness and to ensure the reproducibility of our study, we deposited the real and simulated datasets on the following GitHub pages: In the clustering benchmark, we set the seed for generating pseudo-random numbers globally and applied it to the execution of any method in order to assure the stability of the solutions and reproducibility of the results. Note that, since the scRNAseq R packages we evaluated are often under continuous development, other version of the methods (R packages) than those reported in Results are organized as follows. We first illustrate the performance of the evaluated methods on the 10 raw datasets, then on the 7 normalized FPKM/RPKM counts. Finally, we finish the summary of the main findings obtained on the simulated datasets in the 3 setups described in Within this paper, methods/parameter combinations are referred as string obtained as a concatenation of keys separated by underscores. The concatenation takes the name of the method, the type of additional preprocessing, the dimension reduction technique, the setting of the number of dimensions, the clustering technique and the number of clusters. Each of these keys can take the values reported in As mentioned, we independently applied all 13 methods (corresponding to 143 parameter combinations) to the 10 raw counts datasets after using two basic preprocessing types . Then, we applied only 9 methods (corresponding to a subset of 100 parameter combinations) to the same datasets after applying quality control, filtering and normalization , SIMLR method, sincell (with k-medoids and ward.D chosen as clustering techniques), all combinations of sscClust except SNN and RaceID3 (when k-means was applied). A special case of overestimating the number of clusters method was observed with sincell where a large number of cluster was often returned. For example, sincell used with max.distance technique always returned a number of clusters equal to the number of cells in the dataset whereas in combination with knn it also returned a very large number of clusters.We found that most of the methods performed similarly when changing the preprocessing procedures see 4, althoAs mentioned above, To better investigate this, we computed the PCA of the ARI matrix obtained using the 133 methods/combination as variables and the 10 raw datasets as samples. Overall, In particular, the analysis of In the same spirit, We also superimposed other features, such as dimension reduction or clustering techniques (not shown for brevity). Since such parameters can assume multiple values, the figures do not allow to identify any suggestion that works well for all methods. However, such analysis allowed us to recognize i.e., sscClust with iCor and Seurat with internal number of reduced dimensions, as one of the good performing combinations.We compared run times of the methods across all 10 raw datasets in order to assess their scalability and identify potential issues related to a specific dataset.We used 7 FPKM/RPKM datasets to evaluate the performance of the methods/parameter combinations considered in this study with respect to the number of reduced dimensions when using different dimension reduction techniques. Since FPKM/RPKM datasets consist of already normalized counts we limited the study to those methods/parameter combinations that do not use “method specific” as additional preprocessing and that also allow setting the number of reduced dimensions. In total, we tested 44 methods/parameter combinations on each of the four dimensions: 3, 5, 10, and 15.As in the previous case, we note that some of the methods/parameters combinations failed to cluster some of the datasets   , or often poorly, as, sincell (with many parameter settings) and ascend. Other methods, such as DIMMSC, monocole3, RaceID3, Seurat, SIMLR and TSCAN, can be placed in the middle in terms of overall performance across all datasets, despite the fact that on few datasets they could have reported good performance. However, we should consider that the field of clustering of scRNAseq data is rapidly evolving. Novel methods are continuously emerging and those that we have compared are undergoing to an extensive revision that might improve their performance. It is not easy to explain why certain methods work better than others, since they perform several steps before applying the clustering algorithm. However, one of the reasons is that some methods were originally developed to analyze scRNAseq data collected under specific protocols . Then, the novel challenges (in particular the increasing size and cell heterogeneity) provided by the rapidly evolving scRNAseq technology made them not any more competitive for the complex types of data that are emerging. Reasonably, methods have to be optimized with respect to a specific protocol or dataset size, rather than attempting to find methods that work well on a wide range of scRNAseq conditions. In fact, our study showed that no methods seem to emerge as performing better than others on all datasets. Additionally, our results also showed that there is still space for improving the overall performance of the available methods on large and complex datasets or providing novel and more accurate methods.We have found that despite different basic preprocessing options, there is no global pre-processing strategy which improves significantly the performance of all methods (packages). Instead, we found that the performance of several methods strongly depends on their parameter settings: in Seurat when varying the number of input dimensions; in SIMLR when estimating the true number of cell groups; in sincell when varying the clustering techniques and in sscClust when changing the dimension reduction step. We believe that the impact of the choice of the method-specific parameters on its performance has been underestimated till now, while it turns out to be crucial when using these methods. Unfortunately, we did not identify a golden rule for choosing the parameters. However, depending on the methods used, we identified some better performing configurations: sscClust performed better with iCor as dimensionality reduction step; Seurat with the internal choice of the number of dimensions; Linnorm and SC3 with a set number of clusters (using the true number of cell populations). On the basis of our results, we suggest that users should be more aware of the different possibilities that several methods offer in terms of parameter choices and modes of usage. Moreover, we recommend them to always evaluate the robustness of their partition with respect to changes in the parameter settings. At the same time, method developers should give more attention in better documenting all the possibilities that their methods can offer also testing their robustness with respect to changes in the settings. To this purpose, the benchmark pipeline developed for this study can be easily modified to offer an environment where other/novel methods can be evaluated.We also observed that the poor performance of several methods/parameter combinations is often associated with a poor estimate of the number of clusters . Although a rigorous assessment of the number of cell populations on real data dataset could be debated, our results show that several methods tend to significantly underestimate or overestimate the number of clusters, when compared to the true  cell populations. Therefore, we can say that the estimation of the number of hidden cell populations remains challenging in the scRNAseq data analysis and we hope that novel approaches will provide less biased estimates. Moreover, by comparing the performance of each method when the true number of clusters was imputed with those when it has been estimated from the data, it is possible to quantify the impact that a more accurate estimate of the number of cell populations can have on the overall accuracy.The dataset dimension and complexity turns out to be clearly influential with respect to the running time of the methods and to the overall performance that the methods can achieve. In particular, SIMLR run time increased together with the sample size and was often the longest among other methods by several orders of magnitude . Similarly, scalability issues were observed in SC3, although to a less extent. In contrast, other methods/parameter combinations showed a good scalability in their computational time, as ascend, CIDR, monocle3, pcaReduce, RaceID3 , Seurat (with PCA dimension reduction), sincell, TSCAN or sscClust, limiting the computational time to few seconds/minutes. We want to stress that computational issues are becoming particularly important since modern technologies are now allowing to simultaneously sequence thousands or even tens of thousands of cells, thus it is expected that researchers will have to analyze much larger datasets. Hence, it will be important to provide novel methods that have good scalability properties either in terms of running time and/or computational resources required for their execution. This can be achieved either by designing methods with efficient algorithms and by better exploiting the parallel and high-performance computing in their implementation. From a technical point of view, we also observed frequent failures of some methods under particular cases. For example, SIMLR method failed on most of the simulated datasets with 5000 number of cells. We suspect that the method required large amounts of memory on the high-sample datasets than that available in our system. Other failures, like in monocle3, Linnorm and sincell on FPKM/RPKM datasets were related to the choices on the number of reduced dimensions. In fact, all of them encountered technical errors when used with tSNE dimension reduction and more than 3 number of dimensions. Additionally, Linnorm failed on raw and simulated datasets with a high percentage of dropouts (above 70% of zeros in the dataset) suggesting the low capacity of the method to handle high rates of missing data. Such points are probably less relevant and could be solved with future releases of the methods.Finally, it is also worth to mention that some of the methods, such as ascend, monocle3, SIMLR, sscClust and some combinations of Linnorm and sincell, showed variability in the clusterization despite the global setting of the seed. The fluctuations can be spotted by looking at the accuracy of methods on the identical datasets across three simulation setups  grant agreement n. 665403, EPIGEN project and ADViSE project for financial support.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


We identified the potentially high impact of redirecting investment towards household air quality monitoring and health impact assessment studies, therefore raising the public's and the government's awareness and concern about this issue and its health consequences. Such investments, due to their self-reinforcing nature, can entail high returns on investment, but are likely to give ‘worse-before-better’ results due to the time lags involved. We also discuss the usefulness of the participatory process within similar multi-stakeholder contexts. With important implications for such settings this work advances our understanding of the efficacy of high-level policy options for reducing household air pollution. It makes a case for the usefulness of participatory system dynamics for such complex, multi-stakeholder, environmental issues.58% of Nairobi's population live in informal settlements in extremely poor conditions. Household air pollution is one of the leading causes of premature death and disease in these settlements. Regulatory frameworks and government budgets for household air pollution do not exist and humanitarian organisations remain largely inattentive and inactive on this issue. The purpose of this paper is to evaluate the effectiveness of potential indoor-air related policies, as identified together with various stakeholders, in lowering household air pollution in Nairobi's slums. Applying a novel approach in this context, we used participatory system dynamics within a series of stakeholder workshops in Nairobi, to map and model the complex dynamics surrounding household air pollution and draw up possible policy options. Workshop participants included community members, local and national policy-makers, representatives from parastatals, NGOs and academics. Simulation modelling demonstrates that under business-as-usual, the current trend of slowly improving indoor air quality will soon come to a halt. If we aim to continue to substantially reduce household PM  •HAP is an important health risk in low-income urban settings, such as Nairobi's slums.•Under business-as-usual, the current trend of slowly improving indoor air quality will stop.•For it to continue, a drastic acceleration in take-up of clean cookstoves is needed.•This needs money, and one way to raise money is to invest in HIA and monitoring.•This can turn a cycle of ‘non-attention and no funds’ into one of ‘raised awareness and more resources’. It is estimated that Nairobi has over 150 slum settlements, scattered across the city. These settlements, which occupy <5% of the city's land mass, are home to an estimated 60–70% of the city population . Numerou2.5), especially in the evenings and in households burning charcoal/wood and kerosene  modelling, and health impact assessment (HIA). In the following sub-sections, we introduce the study sites as well as each method, review the structure of the SD model, and validate the model to existing data.2.1Korogocho is a slum settlement to the northeast of Nairobi city about 12 km from the city centre. It borders the Dandora dumpsite, Nairobi's official municipal dumpsite which is the final resting place for mixed waste streams from the city. This has been a source of pollution for residents of surrounding communities, with air pollution from burning garbage as well as soil and water pollution being key challenges. Viwandani is a slum settlement in the industrial zone of the city, with industries being visible sources of air and water pollution in the area. It lies about seven kilometres from the city centre and is home to a youthful, more educated and highly mobile population seeking employment in industries. In contrast Korogocho's population is older and less educated, and the majority have lived in the slum longer, compared with Viwandani's residents. Both slums face a shared challenge of poverty and exclusion especially with regards to the provision of government services such as health and education .2.2The issue of household air pollution concerns various stakeholders besides the residents, including local decision makers, government policymakers, parastatals, and non-governmental actors. Therefore, to increase the chances of research resulting in increased awareness and commitment to change among all actors involved, we organised a series of three multi-stakeholder workshops in Nairobi and used participatory system dynamics to frame the discussions. Participants included a diverse group of stakeholders, including individuals with expertise on air quality and its impacts on health, as well as those working on policy development and implementation. Attendants also included community members from both slums, academia, representatives from the local and national governments, national parastatals, national and international NGOs, and United Nations agencies.In our modelling session, we asked participants to identify the most central variables concerning indoor air quality. These variables were then gradually added to a causal loop diagram on a large whiteboard by asking participants to identify the chains of causality within the system. Following this process, we captured the overall causal structure of the system within the model and identified any emerging feedback loops, i.e. chains of causal relationships involving circular causality.Each workshop was followed by off-site refinement, formalisation and calibration of the system dynamics model, where the model was further elaborated, adding more variables and closing some of the previously open feedback loops. As part of this refinement, we sent out a questionnaire to ten members of the stakeholder group asking them to rate various policies identified during the workshops based on their relative importance to household air pollution. Subsequently, out of a total of fourteen policies, we picked seven which were considered to be the most important, to be included in the model.During the second and third workshop rounds, we sought to verify with the participants whether the model's components, inter-linkages, and resulting behaviour resonated with them and reflected their understanding of the many inter-related issues around indoor air pollution in Nairobi's slums.Participatory system dynamics modelling’ and ‘group model building’ , the life table model calculates changes in the pattern of deaths in the population over time and the corresponding impact of these on the duration of life, expressed as total life years gained or lost in the population. As seen later in the We used a life table model  to quantexposure . Based o2.4The qualitative causal diagram resulting from the participatory workshops was refined, and the health impacts were incorporated into the system dynamics model. We also wanted to understand the results of different policies on this complex model. Since such interactions are too large and complex to simulate mentally, computer simulation is the only practical way to test them . The inh2.4.1The participatory process and subsequent off-site refinement resulted in a model with >150 variables. The model structure is identical for both slums, except that it is differently parametrised for each context.household air pollution highlighted in red as our main indicator, we will explain backwards (against the direction of the arrows) along the chains of causality to investigate the key dynamics of the system as modelled here. The key drivers of the average level of household air pollution (proxied in this study by the concentration of PM2.5) in Nairobi's slums are the levels of outdoor air pollution and ventilation, as external factors, and the proportion of households using clean stoves/lighting internally. This study focuses mainly on exploring the internal factors, i.e. the prevalence of clean appliances. In line with the findings of the workshops, we assume in our model that prevalence of clean lighting is mainly driven by the electricity grid coverage and to some extent by the prices of electric lights. The prevalence of clean cook stoves, on the other hand, is driven by their prices, relative prices of clean versus “dirty” fuels, and finally the levels of public expenditure in providing subsidised appliances. The lower the prices of clean cook stoves and/or clean fuels, the higher the take-up and usage of these by residents of the informal settlements.effective expenditure to reduce household air pollution. This expenditure is modelled to be driven not only by public concern about household air pollution, but also by the extent of enforcement, political will and good governance. To capture this, we have used the World Bank's Worldwide Governance Indicators for Kenya (governance indicator (0.383 out of 1). The future target is set by the user as a policy/scenario variable (in green). While we acknowledge the fact that the situation in the two slums in this study differs from Kenya as a whole, nonetheless this data was the best proxy available for our purpose.The funds available for subsidising clean cook stoves come from the total funds spent for combatting household air pollution, or Kenya . This daPublic concern about household air pollution comes from two sources in the model: either direct monitoring of indoor air pollution levels or through awareness about the health burden associated with household air pollution, which can be estimated through HIA studies. The extent to which either monitoring or HIA are systematically carried out in the informal settlements depends on the levels of funds available for each, which are in turn determined by multiplying the total effective expenditure to reduce household air pollution by the share of this expenditure going to either of these initiatives. The extent of awareness and concern generated through each channel are also driven by the actual levels of indoor air pollution; directly so in the case of monitoring and in the case of health impact assessment, travelling through exposure to air pollution, health burden due to air pollution and attributed to household air pollutionhealth burden . Exposure itself is a consequence of either household or outdoor air pollution. Outdoor air pollution lies outside the scope of this study  would then perhaps think that the issue has to some extent been contained and no longer warrants the previously increased level of allocated funds and decide to divert those additional funds to other more pressing problems, bringing the level of expenditure back close to the initial lower level; hence the use of the label ‘balancing’.In the share of household air quality for monitoring/health impact assessment has the potential to increase the available resources the next time round. This argument makes a theoretical case for allotting a share of the available budget to monitoring and health impact assessment, a policy that we are going to test in Yet, if expenditure for monitoring or health impact assessment studies is increased, once the results of such studies are published, this new information could make the public more anxious about indoor air pollution, which in the model leads to a higher budget allocated to this issue for the next year. Therefore, an increase in the 2.4.3In SD modelling, validation depends on the model purpose, and it consists of an iterative process of building confidence in the usefulness of the model, rather than establishing its objective ‘truth’ . There eBase Run (grey curves) captures the general long-term trend in historical data (black curves) fairly well. Since the focus of this project is long term policymaking, the fact that short-term oscillations are not captured is not considered a limitation of the model for our purpose.The behaviour of the model has also been validated against time-series data from the NUHDSS database. We start the model in 2003, where NUHDSS starts, and validate it using data available until today. For instance, in Prima Gas, which made LPG more affordable for low-income groups by allowing customers to partially refill their cylinders from a mobile refill point for the amount of cash they have in hand; starting at a minimum of 50 Kenyan shillings  and clean lighting (lower plots), with the take up of clean stoves being far slower than that of clean lighting. The generally upward trend is only broken by the most recent datapoint for access to clean lighting in both communities. This fall could have been due to recent efforts to clamp down on illegal and unsafe connections especially widespread in slum areas. The fall is captured well by model simulation, as prevalence in clean lighting is tightly driven by access to electricity in our model, for which historical data is available. The idiosyncrasies of developments in ownership of clean stoves among households are however less straightforward, and the model only manages to capture the general upwards trend, mainly a result of a slow increase in funds available for the provision of clean cookstoves. In particular, the steep increase in the prevalence of clean stoves in both slums during 2014–2015 is thought to be due to a project called hillings . As thishousehold air pollution, posed a challenge to the behavioural validation of the model. This is a limitation that entails a degree of caution regarding the use of the model as the only input to policymaking. Nevertheless, while taking such limitations into account, the model still offers valuable insights for policy, as we will further report in the next sections.The scarcity of available time-series data for important variables in the model, including our central indicator 3Base Run, which is the model's projection of current trends under business-as-usual. Next, we will explore three different scenarios and consider potential implications.In this section, we will start by examining the 3.1Firstly, let us look at future developments of our main indicators according to the model's projections of current trends under business-as-usual. These projections are not merely extrapolations of current trends. Instead, variables can undergo changes in trend, as the behaviour of the model is driven by its structure, and not by its inputs. For brevity, some of the scenario analysis graphs are presented only for Korogocho. In these instances, the results for Viwandani show similar behaviour with identical implications.household air pollution (proxied by PM2.5 concentration) continues to fall slowly (in both slums), before reaching a plateau around 2030. The available data provides reasonable historical evidence on the prevalence of clean appliances since 2003, which enables us to postulate the implied behaviour of household air pollution from our model. This suggests that household air pollution has been slowly falling over recent years, indicating a gradual improvement in household air quality, which is a result of electric lights replacing polluting kerosene lights in most households. An exception to this generally improving trend occurs in the year 2014, when prevalence of electric lighting actually falls  involves a redirection of subsidies from kerosene to LPG and to supporting local manufacturers of clean stoves. It also entails drastically improving the enforcement of any existing regulations, as well as reducing existing corruption that could lead to misallocations of available funds for tackling household air pollution. In the model, these assumptions are proxied by a step-wise 50% increase in kerosene prices, 25% decrease in prices of LPG, and 50% decrease in prices of clean cook stoves and clean lighting. These changes are assumed to be implemented in three steps: the first one in 2017, and then every three years in 2020 and 2023. We also assume a gradual 50% improvement in good governance by the end of our simulation period: 2040.Scenario I (Scenario II (+ monitoring and HIA), we accompany the above changes in policy with gradually ratcheting up the share of the available budget going towards monitoring and health impact assessment, up to 15% for each by 2023. This will gradually bring down the share of the available budget going to the provision and/or subsidising of clean cook stoves to 70% by 2023. It is worth noting that the size of the available budget is not fixed and is endogenously determined under the influence of public concern about household air pollution. The effective amount of funding is also mediated by good governance.In Scenario III (+ outdoor and ventilation), we complement the above indoor-air related policies with a drastic (50%) reduction in outdoor air pollution, and, in the case of Korogocho, a drastic (50%) improvement in ventilation, to demonstrate the potential of improving household air via improvements in outdoor air. In Viwandani, our base assumption for the degree of ventilation in households, estimated based on the limited data available, is already quite high (67%). Note that improvement in ventilation in the absence of improvements in outdoor air quality, can lead to a worsening of indoor air quality, as the outdoor is often more polluted than the indoor in Nairobi's slums.Finally, in our most comprehensive scenario, 3.2.2Base Run. In household air pollution under various assumptions is shown. The graph shows how each scenario performs better than the previous one, thanks to a more comprehensive package of policies implemented. However, the scale of improvements resulting from adding each set of policies is different from other ones.We will now compare the results of these scenarios against each other and against the Base Run), by 2040. This improvement in household air pollution results in a comparable improvement in life years lost to air pollution, as seen in enforcement (Scenario I) does not result in any substantial improvements by the end of our simulation period. If, however, we complement this by investing in monitoring and health impact assessment (Scenario II), we can hope for a much more significant betterment of indoor air quality that is likely to result in roughly proportionate improvements in life years lost to air pollution (Scenario III).ollution . The besgood governance, even when combined, do relatively little to improve household air pollution. This is because currently the amount of funds allocated to combatting household air pollution is so low that incremental increases or reallocations fall short of achieving any substantial improvements over business-as-usual. This means that to see more visible impact, we require available funds to grow by orders of magnitude, which implies that public concern over household air pollution needs to be raised exponentially. This is precisely what investing in monitoring and health impact assessment can achieve thanks to the self-reinforcing nature of the feedback loops involved, as seen in Therefore, as projected by the model, policies like redirecting fuel subsidies to cleaner fuels, reducing stove prices, and strengthening Scenario II), we would still end up far above the WHO guideline for acceptable exposure to annual average PM2.5  to household air quality monitoring and health impact assessment studies, to ‘close the loop’ and bring the issue of household air quality higher up on the list of public/government priorities. Such investments, due to the self-reinforcing nature of the dynamics involved, can entail high return on investment, as the policymaker would be able to leverage the results of such studies to enlarge ‘the size of the pie’ of available money and resources , as well as past expenditures on related policies, among other variables. It is not that such data externally drives the model, rather that such data would have been useful for a more in-depth comparison of simulated behaviour against real-world observations.number of households owning clean lighting was very good, the main driver of this indicator in our model is electricity coverage, which is taken as exogenous real-world data fed into the model, with an assumption of full electricity coverage of slums by 2040. In this respect, all our scenarios are equal. The scenarios differed substantially, however, in the speed of take-up of clean cook stoves, which makes that a key component of the model, far more influential than clean lighting in determining future differences between scenarios. In terms of model validation, however, our simulation only manages to capture the general trend for this variable, and therefore the fit could not be described as an exceptionally good fit (Importantly, while our replication of historical data for the Policies for decreasing household air pollution are certainly not limited to those investigated in this paper. Behavioural change interventions, for instance, could succeed in moving households from cooking inside their rooms to cooking outdoors. Such behavioural policies are not considered within the scope of this study.outdoor air pollution, which was included as a scenario variable whose future value is set by the model user. For a more realistic treatment of the problem of household air pollution, it would be useful to model outdoor air pollution as an endogenous variable that is a composite of pollution that originates indoors and diffuses locally, as well as industrial, waste and transport pollution. Endogenising outdoor air pollution presents a potentially fruitful opportunity for further research in this line.Another limitation of this study is the exogenous treatment of the level of When we asked workshop participants to articulate their hopes for Nairobi slums, only one participant mentioned the reduction of household air pollution. Other issues such as land ownership, services and waste management were much higher on people's agendas. Even though the air pollution-related health burden is known to be very large, to our knowledge there are currently no programmes focusing on air pollution from a health point of view, neither at national nor at county level. This lack of attention  presentsThis project and modelling work was influenced by the limited attention that household air pollution has received so far. Because of this inattention, very little data on household air pollution has been collected to date. Increased investments in researching household air pollution would lead to more abundant scientific evidence, which could be used to produce more useful and reliable policy recommendations.