This study builds on the economics and organization literatures to explore whether and how institutions and organizational structure complement or substitute each other to create specific spaces of alignment where specific individual actors’ motivations co-exist. Focusing on university-industry collaborations, the study examines whether and how different axes of alignment of university and industry motivations are integrated in projects with specific technological objectives and organizational structures, benefitting from the presence of specific institutions designed to facilitate collaboration. Empirically, the study relies on in-depth data on 30 university-industry collaborations in the Netherlands, and provides preliminary evidence that the technological objective and organizational structure of collaboration are malleable variables allowing the integration of both partners’ objectives and expectations. Different institutional incentives for university-industry collaboration favor specific axes of alignment of motivations and certain types of collaborative projects’ design. Hence, our exploratory results suggest that specific organizational and technological structures tend to prevail in the presence of specific institutions. It is well established in the literature that institutions , and which motivations create particular trade-offs . The second research question refers to how particular sets of motivation alignments are associated with the nature, design/organizational structure and institutional setups of collaborative university-industry projects. Empirically, the present study relies on in-depth data on 30 university-industry collaborative projects in the Netherlands. Our exploratory results suggest that specific organizational and technological formats tend to prevail in the presence of different institutions.The paper is organized as follow. Section Different streams of literature have examined incentives and motivations, using diverse research methods and addressing different research questions. Despite the wide amount of work on incentives and motivations, no overarching framework has been proposed to interpret and to shed light on how different axes of motivation alignment relate to the organizational structures in which these motivations are exercised, and to the presence or absence of institutions drawn to support motivational alignment. This study is set up as an attempt to achieve this ambitious objective. Next, we propose a conceptual framework of the motivational alignment process in collaborative projects with specific technological objectives and organizational design. Our framework builds on and can contribute to the economics, organization and institutional approaches.In economics, incentives or motivations are at the center of the analysis as they underlie decisions, behavior and performance. The economics of innovation literature has focused on examining the impact of different innovative behavior and choices on performance  within the university and /or private firm, including part-time researchers who work in a private firm and in a university, TTOs, and previously patented knowledge owned by the university. It also includes facilitators, beyond the organizations that are involved in the collaboration. These institutional facilitators emerge from a set of policy instruments used to stimulate university-industry interaction, such as specific subsidy programs for collaborative projects. In the Dutch case, they include programs offered by the STW foundation, which is part of the Dutch research council, aimed specifically at university-industry interaction.The set of aligned motivations for collaboration among university researchers and private firms depends on several other factors. Alignment stems first from the different (not necessarily conflicting) motivations of the parties in the collaboration, i.e., industry and university. Although in reality these sets of motivations are influenced by such factors as the set of technological opportunities, our case study database does not allow detailed variation of these factors; thus, we consider them as belonging to the exogenous motivations.By their nature, some of these motivations on either side of the collaboration can be expected to complement each other, e.g., university searching for industrial applications with firm’s product development objectives. This complementarity is expected to be a dominant explanatory factor in the set of aligned motivations, but there may also be trade-offs and conflicts among motivations preventing their co-existence. An example of a trade-off in the alignment process might be the conflict between the university objective of opening up new research avenues and the firm’s product development. Opening up new research avenues would seem aligned to basic research, while product development would involve applied research based on the outcomes from basic research. However, the set of aligned motivations depends also on the institutional facilitators. Next, we discuss the main university and industry motivations for collaboration, as well as the incentives that specific institutional facilitators can create for motivational alignment.Firms collaborate with universities mostly to access and develop interdisciplinary scientific capabilities to solve complex industry problems and to support product development, but also to access public sponsorship. Firms may also collaborate with universities to conduct exploratory, non-targeted research to generate ideas, build technological options and search for new products, technologies and markets, and to get access to skilled labor, especially qualified engineers  reflects an agreed division of labor, its respective organization and coordination of the knowledge production and distribution, and agreed rules for accessing resources during development, and appropriating and diffusing project outcomes  or shorter (Master) period of time, which is supervised by a professor who provides informal advice on the developments in the firm related to the project. Being the main forms of implementation of university-industry collaboration, these forms may permit a variety of alignment spaces  are expected to be more frequent in projects where the university’s objective is to test the applicability of its research results; thus, when the knowledge objectives relate to technologies to substitutes for existing ones. Market problems, such as lack of customers, change to the industry partner’s marketing strategy, or competition (a ‘technology race’) may in turn be more frequent in projects that focus on developing near-to-market knowledge. Conflicts emerging from different attitudes to knowledge development and sharing, appropriability and applicability are expected to be more likely in projects where the firm is motivated by objectives of product development exploiting university research and the university researchers are collaborating to find funding or on the expectation of collaboration and funding opportunities in the future.Although the conceptual framework suggests a number of causal linkages, it is not the aim of our empirical analysis to test these causal relations. This is left to future research. Our objective here is to use the framework to guide our thinking about how the various aspects of the collaboration and motivational alignment are related to each other, and to interpret the relationships observed in the cases in our database. In this manner, we hope to advance existing knowledge on the complementarity between organization and institutions in supporting certain axes of motivational alignment.Analysis and development of unidentified relationships and concepts, such as our objective of examining the association among motivational alignment, organization and institutions, require a research design that provides in-depth information on multiple cases  commercialize it.We used a mix of strategies to identify the cases. We searched national electronic libraries for PhD theses completed in the previous five years that acknowledged support from industry and research grants from national research councils. We interviewed the chairs of some research departments in the faculties of mechanical engineering, biotechnology, chemistry, applied physics and electrical engineering, in two technical universities in the Netherlands (Eindhoven University of Technology and Delft University of Technology), and the directors of these universities’ TTOs. We identified professors with large numbers of industrial patents choosing two for our sample.Multiple in-depth case research design requires a different logic to sampling; it is important to select a variety of examples cases of the phenomenon of interest to allow comparison and contrast (Eisenhardt Data collection was done in a retrospective way and involved a combination of primary data (interviews) and secondary data  (Van de Ven and Poole Primary data on each collaboration was collected in between two (only in one case) and six semi-structured interviews with researchers in the firms and universities involved in the projects (three is the median).We converted the information derived from the responses to the protocol questions into binary variables , and four other dichotomous variables refer to the industry motivations to collaborate with university .After a dialectic process between the early codified data and the literature, we retained four university and four industry motivations.Substitute technology and Commercialization, characterize the originality and the development stage of the technological objectives of the project, dimensions commonly used to characterize knowledge development activities and projects .The second step in our analysis, in Section In all 30 collaborations, the university researchers were motivated to collaborate to engage in high-quality scientific research and develop knowledge. Specific reasons included (in order of frequency): (i) obtaining insights into the industrial applicability of previous research; (ii) maintaining contact with industry; (iii) accessing additional funding; and (iv) increasing opportunities for future (collaborative) research. The motivations for collaboration were not exclusive (the sum of the frequencies of these motivations adds to more than 30); the motivation to maintain collaboration contacts contrasts with the other three motivations.We identified four main motivations for firms to propose or engage in collaboration with universities. These are ranked in order of importance: (i) support for product development; (ii) access to public research funding; (iii) solutions to technological problems; and (iv) finding research opportunities. As with university motivations, industry motivations for collaboration are not exclusive. The motivation of accessing a research opportunity contrasts with motivations related to solution to technological problems and support for product development projects.We address our first research question by exploring how alignment of university and industry motivations for collaboration relates to specific project design in terms of its technological objective and organizational structure, and to specific institutional facilitators. We use factor analysis to identify the axes of alignment of university-industry motivations and their institutional and organizational context. In addition to the variables for university and industry researchers’ motivations for collaboration, we include the eleven variables described in Section , and substitute innovations are often the outcome; spin-offs are relatively frequent. Part-time researchers, research sponsoring and STW are infrequent institutional facilitators for the axis academic goal, but dominate on the axis of advance industry research agenda.The first factor refers to the two extremes of the axis of alignment of motivations: applicability of university research , and accessing public research funding (industry) and maintaining collaborative contacts (universities) (the latter two strongly negative). The opposite signs of the loadings of these variables suggest a conflict between these sets of objectives, thus creating the academic goals vs. advance industry research agenda trade off axis. Technical problems are a common barrier finance knowledge development end, we find a rather formal mode of knowledge transfer . Projects are often in the form of PhD (but not MSc) research, are sponsored by public research grants, and can lead to spin-offs. The access technical support end of the axis involves Masters projects and informal modes of interaction.The second factor also maps two opposite motivational alignment extremes. In this case, the primary conflict is, on the one hand, between motivations in the business domain, i.e., research opportunities recognized by firms, and university funding opportunities (strongly positive loading), and, on the other hand, industry product development and solution to technological problems and maintaining collaborative contacts (universities) (negative loadings). This second factor seems to point to trade off finance knowledge development (i.e. exploring research opportunities) vs. access technical support . At the The third factor refers to IPRs on university research as an institutional facilitator, and licensing and/or consultancy contracts. We label this the IPR axis. The main motivational factor associated with this axis is universities exploring future research opportunities. Universities use IPR to maintain access to a research field and to resources. Cultural differences are important here. There is minor industry product development motivation on this axis.The fourth factor characterized by STW as an institutional facilitator, high levels of commercialization, and PhD research projects and spin-offs (no very positive or very negative loadings of any industry and university motivation). This factor also loads high and positive on previously existing IPRs and licensing/consultancy, which is in line with the strong emphasis of STW on IPRs. We label this close to market/STW axis, referring to the fact that projects supported by STW are aimed at commercialization and user involvement.The last factor shows industry and university motivations aligned in the same direction . The two motivations are research opportunities (industry) and applicability (university). These are applied R&D projects, which are characterized by an absence of market-related barriers. We label this the applied research axis.These principal components can be likened to the dishes on a menu of actual collaborative projects, and are used to address our third research question (about the typology of collaborative projects). None of the 30 collaborative projects is described adequately by only one of the five factors, which should be considered analytical tools used to identify the individual processes involved rather than types of cases. To visualize and understand the differences in objectives, motivations and design of university-industry projects that characterize our collaborations, we propose the following typology. K-means cluster analysis is our main analytical tool, and uses data from the 24 variables used in the factor analysis. Table Sponsored Projects; all 11 cases in this cluster received third party research funding. This cluster includes six of the eight cases of STW funding. In relation to the motivational space represented in the PCA, this cluster is clearly focused on advance industry research agenda and is close to market/STW. As in Fig. close to market/STW are positive, but strongly negative for academic goal. The main motivation for university researchers to participate in projects in this cluster is to maintain collaborative contacts (eight out 11 cases). Ten of the 11 cases of industry motivation to participate in accessing research funding are in this cluster. Also, in seven of the cases in this cluster, industry is motivated for product development. Thus, this cluster seems to capture cases where funding and maintain collaborative contacts is the motivation for industry and university participation, respectively. These projects tend to be implemented in the form of PhD research, and their outcomes are likely to be commercialized.We refer to the first cluster Cluster 1), which contains the largest number of cases as , which cExplorative projects. None of the firm’s motivations in this cluster are strong, but the results of collaborative projects tend to be commercialized. In terms of the principal component axes, as plotted in Fig. academic goals (rather than advance industry esearch agenda), applied R&D and close to market/STW. This cluster is characterized also by high frequency of technological problems (7 of the 9 cases). These projects tend to focus on technologies that substitute for existing ones, to be implemented by PhD students, involve high frequency and informality of communication between the partners, and outcomes are likely to be commercialized.The nine projects in the second largest cluster (Cluster 3) are characterized by university researchers being motivated by searching for applications for their basic research ; we call it access technical support, academic goal and applied R&D principal components. These projects are usually implemented by Masters students and focus on researching and developing proof of concepts and prototypes of substitutes to existing technologies, in an environment of intense formal and informal university-industry interaction. We label this cluster Industrial projects because they are aimed at solving technological problems in industry product development projects.The clusters 2 and 4 are smaller (cluster 1 and 3 account for 20 out of the 30 cases). However, since our sample of cases does not pretend to be representative, cluster size is not an indication of importance. Cluster 2 has six cases, and includes a relatively large variety of university motivations. The most observed industry motivations are accessing support for product development and developing solutions for technological problems. In terms of the principal component axes, as in Fig. IPR, finance knowledge development and academic goal axes as in Fig. Contracted. It emphasizes patented university-knowledge. In all the cases, university motivations are in finding funds to undertake research and increasing networks of research and industry partners. Industry motivations are split into getting support product development and finding new research directions and opportunities. This cluster is characterized by a focus on technologies to substitute for existing ones, high frequency of technological and cultural problems and outcomes that can be commercialized. These projects tend to be implemented under licensing agreements and as PhD research, involving frequent but formal communication between the partners.Cluster 4 (four cases) is based on Sponsored and Explorative, reveal a large disciplinary diversity, especially the latter. The two least populated seem biased to specific disciplines: Contracted towards biology and chemical engineering and Industrial to applied physics and electrical/electronic engineering. Concerning prior collaborative experience, Sponsored projects represent to a large extent continuation projects. Only in two out of the 11 cases, the industrial partner had no prior experience with the specific researchers, but in these two cases, there was prior experience with the same university department. These projects have mostly been initiated to explore prior collaborative results or by part-time professors. On the other end, Contracted projects refer mainly to new collaborations, as in none of the four cases was there any type of prior experience between the university and the industry partners. Two cases were initiated by the industry and the other two by the university. In the Explorative and Industrial projects groups, only 2 and 1 projects, respectively, represent new collaborations. These two types of projects, especially in the Explorative group, present higher diversity in the role of the partners in initiating the collaboration. In the Industrial cluster, projects were somehow more often initiated by the industry and researcher mobility and in the Explorative cluster more often by university researchers, part-time professors, research mobility, and prior collaborations.In addition, the four types of projects identified seem to differ in terms of disciplinary diversity, the prior collaborative experience between the firm and the research team or the university department, and role of the partners in initiating the collaboration. The two most populated clusters, Here, we briefly describe four projects that can be considered typical Sponsored, Explorative, Industrial and Contracted university-industry collaborations.Sponsored Projects cluster is the collaborative project that was undertaken as a follow-up development of a prior collaboration between a Dutch technical university and a manufacturer of transmission systems for automobiles. The collaborative project was sponsored by a government program to support environmentally-friendly research projects – especially through university-industry partnerships. The project involved three PhD researchers whose assigned task was to deliver a ‘proof-of-concept’. The project led to four patent applications, and to the creation of a spin-off company since the original partner company was not interested in diversifying and commercializing the invention, which still had to be developed beyond the embryonic stage.One project representative of the Explorative projects cluster is the collaboration between the biomechanical Engineering department of a Dutch technical university, a Dutch academic hospital and a foreign firm. Two university researchers, one working also at the hospital, developed a theory for a novel method to measure flow in the coronary artery, which needed to be tested before being used clinically. Hence, the academics approached the private firm that had earlier developed relevant sensors. The partnership benefited from financial support from STW and from the private firm, which also provided equipment for the research performed at the university. The project was implemented by two PhD students, who conducted most of the research at the university, but there was intensive and quite informal interaction with the other two partners. The collaboration also allowed a firm researcher briefly to visit the university, and a university master student to do a Master thesis at the firm. The project resulted in an improved method based on the theory developed at the university, which came to be used in the partner hospital, as well as in a Belgian hospital.One project representative of the Industrial projects cluster is the collaboration between a Dutch technical university and a governmental institute. An engineer from the governmental research institute approached some of his contacts at the university to get help on developing a new type of mirror, which was seen as a request from industry and experts in the technology area. The university researcher was interested on the knowledge development idea that the governmental research institute needed and a Master student was found to undertake the feasibility study. The master student performed most of the research at the governmental institute, and succeeded in developing a first concept of an extendible deformable mirror, which was patented. Given this success, the team decided to submit a fund request for a follow-up partnership, this time involving also another technical university and industrial firm.A project representative of the Contracted project cluster is exemplified by the collaboration between a foreign firm that learned about the university’s research at a conference and a Dutch technical university. The firm proposed a collaborative project (+2 years) to explore industrial applications of this work and tap into the university’s knowledge. The project was to be funded by the firm and a Dutch research institute. The firm patented the preliminary university results. Throughout the project, the university provided continuous feedback on research results and provided week-long training for some of firms’ researchers. The firm gave no feedback on its in-house testing and application after the first year. By the end of the first year of the project, other patents had been published, but the firm cancelled the collaborative project. The senior university researcher later learned that the firm had created a new R&D facility that allowed it to reproduce the university’s scientific research and proofs of concept and to proceed with development of a new product. Following acrimonious discussion in the university, the researchers found alternative funding sources and were allowed to continue their research in a four-year project, which resulted in publications and prestige for the university.A stereotypical project of the Results suggest that university and industry motivations are aligned in particular combinations. Some motivational spaces are in conflict, i.e., they describe how motivations of one kind (in university/industry) do not go well with motivations of another kind (in industry/university) and create trade-offs during the alignment process. We identified two trade-offs axes. One trade-off we labelled academic goals vs. advance industry research agenda. It is created by the juxtaposition of the university researchers’ interest in finding applications for their basic research results, which is in conflict and is difficult to be reconciled, with an industry motivation for accessing public sponsoring. If firms are motivated by accessing public sponsoring, university researchers seem more willing to join the project in order to maintain collaborative networks. The second trade-off concerns finance knowledge development vs. access technical support and results mainly from conflictual interests and goals of the industry. These results are in line with prior literature that has stressed the exploration-exploitation and the short-run versus long run trade-off in knowledge development activities, and also specifically between the industry and academic knowledge incentives structure March .Three other motivational alignment spaces were identified. In one instance, the field of applied R&D, firm and university motivations were quite in parallel: university researchers were motivated by applicability of research, and firms were motivated by research opportunities in newly discovered technologies (with no immediate target market). This alignment space seems to match with the so-called Pasteur quadrant Stokes . Two othSponsored projects, motivations associated with maintenance of research contacts (university) and access to research funds and new product development (industry) tend to be aligned in projects characterized by low levels of communication and formal organizational formats, often focusing on technologies to complement existing ones. Sponsored projects are strongly influenced by policy measures to fund collaborative research (in particular from the research funding agency STW), and, in several cases, they involve part-time professors.Four types of projects —Sponsored, Explorative, Industrial and Contracted— recombine differently these motivational alignment spaces, observe specific organizational structure and the presence/absence of different institutional facilitators. In Contracted projects. These projects are instead mainly characterized by being facilitated by existing patented university inventions, by university involvement motivated to open up (long-run) future research opportunities, and by industry involvement motivated by exploring new avenues for product development. Contracted projects tend to focus on developing technologies to substitute for existing ones and by high frequency of technological and cultural problems.Formal organizational and communication formats and commercialization of products are also the characteristic of Explorative and Industrial projects rely much less on institutional facilitators. Some projects also benefit from policy sponsored initiatives, but they are less dependent on external incentives or funding. These projects have a much broader motivational basis in industry and university, and combine university-driven and business-driven motivations. When the university motivation is research applicability, projects tend to be characterized by intensive communication and focus on technologies to substitute for existing ones . In this case, the degree of formalization of the communication and interaction between the partners, and the form of project implementation seem to vary according to the industry motivations to collaborate, and to the institutions that support the alignment of these motivations. PhD and Master students have been involved in most of the projects examined.This evidence suggests that institutional facilitators are associated with collaborations that rely on formal implementation forms and on weak and relatively formal communication for problem-solving and knowledge transfer. This may be caused by the fact that external parties are involved in financing and managing the collaboration, the work packages of the partners involved are relatively sequential/ independent or the partners have few incentives to communicate frequently and informally. It might also be that institutional facilitators may be encouraging university-industry motivational alignment distant from Pasteur or Edison quadrants of scientific research proposed by Stokes .7Explorative projects that received public funding differed from the Sponsored projects. Despite benefitting from public funding, the parts managed to create an organizational and technological structure permitting these projects to address more exploratory research objectives and to achieve close to market outcomes. Industrial projects also permit the alignment of university motivation to maintain collaborative contacts and firms’ motivation of product development and/or solve technological problems. Still the organizational and technological structure of Industrial projects differ greatly from that of Sponsored in terms of length, technological scope, organizational structure and resources used (university and firm resources vs. public research funding).Overall, our evidence suggests also that the presence of specific organizational formats tends to prevail in the presence of specific institutions  and two trade-off axes . Four different types of projects —This study contributes to the three streams of the literature on which it builds. To organizational studies, which show that certain organizational structures are associated with certain outcomes and consequently to certain knowledge objectives  or between specific types of projects and performance.Sponsored) are characterized by a peculiar pattern of motivation biased towards advancing industry research agenda, rather than being driven by the research agendas of the university researchers. While we cannot establish how motivations influence the outcome of collaborative projects, our results suggest that university researchers’ “intrinsic” motivations  provide weak motivation in the sponsored projects group. Future research might provide more evidence on whether this may affect project outcomes.Although exploratory, our analysis has some implications for policy. Projects that depend heavily on external funding  projects with high additionality  and avoid a strong bias towards industry projects related to development of new complex products. If public research funding is encouraging university-industry alignment of distant from the academic goal, in the long run this may be reduce the value of collaboration, hence being detrimental to both parties Lam .Contracted) that focus on developing university-patented knowledge seem more likely to be the basis for new collaborative contacts and the development of substitute technologies that usually succeed in developing advanced prototypes and lead to plans for product commercialization. At the same time, these largely industry-financed projects tend to experience managerial problems and several coordination conflicts resulting from the different research culture of the parts, which waste time that could be spent on research or education. Our exploratory results are in line with prior literature that showed that licensing activities may not have a negative effect on research productivity (publications and citations) of academics, but it can decrease ability to get government grants (Thursbyn and Thursby Furthermore, the analysis provides some insights into the role of IPRs in knowledge transfer, an important policy topic. Projects (Finally, our results suggest that policy should recognize the power of collaborative university-industry Masters and PhD theses tools to promote university-industry interaction. In particular, Master theses facilitate the motivational alignment for developing pre-feasibility studies for industry application. However, as some authors have stressed, graduate student traffic is often exploited by academics to strengthen collaboration with firms, and may have consequences for the culture of science and research (Slaughter et al. Our study has some limitations that open up avenues for future research. First, in order to extract detailed information on collaborations, our analysis used unique project level data and, necessarily, relied on a small sample of observations. This restricts the type of statistical analysis possible and makes our results exploratory. Further research is needed on larger samples using additional methods of enquiry and analysis. A larger sample would also permit us to ask whether other specific characteristics of industry and university partners play a role in explaining the space of motivational alignment achieved and the characteristics of the project.Second, we relied on cases of university-industry collaboration in the Netherlands. Some characteristics of the motivational spaces and their relationships with specific collaboration formats might be specific to the Dutch institutional environment in relation to university regulations, academic career progression, science policies, and the functions of research councils. Future research could examine whether these specific findings can be generalized to other countries, given cross-country differences in institutions and careers.


Adolescents in less developed countries such as Zambia often face multi-faceted challenges for achieving successful transitions through adolescence to early adulthood. The literature has noted the need to introduce interventions during this period, particularly for adolescent girls, with the perspective that such investments have significant economic, social and health returns to society. The Adolescent Girls Empowerment Programme (AGEP) was an intervention designed as a catalyst for change for adolescent girls through themselves, to their family and community. AGEP was a multi-sectoral intervention targeting over 10,000 vulnerable adolescent girls ages 10–19 in rural and urban areas, in four of the ten provinces of Zambia. At the core of AGEP were mentor-led, weekly girls’ group meetings of 20 to 30 adolescent girls participating over two years. Three curricula ― sexual and reproductive health and lifeskills, financial literacy, and nutrition ― guided the meetings. An engaging and participatory pedagogical approach was used. Two additional program components, a health voucher and a bank account, were offered to some girls to provide direct mechanisms to improve access to health and financial services. Embedded within AGEP was a rigorous multi-arm randomised cluster trial with randomization to different combinations of programme arms. The study was powered to assess the impact across a set of key longer-term outcomes, including early marriage and first birth, contraceptive use, educational attainment and acquisition of HIV and HSV-2. Baseline behavioural surveys and biological specimen collection were initiated in 2013. Impact was evaluated immediately after the program ended in 2015 and will be evaluated again after two additional years of follow-up in 2017. The primary analysis is intent-to-treat. Qualitative data are being collected in 2013, 2015 and 2017 to inform the programme implementation and the quantitative findings. An economic evaluation will evaluate the incremental cost-effectiveness of each component of the intervention.The AGEP program and embedded evaluation will provide detailed information regarding interventions for adolescent girls in developing country settings. It will provide a rich information and data source on adolescent girls and its related findings will inform policy-makers, health professionals, donors and other stakeholders.ISRCTN29322231. March 04 2016; retrospectively registered.The online version of this article (doi:10.1186/s12889-017-4280-1) contains supplementary material, which is available to authorized users. This paper reviews the design and methods of a multi-arm, randomized cluster evaluation of the Adolescent Girls Empowerment Programme (AGEP) in Zambia. The programme was designed as a multi-sectoral intervention targeted to vulnerable, rural and urban adolescent girls ages 10–19. Embedded in the design of the programme was a rigorous, randomized evaluation that recruited a cohort of eligible adolescent girls invited to participate in the programme and girls residing in control areas. The evaluation sample was interviewed at baseline in late 2013 and early 2014 and is being tracked annually through 2017, with mid-term results collected in 2015 immediately after the end of the intervention and endline results in 2017, two years post programme termination [Adolescent girls in less developed countries face a variety of risks and challenges in achieving positive and successful transitions to adulthood . In ZambThe literature indicates that women and adolescent girls in many less developed settings are lacking the needed assets and capacities required to make more positive and healthier transitions that would facilitate their breaking out of persistent poverty and closing the economic and livelihoods gap between men and women. In such settings, women and girls’ opportunities are inhibited by traditional practices, adverse gender norms and roles, and weak institutions and laws , 5. For To address these disadvantages, the Population Council, working in collaboration with the YWCA, Making Cents International, the National Savings and Credit Bank of Zambia (Natsave) and the Zambian Ministry of Health designed AGEP, a multi-sectoral and girl-centered intervention. The theory of change underlying AGEP Fig.  proposedThe goal of AGEP was to reach a minimum of 10,000 adolescent girls ages 10–19 across ten sites in four of the ten provinces of Zambia; half of the sites were urban areas and half rural areas. Provinces were purposefully selected in collaboration with the donor, weighing feasibility, resources for the programme and evaluation, and the objective of representing vulnerable adolescent girls. As the programme was specifically designed to reach the most-vulnerable adolescent girls, high-density urban housing compounds were targeted, while a more general sample of rural residential communities were targeted.The AGEP intervention operated for a period of two years in each site from late 2013 and early 2014 through late 2015 and early 2016. Implementation of the programme was initiated sequentially by site, after the baseline evaluation survey was completed in each site. The core component of AGEP was group meetings in which 20 to 30 girls met at a local community space for one to two hours per week, most often on weekends. A community-based model was selected as school-based sexual reproductive health programs have been known to have limited impact because teachers find it hard to break out of their typical teaching styles and programs are adversely affected by an already low quality of schooling [All the girls who were assigned to the intervention were invited to participate in AGEP girls’ groups. Two additional programme components were randomly assigned to some of the girls groups. The additional components were: 1) the provision of a health voucher and, 2) an offer to open a bank savings account. Eligibility for these add-on components was determined by whether or not the girl group was located in a cluster that was randomized to receive the additional components of the intervention. The assignment of girls to the programme component is discussed further in the evaluation design section below.Girls who attended AGEP in randomly-selected areas to receive a health voucher targeted to improve access to ten general health and sexual and reproductive health services at partner public and private health providers Table . The desIn addition to the sexual and reproductive, nutritional and general health knowledge building during the weekly girls’ groups, two additional programme elements supported the AGEP health voucher component: 1) the training of health facility staff in providing adolescent girl-friendly health services, and 2) a results-based financing of service provision at health facilities . All parThe results-based financing component of the health voucher also provided a basis from which to reimburse the health facility and provider for service use among adolescent girls. At private and non-governmental organization facilities, AGEP reimbursements were based on negotiated fees for each service provided. At public facilities, as health services are generally free, the payments to the facilities for each service, whose rates were agreed on with MoH and were uniform for all public facilities, were distributed by fixed percentages: to the providers (50%), to the health facilities for supplies (25%), for per-diems to the district health staff to assist in monitoring service provision (20%), and to the District Health Office for overall management of the programme (5%). Reimbursements were a function of the total number of girls with vouchers obtaining services and the number of services that they accessed during the period that the vouchers were usable. It should be noted that unlike other programme elements, due to delays in finalizing the arrangements, the health voucher started after one year of the programme and continued one year after the AGEP girls groups ended.While the girls groups provided capacity building in money management, budgeting and savings, the provision of a bank account was designed to provide a mechanism for knowledge and skills to be operationalized in practice. The bank account, provided to girls who would not otherwise typically have access to financial services, was hypothesized to reinforce girls’ money management skills, promote economic asset building, grow a culture of savings, facilitate economic independence and provide assets in cases of emergencies or other basic needs. Savings accounts, as opposed to micro-credit, were used as studies had shown in other settings limited results for such programs  and thatThe bank account component was implemented in partnership with the National Savings and Credit Bank of Zambia (NatSave). The AGEP “Girls Dream” savings account provided adolescent girls a formal place to store money. The girls who were eligible for the savings account were provided an orientation in the girls’ group sessions and a field trip was organized to the nearest NatSave branch to provide additional information and to open accounts. Girls who were under the age of 18 years were required to have co-signatories present at the account opening. The bank account was specifically tailored to the financial needs of the girls, with a very low minimum opening balance of KW 2.5 (US $0.50) and the ability to deposit or withdraw funds with no fees. The girls who opened accounts could deposit on their own, although for girls under age 18 years the accounts required their co-signatories to withdraw funds. While no direct financial resources in the form of grants or loans were provided to AGEP girls, market research based on focus group discussions conducted during the pilot period suggested that adolescent girls had a variety of sources of income that may be used to build their savings, including cash from parents, paid piecemeal work, agricultural production, selling goods made and transactional sex .As noted previously, AGEP was designed to serve a minimum of 10,000 vulnerable adolescent girls in Zambia aged 10 to 19 in ten sites located within four of the ten provinces of Zambia. The study provinces and the number of sites per province were selected purposefully through discussions with the donor regarding representation of the target population and consideration of the feasibility of operating AGEP while also conducting a randomized evaluation. Study sites within provinces were selected randomly from a sampling frame of potential sites, anchored to proximally-located health and banking facilities. The sampling frames of potential sites for urban and rural areas were developed separately. For urban areas, a list of high-density residential compounds was created, and sites selected randomly from the list, while in rural areas sites were randomly selected among areas that met the following conditions: that they were located within 15 km from a health facility and within 65 km of a NatSave bank branch.AGEP was also designed to draw participants who expressed multiple levels of vulnerability through their residential location and the socio-economic characteristics of their households. To avoid over-representing girls in AGEP who have already experienced the outcomes that were to be prevented by the programme , these indicators were not directly used to identify vulnerability. Instead, an indicator of vulnerability was created using whether girls were behind in school for their age as a proxy, which also permits characterization of the vulnerability of girls younger than those who already have been married, pregnant or dropped out of school. Conceptually, the measure captures adolescents who are at risk of adverse outcomes, given the primacy of education in a child’s life at these ages. Early in the school-going process, many children fall behind in school due to late entry, repetition of grades, and temporary withdrawal from school; all of these events are a reflection of some degree of personal and household vulnerability. The information used to develop the measure of vulnerability was collected along with other household socio-economic characteristics via a household survey in all study clusters, which was subsequently used as a sampling frame to determine eligible girls for participation in the programme with the most vulnerable girls selected from the list to receive invitations to participate.The objective of the AGEP evaluation was to implement a rigorous randomized design to more confidently attribute the changes in girls’ lives to their participation in AGEP. It was expected that the programme would have a positive impact on the longer-term demographic, reproductive and health outcomes of participating adolescents. It was also expected that the impact of the programme on these longer-term outcomes would be mediated through a set of individual assets or skills that the girls acquire through the programme. Given that the AGEP evaluation has multiple components that may lead to improved outcomes, the study was designed and statistically powered to evaluate the impact of each of these components on adolescent girl outcomes, independently for rural and urban areas.The AGEP evaluation was designed as a multi-arm randomised cluster trial where clusters within the ten study sites were randomised to receive different combinations (arms) of AGEP. The experimental and control arms of the study are displayed in Table A cluster in the AGEP evaluation was delineated by Census Supervisory Areas (CSAs) that were compiled by the Zambia Central Statistical Office (CSO) for the 2010 national census. CSAs contained approximately 750 households in urban areas and 300 households in rural areas, although the average number of households varies considerably as population densities vary geographically. CSAs in urban areas were spatially relatively small, perhaps a few hundred metres long and wide, while CSAs in rural areas encompassed multiple kilometres. The assignment of CSAs to the experimental and control arms was conducted through a random selection process at a public lottery. The public lottery was conducted to maximize the transparency and community acceptance of AGEP. Local political and community leaders were invited to participate in the lottery, conducted at a centrally-located public facility. The lottery was conducted via a two-step selection process in which a CSA was first randomly selected for participation among all CSAs in the site and then an AGEP arm assigned through a second random selection process.The overall objective of AGEP was to make a meaningful change in the traditionally observed trajectories of vulnerable adolescent girls in Zambia as they pertained to early family formation, low schooling attainment and poor sexual and reproductive health. The process of change was posited to advance in phases, with the more immediate changes in the girls’ assets and empowerment deriving from their participation in AGEP, translating into positive behavioural change and, ultimately, to longer-term change over the course of four years in which empowered adolescents would realize more positive transitional outcomes. As the concept of empowerment, mediating and longer-term outcomes are potentially broad and multifaceted, a set of representative indicators was used to epitomise their operationalization in the evaluation as noted in Table The study will test several hypothesis to assess the impact of AGEP and its components. The hypothesis tests reflect the causal pathways as illustrated in the theory of change Fig. . The impAs noted above, all three intervention arms included weekly meetings in which girls met with a mentor and each other to engage with a curriculum topic for the week, generally discuss their experiences and to socialize with each other. Given the randomized design, it is possible to assess the impact of the girls groups alone. The weekly girls’ group meeting is hypothesized, in particular, to have positive effects on the girls’ sexual and reproductive health, financial and nutritional knowledge, as well as build their communications, money management and self-advocacy skills. The group meetings also targeted the girls’ social assets, including the number of friends reported, the quality of those relationships and whether they report a role model in their lives. Participation in the groups is also hypothesized to reduce social isolation and provide access to community spaces. These relationships and opportunities are, in-turn, hypothesized to weaken regressive gender norms and roles, move aspirations, and improve mediating behaviours, such as attending school. In the domain of schooling attainment, these changes are hypothesized to improve school participation, performance and, ultimately, attainment. If confirmed, these hypothesizes will result in strong and positive relationships between the indicator of randomization to the girls group only study arm and the indicators noted along this continuum of outcomes.As with the bank account component, the health voucher intervention is hypothesized to directly increase access to health facilities and resources, as well as to increase efficacy and decision-making control over the utilization of health facilities and resources. The effect of the heath voucher component is expected to be distinguishable empirically over and above what is realized through the weekly girls groups alone. In the immediate term, the health voucher is hypothesized to increase the use of sexual and reproductive health services, including the use of modern contraception. Improved sexual and reproductive health, e.g., by obtaining counselling, care and treatment for STIs, is hypothesized to reduce HIV acquisition in the longer-term, while access to contraception is hypothesized to reduce unwanted pregnancy and birth outcomes. In general, it is hypothesized that as specific health issues are prevented or resolved, girls will have more positive assessment of their overall health.As noted in Table Participants for the research study were randomly selected among all invited AGEP participants who were never married at baseline, stratified by site and age group. For comparability, selection of the control girls followed a parallel process for identifying and selecting vulnerable girls, specifically a ranking of girls by their vulnerability score and randomly selecting girls below a set threshold, with the threshold determined by the total number of girls needed for recruitment into the programme at each site. As the ultimate objective of the funder and the programme was to improve the well-being of adolescent girls, the longer-term outcomes in Table The minimally detectable effect sizes were estimated for comparison of each programme component against the control; estimates of effect sizes were not obtained for comparisons between the intervention arms themselves as little prior information was available to guide such an estimation approach. Given a total of ten study sites, it was determined that a total of 40 clusters per arm (four clusters per arm per site) and 20 participants per cluster was the most efficient combination. Baseline (control) estimates of study indicators by age group and residential status were obtained from the 2007 Zambia Demographic and Health Survey . StatistThe sample size calculations indicated that 3200 girls would be needed at the end of the study to assess all key study impact indicators: 800 girls (within 40 study clusters) in each of the three AGEP arms and in the control arm. An additional 400 girls (within 20 clusters) were designated to be sampled for urban areas in nearby high-density components. These girls and clusters were designed to serve as external controls given the potential for contamination between AGEP and control girls within the same site. Never married girls were targeted at baseline as a significant proportion of older girls would have already experienced the key outcomes that were meant to be impacted by the programme. To achieve the target sample of 3600 girls at the end of the study, the sample size was inflated for non-eligibility and non-response at baseline, attrition from the cohort and refusals for biological specimen collection. Thus, the final estimated sample to be collected at baseline was 7200 or 4800 in AGEP clusters, 1600 in internal control clusters and 800 in external control areas.A household survey instrument was implemented prior to baseline to determine eligibility for the intervention and for selecting cases in designated control clusters. The household instrument included a roster of all household members with basic socio-demographic information on each member, household asset ownership, housing quality, recent household shocks, access to financial resources and distances to key resources, including schools, roads, health facilities, banks, etc. Comprehensive annual surveys were conducted at baseline and for four subsequent years. The baseline survey is included as an Additional file Survey instruments were implemented by electronic data capture. Computer-Assisted Personal-Interviewing (CAPI) was used for questions that are non-sensitive. CAPI is a process of data capture in which the interviewer reads the question from a computer screen and enters the participant’s response directly into a handheld or tablet device. For sensitive questions, including sexual behaviour, sexual violence, HIV risk perception, unwanted pregnancy and abortion, Audio Computer-Assisted Self-Interviewing (ACASI) was used. With ACASI the respondent listened with headphones to pre-recorded questions and response categories, while, if desired and if the participant was literate, simultaneously reading the question on the device screen. The participant entered a response by touching a designated number or option. ACASI maximizes confidentiality and privacy of response, as no one could hear or see the question being read, nor the response option selected.Biological markers were collected from adolescents aged 15 and older at baseline (Round 1), after Round 3 (at the end of AGEP), and Round 5 . HIV status was determined via capillary blood draws obtained from finger pricks. HIV tests were conducted by trained and certified voluntary counselling and testing (VCT) staff. In accordance with Ministry of Health guidelines, we conducted serial testing using Determine™ and Uni-Gold™ . Both teThe HSV-2 biological specimens were collected via finger prick. A sample of whole blood was collected and stored in microtainers and laboratory tested. To conduct the laboratory test, serum was derived from the whole blood and the Kalon ELISA assay used, for which the sensitivity and specificity have been found to be high (92% and 98%) respectively in African populations when compared to Western Blot . IndeterThe study also includes an economic evaluation to rigorously assess the cost of implementing AGEP, as well as to assess the incremental cost-effectiveness of the add-on components  relative to the girls group intervention alone.. Direct programme costs were collected from AGEP budgets and financial reports and included both start-up and programme delivery costs for the Population Council and its partners. Participant-specific out-of-pocket and indirect costs data for participation in AGEP and uptake of services were also obtained. The relevant costs for these analyses are the real resource costs of delivering the programme services, not including the costs related to project evaluation. A decision analytic model will be constructed to generate estimates of the incremental costs per negative health outcome averted and positive progress achieved on non-health indicators from participating in AGEP. Incremental cost-effectiveness ratios (ICERS) comparing the add-ons to the girls group only intervention will be calculated to assess whether cost-effectiveness varies by study arm.The primary objective of the evaluation is to assess the impact of AGEP on adolescents’ intermediate empowerment and longer-term outcomes using the above-noted hypothesis as test criteria. Important as well, analysis will be conducted to understand the underlying processes by which change occurred in the key areas of interest. As a first approach for assessing the impact of AGEP on empowerment and longer-term outcomes, an intent-to-treat analysis (ITT) will be conducted using the original randomised assignment to study arm as the primary indicator of impact. The average treatment effect of the programme on the adolescents in clusters randomised to AGEP relative to girls in clusters randomised to the control will be empirically assessed. The analysis will estimate models that are both unadjusted and adjusted for baseline covariates; both approaches accounting for strata and intraclass correlations within clusters. To accommodate for any baseline differences between AGEP and control clusters that may exist in the outcomes, a difference-in-differences (DID) calculation will be made for each indicator. The mid-term assessment after Round 3 focused on the immediate and mediating indicators as primary outcomes, while in the final assessment after Round 5 the mediating and longer-term outcomes will be the empirical focus.As girls were invited to participate in AGEP, many choose not to do so, intermittently missed the weekly sessions or left the programme before the programme was completed. Therefore a secondary analysis will be conducted that uses a measure of participation intensity as the indicator of impact rather than the study’s random assignment. It is important to recognize the statistical estimation problems that arise from self-selection into AGEP, and the degree of participation in it, with the characteristics of girls predicting both their uptake of and exposure to the programme and key outcomes and behaviours of interest. A two-stage instrumental variables estimation approach will be used in which the first stage will predict programme participation, using the exogenous randomised assignment as the key instrumental variable, observed baseline covariates will also be included. The second stage will then use the predicted participation intensity from the first stage along with baseline covariates to test the relevant hypotheses. Appropriate tests, including F-tests on excluded instruments, the Wald F-statistic, and the Hansen statistic for over-identification will quantitatively assess the credibility of the selected instrumental variable.AGEP was an intervention whose objective was to improve important adolescent transitions of girls in Zambia, with the expectation that improved outcomes for girls would lead to life-long improvements in the well-being of women and their children. The rigorous randomized evaluation and cohort study to evaluate AGEP outlined in this protocol paper was designed to address information gaps and the weaknesses recognized in the existing literature, including non-randomized study designs, a lack of information on vulnerable and disadvantaged populations, and short-term durations of assessments . As noteAGEP was a multi-sectoral intervention built off of the core concept of mentor-led, weekly girls’ group meetings whose primary objective was to build and empower girls by 1) enhancing their knowledge and skills, 2) building their self-confidence and efficacy, 3) changing their aspirations, and 4) providing access to services and resources. The AGEP programme is unique in the amount of investment in this effort, with over two years of weekly programme activities devoted to girls’ empowerment across a range of domains, including sexual and reproductive health, financial literacy, life skills and nutrition. AGEP was not, however, limited to the weekly girls groups, as two additional components, a bank account and health voucher, enhanced the potential impact of the group learnings by providing girls direct access to resources that typically did not exist for them or have high barriers to use. The multi-arm randomized evaluation was designed to assess whether each of these programme components provided additional value added in terms of impact.Determining the impact of AGEP is important in order to provide needed evidence for governments, donors, and other stakeholders to develop policies and programs for adolescent girls. Delineating costs of the programme and the investment needed for added programme components can provide additional critical information about the investment needed to scale the programme and the incremental costs of investments in each intervention component. To achieve this additional aim, the study research protocol included an embedded economic evaluation that collected programme implementation and participant costs. The economic evaluation will provide cost-effectiveness information for each study arm and an opportunity to compare AGEP with alternative programmes that address the needs of adolescent girls in the areas of early marriage and childbearing, education and sexual and reproductive health.


To increase discriminatory accuracy (DA) for emergency cesarean sections (ECSs).We prospectively collected data on and studied all 6,157 births occurring in 2014 at four public hospitals located in three different autonomous communities of Spain. To identify risk factors (RFs) for ECS, we used likelihood ratios and logistic regression, fitted a classification tree (CTREE), and analyzed a random forest model (RFM). We used the areas under the receiver-operating-characteristic (ROC) curves (AUCs) to assess their DA.The magnitude of the LR+ for all putative individual RFs and ORs in the logistic regression models was low to moderate. Except for parity, all putative RFs were positively associated with ECS, including hospital fixed-effects and night-shift delivery. The DA of all logistic models ranged from 0.74 to 0.81. The most relevant RFs  in the CTREEs showed the highest ORs in the logistic models. The DA of the RFM and its most relevant interaction terms was even higher .Putative fetal, maternal, and contextual RFs alone fail to achieve reasonable DA for ECS. It is the combination of these RFs and the interactions between them at each hospital that make it possible to improve the DA for the type of delivery and tailor interventions through prediction to improve the appropriateness of ECS indications. A worrisome issue in obstetrics is the longstanding increase in cesarean section rates, as well as the unjustified variations in these rates in clinical practice across public and private hospitals worldwide–3. This Few current clinical guidelines and interventions target these objectives –29. ThosTo our knowledge, very few studies have sought to improve the ability to predict which women are at higher risk of ECS. Those that do are limited to nulliparas, include only a few of the putative RFs, and report no measures of either calibration or DA of the statistical models developed –38. Our The present study is part of a large multifaceted intervention intended to improve the appropriateness of the indications for ECSs in 22 public hospitals of the Spanish National Health Service launched by the Spanish Ministry of Health. Of those 22 participating hospitals, four  were included in this study because their databases were the most reliable in terms of consistency and coverage to ensure that robust predictive models of ECSs could be built. In size and complexity, the obstetric services of these four hospitals belong to level II (out of III) of the Spanish National Hospital Catalogue. They can be considered representative of about 42% of all obstetrics services of the Spanish National Health Service that belong to this level, since they all have a very similar case mix, and attend pregnant women with similar obstetric risk.https://www.datosmacro.com/demografia/natalidad/espana-comunidades-autonomas).The study population consisted of all 6,157 singleton births, with no exclusions, occurring in 2014 at four public hospitals located in three different autonomous communities of Spain. According to the Spanish National Institute of Statistics, these 6,157 births account for 1,5% of all yearly births in Spain . Hospitals A and B account for 26,5% of all births occurring yearly in the Autonomous Community of the Balearic Islands, Hospital C for 12,6% of those occurring in Galicia, and Hospital D for 2,0% of those occurring in Valencia  characteristics that are not captured by any of the independent variables included in the models. They may be predictive of the type of delivery, account for a certain fraction of the medical variations  of ECSs often found in small area analysis, and modify the strength of the associations of the independent RFs and the type of delivery. They are not explanatory of the type of delivery, but their association with it may be indicative of different entrenched, difficult to measure clinical practices across hospitals that are likely to influence the decision regarding the type of delivery and therefore they warrant further investigation. Night-shift delivery was also included as an additional potentially predictive contextual independent variable, since it has been shown to be both a good predictor of the delivery mode, and an appropriate instrumental variable to infer causal associations between the average treatment effect of non-medically indicated cesarean sections  on newborn´s health outcomes .Descriptive statistics were calculated for all fetal, maternal, and contextual variables. Scheduled, emergency, and overall (both scheduled and emergency) C-sections were estimated for the whole population and for each hospital with their corresponding 95% CI.The first step in our analytical approach to identify RFs for ECS was to calculate the prevalence of each putative RF in the overall population and in mothers delivering both by vaginal birth and by ECS, as well as their 95% CI. We then estimated the prevalence ratios of each RF . Finally, we estimated the positive likelihood ratios (LR+) of each RF and their 95% CIs. , as well as a logistic model for the overall sample to find out which fetal, maternal, and contextual RFs (independent variables) were associated with the outcome , as well as the strength of the associations found. Model specification was performed based on stepwise top-bottom variable selection, and taking into consideration the clinical relevance of each variable. Crude and adjusted ORs were obtained, as well as their 95% CIs. The models’ goodness-of-fit was compared by means of the -2log-likelihood ratios and the Akaike information criterion (AIC). Their DA was assessed through their areas under the receiver-operating-characteristic (ROC) curves (AUCs) along with their 95% CI.We then fitted a classification tree , a relatively new and useful predictive technique for studying RFs and outcomes based on the unbiased recursive splitting of the study population sample into subgroups according to the independent variables . The undUnlike CART models, CTREE can handle datasets with both categorical and numerical variables without producing biased splits, and the interpretation of both odds ratios and likelihood ratios is straightforward. Therefore, we used dichotomous variables to enable comparisons with other published studies despite a small potential loss of information. All births were included in the analysis, and anonymity was preserved. A database was constructed by two computer engineers, who also managed the transfer of data. Database quality was periodically audited and was considered reliable in terms of consistency, coverage, and agreement. The database is available upon request. The Spanish Ministry of Health approved this study under the Strategy for Assistance at Normal Childbirth in the National Health System (PI/01445).n classification trees by randomly selecting predictors for each tree. CTREE was used as the base learner, and 500 different trees were created by bootstrapping, rendering more accurate predictions than a single tree analysis. This algorithm allows to estimate the relative importance of each independent variable in the model (i.e. the contribution of each independent variable to the predictive power of the random forest). The methodology to compute relative importance of each variable , and more information regarding CART, CTREE, and RFM can be found elsewhere [We also developed a random forest model (RFM) that fits lsewhere –50. We alsewhere , 50.ECS rates varied from 8 to 15% across hospitals, whereas overall C-section rates were higher (12–21%) . DescripThe prevalence of the putative RFs for ECS in the overall population, as well as in eutocic and ECS deliveries, is shown in The gender of the fetus was neither associated with the type of delivery nor improved either the calibration  or the discriminant accuracy (C statistic) of the final models. Therefore, it was excluded from the final logistic models. BMI was finally included since it did not make any difference to include height and weight separately or BMI in terms of both the calibration (AIC) and the discriminant accuracy (C statistic) of the models. We did choose the most parsimonious models as the final ones. Gestational age was also excluded from the final logistic models due to its high collinearity with the rest of the independent variables that remained in the model for each hospital, and because its inclusion led to biased intercept estimates of these logistic models.According to the final logistic regression model for the overall population , all RFsThe strength of the positive associations was relatively similar in the models for each of the four hospitals and in the model for the overall population. Although pH, induction, and previous ECS appear to be the RFs with the highest ORs, and age and BMI those with the lowest, their relative magnitude at each hospital varied slightly, except for pH, which was substantially higher at one hospital (OR = 7.17). Parity was positively associated with ECS at only one hospital, whereas obstetric risk was positively associated with it at only two.The logistic model for the overall population and those for each hospital fit the data well, as indicated by both the -2log-likelihood ratio and the Akaike criterion. The goodness-of-fit of the population model increased notably when hospital fixed-effects were included. The DA of all five models was notably high, with AUCs ranging from 0.74 to 0.81 .Of the two recursive partitioning models (CTREE and Random Forest), CTREE was used as the base learner for the Random Forest algorithm (n = 500). n = 500 CTREEs with an optimal number of randomly selected variables = 2. Although random forest algorithms tend to be more of a black box in terms of their interpretation, their predictive power  provides reliable predictions even at an individual level. The relative variable importance of all variables included in the RFM is shown in The RFM consisted of a set of The strength of the associations between some putative RFs and ECS, their prevalence, their prevalence ratios, and their LR+ in the overall population were low to moderate, indicating, as in other studies, that single RFs alone offer only a low DA for most outcomes, such as ECS –47.With the exception of scalp pH, the magnitude of the strength of these associations was low and similar across the four hospitals. Likewise, all were positive except for the number of pregnancies, which showed an inverse association. Heterogeneity did not seem to play a relevant role in the study population solely on the basis of this initial analysis. Moreover, only the number of pregnancies seemed to increase the odds of a vaginal delivery, as would be expected.In the final logistic model for the overall population both contextual variables  were positively associated with ECS and increased goodness-of-fit. These variables were associated with higher ECS rates and may thus favor the indication of ECS over vaginal deliveries. Regardless of maternal and fetal characteristics, and as indicated in a number of studies, different entrenched practices across hospitals seem to influence the decision regarding delivery type, similar to how physicians’ desire for night-time leisure influences the decision to perform an ECS at the start of the night shift –11, 39.No single 100% accurate predictive model of the type of delivery has been published to date. In fact, only a few have been published all showing a low predictive and discriminant accuracy. All these contextual  factors that may contribute both to predict and explain variations in both the type of delivery and in the appropriateness of the c-section´s indications  remain unobserved and unknown. The only available way to account for them is by including hospital fixed-effects in logistic models and in random forests as contextual variables . Moreover, their inclusion in the models reduced the biases in the estimates of the measures of strength of the associations without resulting in overfitting, and increase their discriminant accuracy because they account for the abovementioned unobserved predictive factors –11, 39.These results illustrate the usefulness of this analytic approach because they suggest that some hospital characteristics  may explain unjustified variations and inappropriateness of some indications for c-sections that warrant further investigation.Consequently, all fetal, maternal, and contextual factors alone failed to achieve a reasonable DA for ECS rates in different population subgroups at each hospital even after they were controlled for in these models. This is consistent with the well-known fact that the decision regarding the type of delivery hinges not only on different combinations of these RFs and the interactions between them, but also to some extent on variations across individual hospital practices and even individual clinicians’ practices. It can thus be the product of unjustified non-evidence-based clinical practices, which has long been shown in studies of variations in clinical practice with regard to CS using small area analysis –11.Measures of association alone are insufficient to discriminate between those individuals who will develop a given outcome and those who will not  –43. It iThe AUCs of the RFM (0.93–0.95) and the CTREE (0.84–0.92) offer a considerably improved additional analytical approach to the same issue due to the nature of their optimization algorithm, maximum likelihood for logistic and unbiased recursive partitioning for CTREE. Their incremental DA is notably higher than that of logistic models due to the unsupervised detection of interactions in the CTREE model and 500 such CTREEs in the RFM. The reasons for this improvement in DA are mainly twofold. First, it results from detecting associations and interactions among the combinations of RFs used in clinical decision-making regarding the type of delivery at each hospital that are not captured by logistic models. Second, the model also captures heterogeneity (the trees’ branches), among both the hospitals and the clinicians’ decision-making frameworks, that logistic models likewise cannot capture.In terms of implications for clinical practice, we found some medically unjustified differences in ECS rates for hospital D compared to the other hospitals, e.g., in induced births between 11 p.m. and 3 a.m. in which the scalp pH was above 7.20 . Moreover, in the subgroups of deliveries with pH above 7.20 and at least one previous C-section (nodes 25 and 26), the ECS rates climbed to 50% and almost 60%, respectively. The utility of these results lies in that, despite they are neither explanatory not confirmatory, they suggest potential sources of inappropriate ECSs in Hospital D  that should be further investigated .One of the main limitations of this study is that only 4 out 22 obstetrics services were included as explained in the Introduction. These four hospitals could be considered representative of up the 42% of hospitals within the Spanish National Health Service in terms of obstetric case mix, obstetric risk, and number of births and CS rates. However, it is to be expected that studies intended to build a predictive model for the type of delivery fail to have a high external validity with regard to the specific RFs for ECS. As already noted, it is the combination of RFs  at each particular hospital and the interactions between them what makes it possible to improve the DA for the type of delivery. The more the clinical practice varies across centers and clinicians, the more different RF-combination subgroups can be expected to appear in the CTREES given their higher ability to capturing them; hence, the more hospital-specific the combination of RFs and interactions between them yielding the highest DA will be. Given that we performed a 10-fold cross-validation using randomly allocated 90/10% training/test sample sizes, the chances of the RFM being overfitted and the AUCs being overestimated are very low.Another limitation of the study is that scalp pH is a very proximate measure likely linked to fetal distress, so it is not a surprise that it is highly predictive. We did not include cord pH because it is a post-delivery endpoint and as such cannot be considered a predictive variable of the type of delivery. We could agree that scalp pH is linked to fetal distress and can be highly predictive. However, we have included it in the models as a predictive variable for several reasons: i) scalp pH is an intrapartum variable, not a final endpoint. Variations in the cut-off points actually used in clinical practice may explain both variations in the diagnosis of fetal distress, and in the fraction of appropriate and inappropriate indications for ECSs across hospitals ; ii) it has also been shown that both the clinical management of intrapartum  pH and thus of fetal distress varies across hospitals, and that it accounts for a considerable fraction of inappropriateness of ECSs for this specific indication, what could make scalp pH a predictive variable for some but not all ECSs; and iii) tenfold cross validation performed in the CTREE model prevented from obtaining overfitted estimates when including this variable.Therefore, this study’s main contribution is that the information provided by the combination of logistic regressions and CTREES can provide more accurate information than either method alone to help clinicians and managers find the sources of heterogeneity and unjustified variations in ECSs, design and implement hospital-tailored interventions intended to improve the appropriateness of their indications, and reduce unnecessary ECS and their avoidable complications and costs. This comprehensive and complementary statistical methodology, combined with robust data collection and audit processes, makes it possible to analyze an intricate medical decision-making problem with higher discriminant capacity than previous studies.In conclusion, fetal, maternal, and contextual factors alone fail to achieve a reasonable discriminatory accuracy for type of cesarean delivery. We have met our objective by simultaneously considering these factors at each particular hospital by using both logistic regressions and the CTREES for the following reasons. First, this analytical strategy has improved the final discriminatory accuracy of the models for the type of delivery compared with that of the predictive models published to date. Second, the discriminatory accuracy of these models has been validated in our study by means of ten-fold cross-validation. Third, the results allow for further investigating sources of variability and inappropriateness of ECSs. Finally, based on this information, they also allow for tailoring hospital-specific interventions intended to discriminatory accuracy improve the appropriateness of indications for ECS.S1 Tables(XLSX)Click here for additional data file.