The explosion of the data both in the biomedical research and in the healthcare systems demands urgent solutions. In particular, the research in omics sciences is moving from a hypothesis-driven to a data-driven approach. Healthcare is additionally always asking for a tighter integration with biomedical data in order to promote personalized medicine and to provide better treatments. Efficient analysis and interpretation of Big Data opens new avenues to explore molecular biology, new questions to ask about physiological and pathological states, and new ways to answer these open issues. Such analyses lead to better understanding of diseases and development of better and personalized diagnostics and therapeutics. However, such progresses are directly related to the availability of new solutions to deal with this huge amount of information. New paradigms are needed to store and access data, for its annotation and integration and finally for inferring knowledge and making it available to researchers. Bioinformatics can be viewed as the “glue” for all these processes. A clear awareness of present high performance computing (HPC) solutions in bioinformatics, Big Data analysis paradigms for computational biology, and the issues that are still open in the biomedical and healthcare fields represent the starting point to win this challenge. The increasing availability of omics data resulting from improvements in the acquisition of molecular biology results and in systems biology simulation technologies represents an unprecedented opportunity for bioinformatics researchers, but also a major challenge. A similar scenario arises for the healthcare systems, where the digitalization of all clinical exams and medical records is becoming a standard in hospitals. Such huge and heterogeneous amount of digital information, nowadays called Big Data, is the basis for uncovering hidden patterns in data, since it allows the creation of predictive models for real-life biomedical applications. But the main issue is the need of improved technological solutions to deal with them. volume, variety, and velocity [A simple definition of Big Data is based on the concept of data sets whose size is beyond the management capabilities of typical relational database software. A more articulated definition of Big Data is based on the three versus paradigm:velocity . The volAlthough the Big Data definition can still be considered quite nebulous, it does not represent just a keyword for researchers or an abstract problem: the USA Administration launched a 200 million dollar “Big Data Research and Development Initiative” in March 2012, with the aim to improve tools and techniques for the proper organization, efficient access, and smart analysis of the huge volume of digital data , which pStorage represents only one side of the medal. The final goal of research activities in omics sciences is to turn such amount of data into usable information and real knowledge. Biological systems are very complex, and consequently the algorithms involved in analysing them are very complex as well. They still require a lot of effort in order to improve their predictive and analytical capabilities. The real challenge is represented by the automatic annotation and/or integration of biological data in real-time, since the objective to reach is to understand them and to achieve the most important goal in bioinformatics: mining information.In this work we present a brief review of the technological aspects related to Big Data analysis in biomedical informatics. The paper is structured as follows. In Domains concerned with data-intensive applications have in common the abovementioned three versus, even though the actual way by which this information is acquired, stored, and analysed can vary a lot from field to field. The main common aspect is represented by the requirements for the underlying IT architecture. The mere availability of disk arrays of several hundreds of TBs is in fact not sufficient, because the access to the data will have, statistically, some fails . Thus, rThe first and obvious concern with Big Data is the volume of information that researchers have to face, especially in bioinformatics. At lower level this is an IT problem of file systems and storage reliability, whose solution is not obvious and not unique. Open questions are what file system to choose and will the network be fast enough to access this data. The issue arising in data access and retrieval can be highlighted with a simple consideration : scanninMoreover, the transfer rate is inversely proportional to the distance to cover. HPC clusters are typically equipped with high-level interconnections as InfiniBand, which have a latency of 2000 ns (only 20 times slower than RAM and 200 times faster than a Solid State Disk) and data rates ranging from 2.5 Gigabit per second (Gbps) with a single data rate link (SDR), up to 300 Gbps with a 12-link enhanced data rate (EDR) connection. But this performance can only be achieved on a LAN, and the real problems arise when it is necessary to transfer data between geographically distributed sites, because the Internet connection might not suitable to transfer Big Data. Although several projects, such GÉANT , the pan distributed file systems, cluster file systems, and parallel file systems. Distributed file systems consist of disk arrays physically attached to a few I/O servers through fast networks and then shared to the other nodes of a cluster. In contrast, cluster file systems provide direct disk access from multiple computers at the block level (access control must take place on the client node). Parallel file systems are like cluster file systems, where multiple processes are able to concurrently read and write a single file, but exploit a client-server approach, without direct access to the block level. An exhaustive taxonomy and a survey are provided in [If a local infrastructure is exploited for the analysis of Big Data, one effective solution is represented by the use of client/server architectures where the data storage is spread among several devices and made accessible through a  network. The available tools can be mainly subdivided invided in .Two of the highest performance parallel file systems are the General Parallel File System (GPFS) , developA different solution is represented by the Hadoop Distributed File System (HDFS) , a Java- NameNode service, running on one node in the cluster and a DataNode process running on each node that will process data. HDFS is designed to be fault-tolerant due to replication and distribution of data, since every loaded file is replicated  and split into blocks that are distributed across the nodes. The NameNode is responsible for the storage and management of metadata, so that when an application requires a file, the NameNode informs about the location of the needed data. Whenever a data node is down, the NameNode can redirect the request to one of the replicas until the data node is back online. Since the cluster size can be very large , the single NameNode per cluster forms a possible bottleneck and single point of failure. This can be mitigated by the fact that metadata can be stored in the main memory and the recent HDFS High Availability feature provides the user with the option of running two redundant NameNodes in the same cluster, one of them in standby, but ready to intervene in case of failure of the other.At the architectural level, HDFS requires two processes: aThe file system is the first level of the architecture. The second level corresponds to the framework/middleware supporting the development of user-specific solutions, while applications represent the third level. Besides the general-purpose solutions for parallel computing like the Message Passing Interface , hybrid  Map and Reduce functions, commonly used in functional programming. In the Map step the input is partitioned by a master process into smaller subproblems and then distributed to worker processes. In the Reduce step the master process collects the results and combines them in some way to provide the answer to the problem it was originally trying to solve.The first and more famous example is the abovementioned Apache Hadoop , an openHadoop was designed as a platform for an entire ecosystem of capabilities , used inHadoop was the basis for other higher-level solutions as Apache Hive , a distrHadoop is considered almost as synonymous for Big Data. However, there are some alternatives based on the same MapReduce paradigm like Disco , a distrThe traditional platforms for operating the software frameworks that facilitate Big Data analysis are HPC clusters, possibly accessed via grid computing infrastructures . Such apThe data parallel approach, that is, the parallelization paradigm that subdivides the data to analyse among almost independent processes, is a suitable solution for many kinds of Big Data analysis that results in high scalability and performance figures. The key issues while developing applications using data parallelism are the choice of the algorithm, the strategy for data decomposition, load balancing among possibly heterogeneous computing nodes, and the overall accuracy of the results . de novo assembly algorithms, which typically work finding the fragments that “overlap” in the sequenced reads and recording these overlaps in a huge diagram called de Bruijn (or assembly) graph [An example of Big Data analysis based on this approach in the field of bioinformatics concerns they) graph . For a ly) graph , are nowGenerally, we can say that data parallel approaches are straightforward solutions for inferring correlations, but not for causality. In these cases semantics and ontological techniques are of particular importance, as those described in HPC technologies are the forefront of accelerated data analysis revolutions, making it possible to carry out processing breakthroughs that would directly translate into real benefits for the society and the environment. The use of accelerator devices such as GPUs represents a cost-effective solution able to support up to 11.5 Teraflops within a single device  at about $1,500. Moreover, large clusters are adopting the use of these relatively inexpensive and powerful devices as a way of accelerating parts of the applications they are running. Presently, most of the top 10 systems from the TOP500 list (November 2013) are equipped with accelerators, and in particular Titan, the second system in the list, achieved 17.59 PetaFlop on the Linpack benchmark also thanks to its 18,688 Nvidia Tesla K20 devices.Driven by the demand of the game industry, GPUs have completed a steady transition from mainframes to workstations PC cards, where they emerge nowadays like a solid and compelling alternative to traditional computing platforms. GPUs deliver extremely high floating-point performance and massively parallelism at a very low cost, thus promoting a new concept of the high performance computing market. For example, in heterogeneous computing, processors with different characteristics work together to enhance the application performance taking care of the power budget. This fact has attracted many researchers and encouraged the use of GPUs in a broader range of applications, particularly in the field of bioinformatics. Developers are required to leverage this new landscape of computation with new programming models, which ease the developers task of writing programs to run efficiently on such platforms altogether .The most popular microprocessor companies such as NVIDIA, ATI/AMD, or Intel, have developed hardware products aimed specifically at the heterogeneous or massively parallel computing market: Tesla products are from NVIDIA, Fire-stream is the AMDs device line, and Intel Xeon Phi comes from Intel. They have also released software components, which provide simpler access to this computing power. CUDA (Compute Unified Device Architecture) is NVIDIAs solution as a simple block-based API for programming; AMDs alternative was called Stream Computing, while Intel relies directly on X86-based programming. OpenCL  emerged Although these efforts in developing programming models have made great contributions to leverage the capabilities of these platforms, developers have to deal with a massively parallel and high-throughput-oriented architecture , which iBased on Intel's Many Integrated Core (MIC) x86-based architecture, Intel Xeon Phi coprocessors provide up to 61 cores and 1.2 Teraflops of performance. These devices equip the first supercomputer of the TOP500 list (November 2013), Tianhe-2. In terms of usability, there are two ways an application can use an Intel Xeon Phi: in offload mode or in native mode. In offload mode the main application is running on the host, and it only offloads selected  work to the coprocessor. In native mode the application runs independently, on the Xeon Phi only, and can communicate with the main processor or other coprocessors through the system bus.The performance of these devices heavily depends on how well the application fits the parallelization paradigm of the XeonPhi and in relation to the optimizations that are performed. In fact, since the processors on the Xeon Phi have a lower clock frequency with respect to the common Intel processor unit (such as for example the Sandy Bridge), applications that have long sequential part of the algorithm are absolutely not suitable for the native mode. On the other hand, even if the programming paradigm of these devices is standard C/C++, which makes their use simpler with respect to the necessity of exploiting a different programming language such as CUDA, in order to achieve good performance, the code must be heavily optimized to fit the characteristics of the coprocessor .Looking at the performance tests released by Intel , the basClearly, the real competitors of the Intel Xeon Phi are the GPUs devices. At the moment, the comparison between the best devices provided by Intel—Xeon Phi 7100—and Nvidia—K40—shows that the GPU is on average 30% more performing , but theAdvances in life sciences and information technology bring profound influences on bioinformatics due to its interdisciplinary nature. For this reason, bioinformatics is experiencing a new trend in the way analysis is performed: computation is moving from in-house computing infrastructure to cloud computing delivered over the Internet. This has been necessary in order to handle the vast quantities of biological data generated by high-throughput experimental technologies. Cloud computing in particular promises to address Big Data storage and analysis issues in many fields of bioinformatics. But moving data to the cloud can also be a problem; so hybrid solutions of cloud computing are arising. The point can be summarized as follows: data that is too big to be processed conventionally is also too big to be transported anywhere. IT is undergoing an inversion of priorities: the code to perform the analysis has to be moved, not the data. Virtualization represents an enabling technology to achieve this result.One of the most famous free portal/software for the analysis of bioinformatics data, Galaxy by the Penn state University, is available on cloud . The ideFor a complete review on bioinformatics clouds for Big Data manipulation, see . ConcernThe previous sections focused mainly on how to analyse Big Data for inferring correlations, but the extraction of actual new knowledge requires something more. The key challenges in making use of Big Data lie, in fact, in finding ways of dealing with heterogeneity, diversity, and complexity of the information, while its volume and velocity hamper solutions are available for smaller datasets such as manual curation or data warehousing.Semantic web technologies are meant to deal with these issues. The development of metadata for biological information on the basis of semantic web standards can be seen as a promising approach for a semantic-based integration of biological information . On the In domains like bioinformatics and biomedicine, the rapid development and adoption of ontologies  promptedThe semantic web is a collaborative movement, which promoted standard for the annotation and integration of data. By encouraging the inclusion of semantic content in data accessible through the Internet, the aim is to convert the current web, dominated by unstructured and semistructured documents, into a web of data. It involves publishing information in languages specifically designed for data: Resource Description Framework (RDF), Web Ontology Language (OWL), SPARQL (which is a protocol and query language for semantic web data sources), and Extensible Markup Language (XML) .RDF represents data using subject-predicate-object triples, also known as “statements.” This triple representation connects data in a flexible piece-by-piece and link-by-link fashion that forms a directed labelled graph. The components of each RDF statement can be identified using Uniform Resource Identifiers (URIs). Alternatively, they can be referenced via links to RDF Schemas (RDFS), Web Ontology Language (OWL), or to other (nonschema) RDF documents. In particular, OWL is a family of knowledge representation languages for authoring ontologies or knowledge bases. The languages are characterised by formal semantics and RDF/XML-based serializations for the semantic web. In the field of biomedicine, a notable example is the Open Biomedical Ontologies (OBO) project, which is an effort to create controlled vocabularies for shared use across different biological and medical domains. OBO belongs to the resources of the U.S. National Center for Biomedical Ontology (NCBO), where it will form a central element of the NCBO's BioPortal. The interrogation of these resources can be performed using SPARQL, which is an RDF query language similar to SQL, for the interrogations of databases, able to retrieve and manipulate data stored in RDF format. For example, BioGateway  organizeThe support for RDF in high-throughput bioinformatics applications is still small, although researchers can already download the UniProtKB and its taxonomy information using this format or they can get ontologies in OWL format, such as GO . The bigAn ontology layer is often an invaluable solution to support data integration , particuIn the biomedical context a common problem concerns, for example, the generality of the term cancer . A direcA useful instrument for ontology exploration has been developed by the European Bioinformatics Institute (EBI), which allows easily visualising and browsing ontologies in the OBO format: the open source Ontology Lookup Service (OLS) . The sysGene ontology (GO) , which iKEGG ontology (KOnt), which provides a pathway based annotation of the genes in all organisms. No OBO version of this ontology was found, since it has been generated directly starting from data available in the related resource .Brenda Tissue Ontology (BTO) , to suppCell Ontology (CL) , to provDisease ontology (DOID) , which fProtein Ontology (PRO) , which dMedical Subject Headings thesaurus (MESH) , which iProtein structure classification (CATH) , which iThe following ontologies are commonly used for annotation and integration of data in the biomedical and bioinformatics.Linked data describes a method for publishing structured data so that they can be interlinked, making clearer the possible interdependencies. This technology is built upon the semantic web technologies previously described , but rather than using them to serve web pages for human readers, it extends them to share information in a way that can be read automatically by IT systems.The linked data paradigm is one approach to cope with Big Data as it advances the hypertext principle from a web of documents to a web of rich data. The idea is that after making data available on the web (in an open format and with an open license) and structuring them in a machine-readable fashion , researchers must work to annotate this information with open standards from W3C (RDF and SPARQL), so that people can link their own data to other people's data to provide context information.In the field of bioinformatics, a first attempt to publish linked data has been performed by the Bio2RDF project . The proA very important step towards the use of linked data in the computational biology field has been done by the abovementioned EBI , which dBesides improving the search capabilities via ontologies, metadata, and linked data for accessing data efficiently, the usability aspect is also a fundamental topic for Big Data. Scientists want to focus on their specific research while creating and analysing data without the need to know all the low-level burdens related to the underlying data management infrastructures. This demand can be addressed by science gateways that are single points of entry to applications and data across organizational boundaries. Data security is another aspect that must be addressed when providing access to Big Data, in particular while working in the healthcare sector.The overall goal of science gateways is to hide the complex underlying infrastructure and to offer intuitive user interfaces for job, data, and workflow management. In the last decade diverse mature frameworks, libraries, and APIs have been evolved, which allow the enhanced development of science gateways. While distributed job and workflow management are widely supported in frameworks like EnginFrame , implemeVectorBase , for exaThe requirement for processing Big Data in a science gateway is reflected in a few current developments in the context of standardized APIs and frameworks. Agave  providesAn example of a science gateway in the bioinformatics filed is MoSGrid (molecular simulation grid)  that supWhatever technology is used, the distributed data management for biomedical applications with community options for sharing data requires especially secure authentication and secure measures to assure strict access policies and the integrity of data . MedicalGSI (Grid Security Infrastructure)  has beenData is considered the Fourth Paradigm in science , besidesThere are many open issues for Big Data management and analysis, in particular in the computational biology and healthcare fields. Some characteristics and open issues of these challenges have been discussed in this paper, such as architectural aspects and the capability of being flexible enough to collect and analyse different kind of information. It is critical to face the variety of the information that should be managed by such infrastructures, which should be organized in scheme-less contexts, combining both relaxed consistency and a huge capacity to digest data. Therefore, a critical point is that relational databases are not suitable for Big Data problems. They lack horizontal scalability, need hard consistency, and become very complex when there is the need to represent structured relationships. Nonrelational databases (NoSQL) are the interesting alternative to data storage because they combine the scalability and flexibility.From the computational point of view, the novel idea is that jobs are directly responsible of managing input data, through suitable procedures for partitioning, organizing, and merging intermediate results. Novel algorithm will contain large parts of not functional code, but essential for exploiting housekeeping tasks. Due to the practical impossibility of moving all the data across geographical dispersed sites, there is the need of computational infrastructure able to combine large storage facilities and HPC. Virtualization can be the key in this challenge, since it can be exploited to achieve storage facilities able to leverage in-memory key/value databases to accelerate data-intensive tasks.The most important initiatives for the usage of Big Data techniques in medical bioinformatics are related to scientific research efforts, as described in the paper. Nevertheless, some commercial initiatives are available to cope with the huge quantity of data produced nowadays in the field of molecular biology exploiting high-throughput omics technologies for real-life problems. These solutions are designed to support researchers working in computational biology mainly using Cloud infrastructures. Examples are Era7 Bioinformatics , DNAnexuHospitals are also considering to hiring Big Data solutions in order to provide “support services for researchers who need assistance with managing and analyzing large medical data sets” . In partIt is therefore clear that in the Big Data field there is much to do in terms of making these technologies efficient and easy-to-use, especially considering that even small and medium-size laboratories are going to use them in a close future.


CAPS used smartphones to capture digital images of the original records on an electronic CRF (eCRF). In the present study, descriptive statistics are used to report the experience of electronic data capture with remote SDV in a challenging research setting in rural Malawi.Source data verification (SDV) is a data monitoring procedure which compares the original records with the Case Report Form (CRF). Traditionally, on-site SDV relies on monitors making multiples visits to study sites requiring extensive resources. The Cooking And Pneumonia Study (CAPS) is a 24- month village-level cluster randomized controlled trial assessing the effectiveness of an advanced cook-stove intervention in preventing pneumonia in children under five in rural Malawi  recorded in the eCRF with the data in the digital images of the original records.664 episodes of pneumonia were recorded after 6 months of follow-up. Of these 664 episodes, 611 (92%) had a finding of pneumonia in the original records. All digital images of the original records were clear and legible.Electronic data capture using eCRFs on mobile technology is feasible in rural Malawi. Capturing digital images of the original records in the field allows remote SDV to be conducted efficiently and securely without requiring additional field visits. We recommend these approaches in similar settings, especially those with health endpoints. Ranked the 18is rural . Pneumonis rural –4. The dis rural . Advanceis rural . The Glotudy.org  10]..th leastData monitoring in clinical trials is a process that aims to ensure that data are complete, accurate, and verifiable . In clinTraditionally, SDV relies on monitors making multiple visits to study sites requiring extensive travel. Bresclauer et al. reported that SDV accounts for 46% of the time spent on-site by monitors and 40% of the costs for phase II or phase III trials . In anotThe widespread availability of low-cost mobile technologies has led to its growing use as a method for electronic data capture. Walther et al. compared commonly available methods of electronic data capture , and a small laptop computer) with conventional paper-collection methods in rural West Africa . They foCAPS adopted mobile technology to capture data on an electronic CRF (eCRF). This allowed data to be accessed remotely and securely from the United Kingdom (UK). This avoids the heavy financial, human, and time resources associated with traditional paper-collection methods for on-site SDV. In the present study, descriptive statistics are used to report the experience of electronic data capture with remote SDV in a challenging rural setting of Malawi.CAPS is a 24-month village level cluster randomized controlled trial. The trial has recruited over 4500 households representing over 10000 children across two study sites, Chikhwawa District in the South and Karonga District in the North. Participants have been randomized to receive an advanced cook-stove intervention (two Philips fan-assisted stoves with a solar charger per household) or continuation of traditional cooking methods. The primary outcome is the incidence of pneumonia in children under the age of five. Healthcare providers, blinded to trial arm, diagnosed pneumonia using the World Health Organization (WHO) Integrated Management of Childhood Illness (IMCI) pneumonia assessment protocol .The CAPS site we report on here is represented by villages of the remote and rural Chikhwawa district of Southern Malawi, about 30 miles from the city of Blantyre. Chikhwawa is a poorly resourced area with limited infrastructure. Households were recruited from populations mostly living in deprived conditions. To ensure a minimum of six months’ data collection before the child’s fifth birthday, all children up to the age of four and a half years living in the trial catchment area were eligible to participate in CAPS. CAPS began collecting data on the 1st of December 2014. As of 15th of June 2015, 664 episodes of pneumonia had been recorded and included in the present study.The members of the CAPS team, involved in SDV were, one of its co-principal investigators, the trial manager, the data managers, the project manager, 22 fieldworkers, and the SDV worker. Compliance with the protocol and Standard Operating Procedures (SOPs) by field staff was maximized through training events to include GCP training and periodic quality control audits.CAPS was approved by the College of Medicine Research Ethics Committee (COMREC) in Malawi and LSTM Research Ethics Committee (REC) in the UK. Informed and voluntary written consent was obtained from the next of kin, caretakers, or guardians on behalf of the minors/children enrolled in CAPS. This was documented through consent forms that each participant signed. The ethics committees/Institutional Review Boards (IRB) approved this consent procedure.Health workers, including health surveillance assistants, nurses, and medical officers, employed in local facilities, such as health posts, village clinics, health centers, private health clinics, and district hospitals, were responsible for the clinical assessment, classification, and management of pneumonia. Patients were diagnosed according to the WHO IMCI pneumonia protocol.In Malawi, if any child presents to a health facility, health workers make a record of the event  in a free government booklet called the health passport  Fig 2)..Fig 2). A study-specific CAPS sticker was inserted in the health passports of all participants. The CAPS sticker informed the health worker that the child was enrolled in CAPS. In addition to recording the event in the health passport, if an episode of pneumonia had occurred, health workers were asked to record the date, classification, clinical indicators , and outcome of the episode as outlined in the CAPS sticker .At three monthly intervals, fieldworkers employed by CAPS, visited participants at their homes to collect data from the health passport and CAPS sticker (source documents). Fieldworkers also captured digital images of the pages of the source documents relating to the last study visit, up to the latest entry. Data were collected using an eCRF on a Samsung Galaxy S3 Smartphone. Use of the eCRF allowed data to be entered directly from the study site into the master database, making the large number of CRFs manageable.If an episode of pneumonia had been recorded in the health passport since the last study visit, the fieldworkers recorded its occurrence in the eCRF. If the same episode of pneumonia recorded in the health passport had been recorded in the CAPS sticker, fieldworkers transcribed the data from the CAPS sticker into the eCRF. If multiple episodes were recorded in the health passport and/or CAPS sticker within one time-period month, only the first episode of pneumonia was recorded in the eCRF as it could be considered as the same infection.The Open Data Kit (ODK) Collect mobile application was installed on each of the mobile devices as a platform for the eCRF. An eCRF in ODK was in the format of a XML file, ODK Collect interpreted this XML file and displayed it as a questionnaire on the mobile device. When the eCRF was completed, the data were stored in a separate XML file, which was saved to the storage of the mobile device. When Internet connectivity (either via wi-fi or mobile data) became available, the mobile device sent the XML file to the ODK Aggregate server, which was the server software responsible for collating all of the data. When the send was completed and the mobile device received confirmation of this, the XML file containing data was deleted.The Malawi Liverpool Wellcome (MLW) programme provided an administrative and infrastructural hub for CAPS. The ODK Aggregate server was hosted at MLW within their data centre. The base operating system, on which ODK Aggregate ran, was UBUNTU. The database software used to store the data was MySQL. The programming of the eCRF was completed by a data manager at MLW. He was also responsible for the installation, programming and maintenance of the ODK Aggregate server at MLW. Initially, data had to be manually transported in a portable hard drive from Malawi to the UK. Later on, our server at LSTM was able to connect to the ODK Aggregate server at MLW to take a copy of the CAPS data.Initially, Microsoft Access was used to present the data from the eCRF recorded for each episode of pneumonia in the master database with the digital images of the health passport and CAPS sticker. Microsoft Excel was used to record the results of SDV. We later developed a web-based SDV application using a combination of PHP and HTML. PHP is a programming language used to interpret data from MySQL so it can be displayed in a web page. Rather than accessing the data on computers in the LSTM, this allowed SDV to be conducted from anywhere on a password protected web page. This secure remote access provided a user-friendly platform and enabled comparisons to be made easily and in a timely manner.UBUNTU, MYSQL, and ODK aggregate were password protected. Access was restricted to authorized users only. All the mobiles devices and computers used in CAPS were password protected. The mobile devices were returned to the health center in Chikhwawa at the end of each day of data collection, where they were stored overnight and recharged. The data centre at MLW, where the ODK Aggregate server was hosted, was at a secure location managed by their Information Technology (IT) services.SDV was conducted for data critical to trial outcomes for all recorded episodes of pneumonia in the master database for:The occurrence of pneumonia The severity classification (recorded in the CAPS sticker)8 clinical indicators (recorded in the CAPS sticker)Each episode of pneumonia recorded in the master database was recognized as an SDV case, and given a unique SDV case number. If there were no discrepancies in any of the comparisons, the fieldworker had transcribed the data accurately. In this instance, the case required no further investigation, and was highlighted as ‘validated’. If a discrepancy was recognized in any of the comparisons, the SDV case was highlighted as a ‘query’. A query had to be resolved if the SDV case was to be validated. Some queries we were able to resolve immediately, and others required further investigation.A query was also raised, if, upon examining a source document, the study protocol had not been complied with. This gave an indication of the success with which protocol was being undertaken throughout the trial, for example, legibility of digital images of source documents and health-worker completion rate of the CAPS sticker. Any areas of non-compliance were passed onto the clinical trial manager. The clinical trial manager highlighted these areas to fieldworkers to improve compliance.The digital images of the pages of the health passport were examined for a finding of pneumonia. A positive finding of pneumonia was recorded if ‘pneumonia’, ‘lower respiratory tract infection’, or ‘acute lower respiratory infection’ was specified in the health passport, and related to the same episode recorded in the master database. If none of these terms were specified for the episode recorded in the master database or there was no digital image available since the last study visit, a negative finding of pneumonia was recorded. If a negative finding of pneumonia was recorded, the SDV case was highlighted as a query.The digital image of the page of the health passport where the CAPS sticker had been inserted was examined for a finding of pneumonia. A positive finding was recorded if the information in the CAPS sticker related to the same episode recorded in the master database. If the information did not relate to the episode recorded in the master database or there was no digital image available of the CAPS sticker, a negative finding was recorded. Given that the CAPS sticker may or may not have been completed by the health worker, a negative finding in the CAPS sticker was highlighted as a query only if there was no digital image available.In each SDV case, the severity classification and eight clinical indicators in the digital image of the CAPS sticker were compared with the data in the master database. If the data were consistent, a positive finding was recorded. If there was a discrepancy, a negative finding was recorded, and the SDV case was highlighted as a query.There were 664 episodes of pneumonia recorded in the CAPS master database. Of the 664 episodes of pneumonia, 611 (92%) had a positive finding in the health passport and 53 (8%) had a negative finding in the health passport  . All 53 Of the 664 episodes of pneumonia, 280 (42%) had a positive finding in the CAPS sticker and 384 (58%) had a negative finding in the CAPS sticker  . All 134There were 271 cases with a positive finding of pneumonia in both the health passport and the CAPS sticker (41%). There were 44 cases with a negative finding in both the health passport and the CAPS sticker (7%). There were 620 cases which had a positive finding in at least one source document (93%). The digital images of the pages of the health passport and CAPS stickers were clear and legible with few exceptions.The use of a smartphone to capture data on the eCRF and digital images of the health passport and CAPS sticker allowed correction of transcription errors from the UK without the need for any additional visits. As the digital images of the source documents were clear and legible the data were easily verifiable.The process of comparing data in the master database with data in the digital images of the source documents took approximately one to two minutes for each SDV case. Although this process was quick and simple, transferring the large volume of digital images, to be accessed in the UK, required many hours of work by the data managers. Because of slow connection speeds in Malawi, data were transferred on an ad-hoc basis. Furthermore, because ODK Collect stored images as a binary data type known as BLOB, these had to be extracted to convert them back into an image file. This manual process proved to be time-consuming. Data managers also developed the web-based SDV application, in which the data was viewed and the results of SDV recorded.Eight percent of episodes of pneumonia recorded in the master database were not identified in the digital images of the health passport. It is likely that this is explained by missing digital images of the health passport, either not captured in the field or lost during processing. To achieve high quality documentation, we aim to investigate these instances with a new set of digital images at the next follow-up visit. Sixteen percent of SDV cases with a completed CAPS sticker had at least one incorrect value in severity classification or clinical indicators. In these instances, it was possible to amend the master database immediately, without further investigation. Although the SDV was limited to primary and secondary outcomes and study protocol monitoring, it could be used for a range of tasks to ensure the validity and integrity of the trial, including adverse event reporting, consent documentation, and regulatory compliance.This methodology was successfully implemented in a challenging research setting. There had been some skepticism, given assumptions around the lack of infrastructure to support this technologically advanced approach. Where there had been some concern that fieldworkers recruited in rural settings of Malawi would lack experience with mobile technology, it was well accepted by all those who participated. This is in agreement with King et al. in a similar rural setting in Malawi, who found that fieldworkers preferred electronic data capture to traditional paper collection methods .The CAPS sticker was included to help maximize the completeness and quality of information recorded for each episode of pneumonia . If we consider that completing the CAPS sticker was a voluntary additional exercise for health workers, who often work in busy health facilities, our finding that 42% of episodes of pneumonia recorded in the master database had a completed CAPS sticker in addition to the health passport, demonstrates this worked well. We provided pulse oximeters for the use in the assessment of pneumonia, locally tailored educational sessions, and a contribution to the medical equipment fund for the district hospitals involved.The use of an eCRF and digital images of the source documents allowed SDV to be conducted remotely without the need for additional field visits. Although no formal analysis of costs was made, it is likely that the use of widely available low-cost mobiles devices was more efficient than labour- intensive traditional paper-collection methods. These methods will become more important as electronic data capture becomes the norm, particularly in challenging research settings. We recommend these approaches in similar settings, especially those with health endpoints.S1 Supporting InformationA complete table of the results of SDV for the present work.(XLSX)Click here for additional data file.


In 2006, Burkina Faso set up a policy to subsidize the cost of obstetric and neonatal emergency care. This policy has undoubtedly increased attendance at all levels of the health pyramid. The aim of this study was to measure the capacity of referral hospitals’ maternity services to cope with the demand for health services after the implementation of this policy.This study was conducted in three referral health centres . The CHU Yalgado Ouédraogo (tertiary level) and the CMA in Sector 30 (primary level) were selected as health facilities in the capital, along with the Kaya CHR (secondary level). At each health facility, the study included official maternity ward staff only. We combined the two occupational categories (nurses and midwives) because they perform the same activities in these health facilities. We used the WISN method recommended by WHO to assess the availability of nurses and midwives.Nurses and midwives represented 38% of staff at the University Hospital, 65% in the CHR and 80% in the CMA. The number of nurses and midwives needed for carrying out the activities in the maternity ward in the University Hospital and the CMA is greater than the current workforce, with WISN ratio of 0.68 and 0.79 respectively. In the CHR, the current workforce is greater than the number required (WISN ratio = 2).This medical centre is known for performing a high number curative and preventive activities compared to the Kaya CHR. Like the CHU, the delivery room is a very busy unit. This activity requires more time and more staff compared to other activities.This study showed a shortage of nurses and midwives in two health facilities in Ouagadougou, which confirms that there is considerable demand. At the Kaya CHR, there is currently enough staff to handle the workload in the maternity ward, which may indicate a need to expand the analysis to other health facilities to determine whether a redistribution of health human resources is warranted. One of the challenges facing developing countries, particularly in Africa, in achieving the Millennium Development Goals (MDGs) is the availability of health human resources. Not only do these countries have a shortage of human resources, but their distribution is often uneven, especially between urban and rural areas ,2. In liGiven the scarcity of health human resources and the increased use of health services, health workers often complain of being overworked. Burkina Faso, a French-speaking West African country, is also affected by this health human resources issue. In 2003, the State adopted a policy for the regionalization of health worker recruitment in order to promote better geographical distribution of health workers . Even beWe conducted a study on workload in first-line health facilities in Burkina Faso (CSPSs). The results showed that health centres have enough health workers to manage the workload of a free health care system .However, the empirical data indicates that health care workers, particularly at the referral levels, complain of being overworked. Similar studies have been conducted in a number of English-speaking African countries -16. TherThe purpose of this article is therefore to examine the ability of referral hospital maternity wards to meet the public's demand for services following EmONC intervention.This study focuses primarily on nurses and midwives, which are the occupational categories that attend to the majority of maternity activities .This study was conducted in referral health centres  which represent the three levels of the health pyramid of Burkina Faso. We had selected the CHU Yalgado Ouédraogo (tertiary level) and the CMA in Sector 30 (primary level) as health facilities in the capital (urban areas), and the Kaya CHR (secondary level) in a rural area. We chose these two centers in the capital city because the health information system data showed a very high utilization of maternity services at the CHU (8752 inpatients and 3752 deliveries in 2011) and the sector 30 CMA (4199 inpatients and 3371 deliveries). The Kaya CHR was selected because of its health and demographic surveillance system (HDSS) that allowed us to conduct this study .At each health facility, the study included official maternity ward staff only, to detect the maternity wards' ability to keep up in light of the EmONC policy. We combined the two occupational categories (nurses and midwives) because they perform the same activities in these health facilities.We used two tools to collect data: (i) an observation guide to observe and time various tasks performed by midwives and nurses; and (ii) a retrospective activities census guide completed in 2011. In total, the data collection lasted 34 days: 14 days at the CHU, 10 days at the CHR, and 10 days at the CMA. Six investigators were recruited and trained to collect data in the field.We used the Workload Indicators of Staffing Need (WISN) method to calculate nursing and midwife staffing needs in these maternity wards. This method was developed by the World Health Organization (WHO) to calculate optimal distribution and staff deployment . It is a- The available work time is the number of working days in a health worker's year (number of weeks per year (52) multiplied by the number of working days in a week (5) minus the number of non-working days, e.g. annual leave, public holidays, sick leave, training, etc.). Since nurses and midwives are public servants, their work time was calculated based on the standard work schedule for a public servant. The standard is five working days per week, eight working hours per day, 14 public holidays per year, and 10 authorized days of leave.- The annual work time of each activity is the time spent performing the activity in one year, i.e., the duration of the activity multiplied by the total number of times the activity is performed in the year. The activities that comprise the workload were determined using the observation guide.- To determine nursing and midwife staffing needs (A) for service activities, the annual work time for each activity included in the workload  was divided by the total available annual work time. This equals the number of nurses and midwives required for that activity. For support activities, that is, activities in the health care centre for which no statistical data is available, the staffing need calculation is (B) = 1/[1-];- The percentage of time to complete activities per day is calculated as follows: (observed duration/8) x 100, with "8" being the number of working hours per day.- The total number of nurses and midwives required is obtained by multiplying the number of nurses and midwives required for service activities by the number required for support activities (C = A x B). For this calculation, it was assumed that there were no individual activities because they perform the same activities. We used EpiData to enter the data and SPSS Statistics 17.0 and Microsoft Excel for analyses. We used the test for median comparison to interpret the observed durations in the three facilities.This study examined the maternity wards of three health facilities. Each of them is comprised of several health care units, including 11 at the CHU, 9 at the CHR, and 8 at the CMA. In total, 12 health care activities were monitored at the CHU, 10 at the CHR, and 10 at the CMA . For example, a hospitalized patient in intensive care at CHU Yalgado Ouédraogo maternity ward receives on average six times more care time (one hour) per day than a patient hospitalized in the postpartum care unit (10 minutes). We found no significant statistical difference between the times that the different facilities take to perform this activity (p=0.801).Table Midwives and nurses are the best-represented occupational category within these facilities  also assisted with curative activities. CHU deliveries are almost all done by midwives at the unit delivery room. However, in other units such as post-surgical and intensive, most medical care or resuscitations involve nurses and midwives who administer inpatient care. This task-shifting can have an effect on workload estimates in nursing and midwifery.We also found that in the CHU delivery room, interns conduct deliveries in the course of their training and also give treatment to patients admitted to intensive care. These activities should actually be done by midwives and nurses. At the CHR and CMA levels, the distinction is less evident. It must be noted that the Yalgado Ouédraogo CHU is the training centre for medical and health students in Burkina Faso. The number of trainees could lead to underestimates of the number of official staff working in maternity services. Yet at the CHU and CMA levels, the workforce would able to cope with the workload because students were not recorded in this study.The other occupational categories, such as physicians, frequently handle outpatient consultations and assist with other health care activities, as needed. In most of the observed activities , physicians split their time with nursing and midwife staff. Occasionally, some of the activities mentioned here were also performed by other occupational categories, but at proportionally different amounts of times. It was straightforward to determine which activities were performed by physicians and other occupational categories, but it was difficult to separate the activities performed by nurses and midwives, especially at the CHR and the CMA. It was noted that the lower on the health care pyramid, the more the entry- and mid-level health care staff perform the activities in the wards. This differs from the upper level, where specialist physicians are surrounded by every level of health care staff.There are significant differences between the time taken to accomplish some activities at the CHU, CHR, and CMA. This is explained by the fact that some activities, such as family planning consultation and family planning counseling, are done simultaneously at the CHR and CMA, but independently, in two different rooms, at the CHU.In other studies and WHO documents, nurses and midwives were treated jointly and even as a single entity in the estimate of health human resources ,16,18.Even though nurses and midwives are the most numerous among the occupational categories, there is a shortage of official staff at the CHU and CMA, unlike at the CHR, where they currently have enough staff to perform their activities. According to the results, health workers work under considerable pressure at the CHU and CMA with WISN ratios of 0.68 and 0.79, respectively. There is a tendency to say that health workers are overstaffed in big cities, but we find that these numbers are offset by facility and by demand, especially in these two major facilities  after this subsidy policy. In one of our previous studies, we showed that these numbers were rather concentrated at the CSPS level . The worOur results from the CHU and CMA are comparable to the results found in other low-income African countries, such as Tanzania, where a severe shortage of perinatal care workers (nurses and midwives) was identified in all Dar es Salaam health care facilities .Surgical and anesthesia assistants at the CMA were not considered maternity staff, as they perform activities throughout the health care centre, not solely in the maternity ward, as is the case at the CHR and CHU.Midwives and nurses spend more time performing deliveries at the CMA than at the CHU. This is explained by the fact that at the CMA, patients arrive at the beginning of labour. They therefore require care over a longer period, unlike at the CHU, where most patients were referred by lower levels of care  when they are already in labour. Furthermore, this difference is not statistically significant across facilities (p=0.080).Burkina Faso’s policy to apply a maternal health care subsidy (SONU) is beneficial because it responds on the population’s needs. Indeed, it is desirable to also focus on the provision of care, such as hiring additional staff or simply allowing a better geographical distribution of health workers for care. However, Burkina Faso now has a good assisted delivery rate. This is a great achievement for the country. Although the country is experiencing problems in human resources, it is better placed compared to some countries in the sub-region such as Mali or Niger .It is clear that the staffing shortage can have an impact on care, which could result in difficulties in attending to obstetrical complications, less time to perform tasks, compromised quality of care, and difficulty in reducing maternal mortality.However, at the CHR, the supply of staff was much greater than the current workload. This raises the issue of geographical distribution of available human resources in health care facilities. In some African countries, this distribution was found to be very uneven between urban areas, which have a higher ratio of health care professionals, and rural areas -20. The This study should be expanded to unofficial staff to understand their place in the workload distribution, while also incorporating all staff working in maternity .The strength of this study was to know the various activities carried out by nurses and midwives, and also time to complete these activities. The limitations of this study include not taking into account the services and students who are not permanent. The numbers also vary significantly over the course of a year and few health facilities were included in our study.This study showed a shortage of nurses and midwives in two health facilities in the capital of Ouagadougou after the intervention of SONU, which confirms that there is considerable demand. However, the presence of interns, which could affect the workload at the CHU, should also be taken into consideration because a similar study done in a region of Burkina showed that there was no problem in this regard. At the Kaya CHR, there is currently enough staff to handle the workload in the maternity ward, which may indicate a need to expand the analysis to other health facilities, to determine whether a redistribution of health human resources is warranted. Although Burkina Faso is focusing on the accessibility of maternal health care, it should also be focusing on the availability of human resources and, above all, their distribution across health centres to achieve MDG 4.CHR: Regional hospital - Centre Hospitalier Régional; CHU: University hospital - Centre Hospitalier Universitaire; CMA: Medical Centre with Surgical Unit - Centres Médicaux avec Antennes chirurgicales; CSPS: Health centre - Centre de Santé et de Promotion Sociale; DFATD: Foreign Affairs, Trade and Development Canada; EmONC: Emergency obstetric and newborn care; GHRI: Global Health Research Initiative; HDSS: Health and Demographic Surveillance System; IDRC: International Development Research Centre; MDGs: Millennium Development Goals; MVA: Manual vacuum aspiration; SONU: Maternal health care subsidy - les soins obstétricaux et néonataux d’urgence; WISN: Workload Indicators of Staffing Need; WHO: World Health Organization.The authors declare that they have no competing interests.SK conceived of the idea and participated in the design of the study. AL participated in the design of the study, coordinated data collection, and conducted the analysis and writing of the manuscript. VR helped develop the idea, study design, writing, and interpretation. All authors read, commented on, and approved the manuscript.


One of the key principles in the recommended standards is that emergency medical service (EMS) providers should continuously monitor the quality and safety of their services. This requires service providers to implement performance monitoring using appropriate and relevant measures including key performance indicators. In Asia, EMS systems are at different developmental phases and maturity. This will create difficultly in benchmarking or assessing the quality of EMS performance across the region. An attempt was made to compare the EMS performance index based on the structure, process, and outcome analysis.The data was collected from the Pan-Asian Resuscitation Outcome Study (PAROS) data among few Asian cities, namely, Tokyo, Osaka, Singapore, Bangkok, Kuala Lumpur, Taipei, and Seoul. The parameters of inclusions were broadly divided into structure, process, and outcome measurements. The data was collected by the site investigators from each city and keyed into the electronic web-based data form which is secured strictly by username and passwords. Generally, there seems to be a more uniformity for EMS performance parameters among the more developed EMS systems. The major problem with the EMS agencies in the cities of developing countries like Bangkok and Kuala Lumpur is inadequate or unavailable data pertaining to EMS performance.There is non-uniformity in the EMS performance measurement across the Asian cities. This creates difficulty for EMS performance index comparison and benchmarking. Hopefully, in the future, collaborative efforts such as the PAROS networking group will further enhance the standardization in EMS performance reporting across the region. In today’s health care environment, the demand for objective comparative information about the performance of health care organizations and providers has created a need for data-driven evaluation processes . As a paMeasuring quality in EMS systems is challenging . MeasuriPerformance indicators are measurement tools that should be “specific, measurable, action oriented, relevant and timely” . An indiThe structure of EMS agencies differs significantly between the Asian countries. The focus of emergency health care in most of the Asian countries relies heavily on the hospital system, with little emphasis on pre-hospital care other than rapid transport. The quality of care available at all levels appears to be directly related to the economic status of that country and the efforts of government and health authorities to identify and support the development of pre-hospital services. Countries like Singapore, South Korea, Japan, and Taiwan have a much more mature EMS setup compared to other countries in the Asian region ,17. TheyThe data collected comprises of structure, process, and outcome information of the EMS system within each of the EMS agencies. A survey form was created and sent via email to all the representatives of the EMS agencies or the medical directors of the PAROS group. Some of the survey forms were handed to the participating members by hand during the annual PAROS meetings held in various cities within Asia. Some of the data was obtained from the previous publications within the PAROS group that is adopted from cardiac arrest registry to enhance survival (CARES) study based in the USA ,19. TwenThe final response reported in this study is from seven cities  and six countries . General demography of each of the cities is as shown in Table Generally, there seems to be a more uniformity for EMS performance parameters among the more developed EMS systems. The major problem with the EMS agencies in the cities of developing countries like Bangkok and Kuala Lumpur is inadequate or unavailable of data pertaining to EMS performance. This is partly due to lack of uniform and standardized data collection. EMS agencies in Seoul, Tokyo, Osaka, Taipei, and Singapore utilize established electronic patient care record systems for all EMS cases . This alThere are many challenges in reporting the core measures of EMS performance. Variability in data storage and core measure definitions is very common. Not all countries use a systematic database registry system, especially in the developing countries. Not all data systems utilized similar data dictionaries. This resulted in disparity in the way measurements were derived. Some EMS agencies followed the measure exactly, while others had interpreted the measure more loosely. Many local EMS systems use paper pre-hospital patient care records while others use the electronic format. Abstracting information from paper forms is difficult, time-consuming, and not necessarily accurate . In contA few efforts should be considered in order to reduce the limitations in obtaining universal and uniform EMS performance measures in the Asian countries. The establishment of the PAROS in the Asian region in 2010 has significantly changed the landscape of networking among EMS experts in the region. The PAROS is a collaborative research group formed by dedicated pre-hospital emergency care (PEC) providers conducting PEC research in the Asia-Pacific region. The initial step involved the data collection pertaining to the OHCA within the Asian region. Before the study was conducted, basic data on the EMS components were obtained from the involved countries and cities, namely, the general structure, process, and outcome of the EMS system. The regional networking has created standardized definitions across the PAROS network by adopting a consensual common taxonomy and data collection methodology. This will allow consideration in an attempt to create uniform EMS measures in the region and to identify common available core measures from each country or cities. The uniform core measures will create a more meaningful EMS practice comparison and benchmarking in the region. This is a very important step to a very long journey in creating a sustainable unique, low-cost, self-funded model of a collaborative research and education network in EMS. No doubt, there will be a lot more hurdles to be faced but with perseverance and commitment this effort will succeed.EMS is an integral part of every community’s total health care delivery system. Consistent evaluation of clinical and response performance indicators are crucial components in ensuring that first response services are operating at peak efficiency. To achieve this, continuous quality improvement (QI) practices must become an essential and seamless part of normal EMS routines. Unfortunately at the present time, there is an obvious significant difference in the EMS practice of reporting in the Asian region. This creates difficulty for EMS performance index comparison and benchmarking. Data systems that are robust and agile, with the ability to report clinical indicators and performance measures, are a key tool in quality improvement activities. Hopefully, in the future, collaborative efforts such as the PAROS networking group will further enhance the standardization in EMS performance reporting across the region.


Interventions for curing most diseases and save lives of pregnant and delivering women exist, yet the power of health systems to deliver them to those in most need is not sufficient. The aims of this study were to design a participatory antenatal care (ANC) strengthening intervention and assess the implementation process and effectiveness on quality of ANC in Jimma, Ethiopia.The intervention comprised trainings, supervisions, equipment, development of health education material, and adaption of guidelines. It was implemented at public facilities and control sites were included in the evaluation. Improved content of care -immunization, health education, conduct of health professionals, and waiting time) were defined as proximal project outcomes and increased quality of care  were distal project outcomes. The process of implementation was documented in monthly supervision reports. Household surveys, before (2008) and after (2010) intervention, were conducted amongst all women who had given birth within the previous 12 months. The effect of the intervention was assessed by comparing the change in quality of care from before to after the intervention period at intervention sites, relative to control sites, using logistic mixed effect regression.The continued attention to the ANC provision during implementation stimulated increased priority of ANC among health care providers. The organizational structure of the facilities and lack of continuity in care provision turned out to be a major challenge for implementation. There was a positive effect of the intervention on health education on danger signs during pregnancy , laboratory testing , health problem identification , and satisfaction with the service . There was no effect of intervention on conduct of health professionals. The effect of intervention on various outcomes was significantly modified by maternal education.The quality of care can be improved in some important aspects with limited resources. Moreover, the study provides strategic perspectives on how to facilitate improved quality of ANC. The world possess an arsenal of interventions for curing disease and save lives of pregnant and delivering women, yet the power of health systems to deliver them to those in most need is not sufficient. World Health Organization (WHO) has agitated for an urgent need to improve the performance of health systems and developed a framework for action . It has Antenatal care (ANC) is a global health system approach to improved maternal and infant health -6 as ANCFocused Antenatal Care (FANC), applicable also in low-income countries [In 2002 the WHO released new research-based guidelines for ountries . The guiountries -24.In Ethiopia, ANC services have been characterised by unclear guidelines and a lack of training of providers in ANC ,26, and the Maternity Study, was launched in the Jimma area and the surrounding urban communities in Southwest Ethiopia, to improve the quality of care and to develop strategies for improved ANC in low-income settings. The setting and prior needs assessment, using a participatory mixed method approach, has been described previously [An ANC strengthening intervention, eviously . In brieThe objective of this manuscript is to describe the design of the Maternity Study and analyse the implementation process and effectiveness on quality of ANC. The intervention was designed to improve the quality of care by improving the content of care provided -immunization, health education, conduct of health professionals, and waiting time), which then were expected to lead to better identification of health problems and increased overall user satisfaction with ANC . The study should be seen as a contribution to knowledge base of intervention research for improved quality of ANC and improved maternal and child health in low-income settings.The intervention was developed using a bottom-up approach . In the The present study constitutes a complex intervention where a range of interrelated activities are applied to address a number of current practices amongst health care providers and pregnant women. A key concept in a complex intervention approach is to study not only the effect of the intervention, but equally analyse the process of implementation, in order to understand the mechanisms of effect . Furtherwhat works for whom, in what circumstances, in what respects, and how? And further evaluations should provide a revision of how the intervention was initially thought to work. In this process, the program theory (how the activities are expected to lead to the outcomes) becomes the unit of analysis [Pawson et al.  criticizanalysis .The present study was developed with a participatory and explorative approach. We studied the degree of implementation and adaption of activities at the different sites. Moreover, the outcome evaluation included a range of outcomes to allow for measurements of effects across a range of domains. In the result section of this manuscript the implementation process and the outcome evaluation are presented separately, however in the discussion the findings are synthesised to analyse how our program theory managed or did not manage to create changes and why. We studied the effectiveness of the intervention , where tother facilities (private clinics and clinics outside the study area) constituted the control group. According to routine registrations 2009–10, the following annual numbers of women were seen for ANC at the respective facilities: JUSH, 2144; Jimma Town, 1828; Higher 2, 1584; Serbo, 782; Agaro, 1052; and private facilities in Jimma town, 439 .Health facilities in Jimma and Serbo town (17 km from Jimma) were chosen as intervention sites: Jimma University Specialized Hospital (JUSH), Jimma Town Health Centre, Higher 2 Health Centre and Serbo Health Centre. Women attending ANC at Agaro Health Centre (45 km from Jimma) and The timing of the implementation of activities and data collection are shown in Table Intervention health centres developed a prioritised list of needed equipment for ANC and delivery services, based on which, basic medical consumables and equipment were donated by the research project. On the list were gauze, cotton, gloves, syringes, personal protection clothing for health professionals assisting deliveries, bed sheets, blankets, oxytocine, disposable umbilical ties, blood pressure monitors, fetoscopes, stethoscopes, oxygen concentrators, suctions, and stand lamps.Laboratory supplies and equipment were provided to make the routine tests required for ANC available at all facilities. The expected laboratory tests for three months of ANC provision were estimated, and the needed materials were given. At health centres, where the laboratory facilities were available prior to the intervention, women were charged a user fee. With the donations, we started a revolving fund to avoid implementing free services that could not be sustained. The women were asked to pay the following amounts for tests: Haemoglobin 3 Ethiopian birr (ETB) (0.17 USD), syphilis 3 ETB (0.17 USD), blood group and Rh status 4 ETB (0.23 USD), urine dip stick  3 ETB (0.17 USD). The health centres were made responsible for reimbursing these fees to new laboratory reagents. The laboratory technicians were given a two-hour refresher course at their own facility, which ended by displaying posters with the laboratory procedure for these tests.The amount spent on equipment, consumables and laboratory supplies was 56.600 ETB (4.488 USD) at Jimma Town Health Centre, to 46.400 ETB (3.680 USD) at Higher 2 Health Centre, and 36.800 ETB (2.918 USD) at Serbo Health Centre.All health professionals at intervention sites participated in a training package. The training aimed to improve their ANC skills but also to involve them in adapting the WHO FANC guidelines. The initial training consisted of a full-day seminar, where the outline of the new guidelines was prepared. Subsequently, the health professionals in charge of ANC received on-the-job supervision in their facilities. Six months later a subsequent two-day seminar was conducted involving all health professionals. The guidelines were revised and training was given on management of complications during pregnancy, especially local standards for referral.Privacy guidelines were developed to set up principles for the interaction with the clients: If many health staff members or students were present during the consultation, one person should be in charge. The door should be closed, and walking in and out should be limited. The woman should be informed about the procedures undertaken, and mobile phones should be on silent mode.A health education folder was developed based on the first training. It was written in two local languages, and it addressed the specific needs and practices of the surrounding communities with drawings from a local artist. The folder was intended as a job aid for the health professionals. It had to be handed out to the women at the first ANC visit, allowing for reflection with the partner and relatives at home. The material covered 1) an explanation of ANC service, 2) healthy behaviours during pregnancy, 3) danger signs during pregnancy, 4) birth preparedness, and 5) healthy behaviours after delivery. After the folder had been implemented and the use of it monitored, a pictogram on danger signs during pregnancy was developed, because the health professionals expressed that the folder was too detailed for the illiterate women. The pictogram was inspired by a Jamaican project . Both thThe ANC guidelines included 1) a description of timing and content of the four visits 2) a list of pregnancy conditions and management procedures both for the health centres and the referral hospital level, 3) a list of the common laboratory test recommended to pregnant women, how to interpret the results and treatment regimens 4) the privacy guideline and 5) the health education material. The guidelines were distributed to all intervention facilities.Monthly supervisions at the health centres were conducted throughout the intervention period by a research nurse employed by the research project. During the supervisions, the ANC consultations were observed and the laboratory services were monitored. All issues were written in a supervision report. The visits were unannounced, and, when possible, the health centre leader would take part in the supervision. If the health centre leader was not available, the supervision report would be discussed with the health centre leader subsequently, to facilitate local ownership and reflection on identified challenges and possible solutions.In Table The evaluation of the implementation process at the facility level was based on monthly supervision reports.Questionnaire surveys were conducted at the household level, before and after the intervention amongst all women in Jimma, Serbo, and Agaro who gave birth within 12 months preceding the interview dates. Thus, the women included in the after-intervention survey were pregnant either concurrently or just after completion of the intervention (except for the supervisions). Local guides assisted to identify eligible women on a house-to-house basis in the communities. In both surveys, it was also an inclusion criterion that the women had been living in the study area for at least a year. We planned to include 300 women from the control groups and 700 from the intervention group as this would enable us to detect a difference in the proportion satisfied with the service from 50 to 60% or more with 80% power at a 5% significance level.Amharic and it was pilot tested. Data were collected by trained female data collectors under daily supervision. Data were double entered by trained data entry clerks.The questionnaire was designed to gain information on the women’s use and experience with health facilities during pregnancy, and the project outcomes were measured as reported by the women. The questionnaire was identical before and after intervention (except that place of laboratory testing was not included in the baseline survey). In order to diminish recall bias, the women were asked to remember if they had received the different services at least once during ANC. Further, the questions on blood testing were divided; 1) HIV test and 2) other blood analysis . The questionnaire was in Logistic mixed-effect regression was used to compare the change in quality of care before and after the intervention period at the intervention sites, relative to the control sites. Therefore, the regression analyses included the main effect of variable A (before- or after-intervention survey), variable B (intervention or control site), and the interaction-term between A and B. Facilities were included as random effects. Possible confounding by maternal education, parity, and marital status was assess by a comparison of crude and adjusted models. Log rank test was used to assess if the effect of the intervention was modified by maternal education. Analyses presented for 1) a full model, where all sites were divided into control and intervention sites, and 2) a submodel, in which only the intervention health centres were included . Sensitivity analyses were performed to study if the effect of intervention was altered when the control group consisted only of women attending ANC at Agaro Health Centre (other facilities excluded). All analyses were performed in STATA 11.0 .Ethical permission was obtained from the Jimma University Ethical Review Committee of the College of Public Health and Medical Sciences, and permission to study and intervene on the practice at health facilities was obtained from the relevant town and zonal health bureaus as well as the hospital administration. All informants were ensured anonymity and confidentiality, and they gave informed consent after appropriate explanation of the study objectives.During the 12 months of supervision, the ANC providers seemed to gain increased interest in providing ANC: The supervisions gave continuous attention, addressed local problems, created a team spirit, and stimulated increased priority of the programme. At all sites, progress accelerated after the first two supervisions, and, without the supervisions, it is likely that the guideline and materials developed at the trainings would not have translated into practice. It was difficult to include the health centre leaders in the supervisions, partly because the supervisions needed to be unannounced, in part because ANC service was a low priority. Across the facilities, rotating ANC responsibility led to lack of continuity and sense of responsibility, and this turned out to be a major challenge for implementation.Initially there were problems with the privacy guidelines and the health education at two health centres, however during the supervisions solutions were identified and these sustained during the intervention period. At the remaining health centre sufficient continuity was never achieved and therefore adherence to the guidelines was not consistent. The high number of health professional students was identified as the main obstacle for lack of continuity. At the hospital, the health education leaflet was relatively well introduced, but the privacy guidelines, registration procedure, and four-visit model was not successfully implemented. Supervision of the medical interns, who worked on rotation, should have been done by ANC nurses together with the managing obstetricians. However, the nurses did not have enough authority and the managing obstetricians were too busy to give priority to this work. The lack of continuity and low priority of ANC made it difficult to implement the new guidelines, and the intervention was stopped prematurely in March 2010. Subsequently, the only activity involving the health professionals at the hospital was the second training.Throughout the intervention period, the provision of the recommended laboratory tests was a challenge. At the two health centres without laboratory facilities for ANC (but HIV tests) the equipment and the revolving funds were introduced. In March 2010, the Ethiopian Ministry of Health made it free for all women to take the ANC tests, and the revolving fund of our study ceased. Initially, the user fee was the main barrier, whereas during the free services, supply was the main challenge. Further, problems with power cuts, demotivated laboratory technicians , and poor communication between laboratory and ANC staff were evident.Of 1399 women found eligible, 1357 (97%) consented to participate in the before-intervention survey. Similarly, of 2275 women found eligible for the after-intervention survey, 2262 (99%) consented to participate. In the after-intervention survey, more women were identified as eligible in Jimma town and their mean age was slightly higher, whereas no difference was found in other background variables . When the folder/pictogram was given to the women the likelihood that health education would be provided during service was more than double than if no folder/pictogram was given , TT immunization , and problems identified .In the result section we have analysed the implementation process and the outcomes of a complex, health system strengthening intervention for improved ANC in a low-income setup. In the following we will synthesise the main findings and illuminate the mechanisms of the intervention to study what worked where, under which circumstances and contextualize the findings with previous insight in the field.The intervention was successful in increasing the health education provision during ANC, maybe because the material was developed together with the health professionals to address needs and solutions that were applicable in the specific context. The material served as a job aid, and thus the staff used it to remember and prioritise topics. The effects of the intervention on coverage of health education topics were modified by educational level and reflected whether the women received the pictogram about danger signs (most likely women with no education) or the folder covering more topics (most likely women with education). Thus, the differential pedagogical means to health education according to the needs of the woman seemed useful in this setup. The after-intervention survey results revealed that the attitudes towards the folder/pictogram were positive and that the introduction enhanced the understanding of the health education. Health education and counselling are essential components of the international guidelines for ANC, but time allocated to these activities is often short ,41,42, aGood health education is also related to the user–provider interaction. It has been shown elsewhere that health care provision based on empowerment and empathy is associated with good health outcomes ,45. A reThe intervention had positive effects on laboratory tests other than the HIV test. The free ANC laboratory services introduced by the Ethiopian Ministry of Health during the study could have led to the increased testing at both intervention and control sites. After the intervention, more women from the control site in Agaro went to private facilities for testing (30%) than women from intervention sites (10%) did, indicating better performance by the intervention facilities. However, there was no information on these proportions in the before-intervention survey, and thus it is not possible to study changes in laboratory coverage at the public facilities. Previous studies on quality of ANC services in Sub-Saharan Africa have also documented nonexistent or poor coverage and low-quality laboratory facilities for medical conditions other than HIV ,23. The The intervention had a positive effect on health problem identification during ANC service, which is a precondition for providing treatment for pregnant women suffering illness and infectious diseases, which is known to improve also infant health and survival . The incThe measurement of satisfaction with care was relevant as an overall assessment of the women’s evaluation of the care received, and it was expected to improve when the content of care improved. However, satisfaction has previously been shown to be dependent on the expectations towards care ,50. In tAt the national level in Ethiopia, the ANC services, including coverage, number of visits, and content of care, seemed to have improved from 2005 to 2011 ,52. ThusThe changes in quality of care in this study were measured as reported by the women, and were thus measures of perceived content and quality of care. As mentioned previously, perceived quality of care and satisfaction should be interpreted with care . HoweverIt is obvious that when a woman reports that blood pressure was measured, we still have no information about the blood pressure value or the actions taken, if abnormal. Nevertheless, the different contents of care are important precursors for correct treatment, and the changes in health problem identification are therefore considered important.The evaluation only showed positive effects of intervention on some, but not all outcomes, and this specificity of effects could be understood by the process of implementation. For example, the intervention was not fully implemented at the hospital, and when this site was included in the analysis, only few outcomes were significant. In contrast, at the health centres, the intervention was better implemented, and the analyses with only these show stronger effects.To conduct a survey of this size is a logistical challenge in a setup, with no universal registration of deliveries and poor registration of addresses. Prior to data collection, it was not possible to get a reliable estimation of how many pregnant women there were in these urban areas. The before- and after-survey data were collected according to similar protocols, but the experiences from the before-survey had possibly improved the skills of both the research team and the data collectors. Except for the difference in number included, especially from Jimma, there was no important difference in background characteristics in the before- and after-intervention survey.The real-life setting of this study makes it likely that the findings could also have relevance in other contexts, however, not without adaption and user involvement and ownership.This article reported on the design, implementation, and effectiveness of a health system strengthening intervention for improved ANC in a low-income setup. The intervention was successful in improving both the coverage of laboratory tests and health education. The more distal outcomes of health problem identification and overall satisfaction were also improved. However, it was not possible to show any improvements in the user–provider interaction  and here more comprehensive changes in priorities are needed. If these components could be improved, we believe the effects of the intervention on the distal outcomes could have been stronger.The health system in Jimma is dealing with competing needs and priorities with very limited resources. Provision of ANC service has not been given high priority, and the inadequate quality of ANC put women’s trust in the health system at risk . With th