Here we propose the use of a large and multidisciplinary database of scientific curricula in Brazil, namely, the Lattes Platform, to study patterns of scientific production and collaboration. Detailed information about publications and researchers is available in this database. Individual curricula are submitted by the researchers themselves so that coauthorship is unambiguous. Researchers can be evaluated by scientific productivity, geographical location and field of expertise. Our results show that the collaboration network is growing exponentially for the last three decades, with a distribution of number of collaborators per researcher that approaches a power-law as the network gets older. Moreover, both the distributions of number of collaborators and production per researcher obey power-law behaviors, regardless of the geographical location or field, suggesting that the same universal mechanism might be responsible for network growth and productivity. We also show that the collaboration network under investigation displays a typical assortative mixing behavior, where teeming researchers ( Nowadays, scientific collaboration is understood as extremely valuable, as it integrates skills, knowledge, apparatus and resources, allows division of labor and the study of more difficult problems, including interdisciplinary ones. It also brings recognition and visibility and increases the network of contacts of the researchers involved The Internet and the fact that traveling became substantially less costly have facilitated international collaborations. Still, geographical constraints affect the dynamics of research The social nature of collaboration Although in Lotka's analysis Nonetheless, the definition of research collaboration is problematic due to the subjective understanding of its essential ingredients http://lattes.cnpq.br), an online database maintained by CNPq , a government agency that finances scientific research in Brazil. It contains the curricula of almost all researchers in Brazil and their collaborators abroad, as well as information concerning their research groups. The Lattes Curriculum became the standard national scientific curriculum in Brazil, and compulsory for those requiring financial support from the Brazilian government. The curricula present detailed information concerning the researcher, including, but not limited to, full name, gender, professional address, academic titles, field of expertise and list of papers. Researchers are classified in 8 major fields: Agricultural Sciences (Agr), Applied Social Sciences (Soc), Biological Sciences (Bio), Exact and Earth Sciences (Exa), Humanities (Hum), Health Sciences (Hea), Engineering (Eng), Linguistics and Arts (Lin), and Others (Oth). Most information in the curriculum are provided by the researcher themselves, for example, their list of publications.Despite the numerous studies about scientific production, citations and collaborations found in the literature, it is difficult to compare these variables as the databases used in these studies are usually unrelated. Another problem is the small number of samples, due to a low number of respondents in questionnaires or data used only from a specific journal. To analyse the big picture is paramount to work with a dense information database. Here, we used data from Lattes Platform .Due to possible typographical errors From the string matching results, we build a unweighted bipartite network We focus our study on a projection of the bipartite network onto The parameter for the exponential functions were estimated by logarithmic transformation and subsequent linear regression. For the power-law with exponential cutoff distributions, For power-law functions, we perform a logarithmic transformation followed by linear regression to calculate the power-law exponent.TCN includes 275,061 researchers, with 90.4% belonging to the largest component. The total number of identified papers written in collaboration is 623,984, the number of collaborations is 1,095,871 and the network comprises all 8 major fields used by the Brazilian agency CNPq to classify researchers.The extracted papers have publication date extending for several decades, the oldest paper in collaboration being from 1949. By analysing the growth of the network, we show in To analyse the evolution of the largest component, we construct networks with a limited time window spanning five years centered in 1990, 1995, 2000, 2005 and 2010. This was accomplished projecting the bipartite network linking researchers connected to papers published only within the respective time window. A commendable initiative of the Brazilian government is to award scholarships to distinguished researchers among their peers. Doctorates may apply for several levels of scholarship. Applications are judged by a committee based on requestor's project, scientific contributions, participation as a journal editor, among other criteria. These scholarships correspond to a bonus payment in addition to their base salary. The scholarship information is included in the CV by CNPq, not by the researcher, and we obtain the list of researchers awarded when parsing their curricula. For comparison with the TCN, we built a collaboration network with only these researchers, projecting the bipartite network The clustering coefficient A relevant question which naturally arises is how the scientific productivity and collaboration statistics of researchers awarded with scholarships differ from regular researchers. Studying our database, we find that researchers in the SCN represent less than 5% of the researchers in the TCN but contribute with 20% of the production. They are in average more than five times more productive, as measured by publication output. Also, SCN is more cohesive than TCN, as measured by the size of the giant component. To determine whether these characteristics are cause or consequence of their scholarship is not our aim, but previous research on collaborations strategies indicate that those with higher grants are more likely to have more collaborators The assortativity coefficient It is inviting to verify if the production of researchers on Lattes Platform obeys Lotka's Law. As shown in With this database, we can study the time evolution of the cumulative collaboration network by analysing different groups of papers that have been published within a specific range of years. We show in We can use the professional address information included in the curricula to study the differences of collaboration profile due to geographical location. As shown in Finally, the way researchers from different fields collaborate can also be investigated with the data downloaded from the Lattes platform. ield see ,  and , In summary, we have used the Lattes Platform, which contains detailed and unambiguous data of approximately 2.7 million curricula of researchers, as a database for analysing research collaboration in Brazil. It has the advantage of displaying individual curricula, allowing us to study collaborations in a mix of a paper-based approach and questionnaire data.We therefore built collaboration networks including all researchers data from Lattes Platform as June 2012, and found that the network has grown exponentially for the last three decades. The calculated values of the assortativity coefficient and the average nearest-neighbor degree indicate that the networks display assortative mixing, where researchers having high Finally, we confirmed the validity of Lotka's Law for researchers working on different states of Brazil and found substantial correlations between


As an integral part of the European Commission (EC) Imaging Referral Guidelines Project a 1.5-day workshop was held in Vienna on 20–21 September, 2012. At this workshop, models and good practices regarding the appropriateness and use of imaging referral guidelines (Guidelines) in Europe and worldwide were presented, together with the results of a survey of Guidelines in Europe. The latter included ideas, innovations and wishes for future Community action.Stronger measures should be taken by the EC and the European competent authorities for making Guidelines available and used in all EU member states.Evidence-based Guidelines with separate guidance for children should be issued or endorsed by a trusted European organisation.Educational initiatives and electronic requesting in connection with clinical decision support (CDS) systems should be used to improve the implementation of Guidelines.Monitoring of Guidelines implementation and use should be by clinical audit, particularly external audit, but also by local/internal audit.Recommendations for future Community action:The online version of this article (doi:10.1007/s13244-013-0299-8) contains supplementary material, which is available to authorised users. Over 60 participants attended the workshop in Vienna , with reThe programme included over 30 talks, divided into five sessions. Each session allowed ample time for discussions, with enthusiastic participation from the floor.The Workshop’s sessions each dealt with a specific subject within the area of Imaging Referral Guidelines in Europe.Scene Setting, served as an introduction to the topic and gave an overview on the current status of Referral Guidelines in Europe. Experts from International Organisations such as the WHO and IAEA presented their perspective and invited speakers from Europe, USA, Canada and Australia shared their views on Referral Guidelines from a national perspective. This session covered the drivers for the effective use of Guidelines, followed by the experience from a number of countries who had developed and used their own referral guidelines. The key messages included the following:Session 1, European and International Legislation related to radiation protection requires effective justification of imaging tests involving radiation based on evidence-based guidance . In addiThere is a similar process for the production and formatting of Guidelines by those countries who presented their experience –5, with There was acknowledgement of duplication of effort, with the same evidence being reviewed in each individual country reaching the same or similar conclusions. The ownership and development of Guidelines by radiologists may well have an impact on the adoption of, and compliance with, Guidelines by the end-user clinician.The evidence base is often related to comparison of differing modalities in terms of accuracy, safety and efficiency, but there are less robust data on the impact of the use of imaging to patient outcomes. Improvements in process are often considered as a proxy for outcome. Evidence in many pathways is not always present and, if present, is not always high level, widely applicable or robust. A key factor should be the overall efficiency gains within the health system by the correct use of Guidelines.The total number of Guidelines was seen as a potential barrier for adoption/adherence—often in excess of 300.It was agreed that there had to be ease of access by the end user at the time of referral, and for this reason paper versions of the Guidelines were the least effective. Web-based or App-based Guidelines should in principle improve this.Ideally Guidelines should be free to recognised referrers but this requires a stable financial model for their production.Guidelines were often regarded as being owned by the radiology department or radiologists, with no real ownership by the referrer. In some instances certain clinical specialties had developed their own guidance as part of clinical pathways which had the potential to conflict with the radiology guidelines.The increasing concern by referrers of the potential for litigation if they miss a diagnosis leads to unnecessary overuse of imaging.Patient expectations are continuing to increase and their insistence on imaging can over-ride clinical judgement.There were common challenges and barriers at the following stages: awareness, agreement, adoption and adherence. Specific barriers that were discussed included:European or International collaboration on reviewing evidence.Focusing on fewer Guidelines which are likely to have the biggest impact on safe, quality and efficient care.Involve referrer groups in the production and promotion of Guidelines.Integrating Guidelines into the electronic referral process—‘decision support’ software.Increasing dose awareness as part of the guideline process.Improved communication/promotion.Focus on improving the evidence base for the Guidelines that are likely to have the most impact.Production of internationally agreed Guidelines with the ability to customise to national/local circumstances.Education/training/audit.Potential solutions to the barriers and challenges were suggested as follows:Specific Issues, addressed the perspectives of stakeholders from paediatrics, interventional radiology and nuclear medicine, as well as representatives from radiographer societies and patient groups.Session 2, Paediatric radiologyInterventional radiologyNuclear medicineRadiological protection, both dose issues and risk as well as implementation issuesRadiography and the radiographerGeneral practitionersThe patient’s perspectiveThis session gave an excellent overview of referral guidelines from the perspectives of varying and diverse stakeholders in respect to Guidelines with input from:Not surprisingly all speakers were strongly in favour of Guidelines although each speaker saw and described their structure, design and use with differing perspectives.The key presentation relating to paediatrics stressed the greater sensitivity of children to radiation-induced effects and the longer period for effects to be manifested. These risks were far greater with CT and fluoroscopy, rather than radiography. Education is essential and the development and provision of specific guidelines for children is essential. Discussion centred on whether these should be stand-alone Guidelines or separate guidelines integrated into a unified publication.Interventional radiology is a rapidly changing field hence the need for frequent updates with practice guidelines for interventional radiology not fully implemented in current diagnostic radiology guidelines.Interventional radiologists act as clinical gatekeepers ensuring that the design of Guidelines involves multidisciplinary practice, collaboration of a large number of diverse stakeholders and robust education, not only across the healthcare community but also to patients, the public and media.It was noted that procedural risks in interventional radiology were often far higher than radiation-related risks.Nuclear medicine benefits from fewer procedures and Guidelines are committee driven and jointly written with clinical specialities. Interestingly technologists prepare parallel Guidelines. All requests are routinely checked and there is a strong culture of audit.The balance between risk and benefit is pivotal, the assessment of dose alone is not helpful and justification requires knowledge of clinical circumstances. The wording of Guidelines must be carefully chosen, especially related to possible risk of misuse, criteria and appropriateness.Key points of implementation are that Guidelines are legally required, but more importantly that they are useful, should be kept up to date, and should be used and audited and their development funded.Radiographers were portrayed as playing a key role at the interface between patient and technology, being the pivot between referrers, patients and radiologists. They are therefore a key player in the implementation of referral guidelines.A presentation from a general practitioner stressed that excellent communication between the patient, general practitioner and radiologist is essential. This will be facilitated by clear referral guidelines.The patient’s perspective stressed the need for dose information, clear communication (no acronyms), the need to be sensitive to the patient’s needs, and that the patient is a member of the team—“No decision about me without me”.The varying and diverse interpretation of “Specific Issues” demonstrates the need for a broad base of knowledge expertise and experience when designing and implementing Guidelines.Survey Feedback, dealt with the formulation and findings from the survey of Guidelines in Europe, conducted as an earlier part of the EC Referral Guidelines Project. Within this session workshop participants were given an opportunity to present examples of good practices from their own countries followed by enthusiastic discussion.Session 3, The survey feedback summarised the survey. It was web-based and involved multiple organisations from 30 countries. These included radiology and nuclear medicine societies and competent authorities. The questions were comprehensive and related to responsibilities, format, circulation and purpose of the referral guidelines. In addition, the survey participants were asked to share their views on barriers to implementation and their potential solutions. In establishing the survey, consideration was given to ease of understanding, completion and subsequent analysis.The analysis provided a wealth of data. More significant information related to awareness of the legal requirement for Guidelines (60 %) and the difference between this requirement and actual availability. There was variation as to the origin of the Guidelines in each of the countries—some were based on practice within the Member State, while others were heavily or entirely based on the publications from organisations external to their country (e.g. translation of RCR guidelines). The survey provided information on a number of key elements and factors of the Guidelines including the evidence base, the grading of procedures and the associated radiation dose.Innovations for improvement in Guideline use. Examples of innovations in use worldwide and potential solutions drawn from other areas of medicine were discussed.Session 4 dealt with The need was identified to improve Guideline development with specific emphasis on the level of Evidence (currently approximately 30 % of Guidelines are not evidence-based). Guidelines should be informed by secondary research and particularly by systematic reviews where available. There was agreement for the need of European Guidelines, either as a synthesis of several National Guidelines or developed by European bodies.An imaging specialist should be the author.Guidelines should be aimed at referrers.Guidelines should also be of use to patients and the public.Guidelines should be included in the curriculum of medical undergraduates and residents.Suggestions made for Guideline development include:The plan for the future should include developments to electronic requesting  in connection with CDS systems.The suggestion was made for developing selected high-impact Guidelines using simple, fast and transparent software, under the leadership of medical practitioners and validated in small integrated patient groups by motivated physicians.Web-based guidance and adequate resources are required. In addition, tablet and smart phone applications (apps) would be useful.Following presentations from over 30 speakers and lengthy discussions at the European Workshop, Work Package 2 of the EC Guidelines Project, a clearer picture of the current situation has emerged with a coherent message as to the way forward.Data to inform the availability of referral guidelines are not as straightforward as hoped. The survey has shown discordance of responses between some national competent authority and professional society representatives, possibly related to issues of awareness of Guideline existence and the legal requirement for Guidelines. In addition, there are issues of access to national guidelines, which may be related to format or media.In the European survey , respondA further, brief, follow-up survey to ascertain the up-to-date status of Guideline availability was planned following the Workshop to enable the current work in progress to be reflected in the final report.Workshop participants agreed that the availability of Guidelines did not equate with their use. As Guidelines cannot encompass all clinical scenarios, adherence can only be expected in up to 90 % of clinical scenarios. Participants at the Workshop reported that even where Guidelines are available, their uptake and use are limited. Reasons for this may include: sub-optimal clinician buy-in, conflicting advice from other  guidelines, and the difficulty associated with monitoring and encouraging Guideline use.Education—at all levels of training and continuing professional development. Web-based, training courses and inclusion in curricula as well as educational reminders [eminders , 9. Theseminders .CDS systems, particularly those with flexible architecture to fit existing requesting systems. Examples of CDS systems were available from many speakers and there was great interest amongst participants, both radiologists and regulators.Management systems, e.g. Radiological benefit management.Governmental incentives, e.g. Payment for performance [formance , Qualityformance .Tools which can encourage Guideline use include:Measures for reinforcement of Guidelines were considered essential to provide encouragement for use and to ensure sustained benefit. The European survey identified models, particularly external clinical audit as the preferred means, with a secondary role for local internal audit. Self-regulation , 14 by hPreference for European Guidelines, which may best be adapted and amalgamated from mature, trusted, nationally developed evidence-based guidelines. This option was considered to be more practical and expeditious than the centralised development of European Guidelines de novo. Later iterations of European Guidelines should encompass a more inclusive methodology with consensus from experts from multiple European countries.Need to include dose information in a form understood by referrers and patients.Separate section for imaging of children.Stakeholders, including referrers and patient representatives, should be involved.Recommendations based on evidence of efficacy, radiation safety and cost.Areas of agreement for Guideline development methodology which were identified from the European survey and consolidated at the Workshop are:Supplementary Material 1Scene Setting (PDF 2840 kb)Workshop proceedings, Session 1 Supplementary Material 2Specific Issues (PDF 989 kb)Workshop proceedings, Session 2 Supplementary Material 3Survey Feedback (PDF 110 kb)Workshop proceedings, Session 3 Supplementary Material 4Innovations (PDF 714 kb)Workshop proceedings, Session 4 Below is the link to the electronic supplementary material.


Between 2004 and 2008, 24 Dutch hospitals participated in a two-year multilevel quality collaborative (MQC) comprised of (a) a leadership programme for hospital executives, (b) six quality-improvement collaboratives (QICs) for healthcare professionals and other staff, and (c) an internal programme organisation to help senior management monitor and coordinate team progress. The MQC aimed to stimulate the development of quality-management systems and the spread of methods to improve patient safety and logistics. The objective of this study is to describe how the first group of eight MQC hospitals sustained and disseminated improvements made and the quality methods used.The approach followed by the hospitals was described using interview and questionnaire data gathered from eight programme coordinators.MQC hospitals followed a systematic strategy of diffusion and sustainability. Hospital quality-management systems are further developed according to a model linking plan-do-study-act cycles at the unit and hospital level. The model involves quality norms based on realised successes, performance agreements with unit heads, organisational support, monitoring, and quarterly accountability reports.It is concluded from this study that the MQC contributed to organisational development and dissemination within participating hospitals. Organisational learning effects were demonstrated. System changes affect the context factors in the theory of organisational readiness: organisational culture, policies and procedures, past experience, organisational resources, and organisational structure. Programme coordinator responses indicate that these factors are utilised to manage spread and sustainability. Further research is needed to assess long-term effects. This same need provided the impetus for organisation-wide diffusion and quality-improvement programmes that have been designed and implemented in health settings throughout the world in recent years. Examples include the Patient Care Notebook, the 100,000 Lives Campaign, the Framework for Spread in the Veterans Health Administration, and the state-wide collaboratives described by Leape et al.  These weet al. ,6. The pet al. ,8 and a et al. .et al. referred to as, the value of QICs as 'intentional spread strategy' [A recent MQC evaluation study showed that policy measures launched in the Dutch hospital sector since 2000 to overcome the lagging development of quality-management systems have been accompanied by an increase in hospital size and the further development of quality-management systems . A longitrategy' . Moreovetrategy' .In this section, MQC components and the selection procedure for MQC hospitals are described. The following terminology is used: 'programme coordinator' refers to the senior management staff members who play a crucial role in the progress and coordination of the entire programme in each hospital; 'external change agent' refers to the facilitators as well as to the designers of the MQC and its components at the unit and hospital level  level of ambition, (2) experience with multidisciplinary projects, (3) committed strategic management, (4) actors at all levels in favour of participation, (5) willingness to appoint a programme coordinator, (6) sufficient resources and manpower at all levels, (7) implementation of a new reimbursement system on schedule (Diagnosis-related groups-based system with diagnostic treatment combinations), and (8) no significant contraindications .This study focused on the first group of eight hospitals. Hospitals could apply for membership until 16 July 2004. The external change agency received 12 application forms. After the site visits, eight hospitals were selected.For the last 15 years, a validated measuring instrument has been used to measure the developmental stage of the quality-management systems of all hospitals in the Netherlands. A distinction is made in five focal areas  and four developmental stages: (1) orientation and awareness, (2) preparation, (3) experimentation and implementation, and (4) integration into normal business operations . Because1. topics included in management contracts;2. if units work with annual quality plans containing specific quality goals;3. if units are expected to report results periodically, and if so, how often and to whom;4. whether outcomes of different year-one projects are measured regularly.By the end of 2006, the programme coordinators were asked to fill out a table with the second-year spread of QIC projects over new units and patient groups in their hospitals. The questionnaire items and the dissemination table are available in an additional file.In autumn 2006, the programme coordinators were interviewed. Programme coordinators were likely to have the best knowledge of the overall implementation of the QIC projects and the support given by the organisation and the external change agency. They were consulted for information on system and process features that are considered to be relevant for systematic quality improvement and performance management ,22. The 1. How did your hospital organise the internal dissemination?2. What kind of sustainability approach was followed?3. What role did the internal programme coordinator and hospital executives play?4. Were the targets on safety and logistics realised in all the project locations?5. What is the most important lesson your hospital has learned from participation in the programme?The focus of the interviews was on the hospital's supportive structure: vision, facilities to train new teams, the position of experts, how senior management kept track of project progress, and how up-to-date outcome information was generated in the units where projects were implemented. Programme coordinators were also asked about measures taken to spread and to sustain new ways of working and results.Each interview was conducted by two researchers and lasted approximately one hour. Both interviewers made a report of the conversation. One of them made a first version of the interview report and the second interviewer checked this with his or her own transcription. After having reached agreement on the report, it was sent to the respondent who was requested to study the document and to assess whether it reflected the nature of the conversation and the topics dealt with. Based on the feedback, the interview report was corrected and finalised.et al. emphasising the need for innovation to align with the organisation's overall strategy and mission, broad-based support and advocacy (from both top and middle management), attention to human resources (training and support), and meticulous monitoring of the impact of the change [The content of the interview reports was coded. Codes were assigned, firstly, based on a study by Gustafson e change  and, sece change ,12.From the perspective of organisational learning, the level of success of year-one projects was considered relevant to the second-year dissemination and received the code 'previous success'.e.g., training facilities, time, means, reward, attention, and advice) to enable the implementation of current and new project series. 'Information system' encompasses systems that provide timely, up-to-date, and accurate information. Cultural aspects--like the perceived relevance of shared values and beliefs of organisation members regarding the programme, the organisational strategy, or particular QIC projects--were coded 'culture'. 'Structure' includes those characteristics of the organisational structure that are essential to manage the programme.Specific safety and logistics targets explicitly displayed or formalised as norms, performance measurement, and reporting or feedback of results were coded, respectively, 'norms', 'measurement', and 'accountability'. 'Support' involves the supportive measures in place . The committee gave approval for the study to be performed. The study does not fall within the scope of the Medical Research Involving Human Subjects Act.Seven of the eight programme coordinators filled out the questionnaire. They all reported that production agreements and prevalence of pressure ulcers are part of the management contracts. Six of them mentioned patient satisfaction surveys and implementation of improvement projects. The access time for clinical consultation in days was mentioned by five programme coordinators, throughput times by three, and the prevalence of wound infections by two. According to the programme coordinators, most units in the first group of MQC hospitals had annual plans containing patient safety goals (mentioned seven times), efficiency goals (mentioned six times), patient satisfaction and clinical outcomes (mentioned four times), and service quality (mentioned twice). On average, all seven hospitals made it compulsory that their units inform the executives four times a year range 3 to 12) about the level of norm compliance. The outcomes of all first-year QIC projects were measured regularly--except for operating theatre productivity, since these teams were not given a measuring instrument  was mentioned by the programme coordinators.i.e., goals are realised or substantial improvement has occurred) results are likely to be made a norm for other units:Beginning with 'previous success', the programme coordinators suggested that the PDSA cycles Breakthrough projects are based on were also applied at the hospital level, with the goal of disseminating the projects over new units and checking whether results were maintained. It starts with implementing projects within a few pilot units. As soon as management is positive about the merits of these projects A second example:You need a group of enthusiastic people. Improvement topics were chosen after pilot-testing. If an approach proves to be valuable then we consider making it obligatory. I look for such topics. Take working without waiting lists. The first year was not easy. We started in one unit. When it turned out to be a success, it became part of the hospital policy. However, earlier successes do not determine everything:For the second year we took the potential of units into account. This was a bit of a puzzle. Eventually you want to implement each project throughout the hospital, but in practice, the number of projects depends on the available amount of support and the situation within the units. Baseline measurements are useful in this respect. Interviewees considered 'culture' highly relevant:It is essential to generate internal support. By communicating successes within the hospital, physicians will initiate a project. Informal contacts are important. Despite the tendency towards a system-driven push approach, MQC hospitals recognise their limits:In those cases where people or units were not flattered by this strategy, another tactic was followed in which initial goals were maintained. Some people in this hospital appreciate a strong coordinating power, others do not. We listen to them and give room to unit-specific preferences. Another coordinator added:In practice, internal dissemination is a combination of own initiative and obligation, of 'bottom up' and 'top down', and also of 'what do you want' and 'what can we offer' when it comes to supporting projects. The system-driven push approach depends on the elements norms, accountability, measurement, and information system. Coordinators provided numerous examples:Several contracts are used, illustrating what the management expects from hospital staff and how things are to be organized in detail. (...) The annual board letter is very important in this respect. Goals are formulated within the hospital and therefore relevant and legitimate. The board of executives and the chief of the medical staff provide this legitimacy. The hospital uses management contracts and progress is discussed regularly with the board of directors. Agreements are made on internal spread. Hospital units are bound by higher-level management agreements that are linked to performance contracts. This is how spread and sustainability are positioned in the organisational structure. All respondents were convinced that spread and sustainability depend on the structural measurement of performance indicators, made accessible through management information systems:Measurements are crucial. Teams must report their results, and give an indication of the time and frequency of the follow-up measurements. They will have to keep measuring the outcome indicators. The unit manager is responsible for this and the hospital management checks and ensures that it happens. A management information system is being constructed, containing production parameters and quality parameters. These are displayed on a 'dashboard'. The final indicators are decided upon together with the responsible physician. Another example:A monitoring system has been established. Pressure sores, waiting lists, pain and wound infections are measured on a weekly basis. The measurements are imported in the system. As such, the goal of institution-wide diffusion is incorporated in the strategic policy of each hospital, linked to performance monitoring:Better Faster became part of the hospital policy. Every quarter of the year data are reported. Outcomes will eventually be visualised on some sort of 'dashboard'. There are indicators for each topic. (...) The dashboard is the most important sustainability instrument. In this respect, programme coordinators stressed the importance of ICT systems and the need to further develop them:Many hospitals are not equipped for systematic data collection. The tempo in which outcome measures have become more important, is much higher than the possibilities for measurement, registration, generating informative overviews, and using them. This is not a unit level responsibility. The organisation must make sure that the required systems are available. The support element affects these systems, but support means more than that. Units should receive the resources necessary to ease the implementation:New teams are trained by those who have gained experience with a similar project, but we also provide other types of support. At the start of a project we determine how much support or training is needed and who is able to provide it. We assume there is sufficient expertise available within the hospital to support projects. Other coordinators confirmed the relevance of organisational support and approaches to reutilise knowledge and experiences:A solid support platform is necessary for dissemination. In the first year some staff members fulfilled the role of process coach. They made use of the national expertise and QIC training offer. In the second year, they served as internal experts. In the second year, the MQC training model, as ran by the external change agents, was adopted by hospitals. New teams followed internal training programmes:It works fine. Teams are formed that attend training meetings. The sessions take half a day, and are shorter than the ones given during Better Faster. The content, however, is not to be compromised. Elements we copied exactly are: the Breakthrough topics and working methods, reporting of progress, and measurement formats and instructions. The training offer depends on what people want. It requires tailoring. One of the last elements is structure. Various structural aspects have been addressed by the cited programme coordinators. The interviews point out that the hospitals intend to maintain the internal programme structure established during the programme. No signals were given that the hospitals intend to discard the approach with the elements as mentioned; they are essential to the strategy for spread and sustainability. There were, however, respondents who emphasised that the strategy is probably less suitable for some of the QIC projects:We gave all topics a formal place except for process redesign, because these projects are more difficult to realise. Process redesign is a multidisciplinary story with less direct effects for hospital staff, making each implementation trajectory more difficult. Operation theatre productivity was not included in the dissemination scheme either. We have one large team working in three geographically separated hospital locations. Staff is exchanged between these sites. i.e., goals are realised or substantial improvement has occurred) results are likely to be made a norm for other units. This is done formally by integrating the norm in the yearly policy documents that mark the start of the planning and control cycle and serve as a frame of reference for CEOs, management, and staff. Performance contracts are made with unit heads to stimulate the adoption of lessons learned from successful projects in an attempt to obtain similar results (again the left cycle). The support and accountability relations connect both PDSA cycles to each other. Units receive the means necessary to ease the implementation. On average, units report their proceedings to the management four times per annum.The current study adds insight into the mechanism by which MQC hospitals organised sustainability and internal dissemination. Within the MQC, the attention of executives and managers was linked to QIC projects at the unit level. The leadership of hospital executives did influence the extent to which the behaviour of QIC teams and physicians at the micro level was aimed at achieving MQC norms formulated at the macro level. The multilevel QIC encouraged executives to do this in two ways. Not only by--as pointed out in an earlier MQC evaluation --stimulaThis model requires that enough resources are made available and that accurate, timely data can be generated for the sake of accountability. That is to say, unit staff must be equipped to implement new working methods, and information systems should enable the organisation to track the unit's status and progress. Programme coordinators acknowledge that many efforts have been made to optimise hospital information systems. They are convinced that monitoring data are crucial to sustain results and keep the dissemination going.The elements culture, norms, measurement, accountability, previous success, support, information system, and structure are confirmed to fulfill a crucial rule within the strategy for spread and sustainability, as adopted by hospitals participating in the MQC. These elements are similar to the 'possible context factors' identified but left unaddressed in Weiner's theory of organisational readiness for change . Weiner Weiner presents his view on organisational readiness as a way of thinking, best suited for examining organisational changes where collective behaviour change is necessary in order to effectively implement the change and, in some instances, for the change to produce anticipated benefits. The successful internal dissemination and sustainability of QIC projects in units within MQC hospitals can be approached from this organisational readiness theory and its determinants earlier in the chain. The programme can be viewed as an instrument to utilise the configuration of context factors.The second-year dissemination wave consisted of almost 300 projects . This form of organisational learning connects implementation processes at the micro level to management processes at the meso level, leading to new implementation processes at the micro level one year later.The authors declare that they have no competing interests.MLAD was responsible for designing the study; acquiring, analysing, and interpreting the data; and drafting the manuscript. As project leader of the independent programme evaluation, CW was responsible for the design of the study. LV acquired and analysed MQC data. CW, LV, and PPG assisted in interpreting the results and revising the manuscript for intellectual content. All authors have read and approved the final manuscript.Questionnaire and dissemination table. A copy of the questionnaire and dissemination table used in the study.Click here for file


We aim to evaluate the effectiveness of the Good School Toolkit, developed by Raising Voices, in preventing violence against children attending school and in improving child mental health and educational outcomes.We are conducting a two-arm cluster randomised controlled trial with parallel assignment in Luwero District, Uganda. We will also conduct a qualitative study, a process evaluation and an economic evaluation. A total of 42 schools, representative of Luwero District, Uganda, were allocated to receive the Toolkit plus implementation support, or were allocated to a wait-list control condition. Our main analysis will involve a cross-sectional comparison of the prevalence of past-week violence from school staff as reported by children in intervention and control primary schools at follow-up.At least 60 children per school and all school staff members will be interviewed at follow-up. Data collection involves a combination of mobile phone-based, interviewer-completed questionnaires and paper-and-pen educational tests. Survey instruments include the ISPCAN Child Abuse Screening Tools to assess experiences of violence; the Strengths and Difficulties Questionnaire to measure symptoms of common childhood mental disorders; and word recognition, reading comprehension, spelling, arithmetic and sustained attention tests adapted from an intervention trial in Kenya.To our knowledge, this is the first study to rigorously investigate the effects of any intervention to prevent violence from school staff to children in primary school in a low-income setting. We hope the results will be informative across the African region and in other settings.NCT01678846clinicaltrials.gov  Violence against children in schools is common practice in many low-, middle- and high- income countries, and research into prevention and treatment has been outlined as a priority in the World Report on Violence against Children . ChildreWhat data do exist indicate a high prevalence of physical corporal punishment by teachers in African schools. One study of secondary schools in Alexandria, Egypt, found that nearly 80% of boys and 62% of girls incurred physical punishments, such as from being hit with hands, sticks, straps, shoes, and from being kicked . In UganExperience of violence is a well-known risk factor for ill health and for poorer educational outcomes. Those who have experienced violence as children, for example, childhood sexual abuse, are at increased risk for depression ,9, suiciSystematic reviews of interventions to prevent violence in schools provide an overview of existing research, which has largely been focused on childhood sexual abuse ,18, bullThere is a high level of interest among international NGOs, policy makers and governments both in creating child-friendly schools  and in iWe aim to conduct an evaluation of the Good School Toolkit, which is designed to prevent violence against children in schools and to improve the quality of education. The program provides information about alternative discipline techniques and teaching techniques, and employs standard behavioural change strategies  with various actors within the school setting to change violent behaviour. As highlighted previously, available data suggest that corporal punishment by teachers is a common form of violence that children experience in Ugandan primary schools. Corporal punishment is the main form of violence addressed in the Good School Toolkit. The primary objective of the Good Schools Study, therefore, is to assess the impact of the Good Schools Toolkit on children’s experiences of violence by school staff among those attending school in Luwero District, Uganda. Secondary objectives are to assess the impact of the toolkit on child mental health and educational outcomes.The Good Schools Study consists of a two-arm cluster randomised controlled trial, an embedded qualitative study, a process evaluation and an economic evaluation. The study is a partnership between the London School of Hygiene and Tropical Medicine, Raising Voices, Makerere University, and the Institute of Education. This publication describes the design of the trial component, which will measure the impact of the Good School Toolkit intervention. The trial will involve two cross-sectional surveys, one at baseline and one at endline Figure . Our maihttp://raisingvoices.org/download-good-school-toolkit/) and implementation support over an 18-month period. The Toolkit is an established intervention, which has been popular in Uganda since it was developed 6 years ago by Raising Voices. This implementation period was chosen based on Raising Voices programmatic experience and is the period during which the Toolkit is designed to produce changes in a school.The experimental group is receiving the Good School Toolkit , 72% had access to safe water , and 7.2% had access to electricity . Both in Luwero district and at a national level, subsistence farming methods were used by 68% of households, and the majority of households (97%) relied on firewood and charcoal for cooking.Luwero District was chosen because it is within the catchment areas of the implementing partners for this study. There are currently no prevalence or epidemiological data available on children’s experiences of violence or independent data on educational performance. Our study will provide new information for the District.A two-stage selection process has been employed. In the first stage, schools were randomly selected to participate in the trial. In the second stage, within each school, all individual staff members and a random sample of Primary 5, 6 and 7 students will be invited to participate in the follow-up survey.Our implementing partners are able to provide support to a maximum of 21 schools; with the inclusion of 21 control schools the total sample size is 42 schools. These were chosen to minimise selection bias as far as possible and to represent larger schools in Luwero. Using the official 2010 list of all 276 primary schools in Luwero as our sampling frame, we excluded 105 very small schools (with fewer than 40 registered Primary 5 students) and 20 schools with existing governance interventions, and then stratified the remaining 151 schools by the gender ratio of their pupils, into >60% girls, mixed, or >60% boys. From these 151 schools, we selected a random sample of 42 schools, proportional to size of the stratum. Of these schools, 100% agreed to participate in the study. Up to date lists of all Primary 5, 6 and 7 students, and a list of all teaching and non-teaching staff will be obtained from school head teachers at endline.From these class lists, up to 130 children will be randomly selected for individual interviews at follow-up, with the aim of successfully interviewing at least 60 children per school. If a school has fewer than 130 students in P5 to 7, all children will be invited for interview. We chose to focus on students in P5 to 7 students, aged approximately 10 to 14 years, because they would be better able to respond to survey questions about their experiences versus younger children. All school staff will be invited for an interview. Research teams will be in each school for 3 to 6 days; at least one repeat visit will be made to find students who have been absent for that entire period. Students deemed unable to understand the study consent form (who will be therefore unable to provide informed consent) will be excluded.During the planning stages of the study, Raising Voices staff visited Ministry of Education and Sports officials at national and district levels. At the national level, support for the study was indicated by the Ministry Of Education And Sports. At the district level meetings have been held with the District Education Officer in Luwero, who has given permission for the study to take place.Invitations to individual schools selected for participation were issued by letter from Raising Voices to school Head Teachers. These letters were followed by a visit from senior program officers from Raising Voices to explain further details about what would be involved in participating in the study. Consent for participation was sought from head teachers; of these head teachers, 100% agreed to participate.Consent for classroom level assessments, which will involve observations of whole classes, will also be sought from head teachers on the day of the assessment. At follow-up, all staff in participating schools will be invited to participate in individual level data collection by providing informed consent. An opt-out consent strategy will be used for students, consistent with other school-based research on sensitive topics in Uganda . Same asIn participating schools, each P5, P6, and P7 child will also receive a written notice to carry home to his or her parents or caregivers. At baseline, identical procedures were used, and all of these meetings and notices were held before the baseline survey and allocation took place; schools, parents, or children were not aware of whether or not their school would receive the intervention when they completed the survey.Individual students selected to participate in the survey will be approached within their school, and informed consent will be sought. The consent form will be read aloud to each child. This form contains a description of study procedures and will remind children that they do not have to participate and have the right to stop the interview at any time. Children will also be informed that if an interviewer feels a child’s safety is at risk, the interviewer is obligated to discuss the case with the District Probation Officer or with another party responsible for child protection locally.We aim to determine the impact of the Good School Toolkit on violence, well-being and educational outcomes in children attending primary school. Specifically, we will:1. Examine whether there is a difference in children’s self-reported experience of past week physical violence by school staff, in schools that receive the intervention versus those that do not (primary outcome);2. Examine impacts of the intervention on children’s educational achievement , and children’s mental health  (secondary outcomes).Preliminary analysis of baseline data has shown a prevalence of past week physical violence (primary outcome) of approximately 50%, and an estimated intracluster correlation coefficient of 0.06. If we see similar levels at follow-up, allowing for a possible loss of two schools per arm and a conservative estimate of 60 pupils per school, we will be able to detect a reduction of approximately 13% in the prevalence of reported violence in the schools receiving the intervention, with 80% power and a 5% level of significance. One other pilot trial we are aware of that evaluated an educational and behavioural change program observed a more then 50% reduction in ‘negative teacher behaviours’ as rated by classroom observers over the trial period , so we sAll questions related to violence and mental health will be asked during individual interviews with Luganda-speaking staff who receive 3 weeks of in-depth training. Individual interviews will be done using a questionnaire programmed into mobile phones (completed by an interviewer). Some educational assessments will be done on paper in groups.Students’ experiences of violence will be assessed using behaviourally specific questions about acts of violence adapted from the IPSCAN Child Abuse Screening Tool (ICAST) , and theChildren will be administered literacy, numeracy and sustained attention assessments adapted from an intervention study in schools in Kenya, involving one of the study team (Dr Allen) . The StrSchool staff will be asked about perceptions of student discipline and learning needs, attitudes about corporal punishment and violence, as well as their teaching methods, participation in school policy development and culture. Experiences of violence will be queried, using items from the WHO MCS . Mental Interviews will be conducted in open spaces, where interviewer-participant pairs are out of earshot but within sight of others, to protect confidentiality and ensure child safety. All data collected on mobile phones  will be identified only by ID number, for both staff and students. All data collected by mobile phone will be stored on a password protected databases that will be online-accessible only to senior study personnel, and backed-up daily on a password-protected laptop. Students will write their names on standard educational tests, which will be double marked and double entered into a customised database, separate from other sensitive data.Schools were allocated to receive either the intervention or a wait-list control at a public meeting with all school head teachers before the start of the 2012 September school term. Stratified randomisation was used to ensure balance in regard to key factors .Primary analysis will be carried out based upon the groups as randomised (‘intention to treat’). Results will be presented as appropriate effects sizes with a measure of precision . Clustering by school will be allowed for in all analyses. Our main analysis of the primary outcome, the prevalence of past week experience of violence, and secondary mental health and educational outcomes will be cross-sectional analyses comparing the prevalence at follow-up between the two arms of the trial. In cross-sectional analyses, baseline school level summaries will be adjusted for, where appropriate . AdditioThe trial is overseen by the principal investigator (KD) and run by a dedicated manager (JC). A data manager is responsible for monitoring data quality at baseline and follow-up data collection points, and for interim data collection on the intervention implementation. The main data monitoring concerns for a trial of this nature relate to ensuring that cases where children have been exposed to severe forms of violence are detected and that adequate steps are taken to protect them. Comprehensive referral plans have been developed for baseline and follow-up data collection . We will also collect ongoing monitoring data from schools during the implementation process; any child protection concerns that arise during the implementation period will proceed through the same referral pathway.The intervention under study is behavioural and we do not anticipate adverse events occurring as a result of the intervention itself. However we will collect interim qualitative data from representatives of school staff and also from representatives of student committees about unexpected consequences and any adverse consequences of the intervention that have become apparent during implementation. This data will be collected by telephone and in-person interviews by the trial monitoring data officer, and will be entered into a computer on a continuous basis. Any adverse events will be reported immediately to the study manager (JC), the principal investigator (KD), and the implementing partners (DN). This committee will decide upon and enact appropriate responses.In any research on violence against children, ethical considerations and child protection are paramount. We developed comprehensive protocols to ensure that children are protected and referred as necessary over the course of the study, and have received full ethical approvals from both the LSHTM (#6183) and the Uganda National Council for Science and Technology (SS 2520).Our referral strategy for children in need of support involves working within existing child protection structures for the Ministry of Education; the health, legal, and community welfare sectors, with additional support from local NGOs; and implementing study partners to ensure proper procedures are followed. In brief, all children who participate in the study, regardless of what they disclose, will be offered the opportunity to visit with a trained counsellor who is fluent in Luganda. For children who disclose more severe experiences of violence in the past week or past year, the District Probation Officer and the representative from our local Luwero partner NGO, will be informed, and will refer cases onward in accordance with local policy. Children who disclose recent sexual violence, severe physical violence, or injury, or who have otherwise urgent conditions, will be taken immediately to a health centre, and the District Probation officer and local Luwero partner NGO representative will be informed so that further follow-up may take place.The nature of case management and follow-up provided by the existing child protection system is the subject of a nested study within the larger Good Schools Study.To our knowledge, this is the first study to rigorously investigate the effects of any intervention to prevent violence from school staff to children in primary school in a low income setting. We hope the results will be informative across the African region and in other settings.Our study design has some strengths and limitations. The site for implementation of the Toolkit for this evaluation was chosen because it is within the catchment areas for our implementing NGO partners, in the same way that an area for implementation would normally have been chosen. Luwero is typical of Uganda in many socio-demographic ways. We have selected schools to be representative of larger schools in Luwero District, thus representing the majority of pupils in the District. Because of this selection procedure, our results will be generalisable to larger schools in Luwero. We intend to use qualitative and quantitative methods to identify further factors related to uptake of the intervention, and to generate lessons about generalisability to the rest of Uganda and beyond. We will focus at the level of individual schools and beyond .Our main analysis involves a cross-sectional comparison of the prevalence of past-week experience of physical violence by school staff as reported by children. We chose this design, rather than following individual students longitudinally, for several reasons. One, the Good School Toolkit is designed to work at the level of the school, and produce changes not only individuals but in school structures and governance. Thus, we aimed to primarily evaluate impact at the school level. Two, in primary school populations in Uganda, particularly towards the end of Primary school in P7, there is substantial attrition of students over time. The repeated cross-sectional design alleviates potential selection bias introduced by attrition of students between rounds of the survey.In violence research, gold-standard measures are self-reports of behaviourally specific acts; however it is likely that there will be some under-reporting because of the trauma and stigma associated with abuse . As a reIt is possible that some spillover may occur during the trial, which could lead to underestimation of intervention effects. However, this trial is designed as a pragmatic trial, and will produce estimates of the effectiveness of the Good School Toolkit under real-world programmatic conditions, rather than ideal or artificial experimental conditions. The Good Schools materials are publicly available on the Raising Voices website. It is also likely that some school staff will migrate during the course of the trial, and it is possible that some staff from intervention schools will be placed in control schools. However, only intervention schools will receive implementation support on an ongoing basis, which we think will be instrumental to intervention success. School staff will be questioned about any previous exposure to the Good Schools materials, and school-level assessments will query whether other programs similar to Good Schools have been implemented. We will include an analysis of intervention effects by level of exposure in our study outputs.Our study is one of the first trials of a violence prevention program in schools in a low-income setting. The Good School Toolkit is designed to be a low-cost method of intervention, appropriate for wide scale-up in low and middle income settings. The results of this study are awaited by the Uganda Ministry of Education and Sports, and will be of interest to other governments, donors and policy-makers within and outside the East African region.The trial is ongoing, and recruitment of individual participants for the follow-up survey is scheduled to start in June 2014 and to be complete in July 2014.ICAST: IPSCAN child abuse screening tool; WHO MCS: WHO Multi Country Study on Women’s Health and Domestic Violence against Women.Dipak Naker is co-Director of Raising Voices, and developed the Good School Toolkit.KD designed the study, supervised data collection and drafted the manuscript. EA participated in the design of the study and performed statistical analysis. JC managed the fieldwork for the baseline survey and participated in the design of some aspects of the study. CW, JP, EW and DE participated in the design of the study. DN developed the intervention, initiated the idea to do a study, contributed to the design of the study, and obtained funding for the baseline research. All authors read and approved the final manuscript.


Human papillomavirus (HPV) infection is the most common sexually transmitted infection; persistent infection with high-risk types of HPV present a major risk factor for the initiation and progression of a variety of human carcinomas including cervical, colorectal, head, and neck (HN) as well as breast carcinomas. A huge number of studies investigated and reported the incidence of high-risk HPVs in these cancers worldwide particularly in the developed countries; therefore, two HPV prophylactic vaccines against the two most frequent high-risk HPV types (16 and 18) have been developed and used worldwide. However, there are very limited studies about the prevalence of HPVs in the developing countries especially in Africa and some states of the Middle East. In this mini review, we outline the presence of high-risk HPVs in human cervical, colorectal, HN as well as breast cancers in the Syrian population, which was recently explored for the first time by a Canadian/Syrian group. Human papillomaviruses (HPVs) are considered among the major viruses associated with human cancers especially cervical carcinomas . HPVs arEarlier studies revealed that the prevalence of HPV infections in human cancers is related to specific geographic locations worldwide , 25. On ®) and (Cervarix®). Meanwhile, it is important to emphasize that there are few studies regarding the presence and distribution of high-risk HPVs in human cervical cancers in the majority of the developing countries  include neoplasias of the oral cavity, the pharynx , the larynx, and the paranasal sinuses . Head anAccordingly, the incidence of high-risk HPVs in HN cancers in the Syrian population is comparable to the prevalence of these viruses in Turkey. It is worth mentioning that Turkey is one of the neighboring countries of Syria where HPV prevalence studies have been conducted in HN; specifically, there have been four studies exploring the presence and distribution of HPVs in HN cancers using PCR methodology; these studies showed that the presence of high-risk HPVs varies from 14 to 47.6%, and the most frequent HPVs in the Turkish population are HPV types 16, 18, 31, and 33 –60. HoweIn conclusion, it is evident that high-risk HPVs are present in HN cancers in the Syrian population and their presence is comparable to HN cancers in Turkey. However, it is important to highlight that the Canadian/Syrian study was limited to an insignificant number of cases located in a single region of Syria; therefore, it is essential to perform other studies to confirm the incidence and types of these viruses in the Syrian population.Colorectal cancer is the third most common type of cancer, with approximately 1.3 million new cases diagnosed annually worldwide, accounting for about 9.7% of all cancer cases (WHO). Evidence from recent studies suggests that environmental conditions and lifestyle in addition to sequential genetic changes and possibly viral components are major risk factors for colorectal cancer . Therefoin vitro and in nude mice, respectively (Breast cancer is the most common malignancy in women worldwide; and metastatic breast disease is a major cause of morbidity and mortality in breast cancer patients. Several earlier studies reported that high-risk HPVs are present in approximately 50% of human breast cancers worldwide –21; contectively , 74. Regectively . This stectively .The Canadian/Syrian group data of HPVs in breast cancers Table  are simiIn this paper, we have reviewed the incidence of high-risk HPV infections and their associated cancers including cervical, HN, colorectal, and breast in the Syrian population. Although data of HPVs and their related cancers worldwide are evident, the incidence of HPVs in the Syrian population needs more investigations to be confirmed. The presence of these viruses in Syria is limited to one study from the same group for each anatomical site; in addition, these studies are conducted in one region of Syria, which is near the Turkish border. It is important to highlight that presently available HPVs vaccines can protect only against HPV types 16 and 18 and their associated cancers, which are the most frequent types in western countries; so far this does not seem to be the case in the Syrian population based on the Canadian/Syrian group studies [ and TablThe authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


The CanMEDS roles and competencies are being used as the framework to support the development of the Manitoba Practice Assessment Program (MPAP) designed to assess the competence of physicians practicing with a conditional license. Establishing the link between clinical practice and assessment of performance is critical in the development of the MPAP. A first step in establishing this link is to identify activities performed in actual clinical practice as well as the importance of those activities.A descriptive survey design was used to answer the research questions: (1) How do physicians rate the complexity, frequency, and criticality of CanMEDS roles? (2) What is the distribution of perceived importance scores for the CanMEDS roles? Two online surveys, one specific to family practice physicians, and one specific to specialists, were emailed to a sample of Canadian physicians.N = 88).Overall perceived importance scores were calculated for each of the CanMEDS roles. It appears that each role is considered to be at least moderately important. The Medical Expert role was ranked as the most important, followed by the roles of Communicator, Professional, Collaborator, Scholar, Manager, and Health Advocate. There were no significant differences in overall CanMEDS perceived importance scores between family practice physicians and specialists (Given that each of the CanMEDS roles is considered at least moderately important, a variety of assessment tools are needed to evaluate competencies across the entire spectrum of roles. The results underscore the importance of incorporating a multifaceted approach when developing a practice assessment program. The University of Manitoba, Faculty of Medicine, Division of Continuing Professional Development is embarking on the development of an assessment program for practicing physicians called the Manitoba Practice Assessment Program (MPAP). The MPAP is being developed in response to a need identified by the College of Physicians and Surgeons of Manitoba (CPSM). Within the Province of Manitoba, physicians commonly enter practice on a conditional license. A conditional license allows physicians who have completed post-graduate training, but have not yet achieved their Canadian certification through The College of Family Physicians of Canada (CFPC) or the Royal College of Physicians and Surgeons of Canada (RCPSC), to enter supervised practice within a defined time frame during which they are expected to complete the appropriate certification exam. There are a significant number of physicians who are in established practices and have not yet achieved final certification for a variety of reasons. The CPSM requested that the University of Manitoba, Faculty of Medicine develop a process to assess such physicians in their practice settings to enable the CPSM to make a decision about granting registration to these candidates.cloverleafdaisy, or flower, was produced by the RCPSC to visually illustrate the core roles and interconnections among the roles [The framework for the development of the MPAP is the Canadian Medical Education Directions for Specialists (CanMEDS) physician competency framework. The establishment of the CanMEDS framework is rooted in medical education with the framework originally developed to prepare physicians to thrive in our ever-changing healthcare environment . Since 1The CFPC  adopted The CanMEDS framework has been incorporated into undergraduate and postgraduate educational programs both within Canada and internationally -11. SuccThe concept of using the CanMEDS framework as a foundation for an assessment program for practicing physicians is relatively novel as the focus has been on undergraduate, postgraduate, and continuing medical education. Researchers have started to examine the importance rankings of the CanMEDS competencies; however, the context of the research focused on the applicability of the CanMEDS roles and competencies for international physicians ,9. To daIt is important that a credentialing or licensing program be able to provide empirical evidence of the link between the knowledge, skills, and attitudes tested in an examination and the activities performed in practice . AlthougThe goal of the study was to determine the importance of physician’s roles and competencies as perceived by a sample of Canadian physicians. The following questions guided the research:1. How do physicians rate the complexity, frequency, and criticality of CanMEDS roles?2. What is the distribution of perceived importance scores for the CanMEDS roles?A descriptive survey design was used to answer the research questions and achieve the study goal by asking practicing physicians to rate competencies identified as key to their practice. Ultimately, the results will be taken into consideration in the development of the assessment tools and processes that will be used by the MPAP in assessing the competence of practicing physicians.Scott’s Directories: Canadian Medical Directory via a departmental username and password. Information obtained from the directory included the physician’s first and last name, province of residence, and email address. Physicians that met the inclusion criteria, which was a valid email address, were included.After receiving ethical approval from the University of Manitoba Bannatyne Campus Research Ethics Boards (File Number: H2010:357) to conduct the study, a list of Canadian physicians residing in British Columbia, Alberta, Saskatchewan, Manitoba, North West Territories, Yukon Territory, and Nunavut was generated by the principal investigator from the online database called, Constant Contact, was used to send the electronic recruitment notice. Included in the recruitment notice was general participant information, an electronic copy of the consent form, and links to two online surveys. The principal investigator designed and sent the recruitment email to potential participants. If an email was returned undelivered, no attempt was made to resend the email. Physicians were solely responsible for reading the participant information and consent form and determining for themselves whether to participate in the study. Participants were not required to sign and return the informed consent, and completion of the online survey constituted consent to participate. In addition, a statement of consent was included on the introductory page of the online survey. Participants were provided with contact information for all of the investigators as well as the university research ethics board coordinator if they had any questions, issues, or concerns.Due to the large participant list, and to avoid the recruitment notice being identified as unsolicited mail, an email distribution service, namely, Survey Monkey. Physicians with a clinical background in Family Practice were asked to complete the survey entitled, MPAP: CanMEDS Competencies for Family Practice. Physicians with a Specialty Practice clinical background were asked to complete the survey entitled, MPAP: CanMEDS Competencies for Specialty Practice. A period of two months was given to complete the survey. One month after sending out the initial recruitment email, the principal investigator sent a follow-up reminder email to all participants. Since it was not known who completed the surveys, all participants received the follow-up reminder email. Constant Contact has an option for opting out of the email service, and those physicians that opted out of receiving any further emails after the initial recruitment email did not receive the follow-up email. Approximately one month after access to the surveys closed, overall results were sent to all participants who received the initial recruitment email and had not opted out of receiving future emails.After deciding to participate, physicians completed one of two researcher developed online surveys accessible via an encrypted subscription of The surveys were developed by the investigators applying the work of Raymond  and usinAn overall index of perceived importance for each competency (i) within its respective CanMEDS role was calculated using a multiplicative modelIi = overall importance, Fi = frequency, and Ci = criticality [where ticality -17. A coticality . Howeverticality .N = 88) consisted of 23 participants completing the family practice survey and 65 completing the specialty practice survey. Although the 3% response rate fell well short of an initial expected minimum of 10%, based on the work of Tannenbaum and Wesley [junk mail. Strategies to increase the response rate may include targeting specific physician groups through physician organizations and special interest groups, as well as distributing hardcopy surveys as opposed to using an online survey; however, hardcopy distribution will add to the cost of the survey in terms of the mail-out itself and data entry costs. A follow-up focus group to review the results may also be helpful in the validation process. Despite the low response rate, the researchers are unaware of any other work being done to attempt to weight the CanMEDS roles; therefore, the results are an important step in the continuous validation of the CanMEDS framework. Of those physicians that participated, 70 claimed to be in active practice, three referred to themselves as inactive practice, and 15 did not declare their status. Location of participants’ practice was noted as urban (n = 52), rural (n = 3), or northern/remote (n = 1), while 7 noted a combination of locations, and 25 did not note a practice location.A total of 3,294 participants meeting the study criteria received the initial recruitment email. Five hundred and sixty two emails were returned undelivered, and a total of 27 participants opted out of receiving emails from the research team over the course of the study. Thirty five participants accessed and started to complete the MPAP: CanMEDS Competencies for Family Practice survey and 92 participants accessed and started to complete the MPAP: CanMEDS Competencies for Specialty Practice survey. Scores from those participants who did not complete the majority of survey questions were deleted prior to analysis of data. The final sample (d Wesley  who concThe mean ratings of complexity, frequency, criticality, and perceived overall importance for the competencies in each role are shown in Table The finding that there was no significant difference in perceived overall importance scores for CanMEDS roles between participants that completed the family practice survey and the specialty practice survey is an important result as the research team moves forward in the development of a physician practice assessment program. The finding supports the notion that one set of comprehensive practice assessment tools can be developed to assess both family practice and specialty practice physicians using the CanMEDS roles and competencies as the underlying framework; however, additional specialty specific competencies will need to be included in specialty specific assessment tools.The findings support the work of Frank  in that The importance scores of each of the CanMEDS roles will be beneficial in determining a physician’s overall competence as assessed in a practice assessment program. It is expected that the relative importance of each of the roles will be taken into consideration when determining an overall pass or fail grade. Beyond assessment, importance and complexity scores can be used as a reference when developing undergraduate, postgraduate, and continuing medical education curricula.Interestingly, Frank  notes thAs the CanMEDS roles and competencies continue to be adopted in all facets of medicine, it is important that the framework continues to be validated. The small sample size limits the generalizability of the findings and researchers need to continue to examine perceived importance scores of the CanMEDS roles with larger samples and with specific specialties and sub-specialties. Many specialty groups certified by the RCPSC, for example, have adapted the CanMEDS roles and competencies to their own practices , and simGiven that each of the CanMEDS roles is considered at least moderately important, it is assumed that a variety of assessment tools are needed to evaluate competencies across the entire spectrum as opposed to focusing on the role of Medical Expert. Overall, the results underscore the importance of incorporating a multifaceted approach when developing a practice assessment program.CanMEDS, Canadian Medical Education Directions for Specialists; CFPC, The College of Family Physicians of Canada; CPSM, College of Physicians and Surgeons of Manitoba; MPAP, Manitoba Practice Assessment Program; RCPS, Royal College of Physicians and Surgeons of Canada.The authors declare that they have no competing interests.BJS conceived of the study, lead all aspects of the study, and drafted the manuscript. MS participated in the design of the study and surveys. RR performed the statistical analyses and drafted related sections in the paper. All authors read, edited, and approved the final manuscript.BS: RN, BN, MScN, EdS, PhD University of Manitoba, Faculty of Medicine, Division of Continuing Professional Development, Program Advisor Manitoba Practice Assessment Program, Director of e-LearningMS:MD, FCFP University of Manitoba, Faculty of Medicine, Division of Continuing Professional Development Director, Clinician Assessment ProgramsRR: PhD University of Manitoba, Faculty of Education Department of Educational Administration, Foundations, and Psychology Associate Professor