Polypharmacy is common among older adults and it represents a public health concern, due to the negative health impacts potentially associated with the use of several medications. However, the large number of medication combinations and sequences of use makes it complicated for traditional statistical methods to predict which therapy is genuinely associated with health outcomes. The project aims to use artificial intelligence (AI) to determine the quality of polypharmacy among older adults with chronic diseases in the province of Québec, Canada.We will use data from the Quebec Integrated Chronic Disease Surveillance System (QICDSS). QICDSS contains information about prescribed medications in older adults in Quebec collected over 20 years. It also includes diagnostic codes and procedures, and sociodemographic data linked through a unique identification number for each individual. Our research will be structured around three interconnected research axes: AI, Health, and Law&Ethics. The AI research axis will develop algorithms for finding frequent patterns of medication use that correlate with health events, considering data locality and temporality (explainable AI or XAI). The Health research axis will translate these patterns into polypharmacy indicators relevant to public health surveillance and clinicians. The Law&Ethics axis will assess the social acceptability of the algorithms developed using AI tools and the indicators developed by the Heath axis and will ensure that the developed indicators neither discriminate against any population group nor increase the disparities already present in the use of medications.The multi-disciplinary research team consists of specialists in AI, health data, statistics, pharmacy, public health, law, and ethics, which will allow investigation of polypharmacy from different points of view and will contribute to a deeper understanding of the clinical, social, and ethical issues surrounding polypharmacy and its surveillance, as well as the use of AI for health record data. The project results will be disseminated to the scientific community, healthcare professionals, and public health decision-makers in peer-reviewed publications, scientific meetings, and reports. The diffusion of the results will ensure the confidentiality of individual data. Medications represent a large proportion of health care spendings –7. TheirNonetheless, there are circumstances in which the prescription of multiple medications is appropriate and leads to improved health outcomes. Distinguishing appropriate and inappropriate polypharmacy is an important and complex issue that is difficult to tackle . First, Identifying what combinations of medications or trajectories of treatment are associated with health outcomes would allow the development of specific polypharmacy indicators for public health surveillance , 22. SucDetermining health outcomes associated with polypharmacy requires considering concurrent and sequential use of multiple medications, duration of treatment, medical and sociodemographic characteristics of individuals. This type of analysis of large amount of complex data is difficult to perform with traditional statistics, but it can be accomplished with artificial intelligence (AI) methods.However, using AI in large health administrative data may pose ethical challenges, such as those related to re-identifying individuals or using the data sparingly, that is to restrict attention only to the variables necessary to answer the research question. Moreover, the polypharmacy indicators that are created must neither marginalize nor discriminate against any specific population. Taking into consideration the ethical and social aspects involved will be beneficial for researchers, clinicians, decision-makers, and for the population in general.The general objective of the research program is to use AI within an ethical framework to develop polypharmacy indicators in older adults for surveillance and clinical practice in the province of Quebec, Canada.AI axis: Define and detect polypharmacy in administrative databases to identify frequent combinations that correlate with health outcomes.Health axis: Develop polypharmacy indicators in older adults to carry out polypharmacy surveillance in public health and to guide clinicians.Law&Ethics axis: Explore the ethical and social acceptability aspects related to the use of AI for the development of polypharmacy indicators.Three interdependent research axes Fig.  will be We will use the data from the Quebec Integrated Chronic Disease Surveillance System (QICDSS). This database was developed and is managed by the Quebec National Institute of Public Health  for the surveillance of chronic diseases . It contThe three research axes described herein will closely collaborate to achieve the objectives of the project. As shown in Fig. The aim of the AI axis is to define and detect polypharmacy associated with health outcomes. The definition of polypharmacy is expected to incorporate several medications, their type, dosage and duration of use, or specific combinations of medications. For example, based on the QICDSS database, the AI axis will discover medication combinations that are associated with an increased risk of death, hospitalizations or gradual deterioration of the patient state . The main challenge will be to identify clinically relevant correlations between the addition of the new medications and health events among the millions of observations, excluding multiple accidental correlations and coincidences that analysis of such a dataset will inevitably reveal.Objective 1.1 aims to define inappropriate polypharmacy using frequent pattern detection algorithms from big datasets. It involves discovering combinations of medications that will allow prediction of subsequent health events such as hospitalization or death. As benchmarks, we will evaluate algorithms such as Apriori , FP-GrowObjective 1.2 will explore the social dimension of the data. Some subgroups of the population may be more sensitive to specific medication combinations and their outcomes, for example based on their age or medical conditions. The challenge will consist in finding an appropriate representation of individuals allowing them to be partitioned depending on demographic or geographic characteristics. This entails integrating heterogeneous individual-level data that contain attributes that are numeric , binary , or that describe temporal dimension . Deep neural networks have previously been proven to learn patient representation from electronic health records and will be considered for this task of clustering . The freObjective 1.3 will focus on the challenges related to the use of AI strategies in the healthcare, more specifically on the problem of explainability. While demonstrating better predictive performance, the results obtained with neural networks are difficult to explain, which is a limiting factor when one needs to justify why a treatment is being recommended to a patient. Moreover, the complexity of the patterns discovered in the previous objectives and the potential presence of confounding factors  can makeData preprocessing: The data are currently in the form of relational database tables that contain institutional codes. It will be necessary to transform them into groups of items with different levels of granularity of detail. The institutional codes will be arranged in a hierarchical or in ontological manner to allow algorithms to find similar health conditions and medications with the same mechanisms of action or active ingredients. The data preprocessing is a prerequisite for the proper functioning and success of the algorithms of the three objectives. This work will be carried out in accordance with FAIR principles [Reusable) and inteReusable), 42. By The Health axis aims to develop polypharmacy indicators that will be useful for population-level surveillance in order to guide health promotion and disease prevention interventions, and to support clinical practice. Thus, the Health axis will be responsible for translating the information generated by the AI axis into relevant and understandable indicators that will be socially acceptable through recommendations of the Law&Ethics axis Fig. .Objective 2.1 will define health outcomes that reflect a continuum of changes in health status. Since deaths, and to a lesser extent, hospitalizations, are relatively rare events, only a small portion of the population will be affected. We will identify several types of metrics, such as frailty indices or composite health outcomes  that will be useful to define various intermediate stages between states of good health, illness, and death. This approach will enable evaluation of the health outcomes for all individuals. In addition, we will consider institutionalization in the long-term care facilities as one of potentially “negative” health outcomes. The defined continuum of health outcomes will be transmitted to the AI axis and will be used to validate previous algorithms or create new ones, in order to identify polypharmacy patterns and characteristics associated with said outcomes.Objective 2.2 will interpret the frequent patterns and deep representations discovered in the AI axis and transform them into polypharmacy indicators. The Law&Ethics axis will ensure that these indicators are socially acceptable, fair, and do not ignore potential harmful impacts on small subpopulations. Thereafter, each indicator will be validated for clinical and organizational utility on a test set: the data collected between 2016 and 2019. Predictive models will measure the indicator’s effectiveness to predict health events, such as mortality and hospitalizations (multivariable Cox model), and continuum of health outcomes .2, their calibration , and their overall predictive power (observed probabilities compared to predicted probabilities) [Cox models will use age as a time scale , 44; allilities) . Statistilities) . FinallyAn explanatory model will guide the estimation of causal links between indicators and health events. Given the variable nature of medication exposure and potentially confounding covariates, such as comorbidities, the relationship between polypharmacy indicators and clinical outcomes could be subject to time-dependent confounding . We willFinally, the robustness of the results to unmeasured confounding factors will be assessed using the E-value , 51. TheThe indicators will be used for the surveillance that is carried out by the INSPQ. These indicators should be useful and understandable for public health workers, decision-makers, clinicians, and the patients. Thus, the choice of indicators will be made jointly with these potential users with the help of consensus procedures, notably the RAND/Delphi approach , 53. We The collection, use, and communication of personal health information to train AI algorithms raise unique legal and ethical concerns and require a responsible approach . TherefoObjective 3.1 will aim to verify that the use of new correlations or additional emerging attributes in the AI Axis is carried out in an ethical manner. Particular attention will be paid to a set of risks associated with stigmatization, re-identification, and fairness. Stigmatization occurs when data reinforces the labelling of population subgroups, that are often already disadvantaged or taken responsible for their health problems. Data linkage operations may increase the risk of identifying patients from data sets that were previously anonymized. It may happen when few of them share specific characteristics, for example, due to the frequency of the studied phenomena, their nature, or the size of the territory where these phenomena are observed. Finally, the issue of fairness can arise when certain population subgroups are unduly excluded from analysis or under-represented in the dataset. The Law&Ethics axis will act as an internal ethics committee for the project, allowing researchers involved in the AI axis to handle these risky situations.Objective 3.2 will consist of the ethical and social validation of the indicators developed in the Health axis. The approach will be based on grids developed by the INSPQ , 58. TwoObjective 3.3 will focus on the development of markers and guidelines that could frame the ethical reasoning in the context of polypharmacy and AI. We will adopt an approach inspired by narrative ethics, a method primarily developed in medicine, which allows researchers and patients to narrate the challenges and ethical questions they were confronted with during a particular project . This mePatients will be involved in the validation process of polypharmacy indicators and in discussions within the Law&Ethics axis. Among other things, they will ensure that the developed indicators meet their priorities and preferences.The impact of use of various combinations of multiple medications on individual health and health care system is largely unknown. This research program will be performed by a multi-disciplinary team consisting of specialists in AI, health data, statistics, pharmacy, public health, law, and ethics. Such diversity of expertise will allow investigation of polypharmacy from different points of view and will contribute to a deeper understanding of the clinical, social, and ethical issues surrounding polypharmacy and its surveillance, as well as the use of AI for health record data.In addition, the polypharmacy indicators will be developed in partnership with patients, clinicians, researchers, and public health decision-makers, ensuring the proposed indicators to be valid, reliable, understandable, and simple for their use in surveillance and clinics.One of the strengths of the research program is that it will be conducted on an extensive database that contains time-stamped medical and sociodemographic information covering more than 90% of older adults in the province of Quebec, Canada, over two decades.The main limitation of the research is that the data do not contain variables related to race, ethnicity, gender, health habits, and specific clinical data that could influence both medication use and polypharmacy, and their associated outcomes.Each of the axes will meet challenges specific to their objectives. The complexity of the data and their interpretation will make it necessary to test different methodological approaches and will involve a lot of feedback between the axes. The older population is heterogeneous in terms of physiological, psychological, social, and cultural peculiarities. It will not be possible to directly investigate the role of individual factors such as life habits and genetics because they are not contained in the databases. Thus, since many factors other than medications may influence the health outcomes, the interpretation will have to be made with caution.Similarly, the data mining process may generate a very large number of medication use patterns. Among those latter, some may be ambiguous, inconsistent, or even absurd. Therefore, the interpretation of the results will be of particular importance and will be jointly conducted by researchers and clinicians that will use their technical expertise and clinical experience.Our team is committed to popularizing the procedures carried out so that all stakeholders can understand the processing that was done with the data. We are also aware that the advent of new techniques can open the path towards inconsistent approaches, and this is the reason why the Law&Ethics team will produce new knowledge, markers, and guidelines for future research in this area.Our results will be published in peer reviewed scientific journals, presented at local and international scientific meetings, and shared with clinicians and healthcare professionals via the INSPQ leaflets, website, and public health information center.


Incentives contribute to the proper functioning of the broader contracts that regulate the relationships between health systems and professionals. Likewise, incentives are an important element of clinical governance understood as health services’ management at the micro-level, aimed at achieving better health outcomes for patients. In Spain, monetary and non-monetary incentives are sometimes used in the health services, but not as frequently as in other countries. There are already several examples in European countries of initiatives searching the promotion of biosimilars through different sorts of incentives, but not in Spain. Hence, this paper is aimed at identifying the barriers that incentives to prescribe biosimilars might encounter in Spain, with particular interest in incentives in the framework of clinical governance. Both questions are intertwined. Barriers are presented from two perspectives. Firstly, based on the nature of the barrier: (i) the payment system for health professionals, (ii) budget rigidity and excessive bureaucracy, (iii) little autonomy in the management of human resources (iv) lack of clinical integration, (v) absence of a legal framework for clinical governance, and (vi) other governance-related barriers. The second perspective is based on the stakeholders involved: (i) gaps in knowledge among physicians, (ii) misinformation and distrust among patients, (iii) trade unions opposition to productivity-related payments, (iv) lack of a clear position by professional associations, and (v) misalignment of the goals pursued by some healthcare professionals and the goals of the public system. Finally, the authors advance several recommendations to overcome these barriers at the national level. This paper is aimed at identifying the barriers that incentives to prescribe biosimilars might encounter in Spain. Incentives were chosen as one of the main policy actions to stimulate biosimilar use in Spain because they have an important potential leverage, they are relatively underdeveloped in Spain, and they may be controversial in promoting prescription patterns.We particularly focus on incentives in the framework of clinical governance as they are intertwined concepts. Clinical governance would be impossible to implement without incentives, and incentives, if not impossible, would be difficult to establish in different frameworks.Biosimilar medicines significantly help to improve patient access to biological therapies that have revolutionised the prognosis of multiple serious diseases, while contributing to price competition in the market and the sustainability of healthcare systems. If one of the main current problems in health policy is to make access to new medicines compatible with sustainability, biosimilars are part of the solution by freeing up very considerable resources  (p. 7). Biosimilar medicines have a long history in the Spanish pharmaceutical market. Since the approval of the first biosimilar medicine in 2006, within the European regulatory framework ,4,5,6, tHowever, the uptake of biosimilars in Spain is below the European neighbouring countries’ average. This is observed in the antiTNF group, epoetin and human growth hormone, three of the six active ingredients for which there are data available, which means that there is room for improvement in their use . Thus, tThe promotion of biosimilars is part of most Pharmaceutical policy strategies. The Independent Authority for Fiscal Responsibility , which is responsible for ensuring the sustainability of public finances, considers the promotion of biosimilars the most relevant tool for controlling hospital pharmaceutical expenditure . This inThe “Action Plan to promote the use of market regulating medicines in the National Health System: biosimilar medicines and generic medicines” of the Ministry of Health states that “In the Autonomous Communities […] actions will be carried out to link financial or other incentives”  (p. 38).Congreso de los Diputados (the lower Parliament chamber) dealing with the reform of the NHS to tackle with the Covid-19 pandemic included in its report the need to “significantly increase the proportion of biosimilars” [Further, the Commission for Social and Economic Reconstruction of the imilars”  (p. 25).In short, the promotion of biosimilars in general, and the establishment of prescribing incentives in particular, seem to be unavoidable tasks according to decision-makers and policy makers in the short-term. Therefore, this paper is aimed at identifying the barriers that a model of incentives to prescribe biosimilars might encounter in the context of clinical governance in Spain. Both concepts are intertwined as incentives are the instrument and clinical governance the organisational form. This is a pioneering approach, as the research literature on this topic is very scarce.This work is based on a broader study of the incentives that, in the context of clinical governance, can lead to greater use of biosimilar medicines in healthcare . This stIncentivising health professionals, especially prescribing physicians, is a crucial issue for the organisation and reform of the NHS and for policies to promote biosimilars. Payment systems, including pay for performance, and competition, including benchmarking and yardstick competition, are typical financial incentives ,16. HoweBefore moving on, it should be noted that in this study we refer to a very narrow definition of incentive. Specifically, financial incentives that are not necessarily monetary or exclusive to the prescriber. This is critical since many studies about European biosimilar landscape refer to “incentive policies” that are not necessarily financial incentives. For instance, educational campaigns, quotas, or tendering practices are considered incentive tools to increase the uptake of biosimilars and financial incentive would be just another mechanism for that purpose ,19,20,21The use of incentives to influence physicians’ prescribing patterns and encourage their alignment with organisational goals is a policy that has been embodied in various experiences over time and across countries. In Spain, towards the end of the 1990s, financial incentives related to prescription were already applied in several Autonomous Communities. By 2018, there were at least seven autonomous communities applying them. The AIReF, in its 2018 review of public expenditure on medication dispensed through prescription, recommends establishing prescription incentives . The samHowever, when it comes to promoting biosimilars, there are doubts about the most appropriate type of incentives. Some voices are in favour of financial incentives and argue in their support, for example, their contribution to the progress of biosimilars in Germany. “Prescribers need confidence in outcomes, and they and/or the health system need to benefit financially from using biosimilars” . Other oInvolvement of health professionals not only in treatments but in the whole management.Decentralisation of decisions and autonomy of services.Restructuring of services in a multidisciplinary manner aimed at the management of high-quality clinical processes.Measurement and evaluation of performance and remuneration that may include monetary and non-monetary incentives.Incentives are easier to implement in well-organised broader contexts such as health services following the lines of what is known as “clinical governance”. We acknowledge that there is no consensus definition for clinical governance. Our vision of the concept is as follows: This is an organisational form of health services at the micro-level, aimed at achieving better outcomes in terms of patient health, characterised by the following elements:Some biosimilar prescribing incentives have been put in place in Europe. Although it is not the scope of this research, we summarise in The gainshare agreements reached with regional or local leadership in the United Kingdom ,33,34, mIn Spain, a good example of combining incentives and clinical governance is the “Área del Corazón” (Heart area) of the University Hospital of La Coruña, coordinated by Dr Alfonso Castro Beiras in the 1990s . The proAlthough this initiative is no longer running, there is no doubt that clinical governance offers good possibilities for the efficient use of effective and good quality biosimilars by prioritizing health outcomes, motivation, quality of care processes and efficient use of resources. Actually, we might be talking about one of the first gainshare agreements in Spain. This precedent seems significant enough to support a pilot gainshare agreement in Spain like those successfully implemented in the United Kingdom . It is now time to ask what are the barriers and difficulties that incentives to prescribe biosimilars encounter in Spain.As we are particularly interested in barriers to incentives in the framework of a model of clinical governance and both questions are strongly related, we present here different barriers encountered in Spain that we have been found relevant for both concepts.The current payment system is also a barrier primarily for incentives but also for clinical governance. Its regulation, tradition, and the culture it has generated are very much in opposition to the incentives and flexibility required by efficient organisations. The 2006 report on Spain by the European Observatory on Health Systems and Policies noted that after having completed the healthcare transfer process to the Autonomous Communities in 2002 “the most concerning issue was that most of the pay increases affected the fixed components… compared with income related with performance”  (p. 110)In Spain, bureaucratic control has traditionally prevailed over the evaluation of outcomes, at all governmental levels. “The budgetary system has long suffered from being excessive rigid”  (p. 278)The employment relationship of physicians and other health professionals with the services that make up the Spanish NHS  follows the civil service model. The rigidity and the difficulties it entails to achieve efficient management have often been criticised . One of The disintegration of health care in the NHS is an especially important barrier to implementing clinical governance and promoting biosimilars through incentives. Disintegration implies gaps and borderlines, multiplicity of providers, uncoordinated services, neglect of patient preferences, poor measurement of relevant outcomes, and lack of incentives oriented towards the provision of comprehensive care . ClinicaIn Spain, clinical governance is not regulated by law. The draft Royal Decree RD /2015 laying down the basis for the implementation of Clinical Management Units in the Health Services  was an iIn Spain, healthcare managers, such as hospital general managers and others, do not always have the appropriate professional profile and appointments are generally based on discretionary decisions. Open and competitive recruitment and selection processes and periodic performance evaluations are not always the rule. Collective and independent boards of directors controlling the micro-management of elementary organisations such as hospitals and health areas in a decentralised and transparent manner  are scarThe second Amphos Report , prepareThe potential savings due to biosimilar competition expected in Spain for the period 2020–2022 (€2800 million)  may becoAs biosimilars are biological medicines, they must be prescribed by their brand name . In addiHowever, as previously pointed out by Acha and Mestre-Ferrándiz (2017)  (p. 263)Weise et al. (2012)  listed tAnother review aimed at identifying the barriers to the use of biosimilar medicines in Europe medicines points out that physicians act as a barrier to biosimilars in several ways . FirstlyIn Spain, Agustí y Rodriguez (2015)  predicteAs biosimilar medicines are mainly used into the hospital setting, hospital physicians have been the focus of many educational programmes . This is also observed in the constant review of position papers on biosimilars by scientific societies in most relevant specialties, such as Oncology, Rheumatology, Haematology, or Digestive Pathology ,62,63,64In the case of primary care setting, the arrival of new biosimilars is a new challenge. From a survey with over 700 respondents, it appears that 58% of the respondents do not know the definition of biosimilars and 73% do not know that the handling of biosimilars is not comparable to that of generics, for which in Spain prescription by active ingredient is applied . MoreoveIn short, physicians that are informed through official and reliable sources tend to consider biosimilars as alternatives that are efficient for the health system and effective and safe for their patients and are able to convey the trust needed for preventing the nocebo effect (nocebo effect refers to negative expectations of the patient regarding a treatment that translate into negative side effects or outcomes) and ensuring treatment compliance. However, improving the knowledge about biosimilars and afterwards communicating the information to patients require big efforts by physicians. Therefore, it cannot be overlooked that recognising physicians’ efforts through incentives or other formulas aimed at sharing benefits will guarantee their commitment in the medium term. A major barrier to the spread of biosimilar medicines is misinformation and mistrust from the part of patients. The complexity of the world of medicines and their names, especially if they are biological products, makes it difficult for patients to have timely information and knowledge of their characteristics and guarantees . It may The difficulty often arises, not for the naïve patients, but for those whose treatment was initiated with the original product and are encouraged to change it for a biosimilar product (switch). There may be differences in the brand name or appearance . In addiA study on policies to promote biosimilars in 24 European countries found that educational initiatives aimed at patients were rare. Patients are informed mainly through their organisations, through brochures and letters to explain the switch from originator to biosimilar. It recognises that biosimilar policies should include all stakeholders, including patients, and recommends strengthening educational initiatives through instruments such as question and answer (Q&A) documents .However, a very recent study by Vandenplas and collaborators (2021)  emphasisThe physician–patient relationship is absolutely crucial to overcome these information or mistrust issues. There is no doubt that as long as the physician is properly informed and trusts biosimilars, the patient will follow his or her guidelines.It should be borne in mind that having accurate information and access to medicines are rights that are widely recognised in different jurisdictions. According to the Spanish legislation the physician must inform the patient. Indeed, according to Article 10 of the General Health Services Law 14/1986 , patientIn Spain, Calleja et al. (2020)  identifyOne of the barriers to the establishment of biosimilar prescribing incentives is the opposition of trade unions to variable remuneration based on outcomes, targets, or productivity. This exists not only in Spain but also in other countries, and in any sector, not only in health. The study by García-Olaverri y Huerta (2011)  shows thThe disagreement clearly appears in the 2014 document of the State Confederation of Medical Trade Unions  on clinical governance: “Under no circumstance may incentives be linked to savings over the agreed budget, but rather to the level of compliance with it, and with the care and quality targets established in accordance with the provisions laid down in the management contract.” “This implies that the health service that decides to promote clinical governance must allocate additional funds to pay for these incentives”  (p. 10).Professional corporate bodies, especially those of physicians, react positively to clinical governance insofar as it increases their autonomy, responsibility, and decision-making capacity. Other features such as performance assessment, performance-related incentives and transparency and accountability do not generate the same enthusiasm ,80. ThesAlthough it is a very limited group, health professionals not aligned with the goals of the system, poorly committed to the public system, can be a barrier to clinical governance and incentives for good performance in general, and for biosimilar prescribing incentives in particular. Attitudes such as opposition to transparency or to performance assessment leading to differentiated remuneration must be corrected, as they have a very negative effect on the morale of the vast majority of those who are compliant. When it comes to biosimilars, the strong ties that originator companies have with physicians through supporting clinical research or training may influence prescription choices . AdditioEfforts to inform and educate physicians on the pharmacological and clinical characteristics of biosimilars should be continued and intensified, always on a scientific basis.Patients should be informed about biosimilars to ensure their trust on medicines that are approved by the regulatory authorities.We recommend informing all types of unions and professional corporations of the improvements that clinical governance schemes including incentives  can bring about for the NHS, patients, and professional practice.Consensus and support from policy makers is required to implement a growing uptake of biosimilars mainly from the Departments of Health but also from the Department of Finance as its endorsement of financial incentive programs might be necessary.In the long run, structural reforms of the Spanish NHS are required to overcome other barriers to biosimilar prescribing incentives in the context of clinical governance. We refer to rigidity and bureaucracy in management; clinical and health services disintegration; NHS labour relations model; payment systems and governance. Nevertheless, we think that in the short run there is room for new limited experiments, particularly with non-monetary incentives and the gain-sharing design, which will incite less opposition.According to our review we recommend the following actions to overcome the barriers to implement incentives for biosimilars:


Equus asinus) are facing a global crisis. Their welfare, and even survival, is being compromised as the demand for their skins increases. This demand is driven by the need to supply raw materials to produce ejiao, a Traditional Chinese Medicine made from collagen extracted from donkey skins. Since there is no productive chain for donkey skin production outside of China, the global trade is an entirely extractive industry that has resulted in the decimation of some local donkey populations. The donkey skin trade is demonstrably unsustainable, from the ethical issues associated with poor welfare, to the biosecurity and human health risks the trade poses; and it violates both legal frameworks and moral expectations at both a national and global level.Donkeys ( Most countries do not farm donkeys, and the trade therefore relies on animals that are traded both legally and illegally. In some instances, donkeys are gathered on “fake farms,” increasing the risk of disease transmission, until the numbers are sufficient to warrant transport to slaughterhouses , on the transport of animals by land for example it does not specify requirements for donkeys but makes clear recommendations regarding the transport of other equines, such as horses . On the In the same sense, the Council of the European Union (article 1.4.c. of chapter V of Regulation n° 1/2005) establishes that “domestic Equidae can be transported for a maximum of 24 h. During the journey, the animals must be watered and, if necessary, fed every 8 h.” It also stipulates that “the lack of an adequate level of animal welfare is often due to a lack of training. Therefore, any person handling animals during transport must have undergone training, provided only by bodies accredited by the competent authorities” .The inherent welfare implications and challenges associated with the global skin trade necessitate a discussion of the ethical and legal frameworks being infringed. Ethical concerns and scientific knowledge contribute to the development of legislation that has the power to ensure that practices related to animal health and welfare meet social expectations. Regardless of the existence of legislation, the public play an important role in ensuring their expectations pertaining to animal welfare are met and they have increasing opportunities to influence the way animals are treated. However, access to information is essential for people to make decisions, and call for regulations, that align with their ethics. For this reason, the ethical issues related to the skin trade have both legal and public expectation dimensions, and both are required to address issues of cruelty and unsustainability within the trade. This study aims to shed light on legislative frameworks of the global trade in donkey skins.Many countries have legal frameworks governing animal welfare and, while the existence of legislation does not necessarily guarantee that mistreatment will not occur, it establishes a first step that guides regulation, and often reflects the ethical expectations of a society.Brambell Report in 1965 making some recommendations and providing that animals must have five basic freedoms being the freedom from hunger and thirst; the freedom from discomfort; freedom from pain, injury or disease; freedom from fear and distress; and the freedom to express natural behavior. These freedoms have been considered one of the international benchmarks for animal welfare and have been adopted by many countries. Other countries have moved beyond the Five Freedoms and work with a Five Domains Model that was updated in 1994 considering the latest animal welfare science evidence of: (1) Nutrition, (2) Physical environment, (3) Health, (4) Behavioral interactions, and (5) Mental state (positive and negative). The Model emphasizes what matters to animals in welfare terms of their subjective experiences (affects) and the interactions of affects with physiological mechanisms   recognizes animal sentience ; the Indderecho) .While Brazil does not have a Federal Law recognizing animal sentience, some states including Paraíba, Rio Grande do Sul and Santa Catarina have introduced laws that change the legal status of animals.In this context, it is essential to have scientists building high quality evidence focused on what is needed to provide good or high levels of welfare to the animals in our care. There are current, valid methods to determine what is necessary for good welfare. However, in cases where there is insufficient scientific knowledge, or inadequate measurement tools, it is necessary to make decisions-based on the precautionary principle. In relation to animal welfare, the precautionary principle provides the animals with the benefit of doubt related to potential suffering , 25.The global donkey skin trade is largely unregulated, often illegal, and has significant negative impacts including harm to the livelihoods of vulnerable communities, the risk of disease spread, environmental pollution, and threats to other animals . In seveIn Brazil, the Federal Constitution from 1988 (article 225) raised the environment to the status of a fundamental right, entirely intended to protect the environment for present and future generations. This clause cannot be changed except by a new Constitution . It affoThe extensive list of practices that are considered mistreatment includes abandoning a sick, injured, exhausted, or mutilated animal; failing to provide an animal with what is required to meet basic needs, including veterinary assistance; transporting animals on vehicles for more than 12 h, without water and food; corralling animals in such a number that it is not possible for them to move freely; and leaving animals without water and food for more than 12 h . These illegal practices are frequently seen in the donkey skin trade in Brazil and occur in violation of those legislative instruments.Regarding animal mistreatment, a Federal Law criminalizes ill-treatment of animals, providing for a prison sentence of 3 months to 1 year and a fine . In resThe legislative triad  is what protects non-human animals, including donkeys, against cruelty and mistreatment in Brazil.Since there is evidence of donkey mistreatment within the skin trade, the trade also violates the Decree no. 9.013/2017 (article 88), which provides that “the establishment is obliged to adopt measures to prevent the mistreatment of animals and apply actions aimed at animal protection and welfare, from departure at source to slaughter” .The implications of the donkey skin trade in Brazil extend beyond breaches of animal welfare legislation. The Federal Constitution of 1988 (article 215) protects the Brazilian cultural heritage in all its expressions, ways of creating, doing, and living. Donkeys have well-documented historical and cultural importance as an integral part of Brazil's cultural heritage, according to the constitutional provision .Beyond the legal sphere, there are resolutions, ordinances, and normative instructions, from bodies such as the Ministry of Agriculture, Livestock and Supply (MAPA), the Federal Council of Veterinary Medicine (CFMV), and the National Traffic Council (CONTRAN), regarding animal welfare which are also breached by the donkey skin trade.The scientific community recognizes all vertebrates as sentient, that is, capable of experiencing positive and negative emotions in response to stimuli that come through their sensory system . Brain sThe “Cambridge Declaration on Consciousness” affirmed that non-human animals are not only sentient , but they are conscious beings. The declaration was signed by a prominent international group of twenty-six cognitive neuroscientists, neuropharmacologists, neurophysiologists, neuroanatomists, and computational neuroscientists who gathered at The University of Cambridge to reassess the neurobiological substrates of conscious experience and related behaviors in human and non-human animals .Animals' capacity to experience and express emotions has also been demonstrated , 33, 34 Studies have shown the complexity of donkeys' cognitive abilities and emotions. Donkeys have high cognitive abilities and are considered intelligent animals when comparatively scored on an analogous human scale . RegardlDespite evidence demonstrating animals' ability to think and feel, and their intrinsic value, there are countless examples of animal welfare abuses across animal industries globally. The practices that many species are subjected to, and the neglect of their basic needs, give rise to an urgent need for discussion and action. The suffering that donkeys experience in the global trade in their skins is a pertinent example of an issue on which action is urgently needed.It is unquestionable that donkeys have interests that must be considered, and that they are sentient beings that suffer and feel pain, fear, pleasure, and herein are the ethical grounds needed to protect donkeys from mistreatment.The extractive and unregulated nature of the global donkey skin trade results in significant risks including poor animal welfare, environmental degradation, biosecurity and public human health risks, and non-compliance with national legislation in the countries in which the trade operates. Donkeys are sentient and often hardworking beings, and they experience indescribable suffering in this trade. The donkey skin trade is demonstrably unsustainable, and it violates both legal frameworks and moral expectations at both a national and global level. In response to the growing body of evidence demonstrating the donkey skin trade to be inhumane, unsustainable, and potentially unsafe, the slaughter of donkeys and export of their skins should be stopped.YL and PT had the same contribution in writing the first draft and review. ER edited and review. FB and ES review. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Emerging empirical evidence indicates a limited health literacy for a substantial proportion of children and adolescents. Although it is generally agreed upon promoting health literacy as early as possible in the lifespan, there is a lack of interventions addressing children and adolescents and their primary living environments. This article describes the development of Nebolus, a game-based intervention aiming to promote navigation health literacy at the intersection of schools and communities. Its intervention foundation lies in a socio-ecological understanding of health as well as in the Entertainment Education approach. Following an extensive literature search on health-related location-based games, a co-creation process was initiated that involved adolescents, community stakeholders, and design/IT professionals in all phases of the intervention development. The final Nebolus intervention includes three core activities: (1) a Nebolus rally app for adolescents aged 12 to 16 years, (2) an online planning tool allowing local health service providers/professionals to set up own Nebolus rallies, and (3) accompanying teaching material on health literacy in the school setting to be used before and after the Nebolus rallies. This article provides an overview of the intervention layout and discusses strengths and challenges of its development and implementation. According to Sørensen et al.  health ln = 42) focused on adult patients with only four interventions also including patients younger than 18 years. Another recent review focused on health literacy interventions in European countries and was able to identify only one intervention targeting children between 8 and 12 years  addressing adolescents at the intersection of schools and communities. The main intervention aim is to strengthen navigation health literacy of adolescents in their direct living environment . Specifically, this includes the ability to find, understand, assess and use information about and services provided by organizations or professionals in a person's vicinity. This article describes the intervention development including basic conceptual foundations and the methods used during the developmental process. Moreover, an overview of the intervention layout and the implementation strategy is given.Problems in navigating through an increasingly complex health care system have been identified as a challenge for patient centered health literacy , 22. BasSchools as community embedded systems have long been identified as an important avenue for health promotion and nowadays also for health literacy. First, as health knowledge and behavior are already established in the early phases of the life course, activities on health promotion and health literacy should start as early as possible focusing on primary living environments such as schools. Second, health literacy is compatible to the education core mission of schools and share many commonalities with existing curricular requirement and programs , 32. ThiIn light of study findings indicating a higher frequency of limited health literacy for those of lower socioeconomic status, the intervention focus should be on those vulnerable groups from which we know that are hard to reach. In particular, traditional forms of information provision and communication, which are cognitive and rational, reach their limits when it comes to targeting groups with the greatest need for prevention . In the Based on the background and theoretical foundation presented, the development of the Nebolus intervention was carried out in two consecutive steps: (1) literature search on health-related location-based games, (2) participatory-based development of the intervention Nebolus.As emphasized before, video or online games can be regarded as a promising intervention approach that delivers health-related messages in a low-threshold way by also focusing on peripheral cues and heuristics such as an immersive story, appealing characters and game mechanics. However, the evidence is heterogeneous, with some studies showing only small to zero effects of health related videogames . In addin = 18) were published in 2016 and 2017 with most coming from the U.S. (n = 14). In terms of the topics addressed, LbG's with focus on health behavior such as physical activity predominates. Twenty-six of 33 publications included examined aspects of Pokémon Go. In their review and meta-analysis, Khamzina et al. , LbG's are becoming increasingly interesting for public health. Therefore, the goal of the literature review was to provide an initial overview of the thematic issues, potential effects, and implementation experiences of health-related LbG's among young people. To gain insight into the existing field of research an extensive literature search was conducted using Cochrane Library, EBSCOHost, EMBASE, ERIC, Medline PubMed und Web of Science Core Collection. In addition, a hand search of relevant journals was performed . Eligibility criteria and search terms were defined using the PICO scheme : (1) Popa et al.  summariza et al.  report tTo allow that health literacy interventions are tailored to the needs of the target group, a co-creation design method was used. Despite different terms and understandings, co-creation refers to a process that systematically involves those for whom a health intervention is to be developed . Co-creaUser engagement, i.e., active participation of potential users at all phases of the processMulti-stakeholder participation, i.e., involvement of representatives of the public and private sector who are relevant for the innovation or product to be developedCo-creation, i.e., a process that substantially alters the role of users and stakeholders from subjects of research to equally contributors of the innovationReal-life setting, i.e., all co-creation activities take place in real-life environments  to better illuminate the context for which the innovation is being developedn = 14) were conducted in Europe with older adults as the main target group. While the topics addressed ranged widely , all LL applied a multi-method approach  and were embedded in a real-life setting.Living Labs can be organized in three phases of innovation development: (a) understanding the current state and identifying needs of potential users (Exploitation), (b) developing a prototype  including feedback loops (Experimentation), and (c) evaluating the potential impact and added-value of the innovation . Living Labs have been established in different fields of public health such as alcohol prevention  or primaThe main users of the Nebolus intervention are adolescents aged 12 to 16 years, which were actively involved from the very beginning by the establishment of a youth council. Activities included several interactive workshops in which local providers of health services, barriers of utilization and potential strategies how to overcome these were discussed and a game story and potential types of game characters were developed . Later sAs described above, the Nebolus intervention aims to strengthen the navigation health literacy of adolescents aged 12 to 16 years in their direct living environment . This includes promoting the ability to (1) find information about local health service providers/professionals and their activities, (2) understand the services and activities provided by local health stakeholders and professionals, (3) evaluate the quality of local health service providers/ professionals and their offerings, and (4) apply health information obtained through local health service providers and professionals. The secondary objective of the Nebolus intervention is to promote intersectoral collaboration of health service providers/professionals and to stimulate the development of a coordinated prevention strategy at the community level.To achieve these objectives, the intervention Nebolus pursues three core activities: (1) Implementation of Nebolus rallies in the local community for adolescents, (2) Tailored-based development of Nebolus rallies by local health service providers/professionals and (3) Implementation of accompanying teaching material on health literacy in the school setting to be used before and after the Nebolus rallies  to derivThe core of the Nebolus intervention is an app-based rally that guides young people to various real-world locations (stations or levels) in their respective community . At eachNebolus rallies are organized in groups of up to five users. This aims to reduce barriers to interaction with local health service providers and professionals as well as reduce the risk of exclusion of adolescents without access to digital media (smartphone).Nebolus is a universal health literacy intervention that can be tailored to any specific health topic and the local context of the community. To enable tailoring, a browser-based Nebolus planning tool has been developed that allows each local stakeholder/professional to develop an own Nebolus rally. It is intended that a local stakeholder  will take responsibility and coordinate all activities to develop and implement the local Nebolus rally. This includes the recruitment of local health service providers/professionals, the establishment of a local working group with all participating stakeholders, the development of a fictional story and to set up the rally using the online planning tool. To support communities to implement Nebolus on a local level a number of accompanying materials have been developed including checklists, a manual that helps to develop an own fictional story and characters, and a guide for local stakeholders to develop ideas for interaction with adolescents along the different dimensions of navigation health literacy. In addition, a YouTube channel was created, on which tutorials and explain videos will be made available successively. The Nebolus planning tool allows to set up a rally in four steps and does not require any specific IT related skills :Definition of basic properties for each rally, e.g., title of the rally, location, start and end date, main and secondary characterIntroduction into the rally that can be seen by the users before the start of the rally including e.g., a short description, a voice message presenting a background storyOnboarding setup that can be seen by the users with the start of the rally Creation of main and hidden stations including station name and description, address, text instructions for the users, and voice messages that appear before and after the stationwww.nebolus.net) and will serve as inspiration for other communities to develop their own rallies. Each case study will include information about the health topic addressed and the target audience, the story-framework and characters developed and used, the number of stations and local health stakeholder/professional, and information on how the rally has been implemented.For each Nebolus rally a unique rally-code will be created that needs to be entered in the Nebolus app by the users. This is to ensure that each rally is perceived as a unique event, without rallies from other communities being visible. However, each local rally will be documented as a case study on the Nebolus website . Each lesson includes background information for teachers, concrete learning objectives, a timetable, didactic instructions and accompanying material .The aim of the preparatory lessons is to introduce the concept of health literacy and to strengthen individual skills in the HL subdomains. The first lesson addresses the ability to search and find health information and includes group discussions about different information sources and their use as well as worksheets and group exercises about forms of information acquisition. The second lesson deals with the ability to critically reflect on and evaluate health information obtained through various sources. Specific focus will be given on digital health information and particularly on how to deal with information retrieved from social media.In follow-up lessons, pupils are encouraged to develop a mental map of their local community that includes all local health stakeholders/professionals that were visited as part of the Nebolus LbG rally. This forms the basis for discussions within small groups and the class as a whole, for example, about the services offered by local health stakeholders/professionals and the experiences made with these by the fictional character of the Nebolus rally. In addition, barriers to utilization of these local health services  and ways of overcoming them will be discussed.As a result of increasing evidence, a German national action plan for health literacy was adopted under the auspices of the Minister of Health in 2018. It comprises a total of 15 recommendations across four suggested areas of action . ParticuDue to a lack of interventions in childhood and adolescence, Nebolus aims to promote health literacy of adolescents aged 12 to 16 years at the intersection of schools and communities. It addresses two major recommendations of the German national health action plan and–through its focus on navigation health literacy–also provides references to two additional recommendations . In addition, the Nebolus intervention allows for an explicit link to the Health Promoting School approach as it addresses the curriculum level and the school-community network. Through its focus on strengthening cooperation with community health services, Nebolus also contributes to intersectoral collaboration in school health promotion , 66.Nebolus is characterized by several strengths: First, compared to traditional pre-packaged interventions, Nebolus provides an open intervention framework that can be adapted to the specific thematic and local needs. It therefore contributes to a shift from “one-size fits all” measures to targeting and tailoring health promotion and prevention , 68. SecNext to these strengths, several challenges need to be mentioned with regard to the intervention development and implementation: First, not all dimensions of the HPS framework  health literacy in adolescence. With the launch of the intervention, a number of studies are planned to evaluate the impact and the implementation process of Nebolus. The quantitative arm of the evaluation is planned as a cluster randomized trial, with schools within a given community serving as unit of randomization. Moreover, a qualitative evaluation arm includes interviews and focus groups with adolescents and local stakeholders/professionals about the experiences and facilitators and barriers during the implementation. In compliance with the Living Lab approach, active participation of all groups involved is planned during the evaluation.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.VK and EZ-H conducted the scoping review. KD wrote the first version of the manuscript. All authors contributed substantially to the intervention development and contributed to manuscript revision, read, and approved the submitted version.This research was carried out within the eHLastic project , funded by the German Federal Centre for Health Education on behalf of the Federal Ministry of Health.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Improvements in the digital capabilities of health systems provide new opportunities for the integration of patient-reported outcome (PRO) solutions in routine care, which can facilitate the delivery of person-centered diabetes care. We undertook this study as part of our development of a new digital PRO diabetes questionnaire and clinical dialog support tool for use by people with diabetes and their health care professionals (HCPs) to improve person-centered diabetes care quality and outcomes.DiaProfil, in routine outpatient diabetes care.This study evaluates the feasibility, acceptability, and perceived benefits and impacts of using a digital PRO diabetes tool, Overall, 12 people with diabetes scheduled for routine medical diabetes visits at the outpatient clinic were recruited. Purposive sampling was used to optimize heterogeneity regarding age, gender, duration, type of diabetes, treatment modality, and disease severity. Participants filled out a PRO diabetes questionnaire 2 to 5 days before their visit. During the visit, HCPs used a digital PRO tool to review PRO data with the person with diabetes for collaborative care planning. Participants completed evaluation forms before and after the visit and were interviewed for 30 to 45 minutes after the visit. HCPs completed the evaluation questionnaires after each visit. All visits were audio-recorded and transcribed for analysis. Data were analyzed using quantitative, qualitative, and mixed methods analyses.People with diabetes found the PRO diabetes questionnaire to be relevant, acceptable, and feasible to complete from home. People with diabetes and HCPs found the digital PRO tool to be feasible and acceptable for use during the diabetes visit and would like to continue using it. HCPs were able to use the tool in a person-centered manner, as intended. For several people with diabetes, completion of the questionnaire facilitated positive reflection and better preparation for the visit. The use of the PRO tool primarily improved the quality of the dialog by improving the identification and focus on the issues most important to the person with diabetes. People with diabetes did not report any negative aspects of the PRO tool, whereas HCPs highlighted that it was demanding when the person with diabetes had many PRO issues that required attention within the predefined time allocated for a visit.DiaProfil, are feasible and acceptable solutions for routine diabetes visits, and this tool may generate important benefits related to advancement of person-centered care. Further research is now required to corroborate and expand these formative insights on a larger scale and in diverse health care settings. The results of this study are therefore being used to define research hypotheses and finalize real-world PRO evaluation tools for a forthcoming large-scale multisector implementation study in Denmark.The Danish PRO diabetes questionnaire and the digital tool,  Successful diabetes care requires a whole-person, collaborative care approach that focuses on an individual’s biological, psychological, and social health, well-being, functioning, values, preferences, and priorities .Digital patient-reported outcome (PRO) solutions for clinical practice may help improve aspects of the care experience regarding person-centered chronic illness care  and poteDiaProfil, to facilitate the use of PRO data by HCPs and people with diabetes in collaborative care visits using a formative, participatory design involving people with diabetes, family members of people with diabetes, and a multidisciplinary HCP team in all design phases [We developed a new digital tool, n phases . Particin phases -11, can n phases -14. We aDiaProfil provides a user-friendly mobile or internet interface for people with diabetes to complete a psychometrically designed, adaptive diabetes PRO questionnaire before their scheduled diabetes visits. This questionnaire can be completed via mobile phones, tablets, or PCs and provides a one-screen interactive overview (dashboard) of the PRO results for use by HCPs with the person with diabetes during the visit. The PRO dashboard is designed to make the PRO data available in a way that facilitates effective, collaborative, and action-orientated use of the data during the care visit. The aim is that HCPs and people with diabetes use the tool together to gain a common understanding of the overall perspective of the person with diabetes regarding their overall life with diabetes, priorities, and needs.DiaProfil in routine diabetes visits at an outpatient clinic and to undertake the initial exploration of the perceived benefits related to the use of PRO by people with diabetes and HCPs. The specific research questions were as follows:This study aims to evaluate the feasibility and acceptability of the PRO diabetes questionnaire and the first viable version of How do people with diabetes experience the feasibility, acceptability, relevance, comprehension, and adequacy of topic coverage of the PRO questionnaire when used as intended in the context of a routine visit?DiaProfil in connection with a routine diabetes visit in relation to feasibility, acceptability, perceived benefits, clinical utility, and challenges?How do people with diabetes and HCPs experience the use of Specifically, do people with diabetes and HCPs experience the intended and hypothesized benefits of the PRO tool in improving patient participation and quality of the dialog?DiaProfil tool in Denmark.As part of the formative process, this study additionally aims to facilitate the refinement of research hypotheses and finalize evaluation questionnaires for use in future real-world studies on the implementation and effectiveness of the This study was a formative, mixed-methods, single-arm, acceptability, feasibility pilot study that evaluated an office-based digital PRO tool intervention at an outpatient diabetes clinic.The eligibility criteria were age >18 years, diagnosis of type 1 or type 2 diabetes, diabetes duration of at least 1 year, and planned attendance for a routine visit at the diabetes outpatient clinic during the study period. Exclusion criteria were severe mental illness or major cognitive or language difficulties that would prevent the ability to fill out the diabetes questionnaire.We used purposive sampling and consecutive recruitment to maximize the representation of type and duration of diabetes, age, gender, treatment regimen, and disease severity. Eligible participants were identified by the study nurses from the electronic booking system in the diabetes clinic and invited to take part in the pilot study by telephone. The study was described as a pilot test of a diabetes questionnaire designed to help improve the quality of diabetes visits. All participants signed informed consent before study enrollment.The study was approved by the local institutional review board and deemed out of scope for the ethical review board because of the absence of a clinical treatment or intervention and limited risk.The study design and intervention included the following:Each participant received a secure email with a weblink for mobile or web-based access to complete informed consent, the PRO diabetes questionnaire, and an evaluation questionnaire about the PRO questionnaire. The participant was asked to complete this questionnaire 2-10 days before their scheduled visit. If it was not possible to do it at home, the person with diabetes was encouraged to contact the clinic to arrange on-site completion.HCPs were able to immediately access PRO results once completed by the person with diabetes but were asked to only access results immediately before the visit to mimic routine diabetes care.DiaProfil PRO dashboard with the person with diabetes to jointly review PRO results and plan care collaboratively around the priorities of the person with diabetes. Each visit was audio-recorded and transcribed verbatim.At the visit, HCPs used the The HCPs and people with diabetes completed the evaluation questionnaires immediately after the visit about their experiences using PRO. HCPs filled out a web-based form describing any issues, errors, or concerns relating to the PRO results and how they were displayed using algorithms.People with diabetes were interviewed for 30-45 minutes using a semistructured interview guide right after the diabetes visit by a researcher not involved in care for that person with diabetes. The PRO dashboard and questionnaire results were available in the interview to facilitate a detailed discussion with the person with diabetes about any feedback to the individual’s PRO results, as shown on the PRO dashboard. It was emphasized upfront to people with diabetes that their feedback via questionnaires and interviews was kept strictly confidential and would not be shared with the clinical care team.DiaProfil was used by people with diabetes to complete the PRO questionnaire and by HCPs to manage PRO data and use the PRO data during visits.The aim of the PRO diabetes intervention was to increase the active participation of people with diabetes in their own care and improve the quality of the dialog between people with diabetes and HCPs, and overall care quality by focusing on optimizing value for people with diabetes . The basDiaProfil at home 2-10 days before the visit. The diabetes questionnaire consisted of 33-71 items (depending on the activation of branch logic) that measured health, life situation influencing diabetes, general and diabetes-specific social support, psychological well-being, depression, symptom distress , daily life with diabetes, worries about diabetes, confidence in diabetes self-management, blood sugar regulation (including hypoglycemia and blood sugar stability), medical experience and satisfaction, access to HCPs, priority issues for support, and preferred topics to discuss. The HCPs used the PRO dashboard in DiaProfil with the person with diabetes during the visit to review the person’s priorities and issues and collaboratively plan care. The HCPs were recommended to review the PRO dashboard in advance, share the screen for mutual viewing, explain the PRO dashboard and the color-coding, maintain nonverbal communication and eye contact, use open-ended questions and active listening to prompt more information and confirm findings, and cover all flagged PRO issues. Our recommendations for person-centered use of the PRO tool were quite similar to recently recommended strategies for person-centered communication when using PRO data in clinical practice in other studies [People with diabetes completed the PRO diabetes questionnaire by phone, tablet, or PC using  studies .DiaProfil by the VBHC-PRO-DIA  research team and was conducted from 2018 to 2020. We developed DiaProfil as a new tool to facilitate the coordinated use of the national PRO diabetes questionnaire, also developed as part of this project, in different health care settings to improve person-centered, value-based diabetes care. The PRO tool measures diabetes outcome constructs previously established as important to people with diabetes and HCP in Denmark [This formative study is a part of the participatory development process of the digital PRO tool  Denmark .DiaProfil HCP dashboard using a color defined by a predefined scoring algorithm. Green indicates that there is likely no problem for the person with diabetes, yellow indicates a possible issue of concern for the person with diabetes that should be considered, and red indicates that there appears to be a problem that the HCP and the person with diabetes should make sure to review and address.Each PRO item or scale score is shown on the The interface for people with diabetes includes a user-friendly digital interface for questionnaire completion, which was developed and tested with people with diabetes using an iterative participatory process with user-testing to optimize user-friendliness. Only one question is depicted on the screen at a time to facilitate ease of use and lower cognitive burden.The dashboard provides a one-screen instant overview of PRO results by presenting the results in 9 main themes. On the right side of the screen, the person with diabetes’ own priorities for self-management support and topics to discuss are flagged for use as a starting point for the dialog. By clicking or touching the screen, the HCP can access dialog tips, information resources, local treatment, and referral information relevant for each PRO output.DiaProfil app tool and interface for people with diabetes. The user panel also contributed to the scientific development of parts of the PRO diabetes questionnaire.A panel of people with diabetes who represented the target group for the PRO intervention were involved as partners in this study to ensure that the perspective of people with diabetes was considered at all research phases . This wa1c, cholesterol, blood pressure, and complication data), sociodemographic data, and treatment data  were collected from all study participants using chart reviews and questionnaires. PRO and evaluation questionnaires were administered to people with diabetes and HCPs using the DiaProfil platform. People with diabetes completed evaluation questionnaires before and after their consultation, and HCPs completed the questionnaires after the visit. Questionnaires were purposely based on qualitative data collected from workshops held with people with diabetes and HCPs as part of the development of the PRO questionnaire.Clinical charts  diabetes questionnaire (33-71 items)Completed by people with diabetes at home in advance of the visit. Evaluates general health and life situation, well-being, depression, symptom distress, annual check of feet and eyes, daily life with diabetes, worries about diabetes, confidence in managing diabetes, blood sugar regulation and hypoglycemia, medicine experience, access to health care professionals (HCPs), priority areas for self-management support, and preferred topics to focus on for the visit.PRO diabetes questionnaire Evaluation Pilot Questionnaire for People With Diabetes Completed by people with diabetes immediately after the PRO questionnaire; evaluates perceptions of (1) relevance, (2) difficulty, (3) comprehension, (4) topic coverage/comprehensiveness, (5) acceptability, and (6) item-specific issues.PRO Consultation Evaluation Pilot Questionnaire for People With Diabetes Completed by people with diabetes immediately after the PRO questionnaire diabetes visit; evaluates perceived (1) support for autonomy and person-centered communication; (2) PROs impact on dialog, role, and care; (3) potential negative impacts; (4) face validity of scoring algorithms; (5) interest in continued use and advocacy; and (6) suggestions for improvement.PRO Consultation Evaluation Pilot Questionnaire for HCP Completed by HCPs immediately after the PRO questionnaire diabetes visit; measures perceived (1) quality of visit; (2) PROs impact on dialog, roles, visit outcome; (3) challenges with the use of PRO; (4) satisfaction and interest in future use; and (5) clinical validity of items and algorithms.Quantitative data  and qualitative data  were analyzed in SPSS Statistics 25.0 for Windows (IBM Cooperation) and NVIVO 12.0 , respectively. Primary quantitative evaluation data were presented descriptively. The main qualitative analysis used a simple stepwise coding process adapted from thematic analysis and a phenomenological and combined inductive and deductive approach -23. The The characteristics of the 12 people with diabetes enrolled are shown in The results from the quantitative evaluation of the PRO questionnaire by people with diabetes are shown in one or few items of the PRO questionnaire were difficult to understand. The specific items were reviewed with people with diabetes in interviews as part of the formative questionnaire validation and design. Of the 12 participants, 10 (83%) indicated that no important topics were missing from the questionnaire, and 2 (17%) mentioned topics of atypical diabetes type and more on how to access pump technology as desired topics. Both interview and questionnaire data confirmed that people with diabetes felt the color-coded display of their PRO data, including specific cut-off thresholds in DiaProfil, provided a valid and helpful picture of their situation. Furthermore, HCPs provided separate confirmations of the validity of the scored PRO outputs for each person with diabetes after each visit.All the people with diabetes were positive about the relevance of the questions for their diabetes, except 1 person with diabetes who indicated a moderate negative appraisal. All people with diabetes found it easy to complete the entire questionnaire. Of the 12 participants, 6 (50%) indicated no items, and 6 (50%) indicated The mean single-item scores by item for the primary evaluation questions are shown in All 12 people with diabetes rated the person-centered autonomy-supportive communication style of the HCPs positively, with mean scores ranging between 4.5 and 4.8, with a score range of 1-5. People with diabetes felt that their HCPs were focused on their priorities, encouraged them to speak, and made them feel comfortable talking about their needs, and all felt they got the care and advice that they had hoped for. In order of decreasing positive rating, people with diabetes expressed high interest in continued use, that PRO should be offered as standard care to all, that PRO helped focus on what was most important to them, that PRO helped focus the conversation on what mattered most to them, that they would like to use PRO in their future care, that they felt better prepared for the visit, and that the PRO dashboard provided a good picture of their current diabetes situation, needs, and priorities. There was only 1 person with diabetes who indicated a moderate degree of being uncomfortable or having a problem related to the use of PRO. During the interview, where answers were debriefed, she explained that she had had a bad day, was feeling very distressed because of diabetes, and had not felt the HCP understood her issues as they were raised. Despite this, she was very positive about the PRO tool and did not attribute the problem to the tool.The main themes identified from the qualitative analysis of the interviews are shown in People with diabetes expressed that they felt the PRO questionnaire covered all relevant issues, was straightforward to fill out, and facilitated positive self-reflection. In line with the questionnaire evaluations, approximately half of the people with diabetes reported minor issues with understanding one or a few items.Most people with diabetes expressed that they found it positive to complete the questionnaire at home in advance, as it helped them know what the conversation would be about at the visit. Filling out the questionnaire made the people with diabetes feel reassured that they would remember and get to talk about their priority issues with their HCP. This was important as several people with diabetes expressed frustration that they would often forget to talk about the issues that mattered to them during visits. A family member who participated with a study participant explained the following:It is a nice thing to fill it out at home, and the HCP is also better prepared for what you want to ask about. I would like to have this every time.Several people with diabetes had a positive personal experience filling out the questionnaire. One person with diabetes said the following:When I sat down with the questionnaire, I had some time for it, and I felt really positive and surprised. Because it went straight in and touched on some issues where I had to feel and actually remove the shutters and relate to it. ”How am I doing? If I am totally honest, how is it going with this?“ It is easy to say, ”well it’s going to be fine.“ Let’s just keep going as usual. So, in this way it was really an eye-opener for me.Only one person with diabetes expressed an uncomfortable situation related to the use of PRO during the visit, as noted earlier, and it was clarified in the interview that the person with diabetes did not attribute the issue to the PRO tool but attributed it to not feeling fully understood by the HCP.The main results from the HCP questionnaire evaluation of the visits are shown in Themes and illustrative quotes from the HCP’s evaluation of the visits are shown in The results of the content analysis of the audio recordings of the visits are summarized in Many PRO topics prompted by the HCPs were validated as relevant by the person with diabetes and, for the most part, resulted in relevant follow-up dialog and, in some cases, action. However, in line with HCP evaluations, there were several instances where topics raised pertaining to, for example, symptoms unrelated to diabetes were found not to lead to concrete action or follow-up plans. By cross-matching the topics discussed during the visit with the PRO data, we identified some individual errors because of mistakes by people with diabetes during questionnaire completion; however, we could exclude any structural PRO assessment problems and could confirm the clinical validity and utility of the PRO outputs. Observations of a subset of consultations by a clinical diabetes psychologist provided additional reassurance that it was possible for HCPs to incorporate PRO data in a person-centered manner into the dialog. Observations provided input to our future person-centered training for the use of PRO, especially regarding identifying and clarifying previously undetected psychosocial problems.TopicsLife situation impacting diabetesPsychological well-beingDaily life with diabetesWorry about complicationsBlood sugar regulation and hypo/hyperglycemia worryDiet, carbohydrate counting, and smokingMedicine experienceSymptom distress: Sleep, sexual function, foot, neuropathic pain, gastrointestinal, and cardiovascularConfidence in access to health care professionals for diabetesDiaProfil tool were acceptable, valid, appropriate, and effective in improving people with diabetes’ preparation for the visit and the quality of the dialog.Qualitative and PRO data were compared with questionnaire evaluation data to verify the findings and establish robust overall findings regarding feasibility, acceptability, and perceived benefits. Both questionnaire and qualitative data confirmed that the PRO questionnaire and the Both questionnaire data and interviews for people with diabetes and visit transcripts supported that HCPs used the PRO dashboard in the intended person-centered way and achieved a more focused and relevant dialog. The HCP questionnaire and open-ended text confirmed general satisfaction with the use of PRO tools.DiaProfil dashboard was largely green but with flagging of pain, sleep, and blood sugar measurement. She indicated that as she was doing fine and knew what she wanted to talk about; she did not find the questions so relevant when filling out. The person with diabetes expressed afterward that “we covered other topics than usual which made it more personal–I think it was great” and was keen to keep using it. Another point the person with diabetes raised was that a question about medicine required for branch logic was irrelevant as the hospital should already have the information, which was true, and we began to prepopulate that item subsequently. This highlights the significance of iteratively testing every single item with users. The HCP evaluation confirmed that the person with diabetes was resourceful and knew what they wanted from the visit; the HCP also answered neutral (3) on whether the person with diabetes was better or worse prepared because of PRO.Some people with diabetes had a high number of flagged PRO topics in this version of the tool, and some PRO-flagged topics were not relevant for the HCP to act on in the specific visit; however, people with diabetes were satisfied with the use and did not report concerns or negative experiences related to this. We used a multi-informant mixed-methods approach to analyze negative outliers in the questionnaire data. As an example, we looked at the only person with diabetes who had rated the relevance of the PRO diabetes questionnaire as somewhat not relevant, whereas the 11 others had rated it relevant. The person with diabetes had had diabetes for >13 years, was resourceful and empowered in relation to diabetes, and had seen the same physician for several years. The person with diabetes’s People with diabetes were able to fill out the evaluation questionnaires digitally without the need for support after each visit and expressed, in interviews, that the questions were relevant and easy to understand and fill out. We obtained complete evaluation data from both people with diabetes and HCP from all visits. This corroborated our initial finding from user-testing that they were suitable for use in routine care situations. As part of the formative design, the pilot evaluation questionnaires were revised based on our mixed-methods analysis and feedback from people with diabetes for use in a real-world study of how people with diabetes experience use of PRO . ChangesThe PRO tool helped improve the quality of the dialog and visit by facilitating the identification and prioritization of topics according to the needs of people with diabetes. In line with the extant PRO literature, the use of PRO helped introduce new talk topics, supported dialog about difficult psychosocial issues previously left unattended, and facilitated a more active role for people with diabetes ,26,27. PA common concern expressed by HCPs is that PRO questionnaires may raise unrealistic expectations for people with diabetes and that resources are not available to address topics raised by PRO ,31,32, wOur finding that none of the people with diabetes expressed disappointment or unfulfilled expectations may be partly because the questionnaire had been specifically designed to only include items that were perceived as directly relevant for routine care by both people with diabetes and HCPs. Given the importance of expectation setting in the clinical use of PRO, we attempted to give instructions to people with diabetes about PRO carefully to avoid inadvertently raising unrealistic expectations.Our combined analysis indicated good fidelity related to the HCP’s use of the basic recommendations for person-centered use pertaining to the use of open questions to verify and clarify priorities of people with diabetes and ensure coverage of all key topics highlighted by them. The use of open questions and active listening was applied in all visits, and we found it to be an essential component that may be particularly important in the care of people with cognitive or language difficulties who may have difficulties completing the PRO questionnaire as intended. The PRO intervention did not include requirements for HCPs to use specific methods or tools for agenda-setting, structuring the visit, collaborative goal-setting, action-planning, and shared decision-making beyond the general recommendations for person-centered use of the tool. Therefore, this study provided a first opportunity for HCPs to try out the PRO tool in routine visits, identify key communication challenges, and begin to develop individual strategies on how to use the tool in an optimal way. The main challenges reported by the HCPs relate to how to structure the time and balance focus on clinical and PRO issues. An HCP expressed that it could be difficult to juggle tasks during the visit when people with diabetes had a high number of PRO topics, and an HCP found it difficult to review the many topics on the screen while maintaining a natural conversation. In contrast, and interestingly, the people with diabetes did not express concerns or problems related to the number of topics being identified and were overwhelmingly positive about the way the HCPs used the tool. We believe this may be partly because of the HCPs putting in an extra effort to ensure that people with diabetes had a good dialog experience despite the high number of flagged issues.In a few cases, it was noted by HCPs that some flagged PRO topics, especially those related to symptoms, were not related to diabetes or were not relevant to act on in this visit; however, it was still necessary to review the topics as they were flagged on the PRO dashboard. The issue of not being able to act on especially generic PRO issues has also been reported by others . ImportaPeople with diabetes expressed positive experiences related to the completion of the questionnaire. We found several factors that could potentially explain this. The completion of the questionnaire led to a reflective process that facilitated self- and disease-insight related to diabetes. During the design of the PRO tool, people with type 2 diabetes highlighted the potential for this questionnaire to help people with diabetes understand their situation and options for acting for their own health. On the basis of our results, this effect is likely important for a subgroup and dependent on diabetes duration, empowerment, and whether the person with diabetes is completing the PRO questionnaire for the first time. Filling out the questionnaire at home made people with diabetes feel confident that they would get to talk about their priority issues during the visit. This was very important as people with diabetes reported that they do not normally prepare for visits and often forget to ask about issues that are important to them. Furthermore, this appeared to have a potential impact on expectations and motivation related to participating in the upcoming visit.The broad and balanced coverage of topics in the PRO diabetes questionnaire represents the result of an extensive iterative national multistakeholder participatory design process to achieve a core questionnaire acceptable and useful for both people with diabetes and HCPs in different care settings . Our comtouched all the bases. People with diabetes and HCPs did not report that any major topics were missing or that there were topics that were irrelevant. People with diabetes and HCPs confirmed that the scoring algorithms applied for the color-coding responses were suitable and fit for the purpose.Our study supports that it is feasible to use a PRO diabetes questionnaire with a very broad range of topics and that the questionnaire may generate both specific benefits related to each PRO construct as well as related to its use as an overview to facilitate a person-centered dialog . We beliFurther research is warranted to investigate the importance of a broad topic coverage to facilitate the potential therapeutic and empowering effect of self-completing the PRO. It is possible that PRO instruments that focus only on one or a few topics may not provide the same support for disease insight, care navigation, and active participation as instruments which provide a comprehensive 360°-review of diabetes issuesAnother possible benefit related to the broad topic coverage, when compared with, for example, depression-focused diabetes screeners, is that people with diabetes who may be doing well emotionally are still given an opportunity to express other priorities and issues that affect them in relation to diabetes.HCPs noted that an important benefit of the PRO tool was related to obtaining a structured overview of each person with diabetes’ overall situation, which was made possible by the comprehensive topic coverage. In a previous study, we found that there is a need to use a broad set of PRO outcome constructs to evaluate outcomes in diabetes as the needs of people with diabetes vary individually and change over time . Recent Our PRO tool includes 5 items specifically related to depression and diabetes-related distress as it is important to legitimize and prompt dialog on these often insufficiently addressed issues ,52. BalaDiaProfil, and key barriers were found that were addressed by tool design and facilitation support. Further research is required to identify and address perceived barriers and facilitators to HCP adoption in a larger group of HCPs reflecting different disciplines and health care settings [This study provides initial insights on how to evaluate the potential public health impact of the PRO tool using the dimensions of RE-AIM  ,55-57. Isettings ,25,58,59The first stage of reflection and improved disease- and self-insight and motivation facilitated by the questionnaire completion by people with diabetes were found to be important to the study participants and suggest the potential for the intervention to facilitate diabetes-related empowerment ,5,20.Both people with diabetes and HCPs reported that people with diabetes were more actively engaged as a result of PRO. In this pilot study, HCPs used the PRO tool without detailed protocolized steps for agenda-setting, shared decision-making , goal-seThe anticipated beneficial impact of the PRO tool for people with diabetes relates to confidence in diabetes management , self-efHCPs appreciated gaining a structured insight into the lived experience of diabetes among people with diabetes, which facilitated reflection and allowed them to provide more personalized care. The PRO tool facilitated the introduction of daily life and psychosocial issues to achieve a comprehensive biopsychosocial review of the person with diabetes’ situation. As HCPs achieve self-efficacy for the use of PRO with their patients, there is a potential to experience improved work satisfaction and fulfillment. Given the utility of a wide range of topics, we believe the PRO tool has a particularly high potential for improving the individualization of treatments and use of a wider set of support resources in accordance with what can benefit individuals with diabetes the most, thereby potentially promoting better self-management, outcomes, and use of resources.DiaProfil tool was specifically designed to facilitate person-centered chronic illness care [The ess care ,68 and eess care . Furtheress care .An important additional aim of the PRO diabetes tool is to improve care by monitoring outcomes that matter to people with diabetes for value-based care ,25,70. DDiaProfil, before it was fully finalized for clinical use. This pilot study has important limitations: the study was not designed to provide empirical evidence for clinical effectiveness, and there was no attention control group, so it is not possible to rule out bias because of social desirability or effects related to study participation and added attention [Our results should be examined while considering that this was a formative pilot study of the first viable version of the digital PRO tool, ttention .The small number of people with diabetes and HCPs limits the basis for generalization; even so, purposive sampling provided a good basis for examining experiences from a group of people with diabetes, which was diverse in terms of age, gender, type of diabetes, duration, treatment, and disease progression.The 4 HCPs were previously involved in the design of the PRO tool, which limits the generalizability of their experiences. Further research is required to evaluate the adoption and implementation by a diverse group of PRO-naïve HCPs in different health care settings.Despite these limitations, our use of purposive sampling, multi-informant, and mixed-methods data analysis provided an opportunity to show the robustness of our core findings. Further research is required and planned based on the detailed findings in the study to examine the impacts of, facilitators of, and barriers to effective and integrated standard use of our PRO diabetes tool on a larger scale in different health care settings . AdditioDiaProfil tool in a routine diabetes care setting. We found that people with type 1 and type 2 diabetes in different age groups found the digital PRO diabetes questionnaire relevant, acceptable, easy to use, and as having good coverage of relevant topics. Both people with diabetes and HCPs found the digital PRO tool, DiaProfil, feasible, appropriate, and value-adding for use in routine diabetes care. The PRO tool helped improve the preparation and active engagement of people with diabetes and improved the quality of the dialog in line with the objectives of the PRO tool. Filling out the PRO questionnaire had a positive effect on a subgroup of people with diabetes by facilitating self-reflection and better visit preparation, which also contributed to readiness for talking about needed but difficult issues with the HCPs. HCPs were able to use the PRO tool in a person-centered manner and were satisfied with its functionality. Minor issues and challenges were identified, which were addressed as part of the participatory development process for the PRO tool related to some people with diabetes having a high number of PRO topics. Adjustments were made to the PRO tool to address these issues.This is the first study to show the feasibility, acceptability, and perceived benefits of using the Danish PRO diabetes questionnaire and the DiaProfil, was acceptable to people with diabetes and HCPs, provided clinically useful dialog and decision support to HCPs, facilitated active participation of people with diabetes, and overall improved the perceived quality of diabetes care visits. Further large-scale, comparative, and controlled research is now warranted to examine its potential for large-scale implementation and positive public health impact.In conclusion, we found that our newly developed PRO diabetes tool, consisting of the national PRO diabetes questionnaire and the digital PRO tool