Despite growing demand for practicable methods of research evaluation, the use of bibliometric indicators remains controversial. This paper examines performance assessment practice in Europe—first, identifying the most commonly used bibliometric methods and, second, identifying the actors who have defined wide-spread practices. The framework of this investigation is Abbott’s theory of professions, and I argue that indicator-based research assessment constitutes a potential jurisdiction for both individual experts and expert organizations. This investigation was conducted using a search methodology that yielded 138 evaluation studies from 21 EU countries, covering the period 2005 to 2019. Structured content analysis revealed the following findings: (1) Bibliometric research assessment is most frequently performed in the Nordic countries, the Netherlands, Italy, and the United Kingdom. (2) The Web of Science (WoS) is the dominant database used for public research assessment in Europe. (3) Expert organizations invest in the improvement of WoS citation data, and set technical standards with regards to data quality. (4) Citation impact is most frequently assessed with reference to international scientific fields. (5) The WoS classification of science fields retained its function as a de facto reference standard for research performance assessment. A detailed comparison of assessment practices between five dedicated organizations and other individual bibliometric experts suggests that corporate ownership and limited access to the most widely used citation databases have had a restraining effect on the development and diffusion of professional bibliometric methods during this period. Research organizations and research funding agencies have a growing demand for practicable methods of research evaluation, including metrics based on publication and citation data. Such bibliometric indicators remain controversial among the scientific communities affected by performance assessment –5. In reThe methodological focus of this study is on the measurement of citation impact. Other topics of bibliometric assessment, such as emerging research topics, research profiles, international collaboration are excluded. Research productivity in the sense of an input–output relationship is only assessed in a few instances, since adequate data on resource inputs (scientific staff and funding streams) are often not available . MoreoveThis paper is part of a project conducted with the aim of understanding the development of bibliometric assessment methods from the perspective of Abbott’s sociological theory of professions , 18. I sIn this paper, I first present theoretical considerations concerning the application of professional sociology to the field of bibliometric research evaluation. Then I describe data and methods of meta-evaluation, including search strategies, selection criteria, and the content analysis of evaluation studies with regards to their methodological design and metrics. Finally, I present empirical findings and discuss results in light of the theoretical framework.Abbott’s theory of professions is a sociological framework for analysing how professional expertise is socially constructed and institutionalized in modern societies , 18 Fig. It is aIn applying this theoretical framework to the realm of quantitative research assessment, I assume that the demand for professional services predominantly arises from two important groups of potential clients: research organizations and research funding agencies . These oAccording to the theoretical framework, bibliometric assessment is a service provided by professionals. In 90% of studies in this meta-evaluation, bibliometric analyses were conducted by external experts, who often worked on behalf of the organization to be evaluated. Another 9% of studies were performed by bibliometricians employed by large non-university research organizations and funding agencies—namely the Spanish Consejo Superior de Investigaciones Científicas (CSIC), the German Max Planck Society (MPG), and the Swedish Research Council (VR).I distinguish between individual bibliometric experts and organizations that are dedicated to bibliometric assessment services. Individual bibliometric experts are typically academics employed at universities or non-university research organizations, who conduct bibliometric studies as part of their individual research activities. The label “dedicated organizations” includes the Dutch Centre for Science and Technology Studies CWTS; the Nordic Institute for Studies in Innovation, Research, and Education NIFU in Oslo; the consultancy branch of the Web of Science Group owned by Clarivate Analytics, abbreviated here as TR/ Clarivate; ANVUR, the Italian state agency that implements the VQR; and the expert group working at CSIC, a large Spanish non-university research organization. CWTS, NIFU, and TR/ Clarivate are also referred to in the text as “expert organizations” in the sense of , while AAccording to Abbott’s theory, professionals’ work can generally be described as the application of abstract knowledge to complex individual cases. Abstract knowledge lends legitimacy to claims of jurisdiction, tying professional work to the general values of logical consistency, rationality, effectiveness, and progress. Such scientific legitimacy includes definition of the nature of problems, a rational means of diagnosing these problems, and delivery of effective treatment. Moreover, abstract knowledge enables the instruction and training of students entering the profession, and facilitates the generation of new mechanisms of diagnosis, inference, and treatment. Abstract knowledge is typically accumulated by an academic sector closely related to the profession.In a recent study,  investigThe present study complements the two aforementioned studies and examines actual evaluation practice, as visible in mostly publicly available evaluation studies. The included studies use publication and citation data to evaluate the performance of research organizations or funding instruments in Europe, and are published either as study reports (grey literature) or as journal articles see . The autIn this section, I first describe the selection criteria for inclusion of studies, as well as the search strategies used to identify such studies, and discuss generalizability in light of this sampling strategy. Second, I present the coding scheme and procedures applied to extract methodological information from each individual study. In line with the theoretical framework, practices of bibliometric research evaluation are examined for an extended historical time period. The study focuses on work products of professional actors in the form of published evaluation reports (grey literature) or journal articles. In most cases, these evaluation studies have been designed by bibliometric experts for their individual clients, i.e. decision makers from research organizations or funding agencies, and the professional diagnostics proposed by the bibliometricians have been accepted by the respective clients in a contract relationship. Therefore, assessment reports document an important segment of professional work. Other, less codified applications of bibliometric indicators were excluded, in particular performance assessment by private firms and research laboratories, software tools such as SciVal and Incites, and unpublished or ad-hoc use for decision-making by research organizations or funding agencies. While these uncodified applications also seem relevant for the question of de-facto standards, there is currently no systematic information on their usage and diffusion. The selection criteria for professional evaluation studies were as follows:Each evaluation must include a publication and citation analysis. The sample includes studies relying exclusively on bibliometric data, as well as multi-dimensional evaluations that combine bibliometric data with other information, such as peer evaluations, financial and staff data, or case studies , 35. In The objects of evaluation are either research organizations or funding instruments. Research organizations are typically universities and/or their departments or faculties, or extra-university public research institutes. Funding instruments are typically active in supporting research projects or individual researchers at public research organizations, sometimes with the involvement of private firms, and sometimes supporting long-term investments, such as excellence schemes.Evaluation objects (research organizations and funding instruments) must be located in Europe.The evaluation had to be conducted with the stated purpose of informing decision making on behalf of the respective research organizations or funding instrument. Purely academic studies of bibliometric data were excluded from the study sample.All analysed evaluations, including grey literature (project reports) and journal articles, were published between 2005 and 2019.Sampling started in 2015, resulting in 85 evaluation studies for the period 2005–2014. In November/December 2019, sampling was updated, resulting in 53 additional studies for the period 2015–2019. I combined the following search strategies in order to identify a maximum number of studies from diverse sources:Expert organizations and individual experts having a central position in the academic research area of evaluative citation analysis were identified by prior research . RequestBritish Journal of Neurosurgery or the European Journal of Cancer, and four other disciplinary and foreign language journals, that would have escaped a more conventional search strategy based on a pre-defined set of core journals in scientometrics, science policy, and research evaluation.Evaluation studies were extracted from a set of WoS publications identified as “follow-up research on citation impact indicators” . This seth framework program of the European Union. This publicly accessible database contains meta-data, as well as original documents of evaluation studies [I searched for bibliometric studies in the “Science and Innovation Policy Evaluation Repository” (SIPER), which was created under the 7 studies . A searcI used material from , includiI included reports from the Italian national evaluation agency ANVUR which has a legal mandate to evaluate the quality of activities performed by all research organizations receiving public money, and by funding instruments focused on research and innovation . ReportsFinally, I searched the worldwide web for evaluation reports by funding agencies. Some countries and agencies follow high standards of transparency with regards to public research evaluation, including the Swedish Council for Science (VR), the Swedish Environmental Protection Agency (SEPA), the Danish Council for Strategic Research, and the British Wellcome Trust, among others. These web searches identified a total of 23 relevant and publicly available evaluation reports (17% of study sample).The generalizability of the descriptive findings on professional practices hinges on the quality of the sampling process. Randomization was not possible due to the exploratory nature of the study. Relative to the size of their national research systems, some countries are strongly represented while others appear underrepresented. I am confident that this variation is to a large extent due to different levels of bibliometric activity, since some countries have a tradition of quantitative research assessment while others do not , 27, 32.The final study sample includes 138 distinct bibliometric studies, of which 102 (74%) evaluate research organizations and 36 (26%) evaluate funding instruments. The Italian VQR was the largest evaluation exercise within the sample in terms of number of researchers and publications under assessment. Since discretion on impact metrics was given to disciplinary committees, I treated the VQR as two rounds of parallel assessments to examine the methods used in each case n = 20, . The larSome uncertainty remains, since the coverage of individual nations does not only reflect the diffusion of bibliometric methods, but may also result from different national transparency policies. For example, evaluations of the German Max Planck Society are usually kept confidential. In contrast, publication is mandatory under transparency rules in Sweden, such that evaluation reports are generally available on the internet. In line with Abbott´s theory, the meta-evaluation approach should be considered in tandem with case studies of national jurisdictions, such as , to elucAnalyses are presented cross-sectionally over the whole study period 2005–2019 in the main text. Longitudinal analyses comparing three five-year periods  are documented in supplementary materials  Tables. Before answering the question if there are certain de-facto standards in terms of indicators used, I need to ask to what extent there is agreement in evaluation objectives. To a large extent, variation in study objectives is limited by the study selection criteria. The choice of citation indicators usually implies some sort of performance comparison between different units under study. It is important to understand that similar methods of performance assessment can be and are employed for widely different policy and management purposes . For exaMethodologically, this meta-evaluation is based on structured content analysis. I analysed the bibliometric design of each individual study using a scheme of 37 coding questions  in the fThis remainder of this section comments on methodological choices involved in the design of the coding scheme . The topRegarding (3) “the object of evaluation”, I distinguish research organizations and funding instruments, which pose different challenges for evaluation design. In general, publications are linked to authors and authors are linked to research organizations via their institutional affiliations. In this way, research organizations are treated as author aggregates, with large variations in scale. It is more difficult to attribute publications to individual funding instruments because authors, and author teams in particular, typically receive funding from diverse sources and there is a variable time lag between funding input and publication output. Only during the most recent years, funding acknowledgements were in a few cases applied for funding instruments evaluations, and are therefore not included in this analysis.(4) “Citation databases” includes major multidisciplinary databases WoS and Scopus along with Google Scholar and other specialized disciplinary databases, such as PubMed for the medical sciences or MathSCInet for mathematics. These databases are commodities storing abstract knowledge according to .(5) “Quality enhancement of bibliometric raw data” refers to the fact that quality of citation data as offered in conventional licences by WoS and Scopus is insufficient for most bibliometric purposes. It is at least necessary to disambiguate author names and institutional affiliations, but also journal names. Data quality also includes controls of the accordance between the actual research fields of an evaluation object and their operationalization for assessment purposes, which can be regarded as checks for external validity.The topical areas (6) “sampling strategy and data collection”, (7) “research fields under evaluation”, (8) “definition of citation data” refer to methodical details of data collection, for example time period, treatment of self-citations, and citation windows of a given analysis.(9) “Citation impact indicators” belongs to the core of the meta-evaluation. Since a large array of citation impact metrics have been proposed in the literature ( collecte(10) “Utilized statistical methods” refers to information on the significance of performance differences across study units, but also includes an open item recording special features such as innovative tools for analysis or software.I found instances of bibliometric evaluation in many European countries, but the most regular use of bibliometric assessments during the observation period was concentrated in a few countries. Overall, my sample includes studies from 21 countries plus Framework Programs of the European Union. Approximately 26% of the studies, including four cross-country evaluations, were performed by the four Nordic countries Sweden, Norway, Finland, and Denmark, followed by Italy, the Netherlands, and the United Kingdom . The NetThe bibliometric evaluation of public research in Europe during the observation period was largely based on the citation indices contained in the WoS. Of the total sample set, 87% of studies relied on WoS, while 21% used Scopus or a combination of WoS and Scopus . The shaOne important issue in bibliometric assessment is the extent to which bibliometric databases actually cover the investigated research fields , 21, 46.Raw citation data, as provided by WoS or Scopus, require considerable processing before they are adequate for the assessment of authors and research organizations . The maiA related issue is the need to verify the completeness of publication records. If a completed analysis were later found to be missing individual highly cited papers, this could seriously jeopardize assessment credibility in the eyes of stakeholders. Italy and Norway impose a mandatory requirement for each scientist to register a certain number of publications  or all publications (Norway). These national publication records provide the basis for author searches of citation databases for evaluation purposes. CWTS uses a different approach, collecting internal publication records from research organizations, and sometimes sending these records to the authors for personal verification and completion. Personal verification by authors was also used by 21% of the studies by other bibliometric experts.As stated in the method section, the formal evaluation objectives are broadly homogeneous in that they all involve a comparative assessment of research performance. This section further examines the frame of reference, i.e. how the relevant comparison for performance assessment was construed in each case. It is striking that the analysed publication samples vary in size by orders of magnitude. For research organizations, the modal size category is 1,000–10,000 publications. In contrast, for funding instruments, the modal category is 10,000–100,000 publications. There are a total of eight studies with sample sizes over 100,000 publications, five of which are in the area of medical sciences, and three investigate multiple research organizations. .To analyse the frame of reference in greater detail, research organizations were differentiated according to scale (number of institutes or universities) and scope (mono-disciplinary vs. multi-disciplinary), while funding instruments were distinguished according to type of unit funded  rows in . ConcernThe predominance of international field comparisons is affected by three dedicated organizations: CWTS, NIFU, and TR/ Clarivate. A closer look at the subset of 32 studies by other bibliometric experts with international field comparisons reveals two important sources of influence on their choice of methods: at least ten studies explicitly use indicators by and refer to authors from CWTS in their method section , documenting the status of CWTS as a leading expert organization. Second, in eight cases, expected citation rates were purchased directly from Thomson Reuters (F12), based on Essential Science Indicators  or Incites , in addition to the seven bibliometric assessments commissioned directly from TR/ Clarivate. Fewer studies used Scopus  or Google Scholar  as alternative or additional sources. These findings show that while there remains considerable variation in the details of the calculation of international field averages during the period observed, a leading expert organization and a single database provider are important sources of conceptual and methodological convergence.Dedicated organizations generally used field-normalized indicators for impact assessment, most frequently either field-normalized top-percentiles, field-normalized arithmetic mean, or both , while aIn principle, field normalization is applicable to different types of citation metrics , includiJournal impact was used quite frequently (49%), despite repeated criticism that the substitution of a journal’s impact for the actual number of citations lacks validity , 53. WhiOverall, 78% of all studies used field normalization, including the field-normalized arithmetic mean, other field-related percentiles, and field-normalized journal impact . Among tThere has been some debate in the literature regarding the adequacy of WoS SCs as the basis for field normalization. One point of concern is the lack of transparency regarding the methodology used to construct and update the categories , 58. OneNotably,  do not aIn this paper, I analysed the methods used in 138 bibliometric evaluation studies, from a meta-perspective informed by Abbott’s theory of professions , 18. In A detailed review of assessment methods revealed that bibliometric assessment was more frequently performed in the Nordic countries, Italy, the Netherlands, and the United Kingdom, and that WoS was the dominant database used for public research assessment across 21 European countries. Expert organizations that invest in improving WoS citation data were able to set technical standards with regards to data quality. Citation impact was most frequently assessed with reference to the WoS classification of science fields (SCs), which thus far retained the function of a de facto reference standard.My findings demonstrate two main choices regarding the design of bibliometric research assessments. First, there is the choice between international field comparison, national ranking, and other designs, such as inter-group comparisons, as the main standard of performance. Expert organizations such as CWTS, NIFU, and TR/ Clarivate clearly define international field comparison as the predominant professional framework across all categories of evaluation objects, while the Italian VQR deviates from this professional standard by assessing Italian universities via national rankings on composite performance indicators that are unknown elsewhere. Italy does not follow a model of bibliometric professionalism but applies bibliometric assessment as an element of central state governance of universities .The second and related choice concerns field-normalized citation impact versus h-index and h-type indices. I found that all dedicated organizations use field-normalized citation impact, while h-type indices are more prevalent among those bibliometric experts that do not have access to high-quality citation data, either because the authors come from disciplines applying bibliometric assessment, often medicine, or from countries located at the periphery of the European science system.These findings provide support to  that higPerhaps more important than identifying the leading roles of CWTS and NIFU as bibliometric expert organizations, my analysis unequivocally documents the predominance of TR/ Clarivate in terms of defining methodological standards for the performance assessment of public research in Europe. During the observation period, the provider of WoS assumed the most important role in defining de facto standards for bibliometric assessment. All expert organizations, including CWTS and NIFU, based their citation analyses on data licensed by Clarivate Analytics/Thomson Reuters, as did most other bibliometric experts. However, Clarivate Analytics (via its licencing policy) regulates the extent to which different user groups can access citation data. Moreover, WoS subject categories retained their function as de facto reference standards for bibliometric assessment. In addition, there is the effective dissemination of selected impact indicators via the Journal Citation Reports and Incites. Although academic bibliometricians have endeavoured to develop alternative categorizations of scientific fields , 62, 63 These findings suggest that the current ownership structure of the most widely used citation databases has had a restraining effect on the development and diffusion of professional bibliometric methods during the observation period. First, in the current situation, the development of new diagnostic techniques within the academic sector has remained largely disconnected from their application and diffusion in the professional field. A case in point is the development of alternatives for the WoS classification of science fields. A likely explanation for why alternative academic journal classifications have not spread further consists in that they can only be implemented with appropriate inhouse-databases. In contrast, WoS or Scopus science categories are distributed by database providers alongside with WoS / Scopus raw citation data. The CWTS publication-based clusters, on the other hand, represent a professional solution, but because this solution is proprietary, it cannot be widely shared, discussed, and improved in the academic sector.Second, I find that bibliometric methods have spread to a larger number of countries in Europe over the observation period. In principle, the scientific development in countries seeking to catch up scientifically as well as economically should not be assessed with second-rate methods. Lack of access to first-rate citation databases seems to have influenced the methodological choices of bibliometric assessments in countries such as Greece, Romania, Lithuania, and Slovakia. From these instances I infer that shared access to citation databases would be an important mechanism to better connect methodological developments from the academic sector with their application and diffusion in professional assessment practice.The theoretical framework used in this study raises questions regarding professional control over the production of and access to high-quality citation data and analysis tools. Abbott asked how corporate control of expert commodities  will affect the future development of professional knowledge and practice . It seemS1 TableEvaluation studies by the Italian Valutazione della Qualità della Ricerca VQR 2011–2014, published in 2017.(DOCX)Click here for additional data file.S2 Table(DOCX)Click here for additional data file.S3 Table(DOCX)Click here for additional data file.S4 Table(DOCX)Click here for additional data file.S5 Table(DOCX)Click here for additional data file.S6 Table(DOCX)Click here for additional data file.S7 Table(DOCX)Click here for additional data file.S8 Table(DOCX)Click here for additional data file.S9 Table(DOCX)Click here for additional data file.S10 Table(DOCX)Click here for additional data file.S11 Table(DOCX)Click here for additional data file.S12 Table(DOCX)Click here for additional data file.S13 Table(DOCX)Click here for additional data file.S1 Appendix(DOCX)Click here for additional data file.


The public health burden caused by overweight, obesity (OO) and type-2 diabetes (T2D) is very significant and continues to rise worldwide. The causation of OO and T2D is complex and highly multifactorial rather than a mere energy intake (food) and expenditure (exercise) imbalance. But previous research into food and physical activity (PA) neighbourhood environments has mainly focused on associating body mass index (BMI) with proximity to stores selling fresh fruits and vegetables or fast food restaurants and takeaways, or with neighbourhood walkability factors and access to green spaces or public gym facilities, making largely naive, crude and inconsistent assumptions and conclusions that are far from the spirit of 'precision and accuracy public health'. Different people and population groups respond differently to the same food and PA environments, due to a myriad of unique individual and population group factors  and their complex interplays with each other and with local food and PA settings. Furthermore, the same food store or fast food outlet can often sell or serve both healthy and non-healthy options/portions, so a simple binary classification into 'good' or 'bad' store/outlet should be avoided. Moreover, appropriate physical exercise, whilst essential for good health and disease prevention, is not very effective for weight maintenance or loss , and cannot offset the effects of a bad diet. The research we should be doing in the third decade of the twenty-first century should use a systems thinking approach, helped by recent advances in sensors, big data and related technologies, to investigate and consider all these factors in our quest to design better targeted and more effective public health interventions for OO and T2D control and prevention.The online version contains supplementary material available at 10.1186/s12942-021-00266-0. The ongoing pandemic of coronavirus disease COVID-19) has heavily scarred the healthcare capacities and systems across the world during the whole year of 2020 and until now (Q1 2021). At the time of writing, more than 1 per cent of the world population (82 million) were infected (diagnosed/confirmed cases only), and over 1.8 million deaths were claimed by the pandemic 9 has hea.Among NCDs, overweight and obesity (OO) and type 2 diabetes (T2D) are unique because they are both medical conditions  and leading risk factors for other comorbidities, including many NCDs and infectious diseases such as COVID-19 –7. The iThe fact that OO and T2D are unevenly distributed across different socioeconomic statuses (SES), demographic groups and geographies make researchers and policymakers more puzzled to design effective public health interventions , 13. LowTo get our act together for successful future strategies and interventions, this article provides a brief overview of the latest OO and T2D geospatial research trends, especially focusing on novel approaches using precision and accuracy medicine/public health, smart technologies, big data and systems science.OO and T2D have been conventionally and rather naively attributed to a long-term energy imbalance between calories intake from food and beverages and expenditure via physical activity (PA) and behaviour, but the reality of their pathogenesis is far more complex than that. Researchers have extensively investigated the multifactorial and deeply interrelated interactions among multiple biological, sociodemographic and contextual factors of OO and T2D. Clinic-based studies have identified more than 50 genes associated with OO and 36 genes related to T2D that play specific roles in appetite and food intake, insulin action, cholesterol and fatty acid synthesis, and family heredity , 22. PopInternational Journal of Health Geographics (IJHG) on the topic of OO (n = 77) and T2D (n = 64). Examples of OO and T2D articles published in IJHG include studies about youth's obesity [Geographic Information Systems (GIS) have contributed uniquely to OO and T2D literature. Researchers used GIS to investigate the built environment and its associations with OO and T2D. Since 2002 (and up to December 2020), more than 130 articles using GIS have been published in  obesity , 25, old obesity , the bui obesity , 28 and  obesity , 30. Wit obesity , 32.During the past few years, new technologies have been applied in OO and T2D research. Firstly, utilising lifestyle sensors (LS) has gained popularity among researchers. LS are devices that detect and collect changes in the environment or behaviours of an object and relay the collected data to other data processing devices or controllers . Types and levels (intensity and duration) of physical activity, food intake, mobile personal indirect calorimetry, heart rate (variability), blood sugar, blood lipids and blood pressure levels are among the measures that OO and T2D studies have frequently used  or can uThis demonstrator  with proximity to stores selling fresh fruits and vegetables or fast food restaurants and takeaways, or with urbanisation, neighbourhood walkability factors and access to green spaces or public gym facilities, making largely naive, blanket assumptions and crude, incomplete and often inconsistent (across similar studies) conclusions that are far from the spirit and requirements of twenty-first century precision and accuracy public health.Different people and population groups respond differently to the same food and PA environments, due to a myriad of unique individual and population group factors and their complex interplays with each other and with food and PA elements .Furthermore, the same food store or fast food outlet can often sell or serve both healthy and non-healthy options/portions and use both good and bad food processing and cooking methods for different products, so a simple binary classification into 'good' or 'bad' store/outlet should be avoided. Population dietary behaviours, including amounts consumed per individual snack/meal/day or per family are also important, as even the healthiest options can prove unhealthy when overconsumed. Moreover, appropriate physical exercise, whilst essential for good health and disease prevention, is not very effective for weight maintenance or loss , and cannot offset the effects of a bad diet . In factThe research we should be doing in the third decade of the twenty-first century should use a systems thinking approach, helped by recent advances in sensors, big data and related technologies, to cater for this myriad of interconnected factors in our quest to design better targeted and more effective public health interventions for OO and T2D control and prevention. The remainder of this section briefly describes the high-level task clusters that are necessary to successfully execute the feasibility demonstrator roadmap and build prototype smart dashboards for future precision and accuracy public health OO and T2D interventions Fig. .Fig. 1PrTo design and develop effective evidence-based public health interventions for OO and T2D, there should be an agreement on a comprehensive list of population lifestyle data relevant to the study area and study population. Besides standard population variables, such as age, gender, race/ethnicity, SES and family status, a set of important lifestyle risk factors for OO and T2D studies may include health literacy levels , food shopping and dietary habits, behavioural characteristics , relevant health conditions and exposure to environmental pollutants .While studies have often requested participants to write food diaries , later studies have used dedicated smartphone apps to collect and analyse meal records and food barcodes (packaged products) and pictures . AggregMeasures of PA and estimates of energy expenditure levels, e.g., by computing daily walking steps, are commonly used in OO and T2D research. Some studies measured calories spent on PA using LS, GPS/location-based applications or exergaming devices , 46. FitOther measurable lifestyle factors associated with PA in OO and T2D literature include the use of television and other screen devices , sugar-sweetened beverage intake , smoking  and sleep quality , 48. SmaFurther population-level PA indicators to consider include population heart rate variability levels, also derived from fitness/PA wearable and app data aggregates; and, if a platform such as Google Fit is used, average 'Move Minutes' and 'Heart Points' per person/per day in different age groups and country regions, cities and city neighbourhoods; percentages of poor, average and high performers in target populations; also cumulative  population walking steps, 'Move Minutes' and 'Heart Points' per region, city and neighbourhood for a given period. These indicators can be monitored and compared every few months to determine population progress and trends over time, or at different times of the year/seasons, or following specific PA community interventions. Indicators can also be normalised by population number to compare different regions, cities and neighbourhoods.  Further interesting patterns might emerge from user data aggregates that can help city planners better understand and optimise the PA landscape of their city and its population; for example, data might reveal that residents of some neighbourhoods are doing most of their daily exercises in some walkable streets, parks or public gym facilities in a different neighbourhood .Population clinical data (aggregates and trends stratified by population group and neighbourhood/region), such as BMI and body fat percentage , blood sugar and blood pressure levels, cholesterol/blood lipid profiles, data about relevant long-term conditions, etc., can be combined with other population lifestyle factors/data, since OO and T2D are related to, or often coexist with, other chronic diseases, e.g., cardiovascular conditions.Population aggregates from platforms such as Amazon's Halo wearable and health monitoring service (subscription) can provide unique insights about the prevailing moods and stress levels among different population groups over time. Powered by machine learning, speech processing and computer vision, Amazon Halo infers mood and stress levels from the wearer's voice tone, and also measures BFP .actual food and PA environments). Citizen noise pollution monitoring is also possible using smartphones [The rapid advances and improvements in mobile technologies and their affordability over the past two decades have enabled researchers to use smart devices and wearables more actively in their research. For example, a Dutch study obtained a GPS-tracked mobility dataset from its participants for seven consecutive days . Such a rtphones . The anaIt should be noted that the aforementioned lifestyle variables are not an exhaustive list of all the factors one should consider. Researchers also need to prioritise the acquisition of these variables according to their relative importance, data availability and reliability, and ease and cost of collection, by carefully examining previous public health interventions (their level of success and types of data feeding into them), the decision support needs of public health professionals and policymakers, and the available human and financial resources.Identifying the most efficient, appropriate and reliable ways of data collection is as important as selecting the relevant variables. Technological feasibility, as well as traditional issues, such as budget and staffing, should all be examined when planning data collection. There are several important questions that researchers need to carefully consider; for example, are the data purchasable from vendors or need to be collected? Will the data be collected by a research team, by some outsourcing contract, or by crowdsourcing? Which method or instrument would be best for collecting each dataset depending on its nature, real- or near-real-time datasets and those that need to be collected less frequently; for example, using data streams from LS/wearables, smartphone apps, or traditional survey methods ? What levels of detail granularity will be set for the research and data ? How long will it take to collect enough data for the system to reason with, and what will be the ideal update frequency/interval, where applicable  24 h, one week, one month, yearly, or before, during and after some public health intervention, etc.)? For data that are going to be sampled rather than extracted from population aggregates of routine service/app users, we need to determine how many participants should be recruited (and their characteristics) to secure a representative sample of the target population, and what is the most appropriate method to recruit them ? (Big) data quality and reliability dimensions must also be carefully considered for each dataset, e.g., validity, accuracy, consistency, completeness, timeliness, etc.This task cluster focuses on big data governance and management, covering a number of interrelated issues that are critical to the success, scalability and sustainability of the demonstrator, including compliance with relevant data and metadata standards, e.g., OGC  standards; compliance with data protection regulations, ensuring individual data privacy is always preserved, especially when non-aggregate data are collected and processed; establishing adequate data sharing agreements with providers; and the adoption of best practices in ontologies, big data warehousing and cloud security.Specifically, data should be protected from any unauthorised access and potential corruption at all times. Individuals' privacy and confidentiality protection should comply with up-to-date regulations for data storage and management, all while enabling the demonstrator's seamless, secure access to the data for analysis. Data taxonomies and ontologies are particularly essential for managing and reasoning with big data, since the majority of big data are often collected in unstructured form , 53.Data fusion refers to the process of integrating data from different datasets and multiple sources, including managing data uncertainties and incompleteness/missing data points across the fused datasets. For example, a geo-tagged social media dataset aggregated at neighbourhood levels can be 'fused' with other aggregate neighbourhood data, such as governmental census or survey data, to paint a more accurate and detailed picture of the target population. Geo-analytics and GeoAI tools are becoming increasingly essential for crosslinking and reasoning with ('making sense of') the growing amounts of geo-tagged population big data that are continually generated through mobile health  devices/apps and precision medicine practices .The smart public health dashboards should (1) provide public health decision makers and programme planners with a user-friendly one-stop portal for intelligent data analysis, interactive data querying and visualisation in different ways, and distributed team collaboration; and (2) offer lay summaries/views of the dashboards to keep members of the general public informed and engaged. Public health planners can use the dashboards and the generated big data intelligence to interactively design and prioritise new public health interventions, identify their potential target population groups and areas, and perform simulation modelling of intervention costs and impacts under different settings or 'what if' scenarios prior to deployment . VariousPublic health dashboards serve critical functions as information and intelligence communication tools and in decision support. Adequate user involvement in all stages of their development  is key to their lasting success and ultimate user acceptance . DevelopDesigning effective public health dashboards is a highly multidisciplinary and interdisciplinary undertaking that must address a number of essential criteria and tasks concurrently. These tasks cover (1) research integrity, quality assurance and system/evidence updates, (2) multidisciplinary collaboration, (3) risk management, and (4) exploitation of the demonstrator's results.Firstly, a dashboard should achieve scientific rigour. Data, analytics methods, visualisation and underlying research design, as well as the underpinning clinical/public health evidence must all be robust and sound, avoiding the well-documented failings and traps of big data . DashboaSecondly, it is vital to build and maintain strong partnerships, close collaboration and mutual understanding between the multidisciplinary members of the demonstrator team , target users and other stakeholders. The research literature offers some excellent discussions about participatory practices and platforms in dashboard building that should prove helpful in fostering and facilitating these collaborative partnerships , 58.Thirdly, risks assessment and mitigation plans ('plan B') must be put in place at the very start of the work on the demonstrator. Lastly, outputs dissemination activities and exploitation plans should be developed to keep the wider stakeholders and managers engaged and supportive of the work, secure continuing funding, and ensure future viability and expansions of the demonstrator, both in functionality and to other cities/regions and populations.Big data approaches in genomics, epigenomics and bioinformatics, coupled with geographic information (systems), are unveiling the complex interplay of environment, genes and other factors in health and disease . Genes cBut that is not the end of the story; we can still 'tame' (tone down/turn off the expression of) unfavourable genes with suitable external factor modifications and interventions, e.g., by introducing targeted gamified exercise interventions (exergames/mobile exergame apps) for added exercise 'appeal' and PA behavioural sustainability in those individuals and population groups that 'hate' exercise because of their genetic makeup .Indeed, future public health interventions will evolve to precisely identify, and address the specific characteristics and needs of, target population groups or areas, inasmuch the same way as personalised precision medicine is evolving in managing individual patients. In the future, we will know more about our target populations, thanks to population genome data banks, where profiles of large local population samples can be mined and analysed for the presence specific relevant genes, gene variants/mutations and dysfunctions, such as those genes controlling appetite, or implicated in overweight and obesity predisposition, or determining our base exercise behaviour, e.g., fat mass and obesity-associated (FTO), melanocortin-4 receptor (MC4R), leptin (LEP), etc. . We willFirmicutes, while others are 'good', e.g., Bacteroidetes [Similarly, we can have population gut microbiome data banks of large local population samples . A growing number of consumer-oriented labs are already offering 'gut bacteria profiling' services today at affordable prices. Gut bacteria types and diversity/ratios  can be It should be noted that the demonstrator discussed above is all about making sense of 'crude' population data aggregates for public health purposes. Therefore, it is not about collecting ultra-precise individual patient data or the precise clinical management of individual patients. Moreover, there will be other research opportunities related to this demonstrator that were not fully developed or described in this article. For example, task clusters 4 and 5 can additionally help generate new hypotheses for further clinical and epidemiological research beyond the scope of this demonstrator, as an extra advantage of having such big population lifestyle data at hand. An open platform/API (application programming interface) or a 'data cooperative' can be offered as one of the demonstrator's outputs to enable other research groups, nationally and internationally, to interrogate and interact with select sets of the demonstrator's data and analytics, subject to governing data regulations and adequate data privacy and security safeguards. The companion ‘Additional file Geo-tagged big data from smartphones, wearables and other sensors are enabling researchers to conduct innovative studies in OO and T2D.  Public health professionals can greatly benefit from well-conceived big data dashboards and related technologies in unveiling and acting upon the multifaceted challenges of OO and T2D in their target populations. The roadmap presented in this article with its list of high-level task clusters should provide a good start for teams willing to develop these dashboards for smarter public health decision-making in OO and T2D control and prevention in their locales. Dashboards must always be designed and developed in such a way that accommodates and streamlines future updates of not just the software components, but also the consumed datasets  and the underpinning clinical/public health evidence and guidelines (as science progresses). It is hoped this article will initiate and stimulate further fruitful discussions among public health communities worldwide, and inspire many future ground-breaking studies about food and PA environments and population factors in OO and T2D.Reference in the manuscript to any specific commercial product, process or service by trade name, trademark, manufacturer or otherwise does not necessarily constitute or imply its endorsement, recommendation or favouring by the authors or the entities they are affiliated to, and shall not be used for commercial advertising or product endorsement purposes.Additional file 1. A feasibility demonstrator roadmap (supplementary slide set).


Background: Disasters are associated with worse perinatal outcomes, perhaps due to inadequate prenatal care (PNC). Methods: Using 2017–2019 Florida vital statistics, we compared PNC use before and after Hurricane Michael. We categorized counties as most affected (Area A) or less affected (Area B and C). We examined whether Michael’s effects on perinatal outcomes varied by maternity care availability and used the Baron and Kenny method to assess whether delayed PNC initiation mediated perinatal outcomes. Log-binomial regression and semi-parametric linear regression were used, controlling for maternal and ZIP code tabulation area characteristics. Results: Compared to the one-year period pre-Michael, the week of the first PNC was later in all areas in the one-year period post-Michael, with the largest change in Area A , where women were less likely to receive PNC overall  and more likely to have inadequate PNC . Michael’s effects on perinatal outcomes did not vary significantly by maternity care availability within Area A. Delayed PNC initiation appeared to mediate an increased risk in small for gestational age (SGA) births after Michael. Conclusion: Women in Area A initiated PNC later and had a higher likelihood of inadequate PNC. Delayed PNC initiation may partially explain increased risk of SGA. Disaster can produce temporary or permanent “maternity care deserts”, as the March of Dimes terms areas with no delivery or prenatal care . While hThe effect of lapses in PNC after disaster is related to individual risk factors and social determinants of health. Some studies indicate that the number of PNC visits can be reduced without significant harm to maternal or infant health . The selIn addition, while birth outcomes like infant mortality, low birth weight, and preterm birth (PTB) are strong indicators of perinatal health, other important yet overlooked peripartum outcomes may be affected by health care availability. A study of women affected by the Fukushima earthquake and nuclear disaster found that discontinuation of scheduled PNC or switching to another medical institution for PNC was associated with a reduced likelihood of breastfeeding . CaesareEven without disaster, many women in rural areas have challenges accessing care, with longer distances to health care services and limited availability of health care providers . In the In this study, we examine the effects of a major hurricane (category 5) targeting a largely rural region, with respect to availability of PNC, timing of initiation of PNC, adequacy of PNC, and their effects on perinatal outcomes. We hypothesized that adverse birth outcomes and caesarean section rates would rise in the aftermath of the disaster, that breastfeeding would decline, and that these changes would be partially mediated by availability of and access to PNC and obstetric care.Vital statistics data for 2017–2019 were obtained from the State of Florida. Births were categorized as occurring in the year before and after the date of Hurricane Michael . Based on FEMA disaster declarations , FloridaNonhospital care providers: based on the 2019 AtoZdatabases Healthcare Professionals database [database , a list Hospitals: The Agency for Health Care Administration (AHCA) for the State of Florida provided a list of hospitals potentially affected by the hurricane, and the dates of evacuations, closings, and reopenings. Excluded from our analysis are facilities that are unlikely to provide obstetrical care, such as rehabilitation, long-term care, and substance abuse facilities.Availability of maternity care in a ZIP code on a given day was therefore based on (a) whether a delivery hospital was open, and (b) the number of prenatal clinics open. For each delivery, we categorized the availability of healthcare into 3 levels based upon the date of the delivery: in a ZIP code with open delivery hospital(s) and prenatal clinic(s); in a ZIP code with either an open delivery hospital or prenatal clinic but not both; in a ZIP code with neither of these. We limited analysis of maternity care to the most affected area of the state, where residents were eligible for individual FEMA aid (Area A). Births were categorized based on mother’s residence, whether the mother resided in a ZIP code with both a delivery hospital and a clinic, only a clinic, or neither. If a hospital or clinic closed temporarily due to the hurricane, the birth was categorized based on availability when it occurred. The periods between 6 October 2017–6 October 2018 and those during 7 October 2018–7 October 2019 were compared. Access to PNC services before and after Hurricane Michael were evaluated by whether pregnant women had any PNC visits before delivery, the month of first PNC visit, and the Kotelchuck Index. There are four adequacy categories in the Kotelchuck Index: adequate plus, adequate, intermediate, and inadequate . We furtThe birth outcomes examined included incidence of PTB, low birth weight, and small for gestational age (SGA). PTB was defined as a birth before 37 weeks of gestation. The algorithm reported by Klebanoff et al. was used to distinguish spontaneous vs. indicated preterm births . IndicatConfounders were chosen from known risk factors and included in models based on whether the distribution of the factors were different before and after Hurricane Michael. Variables examined as potential confounders were maternal age, race, ethnicity, smoking during pregnancy, alcohol use, education, prepregnancy BMI and enrollment in the U.S. Department of Agriculture’s Supplemental Nutrition Program for Women, Infants, and Children (WIC) program. Covariate missing data was minimal: maternal age (0.0025%), education (0.97%), race/ethnicity (1.28%), prepregnancy BMI (5.63%), WIC enrollment (1.24%), smoking during pregnancy (0.38%), and alcohol drinking during pregnancy (1.51%), so complete case analysis was conducted. To control for sociodemographic and geographic differences between ZIP code areas, we obtained the ZIP Code Tabulation Area (ZCTA) level data from American Community Survey (2018 5-year estimates) for the total population, the percentage of the population with health insurance, the percentage of population living below the poverty level, the percentage of white alone population, and percentage of Hispanic or Latino population. We additionally controlled for the percentage of the population living in an urban area  . The ZCTModels were created to examine whether outcomes differed by area, time period, and whether the area was a maternity care desert or not. First, we examined the effect of Hurricane Michael on PNC receipt and association between PNC and WIC receipt. A semi-parametric linear model was used to model time of first PNC visit comparing the periods before and after the hurricane. Log-binomial regression was used for binary outcomes. The linearity assumption was checked using linear graphs between predictor variables and predicted log probabilities and no violation was found. Collinearity assumptions were checked using the variance inflation factor (VIF). All VIFs were less than 2, which indicates the stability of the regression coefficients. All the estimates were compared unadjusted and after adjusting for potential confounders. Next, we examined whether the effects of the hurricane on perinatal outcomes varied by whether the area was a maternity care desert, adding interaction terms to the models comparing pre- and posthurricane outcomes . In thiFinally, we examined how care receipt might have affected perinatal outcomes. We looked at whether the relationship between hurricane exposure and outcomes was mediated by delayed PNC initiation. Because we did not detect interaction between hurricane exposure and gestational month of PNC initiation, we adopted the Baron and Kenny method : (a) tesStatistical analyses were performed using the software SAS 9.4 . This analysis was approved by the Institutional Review Boards of Tulane University (2019-529-TUHSC), Florida State University, and the Florida Department of Health.The study population is described in The gestational month of the first prenatal visit was later in all three areas using the linear regression models with control for clustering (GEE), but the largest change was in the most affected area , . In areaOf the 429 health facilities, 188 did not treat pregnant women. We approached 68 facilities  was associated with higher risk of low birth weight, spontaneous preterm birth, induced preterm birth, and SGA compared with adequate PNC use in all areas . Those ap-value for Sobel test = 0.0002). In the most affected area (area A), Hurricane Michael was significantly associated with low birth weight and SGA regardless of the time of PNC initiation . AdjustmIn this analysis, we examined the effects of Hurricane Michael on access to care in the largely rural Florida Panhandle, and the relationships this had with perinatal outcomes. Few previous studies have formally addressed this question, particularly among rural populations, largely limiting themselves to adjusting models of birth outcomes for receipt of PNC ,7,13,14.That disasters would reduce access to PNC (due to either facility closures or patients’ inability to reach facilities) seems a logical assumption, but it has not always been confirmed in previous studies. While both this study and studies of Hurricane Katrina found a strong negative effect on PNC , a studyCurrently there is no specific standard or guideline for providing routine PNC during and after disaster. Two disaster planning standards or guidelines for obstetric services published by the American College of Obstetricians and Gynecologists (ACOG)  and StanWe did not find evidence that the effect of Hurricane Michael on adverse perinatal outcomes varied according to health care availability in the most affected area. This could be due to imperfect records of clinics that provide PNC care regarding whether they closed during Hurricane Michael. Some clinics refused to answer our questions after answering the phone; others could not be reached by phone. It is possible that those unreached clinics provided PNC, but a sensitivity analysis where those clinics were classified as providing care found no difference. Although we found evidence that PNC was disrupted by Hurricane Michael, the increase in adverse perinatal outcomes was not explained by delayed PNC initiation overall, which is consistent with previous studies ,12,13,14p < 0.0001) in the most affected areas. Improving partner communication among WIC provider sites, regional health offices and the state health department, and PNC providers prior to and in the aftermath may help improve PNC use and WIC enrollment after a disaster [However, PNC visits could provide an opportunity for pregnant women to receive mental health support, education in environmental health, and linkage with community resources, including nutrition and social service programs, after natural disasters. For instance, the Women, Infants and Children program (WIC), a supplemental nutrition program supported by the US Department of Agriculture, provides health care referrals and nutritional support to low-income pregnant women, postpartum women, and children up to age 5 . Eligibldisaster . Strengths of this study include the large sample size and examination of multiple adverse perinatal outcomes. There are also several limitations of this study. First, we used the measurement of access to care at a ZIP code level rather than the measures specific to an individual woman. Therefore, we do not know how many women were forced to find new sources of care. Second, we did not address maternal mortality or severe maternal morbidity, as such complications are relatively rare and the area under study was sparsely populated. However, such conditions might be more directly affected by lack of care . Third, In this study, we found that women living in rural areas most strongly hit by a hurricane had a later week of first prenatal visit and were more likely to receive less than adequate prenatal care. However, the increase in adverse perinatal outcomes was not explained by delayed PNC initiation, except for SGA. More research is needed to determine the role of PNC in presenting an opportunity to provide pregnant women with support on mental health, education on environmental health, and linkage with community resources after natural disaster.


I had the privilege of serving as Principal- and Co-Convener of the HIV, AIDS and Older Adults Special Interest Group (SIG) for four years (2012 through 2015). During this era, when the proportion of U.S. older adults with HIV was projected to surpass 50%, we witnessed a number of milestones including the NIH Office of AIDS Working Group on HIV and Aging recommendations for critical research focus publication , the first CDC Surveillance Report on people 50 and older with HIV (2013), and the first UNAIDS report on HIV and aging (2013). During this period, the SIG was very successful in raising awareness about HIV and aging through numerous GSA presentations. Topics covered ranged from sexual health, to cognitive function, psychological well-being, social isolation, successful aging, and resilience. These presentations highlighted research findings that have been critical in developing interventions and shaping policy initiatives to support this growing population. Part of a symposium sponsored by the HIV, AIDS and Older Adults Interest Group.


Global Oral Health aims for optimal oral health for all people and elimination of global health inequities through health promotion, disease prevention, and appropriate oral care approaches that consider common determinants and solutions and acknowledge oral health as part of overall health. The purpose of this short communication is to generate a narrative around our proposed definition of global oral health to support establishing guidelines and developing best practices for academic global oral health programs, policies, and practices that aim to achieve a goal of oral health globally.Definitions can generate actionable consensus for a given subject matter by resolving important differences in philosophy and best practices and by streamlining activities for a stronger strategic direction. Interest in the global dimensions of oral health, a generally neglected area of global health, is growing; yet, no previously published research has defined the term ‘global oral health.’ As such, the Global Oral Health Interest Group of the Consortium of Universities for Global Health determined a need for an introductory definition of ‘global oral health’ to guide program planning, implementation, and evaluation. With the oversight of an expert senior Task Force for the Definition of Global Oral Health, we employed a mixed-methods approach using the more common expert consensus-building Delphi technique combined with the lesser utilized Q methodology. This approach allowed us to identify the interconnectedness of global oral health themes and integrate multiple, seemingly disparate, topics into a single streamlined concept. Our resulting definition is as follows:  They identify and resolve important differences in philosophy, best practices, and action in order to streamline efforts and determine strategic direction –3. For eeving it .Oral health is a generally neglected area of health, both on national and global levels, and interest in the global dimensions of oral health has grown ,6,7,8,9.In attempting to ascertain the broad, qualitative nature of disparate perspectives concerning a definition of global oral health, we employed a mixed-methods approach using the more common expert consensus-building Delphi technique , combineBeginning with the Task Force members, and using a snowballing technique for broader representation of expert opinion and philosophy, we identified an additional 12 professionals from other interested organizations beyond CUGH  to participate in the surveys .Table 1.The Phase I survey was designed to be intentionally broad, allowing the expert participants liberty for creative and unrestricted responses to generate sufficient data for later, more structured survey development designed to yield a convergence of opinion. This phase used a semi-structured brief Qualtrics survey instrument in which respondents were prompted to answer a series of five open-ended questions by providing statements about global oral health. The questions were developed de novo and were designed to both capture a broad array of expert opinion statements related to ‘global oral health,’ but also to remain concise enough to mitigate excessive and cumbersome data collection.Phase II utilized the Q-methodology sorting approach to quantify participants’ level of agreement with the results from Phase I. A Q-set of statements was generated from Phase I, and participants were asked to sort them according to their level of agreement with each one, ranging from ‘Most Agree’ to ‘Least Agree.’ The Q-sort followed a quasi-normal-forced choice distribution, in which the boxes at the tail of the distribution accepted the fewest statements, while the middle group (Undecided) permitted the greatest number of sorted responses. Each box was scored, ranging from −5 (Least Agree), to +5 (Most Agree), see Q methodology round were isolated; the final survey also asked respondents to rank these statements in order of preference. Factor arrays were performed to validate which statements were in consensus with which factors. For each factor, we recorded the number of participants who loaded onto  it based on their ranking results, the Eigenvalues (significance of the factor based on respondent ranking), variance explained , and reliability (related to Phase III statement ranking). Grounded in the data, and based on statistical consensus generated from the factor and statement rankings, the research team drafted a definition of global oral health.In Phase III, respondents were asked to rank the three factors in order of preference. A smaller set of individually coded statements that were statistically determined to generate adequate consensus from the Global Oral Health aims for optimal oral health for all people and elimination of global health inequities through health promotion, disease prevention, and appropriate oral care approaches that consider common determinants and solutions and acknowledge oral health as part of overall health.The Phase I round of surveys yielded 35 statements , a standThe GOHIG definition for global oral health shares several common themes with literature reports on oral health in a global context, including integrating oral health and primary care and streamlining efforts for preventing and controlling oral diseases and other NCDs ,17. The Related definitions reinforce but also differentiate themselves from our recommended definition of global oral health. Glick et al., in defining oral health, emphasize its relationship to overall health as well. Review of their approach illustrates that ‘oral health’ can be viewed as a state of being , versus This study presents the first-ever definition for ‘global oral health.’ Definitions are important vehicles for resolving differences in philosophy, strategy, and action and for streamlining priority-setting and determining best practices. The proposed definition is well aligned with important public health and contemporary global health frameworks and complements the GOHIG Global Oral Health Competency Matrix. We hope this proposed definition stimulates international discussion among stakeholders, which in itself is a key aspect of the process towards alignment and consensus building for academic global oral health programs, policies, and practices.


Traditional Chinese medicine (TCM) has long been used to treat chronic kidney disease (CKD) in Asia. Its effectiveness and safety for CKD treatment have been confirmed in documented studies. However, the prescription rule of formulae for Chinese medicinal herbs is complicated and remains uncharacterized. Thus, we used data mining technology to evaluate the treatment principle and coprescription pattern of these formulae in CKD TCM treatment.  Data on patients with CKD were obtained from the outpatient system of a TCM hospital. We established a Chinese herb knowledge base based on the Chinese Pharmacopoeia and the Chinese Materia Medica. Then, following extraction of prescription information, we deweighted and standardized each prescribed herb according to the knowledge base to establish a database of CKD treatment formulae. We analyzed the frequency with which individual herbs were prescribed, as well as their properties, tastes, meridian tropisms, and categories. Then, we evaluated coprescription patterns and assessed medication rules by performing association rule learning, cluster analysis, and complex network analysis.  We retrospectively analyzed 299 prescriptions of 166 patients with CKD receiving TCM treatment. The most frequently prescribed core herbs for CKD treatment were Rhizoma Dioscoreae (Shanyao), Spreading Hedyotis Herb (Baihuasheshecao), Root of Snow of June (Baimagu), Radix Astragali (Huangqi), Poria (Fulin), Rhizoma Atractylodis Macrocephalae (Baizhu), Radix Pseudostellariae (Taizishen), and Fructus Corni (Shanzhuyu). The TCM properties of the herbs were mainly being warm, mild, and cold. The tastes of the herbs were mainly sweet, followed by bitter. The main meridian tropisms were Spleen Meridian of Foot-Taiyin, Liver Meridian of Foot-Jueyi, Lung Meridian of Hand-Taiyin, Stomach Meridian of Foot-Yangming, and Kidney Meridian of Foot-Shaoyin. The top three categories were deficiency-tonifying, heat-clearing, and dampness-draining diuretic.  Using an integrated analysis method, we confirmed that the primary TCM pathogeneses of kidney disease were deficiency and dampness-heat. The primary treatment principles were tonifying deficiency and eliminating dampness-heat. Chronic kidney disease (CKD) is a worldwide public health problem and its prevalence is more than 10%. It eventually progresses to end-stage renal disease in 20–30% of cases, resulting in tremendous social and economic burdens . Many ChSyndrome differentiation and treatment are the core of TCM in clinical practice. There are three basic principles for curing disease with Chinese medicinal herbs: eliminating the cause of illness, dispelling pathogenic factors, and restoring coordination of the internal organs so that excess and deficient yin or yang can be corrected. Most TCM prescriptions consist of more than two herbs. According to the TCM theory, several herbs prescribed together are superior to a single herb for disease treatment because the full extent of their advantages is realized and their disadvantages are inhibited. Different therapeutic effects may result from prescribing different herbs and herb combinations; the toxicity of other coprescribed herbs may also be decreased. In clinical practice, it is necessary to be flexible in modifying the formulae for Chinese medicinal herbs according to the condition of the illness and the status of the patient under the guidance of TCM compatibility principles. The specific prescription reflects both the patient's characteristics and the physician's experience. Only when the principles are unified flexibly can the prescription be congruent with the syndromes to be treated. Then, the expected efficacy will be achieved. Thus, it is necessary to follow the principle but not adhere to the initially established formula. Collection and analysis of formulae are one of the essential methods for learning and passing on knowledge of TCM. However, flexible intervention is extremely complicated in clinical practice, which causes considerable difficulties for clinical research. It also limits TCM from further developing and becoming more widespread. Many researchers and nephrologists are concerned about how to utilize TCM herb formulae safely and effectively in kidney disease. Much attention has been paid to exploring the core herbs used in TCM, fundamental therapy principles, and prescription methods. Therefore, interpretation of herbal formulae is expected to help identify the most effective herbs and improve the treatment of kidney disease. It should also contribute to the development of novel drugs for CKD treatment.Machine learning has been used to analyze the prescription patterns of Chinese medicine from clinical data and to discover potential associations between Chinese medicinal herbs and disease . At presIn this study, we proposed an integrated analysis method to extract and mine TCM medical records. We collected clinical cases in which patients were given herbal prescriptions by one of our hospital's TCM specialists, Professor Lijuan Gong. Then, we analyzed prescriptions for CKD treatment using association rule learning, cluster analysis, and complex networks to explore conventional TCM prescriptions, the law of medication, and potential core prescriptions. We aimed to identify the main treatment principle and herb prescriptions of TCM specialists for CKD treatment using modern data computer technology.2. Exclusion criteria were (1) acute kidney injury diagnosis, (2) receiving renal replacement therapy, (3) incomplete prescription composition or dosage, and (4) not receiving herb formula treatment. In total, 299 prescriptions in 166 patients were enrolled in this study. The study procedures were approved by the Research Ethics Committee of the Affiliated Hospital of Nanjing University of Chinese Medicine and performed in strict conformity with our institutional guidelines. The detailed workflow diagram is summarized in We retrospectively analyzed a patient sample from the Jiangsu Province Hospital of Chinese Medicine and determined the prevalence of prescribed Chinese medicinal herbs in patients with CKD from 2011 to 2016. CKD diagnosis was based on the National Kidney Foundation Kidney Disease Outcome Quality Initiative 2002 guidelines and was defined as abnormalities in kidney structure or function present for >3 months or glomerular filtration rate <60 ml/min/1.73 mWe screened all the clinical information from the clinical cases of outpatients. Patient information was collected, including age, sex, home address, diagnosis, and TCM diagnosis. Herbal prescriptions at each patient's first visit and subsequent follow-up visits were recorded.The data processing methods mainly included establishing the TCM knowledge base, normalizing the prescription's content structure, separating the compound medicine into individual herbs, standardizing TCM terminology, and correcting for manual error.In brief, we first established a Chinese medicinal herb knowledge base according to the Chinese Pharmacopoeia and the Chinese Materia Medica , 16. FirAll the data were processed by R-Studio Version 3.5.3. First, we used the Apriori algorithm to analyze the association rules of the herbs. Second, hierarchical clustering was used to classify high-frequency herbs. Finally, complex network analysis was used to confirm the core herbs prescribed for CKD treatment.X ⟶ Y is an association rule, X (left-hand side [LHS]) and Y (right-hand side [RHS]) represent the set of herb items, σ(X) is the frequency of itemset X, X ∪ Y is the union of itemset X and Y, σ(X ∪ Y) is the frequency with which itemset X and itemset Y appear together, support(X ⟶ Y) is the frequency with which X and Y appear together, and confidence(X ⟶ Y) is the probability that itemset Y appears in the presence of X. The lift is the ratio of the probability of itemset Y appearing in the presence of X to the frequency of Y. Support and confidence are often used to eliminate meaningless combinations; lift is the validity of the rules.The Apriori algorithm is a frequent itemset algorithm for mining association rules. We used it to illustrate the specific rules of TCM in CKD treatment. In our data, each herb was treated as a variable. A fixed combination can be understood as a herb pair in TCM theory. The formulae were as follows:In the hierarchical clustering algorithm, each herb was regarded as a cluster, and N clusters were combined to form a new class based on a similarity measure between objects. The Euclidean metric was used to calculate the similarity between herbs, the formula of which isHerbs used more than 30 times were extracted for word clustering.We used complex network analysis to confirm the core herb in each prescription. Complex network analysis is used to analyze complex interaction laws in complex systems in the real world based on a network model of nodes and edges. The complexities of diseases and human life systems are gradually being recognized such that medical research from a network perspective has become an important topic in current medical research. We regarded the constituent herbs for CKD treatment as nodes and connections between two herbs as edges. Thus, we could rationalize all medical record data into a network of drug nodes and edges using Liquorice  and the multiscale backbone algorithm.Node degree and confidence level are the keys to measuring the association between drugs. Node degree is the number of edges connected to a node, and the confidence level represents the reliability of the data. The multiscale backbone is a statistical model established by identifying significant edges; the parts of the network that are retained are those in which the nodes are strongly associated. We set the screening criteria as a node degree of 36 and a confidence level of 0.95.We analyzed 299 prescriptions of 166 patients with CKD. Of the patients, 66 (39.7%) were male and 100 (60.3%) were female, with a male to female ratio of 1 : 1.51. Age ranged from 4 to 82  years. On the kidney disease spectrum, most patients had glomerular disease, hypertensive kidney disease, or diabetic kidney disease.We analyzed herb frequency in all prescriptions because formulae for Chinese medicinal herbs usually consider two or more herbs. Among the 299 prescriptions, 202 Chinese medicinal herbs appeared 4123 times, as shown in We analyzed the properties and tastes of the herbs. As Chinese medicinal herbs have specific efficacy and can be classified into different specific action categories. Compatibility of Chinese medicinal herbs refers to the combination of two or more herbs with a purpose based on clinical requirements and medicinal properties and actions. It is the primary clinical method of medicinal application and the basis of herbal formulae composition. Here, we used data mining to assess the standard prescription patterns of the formulae for CKD treatment.In clinical practice, paired usage of Chinese medicinal herbs is essential therapeutically and comprises the herb prescription method. Thus, we analyzed herb pairs.First, we used the Apriori algorithm to analyze the association rules of the herbs in all prescriptions. We focused on two parameters: support and confidence level. Support was set as ≥30% and confidence level as ≥85%  and a toX-axis is the LHS and Y-axis is the RHS. The circle's color depth indicates the degree of lift, such that the darker the color, the higher the degree of lift. The size of the circle indicates the degree of support, such that the larger the circle, the higher the degree of support. We found three core herbs of paired prescriptions: Astragali Radix (Huangqi), Rhizoma Dioscoreae (Shanyao), and Rhizoma Atractylodis Macrocephalae (Baizhu). These herbs were usually combined with other herbs, such as Astragali Radix (Huangqi) paired with Fructus Corni (Shanzhuyu), Rhizoma Dioscoreae (Shanyao) paired with Spreading Hedyotis Herb (Baihuasheshecao), and Rhizoma Atractylodis Macrocephalae (Baizhu) paired with Poria (Fulin) to strengthen the spleen, tonify the kidney, and eliminate dampness-heat, respectively.A grouping matrix diagram displaying the general distribution of association rules grouped into categories with similar rules is shown in In Cluster analysis aggregates herbs into categories based on data attributes. The clustering algorithm is an unsupervised learning algorithm that can directly extract valuable information from the input data objects without input of any prior knowledge. It is widely used for data mining in TCM and mainly used for determining the compatibility law between drugs to identify the combination rule of different TCM therapeutic methods. The advantage of this method is that it allows for discovery of potential new prescriptions for CKD treatment. Here, we analyzed 32 core herbs that were prescribed more than 30 times using hierarchical clustering. According to classic TCM theory, five categories were considered suitable in this study. Chinese medicinal herbs are usually applied as a combination of core herbs and other herbs. To further analyze the role of various herbs and their combinations in CKD treatment, we used the complex network method and found that the core herbs in CKD treatment were Rhizoma Dioscoreae (Shanyao), Spreading Hedyotis Herb (Baihuasheshecao), Root of Snow of June (Baimagu), Radix Astragali (Huangqi), Poria (Fulin), Rhizoma Atractylodis Macrocephalae (Baizhu), Radix Pseudostellariae (Taizishen), and Fructus Corni (Shanzhuyu) . These eThis study determined effective herbal prescriptions for CKD treatment by proposing a comprehensive method based on clinical cases, integration of association rules, cluster analysis, and complex network analysis.We analyzed herb frequency and the properties and actions of herbs and assessed the compatibility rules between drugs. Our data showed that the most frequently prescribed herbs were Rhizoma Dioscoreae (Shanyao), Spreading Hedyotis Herb (Baihuasheshecao), Root of Snow of June (Baimagu), Radix Astragali (Huangqi), Poria (Fulin), Rhizoma Atractylodis Macrocephalae (Baizhu), Radix Pseudostellariae (Taizishen), Fructus Corni (Shanzhuyu), Radix Rehmanniae (Shengdihuang), and Fructus Lycii (Gouqizi). The properties of these herbs were mainly being warm, mild, and cold. The tastes of the herbs were mainly sweet and bitter. The herbs were generally distributed to spleen, lung, liver, and kidney channels, and most were deficiency-tonifying herbs, heat-clearing herbs, and dampness-draining diuretic herbs. The most frequently used combinations of herbs obtained using association rules were Radix Astragali (Huangqi) and Rhizoma Dioscoreae (Shanyao). There were three core herbs of paired prescriptions: Rhizoma Atractylodis Macrocephalae (Baizhu), Radix Astragali (Huangqi), and Rhizoma Dioscoreae (Shanyao). Cluster analysis was used to divide the herbs into five clusters. Herbs in cluster 1 mainly targeted the spleen and lung, herbs in cluster 2 were prescribed to nourish the liver-kidney and to clear heat-dampness, and herbs in cluster 3 were prescribed for clearing heat, cooling blood, and clearing dampness. Cluster 4 herbs were prescribed for regulating qi flow, eliminating phlegm, activating blood, and purging turbidity. Herbs in cluster 5 were not frequently used. We used the complex network method to analyze interactions between herbs and found that the core herbs were Rhizoma Dioscoreae (Shanyao), Spreading Hedyotis Herb (Baihuasheshecao), Root of Snow of June (Baimagu), Radix Astragali (Huangqi), Poria (Fulin), Rhizoma Atractylodis Macrocephalae (Baizhu), Radix Pseudostellariae (Taizishen), and Fructus Corni (Shanzhuyu).The properties and actions of Chinese medicinal herbs are the essential bases for the analysis and clinical use of herbs. They are summarized in medical practice and based on the theories of yin and yang, zang-fu, meridian tropisms, and TCM therapeutic principles . ChineseWe identified commonly used herbs and their combinations Tables  and 3. TThe pathogenesis of kidney disease is complex and often manifests as multiple syndrome combinations. The formulae determined by clustering were for composite syndromes. Herbs in cluster 1 were for tonifying the lung, strengthening the spleen, and removing dampness and were prescribed to treat spleen-lung qi deficiency and dampness. Herbs in cluster 2 were for nourishing the liver and kidney and clearing heat or dampness and were prescribed to treat liver-kidney yin deficiency and dampness or heat. Those in cluster 3 were for nourishing yin, clearing heat, and cooling blood. These herbs can be found in Zhi Bai Di Huang Wan and are prescribed to treat fire excess from yin deficiency and blood-heat syndrome. Those in cluster 4 were for regulating qi flow, eliminating phlegm, activating blood, and purging turbidity. They were prescribed to treat phlegm-damp, blood stasis, and turbidity syndromes. These four clusters of prescriptions for different syndrome types can provide useful ideas for CKD treatment. The herbs in cluster 5 had multiple functions, such as tonifying the kidney, releasing the exterior, and promoting digestion. It may be used auxiliarily to target complications and requires further study.Complex network analysis revealed eight core herbs for CKD: Rhizoma Dioscoreae (Shanyao), Spreading Hedyotis Herb (Baihuasheshecao), Root of Snow of June (Baimagu), Radix Astragali (Huangqi), Poria (Fulin), Rhizoma Atractylodis Macrocephalae (Baizhu), Radix Pseudostellariae (Taizishen), and Fructus Corni (Shanzhuyu). Interestingly, these herbs are included in clusters 1 and 2. They were also identified as high-frequency herbs for CKD treatment, so we believed that such a prescription could be used as a basis for further research. Modern pharmacology has also shown that these herbs can improve kidney injury through anti-inflammatory and antioxidant activity  20–29].–29.20–29Our data mining approach has several advantages. First, there was no requirement for the data structure. This is very practical for data mining of Chinese medicine prescriptions because the data structure of most Chinese medicines is not uniform. Second, we combined a variety of data mining methods to perform comprehensive analysis and to ensure that the conclusion was reliable. Finally, we focused on the main treatment methods and uses of herbs, which is an efficient way to learn TCM treatment. Our research also has some limitations. We collected only prescription information but not diagnostic information, so our conclusions cannot be completely confirmed. We also examined only effective treatment cases, not invalid treatment cases, so whether a gap exists between these two case types remains unclear. Thus, it is also unclear whether the core prescription should be corrected. Finally, the safety and efficacy of the core prescriptions were not evaluated and should be investigated in a future study.In summary, we adopted a practical approach based on formulae prescribed in clinical practice. To our knowledge, this is the first study on CKD treatment with TCM which combines frequency analysis, association rule learning, and complex network analysis. Using this approach, we determined the frequency and combination patterns of CKD treatment and discovered possibilities for new CKD treatment prescriptions. We also summarized herbal CKD treatment, finding that CKD syndromes are extremely complicated and various CKD treatment methods exist, such as those for cooling blood and eliminating phlegm. However, we found that the treatment methods were mainly for tonifying deficiency and clearing heat or dampness.Our study found that kidney, spleen, lung, or liver deficiency and dampness-heat were the primary TCM pathogenesis. Thus, the primary treatment principle was tonifying deficiency and eliminating dampness and heat. Furthermore, our study verified that the integrated analysis method can help to explore the TCM treatment strategy. This analysis of herbal TCM prescriptions potentially contributes to the development of novel drugs for CKD.