N = 310). The data is analyzed using the Smart-PLS 3.3.3 software for checking the data for preliminary screening and the measurement of hypotheses. The findings of the study show that the three components of sustainable corporate governance, i.e., concern for employees, sustainable corporate governance awareness, and the environmental aspects have a significant as well as a positive effect on technological innovation and hence on the sustainable economic space. Similarly, the study has recorded a significant moderating effect of IT governance on the relationship of technological innovation and the sustainable economic space. Overall, it can be seen that corporate governance, innovative technology, and a sustainable digital economy share a reciprocal relationship. Where corporate governance helps as a supporting force to keep both innovation and sustainability in action. Whereas IT governance provides enhanced communication and delivery of public services, business, and advanced human capital. The study will be of high advantage for the corporate sector in China for devising and modifying their policies that consider the employee’s concerns for the governance mechanisms prevailing at priority in the organizations. Further, it will be interesting for the organizations to incorporate the IT governance mechanism in their technological innovations for achieving a sustainable economic space.The need for sustainable corporate governance has gained the interests of researchers for a while now and it has been found as a very significant component of successful organizational operations. The current paper has examined the role of sustainable corporate governance in achieving sustainable economic space along with measuring the indirect impact of technological innovation and IT governance on the whole process. This paper has followed the quantitative-positivism approach to measure the hypotheses developed in the study. The population considered in this study are the managers currently employed in the corporate sector in China ( Today, the digital economy offers a diverse range of services that have impacted various traditional economic sectors including banking, transportation, health, education, publishing, and energy. This is because information and communication technologies have changed the dynamics of business as well as personal interactions. The digitally empowered platforms are the major catalyst of economic growth worldwide . SurprisFurthermore, the advent of digital innovation has given birth to multiple social and economic changes . The traGenerally, digital innovation is referred to as a constant process of improvement or a change any product or a business goes through to achieve sustainability in the digital economy . The curThe process of sustainable innovation is aided by both digital technology and management skills to bring forward creative business procedures and innovative solutions . For a sThe current major product of digital innovation is the birth of e-commerce that has shifted the power dynamics between customers and organizations. Today the clients have many major benefits including being able to compare the prices of different goods with just one click. Although, it is to be noted that customers aren’t the sole stakeholders that take advantage of the e-commerce structure almost every stakeholder involved in the process can get benefited. However, whereas the internet is undoubtedly an inclusive tool with responsible economic models and it has helped the economy to grow. It still needs to formulate ethical and lawful boundaries for higher sustainability and security. The frequent issues of privacy and security jeopardize the reliability of the online markets and push customers to have reservations regarding online business transactions. With the advancement of digital technology, a more sophisticated and safer model of interaction for both businesses and customers is required. This sustainable and secure model can help businesses to avoid the multiple ranges of issues including the social and environmental impact of their products . NumerouConsidering the importance of a sustainable digital economy, a large number of countries have implemented IT governments or e-governments by digitalizing the public services . IT goveMeanwhile, due to digitalization the traditional boundaries among almost all the sectors are distorted. The world of the internet is focused on eradicating the class differences in the accessibility of information and public services provided by governments. The current paper aims to explore this multifaced relation between digital economy, sustainability, and the role of good corporate governance in the whole process. It is significant to measure this relationship since the research and development in this sector are important to achieve a unified model for sustainable economic spaces. The study also explores the impact of digital innovation due to its proven significance in achieving sustainable economic spaces. In nutshell, the study acknowledges that with a fast-growing digital economy, it’s almost impossible for a business to achieve a constant process of innovation that is both socially and environmentally sustainable. Hence, sustainable economic growth can only be achieved by regulating these economic spaces under the proper models of corporate governance. The current study aims to explore the effect of corporate governance that is further divided into three different factors on the innovative technologies in order to achieve the sustainable economic spaces.There is ample of theoretical evidence that acknowledges the significance of examining the link between the sustainable digital economy and corporate government, empowered through technology also known as e-government. It would be important to investigate this relation to understand the social, political, demographic, legal, and economic factors that mediate the relationship between corporate governance and a sustainable digital economy. Numerous studies have already examined the development of the relationship between corporate government and sustainable digital economy . The litDuring the last quarter of this century, multiple researchers have acknowledged the fact that actions of any organization can have both internal as well as external consequences. Therefore, an organization should be conscious as well as accountable for its actions to the larger audience instead of only to its stakeholders. This sort of recommendation was first recorded in 1970. Similarly, many philosophers also highlighted the role of business as a member of society as well as the significance of its social performance. The businesses were aware of the requirement to adapt to community climate and be accountable but their constant focus on revenue generation had hindered the social receptiveness. Similar ideas were proposed by The concept of corporate sustainability has become conflicted since the time it has emerged. There are two common assumptions. First that it refers to development that is sustainable in its nature. The second assumption is that to achieve sustainability companies are only required to acknowledge social and environmental issues and make them part of their strategic plans. However, Today most companies are required to adapt to digital as well as innovative technologies in order to support their survival in a sustainable economy while also working for social welfare by answering the concerns of the public . The digIn order to achieve sustainability both digital innovation and economic spaces require regulatory frameworks . For thi1:HInnovative technology has an association with sustainable economic performance.2:HIT governance moderates the relationship of innovation technology and sustainable economic performance.The debate about sustainable digital Innovation cannot be carried alone without discussing the next big issue of corporate governance acknowledged globally. Many companies aim to become global and at the same time, they want to achieve sustainability to keep the competitive advantage. The concept of corporate government started to attract attention at the global level in the mid-1980s. Today, almost all professionals including managers, government officials, auditors, and human resource organizations have recognized its need as well as importance. The code of good governance by Anglo-Americans offered the right amount of motivation to develop the concept further. These codes have been adopted by the economies of both first worlds as well as developing countries into their own set of principles and goals. This set of goals has been chosen above legal standards mostly. With increasing risks for investors, corporate governance has become a safe tool since it fulfills the demand of stakeholders to implement strict corporate principles to guide the businesses. For this purpose, investors have shown interest to invest more to promote the principal implementation of corporate governance . In todaGood corporate governance has been under the discussion for many years, to attain a single clear definition of the concept. However, according to Since the literature review has already established that corporate government encompasses many factors that directly affect the sustainability of a business. The previous research shows that this relationship is not at all clearly understood by many firms. Before understanding good corporate governance, it is important to understand the four major aspects of sustainability presented by 1)Social Impact: It is the measure of the influence of society and stakeholders on the company including both ethical and corporate sets of rules or principles.2)Environmental Influence: This aspect of sustainability deals with the measurement of the effect of a company’s actions on the geographical and physical fabric surrounding the environment.3)Organizational values: This factor is entirely related to the company’s relationship with its internal stakeholder specifically employees.4)Financial Impact: This measures the decent return for the levels and extent of risks taken by a company.These are four major key factors to achieve sustainability in corporate governance. As they capture both internal and external factors, long-term and short-term goals in the context of both present and future. These factors embedded in any model of corporate governance pave a path towards sustainability. Subsequently, this sustainability offers the regulated distribution of both negative and positive effects and helps to remove the conflict between the two as well. Hence, the absence of sustainability in corporate governance leads towards the traditional short-term approach that is no more feasible in the always-changing digital economy . Therefo3:HConcern for employees has an association with innovative technology.4:HEthical and corporate governance awareness has an association with innovation technology.5:HThe consideration for environmental aspect has an association with innovation technology.It is observed that the existing post-industrial social fabric demands an economy based on knowledge and sustained by innovative technology as well as smart human capital resources . The resPrevious studies have only highlighted a few factors including strategies related to digital innovation, sustainability, organizational perspective, environmental impact, division of innovative employees, digital workplace as well as organizational readiness and learning . A compr6:HInnovation technology has a mediating effect on the relationship of concern for employees and sustainable economic space.7:HInnovation technology has a mediating effect on the relationship of ethical and corporate governance and sustainable economic space.8:HInnovation technology has a mediating effect on the relationship of consideration for environmental aspect and sustainable economic space.A following conceptual model  has beenThe current study follows a quantitative methodology with the deductive approach where hypotheses were developed and the study was checked for the effect of certain variables on other variables. This methodology has been used for the minimum bias. The data in this quantitative study has been collected through the self-administered survey method. The population taken for this study is the managerial staff of corporate organizations. The sample selection method used in the current study is convenience sampling, since reaching out to each manager is not easy, though reaching those given consents for being part of the survey was convenient. Therefore, a convenient sampling design was used. The questionnaires were distributed and collected a week later. The usable questionnaires received were 310 and they were used for data analysis. The unit of analysis in this study was the managers of the corporate sector in China.The survey instrument used in this study was the questionnaires. The questionnaire consisted of questions related to each variable. There were six variables in total for this study one independent variable, i.e., sustainable corporate governance that was split into three sub-variables , one dependent variable i.e., sustainable economic space, one mediating variable, i.e., IT innovation and one moderator, i.e., IT governance mechanism. All the scales were adapted from the previous studies, e.g., the scale for sustainable corporate governance was adapted from The data analysis done in this study in three stages. In the first stage, the demographics of the respondents were analyzed using frequency and percentages. The differences were noted on the basis of gender , age with highest contribution from the age segments of (15–20) and (21–25), education with three categories of bachelors, masters and Ph.D. and others; and the nature of job. The details have been mentioned in the Data of the study were analyzed using the software used widely for structural equation modeling (SEM), i.e., Smart PLS 3.3.3. Using this software, the data is analyzed in two stages; the first is the measurement model and the second is the structural model. In the measurement model, the validity and reliability of the data are checked. For validity, factor loadings of the items, average variance extracted, hetero trait mono trait ratio, and Fornell and Larcker tests were used. On the other hand for reliability, Cronbach Alpha reliability and composite reliabilities are used.The measurement model algorithm obtained is shown in The statistics obtained for factor loadings of each variable, alpha reliability and composite reliability along with AVE have been reported in Similarly, the cutoff point of Cronbach alpha reliability is also 0.7 . All theDiscriminant validity of the variables in the current study has been checked through the most commonly used tests namely Hetero trait Mono trait Ratio and Fornell and Larcker Criteria. The results for these two tests are reported in r-square values for the mediating variable technological innovation are 65.2% and that of the dependent variable sustainable economic space is 42.9%.Furthermore, the t-statistic, p-values, the original sample mean values, and standard deviation values. The details of these statistics are reported in The structural model algorithm obtained from the Smart PLS is shown in The structural model gives the output regarding the key indicators for accepting or rejecting the proposed variables. In this study, the direct effects of the variables have been shown in t-statistic = 5.111 and p-value <0.000. Similarly, the second and fourth hypotheses also showed significant positive results for corporate governance effect on technological innovation  and its effect on sustainable economic space . While, H3 has also been accepted at t-statistic = 2.124 and p-value = 0.05.In the first hypothesis of the study, concern for employees has shown a significant and positive effect on technological innovation with t-statistic = 1.992 and p-value less than 0.05. Similarly, the hypotheses H6  and H7  have also been accepted, however, hypothesis H8 could not find any significance in the study as depicted in Moving to the indirect effects, the moderating effect of IT governance mechanism has been found significant in the relationship of technological innovation and sustainable economic space with t-statistic, p-values, the original sample mean values, and standard deviation values have revealed that concern for employees has shown a significant and positive effect on innovation technology with t-statistic = 5.111 and p-value <0.000. It indicates the need for smart and innovative human capital for sustainable innovation  on the innovative technologies. In addition, it examines the mediating role of an innovative technology on sustainable economic spaces and moderating role of IT governance during this process. The results from novation . Similarnovation . The trinovation . Moreovenovation . Howevernovation . Overallnovation . Hence tnovation . Hence, Corporate governance has been a vital component in the organizations and the impacts it forms on the environments around and the economy overall. The current study has also made such an attempt to find the role of corporate governance on the sustainable economic space by splitting it into three constituents of concern for employees, ethics and corporate governance awareness, and the environmental aspect. These three independent variables  have found significant effects on technological innovation and sustainable economic space. The mediating effects checked with technological innovation have also been found to play a partially significant  role in predicting the effect of overall sustainable corporate governance on sustainable economic space for the organizations. Further, the moderating effect has also been found significant that the presence of IT governance mechanisms plays a vital role in using technological innovations for sustainable economic developments.This study has certain implications associated with it for the corporate world. The study will be of high advantage for the corporate sector in China for devising and modifying their policies that consider the employee’s concerns for the governance mechanisms at priority in the organizations. Secondly, it will be interesting for the organizations to incorporate the IT governance mechanism in their technological innovations for achieving and contributing a sustainable economic space. Thirdly, it will lead the organizations in the right way using information technology mechanisms in their organizational procedures and code of conduct that will help the country attaining sustainable economic space at the mass level. However, there are certain limitations as well. This has used the population of managers in China, which can be checked in Europe and other Asian countries as well. Similarly, more indicators of corporate governance and sustainable economic space can be checked for their contribution for the technological innovation and digital progress.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.RQ and YT conceived and designed the concept. CC collected the data and wrote the manuscript. All authors read and agreed to the published version of the manuscript.RQ was employed by the company Chengdu Santai Intelligent Technology Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Background: Within the EU, some of the challenges and perceived risks now facing medical device (MD) developers result from changes in the regulatory framework, emphasizing safety. Therefore, medical technology companies must adopt stricter quality assurance measures so that individual devices can be speedily tracked and retrieved in emergency situations.Objectives: We highlight the challenges and risks faced by the European medical devices industry, particularly those faced by SMEs in the Czech Republic. We address two important research questions: Q1. Do advantages from increased regulation outweigh the additional expenses? Q2. As many MD developers are SMEs, will the new regulatory regime result in some of those companies going out of business and therefore impede future innovation?Methods: The paper focuses on a single case study, with the situation and outcomes discussed in the context of the financial results of a further 50 medical device manufacturers marketing in the Czech Republic.Results: Our findings suggest that the new legislation will result in improved safety, facilitate product recalls, but the cost and administrative burden may be high. The evidence also indicates that some SMEs may be forced to diversify to “non-medical” products, with the inevitable loss of innovative MDs being made available to patients and healthcare providers. The European trade association for the medical technology industry (MedTech) state that “medical technologies can save lives, improve health, and contribute to sustainable healthcare” . The medAttitudes toward regulations are perceived differently. Concern has been expressed in the literature that the current regulatory regime for medical devices (MDs) is inadequate, biased toward commercial interests, that innovation outpaces the development of regulatory controls, resulting in social and patient risk implications –7. de MoThis industry's characteristics are the exceptionally high innovation potential , the aboThe main change under the new MDR regulations focuses on safety and risk reduction, which is to be achieved by strict processes that lead to market authorization . MedicalHowever, from an SME perspective, the new European legislation (highlighted in this paper) may result in a significant proportional increase in costs and an increased administrative burned, resulting in the impossibility for some companies to continue developing new products and, therefore, restrain innovation for the development of MDs.This paper aims to highlight the challenges and risks faced by the European medical devices industry due to recent legislation changes, particularly those faced by SMEs in the Czech Republic. We address two important research questions:Q1. Do advantages from increased regulation outweigh the additional expenses?Q2. As a large proportion of MD developers are SMEs, will the new regulatory regime result in some of those companies going out of business and therefore restrict future innovation?It is essential to highlight and bring to the literature the perceived risks (as a result of the proposed new legislation) facing SME companies that currently develop innovative MDs.Firstly, sector analysis provides an overview of the medical device market and MD companies' structure in the Czech Republic. Secondly, the case study of an SME company describes and analyses the situation from the point of view of regulation and the impact on the functioning of this company. Regardless of the company's size, these problems or requirements will have to be solved by every company.The importance of innovation in the medical device market and the actual impact on society, the market and the economy is often linked to regulations at national and global levels. It is essential to state the context of the current legislative conditions in force and to acquaint them with their possible impacts, which, for example, some authors have mentioned in the past, both negative and positive.Success in medical device development is determined by the variables of innovativeness, financial analysis and planning, user input in the development process, and company employees' engagement in the new product development (NPD) goals. It follows that successful product development and placement on the market depends on a complex interplay of factors relating equally to business strategies, technological solutions, human resources, and end-user involvement. Additionally, a survey found that new global innovations accounted for only 4.4 per cent of NPD projects in larger companies and 9.3 per cent of NPD projects in SMEs .The medical device sector is a rapidly developing industry  subjecteHourd and Williams  conducteNumerous obstacles can hinder the success of the enterprise. These include external cost-related issues, such as lack of funding , high imThe manufacturers of MDs cooperate with clinical workplaces to research and develop new devices and then in the application phase. This co-operation is essential for a high level of medical and nursing care. Therefore, modern MDs represent a fundamental and irreplaceable area for contemporary medicine, which needs to be further developed and innovated . An esseIt is difficult to draw any definitive conclusions when it comes to the relationship between regulation and innovation. The association is a highly complex one, and the impact of regulation on innovation is not always immediately apparent. This impact may be manifested indirectly and gradually, such as in subtle shifts in market structure in competition, business strategies and investment priorities .The new legislation's key goal, highlighted in this paper, is to provide a more transparent, efficient and “consumer-oriented” approach and increase patient safety. The effects of regulation on product quality have been frequently researched in the past. While many theoretical models illustrate the relationships between product quality, competition on the market, and price regulation, only a few studies describe the relationship between regulatory policies and technological innovation. However, a positive relationship between heavy regulation and innovation was established in the Porter hypothesis , which pIn contrast, studies that illustrate a negative impact of regulations on innovation  highlighWhat will be the possible consequences of the new medical device regulations [(MDR) 2017/745]? Will it support or hinder innovation?Medical technologies are characterized by a constant flow of innovation, resulting from a high level of research and development within the industry and co-operation with users. New product development can take from 1 to 2 years in terms of their risk class. Health means are divided into classes I, IIa, IIb, III. The placing on the market of a new or innovated product depends on the complexity of meeting the essential requirements and, above all, on time required for the conformity assessment by the notified body (NB).The following risk classifications and descriptions are taken from .Class I  – Provided sterile “and/or have a measuring function (low/medium risk) or reusable low-risk Class I devices placed on the market in a sterile condition, have a measuring function or are reusable surgical instruments: assessment of the technical documentation relating only to those specific features of the device, such as sterility, measurement, or reprocessing”  highligOne of the biggest problems for companies appears to be in acquiring clinical data, which will enable them to succeed in the conformity assessment or certification process, according to MDR 2017/745. Until now, MDD 93/42/EC requirements were perceived differently; the MEDDEV recommendations specified them. However, they were still recommendations. Within the MDR 2017/745, it is no longer a recommendation but a regulation. Thus, there is an obligation for manufacturers to demonstrate clinical data. However, it is not entirely clear whether clinical data from clinical trials or data from PMCF studies will suffice. Manufacturers believe that a well-designed PMCF process will provide them with sufficient quality clinical data to demonstrate their product's clinical safety. However, the rightly set PMCF process is an intervention in the economic situation of the company. It is the right time for manufacturers to decide what changes they will make to their processes and business model to avoid the financial impact on their business.As the new regulations come into effect, 2021 will be a challenging year for many companies in the MD market. The legislative changes aim to make MDs safer for patients throughout their lifetime. For companies, this means additional control costs, even after an MD has been launched and also new clinical trials for MDs with a lower risk class than in the past.From our research, it is clear that SMEs will be most affected by the change in legislation. Larger companies are better positioned to adapt to legislation changes because they already have a regulatory department to monitor and update the legislative requirements for their currently manufactured medical devices. Article 15 of MDR 2017/745 specifies that a manufacturer is required to have at least one person within his organization demonstrating his expertise in regulatory affairs, but at the same time admits that small firms may have such a person employed externally. From this point of view, the response to legislative changes in small companies will not be as dynamic as in companies that have established an internal regulatory department. These companies will be forced to make many changes that will impact their economic indicators.Our market analysis shows that the larger the company the lower the percentage change in increased costs caused by the MDR new requirements. In our dataset, it was also apparent that, in general, micro-cap companies are specialized in the production of a few MDs in higher safety classes. Therefore, they will be disproportionately affected by new regulations, estimates of increases in certification costs as % of revenues are height. Larger companies focus on the large-scale production of class I MD's where it is harder to compete by small companies due to the economies of scale. That leads to specialization on MD's in more demanding safety classes, which will, unfortunately, be harder hit by increases to certification costs. The above-mentioned leads to a potentially threatening situation for SME's. The performed sectoral analysis further showed that the results obtained in the case study of Mediatrade are relevant in the Czech Republic case, as micro-cap companies see  share maad-hoc and usually even retrospective . It allows to get the contract to a single bidder without a competition . In its reports, the Supreme Court repeatedly draws attention to CPWN's practices in securing public procurement, including at the level of ministers and institutions responsible for the purchase of medical devices and equipment. In the years 2011-2016, the share of these contracts ranged from 31 to 62% of their total volume. A common problem with non-standard methods in the health sector is the vendor lock-in, it means propriety lock-in or customer lock-in that makes a customer dependent on a vendor for products and services. Then the contracting authority often tends to compete for long-term contracts in closed procedures, for higher prices on the account of public funds. One of the specific examples in the field of healthcare is the creation of a cartel for the supply of medical equipment and modern technologies to the Czech hospitals . There ispective . This faspective .Concerning the development of MDs, the EU regulations' changes present many challenges to the MD industry. The new stricter (heavy) regulations aim to improve patient safety through more rigorous quality assurance measures, but there is concern that this may adversely affect SMEs.The Porter hypothesis implies that heavy regulation induces innovation while, on the other hand, there is a contrary view that heavy regulation is a barrier to innovation. Within the context of the new MD development regulations discussed in this paper, it would appear that, with respect to SMEs, heavy regulation may be a barrier to the innovation of new MDs. In contrast, with regard to larger organizations, such regulation may prove to stimulate innovation. However, while some SMEs may move away from MD development to non-MD products, they may transfer their innovative skills to these products to the medical profession's detriment.Regarding research question Q1, our research shows that the new regulations should ensure improved MD safety. However, the economic load may be excessive both to the producer and healthcare provider. A problem is not only the price of the conformity assessment process, which in the Czech Republic varies with regard to the type and risk class of MD in the range of CZK 150–600 thousand within the assessment according to MDD 93/42/EEC but another difficulty for the manufacturer is the obligation to submit clinical data for conformity assessment. Obtaining clinical data that can be used for MDs' clinical trials and subsequently setting up the PMCF system in clinical trials may also be beyond small companies' power. The evidence also indicates that some SMEs may be forced to diversify to “non-medical” products, with the inevitable loss of some innovative MDs being made available to patients and healthcare providers. Q2 has partly been answered under Q1, but as the number of MD developers reduces so will the number of innovative MDs, thus restricting future MD innovation. Following the introduction of the new regulations, further research should be undertaken to verify our findings and perceived implications for the development of MDs by SMEs. On a more positive note, due to the effect of COVID-19, the implementation of the regulations has been delayed until 2021, giving companies the chance to manage the transition more positively and even seek out benefits through the opportunity of a competitive advantage.Limitations regarding our study may arise from two perspectives; (i) the paper deals only with the production of MDs subject to MDD 93/42/EC, resp. MDR 2017/745 and does not, therefore, include MDs in class IVD, and (ii) it is based mainly on a single case study – Mediatrade. Due to the specifics of the selected company, namely its unique business model, previously received state subsidies and historical changes in the Czech accounting practices disallow exact replication of this case study. We have addressed this issue by supporting our research with a comprehensive analysis of the whole industry, which confirms the results obtained from Mediatrade. Nevertheless, we believe that the paper provides a valuable contribution to the literature concerning the innovative development of medical devices.We believe the paper is timely in that it addresses an important current issue regarding the MD industry's future. In further support of the timely relevance of our paper is the concern regarding the current crisis over coronavirus (COVID-19), where innovation in medical devices, especially in the field of personal protective equipment (PPE) and respiratory/intensive care ventilator equipment, should be encouraged and not suppressed through restrictive legislation, although safety must be paramount. This article can also be a basis for further research and addressing issues such as what has happened after that date to confirm the concerns expressed before introducing the new regulations or refute them. How have the SMEs reacted to the new regulations? Has there been a decline in MD innovations? Have some SMEs gone out of business or moved away from MD development? What impact could Covid 19 have on the MD industry?The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.PM, LP, and FL contributed to conception and design of the study. LR performed the statistical analysis and graphical outputs. PM and LH wrote the first draft of the manuscript. PM, FL, LH, LR, and LP wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Sistema Único de Saúde – SUS) is a universal access health system financed by general taxes, and one that has consistently progressed towards enabling comprehensive health care for the Brazilian population [In 2020, Brazil celebrates 30 years of the national health system creation and institutionalization of community participation in the health care sector. The Unified Health System (pulation . HoweverThe origin of the SUS date back to the Brazilian Sanitary Reform (BSR) movement, a broad social movement in the 1980s that defended Brazilian re-democratization, defined health as a social right, and led to the creation of the national health system. As other Latin American countries were moving towards segmented health care models, the BRS movement fought to transform the fragmented and exclusive system and to assure health as a citizenship right . After aIn 1990, Law 8142 instituted the creation of health councils (HC) as part of the organizational structure of the health system. The social bases that influenced the institutional design of SUS advocated for involving the general population in the decision-making process and in controlling health policies. Another element influencing the creation of HC was the expectation of maintaining a state of social mobilization. Thus, the HC would promote citizen education and act as a basis for supporting democratic principles in health. In this article, we discuss the lessons and challenges of institutionalizing community participation in the 30 years of the SUS.Important lessons were learned from the Brazilian experience with community participation in the health sector. Establishing community participation as a principle and institutionalizing HC as part of the SUS structure represents the political decision of Brazilian society to not only have momentary social participation, but also make it a permanent and necessary element that can contribute to societal development. We consider capillarity, inclusiveness, and political influence capacity as important lessons learned from the Brazilian experience in health participation.Capillarity refers to the expansion of participatory bodies across Brazil. The association between the institutionalization of community participation and the decentralized organization of the SUS improved the expansion of participatory bodies in all federated entities at the three government levels, federal government, states, and municipalities. There are 5570 municipal HC in Brazil, 26 state HC, 36 district indigenous HC, and one national HC . AdditioInclusiveness corresponds to the involvement of multiple stakeholders in the participatory process. HC are organizations that have collective representations with the following composition: 50% community representation institutions, 25% health professionals, 12.5% managers, and 12.5% service providers. This architecture seeks to represent several interests directly related to health care provision [Political influence capacity refers to recognizing HC as a decision-making space in health. As defined by Law 8142, HC is responsible for deliberating on health policies. This law also establishes that HC decisions must be approved by the head of the executive branch. This represents an important mechanism to share power between the State and society, as decision-making is no longer the exclusive responsibility of health managers. Deliberation in the Brazilian health system includes making HC as an appropriate public sphere, thereby allowing different social segments to debate and negotiate on health policies that are better suited for the population.In this sense, HCs stand out not only for its capillarity and inclusiveness but also for being a potential space to control health actions and budget execution in the sector. HCs are characterized as bodies that aid community empowerment , that stDespite the progress achieved, HC presents important challenges to become an effective participatory space. We highlight the four orders of performance problems.First, the challenges related to community mobilization and involvement. It has been difficult to maintain high levels of community engagement over the past three decades. The institutionalization of participation was not enough to keep the civil society mobilized to defend the health system. Many HC find it difficult to function regularly, mainly due to the weakening of civil society participation. Because of a decrease in community involvement, managers and service providers play a leading role in conducting the activities, controlling the discussion agenda, and have a strong influence on the attitude of other representatives.Second, the challenges related to the bureaucratization and technification of participation. HCs have a wide range of attributions concerning the deliberation on health policies, monitoring, and inspecting the implementation of these health policies, analyzing accounting balance sheets, and promoting social mobilization . The repThe third challenge is related to effectiveness. Participation requires the ability to influence the health agenda ,8. Altho2 affirm that the Bolsonaro government has imposed successive direct attacks on SUS that threaten the universal character of the Brazilian health system. A weakening of the health care system causes a decrease in community participation as social segments start to discredit the capacity of the public health system.Fourth, the challenges arising from the weakening of the SUS and current ultraliberal policies. Over the past 30 years, SUS has never had enough funding, presenting structural problems that hinder access and affect health care quality ,2. SinceThe Brazilian HC is a mechanism of democratic innovation that can be considered a successful arrangement for community participation in health systems. Capillarity, inclusiveness, and deliberative attribution enabled the establishment of a participatory structure distributed throughout the country, which included community representatives in the decision-making process. After 30 years, the SUS experiences a process of social demobilization that threatens the effectiveness of HC and the SUS itself. Community participation is the only way to strengthen participatory bodies. We highlight the need to resume the process of social mobilization and the democratic project in health.


Community participation is central to the success of primary health care. However, over 30 years since the Alma Ata declaration, the absence of universal community participation remains a major obstacle to combating all types of diseases. This study investigated community participation in water and sanitation activities towards schistosomiasis control in Nyalenda B, an informal settlement in Kisumu City.Eight key informant interviews (KIIs) and 8 focus group discussions (FGDs) were conducted. Additionally, data on NGOs dealing with water and sanitation activities in Kisumu was collected from the local NGO registration Board. Qualitative data was organised into themes and concepts and analyzed using Atlas.ti.Most participants felt that project implementers did not involve them in key levels of project implementation, leading to unsustainable projects and unacceptance from the community. Community structures identified that could be used as avenues of engaging the community in improving water and sanitation situation included the use of organised groups, such as youth, gender-based, farmers and HIV support groups, and merry-go-rounds. Factors mentioned that hindered community participation included negative attitude from community members, poor monitoring and evaluation strategies, limited disclosure of project details, and overdependence from the community.Effective community participation in water and sanitation activities requires a multipronged paradigm that incorporates a change of attitude from the community, information sharing and consultation, improved monitoring and evaluation, transparency and accountability. About one-third of deaths in developing countries are caused by the consumption of contaminated water and, on average, as much as one-tenth of each person's productive time is sacrificed to water-related diseases.2 In addition, 10% of the population in developing countries is severely infected with intestinal worms related to poor faecal waste management.3 Worm infestations continue to be a major public health and socio-economic concern despite efforts and improvements that have been made to reduce helminth transmission worldwide.4 An estimated 207 million people in 74 countries are infected with bilharzia (schistosomiasis),5 90% of whom reside in sub-Saharan Africa.6Large populations in the developing countries do not have ready access to adequate water and sanitation even though it is universally accepted as a basic need. This situation is worse in informal urban settlements in developing countries where environmental contamination, often associated with improper waste and excreta management, is widespread.7 despite the availability of an effective and safe drug. While schistosomiasis has generally always been considered a rural phenomenon, there is increasing evidence on the existence and active transmission of the disease in urban settings.10 This is because of high populations in urban settings which facilitate high rates of disease transmission, larval development sites and poor waste disposal mechanisms.11 This creates a critical need to scale up control interventions against schistosomiasis in urban settings. Schistosomiasis can be controlled using 3 key approaches: improved sanitation, health education and treatment using available and safe drugs. Other control measures include targeting vector snails and avoiding contact with infected waters.12 Regular administration of anthelmintic drugs has strongly been advocated for by the World Health Organization (WHO) as an effective control strategy. The role of community participation in water and sanitation activities in the control of schistosomiasis has been advocated for, though few studies have looked at this in detail.In Kenya, over 9.1 million people are infected with schistosomiasis15 Community participation not only enhances control efforts but also guarantees their sustainment. In community participation, rather than be mere recipients, beneficiaries are actively involved in the planning and execution of the development projects. They also have a say in the nature and direction of the projects.16 A good example is community-directed mass treatment, which has successfully been used for the control of diseases such as lymphatic filariasis and onchocerciasis and is currently being tried for schistosomiasis.18 The strategy is well-accepted by communities since local members are fully involved and take responsibility for delivery of treatment.19 Schistosomiasis control efforts advocated for by the WHO largely focus on chemotherapy. However, these efforts can be augmented by involving communities in control interventions targeting water, sanitation and chemotherapy. Since community structures vary in different settings, they can influence the outcome of control efforts against schistosomiasis. This study therefore sought (1) to describe awareness on schistosomiasis and other water and sanitation-related diseases among community members; (2) to assess the water and sanitation situation in the area; and (3) to identify structures and issues that affect community participation in water and sanitation activities in the control of schistosomiasis in Nyalenda B. Understanding community participation in water and sanitation activities in the control of schistosomiasis in informal settlement areas is critical in designing, implementing and ensuring the sustainment of the control interventions.Although studies prove that improving access to clean water, changing hygiene, behaviour and proper waste disposal go a long way in preventing parasitic diseases, very little emphasis has been placed on this.20 that argues that for communities to be fully involved, there are 4 stages that project implementers need to bear in mind. First, information sharing stage is a one-way information sharing process from project managers to community members. This is done in the case of sharing preliminary findings. Second, the consultation stage involves programme managers not only sharing information with community members but also seeking their opinion on proposed projects. Third, the decision making stage involves engaging the community on project design, implementation and objectives. Fourth, the initiating-action stage is a proactive process where community members take charge of the health gaps in their community and are more involved in the implementation stage of projects.The study adopted Laodicah's model for community participation2 and an estimated population of 32,430, population density of 5,317 people/km2.21 It comprises 8,561 households with a total population: 32,430 people.20 Nyalenda B is divided into 5 health units, which are further subdivided into 9 subunits that are either headed by 1 or 2 village elders depending on the size of the subunit.The study was conducted in Nyalenda B, an informal settlement in Kisumu City, situated along the shores of Lake Victoria in western Kenya. Nyalenda B is 1 of the 8 informal settlements in Kisumu City, with an area of 6.1 km17 Schistosomiasis being a waterborne infection, It is, therefore, no surprise that the presence of intermediate host snails for schistosomiasis have been reported from some of the frequently accessed water points.10 Other studies in the area have also found high prevalences of schistosomiasis9 A map of the study area is shown below in the Kisumu, like the majority of fast-growing cities, faces various challenges such as overcrowding as well as water and sanitation issues that are common with urbanisation. For example, it faces an acute water shortage, and only 40% of the population has access to piped water. The majority of residents rely on unprotected wells and the lake which are subject to high degrees of contamination due to the rampant use of pit latrines and high water tables.This was a cross-sectional qualitative study involving 8 FGDs with local community members and 8 KIIs with individuals in various professions or assuming leadership roles in the community. Purposive sampling was used to select the respondents for the KIIs. The interviews were conducted in the participants' offices or rooms within their working environment. Each FGD with a maximum of twelve participants each were conducted. These were categorised by gender and age. The FGDs were conducted in classrooms in primary schools located within Nyalenda B. Participants were screened at the venue before commencement of the discussion to ensure eligibility and avoid selection bias. Participants for FGDs had to be adults aged 18 years and over, residents of Nyalenda B, and must have lived there for at least 6 months. A semi-structured FGD guide developed by the research team and pilot tested in Obunga, an informal settlement which experiences similar water safety and sanitation issues as Nyalenda B, was used. This guide covered: knowledge on signs and symptoms, community structures, issues and challenges in community participation and the roles of gender and disability. Both English and Swahili were used for data collection.Voice data from both the KIIs and FGDs were tape-recorded and later transcribed. Transcripts that were in Swahili were translated and back-translated to English to ensure that both versions carried the same meanings. Data coding was conducted by the research team. The coding structure evolved inductively with the codes from the narrative data of earlier interviews informing subsequent coding of the following interviews supplemented with field notes from the interviewer and note-taker. A coding frame was developed through open coding, a word-by-word analysis used to identify, name, and categorise explanations and descriptions of the day-to-day reality of participants as related to their perspectives on the water and sanitation situation in their community. Consensus on the coding frame was obtained through discussions between the 2 research assistants who had also participated in data collection. Each transcript was then examined to identify texts relevant to the coding frame. Quotes were later retrieved from the output monitor and arranged according to themes. Our data was validated by triangulation – verification of transcripts with the audio files, and discussions on the coding systems until agreements were reached.The study was part of a larger project evaluating the effectiveness of Community Directed Intervention (CDI) strategy in increasing access to treatment for schistosomiasis and soil-transmitted helminths in Nyalenda. This larger study was reviewed and approved by the Scientific Ethical Review Committees (ERC) of the Kenya Medical Research Institute  and from the ethics board of Maseno University. Additional clearance for our study was obtained from the Provincial administration and Ministry of Public Health and Sanitation (MOPHS) and the Municipal Council of Kisumu.The purpose of the study and its objectives were explained to local authorities, opinion leaders, village elders and community members. Consent was obtained from respondents. Participants were assured of the confidentiality of information during data collection. All personal identifiers were excluded during the discussion. FGD participants were given assigned numbers to anonymise the voice data further.Eight KIIs were carried out. Additionally, 8 FGDs were conducted, with a total number of 85 participants. Of these, 44 were females, and 41 were males. A few (7%) of the individuals had only elementary education – all of these participants were between 29 and 43 years of age. Nearly half (47%) of the participants had attended high school, while 42% reported having some postsecondary education. The majority of adult men were businessmen and fishermen, while most of the women were fishmongers. When the participants were asked about the most common diseases in the area in relation to water and sanitation, malaria, cholera, amoeba and diarrhoea received the most mention with schistosomiasis being brought up only 4 times. This further confirms the neglected aspect of this disease.A 31-year-old male CBO leader said:We have cholera due to poor drainage systems… when it rains, there is stagnant water. You find that these young kids – they play on this stagnant water, and this affects them so much. You find that sometimes they drink this dirty water.A 44-year-old woman, also a CBO leader said:I think in our community, we are experiencing some waterborne diseases like typhoid. Typhoid is rampant in this community, again at times we experience diarrhoea, this is cholera break out, and these are some common illness. Diarrhoea at times is not all that about water, but it is about sanitation.Participants understood the transmission of cholera and its relationship with sanitation but did not mention schistosomiasis. A 32-year-old salaried worker stated:You find children they don't go to the pit latrines for fear that they may drop inside the latrine. So, they just drop their faeces outside in the open – the same place you find the mothers also selling vegetables, the flies go from the faeces to the food they are selling, and they also take the food… this is where cholera comes in.It was the general view of participants that the water and sanitation situation in Nyalenda B is wanting. Factors mentioned included poor drainage systems, low latrine coverage, broken pipes and leakage of the sewerage system. Capturing the poor drainage systems in the community, a 37-year-old small business owner said:We don't have any drainage system in the whole of Nyalenda. What, maybe, the average income earners use is septic tanks and most of these people also don't empty the septic tanks when they are supposed to. Many of them… wait around rainy season; when it is raining, they drain them and the waters from up sweep the waters down the rivers.With regards to low latrine coverage, a 44-year-old male farmer said:I think even another problem is that the latrine coverage is very low…so they end up defecating on the open places, and you find that when the rain falls, it carries it to the river.On broken pipes and leakage of the sewerage system, an adult male said:I have ever used water, and later realised it had passed near some toilet, and the pipe had burst in the toilet, and it had passed very deep inside. There is no proper supervision. They are not put up to the standard that is required.Leakage of the sewerage system is further captured by the sentiments of a 44-year-old leader of a women's group:In this area, some people have dug pit latrines, and sometimes wells are dug near the latrines and water is water; the water in the latrine can flow into the well. When you draw, you may think you are getting clean water from the well, but you will be affected by this.This is despite the NGO registration board reporting that there are 43 NGOs in Kisumu dealing with water and sanitation. Additionally, an interview with the assistant chief in Nyalenda B reveals 4 NGOs working on the same. She was quick to add that there are many who continue to work in the area but do not pass by her office to inform her and she only stumbles on this information occasionally, while interacting with villagers or village heads in the subunits.Participants reported that they were fairly engaged by programme implementers. However, they mentioned that key steps in community participation: Information sharing, consultation and decision making had been left out in most programmes. This, in turn, led to unsustainable projects and unacceptance from the community. On information sharing, a 23-year-old male youth volunteering in a local NGO said:Garbage collecting tanks). People are dumping faeces.I wish that we would be told in advance what is happening. I think because people were not told what to use them for … office. Because it is the CHWs that do most of the things like health talk, Afya Two ... You find that they involve the CHWs. They inform them.I think those who participate mostly the community health worker they participate through the . Second, recent research supports active transmission in this setting.You see, bilharzia is 1 of the few diseases that are not talked about much. They are compared to things like HIV/AIDS, malaria and all that, ……I think the best thing is that when the community can be sensitised on the fact that the bilharzia is also a problem.26 further confounding its level of awareness. On the other hand, it was noteworthy that participants had a fair amoung of knowledge about other waterborne and sanitation-related conditions, such as cholera, amoebiasis, typhoid, and diarrhoea. Like many other informal settlements and as aptly captured by the respondents, Nyalenda B grapples with water and sanitation problems, among them poor drainage systems, low latrine coverage, broken pipes and leakage of the sewerage system. This area has an acute water shortage problem, with only 40% of the population in Kisumu City having access to piped water.17 Consequently, a majority of slum dwellers rely on water sources that studies prove to have high degrees of contamination such as springs, the lake and unprotected wells.27 Ad ditionally, poor drainage systems and low latrine coverage are attributable to high water tables coupled with black cotton soils and rock outcrops.The subtle morbidity or not easily detectable sequel associated with schistosomiasis may give the wrong impression of a less significant health threat that does not warrant seeking medical attention,21 leading to high levels of environmental faecal contamination. Further support of this contamination comes from a study that reported Escherichia coli contamination in 100% of a subsample of fish (Rastrineobola argentea) sold in 6 markets within the City.28 As a result, morbidities related to water and sanitation are on the rise such as diarrhoea and intestinal helminthiasis in this setting.29Studies show that about 11% of the population living in the slums have no latrines and rely on their neighbours' toilets, the wrap and throw (“flying toilets”) or open defecation in the fields,30 For instance, community- and faith-based organisations (CBOs/FBOs) are a major source of support for millions of families affected by AIDS.31A concerted effort is needed by the Municipal council, stakeholders and community members to improve the sewerage infrastructure and improve water treatment and increase access to adequate and safe drinking piped water. A key element of community engagement is participation by the individuals, community-based organisations, and institutions that will be affected by the effort. From the present study, participants identified structures in the community that could be used as avenues of engaging the community in improving water and sanitation situation. Among those that received mentions were: use of organised groups, such as youth groups, gender-based groups, adult women groups, farmers groups, merry-go-rounds, and HIV support groups. Indeed, such partnerships and groups can help mobilise resources and influence systems and serve as catalysts for changing policies, programmes, and practices.33 However, since water and sanitation or health problems have no gender discrimination, the emphasis should be on the pursuit of gender equity in participation and opportunity.Respondents' comments also highlighted the fact that community participation may take a gender-biased dimension, with women expected to take care of water and sanitation and health-related activities. This gender-biased dimension may stem from the fact that women, in sub-Saharan countries are traditionally expected to take the lead in food production as well as the general health and well-being of the family. Therefore, women are naturally at the heart of home-based care management.34 Furthermore, stigma can negatively influence their confidence in advocating for their agenda during programme activities or policy-making processes, therefore necessitating special attention where possible.Ultimately, the consideration of gender issues benefits everyone. Absence or low level of participation from people with disabilities, as seen in our study, may emanate from societal discrimination. Partiality and social intolerance in the community towards people with disability can prove an uphill task for them when they seek to contribute to policy processes or other participatory processes.36Management of water and sanitation services may provide valuable opportunities for the employment of youth in infrastructure maintenance programmes. Perhaps 1 of the ways to assess the community's performance on sanitation would involve use of strategies such as community-led total sanitation approach (CLTS) and assessing its impact on re-infection patterns of soil-transmitted helminthiasis, schistosomiasis and other waterborne and sanitation-related diseases. Community participation is a multilevel spectrum that needs to be nurtured. Several factors were mentioned that hinder community participation including negative attitude from community members, poor monitoring and evaluation strategies, which has led to project unsustainability, limited disclosure of project details to community members, and overdependence from the community. Project failures may occur when community members are unable to influence public action through accountability. Lack of transparency often fosters mistrust and misunderstanding between project authorities and local communities, whereas sustainability is often supported by the continuity of leadership.This study employed qualitative methods for data collection and analysis. This means that the participant's responses were the main basis for results and conclusions. Participant's responses may be biased, based on individual perceptions and attitudes towards the various topics of discussion. To control for this, the moderator took a neutral stance to avoid influencing the responses from the participants. Probing technique was also used to capture the participants' full understanding of the topic of discussion.9 suggesting that the disease is perceived to be of low public health importance. Poor drainage systems, low latrine coverage, broken pipes and leakage of the sewerage systems were mentioned as the leading factors associated with poor water and sanitation conditions in Nyalenda B. Use of organisational groups and partnerships was cited as an important avenue of engaging community members towards improving water and sanitation activities. Our findings suggest that promotion of and organisational development and sensitisation among community groups could well be effective for improving community involvement in water and sanitation projects. Community participation could be enhanced by attitude changes, information sharing, and consultation to improve monitoring and evaluation as well as transparency and accountability.Our results show that there was low awareness of schistosomiasis in this setting, despite the high prevalence as discovered in other studies,


As a joint effort from various communities involved in the Worldwide LHC Computing Grid, the Operational Intelligence project aims at increasing the level of automation in computing operations and reducing human interventions. The distributed computing systems currently deployed by the LHC experiments have proven to be mature and capable of meeting the experimental goals, by allowing timely delivery of scientific results. However, a substantial number of interventions from software developers, shifters, and operational teams is needed to efficiently manage such heterogenous infrastructures. Under the scope of the Operational Intelligence project, experts from several areas have gathered to propose and work on “smart” solutions. Machine learning, data mining, log analysis, and anomaly detection are only some of the tools we have evaluated for our use cases. In this community study contribution, we report on the development of a suite of operational intelligence services to cover various use cases: workload management, data management, and site operations. We formed the operational intelligence (OpInt) initiative to increase the level of automation in computing operations and reduce human interventions. In the approaching High-Lumi LHC (HL-LHC) era, we may be granted the computing resources needed to process, analyze, and store an order of magnitude more than the current LHC data. However, the person–power available to operate this infrastructure will not increase at a similar rate, if at all. Therefore, we pursue the outlined automation objectives in two ways: we organize a regular, technical, cross-experiment forum that brings people from various teams together to share ideas, experience, and code, and we develop tools to automate computing operations, exploiting state-of-the-art technology.The Worldwide LHC Computing Grid (WLCG)  project Notably, machine learning (ML) models applied to the prediction of intelligent data placements and access patterns can help to increase the efficiency of resource exploitation and the overall throughput of the experiments’ distributed computing infrastructures. Time series analyses allow for the estimation of the time needed to complete certain tasks, such as processing a certain number of events or transferring a certain amount of data. Anomaly detection techniques assist with prediction of system failures. Natural language processing (NLP) techniques can be used to analyze logs of services and extract information to automatically assist the operators; each year, thousands of tickets are submitted to ATLAS  and CMS In this article, we cover operational intelligence areas that focus on optimization of computing infrastructure, site operations, and workflow management and data management operations. We summarize activities that have been delivered to production and are already assisting with the automation objectives as well as those that are still in the prototyping phase, the fruits of which will only be harvested in the future.The structure of the article is as follows: in One of the most important aspects of the OpInt project is enabling the collaboration between the different experiments and teams that take part in it. High-energy physics computing is a complex and diverse world, but many components used in software development can be shared across projects. As we are a diverse community, some of the activities aim toward modest evolution and harmonization, rather than revolution, with a highly visible impact on users. Nevertheless, one of the challenges we are facing as we approach the HL-LHC era is the nurturing of cross-team, cross-experiment, and cross-domain collaborations, to share ideas and tools, working toward a common goal, together, making our community stronger and sustainable on the long term. In the following paragraphs, we summarize the areas of the challenges that we are addressing.The OpInt activity heavily relies on the monitoring infrastructure that consists of experiment-agnostic and specific components described later in this section. One of the challenges we address is finding the correct balance of generalization versus customization in the infrastructure configuration, in the tools, the data formats, interfaces, access protocols, and visualization. We believe it is important to offer a shared development platform for all the OpInt-related projects in order to foster code re-usability and enable a quick prototyping of new ideas by using pre-created components. Further description of the monitoring infrastructure is given in Predictive maintenance is a sought-after topic in a variety of applications to complex systems, aiming at identifying maintenance tasks, cutting operational costs, and boosting overall productivity. It is based on the identification of the system state, from diagnosis to prediction, involving decision-making activities to autonomously control the system itself. As such, it also works for computing centers and data centers as well as data infrastructures. The health maintenance of the computing systems can be divided in at least three big activity areas: 1) diagnosis, 2) prevention, and 3) prediction . This acThe intelligent alert system for the CMS experiment, further described in The Jobs Buster is an application that assists with spotting job failures and chasing their root causes in a timely manner. It is built with the OpInt development platform, comparing features of successful and failed jobs taken within the same time frame. The statistics of failed and successful jobs are used to train a failure model. The Jobs Buster is described in Clustering of error messages is a possible way to simplify the analysis of error messages encountered in large-scale distributed systems. With the considerable quantity of failures, the variety of sources and types of errors, and considering the unpredictable textual patterns of error messages, the traditional method of manually reviewing errors is impractical.The error message clustering employs the following categorization: messages encountered only once or several times are considered as anomalies, whereas messages with the same textual pattern and error conditions are grouped together. Groups of similar messages are then described by the common textual patterns and keywords. All messages are linked to the sources, and messages may be of various types and from different sources. The error message clustering is described in The distribution of the large volumes of data collected by the LHC is mainly managed by the file transfer service (FTS) . Each daAnalysis of FTS errors and failures represents a challenge to distributed data management (DDM) infrastructure operations. It is performed by teams of trained shifters and thus represents one of the challenges for OpInt: how can we improve the operations automation, in order to benefit from the expertise of the shifters elsewhere in the community? We have adopted an unsupervised learning approach to minimize the shifters’ effort and to enable the discovery of new failure patterns related to FTS transfers.A further description of the analysis of FTS errors and failures is presented in Rucio  is an opIn this section, we describe the infrastructure that the OpInt activity heavily relies on, with the experiment-agnostic and specific components.The CERN IT department offers a stack of monitoring services, referred to as MONIT , based oThe ATLAS monitoring infrastructure  is basedCMS benefits from MONIT, and its own monitoring infrastructure , which i• Abstract data fetchers to enable getting data from various sources, and a demo implementation of a fetcher that gets data from an Apache Hadoop endpoint.• A module to schedule and execute data-fetching tasks either periodically or on demand.• A module to construct REST APIs to share processed data with other services.• database integration to store fetched and aggregated data.• An abstract template for NLP-processing pipelines. Two of the pipelines that are used in our projects have already been adapted to follow the abstract implementation.• A sample application that can be used to kick-start new projects.In order to share expertise and re-use code and approaches across the board, we started developing the OpInt frameworkThe OpInt framework has already been utilized to run the Jobs Buster application described in The detection of anomalies in data center metrics using machine learning algorithms is an active field of development, with promising advantages in terms of increased quality of service and cost efficiency. The most important players in the field, namely large commercial data centers, often prefer to keep their research findings undisclosed to increase their competitive advantage. The WLCG community has started several initiatives to tackle this subject and fill this gap. One such example is a modular anomaly detection system, developed with open-source technologies, which can be easily adopted and/or adapted by other data centers and service managers . Besidesad hoc solutions for each existing system is counterproductive since a single data center can incorporate hundreds of computing sub-systems. For this reason, general-purpose solutions are preferable even if, initially, they are expected to perform worse than dedicated solutions.Log data are often unstructured, in which the format and semantics may vary significantly from system to system, making an approach toward a general-purpose log-based anomaly detection system extremely challenging. Supervised approaches provide more accurate results at the cost of requiring a large dataset of labeled entries, which is a hard restrictive requirement in the majority of real-world scenarios. Generally, labeled anomalous logs are not available and because of this supervised approaches are seldom used without expensive data pretreatment. In addition, to generate We are exploring feasibility of isolation forest (IF) techniques in the area of predictive site maintenance. The IF technique is a machine learning algorithm commonly applied to anomaly detection problems, for example, to identify anomalous samples in unlabeled data . A novelNLP techniques stand out as one of the most popular classes of strategies. Automated log parsing offers promising results through standard off-the-shelf NLP algorithms . FurtherThe functionality of the compute sites of the WLCG for the ATLAS and CMS experiments is verified by a large number of experiment-specific test jobs. These jobs are steered, controlled, and monitored by the HammerCloud  testing With job shaping, we aim to speed up auto-exclude and recovery decisions made by HammerCloud. This will be achieved by dynamically adjusting the frequency of test jobs based on the latest test job results. Since the auto-exclude decision is based on several functional test jobs processed on a given site, a delay of any individual job can delay the decision as a whole for the site. Therefore, we have analyzed the average run time and the time to start for all test jobs used for auto-exclusion over a week on all sites. The average run time is well below 30 min, which is following the requirements of efficient functional tests: they should be fast and lightweight in terms of resource utilization. The time-to-start peaks below 20 min, but it also has a long tail up to more than 10 h, as shown in . Based oIn order to speed up the decision time in case of jobs remaining in the state “submitted - waiting to start”, for example, if the current job is just stuck, increasing the number of parallel running test jobs would improve the situation. This “submitted - waiting to start” clogging may be a result of a queue mechanism mis-configuration or high demand for capacity at the site serving demands of multiple experiment in a shared environment, the central job scheduler of the experiment may have only a little wiggling space to attempt to unclog the site; one of the approaches is to avalanche more probing jobs, as the job shaping mechanism does. At the same time, in order to avoid wasting compute resources for unnecessary test jobs, the job shaping mechanism adjusts the number of parallel running jobs dynamically: in case of missing job results, five additional jobs per test type are submitted, and as soon as the tool recognizes jobs which have been finished within the last 2 hours, the number of parallel running jobs is decreased to 1 again.As shown in Within the CMS experiment, we developed an intelligent alert system to automate our dashboard with SSB and GGUS alerts. The system is based on the Prometheus and AlertManager RESTful APIs to fetch, create, and update new or existing alerts and supplement them with additional information obtained from SSB and GGUS services.First, we developed a general purpose scraper for the GGUS system by fetching its tickets via HTTP end-point in the CSV format. Then, we used CERN MONIT ElasticSearch instance to obtain SSB alerts.Finally, our system processes this information and constructs necessary annotations on a set of dashboards. All parameters are configurable: from various HTTP end-points to regular expressions used in alert matching as well as dashboard tags.We deployed this service in the CMS monitoring cluster, and it is integrated with our Prometheus and AlertManager services. So far, we annotate dashboards related to cmsweb services, which provide service overview for the experiment’s compute services. The current matching expression watches for network and database interventions and outages as well as issues related to the CERN MONIT infrastructure. The annotations are applied on a tag-matching basis, i.e., we put specific tags on our dashboard and use them for HTTP request annotations.Jobs Buster is an application built within the OpInt framework to detect large groups of job failures and identify the most probable cause. We used a CatBoost library  and the Jobs Buster uses eight features that describe a job, such as the site, the user who submitted a task, and the number of required CPU cores. It also preprocesses the text into diagnostic messages, constructing a corpus of words from all words in all messages for a particular time window. When performing the frequency analysis, a custom list of stop words is excluded. All diagnostic messages of a particular job are converted into a hash and then merged into one string. Once each job descriptor has this concatenated diagnosis string, all failed jobs are split into groups by the diagnosis value, and we train models within each group separately as described previously. This tool is continuously assessing jobs from the ATLAS experiment.detect anomalies (failures that happened only a few times in some period), to identify duplicated issues , to diagnose failures , and to analyze failures retrospectively.Analysis of error messages encountered in large-scale distributed systems has become one of the crucial tasks for the monitoring of computing resources. The monitoring systems utilize error messages to There is already a variety of tools for log and error message parsing that perform clustering using methods such as frequent pattern mining, machine learning clustering, grouping by longest common subsequence, heuristics, parsing trees, and evolutionary algorithms . But, thWe are developing an error messages clustering framework that consists of several stages: data trimming, vectorization, clustering, and clusters description. Data trimming—cleaning, tokenization, and regrouping of data—allows reduction of the initial number of messages by about 90–95%. Error messages are split into tokens, cleaned from insignificant substrings (for the clusterization), and regrouped by the equal cleaned patterns. The vectorization stage is based on the word embeddings technique: at the beginning, each token (or word) is converted to a numerical vector using the word2vec method, and then the average vector for each error message is calculated. At the clustering stage, we intend to explore various machine learning or statistical algorithms to cluster the numerical vectors, such as DBSCAN, HDBSCAN, Optics, Hierarchical, and k-means. Some of these algorithms are density-based as they do not require the initial knowledge of the number of clusters and allow the detection of anomalies. For the DBSCAN and Hierarchical algorithms, the epsilon parameter (intra-cluster distance) is selected automatically based on the data. k-means can be used additionally if we need the deterministic output. The accuracy of the clustering highly depends on the quality of the word2vec model. After training on a large volume of error messages, it becomes the basis for the mapping of error messages to numerical vectors. Larger models can achieve better accuracy in clustering tasks. The clusters description stage searches for common textual patterns and common key phrases for all messages in a cluster. For this purpose, we evaluate the performance of the Levenshtein similarity matching and various keyword extraction algorithms, such as RAKE, TopicRank, and YAKE.One of the main concerns in data transfer operations is to promptly detect and solve issues that affect the functioning of the infrastructure. On our way toward improving automation of DDM operations, we adopted an unsupervised learning approach to minimize experts’ effort and enable discovering new failure patterns related to FTS transfers. This tool targets the operators and aims to assist with understanding the infrastructure issues in the data transfer area.1) vectorization and 2) clustering. In the vectorization step, we concatenate the raw error string with source and destination hostnames, and we use a word2vec model that learns how to map all that information to a vectorial space of a given size, where similar errors are expected to be close together. This is to transform the textual information into a convenient numeric representation and serves as preprocessing for the next steps. A clustering algorithm is the applied to get groups of related errors: we pre-train a word2vec model on a big dataset and run a k-means++ algorithm  the cluster size, 2) the number of unique strings within the cluster, and 3) the number of unique patterns: unique strings after the removal of parametric parts such as paths, IP addresses, and URLs. The model learns to Overall, the idea is for the shifters to look at summary tables and time plots for each of the clusters detected by the algorithm, which act as suggestions of possible issues to investigate further, or later, would create automatic alerts, notifications, or even issue reports. The process described previously can be fully automated after tuning model hyper parameters based on metrics, such as average silhouette width score and within-cluster sum of squared (Euclidean) distances, that measure how compact and separated the clusters are. We have conducted an extensive testing as pre-validation comparing the clusters obtained with this approach against GGUS tickets, showing a reasonable overlap between suggested and reported issues. Although it makes sense to cross-check clustering results with tickets, this comparison has some drawbacks. In particular, the procedure is very sensitive to the choice of the time window. It requires a manual check of the ticket information and the cluster content, which makes the comparison lengthy and not scalable.Data in Rucio is organized using data identifiers (DIDs), which have three levels of granularity: files, datasets, and containers. Datasets and containers are used to organize sets of files in various potentially overlapping groups and to facilitate bulk operations such as transfers or deletions. Users are permitted to perform certain actions on the DIDs: downloads, uploads, or transfers. Due to the large sizes of the collaborations, the experts support a noteworthy amount of requests daily. To reduce the required efforts, we are looking into methods to assist the support team in answering these requests. Ideally, the support would be provided by an intelligent bot able to process and understand the user’s requests and, if possible, trigger appropriate automated action.The NLP assists in processing questions from users, where the objective is to provide satisfactory answers. A prototype bot capable of handling user’s requests up to a certain level of complexity has been developed, acting with emails as inputs. When a new email is received, it goes through the QuestionDetector, and a question object is created. Originally, the support bot was meant to answer incoming support emails. After an initial data analysis, we expanded that idea and introduced new data sources and the option to query the bot directly. A question from a user is treated as a query by the search engine in the next step. Once a question has been detected from a new email or given by a user query, the search engine looks into the support bot data storage and retrieves the most similar documents. Documents are archived questions from emails or GitHub issues and Rucio documentation pages from GitHub as well as FAQ question–answer pairs created by Rucio experts. The search engine subclasses exist for each data source. Once the most relevant documents have been retrieved, we use them as context for the new question in a SQuAD-like  manner tThe NLP applications described in this section are in different maturity states, from prototypes to production deployment. They aim to reduce human person–power cost in DDM operations, targeting partly DDM operation experts, partly suggestion, and chat systems for the user community.• Predictive Site Maintenance: We foresee tuning of anomaly detection approaches applied to data centers’ metrics as well as further exploration and exploitation of granular computing classifiers in predictive site maintenance. We intend to introduce specialized debugging tests to probe further the root cause of the failure with HammerCloud dynamic job shaping, and provide suggestions to the operators, or even assist in the automation of the issue reporting and mitigation.• Jobs Buster: We plan to add long-term models into Jobs Buster, enriching jobs features with information extracted from failures that occurred in the past.• Error Messages Clustering: Further framework developments involve code parallelization of the time-expensive stages.• FTS Logs, Errors and Failures Analysis: We wish to build a reference dataset to store labels for error categories, root causes, priority, and solving actions, assisting in performance optimization. Additionally, further NLP tools such as Question Answering or Named Entity Recognition (NER) look promising to support our target of understanding the root causes and suggesting solving actions for the issues.• NLP Applications in Rucio: Creating a user interface will allow us to deploy the support bot on a test server to a wider set of users, expanding the reach from the beta-testers and from the developers and experts community to the user communities. Their questions and the answers given can then be supervised and examined to create a dataset for the fine-tuning of the models. Additionally, we wish to create a NER tagger to detect Rucio-specific language entities, exploring potential to boost the tool performance significantly, and provide a way to create dynamic answers.We foresee activities developing in a variety of areas:The operational intelligence initiative brings together experts from various WLCG communities and is aiming to reduce the person–power cost of operating distributed computing systems. We presented an overview of activities in varying stages of completeness in the areas of computing centers operation and the workflow and data management, which represent only three of the areas, where innovative approaches can bring substantial improvement while we benefit from the state-of-the-art technologies.The projects described in Although some of the projects have recently entered only the prototyping phase, some are in the development and commissioning phase and some of the activities are already in production and bringing fruits in terms of saving person–power, already now: the intelligent alert system, combined with the monitoring cluster in CMS, and the Rucio NLP applications as well as Jobs Buster and HammerCloud job shaping in ATLAS.The combined monitoring and alerting infrastructure is a fully automated system. It consists of five Kubernetes clusters managed by less than two FTEs . It provides an enormous amount of information to the collaboration. The technology allows to automate many operations, from service deployments (via continuous integration/continuous deployment processes) to dashboard annotations (via OptInt modules). The use of common monitoring tools and technologies allowed the experiment to reduce the number of custom, in-house developed applications that would require the allocation of additional FTEs for development and maintenance. The automated service maintenance via a common set of alert notifications  provides much better insight into operations, allowing operators to timely address the most important and urgent issues.Rucio NLP applications assist with repetitive tasks in the area of DDM operations, particularly targeting the support of the user communities. The Jobs Buster provides insights into jobs failure causes in ATLAS, as it extracts the essential information from the jobs’ logs, and serves it in a comprehensive manner to the operators. The HammerCloud job shaping tool assesses the job failures of test jobs and avalanches a set of probing jobs to collect and abstract further information about the underlying cause of the failure.Since the beginning of LHC data taking (and even before that), digging into plethora of logs spread across different tools and servers, with different format to present the log information, and abstracting the essential information about the system failure, whichever system or systems they were, has been the core activity of the human operators, including the experts. It would account for at least 80% of the operators’ work time. Having automation tools such as Jobs Buster and HammerCloud performing the laborious steps of the failure chasing saga instead of the human operators is a great achievement and still have potential for automation that we are exploring as a next step, with the planned automated reports , describing the underlying cause and suggesting which corrective actions should have been taken. These tools not only reduce the time operators need to spend to identify potential issues but also help to increase the utilization of computing resources.The established procedures of large experiments are well-understood and trusted, such that integrating new ideas and approaches, even general ones, requires significant time and careful integration and validation. The complex heterogenous WLCG infrastructure represents a stimulating environment to apply data science techniques to assist in pushing forward the frontiers of fundamental science, and the initial objectives achieved supported by the positive feedback from the community confirm that integrating further projects into OpInt pipelines is a step in the right direction, demonstrating that collaboration across the table is not only possible but also fruitful, and it is one of the ways to make our community sustainable and capable of addressing the future challenges.