Background: Parental mental illness (PMI) is common and can lead to children developing mental disorders. Family Talk (FT) is a well-known and widely implemented intervention designed to reduce the risk of transgenerational psychopathology. However, given the research to practise “gap,” very little qualitative research, to date, has investigated practitioner experiences in implementing FT. This study aimed to explore the practitioner-perceived barriers and facilitators to the implementation and sustainability of FT within mainstream mental health settings.Methods: This qualitative study was nested within a randomised controlled trial (RCT) of Family Talk [N = 86 families ] within 15 adult (AMHS), child (CAMHS), primary care mental health, and child protection sites in Ireland. Semi-structured interviews and focus groups were undertaken with a purposive sample of clinicians (n = 31) and managers (n = 10), based on their experiences of implementing FT. Interview data were transcribed verbatim, analysed using constructivist grounded theory, and informed by Fixsen's implementation science framework.Results: Service providers highlighted a number of benefits for approximately two thirds of families across different diagnoses and mental health settings (AMHS/CAMHS/primary care). Sites varied in their capacity to embed FT, with key enablers identified as acquiring managerial and organisational support, building clinician skill, and establishing interagency collaboration. Implementation challenges included: recruitment difficulties, stresses in working with multiply-disadvantaged families, disruption in delivery due to the COVID-19 global pandemic, and sustainability concerns .Conclusion: This study is only the second qualitative study ever conducted to explore practitioner experiences in implementing FT, and the first conducted within the context of an RCT and national research programme to introduce family-focused practise (FFP) for families living with PMI. The findings illuminate the successes and complexities of implementing FFP in a country without a “think family” infrastructure, whilst highlighting a number of important generalisable lessons for the implementation of FT, and other similar interventions, elsewhere. Parental mental illness (PMI) is common, with 23% of all families having at least one parent who has, or had, a mental disorder , and a 4Given the prevalence and burden of PMI–and in the context of the principles and values enshrined within the United Nations Convention on the Rights of the Child–there has been a growing recognition in many countries of the need to support families in order to protect children from developing mental disorders , 9. ReasNevertheless, we know from the translation of other evidence-based psychosocial programmes that positive outcomes achieved in controlled research settings may not always be replicated within mainstream service settings , 17. AccTo date, only one published qualitative study of clinicians/managers' experiences in implementing FT has been conducted, despite FT being delivered as part of national initiatives in several countries . Eleven Whilst only one previous study has examined service-provider experiences of implementing FT, a small number of studies have reported on family experiences, which may help to inform workforce practise –32. Workn = 114) and child (n = 69) mental health services in the Republic of Ireland and found that support for families was either non-existent, in the planning stages or ad hoc and small scale (n = 14), identified several barriers to FFP, including lack of practise standards to identify service users as parents, no available structured approach, and an absence of appropriate training  was crucial in supporting the first endeavour to systematically implement FFP for families with PMI in Ireland. The aims of PRIMERA were to: (1) identify/develop, implement, and evaluate family-focused interventions for families with PMI; and (2) inform a “think family” care delivery agenda within mental health services in Ireland. Therefore, following an initial scoping and installation phase, we sought to introduce FFP into mental health provision in Ireland through the implementation and evaluation of FT   for the current “PRIMERA” research  , 40. Thihttps://trialsjournal.biomedcentral.com/articles/10.1186/s13063-021-05199-4;  guidance for complex interventions , 41, 42.05199-4; .n = 31), and managers (n = 10) were identified and recruited for participation in the study, based on their experiences of delivering FT to 55 families within the RCT.A purposive sample of mental health clinicians (n = 56) or to a treatment as usual control group (n = 30), and assessed at baseline and 6-month follow up. At follow up, attrition was 37%, the rate of which doubled due to the impact of the COVID-19 lockdown restrictions (23 vs. 45%). Eligible families were those with a child aged 5–18 years and a parent with a formally diagnosed mental disorder. Eighty per cent of service-users were attending AMHS and 20% were receiving antidepressant medication or primary care psychological support under the care of a General Practitioner  in 15 sites across Ireland, involving AMHS, CAMHS, primary care psychology, and child protection/welfare services . Familietitioner . Due to s 23 vs. 5%. Eligititioner . The 55 titioner .www.emergingminds.com.au), attend monthly supervision, and recruit families and/or facilitate FT. Families were recruited by clinicians in each site from their existing waiting lists. FT was delivered in an outpatient clinic and/or in the home by an FT clinician , parents (n = 25), aged 31–50 years (n = 26), with 14 employed in AMHS, 14 in CAMHS and 3 in primary care and the Tusla child protection agency. More than three quarters were employed as social workers, three as social care workers, and the remaining five as clinical nurse specialists and psychologists. On average, they had been employed as practitioners for 15 years (SD = 6.7), with most (24/31) having worked in multiple settings .Clinicians and managers were selected for interview based on service setting , professional discipline  and site location. All 15 sites were approached and interviews were secured with participants from five sites that recruited 10+ families each, from 3/5 sites that recruited ≤3 families, and from 1/5 sites that did not recruit any families. Three sites could not be contacted and three declined interview due to FT clinicians either leaving the service or having competing demands on their time due to the COVID-19 pandemic. Most of the 31 clinicians interviewed were female (n = 6) were principal/senior social workers, two senior clinical psychologists, one systemic family therapist, and one general manager.Ten managers were also interviewed, half of whom were female, six employed in AMHS, three in CAMHS and one in primary care psychology. Most , while one-to-one interviews with clinicians and managers lasted 30–45 min. Most interviews were conducted using online platforms  due to the COVID-19 pandemic restrictions. An interview schedule/topic guide was devised to investigate: (1) stakeholders' experiences of facilitating/implementing FT; (2) key barriers and enablers to implementation; and (3) factors mediating the longer-term sustainability of FT/FFP in their service/in Ireland. The interviews were conducted by CM, who had met with all service providers several times previously during the exploration and installation phases of FT implementation “My consultant psychiatrists and my team are excellent–and she hears that this work is done. And she's delighted! But she's one of the few psychiatrists who I've seen think systemically.” “More recent referrals have come from team members…that probably has a lot to do with a few more of the talks from myself, a team meeting generating referrals…one of the consultants in the team was quite eager.” “All clinicians/managers appreciated the structured, yet flexible, approach that FT provided in working with families. They also valued its evidence base and its manualised, no-cost, online training. Most also highlighted the importance of the psycho-education provided, and indicated that the skills they had gained were transferrable, although some noted that additional face-to-face training might be helpful for managing more complex cases.I thought the training was really good. I thought it was very accessible… I see the children and the parents get a lot from it… The checklist is really helpful…The structure is invaluable. It's really easy to evidence the work that I'm doing.” .“The fact that it was free, it was online, it's brief, that we could do it ourselves, it didn't require investment from the services–all those things appealed to us.” “I do think given the complexity of cases, you do need to modify, but the structure is there, and the structure is very accessible to most people. And that's one of the big strengths to it.” “Clinicians with prior experience of working in both AMHS and CAMHS were more committed to FFP implementation, having observed at first hand the transgenerational effect on children when they became service users. In addition, cross agency experience gave clinicians confidence and competency in working with the whole family, and facilitated interagency collaboration and co-delivery of FT, which considerably enhanced family recruitment and the quality of programme provision. Furthermore, most participants were social workers and believed that their professional training equipped them to be more persistent with family work when compared with other disciplines on mental health teams; for instance, they felt more competent in assessing family readiness for FT; establishing a positive relationship with families before and during FT; and in working with multiple family members.I spent time in both AMHS and CAMHS. You would see people being referred and you would see there was an inter-generational connection. What you often see is a history of parental mental illness and how that's impacted on them growing up.” “What I liked about it was having the mum and dad and the others all in the room together because while this may be new for some clinicians, it's not odd for the family, because that's the way they work as their every day.” “From my point of view, co-working works really well. The adult mental health practitioners being involved is really important because the children are very badly affected, so having this model of working on those cases, I'll be working with that going forward.” “An important and frequently reported implementation driver for clinicians/managers, was the benefits they had witnessed in approximately two thirds of the families with whom they were working; these included: reduced worry and stigma, a greater understanding of the impact of PMI on family members, a new family narrative around the parents' illness, and improved family communication. Clinicians indicated that parents/partners were typically surprised/upset by how much their children had been affected by tense/volatile home situations, and had hidden their worries and concerns to avoid burdening parents. For children, having their reality acknowledged, was significant as children were usually told nothing was wrong. As parents became more cognisant of their children's needs, family members were motivated to reduce levels of anger/arguments, and to relate to each other in more warm, caring and fun ways, thereby leading to reduced stress and increased family well-being. Clinicians further indicated that the improved family interactions/relationships assisted the PMI's personal and parental confidence and well-being.I think parents being able for the first time to hear their kid's opinions, and that they have opinions on it, they do have questions, and they're not in the dark–that does have a positive impact. Parents can become upset. I have had parents who cry in the feedback session. They can't believe they [children] knew what was happening, but there is some motivating factor in that for recovery. One parent I was working with for over a year had not shown a massive shift, but whatever it was about hearing feedback from her kids, and questions about her mental health, it seemed to motivate her. It did make a difference to her recovery.” “Their life is totally different. The mum had a lot of guilt and shame around her being a mental health patient. It was the first time she talked to the girls and they talked about the frustrations of mum not being available. She's able to speak to both the girls now. Mum is able to cook everyday when she couldn't before so life has become a lot more predictable, which is exactly what they wanted–so hugely beneficial for them.” “For the kids themselves, just to be given that space to talk and have their own voice heard is huge… Because the kids know without maybe knowing what the words are for it, but they know that there's something going on in the household…Takes a huge weight off their shoulders…In one family, both girls were actually blaming themselves for mum's illness because their aunt had told them it was their fault that mum was having relapses.” “That was the best thing he [service user] had done he said and because of the communication with his family, he's doing quite well again. He's more aware of the need to communicate.” “Most clinicians also believed that FT was beneficial for themselves and for their service. FT was reported to be enjoyable and rewarding and had helped to allay long-held ethical concerns about not addressing the needs of family members. In addition, several clinicians noted that FT worked well as a stepping stone for early identification of vulnerable families within their service, could be easily added to treatment plans, and was useful in signposting families to additional supports if required.It's definitely a hugely beneficial piece of work… I could feel it as a practitioner, and they could feel it as a family.” “If you think about it, this intervention is almost social justice. We're doing what we believe is right in developing children's rights.” “It was overall positive and really valuable work.” “Clinicians and managers indicated further that a significant motivating factor for their involvement in FT training and delivery—and another key implementation driver—was the fact that the research was funded by the HSE  and involved a multi-site, national programme aimed at addressing a major service gap in Ireland . Participants also clearly appreciated the wide range of advocacy and support activities undertaken by the research team to scaffold site buy-in, implementation, and family engagement. These included: co-developing a complementary online resource hub to assist clinicians in working with families; co-producing brochures and posters to recruit families; hosting/facilitating access to FFP workshops/masterclasses; co-delivering presentations to site managers and MDTs; providing regular updates by e-zines; and promoting the study through local and national media to raise public and service awareness on the topic . Thus, tWhat attracted it to us was the fact that it was supported by research, it was multi-site, it was a broader ‘Think Family’ agenda which appealed to us… The sense of being part of something bigger. There was a support structure there and we wouldn't have done this in a systematic way unless we were part of the research study.” “More recent referrals have come from team members, and that has a lot to do with a few more of the talks by the research team coming into the service.” “It was great to be part of the research. I feel it was a very exciting time and you guys are doing such an incredible job…I definitely intend to keep going. I would absolutely love to see it more evolved in Ireland. I'm a big believer in it.” “Engaging and retaining families was the primary challenge faced by service providers, and was one which was exacerbated by the COVID-19 restrictions. Clinicians indicated that three to four families had to be approached for every one successfully recruited, and in ten sites there were three or less families recruited . OverallClinicians indicated that for many parents–including those who agreed to attend FT–mental health stigma and concerns about involving their children, was a major concern and key barrier to implementation. Many parents disagreed about what should be discussed with their children, while concerns around social worker involvement with their children, also inhibited engagement. A small number of children in CAMHS also were anxious about discussing the issue with their parents. In addition, many of the cases on waitlists were complex  which may also have affected engagement and retention. Thus, extensive preparatory work by clinicians was needed to allay all of these concerns and fears. Clinicians also reported that some families disengaged before FT commenced/completed due to family crises , relapse in mental health symptoms, having other priorities or finding it too emotionally painful to hear from their children about the impact of their illness on them.As much as we're trying to reduce the stigma of mental illness… It's a massive thing still in Ireland. Especially I think for the parents. I definitely think more open communication is essential in families.” “I think it's probably about five or six families that said no. Their reason for saying no was, ‘don’t like social workers'… or fear that I'm going to start doing a parent assessment and that someone will be speaking to their child.” “The family withdrew… Maybe it was the difficulty of having to talk to her mum about how she was feeling about their relationship. They disengaged with CAMHS… And then COVID hit and to be honest, the crisis of the last couple of months… so that has been it.” “All sites experienced a number of organisational barriers that affected the engagement and retention of families, although some struggled more than others. Resistance to FFP from colleagues was reported as common due to: heavy workloads, staff shortages/high turnover, ideological differences , and feeling ill-equipped to undertake family work due to the individualised, crisis-oriented focus of their professional training. Other barriers included: slow referral processes; difficulties in identifying PMIs; needing to re-secure buy-in with new consultants who rotated on a 6-monthly basis; and colleagues being supportive in theory but not in practise as demonstrated, for example, by their unwillingness to train in FT or to refer families, a tendency to discharge suitable families without notice, and being risk adverse in balancing service-user confidentiality/data protection concerns with family needs.We've had locum six-month positions who are very good psychiatrists, but then they're gone. And they don't have any weight when they're here for six months and they are very dismissible” “Some will say that's not our job, it's a luxury, it's time consuming…Most other disciplines are trained just to work with an individual. So whereas we're going into the messy family life and that's a very frightening thing for services and they'll say to you, ‘oh GDPR’… It's very much a pushback, people aren't comfortable with it at all.” “Health services are reactionary. They deal with crisis after crisis… Which shows how slow we have been to look at preventive intervention… The other reluctance around this is that if you start looking at the psychological and social aspects of mental health, that may potentially reveal the delusion of psychiatry and the medical model.” “n = 7) but not as part of the RCT and, despite support from the research team, struggled to communicate to families the value of taking part in the research.In addition, there was evidence in some sites that insufficient effort may have been invested in recruiting families, which led to some not engaging with FT. For instance, it was reported that FT may have been poorly explained to families, or that parents had been informed by “cold calling” rather than through the building of a prior relationship with them. In addition, several clinicians indicated that negative past experiences of mental health/child protection services amongst some families, had led to their disengagement. Furthermore, some families dropped out following lengthy delays to FT delivery as a result of the COVID-19 pandemic restrictions in Ireland, and particularly in sites where mental health clinicians were redeployed to frontline COVID-19 duties . While iThe main challenge was recruitment. It's because they [colleagues] didn't explain it properly to the parent.” “We have been hugely affected by COVID… And after so much work put into it [FT]. That's been hugely challenging.” “We had a certain amount of time to complete it because of the [research] timelines so there's that added pressure to find families and get them seen. Once that is gone, it will be very good to see this as an integral part of AMHS. I really hope that happens.” “Ten sites recruited three or fewer families, only one of which (site 11) withdrew from the research; they did so because clinicians did not see FT as being a fit with the type of systemic family work which they wanted to undertake. The remaining nine sites were all characterised by limited resources , ideological differences, lack of a champion or practical support from colleagues, and/or lack of organisational readiness to engage families due to joining the study later in its lifetime and especially with the onset of the COVID-19 restrictions. Furthermore, eight of the ten sites had little history of interagency work, which possibly impeded recruitment. Notably, those sites in which child protection services collaborated with AMHS and CAMHS were more successful in engaging families than those who attempted to deliver FT without such interagency support; the latter struggled with clinician buy-in and family recruitment. Child protection practitioners/service providers in Ireland are typically not trained in mental health, and without interagency support, they may have felt less equipped to undertake family-focused mental health practise. In addition, given their limited resources and crisis-oriented focus, they may not have considered families with PMI to meet their criteria/threshold of a child being at risk.I felt a bit overwhelmed…I was the only one that took on the Family Talk intervention even though I spent a lot of time advocating for it… If I was rolling out Family Talk maybe in six- or seven-months' time, I think I would have had more space and the team would have gotten to know me better and trusted me with some of the families to see what social work can do. Within the team, the role of social work was a very basic view of the role of social work .” “Mum has mental health problems, a lot of trauma from her background… The family would really benefit from it [FT]. But Tusla said, no, it doesn't meet our threshold as Dad's a protective parent.” “A small number of clinicians indicated that the family meeting, in particular, was stressful, due to the emotional content being shared, and the requirement to support parents and children spanning a broad age range.What I found difficult was the family meetings, you were sitting with mum, a 16-year-old, an 11-year-old and a six-year-old in the room. You speak differently…You're still getting the essence across, but you're not being as frank about certain issues, or you're making it more child friendly because a child is there.” “Fidelity to FT protocols was also a challenge, with frequent delays/disruptions due to the COVID-19 restrictions. In a small number of cases, clinicians adapted FT using online platforms, which facilitated individual parent and older teen sessions, but was not considered suitable for younger children or family sessions, and therefore completion of FT was delayed. In addition, for families with more complex needs, one third of clinicians indicated that they provided additional parent, child and family sessions beyond the 7-session model, and referred families to further services . As FT was frequently the first (and perhaps only) time parents and children spoke about living with PMI, parents/partners were often angry/upset during initial sessions, while some service users needed time to adjust to not being the sole focus of care. Child meetings were also extended (if time permitted) when complex issues or concerns were raised.Due to the pandemic, I was unable to recommence Family Talk. It was impossible to start the individual meetings again and it just didn't flow straight into the family meeting. Otherwise, I feel the Family Talk would have been very successful.” “A couple of families had a lot of issues, and they needed time–one session with the kids wasn't going to be enough… And they needed follow-on supports that I was able to refer them to.” “Despite the disruptive long-run impact of COVID-19 , six sites have continued to deliver FT beyond the research programme, while the remaining areas hope to use its principles in practise, subject to resource limitations. The top five recruiting sites  appear bWe have continued receiving referrals for Family Talk and are continuing to deliver it to families. I am delighted that staff want it to become embedded in practice and our peer supervision group has become an established forum.” “We still continue here in CAMHS. I still fly the Family Talk flag as much as I can.” “We are going to continue using it in CAMHS. I think it's a very useful service. But definitely the challenge is the recruitment.” “I still use it. I use it in everyday work now.” “The intervention is really great so it's definitely something that we're going to continue to do with families. It should have been here a long time ago.” “A key sustainability issue concerned the perceived “fit” of FT with service remit; while many stakeholders expected AMHS to be the most natural fit for FT–and with CAMHS/Tusla perceived as being more proficient at family work–success in implementing FT appeared to be mediated more by local site resources, organisational culture and the availability of a strong champion. A small number of CAMHS clinicians within one site viewed FT as a mid-level intervention which was not sufficient for complex cases while CAMHS clinicians in other areas, working with equally complex cases, believed FT was appropriate. In addition, four AMHS and CAMHS clinicians believed that while FT principles would inform their future practise, the FT intervention would be better delivered at primary care level, given their lower threshold for access . Conversely, a clinician working in primary care psychology indicated considerable recruitment challenges due to a lack of willingness among parents with moderate anxiety/depression to acknowledge the impact of their difficulties on their children. This participant indicated that recruitment should be easier in AMHS where patients generally have a more defined diagnosis. Therefore, while FT was successfully delivered in all types of service—AMHS, CAMHS, Primary Care, Tusla—thereby reflecting a “no wrong door” approach to service provision , all sitI expected CAMHS to be very family focused. I've been really surprised that it had, like adults, become very much focused on the child is the problem and you fix the child. And you don't look at anything else.” “I think for us to use it to inform our practise but as a package, it probably would work better as a prevention piece on the primary care level.” “This is the problem when you fragment service, and they're not integrated. Tusla is a separate agency. Adult and CAMHS are very separate. This shouldn't be. Because children, come out of one family yet the family might be attending three or four different services, which is part of the problem.” “Given the complexity of some family cases, it was advised that sustainability of FFP in the RoI would be enhanced if FT was implemented as part of a suite of lower and higher intensity interventions. As indicated, clinicians felt the need to deliver extra sessions to several families, and frequently referred to follow-on services/supports, including individual and couple counselling, family and youth services, parent programmes, men's groups, and dialectical behaviour therapy. It should be noted that 76% of families in the RCT were socially disadvantaged and therefore presented with a high level of need.Some families probably need longer intervention… And then when the parent can't overcome stigma or family members are resistant, maybe something lower key in just talking with the parent might help also. But FT has been great for the families that come to it.” “ad hoc manner with little infrastructural support.For longer-term sustainability of FFP in Ireland beyond a small number of committed sites, all participants indicated that FFP is unlikely to flourish within the current medical, individualised, siloed, under-funded, crisis-oriented model of mental health care in Ireland which was perceived as encouraging services to believe it is not their core business to support families with PMI. Other systemic barriers noted by decision makers include a lack of data and accountability of how HSE funding is spent, and initiatives typically being introduced in an All participants highlighted the need for a multi-level, public health approach to raise service and public awareness on PMI, including: introducing a national “think family” policy initiative/practise guidelines; providing dedicated funding for FFP, and mental health services more generally; launching media campaigns to reduce mental health stigma; addressing systemic/interagency barriers to change . In addition, change agents (champions) need access to senior management to effect change at frontline, operational, and strategic levels. Given the movement of personnel within the HSE, multiple FFP positions are needed to ensure sustainable practise see .Interestingly, while all participants agreed that FFP was long overdue in the RoI, there was little consensus on the effectiveness of legislating/mandating FFP or the benefits of introducing standards which may, in practise, be reduced to a meaningless tick-box exercise with little benefits for families or clinical practise. Rather, participants emphasised the benefits of providing training in FFP, such as FT, and having managerial support to deliver FFP to families.I think if you make this kind of thing mandatory or legislative, it adds a little bit to the scary factor, both for families and us working with them… I think a better investment is to train clinicians in it [FFP/FT] and then support them to do it, allow them time. But you need to move beyond the individual, medical model for that.” “n = 45 from 23 families) who participated in a second qualitative study which is reported in a companion paper (ref withheld for purposes of anonymous review). The findings are also consistent with those of studies of clinicians and families who experienced delivering/attending FT in psychiatric settings in Sweden  previous qualitative studies of FT delivery –32, 47, Another key facilitator to implementation was the structured, manualised approach of the intervention, and its freely available online training, which greatly increased its accessibility for busy professionals working across different geographical areas. Nevertheless, some clinicians indicated that they would have welcomed supplementary face-to-face training with international FT trainers, and would have liked the online training to show clinicians working with more complex cases  and across a range of mental disorders. It should be noted that while clinicians in this study did not receive the 2-day, face-to-face FT training, they were required to undertake the online ‘Keeping Families and Children in Mind’ training to familiarise themselves in FFP prior to the FT training. They were also invited to several no-cost FFP masterclass/workshop events organised by the research team, whilst an online resource hub was also co-developed to supplement FT training in Ireland  .not mine, not trained, too busy, too risky” mindset that is a common barrier to FFP implementation  given the range of perspectives and developmental stages of family members. Therefore, it might be useful if the online FT training provided advice on how to tailor the discussion when children of different ages  are present. Secondly, most families indicated that they would have liked more child, family and follow-up sessions whereas one third of clinicians indicated that, for complex cases, they had already provided additional sessions beyond the 7-session model and had referred families to further services. The families' perspective most likely reflects their level of need , as well as the general unavailability of mental health/family supports in the community, whereas the clinicians' perspective reflects working within a context of limited service resources for FFP. Most previous studies of FT have not mentioned the need for additional sessions or follow-on supports, but this may be due to their participants being largely middle class and relatively high functioning , 51, 52.The sustainability of FFP in Ireland was a recurring concern for all participants. Reassuringly, six sites have continued delivering FT beyond the research programme and have established structures to enhance its sustainability, such as integrating FT into organisational procedures and care plans, and providing continued supervision and training of new FT personnel. Therefore, these sites have moved beyond Fixsen's stage of initial implementation, and particularly Site 1, but they have not yet reached full implementation as sustainability is still vulnerable to champions leaving the service . The remThese sustainability concerns raise questions about the perceived fit of FT with organisational remit and capacity. In many ways, given the individualised model of care in AMHS/CAMHS, it was a significant paradigm shift for these services to deliver a whole-family intervention, such as FT. While service providers appreciated the benefits gained from the whole-family model, there were nevertheless indications that FT should be implemented as part of a flexible suite of lower and higher intensity interventions, as recommended by international experts in the field . Higher Another key sustainability issue is identifying the type of service that is best placed to deliver FT/FFP. While AMHS may appear the most natural fit  was interviewed including clinicians and managers across a number of sites, including those that struggled with recruitment. The findings identified a number of barriers and facilitators to implementation and mirror the family experiences of FT reported here in our companion paper.This study is just the second qualitative analysis of practitioner experiences of implementing FT, and the first conducted within the context of an RCT and national programme to introduce FFP for families with PMI across AMHS, CAMHS, primary care and child protection settings (in Ireland). A large and diverse sample of stakeholders  had taken place because it was the focus of a national research programme funded by the HSE in Ireland. While some clinicians indicated that the RCT timeline also impeded recruitment, all RCTs are time-limited which means that some families were not ready to participate within the timeframe of the study or they did not wish to be part of the control group. Lastly, this is the first study of FT to be undertaken, in part, during a global pandemic. The COVID-19 lockdown restrictions halted recruitment, and seriously affected programme delivery and fidelity which led, in turn, to some family disengagement from services. Service providers were also interviewed during the height of the pandemic restrictions, which may have affected their perspectives given the impact of the pandemic on mental health in the general population , 56.Benefits were reported for approximately two thirds of families across different diagnoses and mental health settings (AMHS/CAMHS/primary care/child protection), thereby reflecting a “no wrong door” approach to identifying and supporting families. Key implementation facilitators included: acquiring managerial and organisational support through awareness-raising and buy-in activities; building clinician skill in systemic practise; establishing interagency collaboration across AMHS, CAMHS and primary care; setting up referral and supervision structures, and allowing clinicians sufficient time to engage in FT promotion, recruitment and delivery activities. Recruitment difficulties may be targeted by addressing issues of stigma, readiness, consent and confidentiality during the initial engagement process with families and including quotes/videos from previous FT attendees . In addiThe longer-term sustainability of FFP in Ireland, and elsewhere, requires a multi-level public-health response to address enduring political, cultural, organisational, and family barriers to change. Such a response would include: “think family” policy/practise standards; dedicated funding for FFP; managerial support to implement FFP; initiatives to reduce mental health stigma and recruitment barriers; and a continuum of FFP to broaden its capacity to identify families . “Think Although FT has been implemented in many countries, this is only the second qualitative analysis of practitioner experiences in implementing the programme. Therefore, further qualitative research of practitioner (and family) experiences is required across different cultural/policy contexts, disciplines and settings. Further research is also needed to identify measures and/or supports that might increase family engagement, including, for example, developing and evaluating training videos that teach recruitment strategies. In addition, the facilitators and barriers to implementation identified in this study  could be tested as moderator/mediator variables in quantitative research.In order to develop FT, and more broadly FFP, beyond a small number of committed sites, its longer-term sustainability in Ireland (and elsewhere) requires a careful assessment of the perceived fit of interventions with organisational remit and capacity, and the development of a multi-level public-health response to address enduring political, cultural, organisational, and family barriers to change . While lThe original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.The studies involving human participants were reviewed and approved by four Ethics Committees: the Social Research Ethics Committee in Maynooth University, Ireland (Reference Number SRESC-2018-100), the HSE Research Ethics Committee, Tusla Ethics Review Committee, and the Saint John of God's Research Ethics Committee. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.CM, MF, and SMcG conceived and designed the study. CM conducted interviews and coded transcripts, with 25% of transcripts independently coded by MF, SMcGa, and SO'C. CM prepared the initial draft, with subsequent drafts undertaken by MF, with input from CM, SMcG, and SMcGa. All authors contributed to the article and approved the submitted version.The PRIMERA research programme was funded by the Health Service Executive and the Maynooth University Higher Education Authority COVID-19 Costed Extension Fund.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The World Health Organisation framed responsiveness, fair financing and equity as intrinsic goals of health systems. However, of the three, responsiveness received significantly less attention. Responsiveness is essential to strengthen systems’ functioning; provide equitable and accountable services; and to protect the rights of citizens. There is an urgency to make systems more responsive, but our understanding of responsiveness is limited. We therefore sought to map existing evidence on health system responsiveness.A mixed method systemized evidence mapping review was conducted. We searched PubMed, EbscoHost, and Google Scholar. Published and grey literature; conceptual and empirical publications; published between 2000 and 2020 and English language texts were included. We screened titles and abstracts of 1119 publications and 870 full texts.n = 462/621) than earlier periods. Most of the publications were from Europe (n = 139), with more publications relating to High Income Countries (n = 241) than Low-to-Middle Income Countries (n = 217). Most were empirical studies (n = 424/621) utilized quantitative methodologies (n = 232), while qualitative (n = 127) and mixed methods (n = 63) were more rare. Thematic analysis revealed eight primary conceptualizations of ‘health system responsiveness’, which can be fitted into three dominant categorizations: 1) unidirectional user-service interface; 2) responsiveness as feedback loops between users and the health system; and 3) responsiveness as accountability between public and the system.Six hundred twenty-one publications were included in the review. Evidence mapping shows substantially more publications between 2011 and 2020 (This evidence map shows a substantial body of available literature on health system responsiveness, but also reveals evidential gaps requiring further development, including: a clear definition and body of theory of responsiveness; the implementation and effectiveness of feedback loops; the systems responses to this feedback; context-specific mechanism-implementation experiences, particularly, of LMIC and fragile-and conflict affected states; and responsiveness as it relates to health equity, minority and vulnerable populations. Theoretical development is required, we suggest separating ideas of services and systems responsiveness, applying a stronger systems lens in future work. Further agenda-setting and resourcing of bridging work on health system responsiveness is suggested.The online version contains supplementary material available at 10.1186/s12939-021-01447-w. The World Health Report of 2000 (WHR2000), ‘Health systems: improving performance’ broke ground framing health systems performance and development around three intrinsic goals: good health, fairness of financial contributions, and responsiveness to the expectations of the population – stressing the importance of responsiveness in particular in reducing inequalities, and improving the situation of the worst-off , 2.The potential and significance of a responsive health system is that it should provide inclusive, legitimate, participatory and accountable services, ensure the social rights of citizens, and draw attention to the needs of minority groups –5. More their health system, and decision-makers have enough and the right information to make informed strategic decisions  and [blinded], conducted during 2017–2020, resulting in output articles such as this, as well as a comprehensive documentary database (the database continues to be updated beyond the review end-date). Systematic evidence mapping reviews are increasingly being performed to map diverse literature in public health and health policy and systems research (HPSR) and involves systematic synthesis, organisation and interpretation across a large body of evidence, using rigorous and replicable strategies –44. The While this review approach includes assessment of relevance and quality (of the publication source), it does not set out to assess the rigor of findings within the included studies, nor seek to compare the outcomes or effectiveness of interventions described. This is a common characteristic of evidence mapping reviews, as the approach is designed to describe a large quantity of literature, rather than delve deeply into each included item . The appn = 6, [blinded]) – including experts in responsiveness, governance and accountability, in order to clarify topic scope and foci [We followed recommended phases for evidence mapping synthesis reviews including: 1) determining the scope and question of the topic under review; 2) searching for and selecting evidence; 3) mapping and reporting the findings of existing research; and 4) identifying evidence gaps , 50. In and foci . They suNext, we conducted a qualitative systematized review, keeping records of all searches conducted. Searches were performed using three electronic databases namely: EbscoHost , PubMed, and Google Scholar. The initial staged searches were conducted during June–September 2019. To be eligible for inclusion, a paper needed to include ‘responsiveness’ and ‘health system’ and their variations see . InitialAdditional literature was sourced through reference list searches, expert consultations, hand-searching through Google search results , and through online repositories such as the WHO and World Bank online repositories. These searches were conducted iteratively until saturation was reached and no new relevant materials, nor further topics were found .All abstracts were screened and included if they met the following inclusion criteria: (1) peer or institutionally-reviewed; (2) provided conceptual or empirical information on: responsiveness, accountability  or user feedback within a health system; (3) published in English; and (4) published between 2000 and 2019 . This period of publication was motivated by the inception of the conceptualisation and measurement of responsiveness by the WHO in 2000. No geographical limits were set.We excluded items that met the following exclusion criteria: (1) studies about physiological or biomedical responsiveness to medication or treatment program; (2) responsiveness as a psychometric property of data collection instruments; (3) responsiveness that was not related to health or the health sector; 4) studies on feedback between providers only (e.g. performance feedback); (5) studies that focused on patient-reported outcomes measures ; (6) items where full texts could not be sourced; and (7) items that did not provide substantial information on health system responsiveness in the full text, or used ‘responsiveness’ in a descriptive, non-specific manner.We examined the titles and abstracts/summaries to identify relevant items for further full-text screening. Three reviewers compared the eligible full-text documents and resolved discrepancies through discussions and consensus. During the initial screening process, we categorized items broadly as ‘empirical’ or ‘conceptual’. During the full-text review phase, we also conducted a further quality assessment phase, in which quality of publication source was assessed for all items , and empirical items were further checked for clarity relating to stated aims, methodology (and rigor relating to execution of this methodology), and substantiation of findings. From the remaining items, we then extracted descriptive data into an extraction sheet, including: year of publication, publication type, country, region coverage, country status (economic ranking), study design, populations/samples, contribution , and underpinning ideas and framing of responsiveness see . RefreshOur analysis in the review can be considered mixed methods given that we performed quantitative analysis (descriptive statistics) as well as qualitative (thematic) analysis. More specifically, we generated frequencies tables to determine the bibliographic results of the of the body of evidence, used thematic analysis to identify existing conceptualizations and dominant categorizations of health systems responsiveness.The database search yielded a total of 1084 records, and an additional 134 records found by other means , reviewing the collection of 621 items against consideration of publication rate, geographic location/focus, publication type, and empirical versus conceptual contribution. We have consolidated the graphics in Fig. In the last 20 years, there has been growth in interest and therefore publications on health system responsiveness. However, these are still very small numbers compared to the other goals such as health financing. After publication of the WHR2000, there was relatively limited interest in responsiveness (as indicated by publication numbers), until a decade later, around 2011 , chapters and books, theses, institutional reports , and The President’s Emergency Plan for AIDS Relief (PEPFAR); civil society, research and academic institutions), and commentaries/editorials/letters. Most items were articles 462/621), including 48 review articles (mostly focused on accountability). Commentaries/editorials/letters made up the next largest grouping (47/621) and there were 40/621 institutional reports  service performance and quality indicator, and the preferred method for measurement, via the WHO designed quantitative instrument, is an exit survey at point of care, or household survey of patient experiences.Responsiveness as feedback between users and the system: in this related cluster, the focus is on modes of gathering feedback from patients and patient representatives , and sometimes shows how feedback is utilized for service improvements.Responsiveness as accountability: which mainly reports on processes and structures that support accountability . Specific tools and mechanisms are suggested and assessed, thought to ensure that stakeholders  are answerable and held accountable for their actions.Beyond these eight conceptualizations, the explicit or implicit framing of health system responsiveness across the 621 included studies can be organized into three interrelated dominant ‘categorizations’:These categorizations are indicative, emerging from our review analytics, intended to give the reader a feel for the landscape (rather than to impose rigid classifications/typologies). The categorizations are therefore not totally distinct from each other, with obvious overlaps and relationships between them see Fig. . For exaSorting literature into these categorizations is also complicated by different use of the same terms. For example, to avoid confusion, in the discussion below, we have clustered the varied terms used for ‘individuals’, grouping ‘patients’, ‘clients’, ‘users’ (from categorization 1 and 2) who are all effectively health service users; and grouping ‘citizens’ and ‘community’ (from categorization 3) who form the broader public and could also include users or potential users. Despite these overlaps and complexities, we find the proposed categorizations a useful way of understanding how health system responsiveness is framed and understood across the literature.For categorization 1, health system responsiveness is understood primarily as a performance and service quality indicator , 62. ThiAs depicted in Table Measurement of responsiveness within this category was primarily quantitative, usually applying the WHO’s responsiveness survey instrument , 54, 64.Across items in this category, the WHO responsiveness tool has been validated and adaptations suggested – such as the addition of domains for education and information sharing , 36, effThe publications in categorization 2 Table , focus oactions taken in response to user and public feedback, usually emphasizing the need for robust information systems, and shared decision-making in the development, provision and improvement of services to meet the expectations and needs of the public [system. There are a few efforts in this direction, for example, the Patient Feedback Response Framework proposed by Sheard et al. [normative legitimacy, or providers’ sense of moral obligation and receptivity to user feedback; 2) structural legitimacy emanates from providers’ perceived power within organizations ; and 3), organizational readiness to change, a collective resolve to pursue the courses of action involved in change implementation [In this cluster, authors focus on e public , 78, 79.e public , so there public , 80. In d et al.  offers aentation .Within this categorization, publications can be divided into four themes: 1) receptivity of systems actors, which includes the exploration of users’ and providers’ perspective regarding feedback loops such as complaint management processes, patient experience and user involvement in services –85; 2) tWith regard to methodologies, feedback is usually collected at an individual (micro) level through self-reported instruments such as satisfaction-, quality-, or experience of care surveys , 98–102,In the third categorization, responsiveness is understood as a broader issue of accountability, not only to users, but to the broader public. According to Baharvand  ‘responsIn the available evidence, 32% (196/621) of the items pertained to this category. Most of these were recently published in 2011–2020 (146/196), mostly in the last 5 years (2016 > 2020 = 85); most are empirical (116/196); with most assessments focused on specific services ; and most relate to LMIC settings (89/196), with regional clusters focused on Africa (44/196) and Asia (38/196).Within this category, there is variation in how authors frame accountability, but they can still be usefully divided into those addressing responsiveness as it relates to ‘internal accountability’  or ‘external accountability’  , 110–113Among the papers in this cluster, the preferred method to assess internal accountability is the measurement of performance and quality assurance indicators for various building blocks , usually quantitively measured and narrowly framed , 117. Wioverall system responsiveness (more below).Moving beyond the conceptualisations of responsiveness found in the literature – this review next considers what is presented as best practice to make a health system more responsive. Across the (621) included studies, reports of interventions intended to enhance health system responsiveness focus predominantly on the introduction/strengthening of a particular ‘mechanism’. There is generally much greater mention of mechanism type, functioning and implementation approach than how feedback gathered via these mechanisms is acted upon, or how the system responds; there is also more on how mechanisms affect specific services than how multiple mechanisms/feedback channels impact on Specific mechanisms are considered within all three categorizations, although most frequently mentioned in the Category 3 (accountability) cluster. There, ‘mechanisms’ are understood to be governance tools that facilitate and enhance  accountability within a health system, or  between health system and the public , 13, 136mechanisms  focus on responsiveness ms Table  – in parevaluation of a mechanism; and 2) publications that describe the process of implementing a particular mechanism. The evaluative sub-cluster contains mainly empirical quantitative studies such as quasi-experimental, randomized-controlled or matched interventions designs (pre and post intervention) [voice/participation [In this cluster of 302 items relating to mechanisms Table , most prvention) –147. Thevention) , 149; hevention)  and indicipation –153. Whicipation .The studies focusing on mechanism implementation mainly rely on mixed methodologies and qualitative designs  –157, andAcross the 302 items Table , the mecThe cluster of publications relating to informal feedback and its  impact on health system responsiveness was significantly smaller (21/301) than that examining formal feedback. The most commonly described of this form of feedback was via social media, the studies being primarily descriptive, often relating to the potential for user experiences (or complaints) to be fed through social media to service engagement , 173–176There are also a few other items relating to other forms of media , and its substantive relevance as a concept and area of focus - as a value, a key performance goal, and an important accountability and communication factor. As the WHR2000 argued, improved responsiveness is a legitimate endeavor in its own right, for protecting and enhancing the population’s basic human rights . TherefoHowever, we also confirm that there are still major questions about every aspect of health system responsiveness: its framing (there are many), and theorization (there are few), resulting measurement (varied) and implementation practice (diverse). Without greater specificity, there is a risk that responsiveness remains a descriptive ideal, something mentioned in the introductory or conclusion sections of policies and articles – and the vital real-world application and effect remaining intangible.Conceptual and definitional issues have received little attention, despite this being a standard pre-requisite for empirical research and intervention. Some of this ambiguity emerges as a result of the diversity of the field – and future work in this area should continue to consider context-specificity. However, researchers might also ‘check’ their framing against three initial questions: 1) what constitutes a response?; 2) at what level is response anticipated (provider or systemic)?; and 3) who is the response for ? We do not see it as a task of this evidence mapping review to provide a ‘new’ definition for health system responsiveness. Instead, we would advocate for a broader and collective project of theoretical development, that emerges from context-specific realities, and that builds a dialectical bridge across the multiple interests and ideas described earlier.The lack of coherent framing is important, as this means there is no main coherent idea or theory to test and develop further. This review shows that while authors might use the same term (responsiveness), there are vastly different interpretations lying under this use, drawing from varied applications/sources, rather than iteratively building on clustered ideas, or linking and learning from similar applications in different contexts. The varied clusters of work on health system responsiveness remains largely siloed from each other and often based on individual interests. This has had an impact on the theoretical development , as well as the empirical evidence-base – resulting in wildly diverse conceptualizations of what responsiveness is and how it should be measured, as well as conclusions about how to improve ‘it’.Despite the order artificially imposed in this review, the evidential landscape remains largely ‘chaotic’ (an important finding in itself). The diversity of framing and focus, reflected in differing application of ideas and measurement approaches, makes it extremely challenging for researchers and practitioners seeking to enter this space. This might be a reason for its lesser traction than the other health system goals. Of course, diversity of ideas can encourage new thinking, and we are not encouraging conceptual ‘capture’ – but at this point, after two decades, this diversity appears to be more disabling than enabling.all of the ideas that underpin the compound concept that is ‘health system responsiveness’ would need to be interrogated and operationalised, as further research and implementation depends on achieving better clarity  framed. Tools for assessing health system responsiveness, that encompass a system thinking approach, would still need to be developed. For example, the national scale of the survey tool that emerged from the WHR2000 does not necessarily enable researchers to assess the complex systemic aspects suggested in the framings and categorizations described above. To be fair, the WHR2000 tools were intended to produce quantifiable indicators to allow for easier comparison across countries ; but it highly responsive to the needs of a wealthy patient group, but would not necessarily contribute to a responsive national health system  would not adequately assess whether systems are becoming more/less responsive over time, how systems are adapting to the changing needs of citizens, or how responsiveness relates to systems resilience (building positive adjustments to systems shocks over time). There were few assessments in this review that showed any type of cross-sectional assessment over time. Siloed and once-off service assessments do not show the fluidity of health systems, that change over time. Nor do they enable an understanding of varied levels of responsiveness within systems (or systems within systems), such as the variation between public and private sectors within the same national health system. For example, a for-profit health service might be t needs, . The ‘syThe case for health system responsiveness is also difficult to make because of missing empirical evidence Table . For exaBeyond geography, another major gap of the current literature is population. Although minorities and vulnerable groups are at the centre of the very idea of responsiveness, this review showed how rarely such groups are addressed - and this is a significant gap. All of these require more exploration, as does the broader connection between responsiveness and equity as it relates to a population as a whole.In the current evidence-base, many items focus on whether mechanisms are currently present and functioning or not. It also tends to evidence challenges facing mechanism implementation more often than enablers and success stories. There are only a few examples of short-term and quite limited successes – and even fewer examples available of fully functioning mechanisms, implemented and operating as intended, consistently ensuring citizen voice and feedback gets taken up by the system, and resulting in systemic response, over sustained periods of time. There are opportunities to mine and repurpose existing data on mechanisms for new uses. For example, satisfaction surveys are widely applied in multiple countries, usually at a national scale, but there are few examples of such being leveraged to support work on systems responsiveness . There are opportunities for comparing differences in mechanism performance in different contexts, and for integration of information about multiple mechanisms in the same system, to gain a more complex map of feedback.Researchers  have sought to take broader forms of feedback into consideration – for example applying rights-based approaches, taking broader ‘users’ into account. There is a large body of work on the types of feedback and empirical evaluations that demonstrate that feedback loops contribute quality improvement or systems changes. However, there is limited published literature that synthesizes the ‘how’ or the factors that hinder and enable feedback loops to facilitate a systems response. Further, of the included (621) studies tend to focus on the ‘gathering feedback’ and fewer on responsiveness as ‘the way the system responds or reacts to that feedback’. While there is evidence of feedback loops being in place and functional, what is not as clear is whether/how such feedback engenders response to citizen expectations. There is also as yet no robust explanations provided for how feedback leads towards a more responsive health system. Responsiveness is rarely framed as the actual (systems strengthening) changes made in the health system to address/respond to issues identified.While there is merit to further work determining the effectiveness of mechanisms, there has been a call to move towards exploring the more nuanced aspects of their functioning in context, and in consideration of accountability relationships . FurtherLittle is known about how informal feedback relates to formal mechanisms, or how either/both influence decision-making, or leverage the system to respond. The framing of responsiveness as accountability (more common in C3), lends itself more easily to take informal feedback into account – and generally relates more easily to a systems perspective. For example, pushing beyond the user-provider interaction and includes the public and other actors in the system to hold each other accountable. Another gap is further consideration of ‘multi-level governance’ as it relates to responsiveness – for example, generating perspectives of mechanisms and interactions inclusive of individual, collective and government actions and decisions , allowinThe substantive relevance of having responsive health systems has been convincingly argued – but the evidencing of this claims is not yet fully developed. This leaves health system responsiveness as a ‘nice to have’ or an ideal – rather than a concrete performance goal requiring routine monitoring, attention and resourcing. Although health system responsiveness is understood to be important in many ways – for example, ensuring the social rights of citizens, drawing attention to minority groups, supporting social cohesion, improving population health, improving systems functioning, and ultimately having a health system strengthening effect - at this time, these ideas remain untested hypotheses. There is very little literature providing evidence for these claims or showing how a more responsive health system is a stronger health system.This is one example of why there is still significant work to be done on health system responsiveness. In comparison with the other goals, there appears to have been a lack of prioritization and resourcing of work on responsiveness in the research, policy, and research/intervention arenas , 188. CuThere is major work to be done, for researchers and practitioners. For researchers, improved theoretical development needs to lead to improved  measures and tools – which need to be tested and extended in real world health systems. Better measurement tools (adequate for assessing this complex concept) should result in measurable improvements that can be pragmatically (and routinely) pursued by practitioners. For practitioners, if responsiveness is to move from being a ‘nice to have’ ideal, to a systems performance goal, then it needs to be taken more seriously, and more routinely monitored and considered. Ultimately, the question that remains is: whose responsibility is it, to ensure our health systems become more responsive? The answer might be as simple and as complex as ‘everyone’.Additional file 1. Search terms for review.Additional file 2. Data extraction table for the 621 publications included in the review and analysis.


Driven by people’s psychology of conformity, panic, group polarization, etc., various rumors appeared and spread wildly, and the Internet became a hotbed of rumors. (2) Methods: the study selected Weibo as the research media, using topic models, time series analysis, sentiment analysis, and Granger causality testing methods to analyze the social media texts related to COVID-19 rumors. (3) Results: in study 1, we obtained 21 topics related to “COVID-19 rumors” and “outbreak rumors” after conducting topic model analysis on Weibo texts; in study 2, we explored the emotional changes of netizens before and after rumor dispelling information was released and found people’s positive emotions first declined and then rose; in study 3, we also explored the emotional changes of netizens before and after the “Wuhan lockdown” event and found positive sentiment of people in non-Wuhan areas increased, while negative sentiment of people in Wuhan increased; in study 4, we studied the relationship between rumor spread and emotional polarity and found negative sentiment and rumor spread was causally interrelated. (4) Conclusion: These findings could help us to intuitively understand the impact of rumors spread on people’s emotions during the COVID-19 pandemic and help the government take measures to reduce panic. As a real-world example, when the first Ebola case was diagnosed in the United States, Twitter posts mentioning Ebola jumped from 100 per min to 6000 per min and rapidly produced inaccurate claims that Ebola could be transmitted through food, water, and air . Knapp  believesRumor has a complex social background and psychological root . From thInternet rumor refers to rumor taking Internet media as the main transmission . With thSentiment analysis refers to the automatic analysis of the relevant commentary texts content of research objects such as goods, services, and people, to find the reviewers’ positive and negative attitudes and opinions towards the research objects. Among them, sentiment classification is the most widely used, and its main task is to classify subjective texts emotionally . Some stText-based sentiment analysis is an interdisciplinary research area, involving natural language processing, databases, information retrieval, data mining, artificial intelligence, and other fields . AccordiIn order to get better results for sentiment analysis, scholars have used different classifiers to study text sentiment analysis. According to the 45th China Internet development survey, as of December 2019, the number of Weibo users accounted for 42.5% of the total Internet users . DifonzoSentiment analysis has a good predictive effect in many fields. Gruhl  predicteInitial psychological studies showed that human rumor-making and rumor-spreading behaviors are mutual mappings with inner emotions, which are driven by individual emotions such as anxiety, desire, and fear . Later sAs for the characteristics of rumor spread, Xiaozhe  divided Qi and Lv  believedAs for the social psychology behind online rumors, Zhao  classifiPublic health emergencies mainly refer to major infectious diseases, mass unexplained diseases, major food and occupational poisoning, and other events that have a serious impact on public health . The evoAs for the characteristics of rumor spread in public emergencies, Nie and Ma  believesAs for the psychological mechanisms of rumor makers, rumor mongers, and rumor refuters in public health emergencies, Du  believesSentiment is a kind of psychological activity centered on the individual’s desire and need. It is people’s attitude experience and corresponding behavior responses to objective things. Sentiment itself belongs to neutral words, but different emotional colors endow different emotional words with sentiment polarity. An important spread motivation of rumor mongers is emotional drive . When neToday, with the continuous development of social media, after an event occurs, the information related to the event will be transmitted through the network for the first time. People will also judge the event based on the received information, past experience, and values and generate personal sentiments and express them through the network. In recent years, relevant researchers have made a series of explorations on the detection and identification of Internet rumors and achieved certain results. Qazvinian et al.  tested tAs for the relationship between rumor and sentiment, Prasad  found inInternet rumors affect and even dominate the development direction of public opinions by mobilizing and stimulating group emotions, especially negative emotions, and forming emotional public opinions . In a seAs COVID-19 began to spread around the world in early 2020, rumors about the virus became rife because of its highly contagious nature and because little was known about it. Among them, some false information affected people’s normal life and caused social panic. Thus, what are the categories of rumors that emerged during the epidemic of COVID-19? What is the most popular category of rumors? What are the changes in netizens’ emotions before and after rumor refutation? What is the emotional impact of the “Wuhan Lockdown” on people in Wuhan and non-Wuhan areas? What is the relationship between rumors and emotional polarity? What measures should be taken with regards to netizen emotion change?First of all, in order to explore the types and prevalence of Weibo rumors in the context of COVID-19, we used “COVID-19 rumors” and “epidemic refuting rumors” as keywords to conduct a fuzzy query in Weibo and collected Weibo entries from January 2020 to January 2021 using the Python crawler method. Ren et al.  based onSecondly, we used the stop words of Harbin Institute of Technology as the basis to expand the stop words list and used the JIEBA word segmentation tool to segment words and remove stop words for all blog content. Then, according to the K-means clustering algorithm, which is carried out based on the Scikit-Learn library in Python, subject model analysis was carried out on the crawled texts according to the clustering results. Topic model is one of the text dimension reduction techniques, among which the most common one is term frequency-in-verse document frequency (TF-IDF). TF-IDF is a statistical measure reflecting how important a word is to a document in a collection or corpus. The higher the frequency of a word in a target document and the lower its frequency in other documents, the greater its importance . HofmannBased on the clustering results, we carried out topic model analysis on the crawled texts and got a total of 21 topics. According to the keywords given by each topic, the 21 topics are named and classified into primary and secondary categories. The topics of the first level classification include “Rational treatment of rumors”, “Rumor refutation information”, “World COVID-19”, “COVID-19’s origin”, “COVID-19 vaccination”, “Rumor information”, “Rumors cause social panic”, “Nucleic acid detection of suspected personnel”, and “Epidemic prevention and control”. Among them, “Rumor refutation information” includes “Company rumor refutation”, “Internet rumor refutation”, “Netizens hope the official refute rumors”, and “Rumor refutation all over China”; “Rumor information” includes “The public security bureau detained rumor mongers”, “Series of rumors of tea and garlic wipe out COVID-19 virus”, “Nine latest rumors”, “Market enterprise downtime”, “Internet spread rumors”, “Netizens spread false information”, “Weibo users release rumors”, “COVID-19 rumors in Wuhan”, “COVID-19 rumors lead to many deaths”, and “British 5 g base station spreads COVID-19 virus”.In We took the number of Weibo texts under each topic as the basis for judging the prevalence of rumors under that topic and ranked the topics. The results are shown in From the table below, we can see that in the context of COVID-19, netizens have a strong demand for official refuting of online rumors. In order to study whether rumor refuting information can reduce people’s negative sentiments, study 2 selected “71 cases of this epidemic have been confirmed in Qingdao” as the research object.On 12 October 2020, a rumor titled “71 cases of this epidemic have been confirmed in Qingdao” spread widely on the Internet. On the evening of 12 October 2020, the official WeChat account “Qingdao Rumor Refuting Platform” promptly released information to refute the rumor. Therefore, we selected 13 October as the time node to explore the changes of netizens’ sentiments before and after the release of rumor refuting information.Word embedding, also known as distributed word representation, can capture both the semantic and syntactic information of words from a large unlabeled corpus . A commoHow was the classification done? First, based on the continuous bag-of-words model, the original data set was read, word segmentation and preprocessing were carried out, and the data set of the list of list format was generated. Then, the training set and test set were generated in a 7:3 ratio. Third, the program initialized the Word2vec model, specified vector dimensions, and built vocabulary. Fourth, the model was built on the comment training set to generate the word vector matrix of all the terms corresponding to the whole sentence, and the corresponding vector of the whole sentence was directly averaged by each word vector. Fifth, we generated the modeling matrix and used the transformed matrix to fit the support vector machine (SVM), k-nearest neighbors (KNN), decision tree (DT), random forest (RF) and gradient boosting machine (GBM) models, respectively, to get the classification of negative, neutral, and positive emotional texts.Sentiment analysis, also known as point of view recognition and opinion mining, refers to the analysis process of identifying, extracting, classifying, inducting, and reasoning about points, sentiment polarity, subjectivity and objectivity in the text. Xu et al.  analyzedThis paper analyzed and classified the emotional tendency of the words in Weibo texts and divided individuals’ emotions towards a rumor into positive, negative, and neutral categories.Firstly, the emotional value of each sentence in the texts were calculated.m indicates the number of emotional words modified by dependency, n indicates the number of emotional words not modified by dependency, iSentimentScore indicates the final score of the i-th dependency, and jrawScore indicates the emotional value of the j-th unmodified emotional word.Then, through the weighted sum of the emotional value of the key sentence, the average value obtained is the emotional tendency value of the texts.m indicates the number of key sentences, and f (si) indicates the weight value of emotion.Equations (1) and (2) are used to label the sentences of the training and test sets so that the sentiment analysis method used can be applied and validated later.α and β were set as the threshold values of positive and negative meanings and Z was set as neutral meanings, and the values of emotional inclination were normalized between 0 and 1. The emotional tendency of the whole texts can be judged by classifying the emotional value.Finally, Time series analysis can be divided into deterministic change analysis and stochastic change analysis. The random change analysis methods mainly include the autoregressive model (AR), moving average model (MA), autoregressive average moving model (ARMA), and autoregressive integrated moving average model (ARIMA). The ARIMA model is a time series prediction model for random change analysis, including AR, I, and MA. AR represents an autoregressive model, I (integration) represents a single integral order, and MA represents a moving average model. In this work, the ARIMA is mainly used to analyze the obtained rumors and observe the changes of netizens’ sentiment before and after the release of rumor refuting information. Time series analysis can explore the causal relationship between sentiment value and rumors.Time series forecasting is an important area of forecasting in which past observations of the same variable are collected and analyzed to develop a model describing the underlying relationship. Autoregressive integrated moving average (ARIMA) is one of the popular linear models in time series forecasting in the past three decades . ARIMA mAccording to the rumor refuting information released by “Qingdao Rumor Refuting Platform” about “71 cases of this epidemic have been confirmed in Qingdao”, we took 13 October as the time node to explore the emotional changes of netizens before and after the rumor refuting information was released.According to the above results, we further explored the rumor refuting before and after the changes in the emotional value of netizens. We assign a value of 1 to positive texts, 0 to neutral texts, and −1 to negative texts to calculate the average emotional value of each day, namely:The average daily sentiment value was calculated, and its changing trend is shown in It can be concluded from Study 2 explored the sentiment changes of netizens before and after a rumor was refuted under a certain topic. In study 3, we tried to explore whether netizens’ sentiments related to rumor spreading changed before and after major events during the COVID-19 outbreak. At 10 o’clock on 23 January 2020, Wuhan announced a lockdown, which is a decision that shocked the whole country and even the whole world. For the “Wuhan Lockdown” event, we conducted a time slice and selected the texts within 24 h before and after the official announcement of the “Wuhan Lockdown” at 10 o’clock on 23 January 2020 for sentiment analysis to observe the changes of netizens’ sentiments in Weibo before and after the announcement of the lockdown.From From It can be concluded from After the overall sentiment analysis of the “Wuhan Lockdown” event, we studied the sentiment changes of people in Wuhan and non-Wuhan areas respectively. According to the crawled texts with geographic information, the texts of Wuhan area and non-Wuhan area were distinguished. Then, according to the method of calculating the average sentiment value mentioned in study 2 (see Equation (4)), a chart of the change of average sentiment value in Wuhan and non-Wuhan areas before and after the “Wuhan lockdown” was announced was drawn.As can be seen in However, in non-Wuhan areas, the average sentiment value of netizens about the epidemic was positive, which indicates that people in other regions generally held a positive attitude. When Wuhan announced the lockdown, people in non-Wuhan areas had more positive sentiments and better ability to resist rumors. In study 4, 19,495 valid texts related to “COVID-19 rumors” and “epidemic refuting rumors” were used as the research object to explore the relationship between rumor propagation and emotional polarity in the context of COVID-19. Granger  gives thBefore discussing the causal relationship between rumor propagation and the emotional polarity of text expression in the context of COVID-19, we firstly need to train the Word2vec model and use the model to judge the emotional polarity of all the texts crawled.Firstly, we randomly selected 10,000 pieces of data from the texts for artificial emotion assignment and recorded negative texts as −1, neutral texts as 0, and positive texts as 1. Then, we took 7000 texts from the Weibo sentiment test datasets as a training set for model training. Finally, the remaining 3000 texts were used as a test set for validation. Comparing the positive and negative emotion obtained by the model with the artificial emotion assignment, the results are shown in P), recall rate (R), and F1 value (F1-score) were used as evaluation indexes. For the dichotomy problem, samples can be divided into true positive (TP), false positive (FP), false negative (FN), and true negative (TN) according to the real category and classifier predicted category. Then, the confusion matrix was constructed according to the calculation formula of the evaluation criteria.In this study, accuracy rate , k-nearest neighbors (KNN), decision tree (DT), random forest (RF) and gradient boosting machine (GBM) are the more commonly used algorithms for text classification. Thus, we used the above methods to compare the results of text sentiment classification. From the perspective of classifiers, DT had best performance on SPSS 22.0 was used to explore the correlation between rumor and positive emotion, neutral emotion, and negative emotion, as shown in As can be seen from In order to further explore the causal relationship between rumor and positive, negative, and neutral sentiments, we conducted a Granger causality test.The following conclusions can be drawn from the Granger causality test results of bivariate time series in (1) There is a causal relationship between negative sentiments and rumor. Rumor will have an impact on positive sentiments, and neutral sentiments will have an impact on rumor, positive sentiments, and negative sentiments, indicating that the more negative people feel about COVID-19, the more likely rumors are generated.(2) When a large number of rumors appear, negative sentiments will be generated. As for the neutral texts (such as the rumor refuting information) appearing in the network, some people will approve it and be optimistic about the rumor trend and the development of the epidemic, while others may be dissatisfied with the neutral texts (such as complaining about the delay in releasing the rumor refuting information).In study 1, a Python crawler was used to crawl the Weibo texts from January 2020 to January 2021 with the keywords “COVID-19 rumors” and “epidemic refuting rumors”. After preprocessing the crawled texts, 21 topics and keywords under each topic were obtained via topic model analysis. By summarizing and classifying the topic of rumors, we can better explore the “popularity” of each topic and the relationship between rumors and sentiment polarity in the topic.Li et al.  found thInitial psychological studies show that human rumor-making and rumor-spreading behaviors are mutual mappings with inner emotion, which are driven by individual emotion such as anxiety, desire, and fear . From toKuang and Wu  proposedThere was a significant increase in positive sentiments after the official refuted the rumor. At the beginning of the rumor refutation, the proportion of negative texts was the highest. The reason was that after rumor refutation, some netizens would post their dissatisfaction with the rumor mongering and rumor mongers on Weibo. After netizens vented their dissatisfaction, the positive sentiments about the rumor gradually increased, and the final positive sentiments were higher than before the rumor was refuted. People thought that “fortunately, this was just a rumor and it was not true, the living environment was still safe”. This thought greatly reduced people’s panic and guaranteed people’s safety needs, so the positive sentiments about this rumor began to increase.Finally, we found that the release of rumor refuting information can effectively suppress the spread of rumors, and after the release of rumor refuting information, it can increase people’s positive sentiments and decrease people’s negative sentiments.In the early days of the COVID-19 outbreak, people’s panic and anxiety over the epidemic increased sharply, and the Chinese government announced the decision to “lockdown” Wuhan in order to prevent the further spread of the epidemic.  found thThen, we split the Weibo texts posted by Wuhan and non-Wuhan netizens and discussed the sentiment changes of people in Wuhan and non-Wuhan areas by calculating the average sentiment value. We found that regardless of whether Wuhan is locked down or not, the overall sentiment of Weibo texts in Wuhan was always in a negative state, while that in non-Wuhan areas was always in a positive state. When the lockdown was announced, people in Wuhan, which was at the center of the outbreak and restricted by policies such as home quarantine and no going-out, experienced a rapid increase in panic and unease and showed strong negative sentiments on Weibo. Dyer and Kolic  demonstrAn important spread motivation of rumor mongers is emotional drive . InterneIn addition to negative sentiments, we also need to pay attention to the effect of neutral sentiments on the changes of people’s sentiments and rumors. As can be seen from Based on the above four studies, the following conclusions can be drawn:(1) As for the types of rumors in the context of COVID-19, in study 1, we divided them into 21 topics by using the topic analysis model based on the texts obtained by the crawlers, and the most popular and widespread topic was the rumor “Netizens hope the official refute rumors”, indicating the public expectation for the official release of the true status of the epidemic. (2) In Study 2, the rumor that “71 cases of this epidemic have been confirmed in Qingdao” was taken as the research object to explore the sentiment changes of netizens before and after the release of rumor refuting information. After the rumor appeared, the official timely release of rumor refuting information reduced the rumor spreading, improved people’s positive sentiments, and increased people’s confidence in the fight against the epidemic. (3) In study 3, we studied the overall sentiment changes of netizens before and after the “Wuhan Lockdown” event and found that people’s sentiment changes before and after the lockdown were large. After taking measures for “high-risk” areas, people in “high-risk” areas experienced an increase in negative sentiments, while people in “non-high-risk” areas experienced an increase in positive sentiments. (4) In study 4, we discussed the relationship between rumor propagation and sentiment polarity. It is found that negative sentiments and rumors had mutual causality, and neutral sentiments were positively correlated with rumor, positive sentiments, and negative sentiments. The government and the media should guide the masses to view rumors rationally, so as to change the neutral sentiments to positive sentiments and reduce the generation of negative sentiments.This epidemic was an unexpected public health event. In the face of rumors generated by emergencies, the government and people can correctly view and deal with rumors in the following three ways:Firstly, the government should release true and reliable information on relevant online platforms in a timely manner, refute rumors in a timely manner, and guide people in the correct direction of public opinions. The government also should open information complaint and report platforms, establish a perfect and sound supervision and complaint system, play the role of social supervision fully, and detect and block the spread of rumors in timely manner.Secondly, the government should improve the ability to identify and block Internet rumors, purify the network environment, and reduce the generation and spread of rumors.Finally, the public need to improve their ability to identify rumors, “not believe rumors”, and “not spread rumors” and consciously cooperate with the government and other institutions to create a good public opinions environment for the society.