Eye tracking is commonly used in marketing to understand complex responses to materials, but has not been used to understand how low-literacy adults access health information or its relationship to decision making.n = 21) or literacy-appropriate aid (n = 29) shown on a computer screen. Using 7 content similar slides, gaze patterns, mean pupil fixation time and mean overall time reading and looking at slides were compared. Groups were also compared by literacy level and effect on 'confidence of knowledge' and intended behavior.This study assessed how participants use a literacy appropriate “dirty bomb” decision aid. Participants were randomized to receive a CDC “factsheet”  contains supplementary material, which is available to authorized users. Eye tracking technology assesses how individuals view visual content by monitoring and recording eye movements as a subject processes the stimulus in real time. All systems use cameras, with some mounted to a flat stable surface and others embedded in a computer screen or mounted on lightweight glasses. Eye tracking is commonly used in marketing research , traininWhile eye tracking has been used in a number of health-related studies –19 few hth of 24 developed nations in literacy ability [National Center for Education Statistics (NCES) found that having only basic health literacy was greater in urban areas (16.3 %) compared to non-urban areas (13.1 %), with minorities and those over age 65 being most at risk of having literacy issues [The importance of examining low-literacy populations’ perceptions of and responses to Radiological Terror Events (RTEs) such as a “dirty bomb” is evident. The 2003 National Assessment of Adult Literacy, the most recent national-level assessment of literacy, revealed that 14 % of adults in the United States have below basic literacy, 22 % have basic literacy, 53 % have intermediate literacy, while only 12 % are proficient . Overally issues . This prThe needs of low-literacy populations in the event of a terror or bioterror event are not well understood. This is especially critical because as we move farther away from the September 11, 2001 attacks, a decrease in resources directed toward emergency preparedness has occurred. RTEs, such as the explosion of a “dirty bomb”, present a threat that requires national, state, and local preparedness , but litAn added area of concern is the public's trust of government authorities in times of crisis. While “sheltering in place” in an intact structure is the recommended way to stay safe in the event of a “dirty bomb” explosion, research shows that barriers to complying with government directives include distrust and lack of confidence in government or public health authorities, fear of subsequent attacks, and beliefs that needed supplies or help will not be made available –31. ThesThe present study sought to address this gap by using eye tracking to assess how a systematically-developed, literacy appropriate decision aid, designed to foster understanding about how to respond to a “dirty bomb” event, addressed the needs of adults with limited literacy compared to a higher-literacy sample. The content of the decision aid was based on “Frequently Asked Questions” information developed by the Centers for Disease Control and Prevention (CDC), one of the few materials on RTEs publically available. Results showed clear patterns of differential text use in a small randomized pilot and demonstrated that eye tracking is a viable and useful tool in the quest to design materials for low-literacy groups to enhance decision-making.This study included a multi-phase protocol. Phase I included qualitative focus groups and quantitative surveys. Results of this phase were used in Phase II to develop a low-literacy risk communication decision aid about “dirty bomb” RTEs. Phase III involved testing the developed decision aid in a pilot randomized controlled trial. This paper discusses the results of Phase III, and focuses on the eye-tracking results.Phase I included both qualitative and quantitative data collection. This involved three focus groups with thirty-seven urban 18 to 65-year-olds, representing both men and women recruited from areas with low income and education levels in Philadelphia, PA [phia, PA . From exphia, PA . These qphia, PA , 40.Focus group results were then used to develop a comprehensive survey administered to 50 low-literacy adults, which assessed perception and understanding of a “dirty bomb,” trust of information sources, and intention to comply with government directives about “sheltering in place.” Implemented as a cross-sectional intercept survey, recruitment of respondents occurred in a variety of community-based sites in North Philadelphia, including local hospitals, neighborhood youth and senior community centers, and supermarkets. Four groupings of conceptually-related statements were included in the survey, including: 1. Statements on knowledge of a dirty bomb, trust of information sources if a dirty bomb occurred, and need for information about who was responsible for the event; 2. Statements on action intentions , belief in general preparedness, and preparedness activities such as having an emergency plan, food, and water; 3. Statements on what respondents would be worried about in the event of a dirty bomb, such as safety of food and water, risk of illness, breathing radiation, and its effects and risks to children and pets; and, 4. Statements that assessed participants’ perspectives of how likely a dirty bomb is to occur; likelihood of this event occurring compared to other threats ; beliefs about the trustworthiness of local, state, and federal authorities; and, whether their neighborhood would be treated fairly in response/recovery efforts.Phase II consisted of analyzing survey results using a variety of methods, including perceptual mapping and message vector modeling, and a cluster analysis to identify potential differences within the sample group that might need to be addressed in the design of the resulting decision aid [http://sites.temple.edu/turiskcommlab/).sion aid . Perceptsion aid –45. MessBased on these results, we developed a literacy-appropriate decision aid, optimizing the fit between the subjects’ perceptual maps and the content and format of literacy-appropriate information about a “dirty bomb”. The resulting decision aid included overall information about what a “dirty bomb” is and how to respond should an explosion occur, as well as specific message concepts determined to be important through the perceptual mapping. These concepts included: (1) Emphasizing trust in the information and recommended responses of authorities/experts, (2) Complying with recommended actions, and (3) Getting prepared for an emergency before one occurs. We also emphasized that in such a crisis, it is less important to dwell on who was responsible for the attack (a common need felt by victims of attacks that can impede taking immediate action) than it is to determine how to immediately protect yourself and your family. As well, the decision aid presented straight-forward information about what a dirty bomb is and how to respond if one occurred.http://emergency.cdc.gov/radiation/dirtybombs.asp) and presented in a question/answer format. The reading level of this material, assessed using the Flesch-Kincaid test [th grade reading level and utilized graphics to represent words. The CDC FAQs decision aid was 19 slides, compared to the intervention (low-literacy) decision aid, which was 35 slides. Many slides were content similar, but the intervention decision aid was longer to account for special message content derived from the perceptual mapping but not addressed in the CDC's FAQs. Figure The control material was the CDC’s printed document, “Frequently Asked Questions” for a “dirty bomb”, which is available to the public on the CDC website  and the existing CDC “dirty bomb” FAQs (control) was conducted with a convenience sample of 50 adults  in the Risk Communication Laboratory at Temple University in Philadelphia. The testing occurred over six months, between January and June, 2012. We conducted this pilot study to assess the feasibility of conducting a fully powered randomized trial on whether a systematically developed, literacy appropriate decision aid would help urban residents understand the risks of a “dirty bomb” and how to respond if one occurred, as well as to determine if eye tracking was a useful and valid method of determining how those with low-literacy can or cannot access text.During Participants were recruited for the pilot through community-based methods, including placing flyers in the Temple University Hospital’s General Internal Medicine Clinic, community organizations surrounding Temple University, as well as local pharmacies and supermarkets. Flyers included a “tear tab” with a phone number so those interested could call about participating in the study. We also focused on users of a neighborhood food bank, a senior services organization, and a federal services office. In these cases, research assistants visited the sites, talked with people about the study and screened them for eligibility. The goal was to have participants who closely represented people who matched the socio-demographic groups represented in the focus group and survey analysis.Screening included demographic information  and health literacy level. In-person literacy screening utilized the Rapid Estimate of Adult Literacy in Medicine – Revised (REALM-R) test, which is a word recognition test in which participants are asked to pronounce eleven medical words of increasing difficulty . The firWhen participants arrived at the Risk Communication Laboratory a trained research assistant orally administered all consent forms and instruments, which were tailored for people with low literacy. The eye tracking data collection procedures were explained to participants using pictures of the equipment and samples. Participants were then escorted to the testing lab, which is a small private room, and they were connected to the eye tracking system in preparation for viewing their assigned decision aid. The input from the physiological monitors was fed directly into a computer database for storage and analysis. Eye tracking was performed using an Applied Science Laboratory stationary eye tracker (Eye Trac 6000). This equipment requires a participant to sit in front of a screen with a Velcro band around his/her head, which automatically corrects for any head movement during eye tracking data collection. The control and intervention messages were presented on a computer monitor for each participant to view. Once the system acquired a pupil lock and calibrated the subject’s eye tracking, he/she was told to look at the screen and read the slide presented, and then indicate to the technician when he/she was done. The technician then advanced the slide for the participant until he/she was done looking at all slides. Once the participant was finished, he/she was directed back to another room where the post-education survey protocol was conducted. The full study took approximately 45 min to administer. Results of eye tracking analyses are discussed here.The two groups (Intervention v. Control) are profiled in Table =“Very chaotic tracking with very limited ability to follow text”,= "Mostly chaotic tracking and quite limited ability to follow text,"= “Mixed tracking, where tracking of some areas of the text/slide were precise, and the rest chaotic and inconsistent,"= "Mostly precise tracking and relatively consistent following of text,"= "Very precise tracking and highly consistent following of text."To extract pupil fixation and average time to complete a slide, as well as extracting the gaze patterns, the Eyenal software program  was used (Applied Science Labs). In addition, a 5-point subjective rating scale was developed and tested by the authors to rate the consistency of a participant's gaze patterns, including the ease with which the participant appeared to follow the text. To establish inter-coder reliability in the use of this scale, three study coders first came to agreement about what each scale point meant. The five-point tracking consistency scale was defined as follows:Coder training involved practicing use of the scale to rate eye-tracking consistency until a high level of inter-coder reliability was achieved . After training, coders independently rated each participant's eye-tracking patterns on each slide. In the few cases where rater discrepancies occurred, the coders reviewed the gaze pattern in question together, and discussed their ratings to arrive at a consensus scale value.What is a dirty bomb?Dirty bomb is not the same as an atomic bomb.What to do if you are inside and close to the explosion?What to do if you are outside and close to the explosion?What should I do about my kids and family?Will my food and water be safe?How do I know if I’ve been exposed to radiation?Eye-tracking comparisons were only performed on seven content-similar slides to control for content. These seven slides included similar questions and responses and allowed us to compare eye tracking patterns across similar content. The slides included:As illustrated in Table p = .032), “what to do if you are outside and close to the explosion”  = 2.48, p = .02), “what to do if you are inside and close to the explosion”  = 3.61, p = .001), and “what should I do about my children and family”  = 2.70, p = .012) slides. Intervention participants had higher means on all but one of the slides (“What is a dirty bomb”), which in both groups was very simple and contained only one statement describing what a dirty bomb is.Table Figure p = .002), “What to do if you’re inside”  = 2.03, p = .048), “How do I know about radiation exposure”  = 2.46, p = .02). A fourth slide, “What should I do about my children and family” approaches significance, with the control group having more mean fixations  = 1.86, p = .069). This is indicative of the more densely worded material in the control slides and their low-literacy level. In some cases, control participants had twice the number of pupil fixations compared to the intervention participants, most likely because of the density of the text. This correlates with the observations of real time tracking, where control participants can be seen repeating lines of text or skipping back and forth on the slide.Mean number of pupil fixations on the seven selected slides indicated that the control group spent significantly longer looking at specific words or phrases on three of the slides Table . These ip = .015). Similar patterns are seen in the other slides: “An dirty bomb is not an atomic bomb”  = 5.79, p = .000), “What to do if you’re outside”  = 2.46, p = .018), “What to do if you’re inside”  = 2.91, p = .006), “What should I do about my children and family”  = 3.51, p = .001), “What to do about food and water”  = 5.02, p = .000), and “How do I know about radiation exposure”  = 2.84, p = .007). This correlates with both the eye tracking ratings and the pupil fixation analysis and strengthens the argument that the control participants had a difficult time reading and processing the text-heavy control slides.Table p = .024; What do with family, .409, p = .031; What to do about food/water, .473, p = .006; Radiation Exposure, .406, p = .024). In addition, comparing average time, fixation and tracking ratings by intervention group, the Intervention, Very Low Literacy group had lower time and fixation composite scores  and higher tracking scores (3.32 vs. 2.79). These differences, however, were not significant.In addition to looking at individual slides, composite scores for overall ability to track slides, fixations and time were calculated and then correlated to the dichotomized literacy variable. Pearson Correlations indicates that literacy level was associated with four of the 7 slides on the eye tracking rating ; Know what to do if a bomb blows up  = 2.49, p = .02); My ability to follow instructions if a “dirty bomb” happens  = 2.87, p = .008). These differences were not seen in the Low Literacy by intervention groups.Finally, Independent Samples T-Tests were run comparing effects of the intervention on those with the lowest literacy levels on outcome measures, specifically confidence in knowledge of what a “dirty bomb” is, and intended behavior (i.e. sheltering in place) if one occurred . Table Monitoring of eye tracking in real time and watching video of each participant’s session clearly revealed that in this pilot study the control group participants had a difficult time getting through the dense text. For many of the low-literacy participants, it was obvious that they either chose to randomly glance around the slide text and not systematically work through it, or they began to work through the text, often re-reading given lines several times before giving up and resorting to random looking patterns. In some instances, it seemed that control participants would look off the slide for a period of time before deciding enough time had passed to tell the technician to advance to the next slide. Observing these patterns could be difficult to watch, as the apparent frustration many participants had with the text was evident. These observations were supported by the data, which indicated significant differences in not only the gaze patterns tracked across time, but was also in the measures of time spent at each fixation point, and the total number of fixation points per slide. These differences also remained once tracking fixations and time were combined as a composite score and compared, and when the sample was compared by their literacy level and intervention group. It was clear that the dense text provided in the control condition caused participants to spend more time with trying to get through the text and attempting to read lines more than once. Despite the small sample size of this study, the differences we found in tracking and outcome measures (confidence in knowledge and ability to follow instructions) indicate a large effect size (cohen’s d = .94). However, future studies using eye tracking measures will have to take into account the intent of the trial (eye tracking as outcomes vs. changes in knowledge and behavior) and whether larger sample sizes are required to test the effect of these types of materials are larger audiences.th grade reading level or higher [Clearly, having access to literacy appropriate materials is important. In this study, significant effects on confidence in knowing what a dirty bomb was, how to respond if one occurred and ability to carry out instructions was seen in those with the very lowest literacy. This is a significant finding, indicating that even those with very low literacy can benefit from having information presented in a manner that addresses their needs. Because of the serious consequences of exposure to a “dirty bomb’, it is important that disaster planning and management teams address the accessibility of risk communication materials designed for limited-literacy groups, so that everyone can have equal access to potentially lifesaving information and make informed decisions for themselves and their families. Health literacy studies suggest that a third-to-a-half of the US population has some difficulty using written materials , such asr higher , 51. If Public trust and confidence can help mitigate and improve the ability to manage a terror threat , but thiOn a conceptual level, the goal of effectively communicating a health/risk message to a particular target audience must take into account not only their literacy skills, but also their level of education and experience with the topic being addressed. As such, the design of effective messages strategies requires attention to the task of informing and educating the receiver of the message, and the task of creating an information processing experience that effectively and efficiently optimizes the formatting, graphics, and visual/auditory aspects of message structure. These latter structural elements are critical to optimizing overall processing, learning, motivation, decision making, and behavior. Eye tracking analyses are particularly well suited to understand both the content/educational aspects of message design and the structural information processing aspects, and to understand how difficult it may be to access material that is not developed for those with low literacy. In this pilot study it was clear that low-literacy participants were more comfortable with the decision aid written at their literacy level and were more receptive to its messages; this is especially true in those with the lowest literacy. Eye tracking was thus a viable and useful method for understanding how low literacy adults may respond to and understand emergency communications, informing the development of better, more accessible materials.The limitations of this research include the fact that as a pilot study the sample size is, by design, small. Although this is the case, care was taken to accurately assess the participant's literacy level and to randomly assign participants to the control and intervention test-groups. Future studies should expand the sample size and also expand the geographic scope of the study to explore urban v. suburban differences as well as potential regional differences across the country.There are also other types of eye tracking data that could have been used in the analysis of this study, including saccades and smooth pursuits. These could be useful in larger studies to assess ease of tracking and reading material, especially when video or other interactive activities are incorporated into learning materials. These were not used here. In addition, the subjective eye tracking rating, despite rigorous training of raters and good reliability, is still a source of bias. Perhaps in a larger study this could be compared through a more objective mathematical evaluation of areas of interest and/or machine learning techniques. These measures were not available for this study. In addition, screening participants for visual acuity and vision quality would be advised.Another limitation involves the current focus on the topic of radiological terror events (Dirty Bomb). Individual responses, in terms of information processing, physiological reactions, and knowledge gain will likely vary with different types of content . Public Health officials will have to develop "preparedness" materials for these types of differences as the frequency of such events continues to increase. This may also affect eye tracking results, as information more familiar to an audience may hinder the ability to differentiate as easily between tested materials.A third important limitation of the present study is that it focuses on a particular aspect (literacy) of the audience's ability to access, interpret, and apply critical information about what to do in a particular emergency situation. The eye tracking and message design methodology used in this research can be directly applied to areas other than audience literacy levels. For example, it can be used to design more effective materials for individuals with learning disabilities, or those with limited media-literacy skills, or those needing to access the diverse array of assistive technologies now available.The use of eye tracking is an effective approach to understand how health communication materials are accessed and processed by individuals with limited-literacy skills. We believe that this pilot work clearly demonstrates the feasibility of using these methods on a wider scale to develop more effective health and risk communication messages designed to increase knowledge of and compliance with general health guidelines on the one hand and specific health-based emergency directives on the other. It is also clear that the message design and testing approach has application not only for those with low-literacy, but for those with learning disabilities, those with limited media-literacy skills, and those needing to access the diverse array of assistive technologies now available. The long-term goal is to evolve and refine powerful message design tools that will contribute to effective decision making and the elimination of health disparities evident in educational, social, and personal contexts.CDC, Centers for Disease Control and Prevention; REALM-R, Rapid Estimate of Adult Literacy in Medicine-Revised; RTE, Radiological Terror Event; SILS, Single Item Literacy Screener


The chief ethical rule is the following: thou shalt not have antifragility at the expense of the fragility of others. Taleb, Europe is nicely complex; that is, rich and full of diversity. Lessons learned from the painful past are immense .Two decades after its foundation, the premier organization of European psychotraumatologists, the European Society for Traumatic Stress Studies (ESTSS) is completing its protracted structural transformation and takes an important step to fulfill its key vision, namely, solidify international collaboration among trauma professionals throughout the whole continent. Synchronization of knowledge and experience among all participants is necessary to achieve these ends, and may itself be a creative process. At the end of this transition from a society built around voluntary individual membership to one of an umbrella organization for regional  trauma societies, ESTSS will benefit from tremendous organizational efficiencies as a transnational venue to nurture the common ground in European psychotraumatology. To advance this, ESTSS spreads its knowledge through biannual conferences and the The most robust change in the world of contemporary science seems to be in availability of knowledge. “Open access” journals are increasingly becoming the standard of scientific publishing. Once a domain of youngsters, experiences collected via development of “social” media pave the way to user-friendly academic portals which resemble the former in their designs. The world has turned into a “marketplace” for scientific products; a fantastic one in which goods are not paid for by customers but by producers. In this “intellectual commons,” and not withstanding continued debates and restrictions around intellectual property, knowledge is shared freely. What is the economy behind this inverted relationship? Apparently, knowledge has the tendency to grow if shared. One hopes that this endeavor diminishes the distance between knowledge and truth, and regulates the “market” in such a way that the most valuable scientific output distinguishes itself out of an almost limitless supply.EJPT is a fine example of this philosophy: it is open access , several grotesque changes become apparent in shaping the modern “Zeitgeist” of psychotraumatology: The first is that, 1980 marks the start of the “globalism” which increasingly challenged national “boundaries,” at economic and cultural levels. Free market values and consumerism began to prevail against economic protectionism, and an influx of North American culture started to shatter familiar cultural values. It was also during the 1980s that the European Union welcomed Greece, Spain, and Portugal as member states, which constituted a first step of departure from its predominantly northwestern European identity. The end of “Cold War” led to the reunion of the greater Europe, but not without complications. Increased accessibility of the Internet to a much wider user base has provided tremendous opportunities enabling knowledge to seamlessly cross borders. Last but not least, 1980 was the year of publication of the North America-based highly influential document of psychiatry, namely, the DSM-III : “Dr. Charcot, what you say does not fit the theory.” The response (one of Sigmund Freud's favorite quotes) was: “Theory is good but it doesn't prevent things from existing” Newman, , p. 125.The rule of thumb is that clinicians should learn how to establish mutuality between theory and practice in each particular situation, and how to make their clients and their feedback an active element of this process Kluft, . “Where A recent roadmap for mental health research in Europe created by a group of 60 invited experts identified 20 priorities , Istanbul, Turkey Email: vedats@amerikanhastanesi.org European Society for Traumatic Stress Studies (ESTSS), 2013–2015 Diemen, The NetherlandsPresident International Society for the Study of Trauma and Dissociation (ISSTD), 2008–2009 McLean, Washington DC, USAFormer President


In recent decades, we have observed an intensification of science, technology and innovation activities in Brazil. The increase in production of scientific papers indexed in international databases, however, has not been accompanied by an equivalent increase in the impact of publications. This paper presents a methodology for analyzing production and the impact of certain research areas in Brazil related to two aspects: the origin of the journals  and international collaboration. These two variables were selected for being of particular importance in understanding the context of scientific production and communication in countries with emerging economies. The sample consisted of papers written by Brazilian researchers in 19 subfields of knowledge published from 2002 to 2011, totaling 85,082 papers. To calculate the impact, we adopted a normalized indicator called the relative subfield citedness (Rw) using a window of 5 years to obtain measurements evaluated in 2 different years: 2007 and 2012. The data on papers and citations were collected from the Web of Science database. From the results, we note that most of the subfields have presented, from one quinquennium to another, improved performance in the world production rankings. Regarding publication in national and foreign journals, we observed a trend in the distribution maintenance of production of the subfields based on the origin of the journal. Specifically, for impact, we identified a lower Rw pattern for Brazilian papers when they were published in national journals in all subfields. When Brazilian products are published in foreign journals, we observed a higher impact for those papers, even surpassing the average global impact in some subfields. For international collaboration, we analyzed the percentage of participation of foreign researchers and the connection between collaboration and the impact of papers, especially emphasizing the distinction of hyperauthorship papers in terms of production and impact. In recent decades, we have observed in countries with emerging economies an expansion of production activities at different levels, including the sectors of science, technology and innovation (ST & I) . In BrazHowever, in contrast to the improvement in Brazil’s scientific performance measured according to these parameters, it was possible to verify the low impact of the publications considering the number of citations by the scientific community. According to the Ranking of Countries produced by SCImago , Brazil Among the possible hypothesis that have been developed to explain this phenomenon are: the difficulties of scientific excellence in a emerging and expanding system, the low international collaboration index of Brazilian researchers, the more regional nature of the research problems in certain knowledge areas that are mainly published in Brazilian journals. Regarding this last point, it is estimated that at least half of the papers published by Brazilian scientists are in journals that are published in Brazil .It is worth mentioning that, likewise the growth of production, these issues about the low impact of the publications and the use of journals published in one’s own country are common advents in almost all emerging countries. As a consequence, the publication of journals in these countries plays a distinct role than in the leading countries in science, in which we can see, in addition, a commercial logic . In oppoIn recent years, the international databases began to contemplate a larger number of scientific journals published in less developed countries. Before this, the coverage of the work produced by these countries’ scientists was restricted almost exclusively to what was published in the developed countries. Thus, the indexation integrated the data on papers published in national journals with the data on publications considered to be in the mainstream of science, enabling the expansion of knowledge about the characteristics of the scientific production in emerging countries. According to Meneghini, “the establishment of national journals has created two parallel communication streams for scientists in emerging countries: publication in international journals—the selective route—and publication in national journals—the regional route” .In this context, considering that the improvement in the international ranking of scientific production by Brazilian researchers was accompanied by an increase in the number of Brazilian scientific journals, the aim of this study was to ascertain the scientific production and the impact of some research areas in Brazil, considering the following:Publication in national and foreign journals ;International collaboration (based on papers with Brazilian and foreign authorships).Thus, with this objective, we expect to contribute to better understand the role of the national journals in Brazil’s position in the ranking in the international scenario over the last years. This also could contribute for a diffusion of a methodology potentially useful to other emerging countries to also understand the role of their national journals on their scientific development.The subfields included in this study were those with at least 1 Brazilian journal indexed in the 2002 edition of the Journal Citation Report (JCR). With the application of this criterion, 19 subfields (Web of Science Subject Categories) were identified for the study, as follows: “Agriculture, Dairy & Animal Science”, “Agriculture, Multidisciplinary”, “Biochemistry & Molecular Biology”, “Biology”, “Chemistry, Multidisciplinary”, “Engineering, Chemical”, “Genetics & Heredity”, “Mathematics”, “Medicine, Research & Experimental”, “Microbiology”; “Neurosciences”, “Parasitology”, “Physics, Multidisciplinary”, “Psychiatry”, “Public Environmental & Occupational Health”, “Social Sciences, Interdisciplinary”, “Soil Science”, “Tropical Medicine” and “Veterinary Sciences”. Papers on the “Multidisciplinary Sciences” subfield met the adopted criterion but were not included in the sample because they did not identify and clearly define a thematic specialty. So, considering that the universe of analysis was restricted to papers published in journals of the 19 WoS Subject Categories, we constituted two samples as follows:Sample 1 . This saSample 2. This sample included 85,082 papers  published between 2002 and 2011 and indexed by WoS. These papers were published by authors with Brazilian affiliation. In this sample, the production, impact and international collaboration of Brazilian scientific papers published in national and foreign journals were characterized.The Sample 1 data were collected from WoS on 12-23-2015 and the Sample 2 data were collected from WoS on 01-13-2014. We described in the The comparative analysis of production and impact indicators of scientific activity has at least two challenges: a) different research areas have different production and citation patterns; and b) bibliographic databases have coverage variations over time, which can lead to a mistaken interpretation of changes in scientific performance, i.e., increasing production and impact indicators may be exclusively attributable to the increase in the number of journals indexed by the base.For decades, this known issue has motivated scientometricians to search for formulas that produce relative impact indicators  that allow for absolute performance comparisons of communities of different areas at different periods of time without reservations. Vinkler  revises Regardless of the approach, the starting point of all formulas is the average of citations by papers published by the researchers who constitute the focus of analysis. The controversy regards how to obtain the parameter that will serve for comparison and normalization and how to aggregate the publications by fields –13. Amoni is the number of citations to the i-th paper evaluated, GFm is the mean Garfield factor of journals in the respective subfields according to a Journal Citation Report (JCR) specific edition.According to Vinkler , “the RwTo avoid misleading conceptions: the Impact Factor of the journals (GF) indicate the impact of the journals; the Number of Citations (c) received by the Papers (p) indicates the absolute impact of the papers. Based on these indexes, the Number of Citations (c) received by the Papers (p) normalized by Mean Impact Factor of Journals of the respective subfield (GFm) indicates relative impact of the papers (Rw). This index (Rw) cannot be confused with the use of GF as an indicator of the impact of the papers, an evaluation practice strongly questioned .As a result, the Rw has the number 1 as a reference value for interpreting the impact of the different subfields. If Rw = 1, it can be concluded that the citedness of the papers in question is approximately the same as that of the subfield; If Rw< 1, the papers in question receive less appreciation from the scientific community than the average paper in that subfield; and if Rw> 1, the international scientific impact of the investigated publications is greater than that for the average paper in the subfield .The advantage of the normalization method applied with Rw in comparison to other relative indicators derives from the fact that the factor GFm in the index Rw is calculated by averaging the GF of all journals classified in a specific WoS Subject Category. Other indicators only use the GF of the journals used for publication by the authors being assessed, limiting the comparison into a context defined by the authors themselves . Thus, aHowever, two limitations should be highlighted in terms of choosing the Rw indicator as a tool for measuring the scientific paper’s impact. The first limitation refers to how to define the reference standard of the papers for standardizing the citations: we recognize that the field normalization based on WoS Subject Categories is criticized for being considered a very general classification. An extreme example of this problem is the papers published in journals of high visibility and generally classified as "Multidisciplinary Sciences", independently of the specificity of the paper. In order to attenuate this limitation, we removed from the sample papers published in journals of the subfield "Multidisciplinary Sciences", thus restricting the analysis to papers published in journals of specific subfields, which are more accurately contextualized by themes .The second limitation refers to the fact that Rw is an indicator based on the mean of citations and not on the characteristics of the distribution of citations among publications. More specifically, the impact values measured by means may present distortions due to some highly cited scientific papers. However, a measurement that considers the characteristics of the distribution of citations create some challenges for an adequate definition of reference standard used for standardizing the impact data . These cTo measure the impact, we used a 5-year citation window, adopting the same formula for the 2007 and 2012 editions of the JCR , that isResults for the production and impact of Brazilian papers are presented considering the aspects as previously mentioned: publication in national and foreign journals and international collaboration. Specifically, we emphasize that here we use scatter plots to facilitate descriptive analysis separate from their original purpose of representing correlations between variables.Despite the international nature of science, Even in the US, the world leader in scientific production, we observed a relative low level of endogamy, well below the approximately 80% of endogamy observed in journals published by the emerging countries. The peculiarity of journals published by emerging countries suggests that the scientific production of these countries might have different characteristics depending on the place of publication. The following data on production, impact and international collaboration of the scientific production takes into account the possible need for this distinction. Thus, further analyzes aim the characterization of the Brazilian production when published in its own journals , and when published in journals edited by other countries .In In the first period (2002–2006), Brazil already had outstanding performance in the world ranking in 5 of 19 subfields: “Agriculture, Dairy & Animal Science”, “Agriculture, Multidisciplinary”, “Parasitology”, “Tropical Medicine” and “Veterinary Sciences”. In addition to these subfields, in the second period (Rk BR 2007–2011), Brazil also began to appear among the top five positions in the rankings on “Public Environmental & Occupational Health” and “Soil Science”. Although Brazil did not occupy the top of the ranking, “Social Sciences, Interdisciplinary” is another subfield with improved performance, moving 9 positions in the ranking from the first to the second period.In addition to the position in the world ranking of scientific production, some of these subfields also showed growth rate clearly above the growth of global scientific production. This occurred in “Agriculture, Multidisciplinary”, “Parasitology”, “Public Environmental & Occupational Health”, “Social Sciences, Interdisciplinary” and “Tropical Medicine”.The subfields in which Brazil presented a decrease in performance (Rk BR) during the period were: “Chemistry, Multidisciplinary”, “Mathematics”, “Medicine, Research & Experimental” and “Physics, Multidisciplinary”.Regarding publication in national and foreign journals, we observed continued production distribution trends in the following subfields by type of journal (Brazilian or foreign). “Biochemistry & Molecular Biology”, “Engineering, Chemical”, “Mathematics”, “Microbiology” and “Physics, Multidisciplinary” are the subfields that proportionally publish most often in foreign journals, and, also proportionally, “Agriculture, Multidisciplinary” and “Biology” differ by predominantly publishing in national journals. More specifically, regarding production in national journals, another aspect that we highlight is the publication of papers in languages other than English. “Agriculture, Dairy & Animal Science”, “Agriculture, Multidisciplinary”, “Chemistry, Multidisciplinary”, “Public Environmental & Occupational Health”, “Social Sciences, Interdisciplinary” and “Veterinary Sciences” publish a significant number of papers in other languages, especially in Portuguese, reducing the international audience for the publications.The only subfields that changed their profile publication by type of journal were “Medicine, Research & Experimental” and “Public Environmental & Occupational Health”. The first moved from predominantly publishing in national journals to foreign journals, and the other followed the opposite path.We observed that Brazilian scientific publication in national journals had less than average impact worldwide in all subfields in the 2 periods (Rw< 1). In a few subfields, production was greater than half the worldwide average (0.5 <R w< 1); this occurs in only two subfields in both periods: “Engineering, Chemical” and “Tropical Medicine” . “Agriculture, Multidisciplinary”, “Chemistry, Multidisciplinary”, “Mathematics”, “Parasitology”, “Public Environmental & Occupational Health” and “Soil Science” presented Rw measures within this standard in only one of the two years studied. Although there is a trend toward identifying the use of languages other than English as the cause of the low impact of national journals, our data show that this factor is not sufficient to explain the full problem. Even the national journals classified in subfields where the publications are mainly in English receive a low number of citations.As expected, when Brazilian production is published in foreign journals, we observe a significant increase in the impact of papers in all subfields, many even surpassing the average worldwide impact (Rw> 1). Contrary to common sense, this superior impact to the worldwide average was identified mainly in subfields that publish predominantly in national journals; that is, when studies in these subfields are published in foreign journals, they have greater impact than the worldwide average. This occurred in both periods in the following subfields: “Agriculture, Dairy & Animal Science”, “Agriculture, Multidisciplinary”, “Soil Science”, “Tropical Medicine” and “Veterinary Sciences”. “Engineering, Chemical” and “Physics, Multidisciplinary” are the only two subfields that predominantly publish in foreign journals and have a production impact higher than the worldwide average when they publish in foreign journals. Three subfields presented Rw> 1 in only one of the two years: “Chemistry, Multidisciplinary” and “Public Environmental & Occupational Health” in 2007 and “Psychiatry” in 2012.The comparison of both periods reveals a small oscillation in the impact of Brazilian scientific production in general. The most significant variations occurred in work published in foreign journals, especially in “Agriculture, Multidisciplinary” and “Social Sciences, Interdisciplinary”. Even with the decline in 2012, production in “Agriculture, Multidisciplinary” had higher than average impact in foreign journals in both periods studied.We present below the production and impact results correlated with international scientific collaboration . The datWe observed wide variation between the subfields with regard to the two aspects presented in “Genetics & Heredity”, “Mathematics”, “Microbiology” and “Physics, Multidisciplinary” have the most international collaboration. “Agriculture, Dairy & Animal Science”, “Agriculture, Multidisciplinary” and “Veterinary Sciences” have the least international collaboration.With regard to Rw values, we observed that the impact of nearly all subfields production is higher when the papers published in foreign journals are written with international collaboration . However, in some subfields, the difference in impact is quite small, as in “Agriculture, Multidisciplinary”, “Engineering, Chemical” and “Veterinary Sciences” in both periods.At the other extreme are the subfields whose production impact is much greater when there is the participation of international researchers. This is the case of the subfields “Biology”, “Genetics & Heredity”, “Medicine, Research & Experimental”, “Physics, Multidisciplinary” and “Psychiatry”.Closer analysis suggests the existence of collaborations at differing levels, which may explain the degree of influence of collaborations on the impact of production in specific subfields . In geneVery often, the hyperauthorship papers highlight a group name as the author. Over time, the bibliographic database began to specifically index this information to identify intellectual responsibility for eminently collaborative research . In our In a more specific analysis, we calculated the Rw values with citations that appeared in 2012 against the author-group papers in these 11 subfields. In general, we verified only a small variation in the impact. However, a completely different configuration arises when we apply the same procedure to “Physics, Multidisciplinary”. In the Brazilian production in this subfield, we observed that author-group papers have a much greater impact than other papers with collaboration .Although they represent approximately 21% of all papers published in foreign journals with international collaboration, the author-group papers account for nearly half of the citations received by that type of work in the “Physics, Multidisciplinary” sub-field. As shown in The present study analyzed Brazil’s scientific production and its impact, taking into consideration certain issues that are particularly important to the least developed countries. Next, we discuss the most significant results, especially with regard to the role of national journals and to the methodological aspects of evaluating scientific publications by development agencies in Brazil.Concerning the first aspect, we note that the role played by national journals in emerging countries is different from that in developed countries. Without commercial appeal, the so-called "emerging journals"  play a kThe importance of national journals as a communication channel for research results is observed by analyzing scientific production world rankings. Brazil is in top positions in 7 of the 19 subfields, among them, 6 subfields that predominantly publish in national journals.In contrast, when analyzing Brazil’s position compared with countries that play a leading role in the scientific context, we realize that in addition to the measures given by the rankings, the country contributes with a negligible share of researchers who belong to the editorial boards of prestigious international journals ,26. In aIn Brazil, the gap in this cycle came to exist because the full development of a country’s science research is attained by training researchers to be able to realize and implement more comprehensive overviews of research fields in the editorial aspects of scientific publications. Today, however, this function is performed by Brazilian researchers in a restricted way nearly exclusively through the work of the editorial staffs of national journals . NeverthIn essence, the limited visibility is because submissions to national journals are restricted to works written by Brazilian researchers themselves. The situation is aggravated because the contributions by Brazilians are generally submitted to national journals if they have not been accepted in other international journals or their authors predict that they will not be . The resThe other interesting question raised by the approach that we developed concerns the methodological procedures for measuring the impact of scientific production. The use of scientometric indicators by Brazilian funding agencies has become popular as a tool for evaluating scientific production and impact in areas in which production is predominantly reported in journals indexed by international databases. The adoption of this type of resource is a global trend because of the expansion of the S, T & I sector as a whole. Considering the difficulties faced so far in collecting citation data for specific works, the Garfield (impact) Factor (GF), which is annually published in the JCR, was for many years widely used to indicate the relevance of a contribution by the impact of the journal that published it. This impact assessment method has always received much criticism  and has However, regardless of the object—whether journal or paper—the need to normalize the impact measured by citation patterns in specific (sub)fields was the lesson learned from many years of using these indicators. Unfortunately, in Brazil, some sectors have not yet transposed this learning to evaluation practice. The Brazilian scientific community feels the effects and spreads complaints about the damage suffered by areas of lower impact at the time of the evaluations –30.However, the normalized indicators we used showed that Brazilian papers that were published in foreign journals in 7 of the 19 subfields had greater impact than the world average. We emphasize that the direction is opposite because in these subfields, only one has a high standard of absolute impact; the others have reduced citation potential because they address themes that are considered regional or of an applied nature. The normalized indicator showed a different picture for the evaluation of these subfields.Another point concerning evaluation methodological issues regards the studies that mobilize a large number of researchers, require sophisticated experimental structure and result in hyperauthorship papers. In addition to identifying the individual authors, these papers typically attribute authorship to a group given the cooperative nature of the research. Often, researchers involved in this type of research have much higher standards of productivity and scientific impact . We haveThe vast majority of scientific knowledge does not come from individual research that can sometimes engage thousands of researchers; this arrangement of scientific collaboration is as unusual as the productivity and the impact arisen from it. Our results highlight this distinction present in one of the subfields in which we identified a significant number of author-group papers that in turn had such superior impact that it altered the perception of performance in that area in the Brazilian context. Careful evaluation in this context requires recognizing the inadequacy of using absolute parameters and creating methodological approaches that allow for comparing these studies with others, maintaining the proportions of production and impact inherent to their respective realities.Finally, we highlight that scientific performance analysis using quantitative indicators cannot capture the influence of psychosocial aspects in the citation selection criteria adopted by researchers. However, evidence on the effect of peripheral conditions in recognizing the achievements of papers published by Latin American researchers was obtained by Meneghini, Packer and Nassi-Calo  when theCurrently, our research group is developing a study that adopts a qualitative approach to researchers’ perceptions of these immeasurable aspects by analyzing the products of scientific activity to complement the approach developed here.S1 Appendix(DOCX)Click here for additional data file.S2 Appendix(DOCX)Click here for additional data file.


This pilot study was designed to describe changes in spiritual well-being (SWB), spiritual coping, and quality of life (QOL) in patients with brain cancer or other neurodegenerative diseases participating in a chaplain-led spiritual life review interview and development of a spiritual legacy document (SLD).Eligible participants were enrolled and completed baseline questionnaires. They were interviewed by a board-certified chaplain about spiritual influences, beliefs, practices, values, and spiritual struggles. An SLD was prepared for each participant, and one month follow-up questionnaires were completed. Two cases are summarized, and spiritual development themes are illustrated within a spiritual development framework.A total of 27 patients completed baseline questionnaires and the interview; 24 completed the SLD, and 15 completed the follow-up questionnaire. Increases in SWB, religious coping, and QOL were detected. The majority maintained the highest (best) scores of negative religious coping, demonstrating minimal spiritual struggle.Despite the challenges of brain cancers and other neurodegenerative diseases, participants demonstrated improvements in SWB, positive religious coping, and QOL. Patient comments indicate that benefit is related to the opportunity to reflect on and integrate spiritual experiences and to preserve them for others. Research with a larger, more diverse sample is needed, as well as clinical applications for those too vulnerable to participate in longitudinal follow-up. The experience of serious illness often provokes an unexpected and unwelcome encounter with vulnerability and ultimate mortality. Such experiences invite, and even demand, reflection on established beliefs, patterns of behavior and relationships, and a recalculation of priorities. While conventional medical interventions are essential to the well-being of patients, attention to these less visible but very real issues is also important.10There is evidence that the sharing of a personal spiritual life-story can be therapeutic for patients and also instructive for clinicians, especially those committed to providing respectful, holistic care.discovery, conservation, and transformation. Discovery involves a process of spiritual formation, usually during childhood, that occurs by engaging in and learning from spiritual relationships and experiences. Conservation involves maintaining one’s learned spirituality in ways that are coherent with life roles and meaning. Transformation involves a re-examination of one’s spirituality, often in the context of spiritual struggle, leading to new or deeper convictions.This paper also presents a summary of the interviews of two participants. Themes of spiritual development within the interview are illustrated using a model developed by Pargament and colleagues.This study was approved by the Mayo Clinic Institutional Review Board in 2012. Participants were at least 18 years old, with brain cancer or neurodegenerative illnesses. Excluded were those who were psychiatric inpatients or those thought to be at risk of inflicting self-harm or harm to others. Previous spiritual care from a chaplain at Mayo Clinic was not specified as inclusion or exclusion criteria. Enrollment took place through the Mayo Clinic Cancer Center, but self-referrals were also accepted.Following the introduction of the study and informed consent procedures, patients supplied demographic information and completed baseline questionnaires: the Functional Assessment of Chronic Illness Therapy–SWB (FACIT-Sp-12),Patients were divided into three disease groups: those with advanced brain cancers, those with early-stage brain cancers, and those with non-cancerous neurodegenerative diseases. Continuous data were summarized with means and standard deviations, and categorical data with frequencies and percentages. Then, the percentage of participants in each group who improved from baseline to follow-up and the percentage of patients who maintained a maximum possible score from baseline were summarized.Thirty-two patients enrolled in this study; 5 withdrew, and 27 completed baseline questionnaires and an interview with a chaplain. The ages of the 27 participants ranged from 21.8 to 77.1 years (average 52.0). Approximately half  were female. Most were currently married  and had at least a four year college degree . Half  were Protestant, six (22.2%) were Catholic, one (3.7%) was Muslim, and six (22.2%) indicated no religious preference. Twelve patients (44.4%) had advanced brain cancers, nine (33.3%) had early-stage brain cancers, and six (22.2%) had non-cancerous neurodegenerative diseases. Only two of the 27 patients interviewed had a previous professional relationship with the interviewing chaplain. The length of the interviews ranged from approximately 30 minutes to 2.5 hours. Four patients were interviewed in two separate sessions, one because of fatigue and three because of their desire to add substantial material to their document. Twenty-four patients completed their SLD, and 15 of these completed the follow-up questionnaire. Those who did not included 10 patients who died prior to follow-up and two who did not respond to contact attempts.In several cases, patients scored the highest possible on baseline measures reflecting SWB, religious coping, and QOL, but some improvement was detected in all disease groups. Most notably, 100% (4/4) of those with neurodegenerative diseases reported an increase in SWB on the FACITSp-12 and in positive religious coping. Additionally, 75% (3/4) of this group improved on the SWB LASA and on the FACIT-Sp-12’s meaning/peace subscale. Of those with advanced brain tumors, 66.7% (2/3) reported an increase in positive religious coping and overall QOL, while 75% (6/8) of those with early-stage brain cancers reported improved positive religious coping, and 62.5% (5/8) reported improved SWB on the FACIT-Sp-12 and its meaning/peace subscale. The majority of patients in each disease group maintained the highest possible score on the negative religious coping subscale, with higher scores indicating less spiritual distress, i.e. 66% (2/3) with advanced brain tumors, 50% (4/8) with early-stage brain tumors, and 75% (3/4) with other neurologic illnesses. The score of one patient with an early-stage brain tumor improved on this measure. No statistical testing was done because of the small number per group .Summaries of the spiritual legacy interview transcripts of two patients, identified by pseudonyms, are presented below. These patients were similar in several ways. Both were married men in their mid-60s with adult children, both had been previously employed as engineers, and both had been diagnosed with an advanced brain tumor and died on study. Their spiritual perspectives were very different; one was Roman Catholic and the other an agnostic, but both demonstrate a process of spiritual development that is coherent with Pargament’s model described briefly above.discovered his faith through the influence of his parents and extended family who were devout Roman Catholics. Mr Wheeler attended worship services at least weekly and participated in regular daily prayer with his family. All were activities he treasured. Mr Wheeler credited his parochial school teachers with having a profound influence on him spiritually. They helped him to move towards conservation of his beliefs and making them his own. Activities of spiritual conservation continued after Mr Wheeler became an adult, married, and had children. He and his wife were active in their church and continued to take adult religion classes and support their children in their religious formation.Mr Wheeler was raised on a farm in the mid-western United States. He Transformation occurred when Mr Wheeler’s son became ill at age 5 and died shortly afterwards. Mr Wheeler mourned deeply. He was “sad and depressed for seven years.” His son had made a craft at Sunday school shortly before he died with this refrain from Psalm 136 on it: “God’s love is everlasting.” Mr Wheeler valued it as a message from their son to trust in God, but in his grief he wrestled with its veracity and meaning. Ultimately, Mr Wheeler found that his son’s death had “strengthened” his faith, and one day he “just decided to be happy,” and he was able to move forward. His beliefs were not radically different, but were now informed by his suffering. He found music, including church music, to be a source of consolation for him and a way that he could express his faith. He learned to play the guitar and became active in a men’s choir at his church. Then he described buying “a round table” for the dining room. He had come to a new sense of interconnectedness with his wife and daughters and did not want a position at the head of the dinner table. He said, “A round table fits round prayers,” meaning that he wanted to be sure that his wife and daughters felt equal to him and respected in their family prayer and conversation.Transformation was demonstrated again not in a change in his belief system, but a deeper reliance on it. Psalm 27, which speaks of trusting in God during times of adversity, had been one of Mr Wheeler’s favorite scripture passages, and it became more meaningful to him as he dealt with anger and sadness related to his diagnosis.Mr Wheeler’s brain tumor brought new challenges for him. He described his thinking as “not so good” and like being in “a weed patch.” Unfortunately, music, including his own singing, gave him a headache, and he lost that source of consolation. Most significantly, Mr Wheeler faced an abrupt confrontation with his mortality. Always a kind and sensitive man, he felt an imperative to express his gratitude, love, and important values to his family. He told them: “I gotta sit down with you folks and one by one, I have to thank you … for doing what you are doing.” He reported that he had accomplished this before he died. Mr Wheeler requested that the following statements be included in his SLD: “I’d like my family and others who love me to know that these two sayings have been my mottos: ‘Forgive one another.’ and ‘Be a person of integrity.’ I’d also like to leave them with three more words, ‘I love you’.”conserved this approach to spirituality, summarized by this statement, “I don’t believe in religion.” He was greatly influenced by his education and stated that he put his faith in science because of its predictability. Then, as an adult, he put his trust in his wife, because she, like science, was “predictable” and their relationship “enriched” him. Apart from his wife, Mr Nicholson led a rather insular life without tolerance for mystery and for those who thought differently than he did.Mr Nicholson credited his father and grandfather as having the most influence on him spiritually. He said, “They slanted me towards agnostic.” He was aware of the Ten Commandments as a code of conduct, but said, “That’s as close as I got to religion.” As he grew up, he As a young adult, Mr Nicholson struggled with alcoholism and a negative and critical approach to life. He could be very judgmental when things or people were not as he expected them to be. His decision to quit drinking alcohol “made a huge difference,” especially in terms of his relationships. If he ever got “off track” he said, “Just looking back was enough,” to turn him around.transformation in his spiritual world view and approach to others. His eyes were opened to the goodness of others and he experienced a sense of gratitude and connectedness that was new to him. He also found himself more aware of others in general and “more empathetic to people who have life-threatening diseases.” He was aware of the harsh way he had treated others when he was drinking and, in later years, under stress. He expressed his deep remorse for the “devastating” consequences related to this behavior. While his agnostic perspective led him “not to expect much” as he looked to the future, he began to entertain the possibility of an afterlife for himself. If there was, he planned to do what he could to help his family whenever they needed him. He said, “I’ll leave indicators that I got the message and that I’m operating.” He advised his loved ones, “Be good to people … The act of sharing is pretty important. Make sure you do that throughout your life.”Mr Nicholson’s diagnosis of brain cancer prompted a bounty of unexpected support and help from others in his life. This caused him to reflect and prompted This study provided patients with brain cancers and other neurodegenerative diseases an opportunity to give voice to their spirituality in a comprehensive manner within the context of a relationship with a spiritual care professional, and to prepare an SLD for their loved ones. Though participants had a variety of belief systems and were in the midst of challenging disease processes, enrollment data demonstrated that most patients had relatively high levels of spiritual well-being and quality of life, and relatively low levels of spiritual distress. Follow-up data indicated that most patients either maintained or improved in these areas. This seems quite remarkable and encourages further exploration of the potential for spiritual well-being and spiritual growth during times of medical adversity.This study is limited by its small sample size and the lack of a control group, so definite conclusions based on its results must be curbed. However, spontaneous comments by patients, both within the interview and afterwards, suggest that their experience in the study was helpful to them in several ways. Specifically, patients expressed their gratitude for the opportunity to reflect on their lives and to integrate both positive and challenging experiences related to their diagnosis and disease progression. Patients also reported valuing the opportunity to preserve significant aspects of their spiritual journey for their loved ones and others, and hoped that their words would influence them in positive ways. Several also offered their appreciation for the respectful and “careful listening” of the interviewer. These comments alongside the survey results suggest that the spiritual life review process benefitted patients by affirming their dignity and offering them an opportunity to make a difference in the lives of others.As in any clinical study, it is very important to be mindful of those who did not improve on various measures and to explore reasons for this. Disease progression is a likely contributor, but it is also possible that other adversity occurred. No participants indicated that the study itself was unsettling in any way. It will be important to include a targeted evaluation of the participants’ experience to future protocols. It is also important to understand the relationship of SWB, QOL, and positive religious coping with spiritual struggle. In the results of this study, it appears that doing well spiritually does not preclude spiritual struggle. This needs to be verified, as it could be very helpful information for all who seek to be attentive to a patient’s spirituality.The two summarized cases demonstrate the feasibility of using Pargament’s framework to identify experiences related to spiritual development. The investigators intend to use this model in their qualitative analysis of the interview data. The cases also highlight specific ways that adversity challenged the patient’s spiritual perspective and promoted spiritual change. In the two cases presented, this change was positive and enriching for the patients and their loved ones. It is possible that reflecting on this change during the interview was beneficial and contributed to the results noted above.More research is needed to evaluate the efficacy of the spiritual life review process described in this pilot study. Future protocols should consider larger sample sizes, randomized designs, and earlier follow-up, especially for those with advanced disease. Adaptation of the interview process and SLD development for other patient groups and those beyond the United States could provide benefit for participants and interesting and educational results. Clinically, sharing parts of a patient’s SLD with the health care team could enrich and deepen the bond between them and guide medical care in ways that champion the dignity of each person and the spirit alive within.


In Indonesia there is a pressing need to scale up mental health services due to a substantial unmet need for mental health care. Integrating psychologists into primary health care can potentially deliver affordable mental health services to communities and help to close the treatment gap. Australia is one of the pioneers in integrating mental health into primary health care, and the mental health reforms in Australia may have some implications for Indonesia. The aim of this paper is to examine the Australian experience and to reflect in particular on lessons that may be learnt to inform the development of curriculum for psychologists working in primary health care in Indonesia. Data were collected through semi-structured interviews with 12 Australian experts in primary mental health care. The focus of the interview was on the roles and skills of psychologists working in primary health care with a particular focus on the appropriate curriculum for psychologists. Overall, the Australian experts agreed that psychologists' roles and training should include both clinical skills and public mental health skills. The experts also agreed that psychologists should be able to educate the community about mental health issues and be capable of undertaking research and evaluation of programs. A central theme was the need for strong collaborations with general practitioners and existing agencies in the community so that psychologists are able to make appropriate referrals and also accept referrals. The lessons learnt from the Australian experience, which are most applicable to the Indonesian setting are: (1) the importance of adequate government funding of psychologists; (2) the value of evidence-based treatments such as Cognitive Behavioural Therapy; (3) the need to specifically train psychologists for primary care; (4) the need for flexibility in the psychologist workforce (e.g. location); and (5) the value of continuing supervision for psychologists to support them in their role. Millions of people in Indonesia are affected by mental disorders. They do not affect only one specific group, but are prevalent in all communities and regions throughout the country. According to 2007 data in Indonesia, the prevalence of common mental disorders is reported to be 11.6% in the adult population  initiative was implemented by the Australian Government in 2001  in collaboration with the Faculty of Psychology, Universitas Gadjah Mada in 2004. It was the first pilot project funded by the Local District Government. Starting with only six psychologists in 2004, the project has since expanded to include 25 psychologists, which translates to one psychologist for each primary health care clinic. The expanded pilot project program for the Yogyakarta Municipality (under the same Province) commenced in July 2010 .Publication: The principal investigator reviewed the literature on primary mental health care and recorded the names of authors who were frequently published or cited by others. The principal investigator contacted these authors to seek voluntarily participation in the semi-structured interviews.Snowball sampling: After those contacted agreed to participate, the principal investigator asked them for recommendations for other potential participants to be invited.Recommendation from a key professional organisation: Recommendations were sought from the Australian Psychological Society (APS) as the lead professional organisation in this field in Australia.The participants were Australian experts in primary mental health care across a range of disciplines, who were directly or indirectly involved in the integration of psychologists in primary health care. Participants were identified through one of the following methods:Since there is no uniform agreement in the literature on an optimal sample size for qualitative research Beitin, , the numThe background of participants in the study were psychologists working in primary health care, psychologists working in rural/remote/indigenous area, a representative from the APS, a psychiatrist, a senior lecturer of psychology, a researcher in mental health policy, a GP and a cross-cultural expert who has in-depth understanding of Australian and Indonesian cultures.All identified participants were contacted by sending a formal letter of invitation and a consent form. The interviews were then conducted at a convenient time and place for each participant. Twelve experts agreed to be interviewed by the principal investigator and to be tape-recorded during the interview. Data were being analysed concurrently and after the 12 experts being interviewed and there were no new themes emerging, recruitment of participants was stopped. Audit trail by writing field notes on how the research was conducted and what the principal investigator hears, sees and thinks during the fieldwork was conducted to ensure the quality.The data collected were transcribed verbatim and analysed using a thematic approach. The Nvivo 10 software  patients, (2) other primary health care providers, (3) primary health care as service providers, (4) community and (5) policy.Most participants mentioned that the basic role, and skill, of psychologists in primary health care is providing clinical psychological services. These ranged from conducting assessments to providing treatments. In general, there are four main findings on roles and skills in relation to patients: (1) capability to handle general community populations; (2) assessment skills; (3) diagnostic skills; and (4) knowledge and skills in providing treatment.Capability to handle general broad populations was a strong theme. For example one participant stated that “psychologists largely practice very much like generalists” P1-82. Assessment skill was another important role. For example one participant stated that “the assessment is really the psychologists’ role” (P-10). The participants also highlighted that in conducting an assessment, psychologists should “be able to conduct a comprehensive assessment,” (P-1) and should also understand “how to assess people with various kind [or aspects] of assessment, like mental health status, or cognitive status, so they are trained to do an assessment” (P-9). Furthermore, the participants pointed out that conducting assessments should not only employ existing assessment tools, but should also include developing tools tailored to a patient's culture. As one expert explained: “I'm using some Aboriginal measures I developed. In my research I'm developing Aboriginal measures” (P-3).Providing diagnosis was highlighted by the participants as the next important role or skill. For example, participants emphasised that psychologists should know “how to diagnose them” (P-9). Another participant pointed out the importance to “know what is going on diagnostically” (P-2). This skill is a basic foundation for the next skill, providing treatment.The other set of skills then is treatment, so there are concrete interventions, strategies, that could be offered immediately to take the pressure off, and then subsequently to that, we might help them change their thinking pattern that might have led to this. So, I wouldn't see this as either/or. I would start with a very concrete strategy, to help the person gain some control over their anxiety, or their fear, or their mood, and then we would go back to the other stuff later. (P-12)Knowledge and skill in providing treatment that matches the diagnosis was considered as very important, as demonstrated in these quotes: “to match whatever the disorder was with the best treatments for these disorders” (P-9) and “knowing what are specific treatments are for this specific diagnosis” (P-4). Specifically, psychologists should be able to “deliver these evidence-based treatments” (P-12), as well as “provide effective and appropriate treatments” (P-1). In primary health care, the treatments should contain “some really concrete strategies” (P-3). Another participant explained in detail: … know how to connect with those people and develop relationships, also the appropriate way to refer to other people. You need to know how to speak the right language and know the right words to get another service to accept your patient, to accept the referral. I think those sort of skills are really important. (P-6)There are two main themes about psychologists' roles and skills in relation to other primary health care providers: (1) inter-professional communication and (2) recognising the unique contribution of psychologists. Most participants agreed that communication is the key to building a good relationship between psychologists and other primary health care providers. One expert mentioned the importance of the “ability to communicate effectively with other professionals” (P-1) and another expert highlighted that “we need a standard relationship skill” (P-8). Communication with other health professionals includes an understanding of how to refer and accept referrals. One of the participants highlighted that psychologists should: … providing the consumers with the service, and communicating back to the GP. You know what has happened and what else you'd recommend for the client. But I think sometimes it's up to the GP and the psychologists. So I guess if you know the GP well, they might be supportive, communicate through the telephone, make telephone calls, write letters, they are concerned about risk to the client. (P-1)To communicate with other primary health care providers, especially GPs, psychologists should be supportive and employ different modes of communication including letter and telephone. This skill will help to “maintain the flow of referrals with the GP” (P-1). One participant mentioned that psychologists have to be capable of:I think the capacity to highlight the unique strength of psychologists, is critical. And also the capacity to feel equal and to communicate equally with doctors. Some people still feel intimidated by doctors, and I think that's ridiculous. We've got to develop a sense that they are our colleagues, friends, and that we're all equal. We are doing things differently. (P-10)The ability to emphasise the unique strength of psychologists and to recognise that they are equal to other Psychologists should be self-confident in promoting their unique skill set professionals is also very important. One participant highlighted this:There are two main themes that emerged regarding the roles and skills of psychologists in relation to primary health care service providers: (1) analysis and evaluation skills; and (2) managing stigma.Analysis and evaluation skills are clearly very important to contribute to primary health care as service providers. One participant highlighted that psychologists should be capable to “analyze suicide, social-emotional wellbeing, health and mental wellbeing” (P-3). Two participants expressed that psychologists also should be very capable of “conducting evaluation” (P-6) and “program evaluation” (P-9). The analysis skill would give psychologists strong basics to develop a mental health program and the evaluation skill would help psychologists to develop effective primary health care service.Managing the stigma of mental illness is also another important theme, as stigma can be a big barrier in accessing psychological services. One participant stated that “barriers can be from patients, which can be the stigma”. Seeing a psychologists can be perceived as “if you see a psychologist it means you're mad”. Another participant stated, “ … fear that they are going to get inside and change my mind. And there is, I think, quite a lot of fear about what psychologists do” (P-2). Finally one interviewee expressed, “I would equally feel that if psychology can remain in primary care rather than having to be sent off to some sort of super specialist practice, it would overcome some of that stigma” (P-2). Psychologists working together in primary health care can reduce that stigma of mental illness.Three themes arose regarding psychologist's role in the community; (1) contributing to public health; (2) respecting different cultures and (3) promoting familiarity with existing services in the community. One participant stated, “contribute to public health is important” (P-10) and another noted that one role for psychologists is “ … providing a bit of health education” (P-5). These statements suggest that study participants recognised that psychologists have a responsibility not just to individual clients, but also to the larger community. As a consequence, psychologists have to have a public health perspective.In Aboriginal culture, the way we present to community relations, we call it a family counselling service. We don't call it mental health. As the family counselling service, we present it in more holistic manner so we have adolescents and children upstairs, adult downstairs. And we try to provide integrated approach, where that means we have GPs, family counselling service, and psychiatrists. (P-3)Respecting culture was also viewed as an important skill for psychologists. Psychologists should understand that mental health problems are perceived differently in different cultures and this also has implications for service design, for example:If you're working out there and you don't know who's around you, or you don't know how to refer to these agencies or how to get your patients some food when you don't have anything in the cupboard, it's really very stressful. You can't do therapy and you can't treat somebody's depression if you have no food to eat for the weekend, you constantly need to link other people to the services. (P-6)Familiarity with existing services in the community is also very important, so that psychologists can appropriately refer clients to them. One participant highlighted this point: “I think a lot of this integrated primary work should understand about moving into the community and taking into consideration familiar alliances into the community by taking care of what already exists” (P-3). Respondents indicated that issues with accessing other basic necessities such as food, accommodation and finances should be addressed at the same time as mental health problems. Psychologists should understand that some patients might have complex needs that require social services to be in place. Therefore, the relationship to existing community services is very important:The main findings in relation to policy are that psychologists ought to have (1) good understanding health system and (2) a capacity to advocate for patients within the health system. Psychologists should have a good understanding of the mental health system and the general health system, such as payment scheme for psychologists: “Good understanding of all the different sources of funding, and how it works. So, understand(ing) the government policy” (P-6). Another participant highlighted that psychologists should have an “understanding not just [of] the mental health care system, but (of) the general health care system” (P-7). Therefore, psychologists could involve themselves in “advocacy and navigating health system to prioritise mental health” (P-4).Experts were asked to comment about the integration of psychologists into primary health care in Australia. There are five main findings that are most applicable to the Indonesian setting: (1) the importance of adequate government funding of psychologists; (2) the value of evidence-based treatments such as Cognitive Behavioural Therapy (CBT); (3) the need to better train psychologists for primary care; (4) the need for flexibility in the psychologist workforce (e.g. location); and (5) the value of continuing supervision for psychologists to support them in their role.ATAPS is 12 sessions per year, but, in exceptional circumstances, the GP can give a referral for another six, so they can get 18 sessions in a year, where in Medicare that just changed with the last federal budget, I think they can get 10 sessions. (P-6)Most participants perceived that the implementation of government funding has facilitated the integration of psychologists in primary health care. Specifically, the newly introduced funding model helps patients to be able to afford and access psychologists. Experts highlighted that patients can now avail of sessions with a psychologist for a set number of times a year under the ATAPS funding: … under Better Access and Better Outcomes, people use a lot of common behavioral therapy, CBT, and I think that works really well especially when you've got a time limited number of sessions, so you can equip people into six to twelve sessions, with specific skills. I think people do a lot of that. (P-6)The funding especially encouraged evidence-based treatments and there is a strong focus on CBT as a psychotherapy approach. This appears to be the expectation of many of the GPs referring patients: “Some GPs will give you some more extent of information and others will just like say ‘anxiety-depression’, ‘provide them with CBT’ (P-1)”. Another expert also pointed out that CBT works very well in a limited number of sessions:Well, I think they need to have placement experiences in primary care, not only to learn, but to see, “Is this what I like?”, “Do I like to work in this place?”. Yes, I'd like to see more primary care placements. Because then we get opinions about what the work is like not from your lectures, but from your placement experiences. (P-9)Even with these large scale reforms, insufficient training for preparing psychologists to work in primary health care is still an issue in Australia. One participant mentioned the benefits of primary care placement, as one aspect of the current training that needed to be improved:Well, the reality is that the Better Access model was designed to support GP and psychologists working together in primary health care, but what exactly happened is a lot of psychologists continue to work on their own solo; they are in a private practice setting, and while they are forced to communicate with the GP you know after the six sessions you are meant to write a report and so on. Basically good primary health care psychologists collocated with the primary care general practitioners, so that's my understanding of the model. (P-10)The integration of psychologists into primary health care in Australia does not necessarily mean that psychologists work together with GPs or other primary health care providers in the same clinics. However, one participant did highlight the benefit of co-location with GPs and other primary health care providers:They have to find supervisor and they have to pay for supervision. We want more and more psychologists to have more and more continuing supervision, even when they are qualified, because you need supervisor, especially in the first months after you finish study. (P-9)The Australian system strongly encourages continuing supervision by senior psychologists: “It needs to be supervised, because when you're working in primary care, quite often you're working independently. So, I think supervision is very important and is encouraged in Australia” (P-1). The participant also highlighted that even psychologists have to find and pay their supervisors, and it is a very important aspect:The participants in this study identified clear roles and skills for psychologists to successfully work in primary health care. These roles and skills highlighted by participants relate to their relationship to their patients, other primary health care providers, primary health service providers, policy and also the broader community. The broad spectrum of the roles and skills is consistent with the conceptual model proposed by Talen, Fraser, and Cauley  based onThis study also found a set of roles and skills for psychologists that are consistent with those previously proposed by McDaniel, Belar, Schroeder, Hargrove, and Freeman . For exaThe participants also highlighted the importance of inter-professional collaboration in primary health care. This collaboration includes the ability to communicate through multiple channels such as letter and telephone. The participants also emphasised that psychologists should feel equal to other health professionals and should be feel confident that they can make their own contribution to clinical care. These points are consistent with previous research of integrated care with interpersonal and communication skills highlighted in research by Bray  and BlueThe main lessons that can be drawn from Australian experiences to be implemented in Indonesian were: (1) the importance of adequate government funding of psychologists; (2) the value of evidence-based treatments such as CBT; (3) the need to better train psychologists for primary health care; (4) the need for flexibility for the psychologist workforce (e.g. location); and (5) the value of continuing supervision for psychologists to support them in their role.The importance of funding was recognised as an essential factor to encourage the integration of psychologists into primary health care. There was a strong appreciation of the importance of the ATAPS and BOMHC funding in Australia as a critical step. In the US context, researchers have identified that funding is recognised as a key starting point for mental health providers' willingness to accept patients  the importance of adequate government funding of psychologists; (2) the value of evidence-based treatments such as CBT; (3) the need to better train psychologists for primary care; (4) the need for flexibility for the psychologist workforce (e.g. location); and (5) the value of continuing supervision for psychologists to support them in their role. The application to the Indonesian context needs careful attention due to different legal, policy and culture. Therefore, two further studies were conducted to follow up this study (Setiyawati, Blashki, Wraith, Colucci, & Minas,