Reduced patient portal use has previously been reported among Black Americans when compared with that of the general population. This statistic is concerning because portals have been shown to improve the control of chronic conditions that are more prevalent and severe in Black Americans. At their very simplest, portals allow patients to access their electronic health records and often provide tools for patients to interact with their own health information, treatment team members, and insurance companies. However, research suggests that Black American patients have greater concerns over a lack of support, loss of privacy, and reduced personalization of care compared with other Americans, which results in a disparity of portal use. This qualitative investigation of primary care experiences of Black Americans from across the United States who participated in remote focus groups in April and May 2020 aims to explore the use and perceived value of patient portals to better understand any barriers to optimized treatment in the primary care setting.We performed an inductive thematic analysis of 8 remote focus group interviews with 29 Black American patients aged 30-60 years to qualitatively assess the experiences of Black American patients with regular access to portals. Thematic analysis uncovered the following interrelated themes regarding patient portals in primary care: the optimization of care, patient empowerment, patient-provider communication, and patient burden. In contrast to what has been described regarding the reluctance of Black Americans to engage with patient portals, our focus groups revealed the general acceptance of patient portals, which were described overwhelmingly as tools with the potential for providing exceptional, personalized care that may even work to mitigate the unfair burden of disease for Black Americans in primary care settings. Thus, opportunities for better health care will clearly arise with increased communication, experience, and adoption of remote health care practices among Black Americans. The use of electronic health records (EHRs) is rapidly becoming the norm for primary care services. The American Recovery and Reinvestment Act of 2009 attempted to provide access to EHRs for every American within 5 years through US $19 billion in incentives in the Health Information Technology for Economic and Clinical Health Act .Patient portals, which were first developed by software vendors and health care centers in 1999, provide patients access to their medical charts and other data from their EHRs . PatientCOVID-19 and the need to implement physical distancing to limit the spread of the associated virus have reduced the availability of in-person visits for primary and nonemergent care since early 2020 . As of tWe used focus groups to understand potential barriers to optimized care for conditions disproportionately affecting Black Americans, and participants repeatedly noted the impact of using portals on their care. The perceived utility and value of patient portals were often integral to participant experiences with care and their relationships with providers, health care institutions, and their own health, as detailed herein.Self-identified Black American residents aged 30-60 years with regular access to primary care were recruited nationally using Craigslist and Research Match advertisements. Recruitment continued until thematic saturation  was reacEach interview was an hour long, with a maximum of 5 participants and 2-3 facilitators. One facilitator was responsible for primary facilitation and asking probing questions and at least one facilitator took notes. The focus groups began with the facilitators describing their role and any personal connection to the research and reminding participants that the information disclosed in the interview would remain confidential. The participants then introduced themselves and provided a first name or nickname for use in the focus group to engage with other participants; no other personal identifiers were provided during the session. The subsequent discussion was participant led. At least one facilitator self-identified as Black in every session. Interviews were recorded with video and audio, transcribed verbatim, and deidentified by the primary facilitator.Patterns within the data were analyzed and identified through inductive thematic analysis using NVivo software  . An inteA total of two theoretical frameworks were integrated to direct the discussion of emergent themes related to patient portal use among the participants: the technology acceptance model (TAM) and the health belief model (HBM) . The TAMThe data that support these findings are available on request from the corresponding author, NPB; the data are not publicly available because of ethical restrictions.Optimization of Care, Patient Empowerment, Patient-Provider Communication, and Patient Burden sections, we provide some of the most salient individual examples that emerged from discussion in each of these themes.Eight focus groups engaged 29 participants (22 women) with a mean age of 40 (SD 8) years. Intercoder reliability was found to be in ≥95% agreement, such that secondary codes were not used in the analysis. Of the 29 participants, 22 (76%) discussed experiences related to patient portals. From the inductive thematic analysis, four themes emerged regarding patient portal use among Black Americans in primary care: optimization of care, patient empowerment, patient-provider communication, and patient burden. go-to tool for patients before and after each visit. Before the appointment, these portals were used for scheduling, requests for tests, insurance navigation, and communications. Scheduling is often the first point of contact for patients:Most participants described patient portals as a tool to keep both the doctor and patient informed and organized, as well as to connect with the treatment team. For many, patient portals are integrated into the appointment routine and are considered a [The portal is] a really convenient tool especially for this day and age. I think that the doctor is more accessible that way. It’s a lot harder to have to schedule an appointment and have to go [in-person]. And then if something gets in the way and you have to reschedule, it could all be delayed.Participant 4, group 1, Portland, OregonAfter scheduling, the portal helped patients to prepare for their next visit. As noted by 1 participant, physicians may respond differently to requests for tests. For this participant, use of the portal before appointments complemented her own engagement with her health, as well as her new physician’s willingness to work with her to understand her symptoms:[The portal is] typically how I get my information...When I was desperately trying to find out what is wrong with me, I was always going online or finding something that I thought I might have. Interestingly enough, I would email my primary care physician via the web portal, and he would say “okay, well we’ll test you for this or test you for that...” My last experience before I switched primary care physicians was most harrowing: I didn’t feel heard, and I’m sure a lot of people have had this experience as well where they say “oh, you have this so, here, get this prescription,” or “do this,” or “do that.” So, it’s been quite a journey to finally find a diagnosis for what I have...I like [my new physician’s] approach because he is kind of like “let’s rule it out” versus, you know, “let me figure it out or let me tell you what I think you have....”Participant 23, group 4, Fontana, CaliforniaCommunication with the treatment team before the appointment reduced the possibilities of the participants forgetting or becoming too nervous to address specific health concerns:I think you just have more time when you’re on the computer to think of what you really [want to] tell the doctor....Participant 3, group 1, Denver, ColoradoDuring appointments, the participants reported the efficiency of having personal and family health histories centralized and readily available to the treatment team through their EHR:[The portal] also makes things quicker because [the doctor] could just be like “alright from your last visit I see this, and let’s go ahead and move forward with that.” I think because the doctor already knows what they’re dealing with it probably makes it easier for them to make a more informed decision about [my] care... [When I] switch doctors it kind of feels like starting over...I think it helps having these uniform systems [and patient] information in the portal so [the new doctor] can at least catch up.Participant 4, group 1, Portland, OregonAfter the appointment, treatment teams often contacted patients first  for next steps. Participant access to portals for after-visit summaries and treatment recommendations  streamlined visit follow-up. Some platforms allowed participants to make payments and view insurance information:[The portal is] my best good friend, medically speaking, because it’s what I prefer to use to check my results. [My provider] also follows up with conversations via the portal, so I use the portal quite a bit...from payment options, to referrals, to the information she conveys to me—any recommendations she makes [when] we are no longer face to face, that’s my primary go-to.Participant 13, group 3, Pasadena, MarylandCentralized patient health records, accessible to both the treatment team and the patient, limited redundancies and mistakes:The portal is a very simple system, but I’m very thankful for that. [My primary care physician and I] can go back-and-forth and then if she tells me a certain dosage, I can just look on the portal—“oh what did she say?”Participant 15, group 6, Staten Island, New YorkFor participants with relatively few health concerns, portal use reduced the number of appointments because test results and prescriptions are accessible on the web:We use the portal a lot; sometimes I’m like three or four back-and-forth. In my mind this would’ve been a doctor’s visit back in the day, but we go back-and-forth.Participant 15, group 6, Staten Island, New YorkI rarely have any other visits in-between. I like being able to go get my physical and then almost instantaneously get those results...delivered to me via the portal.Participant 17, group 5, Nashville, TennesseeThe participants generally appreciated the integration of portals into their treatment:I remember the time that you had to ask them for your results and stuff like that. I think it adds [to my treatment] because then you already have a sense of—sometimes before the visit—of things you’re going to talk about. So, I think it’s more helpful and empowering for patients to have that access.Participant 14, group 4, Portland, OregonDistrust was a major topic in the discussion of treatments for diseases that disproportionately affect Black Americans. Distrust was founded primarily on historical, personal, and media-based accounts of malpractice. Providers ignoring medication allergies or preferences was a commonly reported cause for distrust and switching providers. Under these circumstances, portals may facilitate the transfer of records. Patient portals also play a unique role in mitigating distrust, as participants described the importance of personal and family health histories—instead of, or in addition to, racial identity—in the consideration of a new treatment or therapy:I’ve been with my primary care [provider] for years and years and years and I’ve been very vocal about what my concerns are, my family history, and my culture, and I’m already putting that stuff out there. [If] they come back to me like “oh, I agree with you, and per this research...,” I would be more receptive, because it’s like, “okay, you actually heard where I’m coming from. And you’re listening to what my concerns are and why I have these concerns, and you’re trying to work with me to create a solution.” But if it’s definitely from a place of privilege where it’s like “well, you know, you Blacks, the fried chicken and you got—” it’s like “oop, sorry, new doctor! That’s what you’re not going to do.”Participant 23, group 4, La Mesa, CaliforniaThis experience highlights the desire for personalized interactions among the participants and the importance of providing educational resources that connect previous discussions between the patient and provider to any new treatment recommendations. Educational resources were often provided digitally after appointments through secure messaging or portal-based visit summaries. Others obtained this information on their own through internet searches.Patient Burden section). Thorough reporting of health concerns through the portal was especially useful before the visit because crossing the digital divide served as an icebreaker in what can sometimes be timid interactions with the treatment team:User-friendly portals also empowered the participants to discuss sensitive matters more transparently and to engage in their own health care more effectively. Most portal users reported benefiting from additional control of their appointments, although some participants viewed the portals as a hurdle between themselves and their provider rather than an additional avenue for communication  and to obtain second opinions easily.Some participants expected patient portals to reduce personalization of care but instead found that the portals supplemented their interactions with their treatment team. The accessibility of portal communications enabled many participants to address their concerns more thoroughly and accurately, increasing their engagement with their treatment plans. Participants with negative experiences regarding the portal were often dissatisfied with their treatment team at the time and were sometimes seeking a new primary care provider. Overall, communication through portals mirrored communication patterns already established between the patient and their provider :[The portal is] a double-edged sword, right?...The best way of thinking of it I think is that [if] I’m texting with my doctor, you can either give me a short answer or just say “yes/no” without any other follow up or results. And there’s other instances that people use that as a “hey, I saw your message, I actually wanted to give you a call. After I get off the phone with you, I’m going to send you another message and we’ll check in...” So, it all depends really more so on the doctor. If they use that as another touch point to stay in touch, that’s good.Participant 23, group 4, La Mesa, CaliforniaAssessments that were performed that day [are] delivered to me via [the] portal...I’ve also given them permission to notify me by email and by text....I’ll get a [text] message, that might say “you have a new test result” and I like that, and I find it very helpful...I like the connectivity...Whenever I’ve had to reach out to my primary care physician, I get a pretty good return rate. You know, usually within 24-48 hours if I ever have any questions....Participant 17, group 5, Nashville, TennesseeProvider response time was generally important to participants, as was the thoroughness of response:Pretty rapid response time but very minimal in the communication. Just very, like, terse, which is what it is. That was the only method of communication.Participant 16, group 6, Portland, OregonAlthough communication through patient portals was common, portal use failed to supplement in-person communication for some participants. Ineffective communication patterns and delegation of follow-ups from the provider to other treatment team members  compromised satisfaction with portal-based communications. Portal use hindered effective communication in at least two scenarios. First, when a phone call was still necessary to fill the request or answer a question:It’s a longer process for them to get into the portal when I can just call and say what I have to say, instead of waiting a week to get an answer.Participant 22, group 5, St. Louis, MissouriSecond, when digital responses from the provider are delayed, insufficiently detailed, or confusing:When the replies are delayed, that kills me the most. Like, “hey, I sent a message on Monday. Here it is, now Monday the following week. What’s up?”Participant 23, group 4, La Mesa, CaliforniaFor many participants, the nurse or medical assistant was in charge of much of the digital communication with the patients. Delegation to other treatment team members limited digital communication with primary care providers for some participants compared with those who received secure emails and phone call check-ins from their provider. Although some participants were indifferent to delegated communications, personalized messages from the primary care provider were especially appreciated. Weekend, after-hours, and between-appointment communications represented a level of care that exceeded expectations.Portals provide a variety of tools but also rely on the patient to take advantage of these tools outside of the appointment. Some participants had trouble adapting. In addition to technological setbacks, some participants described the portals as an extra step to contact their treatment team, especially in cases where a phone call is inevitable. The sophistication of the portal interface may also play a role in burdening the patient because some participants reported that messages are not forwarded to their email, requiring them to log in proactively to check their inbox. A Portland, Oregon, participant recalled that she had experienced this issue with her past primary care provider:Everything was communicated through a portal. But I had to actively go into the portal to see if there were communications. It wasn’t like if she sent me a message and that would show up in my personal email and then I would know—I had to really be looking and waiting.Participant 16, group 6, Portland, OregonMany participants were aware of the lack of interoperability among portals and were frustrated at having to recount family health history to new providers when switching insurance networks or moving:I bring everything back to my primary care doctor. There’s not a lot of communication between specialists. They do have their own little charts [records] and that can be hard.Participant 18, group 6, San Diego, CaliforniaIn addition to changing appointment routines to adapt to the portal, some participants with long-term relationships with their providers noted the burden that portals placed on hospitals and treatment team members to learn how to implement them in their practice. This physician burden resulted in at least 2 participants losing their long-term primary care provider when the portal was introduced at their place of care:This doctor retired and then, I don't know, he said he was kind of almost forced out a few years back when everything was going digital and he didn’t. He’s on the older side so he didn’t kinda learn the new system and stuff...I did have to switch to someone in the meantime, and I got a referral for her from a family member. But, yeah, she wasn’t as accommodating I’d say, and that’s why I went back to my first doctor.Participant 5, group 2, Portland, OregonThe participant chose the existing rapport she had with her very accommodating provider over a digitalized form of their relationship. Another participant noticed that she rarely received responses to secure messages and wondered if her treatment team had any training on the use of portal-based communications.In the past, structural barriers have limited the initial portal sign-on by Black Americans -13. In oApplying the TAM, the emergent themes in our focus groups suggest that when the perceived usefulness and ease of use of patient portals improve, attitudes toward portals improve and so does the actual use of patient portals among Black Americans . The perAlthough telehealth may replace some elements of in-person visits, the results from this study suggest that portals may be relied upon for the optimization of care before, during, and after appointments when used by Black Americans. Once a telehealth or in-person appointment is made, the personalization of care may still be limited by implicit bias against Black Americans. Patient portals may help reduce this disparity by holding providers accountable for addressing specific health concerns documented on the web, rather than requiring patients to quickly recall and disclose sensitive information in person. Patients run the risk of making mistakes, feeling uncomfortable, and even being traumatized by having to repeat health information every time they meet with a provider when their personal health information is decentralized  or if the patient lacks an EHR altogether . Black pIn contrast, race is often leveraged to assess risk in the treatment of Black Americans for conditions that they are at heightened risk for developing severe symptoms, including COVID-19 ,29. RaceThe impacts of race-based risk assessments for Black Americans at clinics include under- or overprescription of pain medications, reduced options for life-saving surgeries such as cesarean sections during childbirth because of preoperative risk assessments, less aggressive screenings for bone disorders and some forms of cancer because of lower risk of developing the condition , and increased or decreased likelihood of intervention resulting from a decreased likelihood of survival  . AlthougPortal-based communications emphasize the communication patterns already established between a patient and their provider. To this extent, portals provide the treatment team with additional opportunities for exceptional and tailored care. Care centered on the unique needs and experiences of the patient may be an expectation for many patients, particularly those with access to their own health records , but BlaThe potential study limitations include generalizability to populations lacking health coverage, access to internet-compatible devices, or regular access to patient portals. The themes discussed in this paper should be confirmed using quantitative methods, especially considering the increased use of telemedicine over the last year ,14,15. W


Direct contact with nature is paramount in deepening children’s and teenagers’ interest in biodiversity. Learning materials chosen to convey information and engage participants during outings in nature-rich environments are varied and can support rich learning experiences. For this purpose, learning materials can be acquired "off-the-shelf" or developed for site-specific locations or projects. However, there is little guidance on potential techniques for those wishing to generate contextually relevant materials. With the view of responding to this challenge, we propose the cultural probes technique. We demonstrate that the technique, commonly used in qualitative research to generate novel insights in conversation with participants, can instigate innovative and thoughtful approaches to materials designed for children and teenagers to explore nature. We present a toolkit that draws on the literature on cultural probes, inquiry-based learning, and the value of sensory, emotional, and aesthetic experiences in environmental education for structuring interactions with participants. To test our approach, we applied a descriptive research design and mixed-methods approach for collecting questions from youths between the ages of 10 and 18, inspired by a nature walk and a set of exploratory tasks executed through the toolkit. Specifically, we tested our toolkit along a trail in the Nature Park of Terceira, situated in the Azores, a Portuguese volcanic archipelago in the North Atlantic. Here, we present and reflect on the data collected during one visit organized over two days with two groups of participants and one post-trail activity directed at both groups. Results demonstrate that the open-ended and playful nature of cultural probes offers a novel way to engage youths with nature-rich environments through questioning. This contribution further highlights the potential of cultural probes for instigating encounters that tap into the value of sensory, emotional, and aesthetic experience in nature, with positive outcomes for participants. Environmental Education (EE) strives to create "opportunities to explore nature in the outdoors", to provide "information about conservation and environmental issues", and to convey "knowledge and skills that can be used to defend, protect, conserve, or restore the environment" . To thisDrawing on Ardoin et al.’s "exploration of future trends in environmental education research" , EE, as Environmental education materials (EEM) can be key in supporting meaningful interaction with nature, both the classroom and in outdoor settings. When developing and designing printed leaflets, books, audiovisual materials, websites, and content for digital devices, it is essential to consider that “materials should acknowledge that feelings, experiences, and attitudes shape environmental perceptions and issues” . For exaEE guidance indeed recognizes that materials should “meet the specific needs of the site or community where they will be employed", stating that "content should emerge from and address the needs of a local community" . In factHedychium gardnerianum, Pittosporum undulatum), climate change [We developed this study in the Azores, the northernmost of the four Macaronesia archipelagos featuring a Mediterranean hotspot of biodiversity , characte change , 14, thee change , pose sie change .As a backdrop to the study, we have Field Guide (FG), a project  comprisiDesign researchers Gaver, Dunne and Pacenti  first inAs detailed above, the CP technique often entails creating packages that present participants with a series of evocative tasks. Packages typically contain maps, postcards, disposable cameras, diaries, and other familiar objects selected for exploring a given context. Thoring et al.  present While clear instructions accompany most probe items, researchers might find it helpful to add “random probes” through which participants can “decide (for) themselves what to document and when”. Akin to quantitative research methods and despite the qualitative nature of the CP technique, the authors also advocate the use of questionnaires and surveys as complements. Finally, “supporting material” in the form of pens and paper is also valuable. At the same time, the inclusion of a “gift” can help make a cultural probe package more compelling and help further encourage participation.The authors  further In the context of design-led research, Thoring et al.  present EEM should not only convey information, but they also “should build lifelong skills that enable learners to address environmental issues” and that emphasize creative thinking skills, such as the formulation of new questions . From thInformed by a constructivist approach, IBL has mostly impacted the learning sciences with positive implications for environmental education , 24. TheWe are accustomed to questioning in science education . HoweverIn our use of the CP technique, we also considered the work of Sauvé , which iWhile referencing the work of Jickling , Sauvé  highlighwith and in nature. As a result, we formulated the following research questions:In describing in greater detail the CP technique, IBL and EE’s focus on sensory, emotional, and aesthetic experience, we designed this study with the view of presenting the CP technique to those wishing to generate context-specific materials that engage children and teenagers with biodiversity . In so dRQ1: Can the CP technique instigate productive questions regarding the surrounding natural environment on behalf of participants?in and with nature?RQ2: Can CP provide an outdoor learning experience that taps into the value of sensory, emotional, and aesthetic experiences RQ3: Can CP promote participants’ engagement and satisfaction with the activities and with the exploration of nature-rich environments?On the 1st and 7th of December 2019, as part of the activity ‘Rediscovering Nature’, we performed two visits to the trail of Mistérios Negros, located in the Nature Park of Terceira.Cryptomeria japonica), introduced by humans [Hedychium gardnerianum) due to their proliferation rate, high abundance cover and negative impacts.We chose a trail in a protected area  that would expose participants to native, endemic, and exotic species . Native y humans . Some ofWe recruited participants, ages 10 to 18, through the help of two local Scouts groups. Participants and their caretakers accessed information and consent forms via their respective Scouts group. Children and teenagers willingly participated in the study, and we informed them that we might request their participation during a post-trail activity. We also reported to them that they could withdraw their participation at any time. For this study, we obtained ethical approval from the Research Ethics Committee of the University of the Azores.A total of 36 participants  participated in visits to the trail, coming from both councils of Terceira Island, Angra do Heroísmo (Santa Bárbara) and Praia da Vitória (Lajes). Considering the islands’ small area (400 km2), all residents are relatively close to the Natural Park, less than a half an hour ride, although there is no public transportation for these locations. Of all participants, 23 Scouts were between the ages of 10 and 14, and 13 were between 15 and 18. Most participants (n = 33) joined the reflective post-trail activity performed at the university a month and a half later.Regarding participants’ exposure to local biodiversity, we note that teachers working with the Azorean primary and middle school curricula can adapt content and materials to the region’s socioeconomic and environmental context . This flIn this study, we first developed a CP toolkit with the Azorean natural forest ecosystems in mind. We then implemented the toolkit during a trail walk with local children and teenagers. We collected data based on questions formulated by the participants and their final impressions from the trail walk. Later, we conducted a post-trail activity in a classroom setting, where we asked participants about their willingness to return to the trail and why.For its construction, we adopted the six categories commonly used applying the CP technique, as documented by Thoring et al. : (1) "doEfficacy in promoting productive questioning for the future development of EEM (RQ1).Effectiveness in promoting sensory, emotional, and aesthetic experiences in nature (RQ2).Potential to positively engage youth with nature-rich environments (RQ3).As detailed in Secondly, we assessed the efficacy of the technique in promoting sensory, emotional, and aesthetic experiences in nature-rich environments by looking at the frequency with which participants, in the final free, open-ended activity, focused their responses on qualities experienced during the trail walk .Finally, we assessed participants’ engagement with a nature-rich environment by analyzing: the observations registered by adult volunteers during the trail walk; the participants’ evaluative judgments expressed in the final free, open-ended activity ; and participants’ willingness and motivation to return to the trail.Figs We divided the first two kilometers of the trail into 12 demarcated stations and added beginning (Station 1) and endpoints (Station 14) to provide an opportunity to understand participants’ impressions of the surrounding environment. At the beginning of the trail, participants were invited to perform a free-word association exercise in response to a given sentence; at the end, participants were asked to briefly close their eyes for a moment and then write down any words that came to mind . In lineAs shown in In line with the inquiry-based educational literature that emphasizes the positive benefits of devising support strategies for question-based learning , we desiHedychium gardnerianum), an invasive species that poses a significant threat to biodiversity in the Azores [While we employed an IBL strategy in the design of question tokens, the toolkit was designed to explicitly evoke sensory experiences by inviting participants to attune their senses to nature through sensation such as touch, feel, squeeze, smell, and hear, as indicated in Activities 2, 3, 8, and 11 . With ree Azores .To summarize, in We performed two visits on the 1st and 7th of December 2019. In the first visit, we welcomed 15 participants (eight between the ages of 10–14 and seven between 15–18). In the second visit, we welcomed 21 participants (15 between the ages of 10–14 and six between 15–18). Due to the length of the trail, the level of difficulty, and the time needed to cover the terrain, younger participants (ages 10–14) visited Stations 2 to 7, the older participants (ages 15–18) visited Stations 8 to 13, this in addition to Station 1 and 14 which both groups visited.Before initiating the trail activity, we asked participants to perform a free-word association exercise in response to the sentence (here translated from Portuguese): "Write freely the words that come to mind when you think of ’natural forests of the Azores‴. With this exercise, we wanted to gauge participants’ impressions of the local natural environment we were about to visit. At the end of the trail, we asked participants to close their eyes for a moment and then write down any words that came to mind.With the support of 10 adult volunteers during both visits, we welcomed participants to the starting point where they established themselves into groups of two or three members. We followed with an explanation of the activity. Group members were encouraged to adopt different roles, such as note-taking, transportation of the canvas bag, transport, and mobile phone use, while it was explained that the posing of question prompts was a group task. At this point, we also presented the canvas bag and its contents.It is important to note that adult volunteers contributed alongside the research team in the form of structured notetaking, involving: (1) sensory impressions, (2) observations regarding interaction and communication, and (3) any additional notes. For the sensory impressions category, we looked for data relating to participants’ mood in each station . For the interaction and communication category, our focus was on the interaction between participants in each group. For example, how did participants delegate tasks and share resources? Was there a good synergy between group members? Was anyone left out? For the additional notes category, we asked volunteers and participating team members to record extracts of sentences or conversations uttered between participants.We transcribed data collected during the trail walk and the post-trail activity using Microsoft Excel software to synthesize and categorize data. A thematic analysis of the data was necessary to define the categories of the indicators . Themes were developed a posteriori, without a pre-existing categorization system. This inductive process included all responses collected over the two visits. We expand on the operationalizations of the different categories in the Findings section.To ensure the interpretation and validity of the results, we used several quality processes, namely:Triangulation: two researchers with different academic backgrounds (one with an MS in Biology and one with a PhD in Psychology) performed the analysis.Intercoder reliability test: a Cohen’s kappa coefficient was calculated in a random sample of 35 of the 237 units of meaning, categorized independently by four judges, and re-calculated after redefining categories to solve disagreements and until we reached a near-perfect agreement result (0.98).Iterative data processing: performed to include all relevant information from participants as well as accommodate new refinements of the classification system for the only category that resulted in an initial poor agreement kappa coefficient .To analyze participants’ degree of satisfaction with the trail walk, we also submitted their responses to the final free, open-ended activity to a similarity analysis of the semantic field. After translating the responses from Portuguese to English, we used the freeware program IRAMUTEQ  that proCan the CP technique instigate productive questions regarding the surrounding natural environment on behalf of participants?By the end of the trail walk, we retrieved 168 questions. Four were incomplete, making it impossible to categorize them. From the remaining 164 valid questions, 137 were found to be sufficiently distinctive, indicating a high level of diversity of questions posed. Following an inductive process, we analyzed questions with respect to their related activity and the supervisor’s comments concerning participants performance in each activity. As a result, we were able to rank valid questions as either ’productive’ or ’strategic’.The ’strategic’ category corresponds to questions triggered by the invitation to complete the study’s tasks or by design of the activity itself. In these cases, participants overlooked the opportunity to obtain new information through questioning by deriving questions based on those presented on the tokens. For example, we received the question "Describe the sounds of nature", which is similar to the question posed to participants during the activity at Station 11: "(…) make a list of the sounds you identified". Note that our use of the term ’strategic’ differs from King’s , who in In the ’productive’ category, we included internally oriented questions triggered by the desire to question and obtain knowledge or confirm previous knowledge related to the external context. This category comprises externally oriented questions triggered by participants’ exploration of the external context, such as the natural context, or the previously performed activities. In either context, ’productive’ questions are those that refer to a genuine interest in knowing more.An example of an internally oriented question is "Why is biodiversity important?" which does not directly accrue from the activity performed in Station 3 ("The smells of the forest"). Also, a comment from the station supervisor suggests that the group who produced this question was suitably motivated ("They are doing the work with certain dedication."). An example of an externally oriented question is ‘‘How could the water from the lagoon be used?”, which, although it does not directly relate to the corresponding activity, relates to the natural context of Station 10 , where it was produced. Note that this station was also near the lagoon that can be found in Mistérios Negros .From the 164 valid questions, nine-tenths  were found to be ‘productive’, while nearly all the questions were externally oriented , focusing on the immediate natural context or performed activities. Only 11% (n = 18) were categorized as ‘strategic’ questions . The colOverall, differences between age groups are significant (χ2(1) = 5.839; p = .016), and the number of ’strategic’ questions is much smaller in the older group. Nonetheless, both groups generated a higher number of ’productive’ questions; 84% (78 out of 93) for the younger participants, 96% (68 out of 71) for the older participants. Additionally, we crossed the questions posed with their corresponding activities to determine whether some activities prompted more ’strategic’ questions than others. For the younger participants , the disActivities 3 ("The smells of the forest") and 6 ("Who lives here?") raised a more significant number of ’strategic’ questions when compared to other activities. For the older group , ‘strateWhen distributing questions by question tokens used in their production , we obseTokens related to comparison, e.g., "How are… and… similar" and "How do… and… differ", as well as those associated with the utility of something, e.g., "How could… be used to…?" were used less often. We further observed that only 7% of the questions (n = 11) were constructed without the use of tokens–e.g., "Do plants choose their natural environment?". Thus, content-free question tokens appear to benefit participants’ questioning. Additionally, the distribution of ’productive’ and ’strategic’ questions is significantly different between question tokens (χ2(9) = 51,828; p < .001). The token "Describe… in your own words" promoted more ’strategic’ questions than the other tokens. Three-quarters (12 in 16) of the questions produced with this token were found to be ’strategic’, possibly owing to the descriptive nature of the token that encourages the production of questions by adding just one noun.in and with nature?Can CP provide an outdoor learning experience that taps into the value of sensory, emotional, and aesthetic experiences From the last free, open-ended activity, we retrieved 34 responses (two participants did not respond), which were then translated from Portuguese to English. From the analysis, we observed that 22 participants responded with free-word associations  and 12 with phrases .We categorized all 34 responses according to their "dominant theme", or the main idea expressed when attributing meaning to the experience. Responses highlight participants’ perspectives that arise from features of the activity, with our interpretation of the "dominant theme" closely following Litfin’s  definitiThe “experience of the natural context”.The “assessment of the activities”.The “description of the immediate context”.Responses categorized under (1) ‘‘experience of the natural context” are those in which the answer, or the dominant tone, of the answer emphasizes qualities of the lived experience in the natural context. Even when it includes descriptive elements, the description is usually associated with the perception of the natural environment through the senses or feelings. For example, ‘‘nature, joy, happiness, beauty”  emphasizResponses categorized under (2) ‘‘assessment of the activities” correspond to answers whereby the answer itself, or the dominant tone, focuses on assessing the trail walk and/or the activities that it comprises. E.g., the participant response ‘‘This activity was interesting because I was able to pay attention to details and ask questions that I had never thought of before”.in and with nature, as recollected during the open-ended activity. Less than one-third  focused their responses on the assessment of the activity itself, while one-fifth  described the immediate context (Those categorized under (3) ‘‘description of the immediate context” correspond to responses which enumerate, in an emotionally neutral way, elements of the participant’s direct experience. For example, ‘‘nature, birds, river, pond, scouting” describes only the features present in the external context . Results also show that almost half of the participants  emphasized experiences  context .Differences between age groups show that the assessment of the activity is more frequent in the older group, while younger participants represent the totality of descriptions of the immediate context, and the generality of the experiences relating to the natural context  suggests that the CP offered an outdoor learning opportunity that builds on participants’ experience of nature. Moreover, we believe that experiences lived during the study might have been recognized by more participants if we had asked them to do so directly. Nonetheless, the indirect way in which responses were elicited gave us greater access to the relevance and authenticity of their experience. The open-ended nature of the questioning further allowed for assessing participants’ direct awareness of their experience.Were the participants engaged with the CP? How did the CP promote their motivation and satisfaction in natural-rich environments?All participants completed the proposed activity tasks, suggesting that they engaged with the activities, despite a certain awkwardness displayed by a few participants towards some of the more hands-on activities (as communicated in the notes from adult volunteers), e.g., when asked to touch wet vegetation.Furthermore, comments collected by volunteers in each station indicate that participants showed "focus and commitment", a positive attitude ("good synergy between them") and were responsive to all tasks proposed at each station. The volunteers also observed that participant groups engaged in conversations focusing on performed activities or the natural context. These seem conducive to questioning, as demonstrated in the extracts of dialogues between participants collected by volunteers: "What is peat?", "It’s an animal?", "Who lives here?", "Why do invasive species harm the forest?", "What can we use moss for?". Notes were also indicative of curiosity among some groups regarding natural phenomena ("Why do invasive species harm the forest?"). Additionally, the notes taken by adult volunteers suggest that younger participants showed "satisfaction and a positive attitude", while indicating that those participants were also more "enthusiastic" than the older ones.really liked the project and the landscapes were very beautiful.”). ‘Positive’ comments involved favorable explicit evaluations , or favorable implicit evaluations .At the end of the trail, in the activity proposed in Station 14, participants’ comments and evaluative judgments regarding their experience on the trail reveal a high degree of satisfaction with the activities and their exploration of the natural context. After an inductive process, we categorized the degree of satisfaction as neutral, positive, or very positive . ‘NeutraResults show that responses did not reflect negative evaluative judgments. More than two-thirds of participants  expressed ‘positive’ or ‘very positive’ evaluative judgments, with the older ones always positively appreciating the trail walk  of motivations focus on the trail itself, 20% (n = 7) on the natural context, and 9% (n = 3) on the activities performed during the trail walk, with no significant differences found between age groups  and those relating to the utility of something ("How could… be used to…?") were less used. It is possible that such tokens limit the expressive power of participant-generated questions, compared to the other tokens. For example, the question "How do trees and mosses differ?" was created only by adding two nouns to the question. The token that asks participants to identify a concrete purpose or utility for something resulted in questions such as "How could the water from the lagoon be used?". In this instance, participants did not use the complete token, possibly considering that it was not easy to find a concrete purpose when contemplating the utility of the water in the lagoon. Although we have no evidence to suggest how our participants might favor ’doing’ versus ’knowing’ , we speculate that a question token that requires focus on the utility of something can pose challenges.Results gathered in response to RQ1 shed light on the application of IBL in the design of learning materials for EE. Drawing on the broader literature, as in the work of Lazonder and Harmsen , questioConsidering "the impact of culture on inquiry" , designiIn response to RQ2, we can infer that the CP toolkit effectively promoted experiences grounded in sensory, emotional, and aesthetic qualities in a nature-rich environment. Nonetheless, and considering the data collected in response to RQ1, we noticed some discrepancies. Activity 3, for example, raised more strategic questions than productive ones. We believe that this could express the general prevalence given to sight or sound in Western culture. Furthermore, and aligning with the broader literature, e.g., Stevenson et al. , we specWe further noticed that the experience of the trail, as guided by the CP toolkit designed for this study, was very present in the participants’ awareness of the trail walk. However, there are age differences, with older participants tending to make more assessments, while younger people focused more on experiencing and describing the external context. We believe this reflects differences in child and teen learning, which are affected by changes in behavior, cognition, and the brain , as wellFinally, and in response to RQ3, the CP toolkit designed for this study promoted participants’ positive experiences of a nature-rich environment, such as the trail of Mistérios Negros. We believe that the design of the various station activities influenced this outcome. They provided information regarding different elements, such as the native, endemic, and exotic species that characterize the trail, along with geographical information, such as the lagoon. Station activities also engaged participants through their senses, providing a springboard for imaginative exploration, as observed in response to the activities of Stations 11 and 13.We also understand that working with a protected area with pristine nature was paramount to participants’ experience of the CP toolkit. Exposure to nature positively impacts our mood and sense of general well-being . While tThe CP literature, concepts drawn from IBL and the value of sensory, emotional, and aesthetic experiences in EE were paramount in the design of an appropriate CP for FG and the present study. Nonetheless, we derive certain key practical implications that support the creation and application of CP in the context of EE, as follows:Create probes that balance information regarding nature-rich environments with opportunities to experience nature through imagination and the senses directly.Ground the design of CP in relevant pedagogical framework, such as inquiry-based learning.If using IBL, consider participants’ experience with questioning and inquiry, and devise probes that support prior experience or lack thereof.Provide genuine opportunities for feedback without saturating participants with requests for information through questioning. In this instance, subtle integration is key.Our study was limited in time and participant numbers. We recruited participants through two local Scout groups, thereby assembling a convenience sample even though participant demographics align with the project’s target audience: local school-aged children and teenagers (ages 10 to 18). In addition, when conducting the study, we did not have prior information regarding participants’ knowledge of the local environment. Despite this, as we walked the trail, notes from the research team suggest that even though exposure to the outdoors is crucial to Scouts’ activities, participants appeared to know little about local and native species and had little experience on nature trails. For most, this was their first visit to the trail.Furthermore, we were aware that word association exercises could pose challenges. The qualitative nature of the activity demands a capacity on the part of researchers to interpret participants’ subjective responses. However, we were also aware that such exercise could elicit more deep-seated and personal interpretations of the activity which are valuable for future design . We wereContinuous exposure to the CP technique in the context of EE is essential to draw more robust conclusions regarding all three research questions. Despite this, we can say in hindsight that a question-based strategy provided research participants with a rich experience during data-gathering. We retrieved a high number of questions , suggesting that questioning could be a rich topic for the discussion of CP within the context of EE.The purpose of this paper was to present a research method to EE; in doing so, we experimented with the CP technique, a qualitative research strategy that holds potential for involving a local community of participants. Our goal was to understand what interests’ children and teenagers as they interact with nature-rich environments. Akin to the early “community content-based instruction” model , we intein and with nature.We were keen to use the toolkit to invite participants to share their curiosity in interacting with the surrounding natural world with our research team. Preliminary results suggest that a CP toolkit containing question-based activities can afford positive meaningful experiences By situating this study’s findings within EE research, we hope that the unique characteristics of CP can be understood and utilized by the EE community. Although these strategies are born out of design-related inquiry and qualitative research more broadly, our analysis shows that we can apply CP to create EEM. However, further research is needed to support our argument and better implement the CP technique within the range of contexts that characterize contemporary EE.S1 Dataset(XLSX)Click here for additional data file.S1 Fig(TIF)Click here for additional data file.


The Immune Epitope Database (IEDB) freely provides experimental data regarding immune epitopes to the scientific public. The main users of the IEDB are immunologists who can easily use our web interface to search for peptidic epitopes via their simple single-letter codes. For example, ‘A’ stands for ‘alanine’. Similarly, users can easily navigate the IEDB’s simplified NCBI taxonomy hierarchy to locate proteins from specific organisms. However, some epitopes are non-peptidic, such as carbohydrates, lipids, chemicals and drugs, and it is more challenging to consistently name them and search upon, making access to their data more problematic for immunologists. Therefore, we set out to improve access to non-peptidic epitope data in the IEDB through the simplification of the non-peptidic hierarchy used in our search interfaces. Here, we present these efforts and their outcomes.Database URL:  http://www.iedb.org/ The Immune Epitope Database (IEDB)  is a freThe Chemical Entities of Biological Interest (ChEBI) , 3, an oThe ChEBI hierarchy is organized by the chemical features of the structures. However, ChEBI contains many more terms than are required by the IEDB, and the application of ChEBI for IEDB purposes required customization to increase usability for curation, query and reporting. First, the ChEBI hierarchy did incorporate some immunological information, but this was inconsistent and not detailed enough to express the immunological information captured in the IEDB. Second, many IEDB non-peptidic structures lacked ChEBI parent structure assignments that immunologists commonly use. Third, the ChEBI tree, while rigorous, is designed primarily for chemists, and as such it was problematic for IEDB curators to navigate or for immunologists using the IEDB to find structures within the tree.Here, we report the generation of an immunologist-friendly view of ChEBI that, while based on the ChEBI tree, was significantly streamlined and simplified specifically for IEDB applications, addressing the needs of immunologists. We generated two complementary trees: one capturing and classifying compounds by chemical structures and, new to the IEDB, a tree capturing the biological, chemical or application roles (if any) of the immunologically relevant substances, which include roles like ‘pesticide’ or ‘hormone’. All compounds contained in the role tree are also contained in the tree classifying compounds by their chemical structures; the role tree provides an alternative pathway to finding chemicals that may be more commonly known by their functions (e.g. finding ‘progesterone’ in the role tree where it is a child of ‘hormone compound’ may be easier than finding it in the structure tree where it is a child of ‘20-oxo-steroid’). Both trees are immediately visible to the user when they select the non-peptidic molecule finder in the ‘Epitope’ filter panel, after they have first selected ‘Search’ on the IEDB home page. The ‘Finder’ button is located immediately adjacent the ‘Non-peptidic’ radio button in the ‘Epitope’ filter panel on the results page. Once selected, the non-peptidic molecule finder opens as a pop-up within the IEDB, allowing users to both click through the tree manually and search via ‘Name’ or ‘Molecule ID’.Throughout the paper, ontological terms are used to describe the tree classification and its attributes. Each term is defined in the glossary below to facilitate comprehension from the reader not familiar with ontological definitions, while at the same time maintaining rigor in the classification terminology. This terminology is commonly used in ontological applications .Class: Each class within an ontology represents a type of thing in the real world, and particular instances belong to these classes. For example, your pet cat, ‘Fremont’, is an instance of the class ‘cat’, which represents the properties of all cats. ‘Cat’ is a subclass of ‘mammal’.Tree: The overall subclass hierarchy of an ontology. It begins with a root class, usually ‘thing’, which represents all ‘things’ in existence. As you move down the levels of the tree, the classes become more specific.Grouping class: A class that is not utilized by curated data in the IEDB but is used to help sort the tree.Children: A class may have children classes using the ‘subclass of’ property. The children adopt all properties of the parent class. For example, an ‘owl’ is a child of ‘bird’ and, therefore, meets all the requirements of being a bird.Descendants: The ‘subclass of’ property is transitive. A class’s child class may also have children, which still maintain all the properties of their upper-level classes. For example, ‘owl’ is a child of ‘bird’ and ‘bird’ is a child of ‘vertebrate’. Therefore, an ‘owl’ adopts all the properties of the ‘vertebrate’ class.Parents: The opposite of ‘children’; a class has a parent class from which it adopts all properties.Ancestors: The opposite of ‘descendants’; a class adopts all properties from all the classes above it (the ancestors).Siblings: Any classes that share a parent class.Branch: A specific subset of the tree based on a class and all of its descendants.Level: Each ‘subclass of’ relationship creates a new level in the tree.Node: Any term at any level in the hierarchy, whether it be a terminal level, without any children, or a parent level term with many children.Disjoint: A class is disjoint from another class when it cannot share any instances. For example, an instance of ‘bacterium’ cannot also be an instance of ‘eukaryote’, so these two classes are disjoint.Roles: ChEBI contains a role hierarchy that begins directly under the root class (‘thing’). These classes represent the types of roles that a chemical entity can have. Roles are not chemical entities, so you cannot find any chemical entities in the ‘role’ branch.Compounds: We have created an alternate method of browsing ChEBI by creating a ‘compound’ hierarchy that mirrors the ‘role’ hierarchy. Chemical entities that have roles can now be found under their ‘role compound’ parent. This is explained in further detail below.Structural classification: The main basis for browsing ChEBI, in which chemical entities are classified by their structures, creating a ‘structural’ hierarchy.A subset of ChEBI was created by extracting only the chemical entities curated by the IEDB and their ancestors, grouping classes such as ‘lipids’. This does not, however, limit any new chemical entities being added to the IEDB in the future. Specifically, if a chemical entity existing in ChEBI but not in the IEDB is reported to be immunogenic, the IEDB tree will be updated accordingly. This was preferred to the alternative of including all ChEBI entities prior to pruning, to reduce clutter from irrelevant, non-populated entries. A formal update process, similar to what occurs for peptidic curation, is in place.Next, the tree was automatically pruned to remove unnecessary intermediate classes, identified as grouping classes not curated in the IEDB, and with only a limited number of other classes as children. For example, to get to ‘alpha-d glucose’ in the original ChEBI tree, it required 18 levels of grouping classes, some labeled in a manner non-intuitive for immunologists . This waFinally, we created ‘other’ classes to simplify manual browsing of the tree. Any bottom-level children without a specific grouping class were placed in these ‘other’ categories. For example, ‘monosaccharide’ has 7 major grouping categories, and monosaccharides that do not fall under these categories have been placed under ‘other monosaccharide’, to ensure that the 14 extra entities do not clutter the class.Six immunologists within the IEDB team, who had not been involved in the development of the new non-peptidic tree, were identified and requested to complete online testing via a Zoom session. The candidates were provided with de-identified web links to both the old and new non-peptidic tree and were requested to manually locate 10 different chemical entities in each tree by clicking through the branches. The automated search function was disabled and users were unable to search the internet for the chemical entities prior to locating them in the trees. Users were recorded via Zoom and timed as they searched for each entity, sharing their screen with the tester to ensure the correct entity was located. The time taken to locate each entity was recorded, and the task was ended if the entity could not be located within 3 minutes.Currently, the IEDB contains ∼3000 non-peptidic structures. Even allowing for a 3- to 4-fold projected growth (10 000) records, it would be theoretically possible to classify these 10 000 records in 4 levels, with each class at each level containing an average of 10 children. This ideal distribution may be difficult to practically achieve, given the distribution of the records and their classification, but, nevertheless, provided a guiding principle for our efforts.The overall simplification achieved by these processes, as described in more detail below, was remarkable. The structure tree resulting from these various modifications is shown in During this process, limiting redundancies was another goal. ChEBI classifies structures based on their chemical determinants. For example, it is intuitive to allow access to a glycolipid from both the lipid and carbohydrate branches. However, strictly speaking, any organic compound in which the hydroxyl functional group (–OH) is bound to a carbon is considered an alcohol, and to classify carbohydrates as alcohols because of the presence of an –OH group is counterintuitive to immunologists. It also leads to a cumbersome tree for browsing and maintaining. Accordingly, we arbitrarily eliminated some of these occurrences based on our judgment of what users would expect.A high-level branch in the ChEBI tree is dedicated to a ‘mixture’, which is defined as ‘a chemical substance composed of multiple molecules, at least two of which are of a different kind’. This was not a very useful category for our purposes. We accordingly streamlined the structure tree; N-acetyl-alpha-D-galactosaminyl-l-serine residue is placed under ‘serine and serine derivative’. Sugar groups under ‘other glycosyl group’ were placed under ‘other glycosyl [prefix]saccharide group’, unless they contain aminyl groups, in which case they were placed under their respective amino sugar groups. Tosyl and dansyl groups were moved to ‘other benzene’, and, additionally, we created a new node for them called ‘sulfonyl group’ under ‘sulfur molecular entity’. The univalent carboacyl group and its 19 children were placed under ‘carboxylic acid’, since ChEBI defines univalent carboacyl group as ‘a group formed by loss of OH from the carboxy group of a carboxylic acid’. However, some of its members were also placed under other groups; for example, ‘amoxicilloyl group’ is under ‘penicilloyl group’.Likewise, the categories ‘groups’ or ‘derivatives’ encompassed records relating to a prominent part of the molecule that would be most logically found as part of the node describing the molecule. All groups were also relocated. Amino sugars were placed under amino [prefix]saccharide group, under their respective amino [prefix]saccharides. We created a nitrophenyl (NIP) group branch under benzene molecular entity to include a trinitrophenyl group, DNP group, o-nitrophenyl group and p-nitrophenyl group. Phenylazo groups and its children were also moved under benzene molecular entity. The penicilloyl group and its children were placed under penicillin, which now also includes the amoxicilloyl, ampicilloyl and benzylpenicilloyl groups. Amino acid derivatives were placed under their respective amino acids, e.g. Three NIP-related compounds that were under the ‘univalent carboacyl group’ were also placed into a new node below the ‘nitrophenyl group’, called ‘nitrophenylacetyl group’. A few new nodes were created to accommodate special situations. For example, a node for the ‘phenylazo group’ was created under ‘nitrogen molecular entity’, and a node for the ‘carboacyl group’ was created under ‘organic molecular entity’.Next, we returned to the high-level organics nodes to eliminate errors and reorganize the structures to more intuitive formats. In reviewing the ‘carbohydrates and derivatives’ branches, we noted the presence of glycopeptides that are mostly peptides with modifications. This reflected a curation mistake and prompted us to remove the glycopeptides from the non-peptidic tree and recurate the papers. Posttranslational modifications of peptides are indeed curated in the IEDB and accessible through the associated peptidic moiety, with the exception of records relating to recognition of amino acids in isolation. We also noted several glycolipids and phospho sugars. These were placed in separate branches on the same level as monosaccharides and their derivatives, disaccharides and their derivatives, etc. Glycolipid groups with varying numbers of sugar moieties were placed in a new group called ‘glycolipid group’ under ‘glycolipid’.Another example of reorganization/simplification related to ‘ethers’: ‘aromatic ethers’ were found both as a branch in ‘aromatic’ and ‘ethers’. We deleted aromatic ethers from ethers and renamed the ‘ethers’ node to ‘non-aromatic ether’. We moved all phosphatidylcholine entities under a single branch below ‘glycerophosphocholine’, removing the level 4 phosphatidylcholine branch under lipids and the level 2 phosphatidylcholine(1+) branch under nitrogen compounds.Significant effort was also devoted to restructuring amino acid nodes. We implemented the following organizational structure, using alanine as an example amino acid: organic amino compound (level 2); amino acid (level 3); alanine and alanine derivatives (level 4). All 20 natural amino acids were kept at the same level. Under the nitrogen compounds, glycopeptides and oligopeptides were both under ‘amino acid derivative’ and ‘carboxamide’. The ‘carboxamide’ affiliation was eliminated. Leukotriene E4 was preserved under amino acids because it contains a cysteine, but was also retained under ‘other fatty acids’, which is a more intuitive placement. Citrulline was placed in ‘other amino acids’.N-acylsphingosine and sphingomyelin, which were under both the carboxamide branch and the sphingolipid branch. We eliminated the redundancies; retained N-acylsphingosine, glycohydroceramide, glycophytoceramide and sphingomyelin under ‘sphingolipid’; retained galactosylceramide, ganglioside, glucosylceramide and oligoglycosylceramide under ‘glycolipid’ and renamed the carboxamide branch ‘non sphingolipid carboxamide’. For the acetamide branch (level 3), ‘other acetamide’ was removed as a sub-branch because acetamide only had eight members. Likewise, ‘other isocyanate’ was removed as a sub-branch for similar reasons. Penicilloic acid was removed as a level 2 branch because it was already found under penicillin. Chondroitin sulfate and heparan sulfate were consolidated into one level 2 branch called ‘nitrogenous sulfate’ because of being sparsely populated.Additional redundancies within the nitrogen branch included galactosylceramide, ganglioside, oligoglycosylceramide, glucosylceramide, glycodihydroceramide, glycophytoceramide, In the early stages of the reorganization, ‘other organic molecular entity’ was created as a placeholder for entities whose placement was not immediately apparent or intuitive. After these revisions, we returned to the ‘other’ categories, trying to place as many as possible under appropriate branches. In this work, we also relied heavily on logical definitions provided in the ChEBI tree. In the end, the ‘other molecular entity’ category only contained 78 structures as compared to 135 when we started working on the placements.We also reviewed the tree for consistency between plural and singular names (pyridines vs pyridine). We followed the recommended convention for ontologies and controlled vocabularies to use lower case, unless the word would be capitalized in the middle of a regular sentence , and to use the singular form. Other names in the tree were inconsistent, between using ‘compounds’, ‘atom or molecule’ and ‘molecular entity’. We settled on using the term ‘molecular entity’, defined as ‘any constitutionally or isotopically distinct atom, molecule, ion, ion pair, radical, radical ion, complex, conformer, etc., identifiable as a separately distinguishable entity’. This also allows the use of the two broadest terms as ‘inorganic molecular entity’ and ‘organic molecular entity’.The tree was once again inspected for unnecessary redundancies, which were eliminated. As a design goal, no level 2 nodes should be repeated, although we did make an exception for glycolipids, which are in both lipids and carbohydrates. ‘Other’ nodes are always listed last among their siblings. Prior to reorganization, the IEDB’s non-peptidic epitope finder was difficult to navigate, and our goal for a finished product was a tree that was both markedly more streamlined and more intuitive for an immunologist to use. Through the iterative simplification process described above, we have created a product that is demonstrably easier to navigate for an end user.The structure tree resulting from the various modifications described above is shown in The non-peptidic tree shown above is measurably simpler than the old tree along two key axes. First, the old tree is much deeper than the new tree, meaning that the new tree has fewer layers that users must navigate as they search for specific non-peptides. Second, the new tree is also ‘broader’ on average, with more source nodes per parent node and fewer parents with just one child. In the old tree, users would often have to ‘dig down’ many levels, where each level consisted of just one new child node. In fact, the median number of source nodes per parent node in the old tree is just 1 and the average is just 3.5. In the new tree, the median number of source nodes per parent node is 3 and the average is also much higher at 10.2.We argue that it is important to strike a balance between the depth and the breadth of a tree. A ‘tree’ that consists of 100 nodes at the first level has a minimal depth of 1, but the user does not benefit from useful grouping nodes. On the other hand, grouping nodes are of little use when the ‘group’ is just a single node. The new tree strikes a better balance than the old tree, by making the average and maximum depths more manageable for users, while also ensuring that grouping nodes are more effective.The next set of activities was aimed at developing the role tree. As mentioned above, the IEDB curates the immunological data associated with each epitope structure, whether peptidic or not. ChEBI contains some information about certain structures being allergens, haptens or otherwise immunogenic by assigning roles to the chemical entities. Much of this work comes from past collaborations with the IEDB, but this was done at static time points and was not well maintained. Thus, this information was not rigorously controlled, neither at the level of vocabulary nor with actual evidence supporting the attribution. This created potential for data inconsistencies and user confusion.Thus, the ‘role’ branch was reviewed and revised to group roles commonly used by immunologists to better suit the needs of the IEDB. For example, many lower-level role categories were simplified. Roles in ChEBI are typically asserted in the pattern ‘structure subClassOf “has role” some role’ . While providing useful information, these anonymous parents, named because they do not reference a distinct entity within the ontology, are not browsable. To make it possible to browse for a chemical entity by its role, we created a ‘chemical entity by role’ class as a sibling to the ‘chemical entity’ class. The role hierarchy duplicates the structure hierarchy as ‘X compound’ , and chemical entities can now be found under their respective role parent. While ChEBI does include many ‘role’ axioms, not all chemical entities have a defined role. This may be because ChEBI has not yet curated them, or it is not within the scope of ChEBI. As discussed above, ChEBI asserts roles using the pattern ‘structure subClassOf “has role” some role’. In order to be a part of the ‘compound’ hierarchy, a structure or one of its ancestor structures must have this axiom. Ideally, each chemical entity curated within the IEDB should be inspected for possible biological roles.We identified those structures missing a defined role by querying the revised ChEBI tree using SPARQL, a query language that can be used on ontologies. First, we only looked at structures that did not have a good structural parent (those that were IEDB-curated and appeared in a ‘other’ grouping class) and found 464 entities. Once this review was complete, we expanded the query to include any chemical entity without a role. This returned 1873 additional entities to which we assigned roles using the process below.All chemicals in ChEBI that did not have a defined non-immunological role were exported into an Excel spreadsheet. Each row consisted of the ChEBI ID, a link to the corresponding epitope on the IEDB, the name of the compound and the ontological ‘parents’ or grouping classes. We then identified one or more roles for each compound using information in the manuscript(s) that chemical appeared in databases like PubChem and/or internet searches. Once these roles had been identified, they were entered in the spreadsheet as short text descriptions.Of the 2333 chemicals from the Excel spreadsheet mentioned above, we found that a total of 1623 compounds were synthesized/studied and curated in the IEDB only because they stimulate, or are recognized by, an immune response. These included, but are not limited to, mammalian or microbial carbohydrates, oligonucleotides and haptens. For these compounds, we captured a NR (No Role) in the short text description to indicate that they were reviewed and that there was no role to be attached to them and eliminated them from the simplified role tree. Clear roles were identified for the remaining 710 chemicals. These roles included antibiotics and other medications, industrial materials, dyes, food additives, pesticides and reagents, among others.Following completion of the review of all compounds with no previous role assignments, we reviewed the short text role descriptions. If suitable roles existed in the revised ChEBI tree mentioned above, the compounds were assigned existing roles. If a suitable role was not identifiable, new roles were added under in the ‘role’ branch in their appropriate category .The role tree resulting from the various modifications described above is shown in Likewise, ChEBI inconsistently curates the organisms from which the various chemical entities are derived. This is due to only capturing such information when it was present in the original source used for the ChEBI entry. For example, if a ChEBI structure was created from a publication describing it as being derived from a specific species of bacteria, that bacterial source will be included in the entry. However, if that same chemical structure is present in several other species of bacteria that were not mentioned by the publication, ChEBI may not include references to the additional bacterial species in their curated record. The IEDB captures taxonomic information in specific fields that are maintained and linked to the NCBI Taxonomy . AccordiIf our process identified errors or missing structural parents or roles for non-peptidic structures, we sent this information to a ChEBI curator, so that these changes could be made to the actual ChEBI entries. This was actually rare, with most of the effort of this project being a simplification and reduction of the utilizing content present in the ChEBI entries. Once changes were available in the ChEBI website, a new version of the ChEBI ontology was retrieved and our tree-pruning routines, as described above, were run upon it. The result is a new immunologically friendly ChEBI hierarchy with a heavily streamlined structure branch geared toward immunologists.As we encounter new non-peptidic structures that are not already present in our streamlined tree, we first search the ChEBI website to determine if the structure already exists in ChEBI. If it does, we add it, along with information on which of the many structural parents is most immunologically relevant to the IEDB via direct addition to our simplified ontology file that the IEDB’s molecule tree utilizes. At this time, we also review any roles assigned to the structure and provide input to the ChEBI team, if necessary. If the structure does not yet exist in ChEBI, we request its addition via a new term request made to a ChEBI curator.In order to remain up to date with changes in the ChEBI content, the weekly build process of the IEDB incorporates a validation step that checks if all ChEBI structures in our data are still valid. If any structure is not valid, we check for a replacement term in ChEBI. If such a term is not found, we manually inspect and correct the structure.In order to determine whether this new tree was, indeed, more useful and intuitive, we then tested our curators and end users, all immunologists who had not been involved in the development of the revised tree. We created a test that required the prospective user to find 20 common non-peptidic structures by manually navigating both the old and the new ChEBI trees; they were asked to find half in the old tree and the other half in the new tree. The test that was conducted is provided in By using the ChEBI resource to depict non-peptidic structures in the IEDB, we were able to start with a great deal of specialized chemical knowledge, which we were able to prune down to meet the needs of our specific immunologist community. The searchability of non-peptidic data was improved in the IEDB by removing extraneous nodes, simplifying the hierarchy of epitope structures, and by making immunologist-relevant roles available as a new search feature. This work clearly decreased the complexity of the non-peptidic search, as seen by the total numbers of nodes and depth present in the new tree. Through user testing, we were also able to demonstrate significant improvements in the access to non-peptidic epitope data in the IEDB. This initial testing was based on internal users, and our future plans include gathering external user feedback to determine further effectiveness or refinement, following the same process that has been utilized to generally refine IEDB query and reporting processes . Based obaab014_SuppClick here for additional data file.