Characterizing cellular diversity at different levels of biological organization and across data modalities is a prerequisite to understanding the function of cell types in the brain. Classification of neurons is also essential to manipulate cell types in controlled ways and to understand their variation and vulnerability in brain disorders. The BRAIN Initiative Cell Census Network (BICCN) is an integrated network of data-generating centers, data archives, and data standards developers, with the goal of systematic multimodal brain cell type profiling and characterization. Emphasis of the BICCN is on the whole mouse brain with demonstration of prototype feasibility for human and nonhuman primate (NHP) brains. Here, we provide a guide to the cellular and spatial approaches employed by the BICCN, and to accessing and using these data and extensive resources, including the BRAIN Cell Data Center (BCDC), which serves to manage and integrate data across the ecosystem. We illustrate the power of the BICCN data ecosystem through vignettes highlighting several BICCN analysis and visualization tools. Finally, we present emerging standards that have been developed or adopted toward Findable, Accessible, Interoperable, and Reusable (FAIR) neuroscience. The combined BICCN ecosystem provides a comprehensive resource for the exploration and analysis of cell types in the brain. Characterizing cellular diversity is necessary to understand the function of different cell types in the brain. This Consensus View provides a guide to the cellular and spatial approaches used in cell type surveys by the BRAIN Initiative Cell Census Network (BICCN), as well as information on accessing and using the BICCN data and its extensive resources. The National Institutes of Health’s Brain Research Through Advancing Innovative Neurotechnologies (BRAIN) Initiative, launched in 2013, is a major effort to accelerate neuroscience research by providing researchers with tools to study and treat human brain disorders through a comprehensive understanding of the human brain , and larBICCN is a collaborative network of centers and laboratories, including data generating centers, data archives, and data standards developers, which generate, map, and share resources to support several overarching goals. These include generating a high-resolution, comprehensive atlas of cell types in the mouse brain based on large-scale single-cell transcriptome and epigenome sequencing, along with systematic characterization of neuronal morphology, a census of the number and location of cells for each type, new genetic tools to experimentally target brain cell types, and a prototype atlas of human brain and NHP cell types in selected regions of the adult and developing human brain. A standard anatomical template for mapping cell types in the mouse brain was also established through completion and validation of a common coordinate framework . BICCN biccn.org) manages this ecosystem, together with data archives to support logistical organization, data integration, and development of common data standards as well as central maintenance to sustain, compare, and reanalyze data. A major success of the BICCN has been to embrace an operating principle that data should be released quarterly, prepublication, and freely shared under CC-BY-4.0 license unless human protection restrictions apply. In this way, the BICCN Data Ecosystem represents one of the largest resources for single-cell data of the brain and any organ.Each BICCN project has contributed publicly accessible data to multimodal classification of cell types based on transcriptomic, epigenetic, proteomic, morphological connectivity, anatomic distribution, and physiological signatures of cells for further study. To date, the BRAIN Initiative data archives store petabytes of omics, imaging, and neurophysiology datasets generated using over 40 cell profiling techniques and 97 published protocols . The BICCN BRAIN Cell Data Center  . This prBICCN reflects the increasingly collaborative nature of modern neuroscience and has accomplished the deepest coordinated characterization of cell types in any organ to date. Consortia such as the Human Cell Atlas  and HumUnderstanding reproducible features of brain cells is a prerequisite to characterize cell types and to understand their function in the brain, to manipulate them in controlled ways, and to determine variability in brain disorders. Neurons can be distinguished by differential expression of gene classes such as neurotransmitters and neuropeptides, electrophysiological firing patterns, morphology, and by their connectivity, and these modalities form a natural basis for classification. Properties of glial cells, vascular cells, and immune cells in the nervous system are also essential to understand brain function in health and disease . MoreoveThere is general agreement that types should be defined by invariant and generally intrinsic properties and that this classification can provide a good starting point for a census . There aA full characterization of cell types for a given brain region will consist of an enumeration of distinct types characterized across different biological features, including the distributions of their molecular profiles , developmental history, morphology, functionality , and spatial location mapped to a CCF or atlas of the brain. The result of profiling in this way is the development of a taxonomy of different types derived from multiple modality data and their congruences . PresentTo progress toward this goal, BICCN studies used a wide array of approaches Figs , broadlybiccn.org).The cell type profiling techniques developed and used by the more than 30 BICCN projects are presented in The importance of having structure in data grows with increasing annotation and its association with existing knowledge. The hierarchical organization of information is an active area of bioinformatics ,31. AmonData Levels are more than a labeling system and provide an entry point for users of the BICCN data corpus and use–case-directed identification of datasets of particular interest. At project award, each BICCN investigator specified levels of data that their project would generate and BICCN working groups collectively reconcile these definitions by each modality such as 10×-snRNA seq, MERFISH, or electrophysiology to achieve modality specific definitions across groups . While a. CCFv3 is based on a 3D 10-μm isotropic, highly detailed population average of 1,675 mouse brains using 2-photon imaging , which  imaging  and cons imaging . A recen imaging . The CCFHuman and NHP atlases are similarly necessary for structure identification and data mapping . HoweverRRID:SCR_019264)), developed in collaboration with Kitware (www.kitware.com). See Mapping brain data to reference spaces is challenging and uses a range of manual and automated methods of image registration (see BICCN image processing pipelines). Given the challenge of determining anatomical context for any reference atlas even within the mouse, precise image registration requires the whole brain image series (or a reasonable fraction of the brain) to be present with sufficient distinctive anatomical landmarks. Omics data often may not have detailed structural localization and can be positioned within a CCF through coordinate based, visual, or ontological tagging. Human data are typically of this type, where anatomic ontology is known and localized using annotated atlas plates and MNI space from the MRI reference brain volume , a scalable and secure platform codeveloped by the Broad Institute, Microsoft, and Verily for biomedical researchers to share data and run analysis tools such as omics processing pipelines  is an entry point for BICCN resources and provides detailed investigator profiles, consortium news, data access, tools, standards documentation, policies, and overview of scientific progress (BICCN.org (https://biccn.org/data). The catalog organizes datasets by projects, each with one or more associated datasets that may be stored in a single archive or distributed across multiple archives (see Data archives for the BICCN). Users can browse the catalog or use a flexible search function , while optimizing storage costs to store and process data, enabling reproducible data practices, and effectively managing interchange between data producers and computational analysts. The archives serve as active repositories with corresponding.https://nemoarchive.org), Brain Imaging Library , Brain Observatory Storage Service and Database , and Distributed Archives for Neurophysiology Data Integration   that are central to BICCN-related data types: Neuroscience Multi-Omic Data Archive  . The arcive.org) .https://nemoarchive.org) stores and disseminates omics data from the BRAIN Initiative and related brain research projects. NeMO stores both transcriptomic and epigenomic data, including transcription factor binding sites and other regulatory elements, histone modification profiles and chromatin accessibility, levels of cytosine modification, and genomic regions associated with brain abnormalities and disease. Data are organized by projects and within each project further organized by laboratory where data were generated, grant, organism, and the assay type. NeMO is consistent with the principles advanced by the NIH Strategic Plan for Data Science [The NeMO archive  and analyzed results including counts and cluster information (Data Levels 1 and 3), and metadata are submitted to BCDC. Sequence-level data for human samples submitted with access restrictions are made available through an approval process in conjunction with the NIMH Data Archive and NeMO archive. Data submissions rely on the fast transfer technology Aspera (https://www.nemoarchive.org/resources/aspera/). To upload data to NeMO, a user obtains credentials.Data archived at NeMO include raw sequence files as well as derived intermediate files such as BAM files (https://www.brainimagelibrary.org) provides a persistent centralized repository for brain microscopy data and supports dataset deposition, integration into a searchable web-accessible system, redistribution, and analysis tools. It allows researchers to process datasets in-place and to share restricted and prerelease datasets. BIL includes whole-brain microscopy image datasets and their accompanying higher-level derived data such as neuron morphologies, targeted microscope-enabled experiments including connectivity between cells and spatial transcriptomics, and other historical collections of value to the community. In addition to the BICCN, BIL accepts all microscopy data relevant to the BRAIN Initiative, including data from primates, and most mammals and model organisms. BIL accepts both raw and processed data, and Data Levels 2 and 3, such as neuron tracings, which can be linked to lower-level data sources. While BIL does not limit the amount of data deposited per dataset or investigator, users planning to deposit more than 50 TB of data in a single year should contact in advance to discuss data deposition plans. Data contributed to BIL following the Standard metadata for 3D microscopy schema [The BIL  resources for extensive data exploration. The Analysis Ecosystem virtual machine (VM) system has a remote desktop environment to run applications such as Fiji  and Vaa3https://dandiarchive.org) is a web platform for scientists to share, collaborate, and process data from cellular neurophysiology experiments. DANDI works with BICCN and other BRAIN Initiative groups to curate data using community data standards such as Neurodata without Borders  and Brars  and to DANDI web application allows users to browse and search for Dandisets, create an account to register a new Dandiset or gain access to the Dandi Hub analysis platform, add collaborators to a Dandiset, and retrieve an API key to perform data upload. DANDI has enabled streaming access to parts of data using a combination of cloud technologies and storage formats, allowing for more scalable analysis software and visualization technologies. DANDI exposes all data as versioned DataLad datasets [DANDI datasets are referred to as Dandisets and include the dataset and file metadata. Supplied per-file metadata includes instrument, species, sample, subject, and other experimental details. Each Dandiset is organized in a structured manner to help users and software tools interact with it and has a unique persistent identifier that can be used for citation. The datasets , allowinhttps://broadinstitute.github.io/warp/docs/get-started). Detailed documentation and user guides are available through www.biccn.org. Workflow Description Language (WDL) Analysis Research Pipelines (WARP WDL Code) repository contains a collection of cloud-optimized pipelines.The BICCN ecosystem includes production-level, cloud-native data processing pipelines, developed by the Broad Institute’s Data Sciences Platform (DSP) in collaboration with BCDC. While BICCN investigators and other users often process their own omics datasets, standardized pipelines are used to supplement and integrate original analyses with uniformly processed datasets. The pipelines leverage consistent standard file schema and types as well as standardized quality control metrics and metadata. The established cloud-native pipelines replicate processing used by several consortium groups including computational pipelines for processing single-cell/nucleus 10× v2/3, sc/sn full transcript, sn-ATAC-seq, and snmC-seq sequencing data. Each of these pipelines was developed in collaboration with a sponsoring BICCN group and captures their expertise in data processing (https://dockstore.org) for computational biologists, and Terra . Use of a modern WDL separates the code performing scientific tasks from code orchestrating the pipeline on infrastructure and encouraging interoperability for reproducible science (see Section Standards and the BICCN: Towards FAIR Neuroscience.)The Broad Institute Data Sciences Platform resources are actively used by other individuals and consortia, and the approach to the development of molecular pipelines for the BICCN is inspired by FAIR principles . This inhttp://brainarchitecture.org) combines multimodal imaging datasets such as ex vivo radiology and histology in the same animal/subject. The GDM approach overcomes challenges in registering tissue processing procedures such as extraction and fixation that cause brain tissue deformation , and which algorithm is most effective for the image data at hand. Two image processing platforms were developed or extended through the BICCN data ecosystem. (1) Generative Diffeomorphic Mapping (GDM) for image registration and atlas mapping from the Brain Architecture Portal . Some of these resources are described below, with an emphasis on those that facilitate integrative analysis.The BICCN has developed many tools and applications to work with BICCN data. An inventory of these tools describing their application to single-cell analysis is provided under the “Tools and analysis” tab of the BICCN portal RRID:SCR_022793) is an interactive application that aggregates multimodal BICCN data from the primary motor cortex (MOp) atlas at the level of individual cell types in mouse, human, and marmoset. The CTKE integrates the work of many BICCN laboratories and presents aggregate knowledge about cell types in the form of data visualizations and text summaries. Drawing inspiration from Gene Cards , informrds.org;  linking rds.org; , and hisBy leveraging BICCN’s cross-modality mapping of nontranscriptomic data to expression-based taxonomies, CTKE provides rich phenotypic information about cell types and enables its systematic exploration. Cell Type Knowledge Cards for each of the 3 taxonomies are accessible from an interactive sunburst plot . Each caRorb is highly expressed in a certain population of cells and want more information about what type of cells they might be. Searching “Rorb.”CTKE also helps researchers to annotate and interpret their own data. For example, CTKE includes links to Azimuth , a web aRorb more highly than other MOp types  is a web-based suite of data visualization and analysis tools for single-cell data analysis. The portal allows users to explore single-cell, single-nucleus, and spatial transcriptomic and epigenetic profiling data, with flexible plotting tools allowing side-by-side comparisons of any data type. The portal is prepopulated with thematically organized datasets reflecting projects across BICCN. Users can upload their own data for private or public use, utilize curated datasets from other users, select a dataset from the NeMO Archive, or benefit from data collections hosted from peer-reviewed publications. NeMO Analytics simplifies access to BICCN data and provides nonprogrammers with a suite of analytical tools for data exploration, including cell cluster visualization based on expression/cell type, cell cluster comparison, identification of marker genes across datasets, plotting multigene analyses , and note taking  has systematically produced and collected connectivity data for over 10,000 neural pathways in >4,000 experimental cases utilizing a variety of multifluorescent pathway tracing techniques that included double coinjections, triple anterograde and quadruple retrograde tracing, Cre-dependent double AAV anterograde tracing, and rabies viral-based Cre-dependent retrograde methods [www.MouseConnectome.org/Corticalmap). Multiple retrograde tracer injections into different cortical  and subcortical areas  simultaneously reveal multiple cell types, namely, intratelencephalic (IT), pyramidal tract (PT), and corticothalamic projecting (CT) neurons .The Mouse Connectome Project  on the Brain Architecture web portal. Datasets are divided into species and experiment-type specific pages, accessible from the front landing page. Users can filter mouse cell distribution datasets via free text search of metadata for keywords and mouse projection and connectivity datasets via injection region or tracer. A high-resolution viewer capability to display overlays of regional compartments, points indicating cell bodies post-cell detection, and skeletonization on 2D sections of atlas mapped brains. The viewer can display data at multiple resolutions, with zoom to super-resolution capability, beyond the native in-plane 0.46-μm resolution of the images. All software employed for image analytics pipeline, including registration and atlas mapping, cell detection, and process detection and skeletonization via are available both in interactive versions on the Brain Architecture web portal, and for download (of both source code and documentation) on GitHub and Bitbucket repositories. There are additionally cross-linkages with the Broad Institute Single-Cell Portal (https://singlecell.broadinstitute.org/single_cell).A collection of high-resolution 2D images from BICCN collaborators are available through the Brain Architecture Project  and Brain Cell Methylation Viewer (RRID:SCR_020954), interactive visualization tools for functional genomics data; Brainome (RRID:SCR_018162), a genome browser to visualize the cell type–specific transcriptomes and epigenomes of cell types from the mouse MOp; Catlas (RRID:SCR_018690), which provides maps of accessible chromatin in the adult mouse isocortex, olfactory bulb, hippocampus, and cerebral nuclei; Cytosplore Viewer (RRID:SCR_018330) is a stand-alone application (Windows and MacOS) for interactive visual exploration of multispecies and cross-omics single-cell data in several BICCN data resources; and MetaNeighbor [RRID:SCR_014816).Numerous other resources have been key in analysis of BICCN publication datasets, described in Neighbor ,61, a meNeuroMorpho.Org (RRID:SCR_002145) and Hippocampome.org (RRID:SCR_009023), have helped bridge seminal literature information and data with new BICCN-generated data. NeuroMorpho.Org [Hippocampome.org [Tools and Analysis - Brain Cell Data Center (BCDC).Several open-access neuroinformatics resources launched prior to BICCN efforts  but wererpho.Org  providespome.org  is a knoTo be reused and shared efficiently, accessible data need to be described in standard ways, and the development and adoption of standards is thus essential to advancing rigorous science and efficient collaboration . An incrBICCN Standards, Best Practices, and Recommendations have been implemented across BCDC and the BRAIN data archives including metadata and file formats, common processing pipelines, spatial and semantic standards, and identifier systems. BICCN Working Groups focused on harmonizing protocols, data formats, and metadata for transcriptomic, physiological, and anatomical data types. The BCDC coordinated the formation of working groups of consortium members that considered what standards and best practices were necessary for new experimental technologies for which standards were not yet available, including developing QC criteria for a given modality. The BCDC was also responsible for developing strategies to harmonize common metadata across the archives, including submissions checklists, collections metadata, and basic descriptive information for specimens. The Metadata and Infrastructure Working Group  and BICCN Standards, Best Practices, and Recommendations - Brain Cell Data Center (BCDC).Additional standards were adopted over the course of the project as they became available, e.g., the Essential Metadata for 3D Microscopy standard developed with support from the BRAIN Initiative  was receThe BCDC, in partnership with the archives that house the data, ensures that all BICCN data are FAIR according to the principles set out in . The BICBICCN datasets receive a DOI or an equivalent persistent identifier from the archives;The BCDC and archives coordinate on standard metadata to accompany all datasets;Archives implement dataset landing pages for machine-readable rich metadata about the datasets and access.BICCN datasets are versioned;Full-citation metadata is supplied to support data citation;Investigators encouraged to link datasets to detailed experimental protocols deposited in at BICCN group at Protocols.io.Archives are implementing community data standards, including those developed through the US BRAIN Initiative;Archives have implemented common file format and metadata requirements for specific data types;biccn.org;Standards in use in BICCN are documented at Several archives make use of standard identifier schemes for entities linked to the data such as ORCIDS for authors and RRIDs for organisms, antibodies, cell lines, and tools.BICCN has developed ontologies and controlled vocabularies to annotate data and map metadata such as the Brain Data Standards Ontology;Vocabularies are all maintained in GitHub repositories as described on the BICCN standards page.Checklists for standard metadata for experimental types such as Patch-seq and for describing specimens,Contact person identified to answer questions about the data, and code that can be used with the data.All data that do not involve protected health information are made available under a CC-BY 4.0 attribution license;BICCN requires that those using the data follow formal citation principles for citing the data;Archives are making citations available per dataset to assist in proper citation.RRID:SCR_022797), and other single-cell transcriptomics data providers to deepen annotations to use terms from BDSO. A major application of BDSO is to support organization, navigation, and searching of data in the CTKE. Knowledge graphs and APIs were developed for the CTKE , making the reuse, search, and navigation of the BDSO openly accessible. The latest release of the ontology is hosted at BDSO (An important component of cell type classification is a rigorous and precise ontology and nomenclature. The Brain Data Standards Ontology (BDSO)  is an onBICCN Working Groups are presently engaged in continuing collaborative projects (BICCN 2.0) integrating and interpreting new and existing data. In addition to fulfilling the goal of integrating transcriptomic and epigenomic data across the entire mouse central nervous system, these groups are developing methods to identify cell type–specific enhancers that can drive systemic delivery of reporter genes to select subclasses or types of brain cells in mice and primates, producing molecularly annotated wiring diagrams of the mammalian brain, and now beginning work on developing comprehensive human and NHP atlases, through the recently launched BICAN. Additional work is in measuring proteomic signatures of brain cells and further developing integration methods and infrastructure for future atlases.Advances in the development of laboratory techniques and analysis methods for single-cell data in the mammalian brain has made feasible a characterization of its fundamental cell types. Beginning with 10 pilot studies  in develWhile significant progress has been made in most aspects of the original BICCN infrastructure vision, much work remains and is continuing with new BICAN activities. The resources developed through the BICCN have been the result of active collaborations between data generators, analysts, informaticians, and software developers, and the ultimately desired data ecosystem will support data collection, quantification, and a mapping framework for managing data and information across diverse repositories. This ecoystem should maintain consistent data description standards that describe and facilitate best FAIR practices for community use of multimodal single-cell data and its content. From early in the consortium’s activities, requirements for FAIR data management were identified; however, the goal of building a foundational community resource for housing single-cell centered data content in the brain is still a work in progress.An important component of full data integration is the spatial mapping of data enabling users to search by spatial location for data of interest, and common coordinate frameworks for mapping must be in place. There has been general community acceptance of the Allen Mouse Brain Common Coordinate Framework (CCFv3) with several tools now available for pinning specimen level or registering and aligning spatial sections or volumes. Components are now in place for a fully searchable and spatially resolved database, although there remains engineering work to incorporate these into a functional application. Gaps also remain in the BICCN infrastructure. The BCDC data catalog offers an entry point to each project and dataset, yet specimen-level search and access enumerating regional or nuclei-level search is not at present universally available. Further, while the data archives are all capable of accepting and managing large data volumes, and many tools are available for accessing relevant data, the workflow is not yet fully interoperable and there are still inconsistent metadata standards across modalities. This is particularly challenging for users of multimodality data types such as Patch-seq where the associated data types, transcriptomic, morphology, and electrophysiology are stored in different archives.BICCN data represent unprecedented coverage describing the cell type landscape of the mammalian brain, and the stage is now set for completing the BRAIN Initiative 2025  vision oThe ultimate expectation of BRAIN 2025 is to accomplish a full census of neuronal and glial cell types in mouse, human, and NHP, an intellectual framework for cell type classification, and to provide experimental access to the different brain cell types to determine their roles in health and disease. However, there is not yet full consensus on what a neuronal type is, since a variety of factors including experience, connectivity, and neuromodulators can diversify the molecular, electrical, and structural properties of initially similar neurons. There is also increasing evidence that there may not even be sharp boundaries separating subtypes from each other, and cell phenotypes may change over time. Here, taxonomies of putative types and representative cells will provide a frame of reference for studies across labs, and possibly in different organisms, allowing cross-comparison. The data and resources under development in the combined BICCN/BICAN data ecosystems should provide researchers with tools to address these challenges.The extension of the multimodal cell-type atlas of select regions in the human brain to multiple brain regions, particularly those housing vulnerable cell populations, and to different stages of brain development is essential. Such openly available datasets will be key to future studies comparing cell types within their spatial context in the normative brain to those in neuropsychiatric disease, with the addition of transcriptional, epigenetic, morphological, and neurophysiological datasets from postmortem brains, either within the BICCN and BICAN data archives or published in the literature. Combining the BICCN and BICAN data archives with the ability to place cells within a CCF detected from deidentified digital pathology data will also make available large datasets that provide the sample numbers and diverse representation necessary for use of interpretative machine learning analysis applications. Moreover, as techniques for spatial detection of proteins and metabolites achieve multiplexing capabilities as well as cellular resolution, such data may help to uncover disease mechanisms that may be beyond transcriptional and epigenetic detection but, when combined with data currently included in the BICCN datasets, could help explain the neurophysiological changes detected in specific cell types and brain areas as part of a disease phenotype.The BICCN has provided the community with massive high-quality datasets describing the multimodal cell type landscape of the mammalian brain. Substantial resources now exist for the study of brain cell types, and while the supporting data ecosystem is not yet complete, tremendous progress has been made. Increasingly diverse skills are being applied to the architectural design and development of the new BICAN data ecosystem, and we are planning for continuous extension and enhancement of this work to address human-specific challenges. We are only beginning to interpret this valuable data and to understand its importance for the nature of cell types in the brain.S1 TableBICCN Grants: Lists all BICCN investigators and award information. BICCN Modalities: Defines the modalities profiled by the BICCN. BICCN Techniques: Detailed definition of the techniques used by BICCN investigators. BICCN Species: Species profiled including binomial name and NCBI taxon ID.(XLSX)Click here for additional data file.S2 TableDefinition of data levels defined by the BICCN. Columns are detailed definition for each specific modality profiled.(XLSX)Click here for additional data file.S3 TableList of all datasets produced by BICCN investigators and laboratories. Columns define grant, PI, modality, level of data, description of data, collection name, provenance of data and location in archives, tools used in processing data.(XLSX)Click here for additional data file.S4 TableComplete set of software and tools resources generated by the BICCN, including type, RRID, name of resource, location for access and description.(XLSX)Click here for additional data file.S1 Text(DOCX)Click here for additional data file.


Palliative care (PC) aims to prioritize the quality of life and comfort of those with complex and terminal illnesses through the physical, psychological, social, and spiritual needs of the individual. Palliative care can support individuals from the day of their cancer diagnosis through end-of-life and bereavement ,2. In caIn practice, effective communication and trust-building are the cornerstones of delivering patient centered care, including palliative care. To overcome common shortcomings encountered during palliative care discussions, communication tools have been developed to effectively address these gaps and help patients make personally meaningful decisions based on tangible outcomes that are tailored to their specific circumstances ,7–10. FoWhile clinical trials have shown that early integration of PC is correlated with improved pain and symptom control, better quality of life, reduced hospitalizations and survival for individuals with advanced cancers , the literature also suggests that there is a dearth of early palliative care integration for individuals with genitourinary cancers ,12–23. TThere is substantial evidence regarding the positive beneficial effects of early integration of PC within the field of oncology. Several large single-center and multicenter randomized trials of early integration of secondary or subspecialty PC teams in routine oncologic care have shown improvements in patient overall mood and quality of life (QOL), as well as the potential to improve patient survival ,17,24. FDespite the supporting evidence, many cancer patients are still not receiving early palliative care. For example, only 8% of lung cancer patients in the United States see a PC specialist, and these visits are often near death to address end-of-life issues .Despite representing nearly 25% of new cancer diagnoses, with over 350,000 new cases diagnosed each year in the US, few genitourinary (GU) patients have been included in the above palliative care trials  The AdvaRCC makes up approximately 90% of current kidney cancer cases . ApproxiIn the early disease setting where surgery is curative, approximately 20–40% of patients will develop recurrence or distant metastasis after surgery. Multiple trials have explored the potential benefit of oral tyrosine kinase inhibitor systemic therapies following surgical extirpation for high-risk localized RCC while showing progression free survival (PFS) benefit, all have failed to show an OS benefit . Most reTreatment paradigms for RCC that were once concrete in the setting of localized versus metastatic disease with curative surgical resection and palliative systemic therapies as two possibilities of intervention, respectively. With recent advances, the paradigm has now become more nuanced. Through the approval of adjuvant therapies including immunotherapy, systemic interventions are used in curative setting and PC can benefit patients in this setting as well. Newer combination systemic therapies (doublets and triplets) for patients with metastatic disease show long term complete response and possible cure in approximately 10% of cases. Thus, integration of early PC is more important and necessary than ever.As shown in Opportunities for secondary palliative care interventions can also be thought of in the context of curative versus noncurative settings.45 An example of the integration of palliative care in these contextual settings throughout the RCC disease progression is seen in Despite the number of patients undergoing treatment that may benefit from supportive care interventions, PC usage for RCC remains poorly integrated . An analDespite the numerous benefits of early integration of palliative integration for those with advanced RCC, potential obstacles and challenges exist preventing this integration which is derived from similar experiences across the entire cancer care continuum. Only about 14% of patients who need palliative care receive it according to the World Health Organization . BarrierOne recent survey demonstrated that of United States cancer centers, 95% of National Cancer Institute (NCI)-designated institutions had outpatient palliative care programs versus only 40% of non-NCI centers . ResourcProvider barriers for early integration of palliative care includes lack of understanding of palliative care and its assumed association with hospice and inevitably death, lack of exposure or training beyond residency, and lack of understanding of local resources and available at their institution ,58. AddiPatient barriers include lack of understanding of the definition of palliative care, lack of knowledge of available resources, cost, patient-provider emotions, or inaccurate prognostic interpretation of their overall disease state . A natioOncologists themselves are not devoid of emotion – the impact on providers can be heightened when they have established a long-term patient-physician relationship throughout the course of a patient’s cancer diagnosis and treatment. One study demonstrated that the longer an oncologist had known a patient, the less accurate their prediction and disclosure of the prognosis became . FurtherThese challenges or limitations are even more apparent in the era of immunotherapy use in the treatment of RCC. As previously mentioned, immunotherapy can provide a potential for cure in a cohort that were previously thought to have incurable disease. In a study of those with advanced GU cancers, the authors found that anxiety scores were actually lower in patients with accurate understanding of the expected outcomes of immunotherapy and their disease course .Overall, the steps for early integration of palliative care for patients with renal cell carcinoma starts with evaluation of patient function, disease stage and eliciting patient goals, values and preferences. In addition to addressing physical symptoms of cancer and cancer treatment, optimal care involves Identification and addressing emotional, spiritual or practical needs. For those symptoms not able to be addressed by the primary team, other members of the teams including PC team can be consulted. There needs to be coordination of care with the patient’s primary oncology team and other members of the healthcare team, such as nurses, social workers, and chaplains, to ensure that all aspects of the patient’s needs are addressed. Supportive interventions may be offered such as counseling or therapy, to help the patient cope with the emotional impact of the cancer and its treatment. Education of the patient and their family about the disease and treatment, prognosis, and realistic goals of care, to help with shared decision making. Finally, regular assessment of the patient’s needs and adjusting the care plan is necessary to ensure that the patient’s symptoms are well-managed, and their overall well-being is supported throughout their treatment trajectory. The benefits of early integration of palliative care can include improved symptom management, better communication and coordination of care, and greater support for patients and their families throughout the cancer journey.Despite the points highlighted on how to approach this integration, there remains limited data on palliative intervention in the setting of advanced RCC, we present opportunities to overcome possible limitations or barriers to integration in routine oncologic care. In terms of lack of a supportive infrastructure or access to care, with one evident barrier being patient transportation for frequent palliative care visits, the increased acceptance of telemedicine with enhanced focus on nursing phone follow-ups can overcome these structural barriers ,64.Provider barriers can be addressed with improved understanding of PC interventions as well as structured exposure and education to these processes. A survey administered to oncology fellows in 2011 demonstrated that only 26% had completed a palliative care rotation, with only 32% of fellows receiving explicit education on assessing and managing depression at end of life . For thoIn terms of improved provider skills and understanding in order to mitigate PC discussions, one must perform a self-assessment of the individual skillsets of any advanced RCC provider such as a urologist, medical oncologist, or radiation oncologist that are part of the primary PC team. Based on the self-assessment of the primary oncology team members, in addition to the primary palliative care they provide, they must refer to secondary palliative care when appropriate. In addition to the development of basic communication skills, there are specific skills in palliative care delivery including symptom assessment and management, gauging of emotional symptoms, and patient centered discussions on decision-making. We must also consider the multidisciplinary aspect of palliative care administration outside of oncologic providers alone. A study identified the surgeon’s role outside of clinical and operative decision-making, stating that surgeons also work towards understanding patient and family goals, facilitating challenging and tense discussions, and fostering shared decision making. Data supports that early palliative care involvement alongside surgical treatment facilitates goalconcordant decision making, improves patient emotional and spiritual well-being, and enhances overall quality of life .Finally, patient-provider barriers can be overcome with standardization of disease prognosis and decision-making discussions, as well as interval assessment of patient understanding of their treatment course. One group developed a preliminary measure for patient prognostic understanding of their own condition, Prognostic Understanding Perceptions Scale (PUPS), which has been utilized for those with advanced cancers in order to further facilitate patient-physician discussions . This, aDespite the evidence of survival benefit demonstrated by those utilizing early integration of palliative care use, studies do not definitively demonstrate why these patients may have benefited. One focused area of study could include the improved patient understanding of their disease status and shared-decision making models – patients not only need to understand their goals, values, and preferences, but also their treatment options including the risks and benefits of each. Determining more focused goals of providers in these discussions in the interplay of oncologic teams and palliative care physicians as primary team members or in the consultant role, can guide future research into how these mechanisms are best applied.With the current advancements in immunotherapy use in the setting of advanced RCC, traditional definitions of curative and palliative disease states are blurred. This enhances the need for comprehensive and systematic integration of palliative care. With increased integration, comes enhanced awareness of not only structural but patient-provider barriers that once identified can be overcome with improved communication and multi-disciplinary team efforts. Increased areas of study within primary teams for RCC with focus on shared decision making and patient centered care may help develop improved interventions for RCC and across the cancer care continuum.


An IFS approach has led to the development of several intervention strategies within Indigenous communities, which have been highlighted in this review. The successful elements identified in this review may serve to support future food sovereignty-related programmatic and intervention development within Indigenous communities.Despite inherent resiliency and strengths, Indigenous Peoples in the United States and Canada have been impacted by colonialism, which has led to a loss of land, culture, and identity. Loss of land in particular has had substantial impacts on Indigenous food system practices. Indigenous food sovereignty (IFS) has been determined to be a mechanism for Indigenous communities to build their capacity to address food insecurity. A systematic review methodology was therefore engaged to gather and analyze the currently published literature to date to identify common elements of successful IFS interventions within Indigenous communities in the United States and Canada. We carried out a systematic search of the following electronic databases: Academic Search Premier, Agricola, PubMed, CINAHL Complete, Indigenous Studies Portal, the Native Health Database, SocIndex, PsycInfo, and Google Scholar. The Mixed Methods Appraisal Tool was used to apply a methodologic quality score to the included articles. We used a 2-stage process for article selection with 2 independent reviewers screening the titles and abstracts of articles identified. Relevant databases were initially searched up to June 2022 with an updated search occurring in January 2023. Content analysis was carried out on the included articles using qualitative analysis software. Twenty articles met the inclusion criteria of the review. Four main categories of successful elements within IFS interventions were identified, including PROSPERO (number: CRD42022342100). Indigenous Peoples are resilient peoples, with strong ties to culture and land bases around the world. Despite this, over successive generations, colonization has had devastating impacts on Indigenous Peoples in the United States and Canada, resulting in longstanding consequences to individuals, families, and communities ,2. Core In the United States and Canada, Indigenous Peoples experience higher rates of food insecurity ,5. Food Additional factors have also contributed and interacted within the legacies of colonialism, which have additionally contributed to the destruction of traditional food systems. Industrialization has led to the destruction of land and forests, making it difficult and unsafe for Indigenous Peoples to continue traditional cultivation practices . For exaOf important note are the sociocultural components integrated within Indigenous traditional food systems that were altered through colonial beliefs and practices ,5. TheseHistorically, among the Omushkego Cree communities, women were instrumental in the skills of hunting and fishing . Within The negative impacts to Indigenous food systems have culminated in a series of challenges, including diminished access and availability to quality foods, limitations in resources critical to cultivating culturally supported food choices, and changes to the ways in which Indigenous Peoples interact with each other and food. As a result, many Indigenous Peoples have begun to reclaim their traditional practices through the embodiment of food sovereignty practices with the vision of promoting self-determination and the decolonization of food systems ,17. Food1) outline current food sovereignty intervention strategies within Indigenous communities in the United States and Canada, and 2) to determine the elements of successful food sovereignty programs within Indigenous communities in the United States and Canada that may inform future efforts.Although intervention strategies have been highlighted within the literature to promote food sovereignty within Indigenous communities in the United States and Canada , a formaIt is increasingly expected within academic publishing that in the context of Indigenous Peoples, “nothing about us, without us .” With tWe engaged systematic review methodology while following the preferred reporting items for systematic reviews and meta-analysis (PRIMSA) for this review . We prerOur search strategy was inclusive of key terms in the realm of food sovereignty and food security. An example search strategy can be found in We used a 2-stage process for article selection. Two independent reviewers  screened the titles and abstracts of articles identified during the systematic search. A third reviewer was brought in to resolve discrepancies by discussion (NR). The full text articles were then reviewed to determine final inclusion in the review. One reviewer examined 100% of the identified full-text articles (BG), and a second reviewer cross-checked 10% of the full-text articles to ensure consistency (DK). A third reviewer was brought in to resolve discrepancies by discussion (NR). Eligible studies were then retained for data extraction. The first reviewer extracted the following data, including: source , methods , participants  and country/countries recruited from, study eligibility criteria), measures , results (key conclusions from study authors), and miscellaneous (funding source).Two independent reviewers  used the Mixed Methods Appraisal Tool (MMAT version 2018) to assess eligible studies and determined an overall methodologic quality score . A thirdn = 13) of the food sovereignty interventions being based in Canada and 20% (n = 4) being based in the United States. Fifteen percent (n = 3) of the articles identified intervention strategies in communities that span the US and Canadian borders , traditional food access interventions via community programs (n = 5), community resource interventions for preservation of traditional foods and to address barriers (n = 1), and reconnection/reintroduction of traditional food interventions through local initiatives or school-based curricula (n = 4). The age ranges of participants in the various interventions ranged from 1 year old to 90 years old. Genders were well represented with many participants serving various roles in the community from land managers, harvesters, community members, council members, and community advisory board members. Of the interventions, 55% (n = 11) consisted of collaborations among local community organizations and stakeholders within Indigenous communities, while 45% (n = 9) of the interventions were implemented as part of collaborations with entities such as public health organizations, governmental organizations, or university partnerships. Results of the MMAT quality appraisal can be found in summary in The types of interventions carried out in the included articles ranged from garden programs to food preservation interventions see . Overall1) transmission of knowledge and skills within the community through hands-on workshops, 2) enabling cultural connectedness through cultivation practices, 3) direct consumption and preparation of traditional foods through community programs, and 4) active community-based partnerships and collaborations.Content analysis identified 4 main successful food sovereignty intervention elements among the included article base within Indigenous communities in the United States and Canada, including Transmission of knowledge and skills within the community through hands-on workshops was identified as a common element of food sovereignty interventions within Indigenous communities. Most interventions described the involvement of Elders in knowledge transmission as well as in the direct contribution to programs. For example, one of the intervention workshops described one Elder who shared her Lakota/Dakota knowledge on traditional foods and home remedies found around Standing Buffalo First Nation . AnotherEnabling cultural connectedness through cultivation practices was identified as an important element of IFS interventions. Several of the interventions clearly identified a sense of cultural connectedness enabled through gardening . In a stDirect preparation and consumption of traditional foods through community programs was another common element identified within the IFS interventions. Several intervention strategies suggested that traditional food consumption was an important component of programming and therefore included it within their approaches ,44. One Active community-based partnerships and collaborations were also identified as another common element of successful IFS interventions. Several of the intervention strategies were conducted with internal and external partnerships within or outside Indigenous communities . Two of Community-based collaborations also involved local stakeholders such as schools and governmental agencies who assisted with providing services and resources to community members to support, sustain, or preserve traditional foods. One community-based collaboration project consisted of a local health agency and Indigenous community that implemented community champions . These cOur review identified 4 main types of overarching IFS interventions, including those promoting cultural connectedness interventions through traditional food practices; traditional food access interventions via community programs; community resource interventions for preservation of traditional foods and to address barriers; and reconnection/reintroduction of traditional food interventions through local initiatives or school-based curricula. Within these 4 main intervention types, our content analysis identified 4 main elements of successful IFS interventions among the included articles from the United States and Canada. The 4 main elements included transmission of knowledge and skills within the community through hands-on workshops, enabling cultural connectedness through cultivation practices, direct preparation and consumption of traditional foods through community programs, and active community-based partnerships and collaborations. The successful elements identified in this review may serve as a supportive roadmap for developing adaptable local food sovereignty frameworks, and for developing new interventions or initiatives to address IFS in the United States and Canada.1) community ownership, 2) inclusion of cultural knowledge, 3) inclusion of traditional foods, and 4) environmental sustainability of [the] intervention [Prior work by Maudrie et al.  outlinedrvention . Three orvention  had someThe “corn” element of the “Three Sisters” symbolizes the transmission of knowledge and skills within the community. Corn is a sacred plant within many Indigenous communities, being a “must to live” . Corn isThe bean plant within the “Three Sisters” wraps itself around the stalk of the corn plant with an interconnected partnership. The bean plants’ relationship with the corn and the squash see  is basedFinally, the squash in the “Three Sisters” touches the soil and is the embodiment of culturally grounded interventions. Interventions that are grounded platform the touching, smelling, preparation, and consumption of traditional foods that have sustained communities for millennia.The preparation and consumption of traditional foods through interventions also serve to provide Indigenous Peoples with access to quality nutritious foods while addressing food insecurity. The “Three Sisters” themselves work together to ensure appropriate nourishment and protection of each other. The plants’ combined presence brings nutrition and growth to themselves in addition to the Indigenous communities that steward these plants.Overall, there is a lack of published academic literature related to food sovereignty intervention strategies within Indigenous communities in the United States and Canada. This gap limits the sharing of best practices across Indigenous communities that could be helpful for the development of future intervention strategies  and grant mechanisms. Although assumptions of pan-Indigenous approaches need to be reduced given the diversity of Indigenous Peoples within the United States and Canada, an additional focus on “process” in future work may support the development of a more comprehensive food sovereignty framework for Indigenous Peoples that can be adapted to local contexts. Future questions include: Are there additional process or implementation considerations that can overlay and interact with the successful elements of local food sovereignty interventions already identified? Can Although the review identified several elements of successful IFS interventions, there were limitations that should be considered. First, some of the articles did not specify participant numbers. Studies that lacked this information made it difficult to determine whether the intervention was effective across large or small population sizes. Secondly, it is also possible that the search strategy that was developed and operationalized did not find all the relevant articles. Given the consistency of categories identified within the published literature base, however, we are confident that the key elements outlined are relevant to IFS interventions within the United States and Canada. Finally, this review examined only published academic articles. It became apparent throughout the review that there may have been some IFS interventions that were mentioned in other article formats, such as opinion pieces referenced from gray literature sources that may be relevant. Given this, future work could expand the review search to be inclusive of community documents and other gray literature to have a more complete understanding of the parameters surrounding the IFS intervention space outside of the published academic literature.Forced interruptions to Indigenous food system practices have impeded the ability of many Indigenous Nations to feed themselves. The disruptions to traditional Indigenous food systems have created a sense of loss within many Indigenous communities in the United States and Canada. Food sovereignty has served as an approach to reconnect Indigenous Peoples with their food systems and culture. A food sovereignty approach has led to the development of several intervention strategies among and within Indigenous communities that have been highlighted in this review. The successful food sovereignty elements identified may serve to support future food sovereignty-related programmatic and intervention development within Indigenous communities in the United States and Canada.The authors’ responsibilities were as follows – BG, NR: designed research; BG, DK, NR: conducted research; BG, NR: analyzed data; BG, NR: wrote paper; BG, NR: had primary responsibility for final content; and all authors: read and approved the final manuscript.The authors report no conflicts of interest.Shakopee Mdewakanton Sioux Community through a gift to the University of Minnesota from its Seeds of Native Health campaign.The publication of this paper was supported by the Data described in the manuscript will be made available upon request pending communication to the authors.


Hepatocellular carcinoma is a malignant neoplasm of the liver and a leading cause of cancer-related deaths worldwide. The multimodal data combines several modalities, such as medical images, clinical parameters, and electronic health record (EHR) reports, from diverse sources to accomplish the diagnosis of liver cancer. The introduction of deep learning models with multimodal data can enhance the diagnosis and improve physicians' decision-making for cancer patients.This scoping review explores the use of multimodal deep learning techniques  in diagnosing and prognosis of hepatocellular carcinoma (HCC) and cholangiocarcinoma (CCA).A comprehensive literature search was conducted in six databases along with forward and backward references list checking of the included studies. PRISMA  extension for scoping review guidelines were followed for the study selection process. The data was extracted and synthesized from the included studies through thematic analysis.Ten studies were included in this review. These studies utilized multimodal deep learning to predict and diagnose hepatocellular carcinoma (HCC), but no studies examined cholangiocarcinoma (CCA). Four imaging modalities  and 51 unique EHR records  were used in these studies. The most frequently used medical imaging modalities were CT scans followed by MRI, whereas the most common EHR parameters used were age, gender, alpha-fetoprotein AFP, albumin, coagulation factors, and bilirubin. Ten unique deep-learning techniques were applied to both EHR modalities and imaging modalities for two main purposes, prediction and diagnosis.The use of multimodal data and deep learning techniques can help in the diagnosis and prediction of HCC. However, there is a limited number of works and available datasets for liver cancer, thus limiting the overall advancements of AI for liver cancer applications. Hence, more research should be undertaken to explore further the potential of multimodal deep learning in liver cancer applications. Hepatocellular cancer (HCC) and cholangiocarcinoma (CCA) are two types of liver cancer that are responsible for significant morbidity and mortality worldwide  in the medical field, including the cancer and oncology. With the digitization of healthcare records, AI modes can efficiently utilize patient data. Electronic Health Record (EHR) of patients comprises comprehensive information regarding their medical history, diagnoses, treatments, laboratory results, radiology images, genetic profiles, and more ) AND ((“liver cancer”) OR (“hepatocellular carcinoma”) OR (“bile duct cancer”) OR (“cholangiocarcinoma”)). Two Boolean operators were introduced, the OR operator to combine keywords within each category and the AND operator to merge keywords across all categories.The search terms used in this study were:  OR (“deep learning”)) AND  OR  OR  OR , transformers, or neural networks in their methods. The aim of this study was to identify the use of multimodal deep learning techniques in liver cancer application. So, we excluded studies that used only traditional machine-learning techniques. The types of included studies were peer-reviewed articles, dissertations, book chapters, and conference proceedings published from 2018 to August 2023. Only English language texts were included. We excluded studies that did not combine medical imaging and EHR data and also excluded studies that developed models for diseases other than liver cancer. We excluded systematic reviews, abstracts, studies that used languages other than English, and studies that were published before the year 2018. The study selection process was carried out by three authors independently. Conflicts among them were resolved through mutual discussions and through validation by all the authors.The data were extracted from the included studies using a standardized form, including information on the study design, sample size, population characteristics, AI methods, interventions, and outcomes. It was used to ensure accurate and precise documentation of significant information for each study. The data extraction form is provided in Our search terms yielded 363 studies from six different databases . After going through these studies' titles and abstracts, we excluded 276 studies and included 18 studies. The number of excluded studies with their reason for exclusion are listed in the PRISMA flowchart shown in Going through the demographic of the included 10 studies, we can see that 8 studies were from China and only 2 studies were from Brazil. During our literature review, we searched for studies published from 2018 to 2023; however, the final included studies were only published from 2020 to 2022. All the studies, except one conference paper, were journal articles.The research encompassed multimodal AI techniques that are capable of identifying not only diagnostic parameters and biomarkers in EHRs, but also recognizing HCC abnormalities in medical image modalities. All of the studies focused on HCC, and no study involved CCA.The purpose of employing multimodal deep learning models for HCC can be categorized into two main objectives: disease prediction and disease classification or diagnosis. Additionally, these models were utilized for predicting treatment response, determining survival rates, and staging the disease.The primary deep learning architecture employed in these models was CNN. The multimodal deep learning models used a combination of two different deep learning neural network models trained on processing two different types of datasets modalities . The included studies used models like VGG16, VGG19, Inception V3, and ResNet18, to extract and analyze detailed spatial features of medical images in creating multimodal AI techniques  is used to analyze mRNA gene expression data from patients' files, while Cox-regression utilizes its outcome and the outcome from VGG16 (WSI processing model) to predict HCC patient survival , total bilirubin (TBIL), alpha fetoprotein (AFP), carbohydrate antigen 19-9 (CA19-9), carcinoembryonic antigen (CEA), carbohydrate antigen 125 (CA125), hepatitis B surface antigen (HBsAg), and liver function test. All other parameters are specified in The most frequently used parameters are gender (reported in nine studies), age (reported in eight studies), alpha-fetoprotein (AFP) (reported in seven studies), platelet (PLT) count (reported in five studies), albumin (reported in six studies), and prothrombin time (PT) (reported in four studies). These parameters are used in multiple studies and are considered to be important in the diagnosis and prediction of liver cancer. Other commonly used parameters include total bilirubin (reported in four studies), Child-Pugh classification (reported in four studies), hepatitis B virus (HBV) (reported in three studies), ALT (reported in three studies), serum aspartate aminotransferase (AST) (reported in three studies), tumor marker  (reported in two studies), carbohydrate antigen 19-9 (CA19-9) (reported in two studies), carcinoembryonic antigen (CEA) (reported in two studies), and carbohydrate antigen 125 (CA125) (reported in two studies).The model CNN combined with MTNet used the highest number of EHR parameters employing 22 clinical parameters  and accuracy. AUC was used in seven studies  is a method used to evaluate the clinical value of the AI models. It involves comparing the net benefit of using the model to make clinical decisions with the net benefit of using a different decision-making strategy. Additionally, the use of various statistical tests like NRI (Net Reclassification Index), Integrated Discrimination Improvement (IDI), and calibration helped assess the models' performance. The use of different evaluation metrics is summarized in There are certain challenges related to the multimodal deep learning models developed in the included studies  data separately and then combined the results with building the multimodal AI model. There was no single variant of CNN architecture used across all the studies. CT scans were the most commonly used medical imaging modality, followed by MRI. We identified more than 50 clinical parameters and biomarkers related to HCC that were used to train and test the multimodal AI models. However, none of the studies included the history of jaundice or bile duct disorders as part of the clinical parameters, despite their importance as signs of liver abnormalities.It is worth noting that the number of EHR modalities processed does not necessarily correspond to the performance of the deep learning model. Other factors, such as the model architecture and training data also play important roles in determining the model's performance.The multimodal AI techniques lacked multi-centers data and the potential relationship between modalities was not clear. To address these challenges, effective algorithms for extracting multimodal data information should be explored, computational representations based on biological discoveries should be used, larger populations and multicenter studies should be conducted, and feature selection techniques and clinical indexes should be employed.However, this scoping review allows researchers to investigate more uses of multimodal AI for the diagnosis of HCC and to study multimodal AI techniques developed for recognizing more medical imaging modalities. It also encourages researchers to study multimodal AI techniques for purposes other than those mentioned in this paper and to start conducting similar studies in CCA. Datasets utilized by the included studies are mainly from private sources and are used as training and validation datasets. Some studies do not mention the type of validation adopted for the model training and evaluation.While our scoping review offers valuable insights into the use of multimodal AI in liver cancer research, it is important to acknowledge the limitations of our study. One major limitation is that some of the datasets used in the studies were not fully described, leaving questions about their labeling and generalizability. Additionally, the lack of specificity regarding data sources used in the studies could make it difficult to reproduce or compare the models' performances. Finally, we focused on studies published within the past 5 years, which may have limited the scope of our analysis but captured the most recent development in the field of multimodal deep learning for the detection and diagnosis of liver cancer. The specific inclusion criteria on EHR+images data also limited the number of studies.Despite these limitations, our review provides a foundation for understanding the application of Multimodal AI techniques in liver cancer research. By identifying common variables and models used across studies, we can better assess the potential of these models in improving HCC prediction and treatment response.The detection and prognosis of liver cancer, or HCC, have recently been facilitated by recent advancements in deep learning-based AI techniques. In this scoping review, we analyzed 10 studies that investigated the application of multimodal deep learning models in HCC. We did not find any studies related to the use of multimodal deep learning models for CCA. The studies focused primarily on HCC prediction rather than HCC classification or diagnosis, with a particular emphasis on predicting response to TACE treatment. Overall, the studies highlighted the potential of multimodal AI for improving HCC prediction and treatment response assessment, but more research is needed to explore their effectiveness in other areas of liver cancer research. Multimodal AI techniques have the capacity to simultaneously evaluate vast quantities of complex data, including medical images and electronic health records, and infer useful patterns and insights. With the use of this technology, HCC diagnoses might be made more accurately, and the course of the disease could be predicted, which may ultimately lead to better outcomes and a higher survival rate. However, these models face limitations such as the lack of diverse data sets, unclear relationships between modalities, and a focus on explanations and understanding of the underlying mechanisms. Suggestions include exploring effective algorithms, establishing clear inter-modality relationships, and incorporating biological insights. In the context of HCC, the studies reviewed in this work primarily focused on HCC prediction and treatment response assessment using different multimodal AI techniques. However, limitations such as the “black box” nature of deep learning and the need for pathological interpretations persist. Future research should address these limitations, expand to other liver diseases, and incorporate larger populations and multicenter studies for comprehensive understanding and improved diagnostic and treatment approaches.The original contributions presented in the study are included in the article/AS, AA, and BM performed a search in different databases and were involved in the study inclusion, exclusion, and data extraction. MB resolved if there were any conflicts among AS, AA, and BM and reviewed the scoping review process. HA and ZS conceptualized the study and guided this scoping review. All authors agreed on the final draft of the manuscript.


Objective: To identify processes to engage stakeholders in healthcare Simulation Modeling (SM), and the impacts of this engagement on model design, model implementation, and stakeholder participants. To investigate how engagement process may lead to specific impacts.Data Sources: English-language articles on health SM engaging stakeholders in the MEDLINE, EMBASE, Scopus, Web of Science and Business Source Complete databases published from inception to February 2020.Study Design: A systematic review of the literature based on a priori protocol and reported according to PRISMA guidelines.Extraction Methods: Eligible articles were SM studies with a health outcome which engaged stakeholders in model design. Data were extracted using a data extraction form adapted to be specific for stakeholder engagement in SM studies. Data were analyzed using summary statistics, deductive and inductive content analysis, and narrative synthesis.Principal Findings: Thirty-two articles met inclusion criteria. Processes used to engage stakeholders in healthcare SM are heterogenous and often based on intuition rather than clear methodological frameworks. These processes most commonly involve stakeholders across multiple SM stages via discussion/dialogue, interviews, workshops and meetings. Key reported impacts of stakeholder engagement included improved model quality/accuracy, implementation, and stakeholder decision-making. However, for all but four studies, these reports represented author perceptions rather than formal evaluations incorporating stakeholder perspectives. Possible process enablers of impact included the use of models as “boundary objects” and structured facilitation via storytelling to promote effective communication and mutual understanding between stakeholders and modelers.Conclusions: There is a large gap in the current literature of formal evaluation of SM stakeholder engagement, and a lack of consensus about the processes required for effective SM stakeholder engagement. The adoption and clear reporting of structured engagement and process evaluation methodologies/frameworks are required to advance the field and produce evidence of impact. Simulation modeling is an effective research methodology to address complex and “wicked” problems in healthcare and public health.Involving stakeholders in healthcare simulation modeling is assumed to produce better (and more relevant) models which are more readily accepted by problem owners and thereby more likely to be implemented, but there is limited evidence to guide choices of methods for engaging stakeholders to enhance the design and implementation of these models.We document the large gap in the current literature of formal evaluation of SM stakeholder engagement, and a lack of consensus about the processes required for effective SM stakeholder engagement.Processes used to engage stakeholders in healthcare simulation modeling are heterogenous and often ill-defined in the literature, generally involving multiple stakeholder types across multiple simulation modeling stages.via storytelling for non-technical communication of model logic. These enablers may work by providing a common language and mutual understanding between stakeholders and modelers.Possible process enablers of impact are the use of models as “boundary objects” and structured facilitation Adoption and clear reporting of structured engagement and process evaluation methodologies/frameworks are required to advance the field.Healthcare decision-makers are facing increasingly “wicked” problems, which have both a technical (complex and uncertain symptoms and solutions) and social (divergent stakeholder perspectives) dimensions , 2. ConfSimulation Modeling (SM) is an established but historically under-utilized methodology in healthcare . SM creaSM has increasingly been combined with approaches intended to engage stakeholders in the modeling process. Engaging stakeholders, traditionally managers and clinicians in the relevant healthcare field, has been claimed to yield both more technically and socially robust simulation solutions  and imprKnowing how best to involve themselves or others in research is a challenge for clinicians, decision-makers and stakeholders in healthcare, as well as for the researchers. Yet involving frontline clinicians, decision-makers and other relevant stakeholders in research that aims to promote a change in practice is key to translating research “off the shelf and into practice” . SeveralThe aim of this review is to systematically synthesize the evidence on how far and by what means stakeholder engagement in SM results in outcomes with more practical utility and prospect of successful implementation. A key objective is to identify the processes used to meaningfully engage stakeholders in SM research and to analyze the impacts of this engagement in enhancing the design and implementation of healthcare simulation models. In order to accomplish this, we analyzed the extent to which these intended purposes or expected benefits of SM have materialized in applications to healthcare problems, and we identify the contribution of engagement processes to facilitating this.This systematic review is reported according to the PRISMA statement  and usedmodel design stage (population), and (4) reported stakeholder engagement impact (outcome). The search was not limited to a specific study design and did not include a comparator. English-language articles were searched in MEDLINE, EMBASE, Scopus, Web of Science and Business Source Complete databases published from inception to February 2020. Common keywords included: simulation, system dynamics, discrete event or agent based; health care, healthcare, hospital, primary care, public health, health policy or health service; group model building, stakeholder, client, customer, implementation, focus group, interview, steering group, advisory board, advisory committee, co-design, co-production; and participatory simulation or participatory model. Full details are in Eligible articles were original studies that (1) used dynamic SM (intervention), (2) reported a health-related primary outcome (context), (3) engaged stakeholders during the Title and abstract, and then full-text screening see  and locaData were analyzed using summary statistics, deductive and inductive content analysis,  as well The evidence synthesis concentrated on authors' reporting of the participatory process in SM studies, which meant that outcome measures from the studies were not included. Therefore, no formal assessment of risk of bias was necessary either in individual studies or across studies .The search yielded 1,682 titles and abstracts for initial screening, with 119 articles included for full-text screening. Of full-text articles screened, 29 met the eligibility criteria, with a further three identified and included from included articles reference lists , US  and Australia . The most common topic areas were Health and Care Systems Operation (56%) and Epidemiology, Health Promotion and Disease Prevention (41%), with few studies in Health and Care Systems Design (3%) and Extreme Events Planning (3%). None of the included studies addressed the topic of Medical Decision Making. Years of publication ranged from 2000 to 2019, with 78% published since 2014 (n = 21). There was a change in trend in study topic areas over time: initially dominated by Health and Care Systems Operation, and in recent years by Epidemiology, Health Promotion and Disease Prevention , with less engagement with Policy-makers (n = 7), Purchasers/Payers (n = 9) and Patients/Public (n = 6). Each study, on average, engaged two different types of stakeholder participants. Further details on the numbers and types of stakeholders are in The number of studies reporting stakeholder engagement during different stages of SM are shown in n = 13, 48%), interviews , workshops , and meetings , with some variation across SM stages. Specifically, interviews were most common earlier in the SM process while workshops were commonly used in the mid and late stages of the process. Workshops, discussion/dialogue and meetings generally involved direct model interaction, where the model acted as a communication vehicle , allowing stakeholders to physically manipulate and “play” with the model design. Some studies provided descriptions about how they facilitated this input, which ranged from structured and active methods where stakeholders were asked specific questions , (2) better quality/accuracy of the model for its purpose , (3) improved identification or access to better data , and (4) expertise from a range of perspectives . Across the SM stages, increased relevance of the problem was most reported during problem formulation, while better quality/accuracy of the model was most reported during the conceptual modeling phase.There were four types of impacts reported from stakeholder engagement on model design: (1) increased relevance of the problem addressed , (2) “learning” , and (3) better problem solving or decision-making . The fact that models require perspectives and assumptions to be made explicit in a graphical representation which imitates the real system meant that several studies found that interacting with the model created productive discussion between stakeholders, particularly those from different disciplines  , and (3) improved implementation or suitable use of the model .There were three types of impacts reported from the participatory process on the stakeholder participants: (1) productive discussion or shared understanding of the problem  , (2) greIn the overlap between the participatory process and impacts, studies involving direct model interaction were more likely to report benefits for stakeholder ownership of the model (56 vs 11%) and productive discussion & shared understanding (61 vs. 22%), vs. studies without direct model interaction. Read more about the impacts of participatory process on model design, implementation and participants in Some studies provided specific details about the process used to combine stakeholder engagement and SM and how to do this well , 33, 49.We have systematically reviewed the ways in which studies using SM have engaged stakeholders through participatory processes. We reviewed these participatory processes on their reported abilities to improve the design and use of models in healthcare as well as produce desired impacts on stakeholder participants. The reported processes used to engage stakeholders in healthcare SM were heterogenous, but there were common characteristics in terms of the stages, modes and activities through which engagement is facilitated. Studies mostly commonly involved provider stakeholders, across multiple SM stages, using discussion/dialogue, interviews, workshops and meetings as key modes of engagement. In addition to conceptual modeling, many studies engaged stakeholders in the adjacent stages of problem formulation and model verification and validation, as well as during the later stages of experimentation and implementation. Interviews were mostly used earlier in the SM process while workshops were mostly used in the mid and late stages of the process. Key reported impacts of stakeholder engagement included improved model quality/accuracy, implementation, and stakeholder decision-making.via improved model design and stakeholder impacts, may lead to improved model implementation.The communication gap between stakeholders and modelers has been identified as a primary causal factor driving consistently poor rates of model implementation in healthcare SM , 54. ThiFirstly, we found that direct interaction between stakeholders and the model seemed to influence interpersonal communication (between stakeholders themselves and between stakeholders and modelers), leading to stakeholder impacts of more productive discussion and shared understanding, and implementation impacts of greater ownership and acceptance of the model. This is likely because the model operates as a “boundary object” , a visuaMore structured methods of stakeholder engagement, i.e., the use of specific questions  or purpoMany of the methodological issues faced by this review were due to a lack of standardized or detailed reporting of stakeholder engagement, and insufficient reporting was one of the primary reasons for article exclusion. This lack of reporting comprised the details of the process itself and adequate evaluations of the engagement process. Only five studies provided specific details about the stakeholder engagement process , 33, 49,a priori embedded component of the research design.A recent framework from the environmental model building field provides a possible solution to the lack of standardized reporting of stakeholder engagement that may be equally applicable to healthcare SM. Gray et al. propose a four-dimensional reporting framework (4P) which includes: (1) the Purpose for using a participatory SM approach , (2) the Process by which stakeholders were involved in model building or evaluation; (3) the Partnerships that were defined and participants that were chosen; and (4) the Products that resulted from these efforts  . A detaiA limitation of this review is the timeframe, which included studies published until March 2020.The crisis and transformation occurring in health care since February 2020 due to the COVID-19 pandemic is deliberately not captured here. The COVID-19 pandemic has resulted in rapid changes inside health systems including changes in direct care procedures and the adoption of remote care through new technologies, with a corresponding global burden of high health care worker stress. Therefore, health care stakeholder involvement for SM during COVID-19 would involve different, crisis driven approaches, and is the subject of a separate subsequent project.This review explored the process by which studies engaged stakeholders in healthcare SM and the impacts of the engagement process on model design, model implementation and stakeholders. We found that engagement of stakeholders in the SM process was common during multiple stages, involved informal discussion as well as more formal one-to-one interviews or group workshops, and was facilitated by a range of more or less structured activities for model building, with structured activities associated with improved model quality and ability to capture diverse perspectives and expertise. Key enablers reported by authors and stakeholder participants were the use of the evolving model as a “boundary object” to facilitate communication and storytelling to communicate the model logic, complexities and interactions in a non-technical way. We suggest the adoption and clear reporting of structured engagement and process evaluation methodologies/frameworks to enable high-quality stakeholder involvement, improve SM confidence and ownership by healthcare staff and decision-makers, and ultimately lead to implementation of optimal interventions identified by SM that contribute toward better health care systems.The original contributions presented in the study are included in the article/TZ, KL, IM, and JE devised the research and developed the research questions in collaboration with DS and JH. Search strategy was devised by TZ with assistance by KL and JE. Articles were screened by TZ and KL and fulltext extraction performed by TZ with technical guidance and assistance by KL and JE. Manuscript was drafted by TZ. All authors provided inputs into subsequent revisions.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.