Automated analysis of the inner ear anatomy in radiological data instead of time-consuming manual assessment is a worthwhile goal that could facilitate preoperative planning and clinical research. We propose a framework encompassing joint semantic segmentation of the inner ear and anatomical landmark detection of helicotrema, oval and round window. A fully automated pipeline with a single, dual-headed volumetric 3D U-Net was implemented, trained and evaluated using manually labeled in-house datasets from cadaveric specimen ( For a safe cochlear implantation base, it is essential to have a realistic pre- and intra-operative anatomical information about the cochlea and adjacent inner ear structures, not only to select the correct electrode for obtaining an appropriate cochlear coverage3, but to also prevent damage to intracochlear structures4. Furthermore, anatomical information about the inner ear may be used to elucidate the pathophysiology of hearing loss and balance disease in large scale evaluations of radiological data. Non-invasive radiological imaging of the temporal bone via computed tomography (CT) or magnetic resonance imaging (MRI) is currently the default strategy to provide the required anatomical information to the researcher or clinician. Inside the current routine workflow, the data is inspected visually as a series of 2D multiplanar views to mentally comprehend the 3D structure and gather the desired information. The automated creation of advanced 3D visualizations on the basis of semantic segmentation in conjunction with parameter estimation can thus be viewed as highly beneficial to the process of cochlear implantation in particular5 and inner ear clinical research in general. However, the generation of accurate and individualized analysis from radiological data remains time, cost and labour intensive. The generation process is bottlenecked by the step of manual labeling key structures due to the complexity and subtleties of the involved anatomy6 and suffers from inter- and intra-observer variability. To realize the envisioned benefits for clinical purposes and fix the pitfalls of the manual workflow, the two related tasks of a) semantic segmentation of inner ear structures and b) localization of landmarks have to be solved by a fully automated pipeline with minimal user interaction. Success in these two fundamental tasks may act as a gateway to fully automated cochlear duct length measurements, supplanting current formula-based methods relying on Bézier curves fitted through manually set keypoints7. To approach the automatization goal, we consider recent progress in the field of computer vision (CV) that has been spurred disruptively by deep learning with convolutional neural networks8 as the architecture of choice. Achieving state-of-the-art performance in image classification9 and medical image analysis10 tasks alike, they can drastically simplify them via automation by learning from a pre-assembled set of training examples. Careful compilation of the training examples enables the minimization of intra- and inter-observer variability. In contrast to more traditional machine learning algorithms, deep neural nets are regarded as adequate for operating with high-dimensional, unstructured data like volume CT data due to their capacity to internally construct salient data features during the optimization phase. An integrated, performant and data-efficient deep learning framework for automated analysis of inner ear structures could be beneficial for fast preoperative planning and large-scale clinical research.The benefit of cochlear implantation has been shown in various studies and reviews11 for cochlear structure segmentation and inner ear fluid space12 or iterative random-walk with shape priors for full inner ear segmentation in 13 have been suggested in the past. Within the realm of deep learning based methods, there exist several closely related and recent studies that directly share the goal of automated analysis of inner ear structures from volumetric CT data. In the following, we provide a summary of these adjacent works.As a starting point in related literature, traditional methods like atlas-based frameworks14, sequentially staged 2D ResNet and 2D UNet were used to segment the cochlea from 3D ultra-high resolution clinical CT data in the context of automated cochlea length and volume measurement. The proposed system was formulated with 123 volume CT scans. Here, the 2D to 3D gap was bridged by using a 2D patch sampling approach that was iteratively integrated into volumes. The protocol leveraged the primary ResNet stage to generate soft-masking probability maps on three different scales with differing networks. The soft masks were then utilized to constrain the outputs of the secondary UNet stage to the previously identified foreground region. The augmentation protocol for the patches comprised in-plane rotations with an amplitude of In Heutink et al.15 employed a set of 3D CNN variants to perform multi-class segmentation on 150 clinical volume CT temporal bone scans. Three backbones, ResNet, UNet and the newly proposed AHNet were compared. The framework was presented as an integrated pipeline in the 3DSlicer software16 that makes use of largest region extraction and growing/shrinking algorithms with pre-set intensity values to postprocess all convolutional neural network outputs. The augmentation protocol encompassed intensity oscillations (10% of intensity magnitude) and flipping about the first axis. The validation and testing procedure for this work encompassed five-fold CV and a separate testing set of 25 volumes.Another adjacent study by Neves et al.17 leveraged open-source micro CT data of 17 cadaveric specimen from Gerber et al.18 to formulate a 2D, multi-aspect CNN-based framework for inner ear segmentation. The authors introduced this as AutoCasNet, a three-stage cascade of networks specialized on the iterative segmentation of coronal, axial and sagittal aspects of the data, respectively. The authors operated and compared ResNet, UNet and SEUNet  as backbones. No other design choices were tested, while validation and testing was done via 4-fold CV without an independent test set. The formulation incorporated largest region extraction to postprocess the network predictions. No data augmentation was described for this formulation.The work by Hussain et al.20 presented a shape-regularized preoperative pipeline that segments multiple anatomical risk structures from volumetric temporal bone scans to facilitate subsequent trajectory planning. In their first study, a triplet of multi-aspect 2D UNets, similar to the approach by Hussain et al., was used to segment the structures. In contrast to Hussain et al., who adopted AutoCasNet as a sequential cascade of convolutional neural networks, they made use of parallel inference for the axial, sagittal and coronal aspect-wise UNet instances and combined the result using a majority-voting strategy. Follow-up work21 replaced the parallelized multi-aspect set of 2D UNets with a two-staged 3D UNet implementation. There, the primary stage extracted a volume of interest, from which the secondary stage 3D UNet with identical architecture produced the segmentation of the anatomy. Both described systems shared key features. They exploited registration to probabilistic active shape models (PASM) as a separate downstream postprocessing step after network predictions. Further, both were formulated using data from 24 clinical volume CT scans without data augmentation. The validation procedure consisted of a two-fold cross validation strategy without an independent test set.Fauser et al.A summarizing table of all related works concerned with deep learning based automated analysis of temporal bone scans is provided by Table 22 described a pipeline based on a modified 3D UNet, achieving an inner ear segmentation Dice score of 0.86(1) (mean and standard deviation (SD)) for the validation dataset and 0.87(1) for the test dataset. Ahmadi et al.23 employed a 3D V-Net24 based pipeline in conjunction with an atlas-based pre-localization and pre-segmentation step to achieve a Dice score of 0.90(2) for the test dataset. These presented studies are concerned with segmentation tasks in a mono- or multi-class setting.In addition to the presented works operating with CT imaging modality, further studies with volumetric magnetic resonance imaging (MRI) data were conducted recently. We refer to two studies that engaged with inner ear segmentation based on MR data generated by T2-weighted sequences. Notably, Vaidyanathan et al.25 and has been applied successfully in a large variety of tasks in medical imaging26, including brain image analysis27 and retinal image analysis28. For inner ear analysis, multi-task learning was used for reducing metal artifacts in post-implantation CT images29. To further contextualize the presented work with multi-task joint segmentation and landmark detection, we look to craniomaxillofacial (CMF) applications, since for inner ear analysis and otological applications, multi-task learning has not been applied extensively. For example, Liu et al.30 presented a joint segmentation and landmark detection framework for CMF preoperative planning. The automated task comprised the segmentation of midface and mandible bone structures and the localization of 175 bone, teeth and face landmarks. The design reinterpreted landmark localization as multi-class segmentation by casting landmarks as individual classes in the spirit of semantic segmentation. The proposed framework leveraged a layered setup of a quadruplet of specialized 3D UNets operating on different image resolution scales. A primary stage provided a coarse segmentation and coarse-global localization of bony and facial landmarks only, while a secondary stage comprised two refinement models that handled bone segmentation and landmark detection separately. The secondary stage then produced refined segmentations and localization results, using specialized bone and tooth models from the purpose-cropped volume of interest inputs of the primary stage.This work, however, proposed the segmentation and landmark detection as an integrated multi-task problem, since a single, bi-headed network performs both segmentation and landmark detection using a multi-task learning approach. Multi-task learning can produce improved results compared with learning each task separately31 proposed a context-guided UNet based framework termed JSDNet for joint midface and mandible bone segmentation and localization of 15 landmarks for CMF applications. The architecture consisted of a cascade of two sub-networks, subsequently termed FC1 and FC2. The first network was trained to estimate displacement maps from landmark positions, while the second network conducted the joint prediction of the bone segmentation maps and landmark heatmaps using CT scan data concatenated with displacement map data as inputs.Zhang et al.https://doi.org/10.5281/zenodo.8277159) the last contribution element.The main purpose of our project was the development and validation of a fully automated pipeline to jointly generate segmentation maps of inner ear structures and the localization of landmarks from temporal bone CT scans. By framing the problem as a multi-task learning problem, we implemented an end-to-end trained solution algorithm based on the U-Net architecture. In this work, we provide contributions in the following aspects: (1) describing a performant and end-to-end differentiable U-Net pipeline that is evaluated against a range of in-house and open source data from clinical and specimen origin, (2) directly comparing the formulation with single-task UNets, (3) comparing the formulation to recently published state-of-the-art methods with temporal bone analysis applications and to our reimplementation of a joint segmentation and localization architecture from the craniomaxillofacial application space, (4) providing a series of ablation studies elucidating the performance impact of various design choices and (5) releasing the labeled training and validation dataset. The existence of labeled data is vital to the principally data-scarce and restricted field of medical imaging, making the open-source release of the labeled in-house datasets on the Zenodo platform  was comprised of scans of 43 cadaveric temporal bone specimen and was split into two subsets for training (Dataset B (in-vivo) was comprised of in-vivo from pre-operative patients from clinical practice. The data for the clinical dataset B were processed in an entirely anonymized and retrospective fashion. The retrospective anonymized study was conducted in concordance with local guidelines and principles of the Declaration of Helsinki and Good Clinical Practice34 and was approved of the Ethics committee of the University of Würzburg (2019020401). The requirement for informed consent from the study subjects was waived by the Ethics committee of the University of Würzburg due the anonymous data analysis and Article 27, Paragraph 4.1 of the Bavarian hospital law (BayKrG), which permits the use of patient data within the hospital for research purposes or in the research interest of the hospital without the patient’s consent being necessary. The instances of dataset B were exclusively used as a hold-out set and main gauge for the performance of the methodology. From the total 10 dataset elements, a single instance showed otosclerotic lesions and was evaluated separately. This fact is communicated through the 9 + 1 element notation for dataset B. For the described datasets A and B, the inner ear structure was manually annotated under the supervision of a cochlear implant surgeon (K.R.) with extensive experience in cochlear analysis. The segmentation and landmark position annotation was performed with the 3DSlicer software35, an open-source software tool for medical image analysis. The segmentation process comprised manual labeling via brush-based painting assisted by masks generated via voxel intensity thresholding. The foreground labels thus correspond to the inner ear or labyrinth fluid space that is bounded by bone and the membranes round and oval window. As anatomical landmark labels, the positions of helicotrema, oval window and round window were marked as a voxel coordinate vectors. Dataset A is released as open-source at the Zenodo repository (DOI: 10.5281/zenodo.8277159) as part of this work.Design, training and evaluation was performed on the basis of two in-house datasets. Both were scanned on a Siemens Axiom-Artis DynaCT fpVCT scanner  using commercially available software . The following acquisition parameters were utilized: 20s DCT Head protocol, tube voltage 109 kV, tube current 21 mA, pulse length 3.5 ms, rotation angle 200Dataset W by Wimmer et al.36 consists of Dataset O by Sieber et al.37 contains Dataset G by Gerber et al.18 provides in-vivo fluid-filled volumes and other structures. The preparation method also erased the round and oval window membranes  of dimension x, y and z direction.For the open-source datasets, we integrated a histogram and contrast matching scheme in addition to the above clipping and rescaling computation. This was a key aspect to correct for the fluid-space-wise contrast modifications introduced by the various preservation schemes of the datasets W , O  and G (drying). Dataset A from frozen specimen exhibited well-preserved contrast for normally fluid-filled cavernous structures and membrane regions like the oval and round window compared to our clinical dataset. Another open-source dataset processing element was the application of the clipping and normalization procedure separately for each cluster visible in Fig. To enhance both effective training data amount and variability, the traditional form of training time augmentation was performed by stochastically applying a variety of augmenting transformations to a chunk. We used both spatially and intensity-wise transformations in our protocol. For the intensity-wise augmentation, we randomly adjusted image contrast by multiplication with a scaling factor and by application of additive Gaussian or Poisson noise. Lastly, a free angle rotation of the data about a randomly chosen spatial axis was performed. During our ablation studies, we employed two different variations of the protocol. For augmentation protocol Aug38 to be an appropriate architecture for automated medical image analysis, outperforming many alternative techniques. Thusly, a fully convolutional and deeply supervised U-Net model was assembled to solve the joint segmentation and localization problem. Consisting of a common encoder block, two task-specific decoder blocks and skip inter-connections, the contracting encoder path hierarchically extracts high level information while decreasing spatial resolution through strided convolutions. For the dual task of segmentation and landmark localization, two separate decoder heads were envisioned that both draw upon the contextualized high level features extracted by the common encoder backbone. The expanding decoder path successively re-up-samples the feature maps and utilizes the extracted feature information, supplemented with localized details via the skip connections, to segment the voxels into semantic classes and localize landmark positions via heatmap regression, respectively. A visual overview of the utilized 3D U-Net architecture is provided in Fig. 39, a default network architecture with four levels and a double convolution module as the basic building block, consisting of a normalization layer, a For the inherently volumetric fpVCT data, the 3D U-Net architecture was shownsegmentation head performed the voxel-wise classification problem by estimating a pseudo-probability map heatmap head was the second specialized decoder structure that is attached to the common encoder backbone and performed the landmark localization task via the intermediate step of heatmap regression k denotes an index for the landmark, i.e. apex of the cochlea and oval window and round window. The heatmap head is depicted in the architecture diagram of Fig. 41.Gaussian function on the voxel grid with an amplitude 42 optimizer with the weight decay strength hyperparameter The entclass1pt{minimaentclass1pt{minima43, leaky ReLU nonlinearity with slope parameter 0.025, heatmap scheduling and combined Dice-Cross-Entropy-Loss with deep supervision. Starting from this fixed basal configuration, we mono-parametrically changed the following aspects: normalization layer, loss function, encoder depth and channel dimension size, augmentation strategy, activation function and attention gating . For the normalization layer we tested against batch norm. For the loss function we tested the segmentation part against pure Dice loss, squared Dice Loss24, LogCoshDiceLoss44, pure binary cross entropy loss and against disabled deep supervision. For the training data augmentation scheme we tested against an enhanced scheme . For the AdamW settings, we tested against different weight decay parameters We implemented the described modular and dual headed deep neural network architecture in PyTorchWe used Dataset B  and Intersection-over-Union 51 and the volumetric similarity score (VS)52 for the evaluation of segmentation predictions. In addition to the overlap-based metrics, we utilized the Hausdorff distance (HD) as a contour- and spatial distance based evaluation criterion to take the spatial structure and boundary delineation of the segmentation into account. In addition to the default Hausdorff distance that is susceptible to outliers, we calculated the average Hausdorff distance 53. For the ablation experiments, HD and The prediction of the joint segmentation and detection algorithm was evaluated by computing a variety of evaluation criteriaBaseUNet architecture with its multi-task counterpart and our reimplementation of the JSDNet by Zhang et al.31. The results for the overall segmentation and localization performance are shown in Fig. The U-Net was trained using the training protocol outlined above from random initialization until the preset number of iterations was completed. The training loss generally exhibited stable reduction to a low value, while the validation performance reached a high and consistent plateau value  and multi-task (4.8 voxel units) was visible already on the validation data. The superior performance of the multi-task solution continued for test dataset B (5.6 voxel units) and test dataset average (7.2 voxel units). The loss of localization precision was larger for the single-task BaseUNet with 6.3 voxel units and 10.0 voxel units. JSDNet trained with augmentation had comparably larger localization deviations with 7.3, 9.4 and 24.0 voxel units for validation, dataset B and test dataset average respectively. The JSDNet instance that was fitted without training augmentations displayed even bigger landmark localization deviations with 10.6, 28.6 and 28.9 voxel units for the dataset triplet.The next results can be gleaned from the ablation studies. For this, we conducted and evaluated additional 43 experiments with three-fold CV, where mono-parametric variations of the foundational BaseUNet architecture or training protocol were evaluated. Performance summary plots of all ablation experiments are provided in the supplementary materials Sect. gold-status network architecture via a manual and greedy collection of task-optimal design choices using insights generated from the ablation study. The resulting architecture encompassed global attention gating, instance normalization, Mish nonlinearity, augmentation protocol AugWe then produced the final resulting }= {0.89}$$\end{doFor dataset O, the selected instance shown in Fig. Apart from the qualitative considerations, the efficiency benefits of the proposed solution can be stated. In this study, the time to manually and accurately label the relevant anatomical structures and landmarks in a new dataset instance was approximately 600 s using the described threshold-based masking and manual painting method. For the basic, GPU-accelerated PyTorch implementation, the average prediction time amounted to 50 s for a typical input volume with a TTA strategy of ten augmenting transformations and 25 voxels overlap per sub-volume chunk. This time included loading the dataset instance from disk.In addition to the evaluation of the 9 clinically regular instances of dataset B, the single otosclerotic dataset instance was also processed with the described pipeline and algorithm. A significantly lowered segmentation performance as well as landmark localization performance compared to the regular instances of dataset B was observed. The ground truth segmentation map is depicted in Fig. Motivated by the resource constraints and drawbacks of manual analysis of volumetric radiological temporal bone images, an automated pipeline-based framework to jointly produce inner ear segmentation maps and localize three important anatomic landmarks without the requirement of user interaction was developed. After loading the a dataset instance, no further user interaction was necessary for the production of a scan analysis that consisted of the prediction of the inner ear anatomy  and the coordinate-wise localization of the three important anatomical landmarks .The primary objective of machine learning is to optimize a function with limited training data that generalizes well to genuinely new and unseen data instances.The described results provide an interesting evaluation due to the fact of being trained on frozen cadaveric specimen only, while being tested on scans from clinical practice  with AutoCasUNet17. The proposed framework was able to predict segmentation maps with a Dice score of 0.91(2) for the dataset G in a hold-out test setting, enabling direct commensurability to Hussain et al., who also employed dataset G as core training and validation data.When compared with the catalogue of related works presented in Table 54 should be created for inner ear and temporal bone segmentation. This strategy would help to disentangle performance influences that are not genuinely attributable to pipeline and algorithm. Possible candidates therefore are e.g. the voxel size, labeling quality and basal CT imaging setup.We note however that for performance results to be commensurable in all aspects, a single authoritative benchmark dataset like the medical segmentation decathlon datasetFor the localization task, the method was able to provide highly accurate localization of all three anatomical landmarks  and b) in Fig. However, we noted that the global volume-mean standard deviation per voxel is substantially larger than the corresponding value for all non-pathological dataset instances. In Fig. The 10-fold reduction in analysis generation time in comparison to manual labeling is an impressive demonstration of the efficiency gains of the automated pipeline. Using task automation, large scale and multi-centered clinical studies are within reach that may have been unattainable with the earlier lengthy manual methodology. Rapid generation of patient-specific anatomic analysis also facilitates the integration into routine pre- and intra-operative planning for cochlear implant surgeons. The prediction time of the pipeline can potentially be minimized even further by providing a more optimized implementation. For example, a parallelization strategy of the preprocessing TTA augmentation computation could prove feasible. The basal model inference time could be reduced by network pruning and sparsification of the U-Network or by applying weight quantization.15 presented a blinded review with seven expert participants, where automated segmentations with a Dice overlap score of 0.91 were rated highly for accuracy. Since the described pipeline surpasses this segmentation performance, the results are encouraging with respect to real world applications. Concerning the localization performance and the potential future use for automated cochlear duct length (CDL) measurements, we note that for clinical applications the translation of the error propagation from helicotrema and round window localization into the final CDL is relevant. The proposed framework was able to locate helicotrema and round window in clinical data with a mean deviation of 6.4 and 4.5 voxel units. Using a cursory worst-case error analysis with physical units  CT datasets that do not suffer from the inner ear contrast alterations and rotational differences observed in the preserved specimen open-source dataset and thus requiring enhanced processing is desirable.bootstrap a larger collective of datasets via prediction of newly recorded volumes that get quickly reviewed, refined and reintroduced into the training dataset. In our evaluation, we focused on healthy subjects and explicitly treated a pathological dataset instance as out-of-distribution. Special care should be given to the collection of more pathologically altered datasets (e.g. otosclerosis and fibrosis) to better understand the out-of-distribution performance of the model on these dataset instances. The data sourcing for these cases is often even more difficult than for non-pathological cases and may necessitate further specialized data augmentation or transfer learning strategies.The bottleneck for the labeling of more in-house instances was the time-intensive process of manual segmentation. The high segmentation and localization performance of the framework may provide potential to benchmark dataset for temporal bone segmentation and landmark detection, improving commensurability and reproducibility of the various automated methodologies.Another considerable limitation is the absence of fully authoritative ground truth data concerning the segmentation maps and the landmark positions. The ground truth data of datasets A and B was manually generated by a researcher trained and supervised by a qualified clinical researcher and otological surgeon. Subsequently, intra-observer variability may exist in datasets A and B that effect a systematic labeling difference with the external open source dataset catalogue. Incorrect or inconsistent segmentation masks that are considered the absolute ground truth could thusly lead to a low DSC evaluation for the automated prediction. In this possible scenario, the lower DSC would paradoxically indicate a better automated prediction in comparison to the manual annotation. A viable remedy would be a general input data improvement strategy using a multitude of anatomical experts that jointly segment all dataset instances, from which a gold standard and consensus ground truth could be distilled. Such an endeavour could potentially culminate in the development of the earlier mentioned authoritative 57 within a unified fully automated pipeline as a work target. The inclusion of transfer learning and study of learning from multi-modal imaging data including MRI strikes us as another salient line of research. Our future investigations and enhancements to the methodology will focus on these beneficial objectives.Directly addressing the mentioned data limitation, a very worthwhile further line of work could be a scope-maximized evaluation study using more temporal bone CT data. This would ideally entail datasets exhibiting realistic inner ear fluid space contrast and volume orientation according to clinical standard operating procedure. Using the clinical CT data generated by the related works summarized in Table The presented automated 3D U-Net framework and data processing pipeline provided highly accurate segmentations of the inner ear structure combined with good localization of the anatomical landmarks helicotrema and oval and round window via heatmap regression.The practical applicability was demonstrated via the evaluation on representative flat-panel volume CT data with secondary reconstruction sourced from actual clinical practice. Further robust generalization skill was shown on a three-element catalogue of processed open source datasets.The designed model incorporated state-of-the-art deep learning techniques and was implemented with a modular architecture using open source software libraries. Conducted mono-parametric ablation studies illuminated the performance impact of architectural design choices and training protocol hyperparameters and allowed the inference of task-optimal parameters. The bi-headed U-Net based core architecture was end-to-end trainable without postprocessing, enabling insights into the raw performance of deep learning based segmentation and landmark localization models. Ablation studies of the bi-headed solution against single-task formulations yielded a clear multi-task performance benefit for landmark localization. For segmentation performance, equal automated model skill with a small loss in generalization skill to open-source data was visible. In comparison to related works, we were able to show a superior joint segmentation and localization ability. With its encouraging evaluation results, the automated methodology yielded good generalization ability such that the proposed, proof-of-concept framework may possess—with significant improvements and further validation—the potential to be utilized in both clinical practice and research. Immediate, tangible applications of the segmentation and localization results comprise e.g. preoperative planning, intraoperative visualization and large scale exploratory clinical research.Supplementary Information 1.


Cell-state density characterizes the distribution of cells along phenotypic landscapes and is crucial for unraveling the mechanisms that drive cellular differentiation, regeneration, and disease. Here, we present Mellon, a novel computational algorithm for high-resolution estimation of cell-state densities from single-cell data. We demonstrate Mellon’s efficacy by dissecting the density landscape of various differentiating systems, revealing a consistent pattern of high-density regions corresponding to major cell types intertwined with low-density, rare transitory states. Utilizing hematopoietic stem cell fate specification to B-cells as a case study, we present evidence implicating enhancer priming and the activation of master regulators in the emergence of these transitory states. Mellon offers the flexibility to perform temporal interpolation of time-series data, providing a detailed view of cell-state dynamics during the inherently continuous developmental processes. Scalable and adaptable, Mellon facilitates density estimation across various single-cell data modalities, scaling linearly with the number of cells. Our work underscores the importance of cell-state density in understanding the differentiation processes, and the potential of Mellon to provide new insights into the regulatory mechanisms guiding cellular fate decisions. Cell differentiation is a dynamic process that underpins the development and function of all multicellular organisms. Understanding how cells are distributed along differentiation trajectories is critical for deciphering the mechanisms that drive cellular differentiation, pinpointing the key regulators and characterizing the dysregulation of these processes in disease. Cell-state density is a representation of this distribution of cells and is impacted by biological process spanning proliferation to apoptosis –C. For i3. Rapid and coordinated transcriptional acceleration leading to low-density transitory states connecting higher-density regions have been demonstrated to be a hallmark of developmental progression in diverse biological contexts from plants to humans7. Rare transitory cells have also emerged as critical entities in the processes of differentiation1, reprogramming8, and the emergence of metastasis9. Despite the central importance of cell-state density, current approaches for density estimation in single-cell data often produce noisy results and struggle to provide biologically meaningful interpretation 14 due to its consideration of potential cell-state transition trajectories.Mellon’s utilization of neighbor distances and inference of continuous density function is underpinned by two well-established principles of single-cell analysis. First, Mellon assumes that distances between cells in the chosen representation of the phenotypic landscape are biologically meaningful and thus represent a valid measure of cell-to-cell similarity. We refer to such a space as 17.The second assumption Mellon relies on is that density changes from cell-to-cell are smooth and continuous in nature i.e., Mellon assumes that cells with high degrees of similarity possess similar densities. The inherent molecular heterogeneity of cells, primarily due to the subtle differences in gene expression, supports these smooth density transitions. Further, single-cell studies have revealed that cells experience gradual, rather than abrupt, changes in gene expression, providing empirical support for this assumptionMellon first computes distance to the nearest neighbor for each cell in the cell-state space. We then capitalize on the stochastic relationship between density and neighbor distances, where cells in higher density states tend to exhibit shorter distances to their nearest neighbors, whereas cells in lower density states tend to have longer distances . Formall19. GPs are thus ideally suited for density estimation from noisy single-cell data. They achieve their robustness by incorporating the smoothness assumption through a covariance kernel, facilitating sharing of information between adjacent observations. In Mellon, the covariance kernel of the GP encodes cell-state similarity and determines the influence of nearby cells on density estimates at a specific state  in a Bayesian model to approximate this function without assuming a specific functional form . GPs areic state . This coic state –F. Finalic state .regions – connected subsets within the cell-state space with similar density characteristics. Such regions represent collections of closely related cell states that cells appear to inhabit (high-density regions) or traverse (low-density regions) during their differentiation journey , provides an ideal paradigm to understand and model differentiationntiation  to evalu22. Intriguingly, we observed noticeable fluctuation in density within several cell-type clusters, suggesting an inherent heterogeneity, a nuance often masked when cells are grouped together by clustering methods  to fate committed cells (another high-density region) is in part driven by enhancer priming. In this scenario, the loci of lineage-specifying genes are maintained in an accessible state in HSCs through pre-established enhancers. A combination of cell-autonomous and extrinsic factors trigger the upregulation of a small set of master regulators, which in turn rapidly upregulate the expression of lineage-specifying genes in a coordinated manner. Thus, the combined activity of pre-established enhancers in HSCs and lineage-specific enhancers established by master regulators could produce the rapid transcriptional changes that underpin rare transitory cells in low-density states .36. Therefore, we devised a procedure to disentangle primed and lineage-specific peaks associated with a gene using different levels of abstractions  (5.We used the mouse gastrulation atlas.5-E8.5)  for expl.5-E8.5) . Interes.5-E8.5) , highligMellon’s capacity to generate a continuous density function enables it to estimate densities for cell states that were not part of the training data . To demotime and cell-state. Therefore, we can estimate cell-state densities at any desired timepoint situated between the measured instances 21, mouse gastrulation (116k)44 and the iPS reprogramming dataset (250k cells)8 spanning datasets of different sizes and characteristics. For example, using a single CPU core and a default of 5000 landmarks, Mellon required only 100 seconds to process 10k cells and 17 minutes to process 100k cells of the iPS dataset, including the computation of diffusion maps . The Euclidean distance between these cell-states serves as a measure of biological dissimilarity. Mellon outputs a solution –F. The dThe Mellon framework contains the following major components:Mellon first calculates the distance to the nearest neighbor for each cell in the cell-state space, following the first assumption.Nearest-Neighbor Distribution  priors to establish a relation between the densities of highly-similar cell-states, facilitating a continuous density function estimation. Similarity between cell-states is encoded using the covariance function of the Gaussian process. The random variable of the GP, denoted as The GP is defined as follows:The true log-cell-state density approaches negative infinity away from any observed cell state. However, functions sampled by the Gaussian process approach the chosen mean. To approximate the true behavior of density functions we choose a very small value for the mean 19. We therefore chose to use the Matern covariance function with Similarities between cell-states are encoded through the GP covariance function or kernel. Specifically, the kernel function defines the covariate structure between cell-states which translates to the smoothness of the density function. Some commonly used kernels are arbitrarily smooth and allow arbitrary differentiability. Assuming such smoothness can, however, lead to unrealistic resultsThe covariate structure of the Gaussian process is governed by the length scale parameter, denoted as estimate . In areaestimate .This method not only increases the reliability and robustness of density estimates, but it also enables the creation of a continuously changing density function between cell states, offering a nuanced representation of biological phenomena. Unlike the k-nearest neighbor methods for density estimation that assign an equal weight to all k neighbors irrespective of their distances, the continuous covariance function of the Gaussian process accounts for the distance between cells, smoothly adjusting the weight of their contribution. The resulting impact on the local density estimate facilitates a more precise representation of the cell-state landscape.The ideal length scale strongly depends on the availability of data at different points of the cell-state space and encompasses a specific amount of cells needed to support a reliable density estimate of a given state. We therefore derived a heuristic for length scale as function of the mean nearest neighbor distance between cells:Here, Gaussian process computation usually necessitates The sparse GP in Mellon is constructed using a subset of data points, referred to as “landmark cell-states,” that essentially act as inducing points. These landmark states are chosen to capture the essential structure of the cell-state space, providing a representative skeleton for the full GP model. This sparse GP approach translates to an efficient The specifics of the sparse GP implementation play a crucial role in the overall performance of the Mellon and are described in 58. We assessed the accuracy of this approach by comparing the inferred density derived from the landmarks against the density function inferred from a non-sparse, or “no-landmarks” version  compilation using XLA (Accelerated Linear Algebra), a linear algebra compiler developed by GoogleMellon’s scalability to large single-cell datasets is ensured through the use of a Sparse Gaussian Process (GP). The sparse GP allows us to approximate the full GP model, significantly reducing the computational demands while retaining the essence of GP’s expressiveness. This scalability  makes MeFinally, model tractability in Mellon is achieved through the adoption of a length-scale heuristic for the GP covariance function. The covariance function is crucial in a Gaussian Process as it dictates how many nearby points in the input space influence each other in the output space. Typically, the length scale of this function is subject to inference or optimization, often involving computationally intensive iterative processes that require repeated updates of the covariance matrix and its Cholesky decomposition. In Mellon, we sidestep this computational demand by deriving an appropriate length scale with a data driven approach designed to adapt to the varying local densities in the high-dimensional cell-state space . This noTogether, these components create a balance between computational efficiency and model expressiveness, making Mellon an effective and practical tool for cell-state density estimation from large, high-dimensional single-cell data.Mellon, by default, employs the L-BFGS-B optimization algorithm to infer the maximum a posteriori (MAP) estimates of the posterior likelihood. Notably, our implementation provides direct access to the posterior distribution of the density function. This is realized through a JAX function with automatic differentiation, thus facilitating the use of any preferred inference scheme while retaining computational simplicity.This flexibility is crucial because, in Bayesian inference, the MAP estimate can be subject to the transformation of the latent representation and might not necessarily represent the “true” underlying cell-state density. In fact, empirical evidence  indicateIn essence, Mellon’s versatile implementation provides a robust framework for density estimation that can adapt to diverse inference schemes, offering users the freedom to employ the technique best suited to their specific study.11, as implemented in Palantir14, as the representation of cell-state space. Diffusion maps have been widely used in single-cell data analysis owing to their reliable and robust representation of cell-states14. Cellular states in phenotypic landscapes reside in substantially lower dimensions compared to measured gene expression owing to gene regulatory networks inducing a strong covariate structure amongst genes. Therefore, biological similarity between cell states is more closely linked to the distance they can traverse along the phenotypic landscape, rather than solely their direct proximity in gene expression space. Diffusion maps identify the intrinsic structure in single-cell data, mitigating noise by treating the data as realizations of a stochastic process. They not only efficiently reduce noise in single-cell data but also extract a faithful representation of the underlying cell-state manifold.Mellon utilizes diffusion componentsFurther, the distances computed using diffusion maps, termed diffusion distances, are a measure that reflects the interconnectedness of data points along the phenotypic manifold. Importantly, diffusion distance operates along this manifold, which is constructed from the observed cell states, thereby providing a meaningful indicator of biological similarity between cells. Therefore, the use of diffusion distance in the estimation of cell-state density leads to a biologically relevant quantification of cells sharing a similar state.36 or multimodal representations51.Diffusion maps can be constructed for different single-cell data modalities with appropriate preprocessing. We recommend the use of PCA for RNA and SVD for ATAC and histone modification data. “14.The dimensionality of the subspace where the data is represented is determined by the number of diffusion components utilized in Mellon. Mellon results are robust to the number of diffusion components indicating that pinpoint precision in their selection isn’t strictly necessary . HoweverLow-density regions representing rare transitory cells are critical for diverse biological processes. We devised a gene change analysis procedure to identify genes that drive cell-state transitions in low-density regions and thus can be used to describe the dynamic behavior of the biological system. The input is a relevant set of cells We first compute a measure of local variability of a gene for each cell-state: We compute the expression change from a cell when transitioning to each of its neighbors and normalize the change by the distance between the cells in state space to account for the magnitude of the state transition. The maximal normalized change amongst the neighbors of the cell is nominated as the local variability of the gene for the corresponding state. Formally, the local variability for gene We next compute, a low-density change score th percentile are considered to be driving low-density changes  to B-cell fate committed cells (proB) . We used21 to identify metacells from the T-cell depleted bone marrow. SEACells aggregates highly related cells into metacells overcoming the sparsity in single-cell data while retaining heterogeneity. We used the ATAC modality of the multiome data to identify metacells. We used metacells to compare the expression of a gene with the accessibility of each peak in a window of 100kb around the gene to identify the subset of peaks that significantly correlate with expression of the gene   .60 to perform differential accessibility with metacell counts as input. The use of metacells rather than single-cell data for differential accessibility has been demonstrated to provide better sensitivity and specificity37. To identify peaks that are relevant to the B-cell lineage, we compared accessibility in pro B-cell metacells and metacells of the erythroid (EryPre1) or monocyte (Monocyte) lineages and retained peaks with the accessibility fold-change log2FC > 0 in either comparison. While this ensures that the selected peaks have greater accessibility compared to other lineages, it does not exclude ubiquitously accessible peaks. We therefore excluded peaks with log2FC < 0.25 in the comparison between stem-cells (HSCs) and erythroid and monocyte lineages.Metacells and gene-peak correlations were computed using all hematopoietic lineages in our dataset. We performed differential accessibility analysis to identify the subset of peaks with greater accessibility in the lineage under consideration. We used edge RThe resulting set of peaks demonstrate substantially greater accessibility in B-cell lineages compared to all other cell-types –D21. The mean of the Poisson distribution for a cell-type After identifying peaks with greater accessibility in the B-cell lineage, we assigned primed or lineage-specific status to each peak with a simple logic: A peak is annotated as primed if it is accessible in HSCs and lineage-specific if it is not. Accessibility in HSCs was determined using Poisson statistics as described in SEACellsWhere We utilized the primed and lineage-specific peaks to derive primed and lineage-specific scores for the associated genes at single-cell resolution. For each gene Where 61 to preferentially weight peaks which are highly accessible in a small proportion of cells. The MAGIC algorithm2 was then used to perform imputation using normalized accessibility as the input.Given the sparsity of the scATAC data, we used imputed peak accessibility for computing scores. The peak counts dataset was TF-IDF normalizedAccessibility trends along pseudo-time were computed using Mellon. Trends are visualized as a percentage of the maximum value of each trend, to allow for better comparison across genes.We applied primed and lineage-specific accessibility scores to characterize commitment of hematopoietic stem cells to B-cells using the T-cell depleted bone marrow multiome data. We used hematopoietic stem cells (HSC), hematopoietic multipotent cells (HMP), common lymphoid progenitor (CLP) and pro B-cells along the B-cell lineage to investigate the open chromatin landscape . The cel21 to identify metacells using the ATAC modality of the T-cell depleted bone marrow data. Metacells were identified using all cells, resulting in 115 metacells according to recommended heuristic for selecting the number of metacells. Peak accessibility and gene expression correlations were determined using all metacells and the subset of genes with at least 5 peaks were selected for downstream analysis  data in addition to motif enrichment by correlating the expression of a TF to the accessibility of a peak. A combination of a high gene-peak correlation and high motif score is more indicative of potential TF binding compared to a peak with only a high motif score 37. We used our Python adaptation of in silico ChIP-seq using the SEACells metacells as input (github.com/settylab/atac-metacell-utilities). FIMO62 was used to associate TF motifs with ATAC-seq peaks, resulting in a peak by TF matrix of scores indicating the strength of match of the TF motif in the peak sequence. In silico TF binding scores are computed as product of correlation between TF expression and peak accessibility and FIMO motif scores as follows:We used in silico ChIP-seqWhere Minmax normalization is performed as follows:The final in silico ChIP-seq output is a peak by TF matrix, containing a value between −1 and 1 indicating how likely a TF is to bind at a given peak and whether it has a repressive (negative) or activating (positive) effect, or 0 if a peak does not meet the minimum in silico ChIP-seq score (0.15).Peaks correlated with EBF1 expression were ordered using the procedure outlined in the section “Time-series single-cell datasets provide snapshots of the changing cell-state densities at discrete time intervals. Our goal is to compute a time-continuous density function to interpolate cell-state densities at any time between the measured timepoints.We therefore incorporated a time coordinate into the Gaussian process used to generate the log density function and use the covariance of the Gaussian process to link temporally similar cell-states. Effectively the covariance function of time-continuous density has two components: (i) similarity between cells in the cell-state space and (ii) similarity between cells based on their measurement times. Similarity in cell-state space is encoded through the Matern52 kernel with the length-scale parameter as described in all cells from all timepoints, and computed a correlation of cell-state density between timepoints:The length scale should be designed such that the covariance between cells from different timepoints reflects the covariance of densities between those timepoints. Therefore, we optimized the length scale to reflect the empirically observed covariance of density functions between different time points. Specifically, we employ Mellon to compute first time-point specific density functions Where points T –D:P:=pThis matrix Where The optimized length scale is used for the Matern52 covariance kernel for the time coordinate, denoted as tern52lt –F.The resulting covariance kernel for cells Where This construction is easily implemented with Mellon, since it is designed to support any combination of covariance functions, each operating in distinct active dimensions – in this case, either time or cell-state coordinates.Using this covariance function, Mellon can compute a continuous density function over time and state space using all samples, and thus can be used to interpolate cell-state densities at unmeasured timepoints. This function is also differentiable in time and state space, and the change in density over time can be determined using the first derivative .We validated the effectiveness of the time-continuous density function using a leave-one-out cross-validation strategy . We comp14 to derive fate propensities for all cells and selected the subset of cells with high propensity towards a particular fate.Time-continuous density provides a platform to decipher the dynamics of cell-type proportions and fate choices in true temporal order. As proof of principle, we investigated the cell-type proportion dynamics along the trajectory of a particular lineage. We first used PalantirIn the mouse gastrulation data, we applied Palantir using all cells across all timepoints and selected cells which specify the erythroid cells . PalantiThe Palantir fate probability of cell state In this equation, ith cell, and In the above equation, We next determined the joint cell-state density between pseudo-time and real time leveraging the time-continuous density function. We first used Gaussian process as implemented in Mellon to map pseudo-time to each coordinate of the cell-state space. This effectively generates a trajectory traversing the cell-state space by mapping the 1-dimensional pseudotime to high-dimensional cell-state space. Formally, the trajectory for each dimension Where Finally, the time-continuous density function l time t .We used the joint cell-state density The cell-type annotation pseudotime We next marginalized the joint cell-state density Finally, the relative proportion of cell type This provides a quantifiable measure of cell type proportions over time, offering valuable insights into the temporal evolution of cell types in a given biological system.44 across all measured time points: E6.5, E6.75, E7.0, E7.25, E7.5, E7.75, E8.0, E8.25 and E8.5. Data was preprocessed as described in section “Mouse gastrulation data in 14. We selected 25 components, as they encapsulated all significant biological variations. Density results remained stable beyond this point with respect to the number of components . Joint cell-state density over pseudo-time and real-time were visualized using 200 points along pseudo-time and 500 points between every pair of measured timepoints.Palantir63 package.The following procedure was used for preprocessing scRNA-seq data across datasets unless specified otherwise: Raw counts were normalized by dividing the counts by the total counts per cell. The normalized data was multiplied by the median of total counts across cells to avoid numerical issues and then log-transformed with a pseudocount of 0.1. Feature selection was then performed to select the top 2500 most highly variable genes, which was used as input for principal component analysis with 50 components. PCs were used as inputs for leiden clustering and UMAP visualizations. Preprocessing and analysis was performed using the scanpy14 package with default parameters and PCs as the inputs. The diffusion kernel was also used for MAGIC2 gene expression imputation.Diffusion maps were computed using the Palantir64. Batch corrected PCs if applicable were used as inputs for UMAPs, diffusion maps, and imputation.Batch correction where applicable was performed using Harmony with default parameters65.Raw gene counts, ATAC fragment files and cell metadata were downloaded from21.scRNA-seq data was processed using the procedure described in section “64. This dataset is enriched for stem and progenitor cells and thus the associated cell-type information can be utilized to better resolve the cell-types within the HSPC cluster of the T-cell depleted bone marrow data. Batch corrected PCs were used for leiden clustering, and the HSPC cluster of the T-cell depleted data were assigned to different stem and progenitor cell-types based on their clustering with the CD34+ bone marrow data. Clusters associated with the B-cell trajectory were annotated using the markers described in66.All hematopoietic stem and progenitor cells (HSPCs) were grouped as one cell-type in the T-cell depleted bone marrow. To achieve higher granularity among the stem and progenitor cells, we integrated this data with a dataset of CD34+ bone marrow cells using Harmony67 and signature scores were computed by averaging the z-scored imputed gene expression.Mellon was applied with default parameters using 20 diffusion components to compute cell-state density. Gene change scores, primed accessibility scores, and lineage-specific accessibility scores were computed as described above. IL7R signaling targets were downloaded from Nichenet36 pipeline was used for analysis of the ATAC modality. In ArchR, data was normalized using IterativeLSI and SVD to determine a lower-dimensional representation of the sparse data. The first SVD component showed greater than 0.97 correlation with log library size and was excluded from downstream analysis. SVD was used as input to cluster the data with leiden and visualization using UMAPs. SVD also served as input for computing diffusion and MAGIC imputation of peak accessibilities and gene scores. Peak calling was performed within ArchR using only the nucleosome free fragments as described in21.ArchRA handful of cells which passed the RNA QC thresholds did not clear the thresholds in the ATAC modality. RNA preprocessing and analysis was repeated after excluding these cells. Mellon was applied with default parameters using 20 diffusion components to compute cell-state density of the ATAC modality.14 was used to infer pseudo-temporal trajectories of hematopoietic differentiation. Palantir was applied to the RNA modality using default parameters with the number of diffusion components 66 and hence Palantir was run with pre-pro B and naïve B as the terminals. The B-cell fate probability was then computed as the sum of pre-pro B and naïve B probabilities.PalantirCells with increasing probability towards each lineage were selected as the lineage cells highlighted in Cells involved in lineage specification 21; COVID-19 atlas of peripheral blood mono nuclear cells (PBMCs) from healthy donors and critical patients (Covid)69; iPS reprogramming dataset (ips)8 and the mouse gastrulation atlas (mgast)44. These datasets cover a broad spectrum of systems with different complexities, cell numbers and contain discrete and continuous cell-states and cell-types. We compared the densities using Spearman correlation between results obtained from different parameter settings. As shown in The robustness of Mellon was evaluated by recalculating density estimations across a broad spectrum of parameter settings on multiple datasets. We carried out full density inference for an extensive range of length scales, numbers of landmarks, and numbers of diffusion components in the following scRNA-seq datasets: T-cell depleted bone marrow of human hematopoiesis (BM)th percentile of the average density between the pair of runs are used for comparison  designed to emulate a cellular differentiation tree. This tree was conceptualized as a series of velocity vectors, each connecting branching points and each being a slightly perturbed version of the vector of its parent node. For each node in this tree, a unique Gaussian was defined. The Gaussian’s covariance matrix and mean were designed to create a distribution aligning with the velocity vector. Considering the inherent low dimensionality typically exhibited by a cell-state manifold, we adjusted the principal components of these Gaussians using an exponential decay scalar. The two continuous styles mimic the structure of CD34+ bone marrow RNA-seq and T-Cell depleted bone marrow RNA-seq datasets.The synthetic datasets representing single-cell datasets of discrete clusters was also generated using a GMM but with a different configuration. In this setup, we randomly sampled mean and covariance matrices to create an arbitrary GMM, resulting in mostly isolated clusters of simulated cells. This approach provided an alternative, contrasting framework for testing robustness of Mellon.The GMM allowed us to easily sample simulated cell states from both dataset types and to define corresponding ground truth probability density functions. We then utilized Mellon to compute the log-density of these simulated datasets. Ground-truth densities were compared with Mellon densities using Spearman correlations. As shown in See 2. While straightforward, this method tends to produce a noisy density estimation and frequently fails to capture meaningful global trends  to the low-dimensional embeddings generated by tools like UMAPs or Force-Directed Layouts. While these visualization tools are powerful, their main design is not for density inference. They can produce unstable embeddings, and when KDE is applied, the instability in the embeddings directly translates into the density inference, resulting in less reliable outputs. Furthermore, the high compression involved in generating these low-dimensional representations means that they cannot capture all the relevant biological variability inherent in the data. Consequently, these methods often fail to depict all the nuanced details of the underlying cell-state density function .The versatility of Mellon extends beyond density inference, showcasing its robust capability in the swift computation of gene trends, defined as continuous, smooth functions that trace the trajectory of gene expression over pseudotime. Our Gaussian process (GP) regression-centric design not only serves as the backbone for Mellon’s primary application but also efficiently caters to general GP regression, due to scalable features such as the fixed length scale for the Matern52 covariance kernel and landmarks for Sparse Gaussian process regression.Gaussian processes shine in their adeptness at handling high-noise scenarios, for instance, non-imputed gene expression values. This strength enables Mellon to generate smooth gene trends from a selected cellular branch’s temporal ordering using unimputed gene expression values, effectively capturing the dynamics of gene expression as cellular differentiation unfolds .Mellon’s implementation harnesses the power of the JAX library’s vectorization capabilities and low-dimensional latent representations of functions within the GP framework, enabling efficient gene trend computations across a substantial quantity of genes. In tests using a 36-core CPU, Mellon was able to generate gene trends for up to 10,000 genes and 1,500 cells at 500 pseudotime points in about one second. This efficient computation allows high-throughput exploration of gene expression dynamics during cellular differentiation from large-scale single-cell datasets.Supplement 1Supplement 2Supplement 3Supplement 4