Dimensionality reduction techniques.

We explored multiple dimensionality reduction methods to transform each spine’s high-dimensional feature set into a more interpretable, low-dimensional representation that retained the intrinsic relationships between features in the original high-dimensional space. While most existing computational pipelines for dendritic spine analysis use PCA for dimensionality reduction [10,18,21], we hypothesized that non-linear dimensionality reduction methods may better represent the intrinsic structure of the data in low-dimensional space.

We applied five dimensionality reduction methods to the dataset: PCA, Isometric Mapping (ISOMAP), t-Distributed Stochastic Neighbor Embedding (t-SNE), Uniform Manifold Approximation and Projection (UMAP), and Pearson’s Correlation Uniform Manifold Approximation and Projection (PCUMAP). The first four methods were chosen to represent both linear and nonlinear approaches used in dendritic spine and neuroscience research for analyzing complex morphological data. Finally, PCUMAP is a state-of-the-art dimensionality reduction technique that aims to overcome some of the limitations of other methods in representing both local and global structure.

Table 1 summarizes the key components of five dimensionality reduction techniques that were chosen. Each of these techniques has strengths and limitations related to sensitivity and ability to preserve local and global structure. Global linear techniques like PCA are popular for their interpretability, but have their limitations: since PCA assumes linearity, the method cannot capture underlying nonlinear structure in the data [10,36]. Manifold approaches like ISOMAP attempt to solve the issue of nonlinearity by using local Euclidean distances from the original feature space in the low-dimensional space, but in recent years these techniques have been shown to misrepresent neighborhood structure [36], [25]. t-SNE and UMAP are other nonlinear methods that are able to capture local structure, meaning they are effective at creating separable classes and clusters. Additionally, given that these methods are nonlinear, they may be better able to model the continuous nature of spine shape growth [23]. However, both of these methods have issues preserving global structure [36]. For this reason, we also apply a recently developed method, PCUMAP, which aims to both preserve local and global structure using correlation-based loss [37]. Given that all linear and nonlinear methods have pros and cons for our purposes, we decided to test all five using a comprehensive set of local and global feature preservation metrics. The hyperparameters used for each dimensionality reduction method are detailed in S4 Table in S1 File.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of Dimensionality Reduction Methods for Dendritic Spine Analysis.

https://doi.org/10.1371/journal.pone.0349775.t001

Evaluation metrics.

The goal of dimensionality reduction is to obtain a visually interpretable representation of high-dimensional data that preserves as much information as possible from the original data. We created a decision framework to comprehensively evaluate the performance of the dimensionality reduction methods using multiple metrics, focusing on topological preservation (local and global), and taking advantage of the expert-labels to gain further insights into the data structure across dimensionality reduction methods. This enabled us to identify the representation that best balanced structural fidelity and biological interpretability, providing a principled basis for selecting an embedding method rather than relying on heuristic or visual judgment alone.

A. Local Metrics.

Local structure preservation metrics evaluate how well the structure of a low-dimensional embedding matches that of the original high-dimensional embedding [42]. We use multiple common local structure preservation metrics, which are detailed below. For dendritic spine morphological modeling, local structure preservation is prioritized over global structure, since the aim is morphological subtype detection and fine-grained transition analysis. Fine-grained details are better preserved if local neighborhood distortions are low [40].

1. Trustworthiness score (T(k)) measures how well the local structure of the original data is maintained in low-dimensional space, where . [43]. This metric penalizes any points that are nearest neighbors in the low-dimensional space but far away in the original high-dimensional space. Trustworthiness is defined as , where n is the total number of samples (spines), k is the number of nearest neighbors considered, are the k nearest neighbors in the output space, and r(i,j) is the rank of each sample j in the original space. We considered k = 10 nearest neighbors in our implementation. A very small k can be prone to noise or underfitting, so setting k = 10 ensures we are still examining local relationships while mitigating errors due to noise. S6 Table in S1 File details the impact of varying k within the range 5 ≤ k ≤ 50 on the result.
2. Local Continuity Meta Criterion (LCMC) measures the average overlap of the K-nearest neighborhoods of a point in the high-dimensional data and low-dimensional space [44], where . It helps assess how well local relationships between points are preserved in low-dimensional space. LCMC(k) is defined as , where n is the number of samples (spines), k is the number of nearest neighbors, are the k nearest neighbors in the original feature space, and are the k nearest neighbors in the low-dimensional feature space. We use k = 10 to observe local-scale effects while mitigating noise. S6 Table in S1 File details the impact of varying k within the range on the result.
3. Local Structure Preservation Score. We compute an overall local structure preservation score that considers these metrics, using the average of the Trustworthiness and LCMC scores [37]:

We weight LS and GS equally to give equal preference to local and global structure preservation, but this weighting can be tuned to prioritize local or global structure for what is most appropriate for the researcher’s purposes.

B. Global Metrics.

Global structure preservation metrics measure how consistent point-pairwise distances are in the low-dimensional embedding compared to the original high-dimensional data- in other words, points that are close together or far apart in the original high-dimensional space should remain the same way in the low-dimensional embedding [42]. We implement two metrics, Pearson correlation and Spearman correlation, to evaluate the level of global distortion for each dimensionality reduction method.

1. Pearson’s correlation coefficient measures how the pairwise distances between points varies in the original space and low-dimensional embedding space [42]. The score ranges from [−1, 1], where a negative score indicates negative linear correlation, 0 indicates no correlation, and a positive score indicates positive linear correlation. It is computed as the sample correlation coefficient: , where x_i and y_i represent the pairwise distances in the original and embedded space, and , are their means.
2. Spearman’s rank correlation coefficient measures the strength of the monotonic relationship between distances using the Pearson correlation on rank-transformed distances. The value ranges from [−1, 1], where −1 indicates a negative monotonic relationship, + 1 indicates a positive monotonic relationship, and 0 indicates no monotonic relationship. where r(x_i) and r(y_i) are the ranks of the original and embedded pairwise distances, and , are their means.
3. Global Structure Preservation Score. We compute an overall global structure preservation score for ease of interpretability, using the average of the Pearson’s and Spearman’s coefficient values [37]:

C. Overall Structure Preservation Score.

We compute an overall structure preservation score for each dimensionality reduction method, using a weighted sum of the local and global structure preservation scores. The SPS is defined below, where w_LS is the weight of the local structure preservation score, w_GS is the weight of the global structure preservation score, and .

The results presented use and to give equal preference to local and global structure preservation. We also performed a sensitivity analysis to determine how the weighting of LS and GS impacts the selection of dimensionality reduction method (See S7 Table in S1 File). Depending on the researcher’s goals, they can tune this metric to prioritize local or global structure for visualization.

D. Biological Transition Score (BTS).

We additionally aim to establish a metric that can provide insight into the biological relevance of the data in low-dimensional space, focusing on the transition pattern of the data. It relies on annotated labels, but is used only to compare dimensionality reduction methods, not to drive clustering. This approach provides valuable insight into local relationships between spines in the low-dimensionality model, as well as a method of comparison of results across all dimensionality reduction techniques based on biological priors.

We developed BTS, which is a metric that provides a scalar value explaining how well each dimensionality reduction method preserves known biological relationships between different spine types. This metric is based on known literature about the plasticity of dendritic spines. Notably, a study from Bourne et al. examines the morphological transitions between “mushroom”, “stubby”, and “thin” spine types [45]. Their findings show that thin spines are highly plastic, and either mature into mushroom spines or are eliminated. Mushroom spine types are typically highly stable. Most immature dendritic spines are stubby. During long-term potentiation, stubby and thin spines can evolve into mushroom spines, while during long-term depression, stubby spines retract or shrink. However, stubby spines do not commonly evolve directly to thin spines [46]. We applied this domain knowledge toward creating a metric for biological relevance, where common spine transitions are rewarded, and uncommon spine transitions are penalized. Note that BTS is a comparative, heuristic metric for assessing preservation of biologically plausible transitions relatively across embeddings, rather than being a direct measurement of biological transition probabilities.

The score is calculated using both Euclidean Distances and Cosine Distances. Linear methods like PCA prioritize preserving global linear distance, which aligns with Euclidean distance metrics, potentially biasing evaluation in their favor. For this reason, including cosine distance, which measures angular relationships in the feature space, helps assess the preservation of local orientation and offers a more comprehensive view across both linear and nonlinear dimensionality reduction methods [38,47]. We defined BTS as

where w_i,j is the weight of transitions between spine types i and j based on biological plausibility, d_ij is the distance between spines i and j in the low-dimensional space, and NN_i is the set of nearest neighbors of spine i.

Because there is no widely accepted quantitative model describing “transition rates” between spine morphologies, the transition weights used in BTS rely on biologically motivated approximations. The transition weights were assigned as follows, where within-spine clusters are assigned the highest score, common transitions are assigned a score between 0 and 1, and uncommon transitions are penalized with a score of 0. We use:

• Same-type transitions (MushroomMushroom, StubbyStubby, ThinThin): 1.0;
• MushroomThin or MushroomStubby: 0.5;
• ThinStubby: 0.0.

To assess the robustness of the BTS to assumptions about spine transition likelihoods, we evaluated a range of biologically plausible transition-weighting schemes (see S8 Table in S1 File). Across all weighting schemes, within-type transitions (e.g., MushroomMushroom, StubbyStubby, ThinThin) were assigned the highest weight, reflecting local morphological similarity. The weight assigned to biologically common cross-type transitions, (e.g., transitions between MushroomThin spines and MushroomStubby spines) was varied from low to high values to test sensitivity to the strength of these relationships. Finally, transitions between ThinStubby spines, which are rarely observed, were either fully penalized or assigned small nonzero weights to examine the impact of soft vs. hard penalties on uncommon transitions. Combining these weighting schemes span a broad but biologically informed space of transition assumptions, to evaluate whether comparisons remain stable under potential variations of prior biological knowledge.

Probabilistic clustering techniques.

After obtaining a representation of dendritic spine morphology that preserves both local and global structural relationships, we aimed to identify morphological groups based solely on their feature relationships while also capturing transitional phenotypes and quantifying uncertainty, aspects that are often overlooked in the traditional discrete spine classification scheme. To achieve this goal, we first determine the optimal number of spine type groupings for a given dataset, then test three clustering methods- Hierarchical Clustering, Fuzzy C-Means, and Gaussian Mixture Models- using quantitative metrics measuring cluster structure and spread. Each clustering method provides a different perspective and potential benefit when modeling spine structure and transitional shapes, summarized in Table 2. To avoid overfitting and ensure reproducibility, clustering algorithms were run using default parameter settings from their respective implementations, except where explicitly noted.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Comparison of Clustering Methods for Dendritic Spine Transition Modeling.

https://doi.org/10.1371/journal.pone.0349775.t002

1. A. Hierarchical Clustering. We implemented agglomerative hierarchical clustering using Ward’s linkage and Euclidean distance in the low-dimensional feature space [48]. This method results in each data point being assigned a hard membership label according to its cluster, without probabilistic insights. The dendrogram structure of hierarchical clustering can offer valuable insight into nested cluster relationships and structural similarity. While not probabilistic, hierarchical clustering was included to evaluate the extent to which strict structural organization aligns with continuous morphological transitions [10]. Starting with each point in its own cluster, hierarchical clustering continually merges the two closest clusters aiming to minimize the within-cluster variances based on the following equation: , where A, B are the clusters being merged, |A|, |B| are the number of elements in clusters A and B, are the mean vectors of clusters A and B, and D(A, B) is the distance between clusters using Ward’s criterion.
2. B. Fuzzy C-Means (FCM). To allow for partial membership of spines across clusters, we applied FCM, a soft clustering technique that assigns a membership degree between 0–1 to each point for each cluster [49]. This method was chosen for its potential suitability in modeling transitional spine morphologies, since spines at intermediate stages may not belong exclusively to a single class, and probabilistic scoring of spine types could help identify transitional shapes [10]. We used a fuzzifier parameter m = 2, and clusters were formed based on minimizing the weighted within-cluster distances. FCM minimizes the following objective function: . Membership degrees u_ik are updated iteratively using the equation: . is the embedding of spine i, is the centroid of cluster k, is the degree of membership of spine i in cluster k, N is the total number of spines, K is the number of clusters, and m is the “fuzzifier” controlling softness.
3. C. Gaussian Mixture Models (GMM). Finally, we utilized GMM to capture both soft cluster assignments and data density distributions. GMM assumes that data points are generated from a mixture of multivariate Gaussian distributions, and provides posterior probabilities for each spine’s membership in every cluster [50]. Similar to FCM, this approach provides probabilistic scoring for each data point’s membership, but has the additional benefit of modeling uncertainty and transition likelihood. We used the Expectation-Maximization (EM) algorithm for parameter estimation. GMM models the data distribution as a weighted sum of multivariate Gaussians using the following equation: . The soft assignment of point z_i to component k is . is the mixture weight of component k, with , is the mean vector of Gaussian component k, is the covariance matrix of component k, is the multivariate normal density function, and is the posterior probability that point i belongs to cluster k.

Cluster number selection.

Selecting an appropriate number of clusters is a critical step for modeling dendritic spine morphology, especially given the continuous and transitional nature of spine shapes. To determine the optimal number of clusters for each clustering method, we employed two complementary methods that assess cluster separability and geometric structure.

1. A. Silhouette Analysis. Silhouette analysis was used to evaluate clustering performance across a range of candidate cluster numbers k. For each value of k, we computed the mean Silhouette score across all samples, which reflects the balance between intra-cluster cohesion and inter-cluster separation. These scores were plotted as a function of k, and the optimal number of clusters was selected based on the first peak, which results in a solution with strong separability while avoiding unnecessary complication or fragmentation of the data [51].
2. B. Convex Hull (CHull) Method. To further inform cluster number selection, we applied the CHull method, which evaluates the trade-off between cluster compactness and model complexity. The CHull method constructs a convex hull to capture within-cluster dispersion and between-cluster separation, where the “elbow” on the plot represents a solution that balances cluster separation with model complexity [52]. This method is particularly suited for identifying structural organization in datasets with overlapping or continuous cluster structure, which is useful for our dendritic spine application where morphologies, and therefore clusters, are continuous.

Evaluation metrics.

To evaluate the clustering results of these three methods and determine which produces the best representation of dendritic spine shape, we use a combination of hard and soft clustering metrics. The hard clustering metrics focus on separability of the clusters in the low-dimensional space, which gives insights into the general spread of the data, allowing us to determine if there are significant differences between methods. We additionally evaluate multiple soft clustering methods, which measure membership confidence. This gives us insights into the confidence of cluster assignments across methods.

A. Hard metrics.

Hard membership metrics for evaluating clustering models assume that objects either belong or do not belong to a certain cluster, based on inter-cluster and within-cluster similarities. We use multiple common hard clustering metrics, which are detailed below [52].

1. Silhouette Score is a measure of the mean intra-cluster distance and nearest-cluster distance for each sample [51]. This metric was used both to determine the optimal number of clusters in the low-dimensional feature space using each clustering method, and to evaluate cluster separability. The goal of this metric is to evaluate how similar a spine is to its own cluster compared to other clusters. The score ranges between [−1, 1], where a negative score indicates the spine is likely assigned to the wrong cluster, while a score of “1” indicates perfect cluster assignment. The Silhouette score is computed using , where a(i) is the average distance from a sample to all other samples in the cluster, and b(i) is the average distance from sample i to all samples in the nearest other cluster.
2. Davies-Bouldin (D-B) Score measures the average similarity of each cluster with its most similar cluster, where similarity is defined as the ratio of within-cluster distances to between-cluster distances [53]. Clusters that are farther apart will have a lower (better) score, with the minimum possible score of 0. D-B is defined as , where n is the number of clusters, s_i is the average distance of all points in a cluster to its centroid, and d_ij is the distance between the centroids of clusters i and j.
3. Calinski–Harabasz (C-H) Index measures the ratio of between-cluster dispersion to within-cluster dispersion, providing an indication of how well clusters are separated and how compact they are [54]. A higher score indicates better-defined, more separated clusters. The C-H Index is defined as where n is the number of samples and k is the number of clusters, Tr(B_k) is the trace of the between-cluster dispersion matrix, and Tr(W_k) is the trace of the within-cluster dispersion matrix.

B. Soft metrics.

For probabilistic clustering and transitional modeling, soft clustering metrics are useful for understanding the strength or confidence to which a sample belongs to a group [49]. Since dendritic spines exist on a continuum, explicitly modeling probabilistic memberships to each group can be used to inform whether a spine is in a transitional state. We use multiple common soft clustering metrics to evaluate our models.

1. Entropy measures the average uncertainty in the soft cluster assignments by quantifying how uniformly each sample’s membership is distributed across clusters. For a sample i with membership probabilities , the Shannon entropy is defined as and the Average Entropy over all n samples is Lower values indicate higher certainty in assignments, and higher values reflect greater fuzziness or uncertainty in cluster membership [55].
2. Sharpness measures the concentration of membership probabilities. Given the maximum possible entropy , the Average Sharpness is defined as The value ranges from [0, 1], where a value of 1 reflects maximum certainty while values closer to 0 reflect higher fuzziness [49].
3. Average Maximum Probability measures how strongly each sample favors its most likely cluster, and is measured as the average of the maximum membership probabilities across all samples: Higher values indicate that data tends to have a single dominant cluster membership, implying greater membership confidence. Lower values suggest more fuzzy cluster assignments [49].

Local and global structure preservation.

Fig 2B presents local continuity and local trustworthiness. t-SNE, UMAP, and PCUMAP show high continuity and trustworthiness, indicating strong preservation of local structure. PCA maintains original neighborhoods but introduces spurious neighbors in the low-dimensional space, resulting in lower trustworthiness. ISOMAP performed the weakest across local metrics. Table 3 reports the trustworthiness and continuity scores for each method, used to compute LS. S6 Table in S1 File details a sensitivity analysis that varies the number of nearest neighbors used in the calculation of local metrics, showing that the relative ranking of dimensionality reduction methods remains stable across a wide range of neighborhood sizes.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Comparison of five dimensionality reduction methods across local and global structure preservation metrics. Local metrics include T(k) and LCMC, which were combined into an overall LS score. Global metrics include Pearson’s and Spearman’s coefficients, which were combined into an overall GS score. SPS was computed using a weighted sum of LS and GS. BTS was computed using both Euclidean and Cosine distances to represent each method’s preservation of biologically relevant relationships based on expert annotations. The final column reports the average overlap (AO) across all expert-label class pairs (lower indicates better class separation). Bold values indicate the best-performing method for each metric, while underlined values denote the second-best result.

https://doi.org/10.1371/journal.pone.0349775.t003

Table 3 also presents global structure metrics, Pearson and Spearman correlations, which are used to compute the GS. These results are visualized in Fig 2C, which illustrates how each method balances preservation of local and global structure. PCA shows strong global preservation but comparatively high local distortion. UMAP and t-SNE preserve local structure well but have reduced global correlation. PCUMAP provides the most balanced performance, retaining both local continuity and broader global trends.

Structure Preservation Score (SPS).

We computed the overall SPS for each dimensionality reduction method as defined in the Methods section (Table 3). When equal weighting was applied and , PCUMAP achieved the highest SPS among the evaluated methods. To evaluate whether this result depended on the chosen weighting, we performed a sensitivity analysis by varying the weighting parameter between 0.3 and 0.7 to prioritize local or global structure (S7 Table in S1 File). Across all weighting schemes tested, including cases emphasizing primarily local or primarily global structure, PCUMAP consistently achieved the highest SPS. PCA and UMAP generally followed in performance, while ISOMAP produced the lowest SPS values under these criteria. These results suggest that the superior performance of PCUMAP on this dataset is not dependent on a specific weighting choice, but reflects robust preservation of both local and global structure.

Distribution and Biological Transition Modeling.

Fig 2D shows distribution overlaps for the three manually labeled classes (Stubby–Thin, Stubby–Mushroom, and Thin–Mushroom). All methods show low overlap between stubby and thin spines. t-SNE yields the lowest overlaps overall (reported in Table 3 as AO), while PCUMAP shows higher overlap across groups.

BTS was evaluated using both Euclidean and cosine distances to capture complementary aspects of feature relationships (See Methods and S8 Table in S1 File for more details on this metric). Using Euclidean distance, UMAP and PCUMAP achieve the highest BTS values, followed by PCA, while ISOMAP and t-SNE show lower alignment with expected biological relationships. When using the cosine distance, all the methods show a high BTS, suggesting that the angular relationships between feature vectors reflect biologically significant transitions more consistently than the absolute Euclidean distances (Table 3). The BTS calculation incorporates biologically motivated transition weights derived from prior literature on dendritic spine remodeling [45,46]. Because these weights are necessarily approximate, we assessed the robustness of BTS by performing a sensitivity analysis across alternative biologically plausible weighting schemes (S8 Table in S1 File), which showed that relative method rankings remain stable across a wide range of parameter choices.

Determination of most suitable dimensionality reduction method.

Among the dimensionality reduction approaches evaluated, PCUMAP achieves the most balanced preservation of the local structure, the global structure, and the BTS in this data set. Therefore, we selected PCUMAP for subsequent clustering and transition modeling steps.

Clustering in the selected embedding space reveals a continuum of dendritic spine morphologies.

We apply three clustering techniques (Hierarchical Clustering, FCM, and GMM) to the PCUMAP data projection, obtaining a representation of spine connections that uses nonlinear combinations of features to determine groupings (Fig 4A).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Cluster-number selection diagnostics for FCM clustering on the 3D PCUMAP embedding.

Cluster solutions for K = 2–10 were evaluated using complementary structural criteria, including A) the Silhouette score (cluster compactness and separation), B) distortion-based CHull analysis (convex hull elbow), C) the D-B index (cluster overlap; lower values indicate better separation), and D) the standard deviation of cluster sizes (partition balance). The Silhouette score peaked at K = 2, reflecting coarse partitioning into broad groups. Separation-based metrics (D-B index) favored slightly larger values (), while CHull distortion analysis identified an elbow at K = 5, indicating diminishing returns in compactness improvement beyond this point. Cluster size diagnostics revealed substantial imbalance at K = 3, consistent with under-segmentation, whereas larger K values progressively subdivided the embedding structure. Taken together, these complementary metrics support K = 5 as a balanced compromise between under-resolution at small K and over-fragmentation at larger K.

https://doi.org/10.1371/journal.pone.0349775.g003

Decision of cluster number.

To determine the appropriate number of clusters for this dataset and examine how dendritic spine morphologies distribute across groups, we evaluated cluster solutions for K = 2–10 using multiple complementary criteria. Specifically, we assessed the Silhouette score [51], the CHull method [52], the D-B index, and cluster size balance (standard deviation of cluster sizes) (Fig 3). These metrics did not converge to a single optimal K, which is expected given the continuous, branching structure of the embedding. The Silhouette score peaked at K = 3, reflecting coarse partitioning into broad groups (Fig 3A). However, cluster size diagnostics revealed substantial imbalance at K = 3, indicating under-resolution of morphological heterogeneity. In contrast, separation-based metrics (e.g., D-B) continued to improve until K = 6, suggesting additional substructure (Fig 3C). The CHull distortion analysis identified an elbow at K = 5, corresponding to a point of diminishing returns in compactness improvement (Fig 3B). At K = 2, the embedding was divided into two broad lobes, merging morphologically distinct spine types and failing to resolve transitional structure. At K = 8, clusters fragmented in the embedding, producing smaller and less interpretable subgroups without substantial gains in structural separation (S10, S11 Tables in S1 File). Taken together, these analyses indicate that K = 5 provides the most balanced solution, resolving meaningful substructure beyond the canonical three-class organization while avoiding over-fragmentation at higher K.

Hard metrics.

We evaluated hierarchical clustering, FCM, and GMM using hard clustering metrics that quantify cohesiveness and separability (Table 4). All three methods produced reasonably cohesive and well-separated clusters, with FCM showing slightly stronger overall performance (Silhouette score = 0.351). Both FCM and hierarchical clustering yielded lower D-B scores (0.957 and 0.964, respectively), indicating reduced similarity between clusters compared to GMM. FCM also achieved the highest C-H value (381.61), reflecting compact clusters with strong separation.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Comparison of clustering methods applied to PCUMAP dendritic spine shape embedding, using both hard and soft clustering metrics. Hard clustering metrics (Silhouette score, C-H Index, and D-B Index (D-B)) evaluate the quality of discrete cluster assignments. Soft clustering metrics (average entropy, sharpness, and maximum membership) assess uncertainty and confidence in probabilistic cluster memberships. Bold values indicate the best-performing method for each metric.

https://doi.org/10.1371/journal.pone.0349775.t004

Soft metrics.

When comparing FCM’s performance to GMM on the soft clustering metrics, FCM had a “fuzzier” result compared to GMM, with a higher average entropy and lower average sharpness. This indicates that while the dominant cluster assignment for the spines was well separated, individual spines exhibited more distributed membership among the clusters under FCM than under GMM. Probabilistic spread is consistent with the biological continuum of dendritic spine morphologies, allowing individual spines to belong to multiple clusters with varying degrees of membership, better capturing the reality that the boundaries between classical categories are gradual rather than discrete.

Spine distributions across clusters.

As shown in the heat maps in Fig 4B, which compare cluster assignments with expert-provided labels (“mushroom”, “stubby”, “thin”), all three spine types are distributed across the five clusters to different extents. Thin and stubby spines are concentrated primarily within one to two clusters, whereas mushroom spines are distributed across multiple clusters in varying proportions depending on the clustering algorithm. This distribution suggests that mushroom spines themselves represent a broad morphological spectrum rather than a homogeneous category, and traditional classification methods fail to capture the variations within this spine type. Given that mushroom spines are associated with structural stability and long-term memory, capturing the diversity of their morphologies is essential for linking structure to synaptic function.

The connectivity graphs (Fig 4C) illustrate the Euclidean nearest neighbors of each spine, categorized by cluster, in the 3D PCUMAP embedding, providing further insight into local neighborhood relationships consistent with potential intermediate organization. FCM clustering captures more overlap between neighboring clusters. This pattern aligns with the biological expectation that dendritic spines remodel through gradual morphological changes rather than discrete jumps. In comparison, hierarchical clustering produces more rigid connectivity patterns with fewer intermediate states.

Merit of 5 clusters compared to 3 clusters or expert labels.

To determine whether increasing the number of clusters actually captures meaningful morphological heterogeneity, we performed an additional sensitivity analysis directly comparing data structure preservation of (1) expert labels, (2) clustering at K = 3, corresponding to the canonical “mushroom,” “stubby,” and “thin” classes, and (3) K = 5, using identical evaluation metrics, where K is the number of clusters (see S9 Table in S1 File). This comparison is necessary to assess whether increasing the number of clusters leads to artificial fragmentation of existing spine categories or instead reveals stable, interpretable substructure within them. As expected, because K = 3 aligns with the three-class structure of the expert annotations, it showed slightly higher agreement with the ground-truth labels. However, clustering at K = 5 maintained comparable cluster compactness and separation, as indicated by similarly high Silhouette, C-H, and D-B scores. Additionally, K = 5 clustering achieved equal or stronger structural separation compared to the expert ground-truth labels evaluated directly in the same embedding across all metrics, suggesting that the clustering-based representations capture morphological structure that is not fully reflected by the discrete expert classification. These results indicate that increasing the number of spine-type categories can preserve the structural organization of canonical spine types while also resolving additional substructure within traditional categories, providing a more refined and biologically meaningful representation of dendritic spine morphology rather than simply being an artifact of over-segmentation.

Dataset imbalance analysis.

It is also important to note that the dataset is imbalanced in spine types, with mushroom spines comprising the majority of the dataset, followed by stubby spines, and fewer thin spines. While this does align with naturally occurring distributions of spine types reported in prior studies [9,45], this imbalance could potentially influence clustering behavior. To assess the robustness of our results to spine-type imbalance, we performed additional sensitivity analyses comparing clustering outcomes on the original dataset to the result on a balanced subset with equal representation of each spine type (S14 Table, S11 Fig in S1 File). We found that across all clustering methods, balancing the dataset led to more uniform cluster sizes and similar clustering confidence compared to the original dataset, as evidenced by reduced cluster-size variability and stable membership certainty. However, the qualitative relationships between clusters and spine types were preserved after balancing: thin spines remained well separated, primarily in their own cluster, and there was still little to no overlap between stubby and thin spines within the same cluster. These analyses indicate that the observed cluster organization presented here reflects intrinsic morphological structure captured by our method, and is not only driven by spine-type imbalance.

Determination of clustering method and cluster interpretation.

Applying FCM clustering to PCUMAP spine characteristics revealed intermediate morphologies between stubby, mushroom, and thin. Fig 4E shows the distribution of spine labels within each cluster. Fig 4D presents representative spines from each group. While we saw a distribution of spine types across each of the clusters, we observed several consistent feature-based trends (see S10 Fig in S1 File):

• Cluster 1: primarily mushroom spines with larger heads and longer necks.
• Cluster 2: stubby and mushroom spines with short, wide necks, high width-to-length ratios, and homogeneous intensity.
• Cluster 3: predominantly stubby spines with little to no neck.
• Cluster 4: primarily thin spines with long necks and elevated intensity and textural features (fractal dimension, skewness), indicating higher morphological irregularity.
• Cluster 5: primarily mushroom spines with short necks and circular, uniform intensity profiles.

A complete list of all spines and their cluster assignments is provided in S13 Table in S1 File. Together, these results demonstrate that FCM provides a flexible framework for quantifying graded cluster membership and intermediate morphologies within this dataset. Notably, similar mixed-membership structure was observed across all embeddings even with lower BTS (S12 Table in S1 File), indicating that this continuum-like organization is not unique to the selected embedding.

Insights from probabilistic cluster assignments.

Beyond the framework evaluation, applying the selected clustering method to this dataset revealed several biologically meaningful patterns. The distribution of maximum membership probabilities (Fig 5A) shows that mushroom and stubby spines often have uncertain assignments, whereas thin spines are more confidently classified. This pattern is consistent with existing literature, which finds thin spines to be more morphologically distinct, while mushroom and stubby spines represent a continuum of head and neck geometries [7]. The mild imbalance in the dataset, with more mushroom and stubby examples than thin, may also contribute to this spread.

Individual examples also highlight transitional morphologies. Fig 5B shows spines with high membership in multiple clusters. Spines I and II were labeled mushroom by the annotator, while spine III was labeled stubby. However, the clustering approach reveals that both spine II and III show high membership in clusters 3 and 5, revealing a morphological continuum of shape across these three spine types that does not reflect using the traditional hard classification scheme.

To characterize the morphology of these intermediate spines, we compared ambiguous and confident spines using Cohen’s d score (standardized difference between the mean feature value for confident vs ambiguous spines). For each spine, we quantified maximum cluster membership and normalized entropy, and defined ambiguous assignments as spines with maximum membership probability less than 0.60. We found that ambiguous spines tended to have lower elongation/neck-related features (e.g., aspect ratio, LAR, neck length, eccentricity) and higher compactness-related measures (e.g., circularity, solidity), suggesting that the continuum is expressed through gradual variation in shape compactness and elongation rather than abrupt transitions between fully separated classes (Fig 5).

These results support the view that dendritic spine morphology is best described as a continuous landscape of developmental and functional states rather than as a set of discrete categories defined by a limited number of shape-based criteria [9]. Across the evaluated embeddings and clustering strategies, we observe gradual transitions and substantial overlap between traditional spine classes, with probabilistic clustering revealing hybrid morphologies that are obscured by hard classification. By allowing individual spines to exhibit partial membership across multiple clusters, our framework captures intermediate and transitional forms that reflect smooth morphological progression rather than discrete boundaries. These findings demonstrate that the proposed modeling approach preserves the global organization of the traditional classifications while providing increased resolution of intra-class heterogeneity, offering a more nuanced and biologically faithful representation of dendritic spine morphological variation.

Generalization of decision framework to new datasets.

To assess the robustness of our decision framework across imaging conditions, we applied the same dimensionality reduction and clustering selection procedure to an independent lower-resolution dataset from Smirnov et al. [32] (S15-S16 Table, S12 Fig in S1 File). Despite differences in resolution and acquisition parameters, the systematic evaluation of structure-preservation metrics and clustering validity indices again identified a five-cluster solution as optimal. In this dataset, PCA was selected as the preferred dimensionality reduction method, reflecting the lower spatial resolution and reduced nonlinear structure. The emergence of five morphometric groupings across both datasets supports the generalizability of the proposed framework and suggests that the identified clustering structure is not specific to a single imaging modality or resolution.

Primary Dataset: Ghani et al.

For our primary dataset, we used an open-source, labeled dataset of 456 dendritic spines from postnatal (day 7–10) mouse CA1 hippocampal neurons (Fig 6A) [31]. The hippocampus undergoes rapid structural remodeling during early postnatal development [58], resulting in a wide distribution of developing and intermediate spine morphologies. Images were acquired using two-photon laser scanning microscopy (2PLSM), with Z-stacks collected every 5 minutes over a 4-hour imaging period. Expert annotations provide both spine segmentations and categorical morphological labels (“mushroom”, “stubby”, “thin”), along with manually determined boundaries between the dendrite shaft and individual spines. Although manual spine annotation can exhibit inter-rater variability [24], expert-derived labels remain widely used in the field and provide biologically grounded reference annotations for evaluating computational morphology analysis methods.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Result of clustering of dendritic spine feature data in PCUMAP space using 3 clustering methods.

A) Result of clustering using 3 methods on PCUMAP projection of dendritic spine data in 3 dimensions. B) Distribution of mushroom, stubby, and thin spines across the 5 clusters for each clustering method. Mushroom spines are more distributed across the five classes, while thin and stubby spines are generally well-represented within just one to two clusters. C) Transitional graphs showing the distribution of each cluster’s nearest neighbors. The probabilistic clustering methods (FCM, GMM) show stronger edges between clusters. D) Example spines in each cluster from FCM clustering. E) Distribution of spine types across clusters for FCM clustering: mushroom spines are primarily distributed across clusters 1, 2, and 5. Stubby spines are primarily distributed across clusters 2 and 3. Thin spines are primarily contained in cluster 4.

https://doi.org/10.1371/journal.pone.0349775.g004

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Insights from probabilistic cluster assignments.

(A) Maximum membership probability for each spine, color-coded by annotated label. (B) Examples of spines with high membership in more than one cluster. The primary cluster assignment is annotated in red. (C) Top feature shifts using Cohen’s d metric. This shows which features were most associated with ambiguous spine types (max probability ≤ 0.6)‌‌.

https://doi.org/10.1371/journal.pone.0349775.g005

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Overview of datasets.

A) Image sample and expert-identified ROIs (“mushroom”, “stubby”, “thin”) from Ghani et al. B) Image sample and identified ROIs (unlabeled) from Smirnov et al. C) Shape, contour, and intensity features are collected from both datasets for analysis.

https://doi.org/10.1371/journal.pone.0349775.g006

The spatial resolution of two-photon microscopy is sufficient to capture subtle morphological changes in dendritic spine geometry, and is therefore a widely used technique for in vivo dendritic spine imaging and analysis [59,60]. This makes computational approaches developed using 2PLSM data broadly applicable to commonly used experimental imaging pipelines. In contrast, higher-resolution imaging techniques such as electron microscopy (EM) [17] require extensive sample preparation and specialized acquisition workflows [61], while super-resolution optical methods such as stimulated emission depletion (STED) microscopy can be limited by photobleaching and imaging depth [62,63].

As noted by Ghani et al., the limited axial resolution of 2PLSM motivates the use of two-dimensional maximum-intensity projections (MIPs) derived from Z-stacks. MIPs reduce noise associated with limited depth resolution while preserving the major geometric characteristics of spine morphology [31]. Prior comparative studies further demonstrate that morphometric measurements derived from 2D spine projections are strongly correlated with those obtained from full 3D reconstructions [64]. While absolute measurements such as area or volume may be slightly underestimated in 2D projections, the overall structure of morphological variation and separability of spine classes remain statistically comparable to those observed in 3D datasets [64].

An inherent limitation of using 2D images for dendritic spines is the projection-induced variance due to differing spine attachment angles. However, prior work demonstrates that such variation does not fundamentally distort the morphological relationships captured in 2D datasets [64]. Ghani et al. also utilize consistent imaging geometry and manual verification of segmentations by neuroscientists to help mitigate these concerns.

Secondary Dataset: Smirnov et al.

To evaluate the generalizability of our computational framework across datasets, we additionally analyzed a publicly available dataset of labeled dendritic spines from Smirnov et al. [32] (Fig 6B). These images were also acquired using two-photon laser scanning microscopy (2PLSM) and capture dendritic spine morphology in hippocampal neurons, and includes manually annotated spine regions. However, no expert annotations were provided in this dataset.

The Smirnov dataset consists of individual two-dimensional optical sections extracted from time-lapse imaging sequences. During acquisition, a m Z-stack composed of five optical planes was collected at each imaging position per minute. The released dataset contains the resulting two-dimensional images rather than full volumetric stacks. Because repeated acquisitions of the same dendritic segments appear within the Smirnov dataset, we implemented a preprocessing procedure to identify and remove duplicate spine observations. Bounding boxes corresponding to the same physical spine were grouped based on spatial proximity and coordinate annotations, and a single representative crop was retained for each spine. This preprocessing step ensures that each spine contributes only one observation to the dataset to preserve the morphological diversity in the original data.

The differences in projection method, spatial resolution, and acquisition protocols are detailed in Table 5. Because our computational pipeline operates on two-dimensional shape representations, these datasets provide a useful opportunity to evaluate the robustness of the learned morphological embeddings across variations in imaging conditions and data collection procedures, as well as how the decision framework may differ across imaging conditions.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Comparison of imaging scale parameters for the two dendritic spine datasets used in this study.

https://doi.org/10.1371/journal.pone.0349775.t005

Shape, appearance, and intensity-based feature collection.

All preprocessing, feature extraction, and statistical analyses described in this section were performed by the authors using the publicly released datasets. All spines identified by the original dataset publishers were analyzed to limit bias toward spines that are easier to detect computationally.

Shape-based morphological features are fundamental for understanding dendritic spine mechanics, since spine shape is closely linked to synaptic function and plasticity. Existing work to understand dendritic spine shape dynamics points to the actin cytoskeleton and glutamate receptors undergoing rapid and activity-dependent remodeling [34]. Quantitative analysis of these shapes can provide further insight into the mechanisms of synaptic transmission and plasticity. Semi-automatic shape feature collection was performed using SpineJ [35], an ImageJ plugin designed to collect morphological data from high resolution images of the dendritic spine. Because spine ROIs were defined by expert annotations in the original dataset rather than automatically detected by our pipeline, feature extraction was performed on the full set of labeled spines and does not introduce bias toward spines that are easier to detect using automated methods. Details of SpineJ feature collection pipeline can be found in S1 Fig in S1 File.

Intensity features provide complementary information to shape descriptors, especially in 2PLSM images, where intensity patterns can reflect underlying morphological characteristics. Dendritic spine necks typically exhibit a decreased or varied intensity pattern, which may be useful for discriminating spine types with smaller necks that are otherwise difficult to segment based solely on shape [31]. Ghani et. al use Histogram of Oriented Gradients (HOG), a method which computes the direction of gradients as an indicator of spine appearance [31]. Another study uses learned intensity-based data from 2PLSM dendritic spine images for segmentation of dendritic spines [16]. Thus, intensity-based descriptors are valuable both for automated segmentation and for capturing morphological heterogeneity not fully explained by geometric features alone.

Contour-based features, including curvature metrics and convex hull ratios, allow further refinement in characterizing dendritic spine morphology. Curvature analysis can highlight regions of high geometric complexity, such as the transition between spine head and neck, or subtle irregularities along the spine perimeter. The convex hull ratio (CHR) quantifies how closely the spine contour approximates a convex shape, which can be indicative of specific spine subtypes or developmental stages [21]. We captured these additional features using a custom automated feature collection pipeline in Python through scikit-learn [65] and OpenCV [66]. All features are listed in S2 Fig in S1 File.

C. Feature verification.

Although the feature selection process is supported by existing literature on dendritic spine analysis [10,15,21], to ensure that the features selected were actually representative of meaningful spine variations, we conducted a verification of our selected features using a pairwise comparison between spine types for each feature using nonparametric Mann-Whitney U tests (with p < 0.05 for statistical significance) and trained a Random Forest (RF) classifier and assessed feature importance to identify features with low discriminative power for spine type. Details of this analysis can be found in S2 Table and S3 Fig in S1 File. To ensure that feature selection did not overlook multivariate structure, we extended this analysis with additional multivariate evaluations (S5 Fig in S1 File). Specifically, we examined global and class-specific feature–feature correlations to assess potential interaction effects, and performed nested cross-validated forward and backward feature ablation to evaluate redundancy and sufficiency of the selected descriptors. Classification performance plateaued after inclusion of a subset of features, indicating partial redundancy among geometric descriptors while preserving most of the discriminative signal. These analyses led to the 35-dimensional feature set being narrowed down to 31 features, which provides a comprehensive morphometric representation while demonstrating that the retained descriptors are sufficient under cross-validation (S9 Fig in S1 File).