a. Spectral-based GCNNs.

Classical convolution kernels require fixed neighborhood sizes around the central node, which is challenging for graph-structured data due to variable and unordered node neighborhoods. Spectral-based GCNNs leverage graph signal processing, using Fourier transforms to convert graph signals into the frequency domain for convolution operations and then back to the spatial domain via inverse Fourier transforms [35], as shown in Fig 1.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. The system of spectral-based GCNNs.

https://doi.org/10.1371/journal.pone.0324605.g001

Examples include the first-generation GCNN [10], Chebyshev Networks (ChebyNet) [20], and first-order approximations of ChebyNet [21]. The first-generation GCNN’s reliance on Laplacian matrix eigendecomposition leads to high computational costs, limiting practical use. To address this, ChebyNet approximates convolution kernels with Chebyshev polynomials, greatly improving efficiency. Xu et al. [36] further simplified ChebyNet with a first-order approximation, represented as:

(1)

where is the adjacency matrix with self-loops, I_N is the identity matrix, is the degree matrix, W^(l) denotes the trainable weight matrix for layer l, σ(.) is the nonlinear activation function, and represents features at layer l with input H = X.

b. Spatial-based GCNNs.

Spatial-based GCNNs eliminate affinity on graph spectral theory, instead aggregating information from neighboring nodes to update central nodes from a spatial perspective [22,37]. These methods avoid Laplacian matrix constraints, enabling applications to directed graphs. Spatial-based GCNNs, akin to extended ConvNets, perform weighted summation of node information to achieve convolution [38]. Steps include propagating node features to neighbors, aggregating neighborhood features, and applying linear and nonlinear transformations to enhance model expressiveness. Examples include Diffusion Convolutional Neural Networks (DCNN) [26], Graph Sample and Aggregate (GraphSAGE) [39], and Graph Attention Networks (GAT) [40]. DCNN controls information flow via transition probabilities, GraphSAGE uses random sampling to aggregate features, and GAT employs attention coefficients to update node features, offering diverse methods for graph topology convolution. Recent advancements focus on enhancing spatial modeling through dynamic graph topologies and attention mechanisms, as follows:

1. 1) Dynamic Graph Construction: Huang et al. [23] proposed dynamic graph convolutional networks with multi-scale position attention (DGCN-MPA), adaptively fusing hierarchical attention to adjust node relationships across scales. Yang et al. [24] designed a hierarchical adaptive multi-scale hypergraph (HAM-HGNet) that clusters joints into parts based on movement patterns, relaxing fixed topology constraints.
2. 2) Knowledge & Attention Integration: Roy et al. [41] introduced knowledge-driven shift-GCN (KDS-GCN), leveraging graph connectivity priors and coordinate features to enrich representations. Qin et al. [25] enhanced ST-GCN with convolutional block attention modules (CBAM-STGCN), refining spatial-channel features through adaptive weighting.
3. 3) Hybrid Architectures: To bridge Euclidean and non-Euclidean modeling, Wu et al. [27] combined multi-channel CNNs with GCNs, where CNNs capture local temporal patterns and GCNs extract spatial dependencies. Mansouri et al. [28] similarly fused CNNs and GCNs to jointly learn spatiotemporal dynamics.
4. 4) Transformer-Based Extensions: Addressing GCNs’ limited receptive fields, Do et al. [29] proposed SkateFormer, partitioning skeletal-temporal relations for memory-efficient self-attention. Lin et al. [30] developed HCTransformer, decoupling human-centric and context features through domain-invariant interaction learning.

Furthermore, recent advances in graph neural networks across diverse domains have inspired novel skeleton-based HAR designs. For example, Su et al. [42] introduced MFEn-GFDn, a Multi-Feature Extraction Network and Graph Fusion Detection Network, to enhance radar target detection in complex sea clutter by constructing multi-feature graph data and applying graph fusion strategies. Zhang et al. [43] demonstrated the integration of GNNs with brain functional network analysis, highlighting the potential of interpretable GNN architectures for large-scale, heterogeneous graphs in clinical neuroscience applications. In the realm of multimodal video analysis, Wang et al. [44] proposed a Geo-based two-stream framework that builds facial graphs and fuses GCN and CNN features alongside temporal attention, achieving robust personality trait prediction in short video clips. These works underscore the versatility of GNNs in capturing complex relational patterns, motivating their application to dynamic skeleton activity graphs in HAR.

a. Construction of the skelecton graph.

Given a human skelecton, let N represent the number of nodes. The skelecton is naturally abstracted as a graph, where a skeleton with N nodes is represented as an undirected graph G=(V,E). Here, V={v_i|i = 1,2,…,N} represents all vertices of the skelecton graph, corresponding to the nodes of the skelecton, and E represents the edges of the graph, corresponding to the physical connections between nodes. The affinity relationships between nodes are encoded into an adjacency matrix of size N*N, with elements defined as:

(2)

b. Implementation of traditional graph convolution.

Based on the constructed skelecton graph, ST-GCNN (Spatiotemporal graph convolutional neural network) [19] defines the implementation of graph convolution on skeletal data. Given an input feature map , where C_in represents the number of input channels and T represents the number of frames, the process of graph convolution on skelecton data is expressed as:

(3)

where K is the number of spatial subsets, set to 3 in ST-GCNN. represents the normalized adjacency matrix, and is the normalized diagonal matrix, with λ = 0.001 to prevent empty rows in . Note that . is the trainable weight matrix, where C_out represents the number of output channels. is a trainable matrix initialized with all elements set to 1. The operator ⊙ denotes the Hadamard product (element-wise multiplication). Using formula (3), the spatial graph convolution output can be obtained.

a. Activity information extraction block.

Before a graph can adapt to the choreography of a moving body, the model needs a succinct summary of how that body is currently evolving. Raw joint coordinates alone are too literal; they tell us where the joints are but not why they are there. By first translating the input tensor through a low‑rank linear bottleneck, we distil high‑variance motion cues—speed, directional coherence, local synchrony—into a compact semantic vector. The subsequent frame‑to‑frame differencing then accentuates precisely those temporal gradients that classical GCNN layers tend to miss. In short, this block converts unstructured spatial snapshots into a trajectory‑aware “action fingerprint” that downstream modules can reason over.

As shown in Fig 3, the purpose of the activity information extraction block is to obtain a semantic representation of activity information. For a given input feature , the transformation function T(.) is applied to transform the input features. To reduce model complexity and computational resource requirements, a simple linear transformation with a reduction ratio r is adopted for T(.), which is expressed as:

(4)

where represents the semantic features obtained from the input features, and is the weight matrix.

Based on the semantic features P, activity features are further extracted through the extraction function E(.). As shown in the Fig 2, the activity information is obtained by calculating the difference between adjacent frames of the semantic features, and all frame-level activity information is concatenated along the temporal dimension to form M. The process is defined as:

(5)

where concat(.) denotes concatenation, tanh(.) represents the hyperbolic tangent function that maps outputs to (−1,1), and P_t denotes the semantic features of the t-th frame.

b. Graph optimization block.

A skeleton graph is only useful if its edges mirror the latent biomechanics of the action at hand. Relying on a single, anatomically predefined adjacency matrix risks freezing the graph in a posture that may be irrelevant—or even misleading—once the motion unfolds. Our optimisation strategy therefore treats the graph as a living document: we start with hard‑wired physical links, overlay a learnable canvas, and finally let data‑driven activity affinities rewrite the margins in real time. The key idea is to let motion‑dependent similarity matrices pull or relax edges, so that jumping contracts the ankle–knee–hip triad while clapping cross‑links the two wrists. This dynamic tri‑graph fusion produces an adjacency structure that breathes with the action yet stays tethered to plausible human kinematics.

In the k-th spatial subset, the 3A-Graph consists of three components, i.e., A_k, B_k, and C_k. As illustrated in Fig 4, the adjacency matrix A_k and trainable matrix B_k are shared across all frames, making it difficult to capture the dynamic changes in the skeletal graph over time. To model the temporal affinities between nodes at different time periods, the extracted activity features are used to infer a activity-related graph , where s represents the number of time segments. C_k is then used to optimize A_k and B_k, resulting in the 3A-Graph .

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Detailed structure of the graph optimization block within ASGM.

This block dynamically constructs the 3A-Graph by integrating three components: a manually initialized adjacency graph based on physical node connections (A), a trainable graph optimized during network training (B), and a dynamically inferred activity-related graph (C), effectively modeling temporal variations in node affinities.

https://doi.org/10.1371/journal.pone.0324605.g004

Given the extracted activity features , it is first divided into s feature segments along the temporal dimension, denoted as M_i, where i = 1,2,…,s. Each segment has a shape of . Next, two embedding functions θ and ϕ are applied to transform the shape of each segment to . To ensure efficiency in modeling the activity-related graph, θ and ϕ share weights across all feature segments, and the process is defined as:

(6)

(7)

where and are the weight matrices of θ and ϕ, respectively. As shown in Fig 2, θ and ϕ are implemented using two separate 1*1 convolution layers. The resulting embedded features θ(M_i) and ϕ(M_i) both have a shape of .

The shapes of θ(M_i) and ϕ(M_i) are then reshaped to and , respectively, and the activity-related graph C_k is inferred using the inference function D(.). By multiplying the reshaped matrices, a similarity matrix is obtained, where each element represents the similarity between any two nodes. To enrich the feature representation, two distinct branches are used to calculate feature similarity globally, avoiding over-reliance on self-connections and enhancing subsequent optimization. The similarity matrix is normalized using the Softmax function to scale values into (0,1). Each feature segment M_i corresponds to a similarity matrix, and concatenating all similarity matrices produces . As shown in the Fig 4, C_k is defined as:

(8)

Finally, the activity-related graph C_k is used to optimize the initialized matrix A_k and trainable matrix B_k, producing the 3A-Graph . The optimization function O(.) is defined as:

(9)

where A_k is as defined in formula (1); B_k is initialized as a trainable matrix with all elements set to 0, optimized jointly with the network in an end-to-end manner; and α_k is a trainable parameter that adjusts the relative importance of C_k compared to A_k and B_k. The element-wise addition is performed across all s time segments.

Algorithm 1. Details the dynamic optimization process, integrated into ASGM’s forward pass.

Algorithm 2. ASTM-Net.

Input: Input features F ∈ ℝ^{{T × N × C}}; Initialized graph A ∈ ℝ^{N × N}; Trainable graph B ∈ ℝ^{N × N}; Segments s; reduction ratio r

Output: Optimized graph G ∈ ℝ^{N × N}

Procedure:

1. # Activity feature extraction (Eq. 4–5)

  P = Linear(C → C/r)(F) # Eq. 4, T(·)

  M = Concat([tanh(P_{t + 1} - P_t) for t in 0:T-1]) # Eq. 5, E(·)

2. # Split into s temporal segments

  M_segments = Split(M, s) # M_i ∈ ℝ^{{N × (T/s) × (C/r)}}

3. # Compute activity-related graph C

  C = []

  for M_i in M_segments:

   θ = Conv1D(C/r → C/(2r))(M_i) # θ(M_i)

   φ = Conv1D(C/r → C/(2r))(M_i) # φ(M_i)

   S = MatMul(θ, φ.transpose(1,2))/ sqrt(d’)

   C_i= Softmax(S, dim = −1) # Eq. 8

   C.append(C_i)

  C = Stack(C) # C ∈ ℝ^{{s × N × N}}

4. # Fuse A, B, C (Eq. 9)

  G = A + B + α * ReduceSum(C, dim = 0)

  G = LayerNorm(G)

c. ASGM implementation block.

After the graph has morphed into an activity‑aware scaffold, the final step is to project rich node features across this scaffold without drowning in parameters. Splitting the temporal horizon into coarse segments—and using the same 3A‑Graph within each—creates a repeating canvas on which motion patterns can be painted at multiple rhythms, but without replicating weights. By summing the outputs of three complementary spatial subsets we emulate multi‑head attention: each subset selects a different anatomical or functional perspective (self‑loops, limb‑centric edges, cross‑limb shortcuts). The result is a lean yet expressive convolution that captures both the where and the how fast of human motion in a single sweep.

As shown in the Fig 2, given the input skeletal features f_in and the 3A-Graph G_k, a 2D convolution layer with a kernel size of 1 is first applied to transform f_in. Simultaneously, is divided into s 3A-Graph . Each G_ki is used to perform spatial graph convolution for each time period. The spatial graph convolutions across all time periods are fused via summation to produce the activity aware graph convolution result for the k-th spatial subset. Assuming K = 3 spatial subsets, the final ASGM output is obtained by summing the results of all K spatial subsets. The process is expressed as:

(10)

where is the weight matrix, and represents the output of ASGM.

a. Complementary spatial-temporal modeling.

ASGM specializes in adaptive spatial feature extraction by constructing dynamic 3A-Graphs that evolve with activity patterns. Unlike static adjacency matrices in conventional GCNNs [19], the 3A-Graph’s activity-related component (C-Graph) explicitly links spatial node affinities to temporal motion cues through cross-frame similarity learning (i.e.,(8)). This enables localized adaptation to activity-specific interactions, such as strengthening hand-shoulder connections during “overhead press” motions while weakening them in “kicking” actions. However, spatial adaptations alone cannot resolve temporal ambiguities in similar motion phases, e.g., distinguishing the acceleration phase of “jumping” from the deceleration phase of “landing.”

TMGM addresses this through multi-scale temporal modeling via parallel branches with varied receptive fields (Fig 7). The dilated convolution branches (d = 1,3) capture both micro-motion details (e.g., finger tremors during “writing”) and macro-motion trends (e.g., full-body coordination in “falling”), while the pooling branch suppresses transient noise. Crucially, TMGM operates on ASGM-refined spatial features, allowing temporal operators to leverage activity-aware spatial contexts. This bidirectional synergy enables ASTM-Net to simultaneously answer:

1. 1) Spatial “Where”: Which nodes are critically interacting at this temporal scale?
2. 2) Temporal “When”: How do these interactions evolve across motion phases?

b. Efficiency through structural cohesion.

The hybrid design achieves parameter efficiency through shared computation across modules. ASGM’s activity-related graph inference ((i.e., (6)-(8)) reuses temporal difference features (i.e., (5)) that TMGM subsequently processes, avoiding redundant feature extraction. Meanwhile, TMGM’s multi-branch outputs are aggregated before feeding into the next ASGM layer, ensuring progressive refinement of both spatial and temporal features without channel explosion.

a. NTU_RGB_plus_D.

This widely-used large-scale skelecton dataset is collected using three horizontally placed Microsoft Kinect v2 cameras and contains 56,880 samples covering 60 daily and health-related activities [46]. The dataset includes 40 subjects, and each activity is performed by one or two people. The human skelecton in each sample is represented by 25 3D nodes. Two official benchmarks are provided:

1. 1) X-Sub: The 40 subjects are partitioned into 20 for training (37,920 samples) and 20 for testing (18,960 samples). From the training set, 10% (3,792 samples) are randomly selected as the validation set for hyperparameter tuning.
2. 2) X-View: Data from cameras 2 and 3 (37,920 samples) are used for training, with 10% (3,792 samples) reserved for validation, while data from camera 1 (18,960 samples) serve as the test set.

b. NTU_RGB_plus_D_120.

This dataset is an extension of NTU_RGB_plus_D, containing 112,954 samples across 120 activity categories [47]. The number of subjects increases to 106, and the number of cameras rises to 32. Two official benchmarks are provided:

1. 1) X-Sub: 50 subjects (53,230 samples) are designated for training, with 10% (5,323 samples) held out for validation, and the remaining 56 subjects (59,724 samples) for testing.
2. 2) X-Set: Samples with even setup IDs (54,468 samples) form the training set, including 10% (5,447 samples) for validation, while odd-ID samples (58,486 samples) are used for testing.

c. Toyota smarthome.

Toyota Smarthome [48,42] is collected in an apartment with 7 Kinect v1 cameras, recording 31 daily activities (e.g., “drink from cup,” “read book”) performed by 18 elderly subjects (ages 60–80). The Samples include 16,115 video clips with RGB, depth, and 3D skeleton modalities (18 joints per frame). Evaluation Protocols are provided:

1. 1) Cross-Subject (CS): Train on 11 subjects (11,550 samples), test on 7 subjects (2,200 samples).
2. 2) Cross-View (CV1 & CV2):

CV1: Train on front-view (dining room), test on side-view.

CV2: Train on mixed views, test on front-view.

1) Accuracy vs. Parameters.

ASTM-Net achieves 0.31% higher accuracy than DualHead-Net-ALLs with 51.9% fewer parameters, demonstrating the efficiency of its dynamic 3A-Graph parameter sharing. Compared to Shift-GCNN-ALLs, ASTM-Net improves accuracy by 1.59% while using only 73.4% more parameters, validating the cost-effectiveness of its multi-branch design.

2) Computational efficiency.

ASTM-Net reduces FLOPs by 49.7% compared to MST-GCNN-ALLs while achieving 0.82% higher accuracy. Despite lacking FLOPs data for DualHead-Net, ASTM-Net’s 53.6% shorter training time highlights its optimization advantages.

3) Memory footprint.

ASTM-Net consumes 71.5% less memory than MST-GCNN-ALLs, enabled by the lightweight TMGM module and shared 3A-Graphs. Even against Shift-GCNN-ALLs, ASTM-Net reduces memory usage by 28.8%, critical for large-batch training.

Hence, ASTM-Net strikes an optimal balance between accuracy and computational cost.

a. ASGM ablation.

To validate the effectiveness of our ASGM, we construct a baseline model (referred to as Model-A) using the spatiotemporal graph convolutional unit of Sighencea et al. [50] as the STGCB in ASGM. As shown in formula (3), the spatial graph convolution in ST-GCNN combines a initialized adjacency graph and a trainable matrix L through the Hadamard product. In contrast, the proposed ASGM optimizes the initialized graph and the trainable matrix B using the activity-related graph, as defined in formula (9). For a fair comparison, the temporal convolutional layers in the baseline are replaced with TMGM, resulting in Model-B. Table 4 shows the experimental results, where:

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. ASGM Ablation Results.

https://doi.org/10.1371/journal.pone.0324605.t004

“&TMGM” indicates replacing all temporal graph convolutions in the baseline with TMGM.

“&ASGM-n” indicates replacing the first n spatial graph convolutions in Model-B with ASGM.

From Table 4, it can be observed that replacing the temporal convolutional layers of ST-GCNN with TMGM yields better results with fewer parameters. Gradually replacing the first seven spatial graph convolutions of Model-B with ASGM (represented by Models C-F) shows a consistent improvement in validation accuracy, demonstrating the effectiveness and stability of ASGM. Notably, replacing a single spatial graph convolution in Model-B with ASGM improves the activity recognition accuracy by 0.91% on the X-Sub benchmark, further validating the strong performance of ASGM.

b. 3A-Graph ablation.

As described earlier, the 3A-Graph G in ASTM-Net consists of three components: the initialized graph A, the trainable graph B, and the activity-related graph C. To evaluate the importance of each component, we manually remove one part of the topology graph in ASTM-Net (referred to as Model-δ) and denote the results as Models α-γ.

The results are shown in Table 5. It can be observed that the activity-related graph C significantly benefits activity recognition, as removing any part of the 3A-Graph adversely affects the performance of ASTM-Net. Combining all three components achieves the best performance.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. 3a.Graph ablation results.

https://doi.org/10.1371/journal.pone.0324605.t005

The recognition accuracies for Models α&γ are 89.65% and 89.44%, respectively, indicating that the activity-related graph C plays a more critical role than the initialized graph A. Compared to Model-γ, the superior performance of ASTM-Net-Complete validates the effectiveness of C in optimizing A and B.

c. Hyper-parameter ablation.

This section evaluates the performance of ASTM-Net under different hyper-parameter settings, including the reduction ratio r and the number of feature segments s (i.e., the number of time periods). The results are presented in Table 6.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Hyper-parameter ablation results.

https://doi.org/10.1371/journal.pone.0324605.t006

1. 1) Effect of r: Fixing s = 1, we gradually increase r up to 16. The results show that regardless of the value of r, ASTM-Net consistently outperforms the baseline on the validation set, confirming its robustness and stability as a feature extractor. When r is set to 4 or 8, ASTM-Net achieves higher activity recognition accuracy compared to other values of r. Specifically, setting r = 8 results in better accuracy than r = 4 while requiring fewer parameters.
2. 2) Effect of s: Fixing r = 8, we compare different values of s. By learning the activity-related graph at progressively finer granularity (increasing s), the results show that finer-grained graphs do not necessarily optimize the skeletal adjacency graph better. Given that the skelecton trajectories are preprocessed to 64 frames, setting s = 1 means using all frames to learn a single similarity matrix, while setting s = 16 means generating one similarity matrix per 4 frames. The results demonstrate that ASTM-Net achieves the highest accuracy when r = 8 and s = 4.

d. Depth ablation.

To analyze how ASTM-Net scales with model depth and its computational trade-offs, we conduct ablation experiments on the NTU-RGB + D X-Sub benchmark, varying the number of STGCBs. The results are presented in Table 7.

Download:

• PNG
  larger image
• TIFF
  original image

Table 7. Depth ablation results.

https://doi.org/10.1371/journal.pone.0324605.t007

From Table 7: Increasing STGCBs from 4 to 12 improves accuracy by 3.16%, as deeper networks capture richer hierarchical spatiotemporal dependencies. Beyond 12 blocks, accuracy drops slightly, indicating over-smoothing in deep graph convolution layers. Doubling depth from 4s to 8s increases FLOPs by 81.1% for a 2.33% accuracy gain. ASTM-Net-12s achieves the optimal balance, delivering SOTA accuracy with moderate FLOPs. While this study focuses on high-end GPUs, future work will explore lightweight variants and hardware optimization (e.g., quantization) for edge devices. Additionally, automated neural architecture search could further refine depth and branch configurations for specific use cases.

1 ) Dynamic 3A-Graphs with Activity-Conditioned Optimization.

Prior works either use fixed physical graphs [19] or learn global attention weights [25], neglecting the temporal dependency of node relationships. Our 3A-Graph framework innovates by:

1. a) Decomposing graph learning into stable (physical/trainable graphs) and dynamic (activity-related graph) components;
2. b) Conditioning spatial connections on localized temporal features through segment-wise similarity learning (i.e., (8));
3. c) Optimizing graphs via parameter-shared inference across time segments (s = 4), reducing parameters versus layer-specific alternatives [23].

2) Parameter-Efficient Multi-Branch Temporal Modeling.

TMGM redefines temporal convolution through heterogeneous branches that collaboratively capture:

1. a) Short-range dynamics (d=1 branch: 5-frame micro-motions);
2. b) Long-range dependencies (d=3 branch: 13-frame macro-trends);
3. c) Noise resilience (pooling branch: suppressing transient outliers);
4. d) Feature preservation (1*1 branch: maintaining raw signal fidelity);

Unlike transformer-based methods [29] that compute O(T²) attention weights, TMGM’s branched convolutions operate at O(T) complexity, achieving lower FLOPs than MST-GCN [54] while improving accuracy (as shown in Table 3).

1) Integration with Transformer Architectures.

The success of Transformers in modeling long-range dependencies suggests opportunities to enhance ASTM-Net’s temporal modeling capabilities. Replacing TMGM’s dilated convolutions with lightweight self-attention mechanisms could better capture global temporal patterns (e.g., periodic motions in “jumping rope”). Combining ASGM’s dynamic graphs with Transformer encoders may further exploit local-global spatiotemporal dependencies. To validate this, future work could compare the original ASTM-Net with transformer-enhanced variants such as ASTM-Transformer (replacing TMGM with vanilla multi-head self-attention) and ASTM-EfficientFormer (hybrid sparse attention + dilated convolution) on NTU-RGB + D and Toyota Smarthome datasets. Key metrics would include accuracy, FLOPs, and inference speed on edge devices like NVIDIA Jetson. Implementation steps could integrate shifted window attention to reduce quadratic complexity and apply dynamic token sparsification to prune non-salient frames. Quantizing attention weights to INT8 would further optimize edge deployment. Evaluation would focus on accuracy-FLOPs trade-offs and interpreting attention maps for critical motion phases. Challenges such as computational overhead could be mitigated via low-rank approximations or hardware-aware optimizations.

2) Self-Supervised Learning for Label Efficiency.

Current methods rely on fully annotated datasets, limiting scalability. Self-supervised pretraining via masked joint reconstruction or contrastive learning could improve label efficiency and robustness. For instance, pretraining on PKU-MMD V2 (5.4M unlabeled frames) with strategies like masking 40% of joints/frames (L1 reconstruction loss) or generating positive pairs via temporal cropping/jittering could enable robust feature learning. Downstream fine-tuning on NTU-RGB + D (10%/30%/100% labeled data) and Toyota Smarthome would validate label efficiency gains. Success metrics would include achieving >85% accuracy with 10% labels (vs. 70% supervised baseline) and improved robustness to occlusions (RND/CFO metrics). Visualization of pretrained embeddings via t-SNE could verify activity clustering. Cross-modal alignment with CLIP embeddings, if RGB data is available, might further enhance generalization to unseen domains like medical rehabilitation.

3) Privacy-preserving HAR via federated learning.

For sensitive applications like healthcare or smart homes, federated learning (FL) can enable collaborative model training without sharing raw skeleton data. One promising direction is extending ASGM to learn personalized 3A-Graphs on distributed devices while aggregating only graph parameters, avoiding the need to transmit raw data. Another avenue is injecting noise into graph updates during FL to prevent inference attacks on user-specific motion patterns, thus ensuring privacy. Furthermore, optimizing ASTM-Net’s architecture for varying computational capacities (e.g., smartphones vs. IoT sensors) in FL frameworks could further enhance its adaptability.

4) Multi-modal fusion for complex scenarios.

While ASTM-Net focuses on skeleton data, integrating complementary modalities could address its limitations in occlusion-heavy environments. RGB-skeleton fusion using attention-based modules could combine skeletal features with RGB appearance cues to improve performance in scenarios like object interactions (e.g., “open fridge”). Depth maps can refine node distances in the 3A-Graph, improving robustness to viewpoint changes. Additionally, for activities with sound cues, such as “clapping” or “typing,” synchronizing audio features with skeletal motions via cross-modal transformers could enhance recognition.

5) Edge deployment and hardware optimization.

To realize real-time HAR on resource-constrained devices, future work should explore quantization and pruning techniques. Post-training quantization could reduce ASTM-Net’s precision (FP32 → INT8), while pruning redundant graph connections would help to decrease model size and improve inference speed. Hardware-aware neural architecture search (NAS) could be employed to automatically design ASTM-Net variants optimized for specific hardware chips like NVIDIA Jetson or ARM CPUs. Furthermore, enabling incremental learning on edge devices could allow for adaptation to user-specific motion patterns without requiring cloud dependency.

6 ) Real-world deployment and industrial validation.

Although ASTM-Net achieves promising performance on benchmark datasets such as Toyota Smarthome, its practical deployment in real-world scenarios—especially in embedded systems like wearable devices and smart surveillance systems—remains a critical next step. For wearable device integration, hardware-specific optimization is necessary to adapt ASTM-Net to ultra-low-power ARM processors (e.g., Cortex-M7), involving pruning redundant graph connections and employing INT8 quantization for reduced power consumption. Comprehensive energy efficiency evaluations using commercial wearables such as Fitbit Sense and Apple Watch under continuous activity monitoring conditions would further ensure practical viability. Additionally, enabling on-device fine-tuning could facilitate personalized models capable of adapting to individual-specific motion patterns, particularly beneficial in medical scenarios like detecting gait abnormalities in Parkinson’s patients. For smart surveillance applications, ASTM-Net can be enhanced by incorporating multi-camera fusion strategies to robustly handle occlusions in crowded environments, leveraging synchronized skeleton streams from distributed camera systems. An edge-cloud hybrid deployment—deploying lightweight ASGM modules at edge devices for instantaneous alerts such as fall detection, while conducting deeper multi-scale TMGM analyses on cloud platforms—could balance responsiveness with computational efficiency. Moreover, adopting federated learning methodologies would enable privacy-preserving inference, anonymizing sensitive skeletal data across distributed surveillance nodes to ensure compliance with GDPR and CCPA regulations. Finally, establishing industrial partnerships, such as collaborations with healthcare providers like Philips Lifeline to validate the system’s reliability in elderly monitoring or integration with smart home manufacturers such as Bosch or Nest for real-time IoT activity recognition, would significantly facilitate the practical adoption and validation of ASTM-Net in real-world applications.