https://orcid.org/0000-0003-2548-3689
Figures
Abstract
3D point clouds, with their compact structural representation and rich geometric information, have become fundamental data sources for visual understanding tasks in computer vision, robotics, and intelligent systems.Despite extensive progress, many existing methods still place strong emphasis on local geometric modeling while exhibiting limited ability to capture global long-range contextual dependencies. Moreover, the increasing architectural complexity of modern models often leads to high computational cost and memory consumption. In this paper, we propose Point BiLSTM, an efficient and lightweight framework for 3D point cloud classification and segmentation. The core of the proposed method is a bidirectional long short-term memory (BiLSTM)-based sequencer module, which models long-range contextual dependencies among points with linear computational complexity, enabling effective global feature learning at a low cost. Considering the unordered nature of point clouds, we further propose a Mixed Sequence Soft Cross-Entropy Loss that jointly supervises fixed-order and randomly permuted point sequences during training. This design explicitly enhances robustness to permutation ambiguity and improves training stability. Extensive experiments conducted on three widely used benchmarks—ModelNet40, ScanObjectNN, and ShapeNet Part—demonstrate that Point BiLSTM achieves highly competitive performance. In particular, the proposed method attains the fastest inference speed on both idealized and real-world datasets, outperforming current state-of-the-art methods by 30.2% and 54.2%, respectively. In addition, Point BiLSTM significantly reduces computational complexity and memory consumption, providing an effective solution for efficient point cloud learning. Our code will be available at https://github.com/wendaodao04/PointBiLSTM.
Citation: Liu D, Deng Y, Zou J, Fan S (2026) Efficient and lightweight long-range modeling for 3d point cloud classification and segmentation. PLoS One 21(7): e0352793. https://doi.org/10.1371/journal.pone.0352793
Editor: Armin Moghimi, Leibniz University Hannover, GERMANY
Received: March 25, 2026; Accepted: June 15, 2026; Published: July 16, 2026
Copyright: © 2026 Liu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The datasets used in this study are publicly available. ModelNet40 is available at https://modelnet.cs.princeton.edu/, ScanObjectNN is available at https://hkust-vgd.github.io/scanobjectnn/, and ShapeNet Part is available at https://shapenet.org/. The source code supporting the findings of this study is publicly available at https://github.com/wendaodao04/PointBiLSTM.
Funding: This work was supported in part by the National Natural Science Foundation of China under Grant 62473078. There was no additional external funding received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
3D point clouds provide a direct representation of the geometric structure of the real world and have been widely used in intelligent manufacturing [1], robotic manipulation [2] and 3D reconstruction [3]. Unlike regular grid-based data, point clouds describe three-dimensional space in a discrete, sparse, and unordered manner, which gives them inherent advantages in representing complex geometric structures. However, this unstructured nature also poses significant challenges for efficient and robust feature learning. With the rapid development of LiDAR and depth sensing technologies, the scale of point cloud data continues to grow. How to maintain high recognition accuracy while controlling computational complexity and memory consumption has become a critical issue for advancing point cloud understanding methods toward practical applications.Point cloud classification and segmentation are two fundamental tasks in point cloud understanding. Point cloud classification focuses on learning global semantic representations for an entire 3D object. In contrast, point cloud segmentation requires fine-grained semantic prediction for individual points. Compared with classification, segmentation generally requires more effective modeling of local geometric structures and global contextual relationships under irregular point distributions, making it a more challenging task.
Existing point cloud understanding approaches can be broadly categorized into methods based on regular representations [4–8] and methods that operate directly on point sets [9–20]. The former typically transform point clouds into regular structures, such as voxels [4–7], enabling the use of convolutional neural networks [21]. While these methods exhibit certain advantages in local geometric modeling, the voxelization process inevitably introduces spatial information loss, and their computational and memory costs increase rapidly with resolution, limiting their applicability in high-precision or resource-constrained scenarios. In contrast, direct point-based methods avoid intermediate representations and are more suitable for fine-grained geometric modeling. Representative examples include the PointNet [9–11] family based on multilayer perceptrons and subsequent approaches that incorporate local neighborhood aggregation. Although these methods have achieved substantial progress in classification and segmentation tasks, their modeling capability still largely relies on local feature stacking, with relatively limited ability to capture global contextual relationships.
To enhance global modeling capability, Transformer-based architectures and their variants [12,15,22–24] have recently been introduced into the point cloud domain. By leveraging self-attention mechanisms, these methods explicitly model long-range dependencies among points and have achieved strong performance on multiple benchmark datasets. Meanwhile, emerging sequence modeling frameworks such as state-space models [25,26] have also been explored for point cloud feature learning to improve efficiency. However, such approaches typically depend on complex operator designs or exhibit quadratic time and memory complexity [27], making them difficult to scale efficiently as the number of points increases. From a practical perspective, the usability of a model is determined not only by peak accuracy but also by inference speed, memory footprint, and overall computational cost. Consequently, how to preserve global modeling capability while constructing more efficient and scalable point cloud learning frameworks remains an open research problem.
From a data-structural perspective, a point cloud is inherently an unordered set of points. Although prior studies have attempted to impose sequential structures through spatial sorting or local aggregation [13,15,28,29], introducing any fixed ordering may lead to additional uncertainty and potential bias, which has long raised concerns about directly applying traditional sequence models to point cloud tasks. However, the strength of sequence models does not lie in their reliance on a specific “correct order,” but rather in their ability to efficiently model long-range dependencies. If a model can be guided to learn feature representations that are robust to order variations without depending on a particular point sequence, sequence modeling frameworks may offer a lightweight and effective alternative for global point cloud modeling.
Based on this observation, we propose Point BiLSTM, a long-range contextual modeling framework for 3D point cloud classification and segmentation. The proposed method adopts a bidirectional long short-term memory (BiLSTM) network as the core sequence modeling module and employs a residual structure to capture bidirectional dependencies among point features, enabling effective aggregation of global semantic information. To further mitigate the uncertainty introduced by point cloud unorderedness, we introduce a Mixed Sequence Soft Cross-Entropy Loss, which jointly supervises fixed-order and randomly permuted sequence views during training, encouraging the model to learn feature representations that are robust to variations in point ordering.The main contributions of this work are summarized as follows:
- We propose a point cloud learning framework based on residual bidirectional long short-term memory networks, which captures long-range contextual dependencies with linear computational complexity and low memory overhead, making it suitable for both point cloud classification and segmentation tasks.
- We introduce a mixed sequence soft cross-entropy loss that jointly supervises fixed and randomly permuted point sequences, effectively alleviating sensitivity to point ordering while improving training stability.
- Experimental results on multiple standard point cloud benchmarks demonstrate that the proposed method achieves a favorable balance among classification accuracy, computational cost, and inference efficiency.
Related work
Methods based on convolution operators
Many existing approaches apply convolutional operations to 3D point clouds by designing specialized convolution kernels adapted to irregular point distributions [21,28,30–34]. Spherical CNN [32] addresses rotation variance by introducing spherical convolutions, while PointCNN [28] learns a permutation of unordered points to enable convolution-like operations. KPConv [21] defines convolution weights by locating kernel points in Euclidean space, and SpiderCNN [30] formulates continuous convolution kernels using parametric functions over local neighborhoods. InterpConv [33] further introduces interpolated convolution to capture geometric relationships between points and convolution weights, at the cost of expensive neighborhood search and sampling procedures.
Although these convolution-based methods have achieved strong performance on various point cloud understanding tasks, they generally rely on carefully customized kernel designs and complex local operators. Such designs make it difficult to construct a unified and efficient framework that generalizes well across different tasks, while often incurring non-negligible computational overhead.
Methods based on the transformer
The Transformer architecture has achieved remarkable success in natural language processing and image understanding [35,36], which has motivated extensive research on applying attention mechanisms to 3D point cloud processing. A variety of Transformer-based models have been proposed for point cloud classification and segmentation [4,12,14–16,22,23,37]. Point Transformer [14] pioneers the use of self-attention for point clouds by introducing vector self-attention to aggregate local features, while PVT [4] combines sparse voxel attention with point-based relative attention to leverage both voxel and point representations. Fast Point Transformer [37] reduces spatial complexity by designing lightweight local attention, and Stratified Transformer [12] expands the receptive field through hierarchical sampling strategies.
Despite their strong performance, Transformer-based methods generally suffer from high computational and memory costs due to the quadratic complexity of self-attention. As the number of points increases, these costs become a significant bottleneck, limiting scalability and efficiency. This motivates the exploration of alternative mechanisms for capturing long-range dependencies with lower computational overhead.
Methods based on LSTM
Long short-term memory (LSTM) networks and their variants have been widely used in sequence modeling tasks such as natural language processing [38], video analysis [39], and time series prediction [40]. By alleviating the vanishing gradient problem of recurrent neural networks, LSTM models are capable of capturing long-range dependencies in sequential data. In 3D vision, LSTM-based approaches have mainly been applied to temporally ordered point cloud streams. For example, LSTM networks have been used to model temporal information in multi-frame point cloud object detection [41], point cloud sequence prediction [42], and gesture recognition tasks [43]. These methods primarily focus on exploiting temporal continuity across consecutive point cloud frames. A small number of studies have explored LSTM-based architectures for static point cloud analysis [29], such as shape classification using fixed point sequences. Recent studies have further investigated point cloud semantic segmentation under few-shot and multimodal learning settings by leveraging cross-modal semantic priors and multimodal feature interaction [44–46]. These approaches improve semantic understanding capability under limited supervision conditions and reflect recent trends in generalized semantic learning for point cloud segmentation. However, existing approaches are generally limited to specific tasks and do not explicitly address permutation ambiguity or efficient long-range contextual modeling in unordered point sets. Consequently, the potential of classical sequence models for efficient and robust static point cloud understanding remains insufficiently explored.
Proposed method
Our proposed Point BiLSTM architecture is illustrated in Fig 1. It follows a hierarchical point processing pipeline, where standard sampling and neighborhood grouping operations are used to extract local structures. In the following sections, we describe each component in detail.
It includes classification and segmentation networks. The specific details of the residual sequencer block are illustrated in Fig 2. “” represents the loss function proposed in this paper.
Bidirectional LSTM
Bidirectional Long Short-Term Memory (BiLSTM) networks are a class of recurrent models designed to capture long-range dependencies in sequential data by jointly modeling forward and backward contextual information [47]. Unlike unidirectional recurrent models, BiLSTM processes an input sequence in both temporal directions, enabling each element to be informed by preceding and succeeding context simultaneously.
Given an input point feature sequence , a BiLSTM consists of a forward LSTM and a backward LSTM. The forward LSTM encodes contextual information in the original order, producing hidden states
, while the backward LSTM processes the reversed sequence and produces hidden states
. The output representation at each position is obtained by combining the hidden states from both directions:
where denotes a feature fusion operation, implemented as concatenation followed by a linear projection in our framework. The resulting representation
encodes bidirectional contextual information and serves as the output feature for the corresponding point.
In the context of static point cloud understanding, we treat point features as sequence elements without assuming a semantically meaningful order. The advantage of BiLSTM in this setting does not lie in exploiting a specific point ordering, but in its ability to efficiently propagate contextual information across the entire sequence with linear computational complexity. This property makes BiLSTM particularly suitable for modeling long-range dependencies in point clouds, providing an effective alternative to attention-based mechanisms with significantly lower computational and memory cost.
In our framework, the BiLSTM module functions as the core sequence modeling component and is integrated into a residual architecture, as described in the following subsection.
Residual sequencer block
Although point clouds are inherently unordered sets, organizing point features into sequences provides an effective way to model long-range contextual dependencies. To explicitly incorporate spatial information into the sequence modeling process, we introduce a relative positional encoding term , which represents the relative positional relationship between neighboring points. This positional encoding is fused with the features learned by the BiLSTM to enhance spatial awareness during sequence modeling. Specifically, the positional encoding function and the proposed Sequencer Layer are defined as follows:
Here, and
represent the 3D point coordinates for points i and j, respectively. The encoding function
is an MLP consisting of two linear layers and one ReLU nonlinearity.
denotes a set of points in the local neighborhood of
(K nearest neighbors). The mapping function
is an MLP with one linear layer and a ReLU nonlinearity.
represents the BiLSTM network. The positional information
is directly fused with the feature information
through a multiplicative relation, enhancing the spatial awareness of the feature representation. This approach allows the BiLSTM to not only capture the local context of each point but also combine the spatial information of the points, facilitating the capture of global spatial dependencies. The sequencer layer is illustrated in Fig 2.
By fusing sequential features with relative positional encodings, the proposed Sequencer Layer explicitly injects spatial awareness into the sequence modeling process. This design enables the BiLSTM to capture local geometric relationships while effectively modeling long-range contextual dependencies, thereby achieving global feature aggregation without relying on attention mechanisms. The overall structure of the residual sequencer block is illustrated in Fig 2 and highlighted in Fig 1. The block consists of a BiLSTM module, a linear projection layer, and a residual connection. The linear projection layer is employed to align the feature dimensions between the input and output of the sequencer, ensuring optimization stability and effective information propagation when stacking multiple layers.
Mixed sequence soft cross-entropy loss
Soft cross-entropy loss is widely adopted in point cloud classification tasks. It can be regarded as a variant of the standard cross-entropy loss with the introduction of label smoothing [48]. The standard cross-entropy loss is defined as
where K denotes the number of classes, represents the predicted probability for class i, and
denotes the ground-truth distribution:
To improve generalization ability and avoid overconfident predictions during training, label smoothing is introduced. The smoothed label distribution is defined as
where is a small hyperparameter controlling the smoothing strength.
However, point cloud data inherently exhibit an unordered structure, which introduces ambiguity when modeling them as sequences. To better exploit the complementary characteristics of ordered and unordered representations, we propose a mixed supervision strategy for sequence learning.
Specifically, two types of point cloud sequences are constructed. The first sequence is generated by fixing a random point order, which provides a consistent sequential view for feature extraction. The second sequence is produced through random permutations, which introduces diverse structural variations and enhances robustness. As illustrated in Fig 3, these two sequence views contain complementary geometric information that can benefit downstream tasks.
(a) A point cloud sequence with a randomly fixed order. (b) An arbitrarily permuted point cloud sequence with a different ordering from (a). Spheres with the same color represent the same point, and the numbers on the spheres indicate the point order in each sequence.
To jointly learn from these two views, we propose a mixed sequence soft cross-entropy loss defined as
where represents the loss computed from the fixed-order point cloud sequence, while
corresponds to the loss obtained from the randomly permuted sequence. The parameter
controls the trade-off between these two supervision signals.
By integrating both ordered and permutation-based supervision, the proposed M-Loss encourages the network to learn more robust and comprehensive representations from point cloud sequences.
Experiments
In this section, we comprehensively evaluate the Point BiLSTM model on several benchmark datasets. Specifically, ModelNet40 [49] and ScanObjectNN [50] are used for shape classification, ShapeNet parts dataset [51] serves as the dataset for part segmentation, respectively.Subsequently, detailed ablation experiments demonstrate the effectiveness of Point BiLSTM in the process of point cloud feature learning.
Datasets and implementation details
Datasets. We evaluate the proposed Point BiLSTM on three widely used benchmarks for 3D point cloud understanding, covering both classification and segmentation tasks.
ModelNet40 [49] is a synthetic CAD dataset consisting of 12,311 shapes from 40 object categories. Following the official split, 9,843 shapes are used for training and 2,468 for testing. This dataset is commonly adopted to evaluate the shape-level classification ability of point cloud models under clean and well-aligned conditions.
ScanObjectNN [50] is a real-world point cloud dataset captured by RGB-D sensors. It contains 15 object categories with 2,902 unique object instances and includes challenging factors such as background clutter, occlusion, and noise. In our experiments, we focus on the most challenging variant, PB_T50_RS, which introduces random background points and partial observations. ScanObjectNN is considered a strong benchmark for evaluating the robustness of point cloud classification methods.
ShapeNet Part [51] is a large-scale dataset for 3D part segmentation, containing 16 object categories and 50 annotated part labels. Each shape is associated with fine-grained per-point part annotations, making it a standard benchmark for evaluating point-level segmentation performance.
Implementation details. For all experiments, we uniformly sample a fixed number of points from each shape as network input. Specifically, 1,024 points are sampled for both ModelNet40 and ScanObjectNN, while 2,048 points are used for ShapeNet Part segmentation following common practice. The point coordinates are normalized to zero mean and unit sphere before being fed into the network. For classification tasks, stochastic gradient descent (SGD) with a momentum of 0.9 is employed for optimization. The initial learning rate follows the standard setting used in previous works and is decayed using a cosine annealing strategy. All classification models are trained for 300 epochs on ModelNet40 and 350 epochs on ScanObjectNN with a batch size of 32. For the part segmentation task, we follow the standard training protocol adopted in previous methods, including data augmentation operations such as random scaling and jittering, and train the model for 350 epochs. The proposed mixed sequence soft cross-entropy loss is applied to both classification and segmentation tasks. Unless otherwise specified, all hyperparameters are kept consistent across different datasets to ensure fair comparisons. In all experimental result tables, bold numbers indicate the best performance in each category, while the symbol “-” denotes that the corresponding metric is not reported in the original paper or official implementation. Test Speed represents the number of samples processed per second. All experiments are conducted on two Tesla P40 GPUs, and all reported test speeds are evaluated under the same hardware environment to ensure fair comparisons. The parameter counts and FLOPs reported in the comparison tables are obtained from the corresponding references.
Results on ModelNet40
The classification results on ModelNet40 are summarized in Table 1. Point BiLSTM achieves an overall accuracy (OA) of 93.4%, which is competitive with recent state-of-the-art methods.Compared with PointMLP, our method yields slightly lower mean class accuracy (mAcc), but requires significantly fewer parameters (3.99 MB vs. 12.6 MB) and achieves substantially higher inference speed (574 vs. 154 samples/s). Compared with the previously fastest method, Multi-DL, the proposed Point BiLSTM achieves more than a 30% improvement in inference speed, while simultaneously attaining higher classification accuracy. This highlights the efficiency advantage of the proposed sequence-based modeling strategy.An important observation from Table 1 is that Point BiLSTM achieves a favorable balance between accuracy and computational efficiency.While several high-performing methods rely on deep MLP stacks or attention-based mechanisms, our approach attains comparable performance with a much lighter model. Notably, the inference speed of Point BiLSTM is approximately 3.7× faster than that of PointMLP, indicating that bidirectional sequence modeling enables effective long-range context aggregation without excessive architectural depth. Moreover, compared to attention-based models such as PCT and PointConT, Point BiLSTM avoids quadratic complexity with respect to the number of points, resulting in reduced FLOPs and faster execution. These results demonstrate that Point BiLSTM can effectively capture global shape semantics while maintaining high computational efficiency.
Results on ScanObjectNN
The experimental results on the ScanObjectNN dataset are reported in Table 2. Point BiLSTM achieves an mAcc of 86.7% and an OA of 87.9%, demonstrating competitive performance under challenging real-world conditions with background noise and occlusions. Although Mamba3D attains slightly higher overall accuracy, it involves a substantially larger model size (16.9 MB vs. 3.99 MB) and exhibits significantly lower inference speed (43 vs. 407 samples/s). In contrast, Point BiLSTM provides a more favorable trade-off among memory consumption, computational complexity, and accuracy, making it more suitable for scenarios with limited computational resources. Further observations from Table 2 indicate that Point BiLSTM exhibits a relatively small gap between mAcc and OA, suggesting more balanced performance across different object categories. This implies that the proposed bidirectional sequence modeling strategy is effective in capturing global contextual information even in the presence of background interference and partial observations.
Fig 4 compares the training convergence behavior of different models. Point BiLSTM demonstrates stable convergence throughout the training process. Compared with PCT, the proposed method shows improved training stability, which can be attributed to the explicit modeling of point order variations introduced by the mixed sequence soft cross-entropy loss during training. Although PointMLP exhibits slightly smoother convergence curves, it relies on more complex nonlinear mapping structures, leading to higher training and inference costs. Overall, the experimental results on ScanObjectNN further verify that Point BiLSTM remains effective in realistic point cloud classification scenarios and exhibits strong robustness.
Results on ShapeNet part
In addition to shape classification, we further evaluate the proposed Point BiLSTM framework on the part segmentation task, and the quantitative results are reported in Table 3. Point BiLSTM achieves an instance-level mIoU of 85.5%. Although the proposed framework is primarily designed for efficient global context modeling, it still demonstrates good generalization capability on the part segmentation task. The results indicate that bidirectional sequence modeling is effective in capturing structural consistency and long-range contextual dependencies, which are particularly important for part segmentation of objects with coherent global geometry. For objects with highly complex local structures or fine-grained part decompositions, methods that emphasize intensive local feature aggregation may achieve higher segmentation accuracy. This observation reflects a common trade-off between global context modeling and fine-grained local representation, and is consistent with the design philosophy of Point BiLSTM, which prioritizes efficiency and long-range dependency modeling rather than exhaustive local detail extraction. Compared with attention-based approaches, Point BiLSTM attains comparable instance-level segmentation performance while employing a considerably simpler network architecture. This suggests that explicit bidirectional sequence modeling can serve as a viable and efficient alternative for capturing global dependencies in part segmentation, without relying on computationally expensive self-attention mechanisms. In addition, Fig 5 presents qualitative visualizations of part segmentation results.
Points aremarked with different colors based on the parts to which they belong. The first rowshows the model’s predicted results, and the second row displays the Ground Truth.
Ablation study
Effect of network depth and sequence length. We first investigate the influence of network depth by varying the number of residual sequencer blocks from 1 to 4, and the results are summarized in Table 4. When only a single block is used, the model exhibits limited representation capacity. Increasing the depth to two blocks leads to clear performance improvements. However, further increasing the depth to four blocks results in performance degradation, accompanied by increased computational and memory overhead. These results indicate that blindly increasing network depth does not necessarily benefit point cloud learning. A moderate depth achieves a better balance between representation capability, training stability, and efficiency. In all subsequent experiments, the number of residual sequencer blocks is set to two.
We further analyze the influence of sequence length on sequential contextual modeling performance, and the corresponding results are presented in Table 5. When the sequence length is relatively short, the model is unable to sufficiently capture long-range contextual dependencies, resulting in limited feature interaction capability. As the sequence length increases from 64 to 256, the classification performance consistently improves, demonstrating that richer contextual interaction contributes to more discriminative global representation learning. However, further increasing the sequence length to 512 yields only marginal performance gains while significantly increasing computational and memory costs. Moreover, excessively long sequences may propagate redundant or noisy contextual information during recurrent interaction, which can negatively affect model robustness. Therefore, an appropriate sequence length achieves a better trade-off between contextual modeling capability and computational efficiency. According to the experimental results, the sequence length is set to 256 in the final model configuration.
We also investigate the influence of different sampling strategies on sequence token construction and contextual interaction quality. The corresponding results are summarized in Table 6. Random Sampling exhibits relatively unstable performance due to irregular spatial point distributions, which may weaken sequential contextual interaction and reduce feature consistency. Uniform Sampling improves spatial coverage to some extent and achieves better performance than Random Sampling. In contrast, the proposed framework achieves the best performance when using FPS, since FPS preserves more uniformly distributed and representative geometric structures, thereby facilitating more stable long-range contextual dependency modeling. These results further demonstrate that representative spatial point distributions are essential for effective sequential feature learning in point cloud analysis.
Effectiveness of mixed sequence soft cross-entropy loss. To evaluate the effectiveness of the proposed mixed sequence soft cross-entropy loss and investigate the sensitivity to the mixing coefficient , we vary
from 0 to 1 with an interval of 0.25. The corresponding results are summarized in Table 7. Here,
indicates that supervision is performed using only randomly permuted sequences, whereas
corresponds to supervision using only fixed-order sequences. Intermediate values of
represent different mixing ratios between fixed-order and randomly permuted sequence supervision. As shown in Table 7, relying solely on a single sequence view leads to inferior performance compared with mixed supervision. In particular, when
, the model achieves the lowest performance, suggesting that supervision based only on randomly permuted sequences weakens the stability of sequential contextual learning. As
gradually increases from 0 to 0.5, both mAcc and OA consistently improve, indicating that introducing fixed-order sequence supervision provides more stable and discriminative contextual representations. However, further increasing
from 0.5 to 1.0 results in performance degradation, implying that excessive dependence on fixed-order sequences may reduce robustness to point-order variations. Overall, the best performance is achieved when
, demonstrating that a balanced combination of fixed-order and randomly permuted sequence supervision provides the most effective trade-off between order consistency and permutation robustness. These results further validate both the effectiveness of the proposed mixed sequence soft cross-entropy loss and the importance of selecting an appropriate mixing coefficient.
Effectiveness of residual sequencer block. We further investigate the contribution of the residual sequencer block and different sequence modeling units. Under the same network depth and loss function settings, three variants are compared: without sequence modeling, with LSTM, and with BiLSTM. The corresponding results are presented in Table 8. Removing the sequencer block leads to a significant performance degradation, indicating that sequential contextual interaction plays a critical role in point cloud representation learning. Introducing LSTM substantially improves performance by enabling long-range contextual dependency modeling. Furthermore, replacing LSTM with BiLSTM consistently achieves better results, demonstrating that bidirectional sequence modeling is more effective than unidirectional modeling for capturing contextual relationships in unordered point cloud data. By aggregating contextual information from both forward and backward directions simultaneously, BiLSTM provides more comprehensive sequential feature interaction and stronger global representation capability. We additionally analyze the effectiveness of incorporating relative positional encoding into the residual sequencer block, and the corresponding results are summarized in Table 9. Without relative positional encoding, the model performance decreases noticeably, indicating that sequential feature interaction alone is insufficient for fully capturing geometric relationships in point clouds. After introducing relative positional encoding, both mAcc and OA are consistently improved, demonstrating that explicit geometric priors enhance the spatial awareness and discriminative capability of sequence-based contextual modeling.
Effectiveness of a randomly assigned but fixed point cloud order. Finally, we study whether imposing a specific deterministic order benefits model performance. We compare randomly fixed orders with orders obtained by sorting points along the x-, y-, and z-axes. The results are reported in Table 10. All ordering strategies achieve comparable performance, with no consistent advantage observed for axis-based sorting. Moreover, deterministic sorting introduces additional preprocessing cost. These results indicate that the proposed framework does not rely on a specific point order and is capable of learning order-invariant representations through sequence modeling and mixed-sequence supervision.
Failure case analysis
Although our method achieves competitive results on multiple public datasets, it still has certain limitations. As shown in Fig 6, tables and chairs contain highly similar structural components, particularly at the junctions between supporting legs and flat surfaces. These locally similar geometric structures may be misinterpreted as belonging to the same semantic category during sequence-based contextual modeling, leading to contextual feature ambiguity. While modeling long-range dependencies enhances global feature aggregation and cross-region contextual interaction, the model may still struggle to distinguish subtle differences in highly symmetric or repetitive local geometries, which can affect fine-grained semantic boundary discrimination. This highlights an inherent challenge of sequence-based modeling: local geometric similarity can conflict with global contextual information, causing similar local structures to be incorrectly aggregated into the same contextual representation and resulting in feature confusion. Future work may address these limitations by incorporating more discriminative geometry-aware contextual interaction mechanisms and adaptive sequence construction strategies to further improve the robustness of sequence-based point cloud understanding.
Geometrically similar local structures, such as the junction regions between table/chair legs and flat surface structures, may introduce contextual ambiguity during sequence-based feature interaction, leading to feature confusion. Different colors are used to facilitate distinguishing between the legs and flat surface structures of tables and chairs.
Conclusion
In this paper, we presented Point BiLSTM, an efficient framework for 3D point cloud classification and segmentation based on bidirectional sequence modeling. By introducing a residual BiLSTM-based sequencer, the proposed method is able to capture long-range contextual dependencies with linear computational complexity, providing an effective mechanism for global feature aggregation. To address the unordered nature of point clouds, we further proposed a mixed sequence soft cross-entropy loss, which jointly supervises fixed-order and randomly permuted point sequences. This design explicitly encourages robustness to point order variations and improves training stability. Extensive experiments and ablation studies on multiple benchmark datasets demonstrate that Point BiLSTM achieves competitive performance while significantly reducing model complexity, memory usage, and inference latency. These results highlight that explicit bidirectional sequence modeling can serve as a lightweight and effective alternative for global context learning in point cloud understanding. Rather than replacing existing paradigms, this work provides a complementary perspective on point cloud modeling, bridging classical sequence learning and modern 3D perception. Future work will explore extending the proposed framework to other 3D tasks, such as object detection and scene-level understanding.
References
- 1. Liu Y, Zhang C, Dong X, Ning J. Point Cloud-Based Deep Learning in Industrial Production: A Survey. ACM Comput Surv. 2025;57(7):1–36.
- 2.
Yang Z, Chen L, Sun Y, Li H. Visual Point Cloud Forecasting Enables Scalable Autonomous Driving. In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 14673–84. https://doi.org/10.1109/cvpr52733.2024.01390
- 3. Yan L, Song J, Xie H, Wei P, Li G, Zhu L, et al. LiDGS: An efficient 3D reconstruction framework integrating lidar point clouds and multi-view images for enhanced geometric fidelity. International Journal of Applied Earth Observation and Geoinformation. 2025;142:104730.
- 4. Zhang C, Wan H, Shen X, Wu Z. PVT: Point-voxel transformer for point cloud learning. International Journal of Intelligent Systems. 2022;37(12):11985–2008.
- 5.
Mao J, Xue Y, Niu M, Bai H, Feng J, Liang X, et al. Voxel Transformer for 3D Object Detection. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 3144–53. https://doi.org/10.1109/iccv48922.2021.00315
- 6.
Liu Z, Tang H, Lin Y, Han S. Point-voxel CNN for efficient 3D deep learning. In: Advances in Neural Information Processing Systems, 2019.
- 7.
Riegler G, Ulusoy AO, Geiger A. OctNet: Learning Deep 3D Representations at High Resolutions. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 6620–9. https://doi.org/10.1109/cvpr.2017.701
- 8. Guo Y, Wang H, Hu Q, Liu H, Liu L, Bennamoun M. Deep Learning for 3D Point Clouds: A Survey. IEEE Trans Pattern Anal Mach Intell. 2021;43(12):4338–64. pmid:32750799
- 9.
Charles RQ, Su H, Kaichun M, Guibas LJ. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 77–85. https://doi.org/10.1109/cvpr.2017.16
- 10. Qi CR, Yi L, Su H, Guibas LJ. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems. vol. 30; 2017
- 11.
Ma X, Qin C, You H, Ran H, Fu Y. Rethinking network design and local geometry in point cloud: A simple residual MLP framework. 2022. https://arxiv.org/abs/2202.07123
- 12.
Lai X, Liu J, Jiang L, Wang L, Zhao H, Liu S, et al. Stratified Transformer for 3D Point Cloud Segmentation. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 8490–9. https://doi.org/10.1109/cvpr52688.2022.00831
- 13. Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, Solomon JM. Dynamic Graph CNN for Learning on Point Clouds. ACM Trans Graph. 2019;38(5):1–12.
- 14.
Zhao H, Jiang L, Jia J, Torr P, Koltun V. Point Transformer. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 16239–48. https://doi.org/10.1109/iccv48922.2021.01595
- 15. Guo M-H, Cai J-X, Liu Z-N, Mu T-J, Martin RR, Hu S-M. PCT: Point cloud transformer. Comp Visual Med. 2021;7(2):187–99.
- 16.
Wu X, Lao Y, Jiang L, Liu X, Zhao H. Point Transformer V2: Grouped Vector Attention and Partition-Based Pooling. In: Advances in Neural Information Processing Systems 35, 2022. 33330–42. https://doi.org/10.52202/068431-2415
- 17.
Shi S, Wang X, Li H. PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 770–9. https://doi.org/10.1109/cvpr.2019.00086
- 18.
Zhao H, Jiang L, Fu C-W, Jia J. PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 5560–8. https://doi.org/10.1109/cvpr.2019.00571
- 19.
Zhang F, Fang J, Wah B, Torr P. Deep FusionNet for Point Cloud Semantic Segmentation. Lecture Notes in Computer Science. Springer International Publishing. 2020. p. 644–63. https://doi.org/10.1007/978-3-030-58586-0_38
- 20.
Te G, Hu W, Zheng A, Guo Z. RGCNN: Regularized graph CNN for point cloud segmentation. In: ACM International Conference on Multimedia, 2018. 746–54.
- 21.
Thomas H, et al. KPConv: Flexible and deformable convolution for point clouds. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. 6411–20.
- 22. Liu Y, Tian B, Lv Y, Li L, Wang F-Y. Point Cloud Classification Using Content-Based Transformer via Clustering in Feature Space. IEEE/CAA J Autom Sinica. 2024;11(1):231–9.
- 23.
Park J, Lee S, Kim S, Xiong Y, Kim HJ. Self-Positioning Point-Based Transformer for Point Cloud Understanding. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 21814–23. https://doi.org/10.1109/cvpr52729.2023.02089
- 24. Zhou W, Zhao Y, Xiao Y, Min X, Yi J. TNPC: Transformer-based network for point cloud classification. Expert Systems with Applications. 2024;239:122438.
- 25.
Han X, Tang Y, Wang Z, Li X. Mamba3D: Enhancing local features for 3D point cloud analysis via state space model. In: 2024. 4995–5004.
- 26.
Liang D, Zhou X, Wang X, Zhu X, Xu W, Zou Z. PointMamba: A simple state space model for point cloud analysis. 2024. https://arxiv.org/abs/240210739
- 27.
Lu D, Xie Q, Wei M, Gao K, Xu L, Li J. Transformers in 3D point clouds: A survey. 2022. https://arxiv.org/abs/2205.07417
- 28.
Li Y, et al. PointCNN: Convolution on X-transformed points. In: Advances in Neural Information Processing Systems, 2018.
- 29.
Liu D, Deng Y, Shang P, Zou J, Fan S. A Multi-scale Deep LSTM Model for 3D Point Cloud Classification. In: 2025 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), 2025. 1–5. https://doi.org/10.1109/bmsb65076.2025.11165580
- 30.
Xu Y, Fan T, Xu M, Zeng L, Qiao Y. SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters. Lecture Notes in Computer Science. Springer International Publishing. 2018. p. 90–105. https://doi.org/10.1007/978-3-030-01237-3_6
- 31.
Atzmon M, Maron H, Lipman Y. Point convolutional neural networks by extension operators. In: 2018. https://arxiv.org/abs/1803.10091
- 32. Esteves C, Allen-Blanchette C, Makadia A, Daniilidis K. Learning SO(3) Equivariant Representations with Spherical CNNs. Int J Comput Vis. 2019;128(3):588–600.
- 33.
Mao J, Wang X, Li H. Interpolated Convolutional Networks for 3D Point Cloud Understanding. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019. 1578–87. https://doi.org/10.1109/iccv.2019.00166
- 34. Baek H, Yun WJ, Park S, Kim J. Stereoscopic scalable quantum convolutional neural networks. Neural Netw. 2023;165:860–7. pmid:37437364
- 35.
Parmar N. Image Transformer. In: Proceedings of the International Conference on Machine Learning, 2018. 4055–64.
- 36. Zhang N, Yu L, Zhang D, Wu W, Tian S, Kang X, et al. CT-Net: Asymmetric compound branch Transformer for medical image segmentation. Neural Netw. 2024;170:298–311. pmid:38006733
- 37.
Park C, Jeong Y, Cho M, Park J. Fast Point Transformer. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 16928–37. https://doi.org/10.1109/cvpr52688.2022.01644
- 38.
Wang S, Jiang J. Learning natural language inference with LSTM. In: 2015. https://arxiv.org/abs/1512.08849
- 39.
Wang Y, et al. Eidetic 3D LSTM: A model for video prediction and beyond. In: International Conference on Learning Representations; 2018.
- 40. Jin N, Yang F, Mo Y, Zeng Y, Zhou X, Yan K, et al. Highly accurate energy consumption forecasting model based on parallel LSTM neural networks. Advanced Engineering Informatics. 2022;51:101442.
- 41.
Huang R, Zhang W, Kundu A, Pantofaru C, Ross DA, Funkhouser T, et al. An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds. Lecture Notes in Computer Science. Springer International Publishing. 2020. p. 266–82. https://doi.org/10.1007/978-3-030-58523-5_16
- 42. Zhang C, Fiore M, Murray I, Patras P. CloudLSTM: A Recurrent Neural Model for Spatiotemporal Point-cloud Stream Forecasting. AAAI. 2021;35(12):10851–8.
- 43.
Min Y, Zhang Y, Chai X, Chen X. An Efficient PointLSTM for Point Clouds Based Gesture Recognition. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 5760–9. https://doi.org/10.1109/cvpr42600.2020.00580
- 44.
An Z, et al. Generalized few-shot 3D point cloud segmentation with vision-language model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 16997–7007.
- 45.
An Z, et al. Multimodality helps few-shot 3D point cloud semantic segmentation. In: International Conference on Learning Representations. 2025. 55414–35.
- 46.
An Z, et al. Rethinking few-shot 3D point cloud semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024;3996–4006.
- 47.
Graves A. Supervised sequence labelling with recurrent neural networks. Berlin, Germany: Springer. 2012.
- 48.
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the Inception Architecture for Computer Vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 2818–26. https://doi.org/10.1109/cvpr.2016.308
- 49.
Zhirong W, Song S, Khosla A, Fisher Yu, Linguang Z, Xiaoou T, et al. 3D ShapeNets: A deep representation for volumetric shapes. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. 1912–20. https://doi.org/10.1109/cvpr.2015.7298801
- 50.
Uy MA, Pham Q-H, Hua B-S, Nguyen T, Yeung S-K. Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019. 1588–97. https://doi.org/10.1109/iccv.2019.00167
- 51. Yi L, Kim VG, Ceylan D, Shen I-C, Yan M, Su H, et al. A scalable active framework for region annotation in 3D shape collections. ACM Trans Graph. 2016;35(6):1–12.
- 52. Liu X, Han Z, Liu Y-S, Zwicker M. Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-Based Sequence to Sequence Network. AAAI. 2019;33(01):8778–85.