Framework overview

This paper proposes a concept-aware semantic consistency modeling framework for multi-translation comparison of the Analects, as illustrated in Fig 1. Given multiple English translations, we first construct sentence-level aligned groups based on shared chapter and sentence IDs. We then perform unified cleaning and structural preprocessing on the input texts. Next, we introduce two optional normalization modules, namely concept-term normalization and proper-noun normalization. These modules reduce surface variation caused by lexical choice and naming differences. The processed sentences are then fed into a shared sentence encoder that maps them into a common semantic representation space. This space serves as the basis for subsequent consistency modeling and discrepancy analysis.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Concept-aware semantic consistency modeling framework for multiple English translations of the Analects.

https://doi.org/10.1371/journal.pone.0358227.g001

At the representation learning stage, the framework uses sentence-level alignments indexed by the same chapter and sentence IDs as supervision. It applies supervised contrastive learning to bring different translations of the same saying closer in the representation space. It also pushes apart semantically unrelated sayings. In addition, we introduce a concept-aware constraint. This constraint encourages expressions that share the same core concept to remain close across translations. The combination of sentence-level supervision and concept-level constraints enables the framework to capture both local semantic consistency and broader patterns of concept transfer. Finally, we evaluate the learned sentence representations through two primary downstream tasks, together with a supplementary score-label association analysis. This design yields a quantifiable, comparable, and reproducible framework for empirical analysis.

Input processing and normalization

The input to our framework consists of M English translations of the Analects, denoted by the translation set . To ensure comparability across translations, we use chapter and sentence IDs as a unified index. We then organize the translations into sentence-level aligned groups. Specifically, for each chapter–sentence index (c,i), we construct a translation group , where denotes the sentence from version T^(m) at that position. Because the available translations differ in formatting and completeness, some indices may be missing in certain versions. In such cases, the corresponding position is retained and marked as NULL. Table 1 shows an example of this processing procedure, including the aligned sentence groups, concept tags, and normalized inputs.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. An example of input processing for aligned sentence groups of the Analects.

https://doi.org/10.1371/journal.pone.0358227.t001

At the structural processing stage, we first perform chapter parsing and sentence segmentation for each translation. We then extract the chapter–sentence hierarchy and map each translated segment to the unified index (c,i). The alignment step follows a hard alignment strategy based on identical chapter and sentence IDs. Hard alignment was applied only when chapter and sentence identifiers were unambiguous. After text extraction and structural recovery, we checked identifier continuity, duplicate records, missing entries, and abnormal text lengths. Particular attention was given to the Giles translation because of its partial and thematic organization. Uncertain or unmatched passages were marked as missing and excluded from positive-pair construction and retrieval evaluation. Based on the aligned groups, sentence-level positive samples are constructed from non-empty translations under the same index, whereas samples from different indices are treated as negative candidates.

To reduce the effect of non-content noise, we further apply unified cleaning and quality control to the aligned sentence groups. First, we remove non-body content and normalize whitespace, punctuation forms, and abnormal characters. Second, we filter anomalous fragments according to length and structural features, such as overly short incomplete sentences, excessively long concatenated sentences, and entries that contain only indices or symbols. Finally, we compute basic corpus statistics, including the number of valid sentence groups, missing rates across translations, and sentence-length distributions. These steps improve the consistency and reproducibility of the input data at the whole-text level.

After cleaning, we construct normalized inputs to reduce the effect of surface variation on cross-translation consistency learning. We use two switchable normalization modules. The first is concept-term normalization. This module maps different renderings of the same Confucian concept to a unified concept label based on a predefined concept lexicon. For example, expressions such as gentleman, superior man, and related variants can be linked to the same underlying concept tag when they refer to junzi.

The concept lexicon was developed as a manually curated expert knowledge resource for the five English translations examined in this study. Candidate lexical variants were collected from the translations and were assigned to the same concept label only when contextual examination indicated that they referred to the same underlying Confucian concept. The resulting lexicon contains 37 concept labels and 266 lexical variants, covering 431 of the 505 aligned groups in the Core Aligned dataset, with a coverage rate of 85.35%. Each concept label contains 7.19 variants on average, and each covered aligned group contains 2.92 concept labels on average. The lexicon was finalized before representation-learning training and was subsequently treated as a fixed deterministic resource. The lexicon contains no consistency labels, split identifiers, similarity scores, or evaluation results, and it was not revised according to validation or test performance. Concept normalization introduces an explicit manually curated expert prior rather than learning concept equivalence through representation optimization. Because lexical variants from all five translations were consulted during lexicon construction, including expressions occurring in held-out groups, this resource should be understood as a closed-corpus expert lexicon rather than a training-only or externally defined lexicon.

The second module is proper-noun normalization. This module handles transliterations, aliases, and spelling variants of named entities. It is designed to reduce artificial divergence caused by differences in romanization or naming conventions across translations. Both modules operate only on the target lexical items. They leave the remaining sentence context unchanged and preserve the original forms for subsequent error analysis and case studies.

In addition to normalization, the framework also records concept annotations for each aligned sentence. These annotations serve two purposes. First, they provide explicit semantic tags for downstream concept-aware supervision. Second, they improve the interpretability of low-consistency cases by making concept-level variation directly traceable. In Table 1, the columns “Concept Tags” and “Concept-normalized Form” correspond to the annotation results and normalized inputs, respectively. After these steps, each non-empty translation sentence is tokenized and converted into a model-readable input sequence using a unified maximum length and truncation strategy.

Sentence representation learning with supervised contrastive learning

After input processing, we use a shared sentence encoder to learn unified representations for aligned translations, as illustrated in Fig 2. Given a non-empty translation sentence , where m indexes the translation version and (c,i) denotes the chapter and sentence ID, the encoder first produces a contextual sentence representation . This representation is then passed to a lightweight projection head and normalized to obtain the contrastive representation:

(1)

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Architecture of the Proposed Concept-aware Contrastive Learning Framework.

https://doi.org/10.1371/journal.pone.0358227.g002

Here, denotes the encoder output, and denotes the projected representation used for contrastive learning. The encoder parameters are shared across all translation versions. As a result, sentences from different translations are mapped into a common semantic space. This design improves cross-version comparability and provides a unified representation basis for semantic consistency modeling.

The use of a shared encoder is important for multi-translation comparison. Different English translations of the same saying often vary substantially in wording, syntax, and stylistic preference. If each version were modeled independently, the resulting representations would be less directly comparable. By contrast, the shared encoder forces semantically related sentences from different translations to be represented within the same embedding space. The projection head separates the representation used for contrastive optimization from the encoder output. This design facilitates contrastive training while preserving the semantic information encoded by the backbone model. In our setting, it also provides a cleaner representation space for modeling cross-translation alignment.

At the sentence level, we construct a supervised contrastive objective from the aligned sentence groups. For an anchor sample , all non-empty translations with the same chapter and sentence ID are treated as sentence-level positive samples. These samples correspond to different English renderings of the same source saying. By contrast, samples with different indices are treated as negatives. Let denote the set of valid anchors and P(a) the sentence-level positive set for anchor a. The sentence-level supervised contrastive loss is defined as

(2)

where is the training batch, denotes cosine similarity, and is a temperature parameter. This objective explicitly encourages different translations of the same saying to be close to each other, while pushing apart semantically unrelated sayings. In this way, the model learns translation-consistent sentence embeddings rather than representations dominated by surface lexical variation. This sentence-level contrastive objective forms the basic alignment mechanism of the proposed framework and serves as the foundation for the concept-aware constraint introduced in the next subsection.

Concept-aware constraint and semantic consistency inference

Although sentence-level supervised contrastive learning can align different translations of the same saying, it does not explicitly model the stability of key concepts across different sentence contexts. This limitation is important for the Analects, where many core concepts, such as ren, li, and junzi, recur across sayings and are often rendered in different lexical forms. To further strengthen concept-level semantic consistency, we introduce a concept-aware constraint in addition to sentence-level supervision. Let denote the set of concept tags associated with sentence . If two samples share at least one concept tag, they are treated as concept-level semantic neighbors. In this way, the model is encouraged to preserve not only sentence-level alignment for parallel translations, but also concept-level coherence across different sayings.

For an anchor a, the concept-level positive set is defined as

(3)

where denotes the current training batch. Based on Q(a), we define a concept consistency loss . This loss takes the same contrastive form as , but uses concept-sharing neighbors as positives rather than sentences aligned by the same chapter and sentence ID. The final training objective is

(4)

where controls the relative contribution of concept-level supervision. The two losses play complementary roles. The sentence-level loss aligns translations of the same saying. The concept-aware loss promotes coherence among sentences that share core concepts across different contexts. Together, they enable the representation space to capture both local translation alignment and broader patterns of concept transfer.

At inference time, we use the trained shared sentence encoder to generate a final representation for each non-empty translation sentence. Semantic consistency across translations is then measured by cosine similarity in the learned representation space. Specifically, for any two translation sentences and , their consistency score is defined as

(5)

where and denote the -normalized encoder representations used for downstream evaluation. A higher score indicates stronger semantic proximity between two translations. This score provides a unified basis for cross-translation retrieval and low-consistency detection. It is also used in a supplementary analysis of the association between continuous similarity scores and binary human labels.

Dataset construction

The dataset used in this study consists of five publicly available English translations of the Analects, namely those by James Legge (https://www.gutenberg.org/ebooks/3330), Leonard A. Lyall (https://www.gutenberg.org/ebooks/24055), Lionel Giles (https://www.gutenberg.org/ebooks/46389), William Jennings (https://archive.org/details/dli.ministry.25525), and William Edward Soothill (https://archive.org/details/analectsofconfuc00confrich). Table 2 summarizes the source formats and coverage of these translations. All versions are obtained from openly accessible digital sources. They provide representative material for modeling semantic consistency across multiple English translations of the Analects. Among them, the versions by Legge, Lyall, Jennings, and Soothill are complete translations of the full text, whereas Giles’s version adopts a selective and thematically reorganized format.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Source information of the English translations used in this study.

https://doi.org/10.1371/journal.pone.0358227.t002

To construct a computable parallel corpus, we first perform version-level structural parsing on the raw texts. For versions with relatively clear chapter structure, we establish a unified index and map each translation onto a shared chapter–sentence skeleton. For versions with substantial noise or different editorial formats, we first conduct main-text extraction, paragraph recovery, and chapter localization, and then perform sentence segmentation and index alignment. To ensure cross-translation comparability, we adopt a hard alignment strategy based on identical chapter and sentence IDs. If an entry is missing or cannot be reliably aligned, its index position is retained and marked as NULL, thereby preserving a unified whole-text structure. The corpus contains 505 aligned chapter–sentence groups. Legge contains 497 valid groups and 8 missing groups, corresponding to a missing rate of 1.58%. Lyall contains 504 valid groups and 1 missing group, with a missing rate of 0.20%. Confidence filtering retained 311 Giles groups, 468 Jennings groups, and 500 Soothill groups, corresponding to missing rates of 38.42%, 7.33%, and 0.99%, respectively. Across the ten translation-pair combinations, the retained data yield 4,104 pair instances. The aligned groups were divided into training, validation, and test sets using a 70%/15%/15% split. The 70%/15%/15% partition controls the data used for representation learning and model selection. The manually curated concept lexicon was not estimated from these representation-learning subsets. Instead, it was finalized as a fixed expert resource before model training. Test labels, model predictions, and evaluation results were not used in its construction or revision.

Based on the aligned corpus, we further apply unified cleaning and normalization procedures. First, we remove obvious non-body content and standardize whitespace, punctuation forms, and abnormal characters to reduce noise introduced by different digitization sources. Second, we filter anomalous fragments according to length and structural features. Third, we annotate core Confucian concepts using a predefined concept lexicon and normalize variants of person names and proper nouns. This processing pipeline preserves both the original expressions and their normalized forms in the final dataset, thereby supporting reproducible comparison and downstream analysis. The processed dataset and associated materials underlying the findings are publicly available in the Zenodo repository at https://doi.org/10.5281/zenodo.20660361.

Baselines

To comprehensively evaluate the proposed method, we construct a baseline suite from three perspectives: traditional text similarity methods, embedding-based semantic similarity methods, and internal representation-learning baselines with ablation variants. This design serves two purposes. First, it covers different modeling levels, ranging from surface lexical matching to contextual semantic representation. This allows us to assess the overall advantage of the proposed framework over existing similarity-based methods. Second, through internal comparisons and module ablations, it helps determine whether the performance gains mainly come from sentence-level supervised contrastive learning or from the concept-aware constraint and normalization design.

As shown in Table 3, the external baselines include traditional similarity methods and embedding-based semantic models. TF-IDF and SimHash serve as traditional baselines. The embedding-based baselines include Word2Vec [60], GloVe [61], BERT [62], Sentence-BERT [63], and SimCSE [64]. Word2Vec and GloVe represent static word embeddings, whereas BERT uses contextual representations with mean pooling. Sentence-BERT and SimCSE provide stronger sentence-level representations through contrastive pretraining. Both models are evaluated using publicly available pretrained checkpoints without task-specific fine-tuning. Word2Vec, GloVe, and BERT have also been used in previous studies of semantic comparison among English translations of the Analects, providing direct external reference points. For all embedding-based baselines, sentence representations are normalized and compared using cosine similarity under the same data splits and evaluation protocols.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Baseline methods used in the experiments.

https://doi.org/10.1371/journal.pone.0358227.t003

For internal comparisons, we further include representation-learning and ablation baselines. Encoder-only denotes the setting in which only the shared sentence encoder is used to generate sentence representations, and cosine similarity is computed directly without contrastive training. SupCon denotes the setting that uses only sentence-level supervised contrastive learning, without the concept-aware constraint. We also consider three ablation variants, namely without concept normalization (w/o concept normalization), without name normalization (w/o name normalization), and without the concept-aware loss (w/o concept-aware loss). These variants are used to evaluate the contributions of concept-term normalization, proper-name normalization, and the concept consistency loss, respectively. These internal baselines allow us to analyze the role of each component in cross-translation semantic consistency modeling and to verify the necessity of the full model design.

Experimental settings

The overall experimental setup is illustrated in Table 4. We construct the training, validation, and test sets on a unified chapter–sentence indexing scheme, where all translated sentences associated with the same chapter–sentence index are always assigned to the same data split in order to avoid information leakage. On this basis, we conduct two primary task-oriented evaluations, namely cross-translation retrieval and low-consistency detection, together with a supplementary score-label association analysis. In cross-translation retrieval, a sentence from one translation is used as the query, and the goal is to retrieve the sentence with the same chapter and sentence ID from the candidate set of the other translations. All translated sentences associated with the same chapter–sentence ID were assigned to the same split. This group-level partition prevents sentences corresponding to the same source saying from appearing in different representation-learning subsets. The representation-learning test split was excluded from model training and representation-level hyperparameter selection.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Experimental settings of the proposed model.

https://doi.org/10.1371/journal.pone.0358227.t004

Low-consistency detection was evaluated using 300 human-annotated translation pairs sampled exclusively from chapter–sentence groups in the held-out representation-learning test split. Consequently, neither the annotated pairs nor any sentence from their corresponding aligned groups was included in representation-learning training or validation. Two annotators with backgrounds in translation studies and classical Chinese texts independently labeled each pair as high consistency or low consistency. They were blind to all model outputs and similarity scores. Spearman’s and Kendall’s were used to measure the association between the continuous model scores and these binary labels. The annotators agreed on 275 pairs (91.67%), with Cohen’s , and disagreements were resolved through adjudication. The 300 annotated pairs were further divided into a human-validation subset and a human-test subset using a 67/233 group-disjoint partition. All pairs derived from the same chapter–sentence group were assigned to the same subset. The human-validation subset was used only to select the decision threshold by maximizing F1. The selected threshold was then fixed and applied to the human-test subset. Accuracy, F1, and AUC were calculated on the human-test subset.

At the implementation level, we adopt a representation learning model composed of a shared sentence encoder and a lightweight projection head. The backbone encoder is bert-base-uncased, and its output is mean-pooled to form sentence-level representations. The projection head is implemented as a two-layer feed-forward network that maps sentence representations into the contrastive learning space. The maximum input length is set to 128 in order to balance the sentence length of classical translations and training efficiency. Model training uses the AdamW optimizer, with an initial learning rate of , a weight decay coefficient of 0.01, and a batch size of 32. Both the supervised contrastive loss and the concept consistency loss use cosine similarity, with the temperature parameter set to 0.07 and the concept loss weight set to 0.2. Training is conducted for at most 10 epochs.

To ensure fair comparison among baselines, we use the same data splits and the same sentence-level similarity computation for all embedding-based and representation-learning methods. For internal comparisons and ablation experiments, all settings remain unchanged except for the removed module, including the backbone encoder, input length, optimizer, learning rate, and batch size. For external baselines such as TF-IDF, SimHash, Word2Vec, GloVe, and BERT, we follow either standard implementations or commonly used settings in related work as closely as possible, so as to ensure the interpretability of the comparison results. Sentence-BERT and SimCSE were evaluated using their publicly available pretrained checkpoints without additional fine-tuning.

For evaluation metrics, cross-translation retrieval is evaluated using Recall@1, Recall@5, and MRR, whereas low-consistency detection is evaluated using Accuracy, F1, and AUC. Spearman’s and Kendall’s are additionally reported as supplementary measures of association between continuous model scores and binary human labels. To reduce the effect of random initialization, all trainable neural models were run five times using the same data split and matched random seeds. We additionally conducted group-level paired permutation tests over the held-out evaluation instances. For retrieval, all queries derived from the same chapter–sentence group were treated as one cluster. For the human-based evaluation, all annotated pairs associated with the same chapter–sentence group in the human-test subset were kept together. For descriptive run-level reporting, we present the mean, standard deviation, and 95% confidence interval across five matched runs. Inferential comparisons between the Full model and SupCon were conducted using 10,000 two-sided group-level paired permutations over the held-out evaluation instances. Holm correction was applied across the three representative metrics. Unless otherwise stated, all experiments are conducted on a single NVIDIA GPU, and the models are implemented with PyTorch and HuggingFace Transformers.

Ablation study

To analyze the contribution of each component to the final performance, we further conduct an ablation study on the sentence-level dataset, as shown in Fig 5. Overall, the Full model achieves the best results on all eight evaluation metrics. This result indicates that concept normalization, name normalization, and the concept-aware loss all contribute positively to semantic consistency modeling across translations. Specifically, the Full model attains 0.936 on R@1, 0.952 on MRR, 0.807 on F1, 0.867 on AUC, 0.614 on Spearman’s , and 0.532 on Kendall’s .

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Ablation Study Across Eight Evaluation Metrics on the Translation Dataset.

https://doi.org/10.1371/journal.pone.0358227.g005

Comparing the two normalization modules, removing name normalization causes a relatively small performance decrease. For example, R@1 decreases from 0.936 to 0.929, F1 from 0.807 to 0.789, and Spearman’s from 0.614 to 0.566. Removing concept normalization leads to a larger decline. R@1 drops from 0.936 to 0.902, Accuracy from 0.840 to 0.795, F1 from 0.807 to 0.771, AUC from 0.867 to 0.839, and Spearman’s from 0.614 to 0.532. These results confirm the contribution of the explicit concept prior. However, the model retains substantial performance without this module. Thus, the overall gains arise from both representation learning and concept normalization rather than from the encoder alone.

Removing the concept-aware loss causes the largest overall performance decline. Compared with the Full model, this variant reduces R@1 from 0.936 to 0.896, MRR from 0.952 to 0.919, Accuracy from 0.840 to 0.813, F1 from 0.807 to 0.758, AUC from 0.867 to 0.829, Spearman’s from 0.614 to 0.527, and Kendall’s from 0.532 to 0.478. These declines indicate that concept-aware supervision improves retrieval and low-consistency detection and strengthens the association between model scores and the binary human labels. The ablation without concept normalization further shows that the manually curated lexicon contributes to performance, while the model retains substantial effectiveness without this resource. The reported results therefore reflect contributions from both representation learning and the explicit expert prior, rather than reliance on the lexicon alone.

Hyperparameter sensitivity analysis

To evaluate the sensitivity of the proposed method to key hyperparameters, we further examine the effects of the concept-aware loss weight and the temperature parameter on model performance. We select R@1 and F1 as the main evaluation metrics because they are representative of cross-translation retrieval and low-consistency detection, respectively. This analysis verifies the reasonableness of the default hyperparameter settings. All default representation-level hyperparameters were fixed before evaluation on the human-test subset. The analysis in Fig 6 was conducted post hoc to assess sensitivity and was not used for model selection.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Sensitivity of the Full Model to the Hyperparameters and .

https://doi.org/10.1371/journal.pone.0358227.g006

For the concept-aware loss weight , performance first improves and then declines as increases. When , the model is equivalent to the w/o concept-aware loss variant. As increases from 0 to 0.2, R@1 improves from 0.896 to 0.936, while F1 increases from 0.758 to 0.807. This result shows that moderate concept-level supervision benefits both retrieval and low-consistency detection. However, larger values of reduce both metrics. An overly strong concept-aware constraint may weaken sentence-level alignment and impair overall performance.

The temperature parameter shows a similar pattern. As increases from 0.03 to 0.07, R@1 rises from 0.917 to 0.936, and F1 reaches its highest value of 0.807. Further increases in gradually reduce both metrics. A large temperature produces a smoother similarity distribution and weakens the distinction between positive and negative samples. The prespecified settings and lie near the observed performance peak in this post hoc sensitivity analysis.

Case study

To further understand how the proposed framework behaves on concrete translation instances, we conduct a qualitative case study on several representative sentence groups from the translation dataset. As shown in Table 8, the selected examples cover three typical situations: semantically consistent renderings with substantial lexical variation, concept-centered translation divergence, and boundary cases involving interpretive paraphrase. These cases complement the aggregate evaluation results and allow us to examine how concept-aware semantic consistency modeling operates at the level of individual sayings.

Download:

• PNG
  larger image
• TIFF
  original image

Table 8. Representative case studies for cross-translation semantic consistency analysis.

https://doi.org/10.1371/journal.pone.0358227.t008

Cases C1 and C2 illustrate situations in which the translations remain highly consistent in meaning despite substantial surface variation. In C1, the shared concept of learning is expressed through different lexical realizations, such as learn, do, practise, and acquire knowledge. In C2, the idea of companions arriving from afar is rendered as friends, associates, or men of kindred spirit. These examples show that lexical overlap alone is not sufficient for reliable comparison. Semantically equivalent translations may differ greatly in wording. The proposed framework handles such cases more effectively because concept normalization reduces lexical mismatch, while concept-aware supervision keeps semantically aligned renderings close in the representation space.

Case C3 and the boundary case C4 further show why concept-level modeling is necessary, while also revealing its current limitations. In C3, the source concept junzi is translated as a man of complete virtue, a gentleman, men of the superior order, and a true philosopher. These renderings reflect different interpretive choices rather than simple lexical variation. This case highlights the importance of explicit concept-aware learning, because surface-form similarity alone cannot reliably capture the semantic continuity among such expressions. At the same time, C4 shows that highly interpretive translations remain challenging even for the Full model. When a translation shifts from conceptual transfer to broader explanatory paraphrase, the boundary between semantic equivalence and semantic reinterpretation becomes less clear. This observation suggests that, although concept-aware supervision substantially improves cross-translation comparison, modeling deeper interpretive divergence in classical texts remains an important direction for future work.

Threats to validity

Internal validity. The internal validity of this study is mainly influenced by the data processing pipeline and the experimental setup. First, all English translations are obtained from publicly accessible digital resources. The reconstruction of the Jennings and Soothill texts involves OCR cleaning, main-text extraction, and structural recovery. These steps may introduce a limited amount of noise or alignment error. Second, sentence-level alignment mainly relies on chapter/sentence IDs and rule-based recovery. Although missing entries and anomalous fragments are handled in a unified manner, the effect of local mismatches cannot be fully excluded. In addition, the reported results may still be affected by parameter settings, random initialization, and implementation differences. The ablation study measures the overall contribution of the concept prior, but it does not establish whether the encoder can learn the same conceptual abstraction independently from original, non-normalized inputs. Low-consistency detection is evaluated on a held-out human-annotated subset rather than solely on threshold-based similarity labels. This reduces the risk of circular evaluation, although the annotation of semantic consistency may still involve expert judgment. The concept lexicon constitutes another potential source of corpus dependence. Although it was manually curated, fixed before model training, and contained no human consistency labels or model-derived information, its lexical variants were collected from the same five translations evaluated in this study. Consequently, some expressions occurring in the held-out splits were already represented in the normalization resource. This setting does not involve label leakage because no human consistency labels, model predictions, or evaluation results were used. Nevertheless, the normalization resource has access to lexical forms occurring in the held-out data and therefore represents a closed-corpus evaluation setting rather than a strict unseen-corpus setting. The present findings should therefore be interpreted as evaluating a manually curated closed-corpus expert lexicon.

External validity. As a Confucian classic, the Analects is characterized by concise expression, dense conceptual content, and substantial interpretive openness. These properties make concept-aware semantic consistency modeling particularly suitable for the present task. However, it remains unclear whether the same advantages can be directly generalized to other types of classical texts, modern texts, or corpora with lower concept density. Moreover, the current study focuses only on the comparison of multiple English translations of a Chinese classical text. Its applicability to other language pairs, texts from other cultural traditions, and datasets with different translation formats has not yet been fully validated. The five translations were published between 1861 and 1910 and may not represent modern translation strategies, particularly the retention of romanized concepts such as ren, li, and junzi. Accordingly, under the current evidence, the framework should be understood as a computational aid evaluated within this specific cross-translation setting, rather than as a validated general-purpose method for translation-quality assessment. Future work will compare the present closed-corpus lexicon with externally defined and training-only lexicons to evaluate lexical generalization under stricter unseen-corpus settings.

Construct validity. This study operationalizes cross-translation comparison through two primary evaluation tasks, namely retrieval and low-consistency detection. R@1, R@5, and MRR measure retrieval performance, while Accuracy, F1, and AUC measure binary detection performance. Spearman’s and Kendall’s are reported only as supplementary measures of association between continuous model scores and binary human labels. Because the human annotations contain two categories rather than graded consistency levels, these coefficients should not be interpreted as agreement with a fine-grained semantic ranking. More detailed human annotations would be required to evaluate whether the model captures multiple degrees of semantic consistency.