2.1. The South China Sea in the digital age

The digital humanities field represents a fundamental interdisciplinary convergence, uniting traditional humanistic inquiry including history, literature, and cultural studies with rigorous computational methodologies [15,16]. This computational shift enables scholars to leverage data analysis techniques to interpret humanistic data, revealing latent patterns, themes, and connections within large, previously intractable datasets [17]. Central to this transformation is the Knowledge Graph (KG), a crucial data structure that integrates information from disparate sources, transforming current resources into representations interpretable by both machines and humans [8]. While general-purpose KGs such as Wikidata and DBpedia provide broad coverage, they often lack the necessary semantic depth and domain specificity required for complex applications, leading to fragmentation and insufficient representation in specialized research areas [9,18]. Consequently, a growing research trend focuses on leveraging machine learning (ML) and natural language processing (NLP) to construct domain-specific knowledge graphs across fields such as law, cultural heritage, and political science, enabling the accurate modeling of highly specialized knowledge [17,19]. Recent advancements in AI-assisted knowledge representation, semantic modeling, and structured information extraction have further strengthened this trend, particularly through hybrid machine learning and deep learning frameworks combined with explainable AI techniques. These developments demonstrate the potential of automated yet interpretable methods for handling complex, heterogeneous data in high-stakes domains [20–22].

The South China Sea (SCS) region is a critical academic and geopolitical subject, characterized by complex territorial disputes, rich natural resources, and immense strategic significance [23]. Research on the SCS is inherently multifaceted, spanning international relations, law, security, history, and culture [24]. The necessity of understanding the interconnections between these diverse disciplinary facets has driven a growing emphasis on interdisciplinary approaches [25]. Although historical and social science perspectives traditionally dominate this research, there is an increasing adoption of computational methods to achieve the digitization of research in this area [26].

Recent computational efforts focused on the SCS have primarily aimed to structure and analyze complex data. For instance, Wang et al. [4] focused on constructing a knowledge map of documentary evidence for rights protection by transforming raw data into a structured layer of evidentiary entries. Other studies have applied advanced ML to classification tasks, such as Peng et al. [27], who proposed a BERT-based system for fine-grained, multi-label classification of evidentiary data to reveal latent relationships. Furthermore, specialized NLP has been crucial in dealing with resource-scarce historical texts, as seen in the work of Wei et al. [26], who developed an ALBERT-based model to enhance toponymic entity recognition in the Genglubu text. Similarly, Peng and Li [3] have employed social network analysis (SNA) to quantify the temporal dynamics of geopolitical interactions, offering a data-driven framework for regional relations.

While these pioneering studies have significantly advanced the innovation and modernization of SCS knowledge representation within the context of artificial intelligence, several critical gaps remain. Compared to digital resources available in the natural sciences, research on the creation of integrated digital infrastructure in the humanities and social sciences within this region remains notably limited [28]. Furthermore, most existing computational studies employ either specialized, semi-automated methods that require substantial manual effort or focus on solving isolated NLP tasks, which collectively reduces overall efficiency and scope. Crucially, the fundamental challenge of multiplicity and heterogeneity across diverse data sources including historical records, legal documents, and cultural artifacts continues to impede comprehensive analysis via current sequential NLP methodologies. Addressing such heterogeneity requires pipeline architectures that incorporate principled normalization, feature verification, and interpretability mechanisms—design considerations that remain active challenges across AI-driven structured information extraction tasks [22,29,30]. This paper addresses these limitations by developing the SCS-KG, designed to provide a cohesive, structured, and interoperable resource that enables multi-disciplinary discovery and analytical reasoning across fragmented historical, cultural, and geopolitical narratives.

2.2. LLM-based knowledge graph construction

Knowledge Graphs (KGs) are fundamental for organizing and representing information in a structured, machine-interpretable format. Traditional methods for KG construction including manual curation, rule-based systems, and classical machine learning (ML)/natural language processing (NLP) approaches, have proven effective but face inherent limitations [12,13]. Specifically, these methods often struggle with contextual reasoning and exhibit poor adaptability to novel or complex information due to their reliance on rigid, predefined rules and categories [11]. This structural rigidity hinders their performance when scaling to diverse, high-volume, and heterogeneous datasets that characterize modern domains [14].

Recent advancements in Large Language Models (LLMs) offer a promising solution to these challenges, supporting more flexible approaches to knowledge extraction, representation, and reasoning. LLMs facilitate various aspects of KG construction by leveraging their superior ability to process and generate natural language, enabling robust zero-shot extraction of entities, relationships, and attributes [14]. Beyond simple information extraction, LLMs show strong reasoning performance through in-context learning (ICL), reducing the need for labeled datasets. Methods for integrating LLMs range from zero-shot prompting [31], where the task is reframed as a multi-turn Q&A (e.g., ChatIE), to more structured approaches like KICGPT [32], which integrates LLMs with traditional triple-based retrievers for knowledge graph completion. While training-free methods such as ICL and prompt engineering are efficient, other studies emphasize the value of fine-tuning to enhance structural awareness, arguing that structural embeddings contain richer semantic information than text-based prompts [33]. The reliability of automated knowledge extraction pipelines is further contingent on the integration of interpretability mechanisms and multi-stage verification strategies, which improve the semantic validity of extracted content and reduce noise introduced during automated processing [21,29].

Despite these rapid advancements, the core limitations of LLM outputs remain a critical challenge, notably regarding accuracy and reliability due to hallucinations and contextual misunderstandings [34,35]. This challenge has motivated the development of robust hybrid frameworks that integrate LLMs with external knowledge and verification systems to bolster credibility. Approaches like KGGen [5] and SAC-KG [11] utilize multi-stage methodologies involving generator, verifier, and pruner components to mitigate graph noise, incompleteness, and knowledge hallucination. In these frameworks, the LLM is treated as a domain expert operating under a structured governance layer. The importance of such governance layers is further underscored by recent findings showing that pipeline designs incorporating structured normalization, feature pruning, and explainability stages consistently yield more robust and trustworthy outputs in automated knowledge extraction tasks [20,22].

Despite these documented advancements, a significant domain-specific gap remains: to the best of our knowledge, no prior work has constructed a comprehensive KG focused on the complex, heterogeneous South China Sea domain, nor applied advanced LLM-integrated techniques to knowledge extraction and validation in this critical geopolitical and historical context. The present framework addresses this gap while differing from existing approaches in three respects that are particularly salient for this domain. First, it targets open-ended entity-relation extraction from heterogeneous historical, legal, geopolitical, and cultural texts rather than classification or detection tasks. Second, it combines LLM-based extraction with embedding-based normalization and LLM-assisted triple verification to address semantic redundancy and factual noise within a single integrated pipeline. Third, it evaluates the resulting KG not only through extraction metrics but also through domain-specific reasoning scenarios, positioning it as a domain-adapted KG construction workflow rather than a general explainable AI classifier.

3.1. Entity and relation extraction

The extraction of entities and relations uses LLMs to identify and extract entities along with their corresponding relations from natural language and directly outputs them. Given LLMs’ proficiency in language comprehension, the Generative Relation Extraction method is adopted [6]. However, lengthy documents often pose significant challenges due to LLMs’ limited context length. To address this, the original document is first split into smaller, more manageable chunks to fit within the LLM context window and at the same time preserve the semantic integrity of the text [34]. Each chunk is then analyzed by the LLM to generate a structured representation of the information contained within it, which includes the identification of key entities and their relations as well as the provision of their supporting sentences and explanations. The LLM is also prompted to label an entity type to the extracted entity from a predefined set of entity categories. The predefined entity categories are based on previous studies on named entity recognition (NER) with modifications to better fit current data [10,32]. By indicating what specific category the extracted entity belongs to, we can easily find out whether there are misclassifications of the entities by the LLM [32]. We impose strong constraints on the output format of the LLM via the Langchain OutputParser to ensure the output format (in JSON) of LLM. The prompt for the entity and relation extraction and the expected output format are illustrated in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Entity and relation extraction prompt and output format.

https://doi.org/10.1371/journal.pone.0351132.t001

3.2. Entity and relation normalization

After extracting the entities and relations from the source texts, we may have duplicate or synonymous entities and redundant relations. The normalization process aims to merge the extracted entities and relations which have the same underlying meaning, e.g., “海南 (Hainan)” might be clustered with “海南岛 (Hainan Island)” and “贸易往来 (trade exchanges)” with “贸易联系 (trade connections)”, “贸易交流 (trade communications)”, “商贸往来 (commercial exchanges)”. This reduces the likelihood of semantically similar duplicate entities or relationships and prevents the formation of sparse KGs. The normalization process involves several steps, including embedding, clustering, and mapping. It helps to create a more accurate and efficient knowledge graph by reducing redundancy and standardizing entities and relations.

In our approach, the nodes and edges are first vectorized into numerical vectors using the text-embedding-3-large model. Once the embeddings are obtained, similar entities and relations are grouped via hierarchical clustering using the average linkage method, with cosine similarity as the similarity measure (see Equation 1 below):

(1)

where and denote two vectors, and the i-th components, and n is the vector dimensionality. A cosine similarity value closer to 1 indicates a higher degree of similarity.

After clustering, a representative element for each cluster is determined. This representative is identified as the item exhibiting the highest similarity to the cluster’s weighted average embedding (weighted by the norm of each embedding vector), and serves as the canonical form for standardizing all cluster members.

3.3. Triple verification

Although normalization process effectively reduces redundancy and duplicate triples generated by the LLM, errors in the outputted triples may still persist. To further improve the quality of LLM-generated outputs, we introduce a Triple Verification step designed to identify and eliminate erroneous triples. All extracted triples are evaluated using the LLM (DeepSeek) which is provided with a structured prompt to assess the correctness of each verified triple. The LLM assesses each triple based on its accompanying supporting sentences and explanations, which offer contextual information to facilitate accurate evaluation. The LLM assigns a confidence score to each triple from a score range of 1–5, where 5 indicates strong confidence in the triple’s correctness and 1 denotes a high likelihood of error. This scoring system enables a more nuanced evaluation of triple correctness and facilitates differentiation between similar triples. Triples receiving confidence scores of 1 or 2 were excluded from the final SCS-KG to ensure high reliability. The defined score range reduces ambiguity in the LLM’s responses, encourages more definite assessments of the triples, and quantifies uncertainty regarding each triple. Each confidence score is supplemented with a rationale that explains the evaluation outcome, further enhancing the LLM’s reasoning ability and mitigating hallucination issues during the assessment process. The prompt for the entity and relation verification and the expected output format are illustrated in Table 2.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Prompt for triple verification and output format.

https://doi.org/10.1371/journal.pone.0351132.t002

4.1. Datasets and experiment setup

We compiled a corpus of raw textual data from 20 published Chinese books, 63 selected research articles sourced from the China National Knowledge Infrastructure (CNKI)-China’s largest comprehensive academic database-drawn from Peking University Core and Chinese Social Sciences Citation Index (CSSCI) journals, and official publications available from institutional websites (e.g., the Ministry of Foreign Affairs of the People’s Republic of China, the National Institute for South China Sea Studies, the Chinese Academy of Sciences, and the South China Sea Institute of Oceanology). All sources are publicly accessible through academic databases (e.g., CNKI), institutional websites, published books, or peer-reviewed journals, subject to standard access terms (e.g., institutional subscriptions for CNKI or library access for books). The collection and analysis methods complied fully with the terms and conditions of the respective sources, including CNKI’s usage policies for academic research purposes and the public-domain or fair-use provisions applicable to official government/institutional documents. No personal data or restricted content was collected or processed. The underlying raw texts include third-party copyrighted materials and content accessed under license (e.g., CNKI articles and published books) and therefore cannot be redistributed by the authors due to copyright and licensing restrictions. A complete list of sources is provided in the S1 Appendix to enable other researchers to independently obtain equivalent data. The corpus consists exclusively of Chinese-language materials, drawing primarily from official publications and academic sources affiliated with Chinese institutions. Accordingly, the resulting knowledge graph reflects the perspectives, terminology, and evidentiary emphases present within the selected corpus rather than a fully neutral representation of all international viewpoints concerning the South China Sea. After excluding irrelevant content such as references, appendices, copyright pages, and journal information, we constructed a corpus comprising 4,759,368 tokens. To handle this text, we use the RecursiveCharacterTextSplitter from Langchain configured with a chunk size of 500 tokens and 100-token overlap between chunks. These two values were chosen because preliminary experiments show that this combination preserves most sentence integrity while maximizing compatibility with the model’s context window length.

DeepSeek-V3 (https://platform.deepseek.com/) was chosen as the backbone model due to its outstanding proficiency in Chinese language processing and its effectiveness in knowledge-intensive tasks. As an open-source mixture-of-experts (MoE) model featuring 37 billion activated parameters, DeepSeek-V3 achieves superior performance on Chinese factual knowledge benchmarks, such as Chinese SimpleQA and C-Eval [36]. Owing to its robust reasoning capabilities and extensive training across diverse knowledge domains, DeepSeek-V3 is well-suited to effectively process and integrate the complex information necessary for knowledge graph construction. To ensure high accuracy and response consistency, the temperature of the LLM was set to 0.1.

For knowledge normalization, hierarchical clustering was performed using the average linkage method, with a similarity threshold of 0.85 applied for both cluster formation and subsequent merging of entities and relations. Entities and relations were grouped and consolidated if their cosine similarity with representative cluster element exceeded this threshold. The threshold of 0.85 was selected after inspecting clustering outcomes across a range of candidate values (0.80–0.95). Lower thresholds led to excessive merging of semantically distinct concepts (reduced cluster purity and increased false positives), while higher thresholds missed many legitimate synonym variants and variant expressions (lower recall and insufficient redundancy reduction). The value 0.85 achieved the optimal trade-off between precision and recall for both entities and relations.

To quantitatively assess extraction quality, we constructed a test set by randomly sampling 200 text segments from the corpus. Given the scale of manual annotation required for comprehensive entity and relation labeling, we employed a frontier LLM (GPT-5) as an independent annotator to produce initial gold-standard labels. GPT-5 is architecturally and operationally distinct from DeepSeek-V3 used in our extraction pipeline, thereby mitigating concerns of circular evaluation. To validate the reliability of this LLM-based annotation, we randomly selected 80 annotated samples for human review. The human review was conducted by the first author and a research assistant familiar with knowledge graph construction and South China Sea studies.

Both reviewers independently examined whether extracted entities, relations, and supporting evidence were consistent with the source text and semantically appropriate in context. Annotation quality was assessed based on the correctness of entity boundaries, entity types, relation labels, and factual consistency between triples and the corresponding source passages. Disagreements were subsequently resolved through discussion and consensus-based adjudication. The sampled human review yielded an entity-level accuracy of 89.7% and a triple-level accuracy of 85.4%. Discrepancies identified during human review were corrected to form the final gold standard containing 4811 entities and 4179 relation triples. We report Precision, Recall, and F1 at both the entity and triple levels, using relaxed matching (partial string overlap) to account for boundary ambiguity in Chinese text.

4.2. Main results

In this section, we present the results of entity and relation extraction, normalization of entities and relations, and triple verification.

4.2.1. Results of entity and relation extraction.

The entity and relation extraction process generated 78,542 nodes and 891,195 edges. As shown in Fig 2, location (LOC) entities dominate (60%), followed by organization (ORG) (12%) and person (PER) (9%). Other types such as quantity (QTY), business (BUS), technology (TEC), and time (TIM) each account for about 1%. A total of 894 distinct relation types were identified, reflecting the dataset’s diverse semantic structure. The most frequent relations include Geographical Location (26%) and Affiliation (19%), underscoring the centrality of spatial and administrative connections consistent with the predominance of LOC entities. Other major relations including Resource Exploitation (11%), Cultural Dissemination (3%), Trade Activities (2%), Alias (2%), War (2%), Writing (2%), Establishment (2%), and Implementation (1%) collectively indicate the multifaceted nature of the South China Sea domain, encompassing geopolitical, cultural, and economic dimensions.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Entity and relation category distributions.

(a) Entity type proportions showing the dominance of LOC entities (60%), followed by ORG (12%) and PER (9%); (b) Top relation types highlighting spatial (Geographical Location, 26%) and administrative (Affiliation, 19%) connections.

https://doi.org/10.1371/journal.pone.0351132.g002

Representative entities (Table 3) illustrate the breadth of extracted knowledge. LOC entities include Chinese provinces (e.g., Hainan, Guangdong, Fujian), neighboring states (Vietnam, Philippines, Malaysia, Indonesia), and other countries with strategic interests (United States, Japan, United Kingdom, France). Maritime features such as the Nansha, Xisha, Zhongsha, and Dongsha Islands serve as focal points of sovereignty disputes. The ORG category covers governmental bodies (e.g., Qing government), international and regional organizations (UN, ASEAN), military forces, and media institutions. EVENT entities span historical events (The Opium War, 1974 Xisha Sea Battle), territorial disputes (Huangyan Island dispute), and international agreements (Declaration on the Conduct of Parties in the South China Sea). The PER category includes historical figures (Zheng He, Emperor Wu of Han), ethnic groups, and collectives (Chinese fishermen, Huaqiao). CULTURE entities encompass maritime relics, navigational texts, temples, religions, and art forms, while TIME, TECHNOLOGY, BUSINESS, and QUANTITY entities capture temporal markers, technological advancements, economic activities, and quantitative data relevant to the region.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Entity categories and typical tokens.

https://doi.org/10.1371/journal.pone.0351132.t003

Analysis of frequently co-occurring entity pairs (Fig 3) reveals that LOC-LOC pairs are the most common, reflecting strong spatial linkages. Other high-frequency combinations PER-CUL, ORG-ORG, ORG-EVT, and PER-PER highlight interactions among individuals, organizations, and cultural or event contexts. Examples include Fu Hongguang-depicted-Map of Xisha and Nansha Islands (PER-CUL) and ASEAN 10 + 3-similar cooperation model-APEC (ORG–ORG). Pairs such as ORG-LOC and PER-ORG reflect spatial or affiliative ties, while CUL-CUL and PER-LOC indicate cultural and locational associations. Overall, these patterns emphasize the prominence of geographical, organizational, interpersonal, and cultural relationships within the South China Sea knowledge structure.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Pattern of co-occurring entity category pairs.

LOC–LOC pairs are most common (strong spatial linkages); other frequent combinations (PER–CUL, ORG–ORG, ORG–EVT) highlight interpersonal, organizational, and cultural interactions.

https://doi.org/10.1371/journal.pone.0351132.g003

4.2.2. Results of entity and relation normalization.

After the normalization process, a total of 765,043 triples were generated. To ensure semantic consistency, both entities and relations underwent mapping based on similarity scores calculated by the LLM. The normalization stage effectively grouped linguistically or semantically related expressions, thereby reducing data redundancy while retaining meaningful distinctions among concepts (see Fig 4).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Examples of entity and relation mappings.

(a) Entity cluster centred on Hang Haijia groups near-synonyms with high similarities (0.8551-0.8927); (b) Relation cluster centred on Jingmao Wanglai groups lexical variants with similarities 0.8736-0.9143.

https://doi.org/10.1371/journal.pone.0351132.g004

For entity mapping, the results indicate effective clustering of near-synonymous expressions (Fig 4a). For instance, the representative entity “南海周边国家” (countries surrounding the South China Sea) was semantically aligned with variants such as “南海周边有关国家” (relevant countries around the South China Sea, similarity 0.9338), “南海区域周边国家” (countries in the region surrounding the South China Sea, 0.8834), and “南海周边各国” (various countries around the South China Sea, 0.9332). Similarly, “南中国海问题” (the South China Sea issue) corresponded closely to “南中国海争端” (South China Sea dispute, 0.8676) and “中国南海问题” (China’s South China Sea issue, 0.8605), reflecting consistent clustering of conceptually similar entities. Another representative case is “200海里专属经济区” (200-nautical-mile exclusive economic zone), which mapped to several near-identical expressions, such as “200海里专属经济区域” (0.9816), “200海里经济专属区” (0.9256), and “200海里经济水域” (0.8805), illustrating the model’s precision in normalizing technical terms. Furthermore, named entities such as “乾隆皇帝” (Emperor Qianlong) were correctly linked to “乾隆帝” (0.9287) and “乾隆” (0.8937), indicating the model’s capacity to handle entity abbreviation and historical name variation.

In relation mapping, the normalization process also yielded coherent clusters of semantically similar expressions (Fig 4b). For example, the representative relation “被称为” (is called) was mapped to “被称作” (is called as, 0.9600), “被尊称为” (is honored as, 0.8512), and “被称之为” (is known as, 0.9596), reflecting high lexical and semantic overlap. Likewise, “宣称主权” (claim sovereignty) corresponded to “宣布主权” (declare sovereignty, 0.8985), “声称主权” (assert sovereignty, 0.8998), and “宣示主权” (proclaim sovereignty, 0.9236), accurately capturing lexical variations in expressing sovereignty claims. Other examples include “建立贸易关系” (establish trade relations), which aligned with “建立通商贸易关系” (0.9391) and “通商关系建立” (0.8625), as well as “和平解决纷争” (0.9498) and “和平解决争议” (0.9542) mapped to “和平解决争端” (peacefully resolve disputes).

Overall, the mapping results indicate that the normalization process effectively reduced lexical diversity without sacrificing semantic richness. High similarity scores (mostly above 0.85) suggest that the LLM successfully identified synonymous or near-synonymous expressions across both entities and relations. This normalization not only minimizes redundancy but also enhances the coherence and connectivity of the resulting knowledge graph, ensuring that conceptually equivalent terms and relationships are uniformly represented across the dataset.

4.2.3. Results of triple verification.

The verification procedure aimed to identify and reduce potential inaccuracies within the triples generated by the LLM. All triples from the normalization stage were re-evaluated for accuracy and consistency. Among the assessed triples, 505,896 (66.13%) received a confidence score of “5”, 92,760 (12.12%) scored “4”, 53,362 (6.98%) scored “3”, 67,341 (8.80%) scored “2”, and 45,684 (5.97%) scored “1”. Overall, more than 78% of the triples achieved high confidence levels (scores ≥ 4), indicating the robustness of the verification mechanism in enhancing data reliability (Fig 5a).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Triple verification results.

(a) Confidence score distribution shows >78% of triples scored 4–5(high reliability); (b) Low-confidence triples (scores 1–2) primarily stem from incorrect relation classification (40.8%) and factual hallucinations (34.4%).

https://doi.org/10.1371/journal.pone.0351132.g005

High-scoring triples were supported by explicit textual evidence, whereas medium- and low-scoring triples often reflected weaker contextual associations or implicit relationships. Further examination of low-confidence triples (scores 1–2) revealed four major error sources: incorrect relation classification (40.8%), factual errors or hallucinations (34.4%), overgeneralization (21.1%), and entity boundary recognition errors (3.7%) (Fig 5b).

Relationship misjudgment commonly occurred when the model incorrectly classified semantic links between entities, for example, interpreting “United Kingdom-Nansha Islands-geographical location” as a locational rather than exploratory relation. Factual errors often stemmed from fabricated or inconsistent details, such as the misrepresentation of “Eastern Xisha Islands” instead of “Xisha Islands”. Overgeneralization errors involved extrapolating conclusions beyond textual scope, as in extending the inapplicability of one legal framework to all similar systems. Finally, boundary errors arose when entity scopes were either too narrow or too broad, as seen in “South China Sea–geographical location–Subic of Philippines”, which improperly localized a regional reference.

Overall, the verification results suggest that the LLM-based validation framework effectively distinguishes reliable from erroneous triples, reduces semantic ambiguities, and enhances the overall accuracy and consistency of the knowledge graph.

4.2.4. Quantitative evaluation.

To complement the structural and descriptive analyses presented above, we conducted a quantitative evaluation using the LLM(GPT-5)-annotated test set described in Section 4.1. Table 4 presents the extraction performance of our proposed method alongside two baseline approaches.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Comparison of entity and relation extraction performance across different methods.

https://doi.org/10.1371/journal.pone.0351132.t004

As shown in Table 4, our DeepSeek-V3 based pipeline achieves competitive performance, with an entity-level F1 of 88.35% and a triple-level F1 of 84.72%. The traditional NLP pipeline (SpaCy+OpenIE) achieves noticeably lower scores, particularly on relation extraction (Triple F1 = 55.07%), reflecting the limitations of rule-based and statistical methods in handling domain-specific semantic complexity and long-range contextual dependencies in South China Sea texts. GPT-4o shows slightly higher entity-level precision but lower recall compared to DeepSeek-V3, suggesting a more conservative extraction strategy. In contrast, DeepSeek-V3 demonstrates stronger coverage in identifying heterogeneous entities and implicit semantic relations across historical, legal, and geopolitical materials, achieving the highest triple-level F1 (84.72%) among all evaluated methods.

4.3. Stage-wise evaluation

To further evaluate the contribution of each module, stage-wise experiments were performed on the entire framework. Table 5 presents core metrics used to assess the quality and structure of the knowledge graphs at three stages: Initial Extraction, Normalization, and Verification. These include: (1) basic metrics that quantify the graph’s scale and sparsity; (2) connectivity metrics that evaluate the extent of interconnection and component structure; and (3) structural metrics that analyze local and global topological properties. Basic metrics capture the graph’s size and connectivity density (see Table 5). The number of nodes and edges reflect the total counts of entities and relationships, respectively, while density characterizes sparsity according to the following formula:

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Knowledge graph quality metrics by stage.

https://doi.org/10.1371/journal.pone.0351132.t005

(2)

where E denotes the number of edges, and N denotes the number of nodes. Nodes and edges decreased from 78,542 and 891,195 (extraction stage) to 59,836 and 652,018 (after verification). At the same time, graph density rose from 0.000145 to 0.000183 (+26.2%). This reflects removal of redundant elements and greater compactness. Connectivity metrics evaluate how nodes form connected subgraphs. Key indicators are the number of connected components (disjoint subgraphs) and the Largest Connected Component (LCC) ratio (proportion of nodes in the largest subgraph). The LCC ratio is calculated as:

(3)

The initial extraction generated 1,523 components with a LCC ratio of 76.32%. Normalization merged small components, yielding 1,214 components and raising the LCC ratio to 82.54%. Verification reduced components to 982 and raised the LCC ratio to 87.46%, consolidating weak structures into the main component. Structural metrics assess both local clustering and global degree distribution within the graph. The average clustering coefficient quantifies the extent to which a node’s neighbors are interconnected, and is defined for node i as , where is the number of edges among neighbors and is the degree of node i. It increased from 0.135 (extraction) to 0.192 (verification), showing stronger local relationships. The average shortest path length reflects the efficiency of connectivity among nodes. It decreased from 4.28 to 3.32, indicating better network efficiency. This reduction may result from the elimination of redundant edges or from the normalization process, which prioritizes efficient global connectivity over the formation of highly clustered local neighborhoods.

Degree centrality was also calculated to identify key entities across the three stages. Degree centrality quantifies the number of direct connections a node has, with higher values indicating greater centrality within the network. The degree centrality (C_D) of node i in a graph with n nodes is given by:

(4)

where represents the number of edges incident to node i. High degree centrality marks core concepts in the graph. Examination of the top 30 entities revealed that mapping successfully occurred during the normalization process. For example, entities such as “中国南海” (South China Sea), “南中国海” (South China Sea), and “中国南海海域” (South China Sea waters) were all mapped to “南海” (South China Sea), and “海南岛” (Hainan Island) was mapped to “海南” (Hainan). The node “南海” (South China Sea), which exhibited the highest degree centrality (normalized value = 1), was selected as the central node for constructing a local knowledge graph of the South China Sea corpus.

The subgraph was generated using the top 200 entities most strongly associated with “南海”. In the initial extraction stage, the subgraph exhibited a density of 0.156, indicating a moderate level of connectivity among these entities. An average clustering coefficient of 0.56 indicated the presence of localized community structures, with nodes tending to form cohesive groups. The largest connected component ratio (1.0000) confirmed full connectivity within the subgraph, while an average shortest path length of 1.9015 reflected a “small-world” property, whereby most nodes were accessible within a few steps via “南海”(South China Sea) as the central hub.

Normalization improved density to 0.2031 (+30.2% from initial), raised clustering coefficient to 0.67, and slightly shortened average path length to 1.8970. Verification further adjusted density to 0.2001, increased clustering to 0.73, and reduced average path length to 1.61 (–15.3% from initial). This pronounced reduction reflects the retention of high-weight, semantically meaningful edges and the emergence of shortcut connections between distant communities, thereby facilitating more efficient knowledge inference across domains.

To further quantify the individual contribution of each module, we conducted an ablation study by separately removing the normalization and verification stages while keeping all other settings unchanged. In addition to Entity F1 and Triple F1, we also introduce Cluster Purity as a semantic consistency metric. Cluster Purity measures the extent to which semantically related entities are grouped into coherent clusters after graph construction. Higher values indicate lower semantic redundancy and stronger conceptual consistency among connected entities.

Table 6 reports the results of the ablation study examining the contribution of each pipeline module. Removing the normalization module decreases Entity F1 from 88.35% to 84.10% and Triple F1 from 84.72% to 78.65%, while also lowering Cluster Purity from 0.92 to 0.79, Density from 0.000183 to 0.000145, and the LCC Ratio from 87.46% to 76.32%. These results indicate that normalization plays a critical role in merging synonymous entities, reducing semantic redundancy, and improving graph coherence and connectivity. Removing the verification module results in a smaller but still meaningful decline in Triple F1 from 84.72% to 81.25%, whereas Entity F1 remains relatively stable at 87.92%. This pattern is consistent with the verification module’s function as a relation-level quality filter that removes weakly supported or hallucinated triples. The decreases in Cluster Purity, Density, and LCC Ratio further suggest that verification improves both semantic consistency and structural reliability. Overall, the ablation results validate the effectiveness of the proposed three-stage pipeline, with normalization primarily enhancing semantic consistency and graph connectivity, and verification improving relational precision and factual reliability.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Ablation study results showing the contribution of each pipeline module.

https://doi.org/10.1371/journal.pone.0351132.t006

4.4. Application scenarios

4.4.1. Evidence-based query answering.

This scenario focuses on validating the effectiveness of the SCS-KG by addressing the complex, domain-specific query: “What historical evidence and claims concerning jurisdiction over the Nansha Islands and adjacent waters are represented in the SCS-KG corpus?” We compared responses generated under three conditions: DeepSeek alone, DeepSeek augmented with textual context, and DeepSeek integrated with the structured knowledge graph (DeepSeek+KG) to evaluate the KG’s ability to enhance response accuracy and contextual depth through structured domain knowledge. The following analysis is based on the selected source corpus represented in the SCS-KG and is intended to evaluate structured evidence retrieval rather than adjudicate competing sovereignty or jurisdictional claims.

The responses generated by the standalone DeepSeek model and the text-augmented DeepSeek model exhibited significant limitations (Fig 6). DeepSeek alone provided a broad chronological overview, often omitting specific dynastic details. The response was generic, relying on high-level references to official maps, and diminished in precision by vague or potentially confusing statements (as highlighted in red in Fig 6). For example, the response indicates that during the Qing period the islands were included in official maps and administered under “Guangdong/Hainan”. Historical scholarship indicates that during the Qing Dynasty, the islands were consistently recorded in officially sanctioned imperial maps and administered under Guangdong Province within which Hainan was governed as the Qiongzhou Prefecture at that time [37]. Therefore, the reference to “Guangdong/Hainan” requires clarification, as Hainan was historically a prefecture under Guangdong until modern times, which could be confusing. Although the answer correctly notes that 1946 reclamation of territory from Japan based on the Cairo and Potsdam Declarations, the inclusion of the statement “No post-war treaties transferred sovereignty” lacks nuance and source attribution. According to historical and legal scholarship analyzing China’s position, both the Cairo Declaration and the Potsdam Proclamation are regarded by China as internationally significant instruments which explicitly stipulated the return of territories seized by Japan, forming a key post-war legal basis for China’s stated post-war legal position regarding the South China Sea islands [37,38].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Answers generated under three scenarios.

Red highlights indicate areas of vagueness, ambiguity, overgeneralization, or insufficient evidence grounding in the first two responses, in contrast to the more precise and source-linked output on the right.

https://doi.org/10.1371/journal.pone.0351132.g006

Augmentation with general text offered some improvement in granularity but introduced new over-generalizations and potential inaccuracies (as highlighted in red). The DeepSeek+Textual Context response made unsubstantiated claims and overstated the evidentiary support for certain activities. Furthermore, ambiguous temporal references, such as the dual year reference for the publication of official island names, required clearer distinction. For instance, the assertion that the Tang Dynasty placed the islands under Zhenzhou (present-day Hainan) is not directly traceable to specific primary sources in the corpus. While some historical analyses and local records suggest this administrative allocation [39,40], contemporaneous primary Tang records concerning Hainan’s governance do not explicitly detail direct administrative control over the more distant South China Sea islands. Similarly, the statement that the Han Dynasty incorporated “Hainan and nearby waters” into its jurisdiction conflates distinct administrative scopes without source-level differentiation. Although the Han established commanderies like Zhuya and Dan’er on Hainan Island, extending this claim to constitute direct and sustained jurisdiction over specific South China Sea islands is an interpretation not fully substantiated by robust supporting evidence according to historical interpretations [37,41,42]. The claim that Guo Shoujing conducted specific astronomical surveys targeting the South China Sea islands is not substantiated by specific source references in the response. Although Yuan Dynasty historical records mention his “Sihai Ceyan” (Four Seas Survey) in the southern seas, direct and documented evidence pinpointing these activities to the Xisha or Nansha Islands remains limited and is partly based on later historical reconstructions [39]. Additionally, the reference to China publishing official island names in “1935/1947” needs clarification. The Republic of China’s committee for the Review of Maps of Lands and Waters released a map designating island names in 1935 [40,43]. The reference to 1947 pertains to a post-World War II period when the Chinese Ministry of Internal Affairs officially renamed the islands and produced a location map of the South China Sea Islands [43,44]. A clearer distinction between these two separate official acts is necessary to avoid conflation.

By contrast, the DeepSeek+KG approach produced a more source-grounded and granular account. Leveraging the SCS-KG’s structured domain knowledge (Fig 7), this response systematically delineated historical records and jurisdiction-related claims represented in the corpus. It linked specific historical records and claims to explicit primary sources (e.g., Yi Wuzhi, Genglubu), concrete historical events, and clearly delineated administrative units. This structured approach produced responses that were more detailed and more consistently traceable to specific source texts contained within the corpus, helping reduce some of the oversimplification and generalization issues observed in the other two methods.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Subgraph for the query.

The structured graph connects historical entities, events, primary sources, and multi-source evidence in a way that enables more source-grounded and contextually detailed reasoning and mitigates some of the generalization and precision issues seen in non-KG approaches.

https://doi.org/10.1371/journal.pone.0351132.g007

The case study indicates that the SCS-KG can integrate historical, geographical, and cultural data to provide more source-grounded and contextually detailed responses within the selected corpus context. The results suggest three key analytical strengths of the SCS-KG: Integrative Power (e.g., systematically linking fragmented data such as Yuan-era surveys to Ming-era navigation maps), Multi-Source Corroboration (e.g., linking Song-era administrative records with corresponding historical texts), and Nuanced Discovery (e.g., revealing implicit relationships between entities, such as the correlation between temple distributions and historical trade routes). By structuring domain knowledge into semantic triples and visual graphs, the SCS-KG supports a more systematic and nuanced investigation of complex interdisciplinary issues, demonstrating potential advantages over standalone language models in structured reasoning and evidence grounding, thereby supporting research use cases and potentially informing decision-support contexts.

4.4.2. Implicit relationship discovery.

Historical narratives of the South China Sea rely on fragmented evidence, making the connections (i.e., latent, multi-hop relationships) between macro-level historical governance (e.g., imperial policies) and micro-level social practices (e.g., folk navigation, fishing) difficult to establish and visualize using conventional keyword-based document retrieval or linear archival research [27]. This analysis aims to address the following question: How can the SCS-KG be used to discover and visualize multi-hop evidence chains that connect macro-level historical governance with micro-level social practices?

To explore these latent linkages, we employed a constrained multi-hop path query approach [45], which identifies the shortest or all possible paths within a maximum hop length () while filtering out semantically irrelevant connections. For this analysis, we set the maximum path length to balance interpretive depth and network coherence.

The query revealed several converging relational chains centered on Zheng He and the Genglubu through a coherent, 5-hop latent path, as illustrated in Fig 8. It connects state-level initiatives (the Ming Dynasty’s sponsorship of Zheng He’s voyages) with enduring grassroots practices (fishermen’s use of the Genglubu for navigation) through key intermediate Entities (Zheng He’s Nautical Chart and the Genglubu document). This structure illustrates how state-level initiatives and local navigational traditions are historically intertwined through the circulation and adaptation of maritime knowledge.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. Relationship path between Ming Dynasty and Hainan Fisherman.

The 5-hop chain links Ming state voyages to Hainan fishermen’s navigational practices, uncovering latent continuity in maritime knowledge that conventional methods struggle to reveal.

https://doi.org/10.1371/journal.pone.0351132.g008

This result systematically integrates previously fragmented evidence across different levels of historical activity, constructing a novel, multi-level historical narrative demonstrating continuity in maritime knowledge and practice. The discovered path suggests the SCS-KG’s capacity to move beyond simple document retrieval and facilitate the reconstruction of complex, non-linear historical relationships.

4.4.3. Entity-centric knowledge profiling.

Analyzing complex international relations and geopolitical processes requires the rapid, comprehensive profiling of key diplomatic instruments. This profiling depends on the structured aggregation of multi-dimensional information, including the identification of signatories (states/organizations), the underlying legal and institutional frameworks, core principles, spatial-temporal context, and subsequent developments or cooperative mechanisms. Traditional document-based approaches, however, remain linear, labor-intensive, and prone to information fragmentation and inconsistency across heterogeneous sources [46].

This scenario addresses the question: Can the SCS-KG serve as an analytical tool that enables “one-stop” aggregation and structured visualization of multi-dimensional information surrounding a major diplomatic agreement?

To address this, we adopted a 1-Hop Neighborhood Query approach, also known as an Ego-network Expansion. This graph-native method centers on a single entity (the ego node) and retrieves all one-hop neighbors (directly connected entities) and their semantic relationships (edges).The central entity (ego node) for this analysis is the Declaration on the Conduct of Parties in the South China Sea (DOC).

The query successfully generated a structured subgraph (Fig 9) that visually represents the core entity network of the DOC. Beyond showing connectivity, the visualization reveals distinct clusters that correspond to the agreement’s multidimensional nature. The first cluster (in light blue) encompasses the signatories, including China and the ASEAN member states, which collectively embody the multilateral character of the declaration. The second cluster (in purple) corresponds to the legal foundations, such as the 1982 United Nations Convention on the Law of the Sea (UNCLOS), which provides the normative and legal context for the DOC’s formulation. The third cluster (in pink) highlights the core objectives of the agreement, centering on notions of peaceful dispute resolution, mutual trust, maritime cooperation, and regional stability. The fourth cluster (in blue) shows that the South China Sea itself emerges as a distinct and central cluster, representing the spatial and geopolitical focus of the entire framework.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 9. Structured Geopolitical Profiling of DOC.

The visualization aggregates multidimensional elements (e.g., signatories, legal frameworks, objectives, and spatial context) into thematic clusters, illustrating SCS-KG’s efficiency for geopolitical analysis.

https://doi.org/10.1371/journal.pone.0351132.g009

By categorizing these neighboring nodes, the SCS-KG not only aggregates information but also reveals the structural logic and thematic coherence of the DOC’s knowledge domain. The resulting entity-centric profile offers an integrated analytical perspective that connects actors, legal norms, policy goals, and institutional trajectories within a single visual frame. This suggests the SCS-KG’s potential as a rapid, structured, and comprehensive analytical instrument for geopolitical and policy research. By enabling instantaneous aggregation and contextualization of heterogeneous information sources, the graph significantly enhances analytical efficiency and provides a systematic foundation for the exploration of complex diplomatic processes.