Ethics statement

This study analyzed only publicly available, de-identified and synthetic datasets—the CDISC SDTM/ADaM Pilot Project data and the open-source pharmaversesdtm package [7]—and did not involve human participants, identifiable patient data, or the collection of new data. Accordingly, institutional review board (IRB) approval and informed consent were not applicable to this work.

Two-layer architecture

The original design specified three layers: L1 (SHACL), L2 (probabilistic DAG), and L3 (agentic). After Gate B analysis determined that only 1 of 20 archetypes required multi-step reasoning beyond declarative graph constraints, L2 was dropped. The L1/L3 numbering is retained to match the pre-registration protocol. CAVE-Onc processes CDISC submission data through the following pipeline (Fig 1):

1. XPT-to-RDF adapter. SAS Transport (XPT) files for nine oncology domains (TU, TR, RS, EX, DM, AE, DS, TA, SUPPDM) are converted to RDF triples using a custom adapter that preserves variable labels, controlled terminology bindings, and RELREC foreign-key relationships under a dedicated CAVE namespace. The adapter handles trial-design domains without USUBJID (e.g., TA) via hash-based IRIs and expands SUPPDM qualifiers (QNAM/QVAL) into semantic cave:SUPP_{QNAM} triples for downstream cross-domain reasoning.
2. Layer 1 (SHACL validation). The RDF graph is validated against 111 SHACL shapes using pySHACL (v0.31) [10,11], an rdflib-backed Python SHACL validator. Shapes are partitioned into three categories: 85 CORE-ported shapes (see SHACL shape porting pipeline), 8 RECIST derivation shapes (see RECIST 1.1 derivation shapes), and 18 archetype-specific SHACL-SPARQL cross-domain constraint shapes (see Archetype-specific SHACL-SPARQL shapes). L1 produces a structured violation report as (USUBJID, shape_id, severity) tuples. Shapes targeting trial-design domains (TA) or supplemental qualifier domains (SUPPDM) produce flags without an associated USUBJID; these appear as “unknown” domain in the flag breakdown. Note: the original design specified Trav-SHACL over an Oxigraph triple store, but pySHACL was selected for its mature SHACL-SPARQL support; migration to Trav-SHACL is recommended for production scaling (see Discussion).
3. Layer 3 (LangGraph agent). Subjects with L1 violations or those flagged by routing heuristics enter the CaveAgent, a LangGraph state machine with typed Pydantic schemas. The agent has access to SPARQL query tools over the RDF graph and a RECIST Table 7 lookup table [4]. L3 invocation rate is a primary reported metric.
4. Audit store. All validation traces are written to an append-only SQLite WAL store with a Merkle hash chain linking consecutive entries, providing a tamper-evident audit trail foundation compatible with 21 Code of Federal Regulations (CFR) Part 11 requirements [12].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. CAVE-Onc two-layer validation architecture.

XPT files are converted to an RDF knowledge graph, validated by 111 SHACL shapes (L1), and selectively routed to a LangGraph CaveAgent (L3) for RECIST Table 7 reasoning. All traces are recorded in a Merkle-chained audit store.

https://doi.org/10.1371/journal.pone.0350376.g001

SHACL shape porting pipeline

We systematically ported CORE rules scoped to oncology domains (TU, TR, RS, EX, DM) into SHACL shapes:

• Source: CORE v0.15 cache snapshot (commit 88837395), oncology-filtered to n = 122 rules.
• Classification pipeline: Rules were classified as single-domain (portable to SHACL) or cross-domain (unportable). Single-domain rules underwent polarity auditing and recursive De Morgan expansion.
• Port success: 85/122 rules (69.7%) ported with zero expressiveness compromise. The 37 unportable rules fell into two buckets: 31 cross-domain join rules and 6 row-set uniqueness rules.
• Port-kind distribution: De Morgan expansion (50/85, 58.8%), polarity flip (30/85, 35.3%), no change (5/85, 5.9%).
• Per-domain breakdown: DM = 74 shapes (67.9% port rate), EX = 5 (71.4%), TR = 4 (100%), RS = 2 (100%).

Full porting statistics, including per-rule classification and polarity-audit outcomes, are in S2 Table in S1 File.

RECIST 1.1 derivation shapes

Eight SHACL-SPARQL shapes were authored from RECIST 1.1 clinical criteria [4] to validate derivation logic between TR (tumor measurements) and RS (response assessments). These shapes enforce target-lesion JOIN patterns, lymph-node short-axis VALUES, two-stage baseline sub-queries, and confirmation-window requirements. Shapes were reviewed and approved by an independent clinical reviewer.

Archetype-specific SHACL-SPARQL shapes

Eighteen SHACL-SPARQL shapes were authored to target 18 of the 20 contradiction archetypes (A01–A15, A17, A18, A20) using cross-domain SPARQL constraints within sh:sparql blocks. Of the remaining two archetypes, A16 (duplicate USUBJID across studies) is a row-set uniqueness contradiction with no dedicated archetype shape (the duplicated record re-triggers the existing domain shapes, so it is credited structurally as a side-effect rather than by a bespoke cross-domain constraint), and A19 is handled by the L3 agent. These shapes implement nine distinct constraint patterns: tumor response consistency (A01, A06, A18), new lesion detection (A02), temporal consistency (A03, A07, A20), visit alignment (A04), supplemental qualifier cross-reference (A05, A13), valueset cross-reference (A08), conditional existence (A09, A12, A14), date derivation (A10, A11, A15), and cross-record consistency (A17).

CaveAgent with RECIST Table 7 lookup

The L3 CaveAgent implements RECIST Table 7 overall response verification. Given a subject’s visit-level data across TR, RS, and TU domains, the agent: (1) queries the RDF graph for target lesion Sum of Longest Diameters (SLD) change from baseline; (2) queries non-target overall response; (3) queries new lesion status; (4) applies the RECIST Table 7 decision matrix to determine the expected overall response; (5) compares the expected response against RSORRES and emits a trace if they conflict. The agent operates within LangGraph’s constrained state machine, ensuring deterministic execution paths.

Contradiction injection corpus

Twenty contradiction archetypes were enumerated with clinical review (Gate B) covering nine domains (RS, TR, TU, DM, EX, AE, DS, TA, SUPPDM); the full per-archetype catalog is in S1 Table in S1 File. By provenance the archetypes fall into two groups: ten (A08–A17) were derived during the Gate A gap analysis from CORE’s own conformance-rule corpus—each corresponds to an existing CORE rule that CAVE re-expresses as a cross-domain shape—while the other ten (A01–A07, A18–A20) are RECIST 1.1 overall-response and protocol-extension contradictions with no corresponding CORE rule. This split underlies the Track B baseline comparison (see Results). Each archetype was injected into clean pharmaversesdtm data [7] using a RELREC-preserving injector that modifies clinical values at the semantic layer while maintaining referential integrity.

Evaluation design

Evaluation was pre-registered (OSF: https://osf.io/fx2ky) with four hypotheses (Table 1).

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Pre-registered evaluation hypotheses.

https://doi.org/10.1371/journal.pone.0350376.t001

Track A (regression/complementarity): CDISC Pilot 1 data (52 subjects, oncology domains DM and EX) was validated by both CAVE L1 and CORE CLI. Flag sets were compared as (USUBJID, rule_id) tuples. Note: pharmaversesdtm RECIST data (6 subjects) was used for Track B injection only; Track A results are reported on Pilot 1.

Track B (novelty): Each of 20 archetypes was injected into the clean corpus. Four configurations were run: two empirical industry baselines (the Pinnacle 21 Community FDA production engine and the CORE engine), L1-only (ablation), and full CAVE (L1 + L3). Detection was measured as binary (detected/not detected) per archetype.

Audit trail

All validation runs produce append-only traces in a SQLite WAL store. A Merkle hash chain links consecutive entries, enabling post-hoc tamper detection. This provides a foundation for 21 CFR Part 11 compliance [12]; full Part 11 implementation (e-signatures, RBAC, IQ/OQ/PQ) is out of scope.

Track A: Rule-class complementarity

On clean reference data (52 subjects from CDISC Pilot 1, oncology domains DM and EX), CAVE L1 and CORE produced substantially different flag sets (Table 2).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Track A flag comparison on clean data.

https://doi.org/10.1371/journal.pone.0350376.t002

The Jaccard similarity between the L1 and CORE flag sets, computed over deduplicated (USUBJID, rule) pairs, was 0.004 (24 shared pairs), confirming H1 (threshold < 0.10). Two of 18 CORE rules achieved recall = 1.0 (CORE-000655, CORE-000656), both addressing arm assignment consistency in DM; however, L1 produced 306 flags per rule versus CORE’s 12, reflecting the broader matching scope of SHACL shapes operating over the full RDF graph rather than a filtered domain DataFrame. The remaining 16 CORE rules had recall = 0.0, consistent with the complementary-class hypothesis (per-rule recall for all 18 CORE rules is in S3 Table in S1 File).

Track B: Contradiction detection

On the 20-archetype injected corpus of RECIST-enriched pharmaversesdtm data, CAVE detected 20/20 archetypes (100%; exact binomial 95% CI [13]: 83.2%–100.0%) (Table 3). This count is reproducible from the released shapes under the same subject-specific criterion used for the held-out and real-data studies—each archetype’s own shape (or, for A19, the L3 agent) fires on the injected subject, and the two cross-subject/structural archetypes (A16, A17) fire cohort-wide—so it does not depend on a global flag-count delta. We ran the CDISC Rules Engine (CORE v0.15) on the same injected corpus, counting an archetype as CORE-detected only when a new CORE violation whose rule semantics are the injected contradiction fires on the target subject (a per-archetype adjudication reported in full in S12 Table in S1 File): it detected 8/20 archetypes (40%; 95% CI: 19.1%–63.9%). All eight CORE detections fall among the ten archetypes that were deliberately seeded from existing CORE conformance rules (e.g., ARMCD absent from the TA codelist, RFXSTDTC/RFXENDTC reference-date aggregation, duplicate USUBJID, ARMCD–ARM not one-to-one), which CAVE re-expresses as ported SHACL shapes. We additionally ran the branded Pinnacle 21 Community FDA production engine (FDA 2405.2)—the validator pharmaceutical sponsors submit with—on the identical corpus under the same adjudication (per-archetype adjudication in S11 Table in S1 File): it detected 6/20 (30%; 95% CI: 11.9%–54.3%), again entirely within the CORE-seeded structural class. The two engines ship slightly different structural rule sets (the FDA engine lacks CORE’s reference-date-aggregation and multiple-race checks but adds a missing-exposure check), a portability difference; neither expresses the cross-domain class. On the ten non-CORE archetypes (the cross-domain RECIST overall-response, confirmation-window, and multi-domain temporal/value contradictions that domain-scoped rules cannot express), both industry engines detected 0/10 while CAVE detected 10/10 (McNemar’s exact test [14] p = 0.002 for each). Across all 20 archetypes, CAVE detects 12 archetypes CORE misses and 14 the FDA engine misses (McNemar exact p < 0.001 for both). This split is the crux of the expressiveness claim: the industry validators catch the structural cross-domain checks they already encode, but none of the RECIST semantic contradictions.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Track B detection results by configuration. Two industry validators were run empirically on the injected corpus (SDTMIG 3.4): the branded Pinnacle 21 Community FDA production engine (FDA 2405.2) and the open CDISC Rules Engine (CORE v0.15). The 20 archetypes split into ten CORE-seeded structural checks (A08–A17, derived from existing CORE rules) and ten non-CORE cross-domain RECIST/semantic contradictions (A01–A07, A18–A20).

https://doi.org/10.1371/journal.pone.0350376.t003

Of the 20 detected archetypes, 19 were detected at L1—18 by the archetype-specific SHACL-SPARQL cross-domain constraints and A16 (duplicate USUBJID) structurally through the domain shapes its duplicated record re-triggers—and 1 (A19) by the L3 agent exclusively (per-archetype results are reported in S4 Table in S1 File). H2 (CAVE novelty) was supported: on the ten non-CORE cross-domain archetypes CAVE detected all ten that CORE missed (McNemar p = 0.002), and across all 20 archetypes CAVE detected 12 that CORE missed (p < 0.001). The eight archetypes CORE also detects are exactly those seeded from its own conformance rules, which CAVE re-expresses as ported SHACL shapes—a portability rather than novelty result. H3 (agent layer value) was supported: L3 produced 1 trace for A19 that L1 alone did not detect. Fig 2 shows the detection pattern across all configurations.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Contradiction archetype detection across the four configurations of Table 3: The two industry baselines (Pinnacle 21 FDA, CORE), the L1-only ablation, and full CAVE (L1 + L3).

Blue indicates detection; light grey indicates non-detection. A19 (dashed border) is the only archetype detected exclusively by the L3 agent; all others are detected by L1 SHACL-SPARQL shapes. The industry engines detect only CORE-seeded structural archetypes (A08–A17) and none of the non-CORE cross-domain class.

https://doi.org/10.1371/journal.pone.0350376.g002

Ablation analysis

The L1-only ablation detected 19/20 archetypes and missed A19. Adding L3 increased detection to 20/20, with the single additional trace corresponding to the RECIST Table 7 contradiction. This ablation confirms that L3 adds A19 detection beyond L1 alone, demonstrating the agent layer’s unique capability for cross-domain semantic reasoning.

Generalization beyond constructed archetypes

Because the 20 archetype shapes were authored with knowledge of the injected patterns, the 20/20 result is a construction validation rather than a generalization estimate (see Limitations). To probe transfer to unseen contradictions, we constructed five held-out archetypes (H01–H05) after the shape library was frozen, each targeting a pattern none of the 18 archetype shapes was designed to catch, and evaluated them against the existing L1 shapes and L3 agent with no new shape authoring (Table 4). Detection was credited only when a new violation appeared on the injected subject, rather than from any change in the global flag count—a subject-specific criterion that avoids the over-crediting a naive global-delta count can produce from baseline enrichment noise (see Limitations). The existing shapes detected 3/5 held-out archetypes. Of these, 2/5 caught the intended cross-domain mechanism (H01 via the new-lesion shape used for A02; H04 via the value-robust shape used for A18), H03 was caught only coincidentally by the L3 Table 7 agent, and two genuinely novel patterns (H02 consent-after-exposure; H05 exposed-but-unassigned) were missed and would require new shapes. This partial, shape-dependent transfer is the expected behaviour of a genuine expressiveness test and distinguishes CAVE-Onc from a corpus-specific rule set.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Held-out archetype generalization (authored after the shape library was frozen; no new shapes added).

https://doi.org/10.1371/journal.pone.0350376.t004

Runtime performance

In the enriched Track B benchmark, L1 validation over 111 shapes required 110.032 s per archetype, L3 agent execution required 0.025 s, and the full pipeline required 110.057 s. The clean-data L1 baseline was 103.390 s, yielding a pipeline overhead ratio of 1.06. H4 (runtime efficiency) was supported: the full CAVE pipeline incurred only 6% overhead over the clean-data baseline, well within the pre-registered 10 threshold (Fig 3). L3 agent execution added a negligible 25 ms per archetype. This 25 ms is the full RECIST Table 7 derivation (querying overall, non-target, and new-lesion response and applying the lookup) executed over all subjects on every archetype run (3.1 ms per subject across 8 subjects); it is not an A19-specific inference cost, as the agent has no inference-skipping fast path. Supplementary scaling analysis (S8 Table in S1 File) shows sub-linear wall-clock growth from 8 to 80 subjects (7.75 time for 10 data).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Mean validation time per archetype by component.

A. Full scale comparing the clean-data L1 baseline, the injected-data L1 run, and the full CAVE pipeline (the 1.06 overhead ratio is CAVE relative to the clean-data L1 baseline). B. Zoomed view of L3 agent overhead (25 ms).

https://doi.org/10.1371/journal.pone.0350376.g003

Formal detection metrics

At the archetype detection level, CAVE produced 20 true positives and 0 false negatives, corresponding to precision = 1.000, recall = 1.000, and F1 = 1.000. The McNemar comparison against CORE was significant (p < 0.001), and the exact binomial 95% CI of [83.2%, 100.0%] reflects the finite sample size (n = 20 archetypes); bootstrap confidence intervals are reported in S7 Table in S1 File.

L3 invocation rate and cost

L3 (CaveAgent) executed on all 20 archetype runs but triggered detection on only 1 (A19), yielding an effective detection rate of 5% (1/20). We distinguish detection from compute: the agent’s Table 7 derivation runs on all 20 archetype runs (see Runtime performance), so the 5% figure describes how often that derivation surfaces a contradiction, not how often the agent executes. The primary A19 detection is fully deterministic: the LangGraph state machine executes SPARQL queries over the RDF graph and applies the RECIST Table 7 lookup, incurring $0.000 API costs per subject while guaranteeing perfect reproducibility.

To evaluate the robustness of LLMs in cross-checking these deterministic findings, a dual-channel verification benchmark was executed across four major LLMs (DeepSeek V3, GPT-4o, Claude Sonnet 4, and GLM-4.6). DeepSeek and GPT-4o achieved 100% agreement with the deterministic logic (the full per-case agreement matrix is in S9 Table in S1 File). Claude and GLM both achieved 75% agreement, notably preserving the non-standard NON-CR/NON-PD terminology for target lesions instead of normalizing it to SD. This highlights a clinically defensible terminological ambiguity that multi-model cross-validation surfaces. In addition to verification, the L3 agent successfully generated clinician-readable narrative explanations for all 20 archetypes, demonstrating its utility as a semantic explainability layer. LLM API costs for cross-verification were negligible (on the order of $0.002–$0.04 per call at provider list prices, given the short prompts and 300-token completions).

Expert validation (G8/G9)

Three domain experts were recruited based on oncology CDISC experience (10 years each in statistical programming or data standards review at pharmaceutical/biotech companies). Each reviewer independently received the same workbook containing all 20 archetype descriptions, detection examples, and L3 agent traces. Reviewers rated each archetype as valid, invalid, or uncertain and provided qualitative rationale. They also rated trace readability and system confidence on 5-point Likert scales. No compensation was provided. Fleiss’ [15,16] was computed for the 3-category validity ratings. Table 5 summarizes results.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Expert validation results (G8: archetype validity, G9: trace readability).

https://doi.org/10.1371/journal.pone.0350376.t005

No reviewer rated any archetype as invalid. Four archetypes received majority “uncertain” ratings (A04: visit-window violation, A07: PR confirmation, A12: death flag, A20: iRECIST confirmation; three of these were unanimous, one majority), all reflecting protocol-dependent interpretation rather than clinical invalidity. Reviewers consistently recommended that these archetypes be configurable per study protocol rather than removed. To substantiate this classification, we subjected these four archetypes to an LLM uncertainty triage layer using the same four models (DeepSeek, GPT-4o, Claude, GLM). The models achieved 100% unanimous consensus, correctly classifying all four as protocol_dependent rather than genuine violations or false alarms (see S10 Table in S1 File). The L3 agent trace for A19 received a unanimous 4/5 readability score, with reviewers noting it would benefit from additional subject identifiers and audit-replay metadata.

Real-world data validation

To test whether detection transfers beyond the synthetic benchmark, we obtained two independent oncology trials from Project Data Sphere and mapped each from its legacy sponsor format to the SDTM model the engine consumes: trial 123 (Synta 4783−08; 325 subjects) and trial 107 (CA012, metastatic breast cancer; 227 subjects). Mapping fidelity, the principal validity threat, was locked by regression tests: the sum-of-longest-diameters invariant ( target LDIAM per visit) held in 760/815 (93%) of Synta visits (the 55 exceptions being systematic lymph-node short-axis cases, a disclosed mapping limitation) and in 549/549 (100%) of CA012 visits, with zero orphan lesion links in either trial. All validation ran on the Oxigraph backend over the full cohorts.

We ran two complementary studies on each trial (Table 6). In the naturalistic specificity study (E1) the engine validated the unmutated real data and stayed highly specific, raising only 0.06 (Synta) and 0.09 (CA012) contradiction-archetype flags per subject; the genuine RECIST-derivation signal was dominated by the SLD-math invariant, and on CA012—which carries assessment dates—the 28-day confirmation rule (S7) was exercised for the first time. In the mutation-transfer study (E2) we injected each of the 20 archetypes into the real trial structure and credited detection only when the archetype’s own shape fired on the injected subject but not on that subject’s clean baseline—the same subject-specific criterion used for the held-out probe (Generalization beyond constructed archetypes). On Synta, 10 of 11 applicable archetypes were detected (90.9%); on the richer CA012, 16 of 18 (88.9%) (S2 Fig in S1 File). Across both trials, 16 distinct archetypes were detected on at least one real trial.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Real-world validation on two independent Project Data Sphere oncology trials (Oxigraph backend, full cohorts). E1: naturalistic specificity on unmutated data; E2: per-subject mutation-transfer recall on applicable archetypes (those whose required domain/dates the trial provides).

https://doi.org/10.1371/journal.pone.0350376.t006

Applicability was determined by data availability, not detection ability. Archetypes requiring an Exposure (EX) domain or fine-grained dates—which the Synta package omits—were not exercisable on Synta but became so on CA012, which ships dosing records and assessment-day offsets, automatically widening the applicable set from 11 to 18. The residual gaps are nameable: A03 requires an attributed Adverse Event domain—Synta provides none, and the solicited-toxicity AE we map for CA012 records no drug-causality variable, so the attributed-AE join cannot be exercised; A19 (the L3 Table 7 detector) requires the benchmark’s single overall-response row and the NTOVRLRESP/NEWLEC test codes that real multi-row RS lacks; and the two temporal-confirmation injectors (A07, A20) could not manufacture a fresh contradiction on CA012’s sparse four-visit schedule. The latter two are injector artifacts, not detector gaps: A07’s shape fires naturally on four clean CA012 subjects, and a schedule-aware variant injector—kept separate from the validated benchmark generator so the headline recall and the Track B result are unchanged—reproduces both contradictions on CA012’s real structure, where the A07 and A20 detectors then fire. The lone Synta miss (A18) was an injector artifact—it targets the screening visit, which carries no overall-response row—and was detected on CA012, where that visit does. These results indicate that the engine’s contradiction-detection logic generalizes across two independent real trials wherever the supporting domain is present, and that the boundaries reflect input completeness rather than tailoring to the constructed corpus. We frame this as a real-data specificity and transfer study on mapped submission-grade data, not an estimate of contradiction prevalence in regulatory submissions, which would require labelled real contradictions.

Download:

• PNG
  larger image
• TIFF
  original image

Table 7. Maintainability of the A19 RECIST Table 7 logic encoded as a single SHACL-SPARQL constraint versus the L3 agent (verified functionally equivalent).

https://doi.org/10.1371/journal.pone.0350376.t007