2.1: Literature review: Translation methods for Japanese attributive clauses

[21] proposed two translation methods for Japanese attributive clauses. Method A involves directly rendering shorter attributive clauses as attributive structures in Chinese (the original text"短的定语可以直接照译"). Method B is applied to longer attributive clauses, where it is often necessary to relocate the attributive clause to the end of the main sentence (the original text"较长的递加定语多数要后移"). Additionally, [21] proposed four specific translation methods for longer Japanese attributive clauses as mentioned in Method B. Method B-1: Translate the attributive clause into Chinese directly without changing the modification relationship (the original text"不变语序, 直接连译"). Method B-2: Translate the attributive clause into a separate sentence, followed by the main clause(the original text"定语提位, 代指下连"). Method B-3: Translate the main clause first and then translate the attributive clause(the original text"先抓住干, 后理分支"). Method B-4: Divide the attributive clause into two parts, use one part as a modifier before the modified noun, and translate the other part as a complement(the original text"部分定语不变, 部分定语后置").

[22] proposed three translation methods for translating longer Japanese attributive clauses. Method C: Long Japanese attributive clauses can be translated into long attributive clauses in Chinese(the original text"長い連体修飾語をそのまま訳して中国語でも長い連体修飾語とする"). Method D: Use long attributive clauses as predicates, with the center words serving as subjects, and then continue the sentence with appropriate words(the original text"長い連体修飾語を述語とし、その中心語を主語として、独立したセンテンスにする。そして、そのあとに適当なことばでそれを受けて話をつづける"). Method E: Position the center word first (or along with a short attributive clause) and process the long attributive clause as a supplementary explanation that complements the center word with appropriate words (which may not always be necessary) (the original text"中心語、またはプラス短い連体修飾語をさきに出し、長い連体修飾語は適当な言葉で、使わない場合もあるが、中心語を受けてそれを補足説明する形として処理する").

Although [21, 22] proposed translation methods for Japanese attributive clauses, they did not elaborate on the characteristics of the "attributive clause + modified noun" structure in Japanese that each method is best suited for, nor did they discuss whether the semantic role of the modified noun influences the selection of a translation method. This lack of detail limits the development of appropriate prompts for LLMs based on these methods.

2.2: Large language model and prompting for machine translation

There are substantial differences between the output processes of LLMs and NMT.

Fig 1 illustrates that conventional NMT tools directly present the translated version of the original input sentence. In contrast, LLMs require a specific template to be applied for formatting the original sentence to generate the translation, such as, ’Translate the following sentence to Chinese: {original input sentence}’. In other words, given a pretrained and fixed LLMs, MT prompting first converts each test input to a prompt according to a template and then generates the translation by feeding the prompt to the LLMs. This highlights that the quality of the output from large language models is largely dependent on the quality of the prompts [15]. Therefore, by optimizing the design of prompts, we can significantly enhance the output quality of LLMs-based translations [23].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Illustration of our proposed prompt-enhanced MT framework compared to conventional NMT tools (e.g.

Baidu translation) and vanilla LLM-based MT. Note: The machine translation site utilized in this study generated all translated texts, which were obtained on March 26, 2023.

https://doi.org/10.1371/journal.pone.0313264.g001

The central focus of this paper is to explore how the learning capabilities of large language models can be leveraged to incorporate linguistics theory into prompt design, thereby guiding these models to produce translations that better meet user expectations.

Data selection and collection

The study compiled a parallel corpus consisting of five officially published modern Japanese novels and their corresponding Chinese translations. The selected novels include Half a Confession (《半落ち》 [24]), Convenience Store Woman (《コンビニ人間》), Secret (《秘密》 [25]), Wings in the Darkness (《翼のある闇》 [26]), and The Housekeeper and the Professor (《博士の愛した数式》 [27]). Their respective Chinese translations are Half a Confession (《半落》 [28]), Convenience Store Woman (《人间便利店》 [29]), Secret (《秘密》 [30]), Wings in the Darkness (《有翼之暗》 [31]), and The Housekeeper and the Professor (《博士的爱情算式》 [32]). These novels span a range of genres, including suspense, ethics, and science fiction, with language that closely mirrors everyday speech, ensuring that the corpus is both accurate and rich in linguistic diversity.

Through manual annotation, 64 sentence pairs featuring a "nominative inner relation attributive clause + modified noun" structure was extracted from the parallel corpus, along with their corresponding Chinese translations. Dialectal variations were excluded during data selection to enhance the generalizability of the study’s findings.

Potential biases

To mitigate potential biases associated with using a single source of data, the study included a diverse range of materials covering different themes. However, it is important to note that the linguistic expressions found in literary works may differ from those used in everyday language, which could influence the evaluation of translation quality.

Statistical methodology

To provide a clear and concise representation of how prompt optimization affects translation quality, this study employed descriptive statistical methods. Descriptive statistics were used to summarize and interpret the experimental data, revealing overall trends in translation quality scores. However, it is important to recognize that descriptive statistics do not allow for significance testing. Consequently, the study’s conclusions focus on illustrating general trends rather than confirming the statistical significance of prompt optimization. Future research could incorporate more rigorous statistical techniques, such as paired t-tests, to assess the significance of the differences in translation quality before and after prompt optimization.

5.1: Translation patterns of Japanese "nominative" inner relation attributive clauses

When translating Japanese "nominative" inner relation attributive clauses into Chinese, the decision to adopt the "splitting long attributive clause and main clause into two separate sentences" translation method poses a challenge in determining whether the modified noun should initially fill in the valence complement of the verb in the attributive clause or the valence complement of the verb in the main clause. Prioritizing the valence complement of the verb in the attributive clause will result in the content of the attributive clause appearing before the main clause, while prioritizing the valence complement of the verb in the main clause will make the content of the main clause appear before that of the attributive clause.

If the modified noun gives priority to filling in the valence complement of the verb in the attributive clause, then the translation pattern is:

Pattern I: (Inner relation attributive clause + modified noun) & Main clause (Japanese) = > Attributive clause & modified noun, main clause (Chinese)

If the modified noun gives priority to filling in the valence complement of the verb in the main clause, then the translation pattern is:

Pattern II: (Inner relation attributive clause + modified noun) & Main clause (Japanese) = > Modified noun & main clause, Inner relation attributive clause (Chinese)

In (11), the translated sentence of (10), the modified noun "平介 (Heisuke)" gives priority to filling in the valence complement of "帰宅した (came home)", which is the verb in the attributive clause. The content of this attributive clause "この日夜勤明けで、午前八時ちょうどに帰宅した (came home from his night shift at exactly 8 a.m.)" comes before the main clause "四畳半の和室に入るなり、テレビのスイッチを入れた (entered the small tatami room, and turned on the television)". On the other hand, in (12), "平介 (Heisuke)", the modified noun, gives priority to filling in the valence complements of the verbs "入る (entered)" and "入れた (turned on)", which are verbs in the main clause. The content of the main clause comes before that of the attributive clause.

(10)この日夜勤明けで、午前八時ちょうどに帰宅した平介は、四畳半の和室に入るなり、テレビのスイッチを入れた。(Heisuke came home from his night shift at exactly 8 a.m., entered the small tatami room, and turned on the television)

(秘密(Naoko: A Novel [33]))

(11)平介这一天下夜班, 回到家刚好是上午八点, 一走进四叠大小的和室, 便打开了电视。

(Pattern I, Translated by the author)

(12)平介一走进四叠大小的和室, 便打开了电视。他这一天值完夜班, 回到家刚好是早上八点。

(Pattern II, Translated by the author)

5.2: Exploring the relationship between the semantic role of modified nouns and translation pattern selection

How should we decide whether the modified noun should be placed in the valence complement of the attributive clause verb or the valence complement of the main clause verb? We propose a hypothesis: the priority order depends on the strength of the subordinate relationship between the semantic role of the modified noun and the verb. If the modified noun’s semantic role has a stronger subordinate relationship with the attributive clause verb, it should be placed first in the valence complement of the attributive clause verb. Conversely, if the modified noun’s semantic role has a stronger subordinate relationship with the main clause verb, it should be placed first in the valence complement of the main clause verb. If there is no difference in the strength of the subordinate relationship between the modified noun and the main clause verb and that with the attributive clause verb, the modified noun can be placed in either valence complement first.

[34] proposed that Japanese language represents various subordinate relationships between nouns and verbs with different semantic roles. Table 1, drawn from Nitta’s research, lists the strong and weak subordinate relationships between each semantic role and verb. In this study, we will employ this theory to examine and evaluate the strength of the relationship according to the classifications in the Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Degree of subordination of verbs.

Note: Fig 2 was compiled by the author based on [34]’s research.

https://doi.org/10.1371/journal.pone.0313264.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Semantic role relationship between modified noun and attributive/main clause verb in inner relation attributive clauses.

https://doi.org/10.1371/journal.pone.0313264.t001

To validate the aforementioned hypothesis, this paper extracted 64 instances of data pertaining to "nominative" inner relation attributive clauses from five Japanese novels and examined the translation patterns of their respective Chinese versions. The analysis results have been summarized in Table 1.

When the semantic role relationship between the modified noun and the verb in the main clause is “nominative,” the strength of the subordinate relationship between the semantic role of the modified noun and the verb in the main clause and the subordinate relationship between the semantic role of the modified noun and the verb in the attributive clause are equal. It is abbreviated as “attributive clause verb = main clause verb.” When the semantic role relationship between the modified noun and the verb in the main clause is other than nominative, then the subordinate relationship between the semantic role of the modified noun and the verb in the main clause is weaker than that between the semantic role of the modified noun and the verb in the attributive clause. It is abbreviated as “attributive clause verb > main clause verb.” The correlation between the degree of subordination between semantic roles attributed to modified nouns and verbs and the selection of translation mode is presented in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Degree of subordination between attributive/main clause verbs.

https://doi.org/10.1371/journal.pone.0313264.t002

Based on the results presented in Table 2, it is evident that Pattern I has a significantly higher number of translations than Pattern II. This supports the previously proposed hypothesis that the choice of translation mode for "nominative" inner relation attributive clauses is linked to the degree of subordination between the semantic role of the modified noun and the verb. As "nominative" inner relation attributive clauses have a "nominative" semantic relationship, which represents the highest level of subordinate attention intensity, when translating into Chinese, it is recommended that priority be given to using the modified noun as the valence complement of the attributive clause verb–nominative.

6.1: Machine translation linguistic metric

To accurately assess the quality of machine translation, it is essential to establish comprehensive evaluation criteria. Building on [35] Japanese-English machine translation evaluation framework, we have developed Japanese-Chinese machine translation evaluation criteria that take into account the unique characteristics of Japanese-Chinese translation. The resulting evaluation framework is presented in Table 3 for a more intuitive and informative reflection of machine translation quality.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Japanese-Chinese machine translation quality evaluation criteria.

https://doi.org/10.1371/journal.pone.0313264.t003

6.2: Experiments and findings

Although the linguistic conclusion is valuable in enhancing translation quality, it may not be immediately applicable to improve the output of translation websites like Google and Baidu. It’s because these websites operate solely in the "input-output" mode, with no provision for users to debug the model. Fortunately, LLMs like ChatGPT are interactive translation tools that function within a dialogue-based framework, allowing users to iteratively obtain superior outputs by adding prompts. This paper aims to convert the linguistic conclusion derived in Section 5 into a prompt and evaluate its effectiveness by comparing the translation quality before and after implementing the prompt.

The recommendation for translating "nominative" inner relation attributive clauses into Chinese is to prioritize using the modified nouns to fill the valence complement of the attributive clause verbs, which means filling the nominative case of the attributive clause verbs with the modified nouns. Additionally, "the content of the attributive clause is placed before the content of the main clause" can be transformed into three-step prompt chaining:

Prompt A: What is the noun modified by the attributive clause in the following sentence? "SL".

Prompt B: Please place the noun modified by the attributive clause in the subject position of the attributive clause, and then separate "SL" into two sentences.

Prompt C: Translate the following sentence into Chinese. " SL‴

These three prompts—Prompt A, Prompt B, and Prompt C—form a chain in which "SL" refers to the original Japanese text in Prompts A and B, while "SL‴ in Prompt C is the modified Japanese text that has been split into two sentences based on the LLM’s response to Prompt B.

To assess the efficacy of the prompt, we selected five corpora featuring Japanese "nominative" inner relation attributive clauses and conducted translation experiments using ChatGPT. The experimental procedure entailed the following steps (Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Experimental procedure for assessing prompt efficacy on Japanese inner relation attributive clauses.

https://doi.org/10.1371/journal.pone.0313264.t004

By adding a prompt, the Japanese translation of "社員の真似をして、勢いよくお辞儀をした私に、女性は笑って「ありがとうね、またきます」と言い、レジから去って行った"changed from "看到店员的样子, 我也效仿着热情地鞠了一躬, 然后那位女性笑着对我说: “谢谢啦, 下次还会再来的。” 她说完后离开了收银台" (A1) to "我模仿店员的样子, 热情地鞠躬。女性笑着对我说: “谢谢你, 再见”, 然后离开了收银台" (A4). The translation quality has also been improved from 4 to 5.

This paper conducted the above experiments on five data samples. We performed the same experiment on the remaining four Japanese texts. The translation produced by Result A1 corresponding to Prompt Q1 is referred to as TL1, and the translation generated by the final Result A4 corresponding to the adjusted prompt combination is referred to as TL2. Table 5 below presents TL1 and TL2 for each text. Fig 2 displays the translation quality scores for TL1 and TL2 for each set of SL. Fig 3 illustrates the translation method choices for TL1 and TL2 for each set of SL.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Translation quality changes before and after prompt optimization (translation quality score).

https://doi.org/10.1371/journal.pone.0313264.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Translation quality changes before and after prompt optimization.

(SL and TL).

https://doi.org/10.1371/journal.pone.0313264.t005

In Fig 3, the X-axis represents translation quality scores, and the Y-axis represents the five SL sets. The blue bars indicate the translation quality scores before optimization (TL1), while the orange bars represent the scores after optimization (TL2).

From Table 5 and Fig 3, it is evident that the quality scores of all five translations (TL2) after prompt optimization are higher than those produced using the zero-shot prompt (TL1). Specifically, by integrating linguistic prompts into ChatGPT, the average translation quality improved significantly from 3.2 to 4.6, marking a notable 43.75% increase. This demonstrates that designing prompts based on linguistics theory can indeed enhance the quality of Chinese translations of Japanese sentences with complex attributive clause structures.

Moreover, (13), (14), and (15) illustrate that TL1 incorporated linguistic details absent in SL, such as "I saw the behaviour of the clerk," whereas TL2 reflected content akin to that of SL, showing no signs of over-translation or under-translation, or mistranslation. Additionally, TL2 features a subject placement consistent with Chinese expression norms, rendering the language more natural, fluent, and concise compared to TL1.

(13)社員の真似をして、勢いよくお辞儀をした私に、女性は笑って「ありがとうね、またきます」と言い、レジから去って行った。

(SL)

(14)看到店员的样子, 我也效仿着热情地鞠了一躬, 然后那位女性笑着对我说: “谢谢啦, 下次还会再来的。”她说完后离开了收银台。

(TL1)

(15)我模仿店员的样子, 热情地鞠躬。女性笑着对我说: “谢谢你, 再见”, 然后离开了收银台。

(TL2)

In Fig 4, the blue bars represent the translation method that directly translates the Japanese "attributive clause + modified noun" structure into the corresponding Chinese structure (Translation Method 1), while the orange bars represent the method that does not directly translate this structure (Translation Method 2). The X-axis shows the translations generated using the zero-shot prompt (TL1) and those generated after prompt optimization (TL2), while the Y-axis indicates the frequency of each translation method.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Translation method changes before and after prompt optimization.

https://doi.org/10.1371/journal.pone.0313264.g004

From Table 5 and Fig 4, we observe that with the zero-shot prompt, one set of SL (SL1) used Translation Method 1, while four sets of SL (SL2, SL3, SL4, SL5) employed Translation Method 2. However, after applying the optimized prompt, all five sets of SL adopted Translation Method 1. This demonstrates that, with the optimized prompt, the large language model exhibits greater consistency in its choice of translation strategies. This consistency further suggests that incorporating linguistics theory into prompt design ensures greater stability in translation quality when large language models process Japanese sentences with complex attributive clause structures.

Practical applications

1. Meeting user demand for high-quality translations in machine translation tools: By integrating linguistics theory into prompt design, this study optimized prompt combinations, resulting in translations that are more natural and fluent when large language models handle Japanese-to-Chinese translation tasks involving complex "attributive clause + modified noun" structures. These optimized translations reduce instances of mistranslation and over-translation, allowing users to access more accurate, high-quality translations, thereby minimizing the cognitive effort required to comprehend machine-generated texts.
2. Enhancing human-machine collaborative translation efficiency: In human-machine translation workflows, the quality of machine translations directly impacts the amount of post-editing required by human translators. The findings of this study significantly improve the quality of translations for Japanese-to-Chinese tasks, particularly for sentences with "attributive clause + modified noun" structures. This improvement reduces the time and effort required to correct errors such as omissions, mistranslations, and awkward sentence constructions. Consequently, human translators can focus more on higher-level tasks, such as refining linguistic style, ultimately enhancing overall translation efficiency.
3. Supporting Japanese learners in mastering complex sentence structures and improving language education: The optimized prompt combinations can be valuable tools for intermediate and beginner-level learners of Japanese, helping them better understand complex sentence structures. This not only aids learners in grasping the nuances of Japanese grammar but also enhances the effectiveness of language education, contributing to improved learning outcomes.

Theoretical contributions

This study highlights the immense potential of combining linguistics theory with large language models in machine translation. This approach not only improves translation quality but also introduces an innovative framework for addressing the interpretability of large language models, often referred to as "black boxes." The proposed framework offers new insights into how machines process multilayered linguistic structures, providing a novel avenue for research in both Japanese linguistics and Japanese-Chinese translation studies.

Unlike previous studies, which either focused solely on linguistic or translation theory or explored how to improve large language models through the optimization of training data without incorporating linguistics theory or providing interpretability, this research uniquely combines linguistics theory with prompt design for large language models. This innovative approach not only results in significant improvements in translation quality but also introduces interpretability into machine translation systems, offering a fresh and valuable contribution to the field.