Dr. Ming Qian

Proceedings of the 26th International Conference on Human-Computer Interaction (HCII 2024), Washington, DC (July 2024).

This study evaluates a novel human-machine collaborative machine  translation workflow, enhanced by Large Language Model features, including pre-editing instructions, interactive concept-based post-editing, and the archiving  of concepts in post-editing and translation memories. By implementing GPT-4  prompts for concept-based steering in English-to-Chinese translation, we explore its effectiveness compared to traditional machine translation methods such as  Google Translate, human translators, and an alternative human-machine collaboration approach that utilizes human-generated reference texts instead of concept  description. Our findings suggest that while GPT-4’s discourse-level analysis and augmented instructions show potential, they do not surpass the nuanced understanding of human translators at the sentence-level. However, GPT-4 augmented  concept-based interactive post-editing significantly outperforms both traditional  methods and the alternative method relying on human reference translations. In  testing English-to-Chinese translation concepts, GPT-4 effectively elucidates nearly all concepts, precisely identifies the relevance of concepts within source  texts, and accurately translates into target texts embodying the related concepts.  Nevertheless, some complex concepts require more sophisticated prompting  techniques, such as Chain-of-Thought, or pre-editing strategies, like explicating  linguistic patterns, to achieve optimal performance. Despite GPT-4’s capabilities,  human language experts possess superior abductive reasoning capabilities. Consequently, at the present stage, humans must apply abductive reasoning to create more specific instructions and develop additional logic steps in prompts, which  complicate the prompt engineering process. Eventually, enhanced abductive reasoning capabilities in large language models will bring their performance closer  to human-like levels. The proposed novel approach introduces a scalable, concept-based strategy that can be applied across multiple text segments, enhancing  machine translation workflow efficiency.

For More Information

To learn more or request a copy of a paper (if available), contact Ming Qian.

(Please include your name, address, organization, and the paper reference. Requests without this information will not be honored.)