A COMPARATIVE ANALYSIS OF THE RELIABILITY OF CHATGPT AND HUMAN TRANSLATION WITH REFERENCE TO LITERARY TEXTS

Abstract

The study entitled, A comparative Analysis of the Reliability of ChatGPT and Human Translation with Reference to Literary Texts, aimed to assess the reliability of ChatGPT and human translation with respect to literary texts. It explores the effectiveness of both translation procedures in terms of accuracy, fluency, coherence, cultural relevance and general acceptability. The study is descriptive in nature and was conducted with mixed-method approach, which is a combination of qualitative and quantitative methods. This study has also taken into account a corpus of three literary texts. The source texts were translated by ChatGPT and compared with human translations. The identification and categorization of translation errors were based on a modified Multidimensional Quality Metrics (MQM) framework for the quantitative analysis, as well as the evaluators’ assessment of the translation acceptance provided by a ranking questionnaire. Descriptive statistical evaluations were used to compare the acceptance of the two translation techniques. The qualitative analysis was based on feedback provided by six evaluators as well. The results indicate that both the human and ChatGPT translations were of acceptance in most cases. However, the human translations were always deemed more acceptable, and showed higher inter-rater agreement. It has been found that Human translation achieved a mean score of 8.42, compared with 7.89 for CHT, showing that human translation was significantly better in the translation of literary texts in terms of better preserving style, rhetorical devices, emotional nuances and cultural embedded meanings. While the translation capability of the ChatGPT was effective, it had limitations when it came to stylistic and cultural nuances. The study concludes that ChatGPT has considerable potential to support professional translation tasks but cannot replace human translators in tasks that require creativity, cultural understanding and literary subtlety. The results are a contribution to the emerging field of AI-assisted translation, and they provide tangible benefits for translators, educators and scholars.