Translating ambiguity: A comparative evaluation of AI Chatbots in rendering Arabic full homonyms into English
Main Article Content
Abstract
Homonymy is one of the most persistent challenges in translation, particularly for AI-based systems. This study examines the abilities of three prominent AI chatbots, ChatGPT 5, DeepSeek (V3.1), and Gemini (2.5 Flash), in the English translation of Arabic homonyms. The dataset comprised Qur’anic verses and classical poetry. The homonyms were first collected, then analyzed for contextual meaning, and then translated by the chatbots. The data were presented without diacritics to replicate the conditions of authentic Arabic text. After the items were translated, the data were assessed using the TAUS Adequacy/Fluency model to determine the extent to which the systems preserved meaning while maintaining linguistic naturalness. The results showed that while all chatbots often achieved high fluency, their adequacy slightly differed across text types and homonym categories. Gemini outperformed the other models overall, followed by ChatGPT and DeepSeek. The Qur’anic translation performed best due to the high prevalence of such texts in online corpora. However, the performance declined noticeably in the poetic and classical domains. While large language models (LLMs) represent a significant improvement over traditional machine translation (MT) systems in handling lexical ambiguity, they remain unable to reliably disambiguate complex Arabic homonyms without contextual or cultural grounding. The study concludes that AI systems require substantially more training, particularly in Arabic-language contexts. The study therefore recommended further research on homonym translation, focusing on different contexts as well as the rendition of homonyms outside of pairings.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This open-access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.
You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially. The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
No additional restrictions You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
How to Cite
Share
References
References
Abdelmughni, H. M. A. (2025). Artificial intelligence in Arabic–English literary translation: Technological innovations, linguistic accuracy, and ethical implications. International Journal of Language and Literary Studies, 7(3), 107–129. https://doi.org/10.36892/ijlls.v7i3.2120
Abul-Saud, M. (2010). Irshad al-‘aql al-salim ila mazaya al-kitab al-karim [Guiding the sound mind to the merits of the Noble Book]. Dar Alkutub Alilmiyya.
Aburahme, M. (2021). The life of the Arabic language, its merit, and means of its development in the thought of Muhammad al-Khadr Hussein. Jordan Journal of Applied Science–Humanities Series, 27(2), 110–121. https://doi.org/10.35192/jjoas-h.v27i2.8
Abu Romman, O. (2025). Conceptual metaphor theory in cognitive linguistics. Jordan Journal of Applied Science–Humanities Series, 43(2), 127–142. https://doi.org/10.35192/jjoas-h.v43i2.1182
Al-Abbas, L. S., & Haider, A. S. (2020). Evaluating the accuracy and consistency in rendering Quranic terms with overlapping meanings into English. AlBayan: Journal of Qur’an and Hadith Studies, 18(2), 111–137.
Al-Aloussi, M. (1995). Rouh al-ma‘ani fi tafsir al-Qur’an al-‘atheem wa al-sab‘ al-mathani [The spirit of meanings in the exegesis of the Great Qur’an and the seven oft-repeated verses]. Dar Alkutub Alilmiyya.
Al-Darabee, M., Farghal, M., & Haider, A. S. (2025). Netflix versus Google Translate: A case study of the English–Arabic translation of scatological terms. Studies in Linguistics, Culture and FLT, 13(2), 77–95.
Aldawsari, H. A. H. (2023). Comparing the performance of Google Translate and SYSTRAN on Arabic lexical ambiguity. Arab World English Journal for Translation & Literary Studies, 7(3), 19–34. https://doi.org/10.24093/awejtls/vol7no3.2
Aldawsari, H. A. H. (2025). Evaluating the performance of large language models on Arabic lexical ambiguities: A comparative study with traditional machine translation systems. World Journal of English Language, 15(3), 354–363. https://doi.org/10.5430/wjel.v15n3p354
Al-Ezzi, R., & Al-Qudah, I. (2024). English translation of verbal humour in Egyptian comedy films. World Journal of English Language, 14(1), 157–157. https://doi.org/10.5430/wjel.v14n1p157
Ali, A. Q. (2022). The challenges of translation of homonym [Unpublished bachelor’s thesis]. University of Diyala.
Ali, E. A. M. (2020). Lexico-semantic issues on translating homonymy and polysemy: A comparative study based on the Noble Qur’an. QUR’ANICA: International Journal of Qur’anic Research, 12(1), 45–68. https://doi.org/10.22452/quranica.vol12no1.3
Ali, M. A., Brakhw, M. A., & Nordin, M. Z. F. (2014). Transferring polysemic words from Arabic into English: A comparative study of some samples from the Holy Qur’an. Australian Journal of Basic and Applied Sciences, 8(23), 38–43.
Al-Jarf, R. (2025). AI translation of full-text Arabic research articles: The case of educational polysemes. Journal of Computer Science and Technology Studies, 7(1), 311–325. https://doi.org/10.32996/jcsts.2025.7.1.23
Alkarazoun, G., & Riziq, D. (2025). A syntactic and pragmatic analysis of the colloquial expression ʔinno ‘that’ in Jordanian Arabic: Evidence from social media conversation. Languages, 10(9), Article 205. https://doi.org/10.3390/languages10090205
Al-Qudah, I. (2022). Censorship as enabling: Importing, distributing, and translating foreign films in the Arab Middle East. Heliyon, 8(8), Article e09977. https://doi.org/10.1016/j.heliyon.2022.e09977
Alqudah, I. (2025). Who censors what and how? A linguistic inquiry into censorship in Arab audiovisual media. Forum for Linguistic Studies, 7(7), 402–413. https://doi.org/10.30564/fls.v7i7.9990
Al-Salman, S., & Haider, A. S. (2024). Assessing the accuracy of MT and AI tools in translating humanities or social sciences Arabic research titles into English: Evidence from Google Translate, Gemini, and ChatGPT. International Journal of Data and Network Science, 8(4), 2483–2498. https://doi.org/10.5267/j.ijdns.2024.5.009
Alsharif, B., & Khasawneh, R. (2025). Beyond the literal: Machine translation performance and strategies in rendering audiovisual political idioms. Research Journal in Advanced Humanities, 6(2). https://doi.org/10.58256/4442fg06
Al-Zamakhshari, M. (1987). Al-kashaf ‘an ghawamidh al-tanzeel [The unveiler of the obscurities of revelation]. Dar Alkitab Alarabi.
Ammari, R., & Al-Ahmad, W. (2023). A corpus-based study of English synonyms: Small, little, tiny, and petite. Jordan Journal of Applied Science–Humanities Series, 36(2), 1–20. https://doi.org/10.35192/jjoas-h.v36i2.542
Bin Hariz, M. A. (2023). Problems of translating synonyms and homonyms in the Holy Qur’an: A comparative study of two Qur’anic translations by Yusuf Ali and Ahmed Ali. Electronic Journal of University of Aden for Humanities and Social Sciences, 4(1), 129–136. https://doi.org/10.47372/ejua-hs.2023.1.239
Bououden, R. (2025). Evaluating Arabic–English neural machine translation: Challenges across different text types. Journal of Languages and Translation, 5(Special Issue), 59–73.
Farghal, M., & Haider, A. S. (2024). Translating classical Arabic verse: Human translation vs. AI large language models (Gemini and ChatGPT). Cogent Social Sciences, 10(1), Article 2410998. https://doi.org/10.1080/23311886.2024.2410998
Farghal, M., & Haider, A. S. (2025). A cogno-prosodic approach to translating Arabic poetry into English: Human vs. machine. 3L: Language, Linguistics, Literature, 31(1), 255–271.
Fraihat, B. (2024). The effect of implementing a remedial plan based on reading strategies on improving the reading comprehension achievement of Jordanian EFL eleventh-grade students in the vocational stream. Jordan Journal of Applied Science–Humanities Series, 39(1), 18–26. https://doi.org/10.35192/jjoas-h.v39i1.729
Garcia, M. (2021). Exploring the representation of word meanings in context: A case study on homonymy and synonymy. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.acl-long.281
Haider, A. S. (2019a). The representation of Al-Megrahi’s release in Arabic and English newspapers in 2009 and 2010: A corpus-assisted discourse study. Dirasat: Human and Social Sciences, 46(1, Suppl. 2), 297–317. https://doi.org/10.35516/0103-046-986-019
Haider, A. S. (2019b). Syrian–Lebanese relations: A corpus-based critical discourse analysis of Bashar Al-Assad’s speeches and interviews. Dirasat: Human and Social Sciences, 46(4), 551–570. https://doi.org/10.35516/0103-046-004-040
Haider, A. S. (2019c). Using corpus linguistic techniques in (critical) discourse studies reduces but does not remove bias: Evidence from an Arabic corpus about refugees. Poznan Studies in Contemporary Linguistics, 55(1), 89–133. https://doi.org/10.1515/psicl-2019-0004
Haider, A. S. (2026a). AI as a language tutor: Jordanian university students’ perceptions of AI-generated feedback on their English. Research Journal in Advanced Humanities, 7(4), 1–17.
Haider, A. S. (2026b). Trust in AI translation: Reception and adoption among Jordanian business professionals. Research Journal in Advanced Humanities, 7(4), 1–15.
Haider, A. S., Akar, R. A., Saed, H., Tair, S. A., & Ismail, H. H. (2025). AI-powered Arabic translation of business and finance acronyms and abbreviations: Evidence from Gemini, ChatGPT, and Google Translate. In Innovation, technologies, and business management (pp. 519–528). Springer. https://doi.org/10.1007/978-3-031-84889-6_62
Haider, A. S., & Alkhatib, R. (2024). Subtitling English legal acronyms into Arabic: Human vs machine. Kutafin Law Review, 11(4), 810–842.
Haider, A. S., Alzghoul, O., & Hamadan, Y. (2023). Creating and experimenting a new parallel corpus of English–Arabic subtitles of culinary shows: A useful guide for translating food across cultures. World Journal of English Language, 13(8), 358–372. https://doi.org/10.5430/wjel.v13n8p358
Haider, A. S., & Hussein, R. F. (2022). Modern Standard Arabic as a means of euphemism: A case study of the MSA intralingual subtitling of Jinn series. Journal of Intercultural Communication Research, 51(6), 628–643. https://doi.org/10.1080/17475759.2022.2106289
Haider, A. S., Obeidat, M. M., Hamdan, Y., Abu-Rayyash, H., & AlZghoul, O. (2025). SDL Trados Live Cloud vs. traditional subtitling workflows: An empirical evaluation of performance. Research Journal in Advanced Humanities, 6(4). https://doi.org/10.58256/fc624p07
Haider, A. S., & Shohaibar, R. S. (2024). Netflix English subtitling of idioms in Egyptian movies: Challenges and strategies. Humanities and Social Sciences Communications, 11, 1–13. https://doi.org/10.1057/s41599-024-03327-4
Hammouri, D. M. (2025). The role of artificial intelligence in generating Arabic neologisms: A study of gaming localization. Research Journal in Advanced Humanities, 6(4), 1–15.
Hauer, B., & Kondrak, G. (2020). One homonym per translation. Proceedings of the AAAI Conference on Artificial Intelligence, 34(5). https://doi.org/10.1609/aaai.v34i05.6296
Holes, C. (2004). Modern Arabic: Structures, functions, and varieties. Georgetown University Press.
Ibn Ashour, M. (1984). Al-tahreer wa al-tanweer [Verification and enlightenment]. Aldar Altunisiyya Linnashr.
Ja’afreh, N. (2023). A pragmatic contrastive analysis of apology strategies in Jordanian Arabic and English language. Jordan Journal of Applied Science–Humanities Series, 37(2), 69–86. https://doi.org/10.35192/jjoas-h.v37i2.601
Jahameh, S., & Al-Jazzar, F. (2025). The technological impacts of graphic design: Visual adaptations of English borrowed terminology in Arabic—A case study of Arabic typography. Jordan Journal of Applied Science–Humanities Series, 44(2), 154–173. https://doi.org/10.35192/jjoas-h.v44i2.1853
Jaradat, A., Hammouri, D. M., Gwasmeh, M., & Haider, A. S. (2026). Reflexivity and reciprocity in two Arabic varieties: Evidence for REF-REC category. Languages, 11(2), Article 26. https://doi.org/10.3390/languages11020026
Khasawneh, R., & Alsharif, B. (2025). Translating idiomatic expressions: A systematic review. Forum for Linguistic Studies, 7(10), 881–893. https://doi.org/10.30564/fls.v7i10.11172
Khatatbah, D., & Ashour, M. (2024). The role of school administration in activating the use of information technology at schools in Irbid Governorate. Jordan Journal of Applied Science–Humanities Series, 38(2), 121–137. https://doi.org/10.35192/jjoas-h.v38i2.718
Khudjaeva, R. N. M. (2023). Conceptual analysis of homonyms translation in English fiction literature. Journal of Modern Educational Achievements, 12(12), 318–322.
Klepousniotou, E. (2002). The processing of lexical ambiguity: Homonymy and polysemy in the mental lexicon. Brain and Language, 81(1–3), 205–223. https://doi.org/10.1006/brln.2001.2518
Li, S., Chen, J., Yuan, S., Wu, X., Yang, H., Tao, S., & Xiao, Y. (2024). Translate meanings, not just words: IdiomKB’s role in optimizing idiomatic translation with language models. Proceedings of the AAAI Conference on Artificial Intelligence, 38.
Meqdad, S., Al-Bayyari, M., & Al-Taher, M. (2023). Semantic disparity between Amr (command) and Nahy (negative command) in the Noble Quran: An interpretive/contextual study. Jordan Journal of Applied Science–Humanities Series, 36(1), 1–10. https://doi.org/10.35192/jjoas-h.v36i1.516
Naib, A. (2025). Algerian viewers’ attitudes toward taboo language: A case study of Mourad Oudia’s humorous YouTube content. Jordan Journal of Applied Science–Humanities Series, 44(1), 167–191. https://doi.org/10.35192/jjoas-h.v44i1.1830
Obeidat, M. M., Haider, A. S., Tair, S. A., & Sahari, Y. (2024). Analyzing the performance of Gemini, ChatGPT, and Google Translate in rendering English idioms into Arabic. FWU Journal of Social Sciences, 18(4), 1–18. https://doi.org/10.51709/19951272/Winter2024/1
O’Brien, S. (2012). Towards a dynamic quality evaluation model for translation. The Journal of Specialised Translation, 17, 56–77.
Rabadi, R. I. (2015). Homonymy and its effect on students of translation at Jordanian universities. Arab World English Journal, 6(4), 26–38. https://doi.org/10.2139/ssrn.2843923
Ramadhan, H. S. (2023). Polysemy in English and Arabic: A contrastive study. The Middle East International Journal for Social Sciences, 5(1), 48–58. https://meijss.org/wp-content/uploads/2023/08/MEIJSS-4858.pdf
Riziq, D., & Hamdan, J. (2023). The semantic structure of motion verbs in Jordanian Arabic: Talmy’s typology revisited. Italian Journal of Linguistics, 35(2), 143–170. https://doi.org/10.26346/1120-2726-196
Ryding, K. C. (2005). A reference grammar of Modern Standard Arabic. Cambridge University Press.
Tannous, B., & Haider, A. S. (2025). Assessing the accuracy of AI tools (Google Translate and Gemini) in translating Arabic idiomatic expressions and proverbs into English. International Journal on Artificial Intelligence Tools, 34(4–5), Article 2550016.
Versteegh, K. (1997). The Arabic language. Edinburgh University Press.
Wael, T., Elrefai, E., Makram, M., Selim, S., & Khoriba, G. (2024). Pirates at ArabicNLU2024: Enhancing Arabic word sense disambiguation using transformer-based approaches. In Proceedings of the Second Arabic Natural Language Processing Conference (pp. 372–376). Association for Computational Linguistics.
Wright, W. (2011). A grammar of the Arabic language. Cambridge University Press.
Yang, C., & Zhang, Y. (2024). The application of Chinese–English machine translation in cultural terminology translation: A case study of homographs and polysemous words. In 2024 7th International Conference on Artificial Intelligence and Big Data (ICAIBD) (pp. 226–231). IEEE. https://doi.org/10.1109/ICAIBD62003.2024.10604629