Improving RAG Performance with Translation-Based Embeddings for Non-English Documents

Although multilingual embedding models are advancing rapidly, they still perform more weakly on non-English text than on English. This gap limits how successfully Retrieval-Augmented Generation (RAG) based question-answering systems built in languages other than English find the right context. The paper by Can İşcan, Muhammet Furkan Özara and Akhan Akbulut from the Next4biz R&D Center focuses on exactly this problem.

In the proposed method, documents are first translated into English and the embedding vectors are produced from that translation, while the text in the original language is stored as metadata. This makes it possible to benefit from strong English embedding models and still present the original content to the user. The team built a question-answering pipeline that uses GPT-4o for answer generation and compared the conventional approach with the translation-based approach across different embedding models within the RAGAS framework.

According to the initial findings, translation-based embedding outperforms the conventional method on context precision and context recall in non-English scenarios, and the approach can be adapted to different languages and translation models. As stated in our R&D team's project article, this approach is used in Next4biz chatbot systems that work with large language models such as OpenAI, LLaMA and Qwen.

The paper was presented at ASYU 2024 – the Innovations in Intelligent Systems and Applications Conference – held in Ankara on October 16–18, 2024 and hosted by Gazi University, and appeared in the conference proceedings published by IEEE (pp. 1–6).

Prof. Dr. Akhan Akbulut

Professor Doctor Akhan Akbulut worked in the Computer Engineering departments of Istanbul Kültür University and NC State University. He serves institutions such as TÜBİTAK, Ministry of Industry and Technology, TÜSEB, and KOSGEB. He researches Distributed Systems and Artificial Intelligence and has over 100 international journal articles and conference proceedings from his work.