Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer

arXiv:2606.30943 · cs.CL · Submitted 2026-06-29 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer".

Jane: The paper was written by M. K. Arabov from Kazan Federal University and Institute of Computational Mathematics and Information Technologies and Department of Data Analysis.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: Jane, you have to listen to this new title we just pulled from arXiv. It is called "Bridging Scientific Heritage: An Arabic–Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer."

Jane: That sounds like a massive project, Tom, but I love the idea of "bridging" something as important as scientific heritage.

Tom: It really is, especially since it's written by M. K. Arabov from Kazan Federal University in Russia. He is looking at how we can actually move research between Arabic and Russian speakers without the language barrier getting in the way.

Jane: I think that's such a beautiful way to frame it because both of these cultures have such deep histories in fields like mathematics and astronomy.

Lu: It makes me incredibly excited because if this works, we could see a sudden surge in global collaboration. Imagine a researcher in the Gulf studying renewable energy suddenly being able to read Russian breakthroughs in physics instantly.

Meng: I can see the potential, Lu, but I wonder about the actual data requirements to make that happen. Arabov seems to be focusing on building the specific foundation needed for that kind of technical exchange rather than just hoping a general model works.

Jane: That's a great point, Meng, because you can't build a bridge if you don't have the right materials to start with.

Lalam: This research really speaks to how we can use technology to keep our collective human intelligence connected. By making these two scientific worlds more accessible to each other, we are helping ensure that cultural and intellectual legacies aren't lost just because of a language gap.

Tom: It feels like he is setting the stage for a much larger shift in how scientists talk to one another.

Jane: We should see exactly what he used to build that foundation, so let's look at how this corpus was actually put together.

Summary: Jane: Now that we know the goal, we need to talk about the actual work Arabov did in "Bridging Scientific Heritage: An Arabic–Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer."

Tom: He didn't just write a theory; he actually constructed a hybrid parallel corpus with about twenty-seven thousand sentence pairs.

Jane: And it isn't just strictly technical stuff, is it?

Tom: No, that's the clever part because he mixed in scientific abstracts with general sources like news and even conversations.

Lu: That mixture is so smart because it helps the AI understand how to be precise with a medical term while still sounding natural in a sentence.

Meng: I was reading about his model selection too, and he tested three different architectures: mT5, NLLB, and Qwen. He wanted to see if a smaller model could handle the load or if we really needed those seven-billion parameter models to get anything useful done.

Jane: It sounds like he was searching for the right balance between being powerful and being efficient.

Meng: He definitely found a limit, because the results showed that smaller models like the five hundred eighty-million parameter mT5 basically couldn't produce meaningful translations for this specific pair.

Lalam: This focus on diverse data is so important for creating an inclusive digital landscape. By including everything from religion to news, he is making sure the AI understands the nuance of how people actually communicate across different domains.

Tom: It's a huge amount of work to curate and organize twenty-seven thousand high-quality examples like that.

Jane: It really is, and now we have to see if all that training actually translated into better performance.

Improvements: Tom: We are finally getting to the results, specifically how much these models improved once they were fine-tuned on this new data.

Jane: The jump in quality was huge, especially for the Qwen2 point 5 model.

Tom: It was impressive! Using a technique called QLoRA, the Qwen model hit a BLEU score of twenty-three point one five, which is a massive step up from how it performed without any training at all.

Jane: For those of us who aren't engineers, LoRA is basically like giving an existing AI a specialized textbook so it can learn a new skill without having to relearn everything it already knows.

Lu: And because he used QLoRA, this is so much faster and cheaper than full training! I can see this being used to create tiny "expert" models for every single scientific niche, from aerospace to botany.

Meng: That efficiency is what caught my eye, because they were able to do this on much more modest hardware. You don't need a massive supercomputer cluster to run this kind of fine-tuning; you can actually do it on consumer-grade gear.

Tom: But there was a bit of a reality check in the findings, wasn't there?

Jane: There was, because they tried "few-shot prompting"—where you just give the AI a few examples in its chat window—and it didn't help at all.

Lalam: That really proves that for specialized science, you can't just give an AI a few hints and expect it to be an expert. You have to deeply embed that knowledge through actual fine-tuning so the model truly understands the technical language.

Meng: It's a good reminder that prompting isn't a magic wand for everything.

Tom: It really shows that real domain expertise requires real, dedicated training.

Jane: Now that we've seen the numbers and the methods, let's wrap this all up and look at what this means for the world.

Conclusion: Tom: This has been a fascinating look at "Bridging Scientific Heritage: An Arabic–Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer."

Jane: It really highlights how much effort goes into making sure scientific knowledge can flow freely between different cultures.

Lu: I see a future where no discovery is ever stuck behind a language barrier because we have these specialized linguistic bridges ready to go.

Meng: And from an engineering standpoint, the fact that this is so efficient means actual research labs can start using these tools today without needing a massive budget.

Lalam: This work supports the idea of global partnerships, helping us meet sustainability goals by making sure every researcher has access to the world's collective intelligence.

Tom: It's been such an inspiring conversation, and I think we've all learned a lot about how much potential there is in these specialized models.

Jane: We are definitely leaving this session feeling excited about the future of scientific collaboration.

Tom: Thanks to everyone for joining us today; we'll be back after the break with a look at how AI handles spatial reasoning in three dee environments.

Jane: Stay tuned, we'll be right back!

Kazan Federal University · Institute of Computational Mathematics and Information Technologies · Department of Data Analysis

cs.CL

Submitted: 2026-06-29

Updated: 2026-09-13

Comments: 21 pages, 12 tables, 1 figure. Preprint

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 79/100

The gist: This paper presents a benchmark for Arabic–Russian scientific translation, addressing a "language barrier" that "impedes the exchange of research results" between these two major scientific

Key concepts

Parallel Corpus: A dataset containing paired sentences in two different languages. This research used a hybrid corpus of 27,000 Arabic–Russian pairs, combining scientific abstracts with news and conversations to help AI models master both technical precision and natural language flow across different domains.
QLoRA: An efficient fine-tuning technique that allows large language models to learn new skills without full retraining. Because it is faster and cheaper, it enables researchers to create specialized "expert" models on modest, consumer-grade hardware rather than requiring massive supercomputer clusters.
Few-shot Prompting: A method of giving an AI a few examples within a prompt to guide its output. The study found this approach failed for specialized scientific translation, demonstrating that deep knowledge must be embedded through dedicated fine-tuning rather than just providing a few hints.

Terminology

Fine-Tuning — 39× in this paper · explained in 5 episodes
Benchmark — 30× in this paper · explained in 2 episodes
Adaptation — 10× in this paper · explained in 1 episode
Parameter-Efficient Fine-Tuning — 7× in this paper · explained in 2 episodes
LoRA Fine-Tuning — 5× in this paper · explained in 3 episodes
Low-Rank Adaptation (LoRA) — 4× in this paper · explained in 9 episodes
Transformer — 4× in this paper · explained in 2 episodes
Accuracy — 3× in this paper · explained in 1 episode
Adaptability — 3× in this paper · explained in 1 episode
Quantization — 3× in this paper · explained in 12 episodes
Semantic Similarity — 3× in this paper · explained in 2 episodes
Benchmarking — 2× in this paper · explained in 4 episodes
Embeddings — 2× in this paper · explained in 2 episodes
Hallucination — 2× in this paper · explained in 4 episodes
Hybrid Model — 2× in this paper · explained in 2 episodes
International Conference on Learning Representations — 2× in this paper
Low-Resource Languages — 2× in this paper · explained in 4 episodes
No Language Left Behind — 2× in this paper
Cooperation — 1× in this paper · explained in 2 episodes
Distillation — 1× in this paper · explained in 2 episodes
Domain Adaptation — 1× in this paper · explained in 5 episodes
Human-in-the-Loop — 1× in this paper · explained in 3 episodes
In-Context Learning — 1× in this paper · explained in 4 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
Open-Source Arabic Corpora and Processing Tools — 1× in this paper
Resource Constraints — 1× in this paper · explained in 3 episodes
Transferability — 1× in this paper · explained in 4 episodes
alpha — 1× in this paper · explained in 1 episode

Summary

This paper presents a benchmark for Arabic–Russian scientific translation, addressing a language barrier that impedes the exchange of research results between these two major scientific communities. By providing a new hybrid parallel corpus and evaluating various large language models (LLMs), the work aims to facilitate sustainable knowledge transfer in critical fields such as renewable energy and water management.

The Dataset Construction

The researchers constructed a hybrid parallel corpus of approximately 27,000 training examples to ensure both domain adaptability and linguistic diversity. This corpus combines specialized scientific data with general-domain text to provide a robust foundation for training. The scientific portion was derived from medical and technical translation pairs, while the general-domain portion was curated from several sources to maintain natural language fluency.

The final training set includes:

A scientific corpus containing 15,000 entries from sources such as PEACH (Healthcare), MeSpEn Glossaries, Misraj/Tarjama-25, and NAMAA-Space/ASCAT.
A general-domain subset consisting of up to 3,000 examples each from:
Religion
Bible
Dictionary (filtered for a minimum of two Russian words)
Tatoeba
News
Conversation

Model Selection and Training

The study evaluates three multilingual LLMs across different architectures and size regimes to identify the minimum model capacity requirement for scientific translation. The researchers employed parameter-efficient fine-tuning techniques, specifically Low-Rank Adaptation (LoRA) and its 4-bit quantized variant, QLoRA, to adapt the models.

The experimental framework utilized:

mT5-base (580M), a compact encoder-decoder model.
NLLB-200-distilled-1.3B (1.3B), a mid-sized encoder-decoder model.
Qwen2.5-7B-Instruct (7B), an instruction-tuned decoder-only model.

The researchers systematically tested LoRA ranks of 8, 16, 32, and 64 to determine their impact on translation quality across these distinct architectures.

Evaluation and Results

Performance was measured using a rigorous comparative analysis involving four metrics: BLEU, chrF, BERTScore, and COMET. The findings indicate that model capacity is a primary determinant of translation quality, with the 7B Qwen model outperforming others. Specifically, the Qwen2.5-7B model fine-tuned with QLoRA (rank 8) achieved a BLEU score of 23.15 and a COMET score of 0.758, which was +4.36 BLEU and +0.051 COMET above the zero-shot baseline.

Key observations from the results include:

A minimum model capacity threshold is necessary, as models below 1B parameters perform poorly in zero-shot settings.
Few-shot prompting with three examples does not improve performance, suggesting that domain-specific fine-tuning is required.
For smaller models like mT5, increasing LoRA rank leads to monotonic improvement, whereas larger models like NLLB show diminishing returns after rank 32.
All models exhibited a consistent compression effect, producing translations that were shorter than the reference texts.

Broader Implications

By enabling high-quality, automated translation of scientific content, this work addresses a knowledge transfer barrier between Arabic-speaking and Russian-speaking research communities. This facilitates bidirectional knowledge flow in areas like climate resilience and healthcare, contributing to sustainable partnerships (UN SDG 17) and innovation infrastructure (SDG 9).

Improvements for AI systems

1. Architecture Transition: Decoder-Only Instruction-Tuned Models

Improvement: Replace traditional encoder-decoder architectures (e.g., NLLB, mT5) with decoder-only, instruction-tuned architectures (e.g., Qwen-series) for Arabic–Russian scientific translation.
Capability: The system will achieve higher translation quality (higher BLEU and COMET scores) by leveraging superior instruction-following and larger parameter capacities for complex linguistic mapping.

2. Parameter Capacity Thresholding

Improvement: Enforce a minimum model capacity requirement of 1B parameters for all Arabic–Russian translation tasks.
Capability: The system will avoid the catastrophic failure in semantic coherence and morphological handling observed in sub-1B parameter models (like mT5-base).

3. Dynamic LoRA Rank Allocation

Improvement: Implement a size-dependent Low-Rank Adaptation (LoRA) strategy: utilize Rank 32 for models 1B parameters and Rank 64 for models <1B parameters.
Capability: The system will optimize the trade-off between computational efficiency and translation accuracy, preventing the diminishing returns seen when over-parameterizing larger models.

4. Hybrid Domain Data Composition

Improvement: Construct training pipelines using a hybrid corpus ratio of approximately 75% domain-specific scientific abstracts and 25% diverse general-domain data (news, religion, and conversation).
Capability: The system will maintain high-precision technical terminology accuracy while preserving natural language fluency and linguistic diversity.

5. Length-Aware Decoding Integration

Improvement: Incorporate length-penalty adjustments in beam search decoding or length-normalized loss functions to mitigate the observed-2.1 to-3.1 token compression bias.
Capability: The system will produce translations that match the structural completeness and information density of the source text, rather than defaulting to overly concise, truncated outputs.

6. PEFT-Centric Deployment (Replacing Few-Shot Prompting)

Improvement: Deprecate in-context learning (few-shot prompting) in favor of Parameter-Efficient Fine-Tuning (PEFT) using 4-bit quantized LoRA (QLoRA).
Capability: The system will eliminate the performance degradation and bias introduced by few-shot attempts, ensuring the model is explicitly adapted to the specific morphological and syntactic requirements of the Arabic–Russian scientific domain.