Reasoning about In-Context Samples for Machine-Translation
cs.CL
Submitted: 2026-08-27
Updated: 2026-08-27
Code: https://github.com/Maxwell1447/Fragment-Based-Reasoning
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large Language Models (LLMs) can be trained to perform chain-of-thoughts reasoning in order to improve the reliability of their responses.
Terminology
Abstract
Large Language Models (LLMs) can be trained to perform chain-of-thoughts reasoning in order to improve the reliability of their responses. In this work, we investigate how explicit reasoning can be leveraged for LLM-Based Machine Translation (MT) with in-context samples. We introduce a novel fragment-based reasoning framework in which the model first extracts parallel source-target fragments from retrieved similar exemplars, and uses these fragments as intermediate reasoning traces to produce the final translation. To train our model, we distill silver fragments and drafts from a large teacher model. Our experiments with the Qwen3 model family, over 6 languages, including up to 5 domains per language, demonstrate that fragment-based MT significantly outperforms alternative methods like standard k-shot or basic drafting.
Sources
- OpusCleaner and OpusTrainer, open source toolkits for training Machine Translation and Large language models
- Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysis
- Training Verifiers to Solve Math Word Problems
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning
- TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinement
- R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning
- How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
- Nearest Neighbor Non-autoregressive Text Generation
- Unlocking Reasoning Capability on Machine Translation in Large Language Models
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Prompting PaLM for Translation: Assessing Strategies and Performance
- Qwen3 Technical Report
- LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens
- OPT: Open Pre-trained Transformer Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering