ODE-Based Transformer Decoders for Iterative Sign Language Translation

Sign language translation has achieved strong results with Transformer architectures, yet recent improvements largely rely on scaling model capacity at the cost of increased computation. We propose a parameter-efficient alternative that improves expressiveness without increasing model size. Rather than scaling capacity, we focus on enhancing the update dynamics of iterative refinement decoders, where each refinement step corresponds to one internal decoder iteration that progressively improves the latent representation before translation generation. We reinterpret residual refinement updates from an Ordinary Differential Equation (ODE) perspective and replace them with higher-order numerical integration schemes, namely Runge–Kutta methods (RK-2 and RK-4). These methods perform multiple function evaluations within each refinement step to produce more accurate and stable representation updates without adding decoder parameters. To the best of our knowledge, this is the first application of ODE-inspired update dynamics to sign language translation. RK-2 achieves 22.96 BLEU-4 on the PHOENIX14T test set and 19.34 BLEU-4 on the CSL-Daily test set, outperforming the IPSLT baseline on both benchmarks, with fewer decoder layers and refinement iterations on CSL-Daily. These results suggest that stronger refinement dynamics can improve translation performance under parameter-efficient decoder designs, providing a complementary alternative to conventional model scaling.

Improvements to AI Systems:

1. Parameter-Efficient Decoder Enhancement via Higher-Order ODE Solvers

• Replace standard residual update rules in iterative refinement decoders with Runge–Kutta (RK-2, RK-4) integration schemes. This improves the stability and accuracy of latent representation updates without adding parameters, allowing smaller models to match or exceed the performance of larger baselines.

2. Dynamic Refinement Step Control

• Use the ODE perspective to adaptively decide the number of refinement iterations per sample (e.g., fewer steps on simpler inputs, more on complex ones). This reduces inference computation while maintaining translation quality, as demonstrated by CSL-Daily results with fewer iterations.

3. Generalizable Update Dynamics for Sequence-to-Sequence Tasks

• Apply the same RK-based refinement to other iterative decoders (e.g., speech translation, text-to-SQL, or image captioning) where progressive latent refinement is used. The method is task-agnostic and does not require architectural changes.

4. Improved Latent Representation Smoothness

• Higher-order integration reduces error accumulation across refinement steps, leading to more coherent and contextually accurate translations, especially for long or syntactically complex sentences.

What the Improved AI System Can Do:

• Achieve state-of-the-art sign language translation (e.g., BLEU-4 22.96 on PHOENIX14T) with a smaller model footprint, making it deployable on edge devices or real-time systems.

• Translate with fewer computational resources (fewer decoder layers and refinement iterations) while preserving or improving accuracy, enabling faster inference in live interpretation.

• Adapt to new languages or domains without retraining the full model—only the refinement dynamics are adjusted, reducing fine-tuning costs.

• Provide more stable and consistent translations across repeated runs, reducing variance in output quality for production systems.

Sign language translation has achieved strong results with Transformer architectures, yet recent improvements largely rely on scaling model capacity at the cost of increased computation. We propose a parameter-efficient alternative that improves expressiveness without increasing model size. Rather than scaling capacity, we focus on enhancing the update dynamics of iterative refinement decoders, where each refinement step corresponds to one internal decoder iteration that progressively improves the latent representation before translation generation. We reinterpret residual refinement updates from an Ordinary Differential Equation (ODE) perspective and replace them with higher-order numerical integration schemes, namely Runge--Kutta methods (RK-2 and RK-4). These methods perform multiple function evaluations within each refinement step to produce more accurate and stable representation updates without adding decoder parameters. To the best of our knowledge, this is the first application of ODE-inspired update dynamics to sign language translation. RK-2 achieves 22.96 BLEU-4 on the PHOENIX-2014-T test set and 19.34 BLEU-4 on the CSL-Daily test set, outperforming the IPSLT baseline on both benchmarks, with fewer decoder layers and refinement iterations on CSL-Daily. These results suggest that stronger refinement dynamics can improve translation performance under parameter-efficient decoder designs, providing a complementary alternative to conventional model scaling.