AI papers — 2026-10-15

Today we are diving into how we can stop language models from forgetting what they learned on old tasks while learning new ones because that catastrophic forgetting is a huge hurdle for continual adaptation. The most important thing I want to talk about is whether simpler methods of parameter interpolation can actually replace the complex rehearsal buffers and auxiliary objectives that have been the standard solution.

We looked at Sequential Fine-tuning with Averaging, or SFA, and compared it to replay-free Lookahead which was originally designed for optimization stability. In Qwen3-14B adapter continual learning, both fixed configurations of these methods outperformed the replay and adaptive-merging setups in terms of mean final accuracy while using up to thirty six point three percent less training time. This suggests that simple interpolation deserves serious consideration before we get bogged down in more complicated forgetting mitigation techniques.

Another piece of work that touches on how models learn structure is Draft-Thinking, which proposes a lightweight adaptation framework for learning a model-adapted draft chain of thought. This method learns a compact working derivation directly supporting the final answer, and it tackles the Cross-Scale Draft Mismatch by using progressive draft adaptation to calibrate reasoning granularity to the target model. This contrasts with methods that rely on post-hoc compression or explicit length penalties, as Draft-Thinking achieves substantial reductions in reasoning cost across six mathematical and scientific benchmarks.

Moving toward representation transfer, CORTEG investigates whether large pretrained foundation models can be adapted for cross-modality representation transfer from scalp to intracranial brain recordings. This framework combines a pretrained EEG FM backbone with an electrode-aware spatial adapter and a dual-stream tokenizer, achieving state-of-the-art mean performance on both public finger trajectory regression and private audio envelope regression. This work provides evidence that scalp EEG pretraining can be repurposed across different recording modalities for data efficient neural decoders.

Finally, we have some interesting insights into how attention mechanisms behave during generalization. Making Transformers Logical explores how to correct the systematic errors in generalization by injecting explicit structural priors directly into the attention mechanism via a learnable attention-logit bias initialized from task-level interaction structure. This mask-based initialization achieves near-perfect extrapolation on Boolean reasoning tasks, showing that attention masks can serve as a substrate for encoding persistent inductive bias.

The work on genesis steering is particularly important because it addresses how we can better control large language models during inference by using contrastive examples from the model's own training data to guide its behavior, which is crucial for ensuring safety and desired outputs. This technique relies on deriving a genesis vector from these internal examples, and when that data isn't available, they suggest using membership inference literature to create a proxy dataset. This contrasts with the work on equilibrium forcing, which tackles adaptive video generation without noise conditioning by focusing on how to generate coherent visuals based on some form of equilibrium.

The research into spatial reasoning in multimodal large language models is also significant because it seeks to improve the perception and reasoning capabilities of these models when they are used in embodied agents and robotics. This survey categorizes various fine-tuning methods, noting that dataset construction and evaluation approaches show substantial variation across different techniques like rules, prompting, or human-in-the-loop methods.

Furthermore, the development of MA-Bench is noteworthy as it provides a way to measure the operational planning ability of language models when faced with complex and contradictory information. This benchmark uses mission analysis principles to grade how well a model can turn messy documents into a plan, showing that models are strongest at stating facts but weakest at making inferences.

The work on xLightGCN offers an important contribution by providing an exact, graph-based method for generating explanations for decisions made by recommendation systems. This approach uses LightGCN like architectures to create interpretable explanations, which is vital for establishing trustworthiness in user-facing AI systems.

The work on DeLoop is particularly important because it tackles a fundamental problem in making language model agents capable of handling long tasks without getting stuck in repetitive cycles. We see this looping behavior naturally occurring in agents like Qwen, Llama, and Gemini, and this issue becomes much more pronounced when the agent's memory is squeezed into a limited context window. DeLoop proposes a self-supervised training method to combat these loops by using an agent's own repeated actions as supervision. This means it learns from seeing a loop as a negative example and then learning from the first subsequent non-looping action in that same sequence as a positive example, all through direct preference optimization without needing external rewards or judges.

This approach shows promise because on CodeGym under compression, DeLoop managed to cut the share of Qwen3-8B tasks containing loops from fifty-seven point three percent down to six point one percent while using about one third fewer steps per task and maintaining similar success rates. Furthermore, when applied to Qwen3-32B, DeLoop not only reduced both the loops and the number of steps but also boosted task success from thirty-seven point seven percent up to forty-four point eight percent. This demonstrates that this method can improve efficiency and performance across different model sizes for long tasks.

This idea of managing agent options is further explored in LAGO, which introduces a framework where specialized language agents are treated as reusable, temporally extended options rather than being locked into single monolithic tool-use sequences. A meta-agent can retrieve existing options from a persistent library or create new ones on the fly, dispatch sub-tasks, and admit novel successful options to that library without needing to update the model weights. This suggests that decomposition quality plays a role in how well these agents resolve complex tasks across different models.

This contrasts with the focus on safety, where AgentSafeEvo introduces an adversarial co-evolutionary reinforcement learning framework designed to strengthen LLM agent safety by pitting an attacker against a defender in a game spanning various tasks and attack types. This method specifically addresses how malicious content can manipulate agent decisions through user queries or tool descriptions.

Meanwhile, ELARSpeech is addressing a different kind of challenge by creating ELARSpeech, a massive multilingual speech resource containing thirty thousand two hundred hours of speech across six hundred seventy-five underrepresented languages. This resource aims to help extend Automatic Speech Recognition and Text-to-Speech capabilities into the linguistic long tail, providing both a training material and a testbed for further model development in these areas.

The most significant work here is about behavior transfer under model swapping in multi-session agents because it directly addresses how agents maintain continuity when they switch between different language models to manage performance versus cost. This is particularly important because if an agent loses its learned reasoning patterns during a model switch, the entire orchestration fails. The study by AKMPC found that recipients tend to adopt the reasoning and action patterns from the models that generated their histories, though this transfer is substantially reduced when summarization is used instead of raw history transfer.

This finding suggests that effective multi-model orchestration needs to consider not just which model to employ but also which behavioral patterns should be carried forward. This idea connects to the work by Articulation Upper Bounds for Language Models, though the abstract for that piece is not detailed enough to draw a direct link regarding agent behavior.

Then there is the memory-efficient subspace optimization with residual correction framework by Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon, which tackles the practical challenge of reducing LLM training memory costs. This method improves convergence rates for momentum SGD and is extended to Muon, showing that it maintains an O(T-1/4) convergence rate even with multi-stage updates.

In terms of learning from long sequences, the work by Long-Horizon RL Needs Muon examines how agents can coordinate reasoning and tool use over extended interactions, suggesting that using Muon in a single-rollout asynchronous recipe improves training success compared to baselines. This relates to the optimization concerns raised by When Does Momentum Help in Reinforcement Learning with Verifiable Rewards?, which explored how increasing the momentum coefficient beta can preserve nonvanishing gains when gradients are estimated from sampled responses in RLVR settings.

Finally, the severity-weighted character error rate introduced by Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCR provides a nuanced way to measure errors by having an LLM judge the severity of each edit across many languages, which offers a more granular view than standard character error rate metrics.

The work that matters most is Hindsight-Privileged RLVR because it tackles a fundamental problem in reinforcement learning with verifiable rewards: assigning credit to individual reasoning steps within a single rollout. This is crucial because current methods struggle to attribute success or failure at the token level, which limits our understanding of how language models actually arrive at correct answers. HP-RLVR solves this by using the trajectory's own future as a source for token-level updates, scoring each token based on its history and the realized outcome.

This approach is significant because it allows for bounded credit for both successful and failed attempts without needing a reference solution or a learned critic, which simplifies reward attribution. This method is faster than Group Relative Policy Optimization on mathematics and code generation tasks, achieving comparable overall accuracy while showing better performance in specific areas like mathematics.

ActiveEvidence also contributes to this field by focusing on the difficult decision of when to acquire new visual evidence in a multimodal agent setting. It introduces a Counterfactual Evidence Interaction Graph to represent evidence value and interaction harm, allowing the system to stop acquiring data once it has a high confidence certificate that further interaction won't change the outcome significantly. This is important because it makes evidence acquisition more efficient by evaluating only interactions that can displace the current choice.

Dynamic Graph World Modelling addresses how entities and their relationships evolve over time, which is vital for modeling complex environments where structure changes dynamically. DyG-WM proposes a framework that predicts what has evolved, how it evolved, and what will evolve next by using historical context encoders to summarize trajectories and path-signature encoders to capture structural changes.

Split the Differences Pool the Rest Provably Efficient Multi-Objective Imitation offers a way to recover policies that lie on the Pareto front when multiple experts provide conflicting demonstrations. MA-BC systematically partitions divergent expert data while pooling state action pairs where no conflict exists, proving it converges faster than independent learners and establishing a lower bound for multi-objective imitation learning.

Finally, Fully sequential Gaussian process regression for preference learning provides a fast closed-form update method for continuous global optimization problems where only relative performance between two points is observed. This technique uses a fully sequential sampling policy to control error accumulation, offering new convergence rates for approximate Bayesian inference in this setting.

The most significant work here concerns how energy consumption scales with model length, specifically comparing diffusion and autoregressive architectures. This matters because as large language models get bigger, their operational cost becomes a major bottleneck for deployment. The paper by Energy Consumption Analysis of Discrete Diffusion and Autoregressive Language Models shows that at longer sequences, diffusion models are substantially more energy efficient than autoregressive ones when considering the KV-cache overhead, achieving about two times the efficiency of AR models with KV-cache for sequences between two hundred and six hundred tokens.

This efficiency gain is important because it suggests a viable path toward scaling these powerful generative systems without incurring prohibitive energy costs. This finding contrasts sharply with earlier concerns about how different model types handle long context processing.

Another piece of research delves into the practical deployment challenges of real-time sensing systems, specifically Comparative Evaluation of Deep-Learning based DoA Estimators for Real-Time ISAC Beamforming, which evaluates various methods for estimating Direction-of-Arrival in Integrated Sensing and Communication systems. This work is crucial because it systematically shows that different estimation techniques occupy distinct points in the accuracy latency performance design space, indicating that a single best method does not exist across all operational needs.

Building on this practical assessment, the study Explain Like I'm 5 or Whatever I Choose: Evaluating the Potential for Interactive Complexity in Language Model Responses explores how interactive complexity affects language model responses, which speaks to a different kind of operational cost related to user engagement rather than pure inference energy.

Finally, we have work on the theoretical limits of neural network width, such as The minimal width of neural networks and immersions of real projective space, which establishes mathematical lower bounds on how wide a network must be to approximate certain continuous functions. This sets a fundamental constraint on the architecture itself, suggesting that achieving high performance might require specific structural properties related to topological embeddings.

The most important work here is the effort to make streaming inverse text normalization more efficient because it directly addresses the real-time needs of converting spoken audio into written text for applications like live transcription. This is tackled by adapting a pretrained text-to-text model to handle streaming, which means instead of relying on slow hybrid systems that mix neural tagging with fixed rules, they introduce architectural changes and a specific decoding policy to keep up with the flow of speech.

This adaptation is crucial because standard encoder-decoder models are not naturally suited for streaming due to their global attention mechanisms. The proposed system shows accuracy comparable to non-streaming methods on Vietnamese datasets while maintaining the low latency required for real-time use, which is a significant step forward from older techniques like those described in Towards Efficient Simultaneous Inverse Text Normalization with Pretrained Text-to-Text Language Model and Read-Tag-Write Policy.

Building on this idea of leveraging large models, DualView presents a parameter efficient way to use LLMs for multioutput text regression, which is important because it shows that smaller models can sometimes outperform unsupervised methods when using specific evidence from the LLM. This method extracts both broad and output specific representations from a frozen model and uses a dual-branch structure to generate the full score vector, requiring significantly fewer parameters than stronger baselines across various tasks.

Another area of concern is how prompts affect model performance, which is explored by RCCM, which examines how reasoning prompts can help one deployment while hurting another. This work suggests that simply matching a reference answer does not guarantee delivery of the correct output and that prompt benefits must be viewed alongside execution costs, as shown in RCCM: Diagnosing Gains and Harms in Reasoning-Prompt Transfer.

Furthermore, understanding model limitations is key, which leads to studies on interpretability. Capability not equaling interpretability is highlighted by research showing that foundation models are often less interpretable than size-matched supervised models, even though their raw capability doesn't always correlate with performance gains. This gap is addressed by focusing on localizability and semanticity in vision models, as explored in Capability not equal to Interpretability: Human Interpretability of Vision Foundation Models.

Finally, there is work looking at how cultural knowledge evolves over time, which is important because it challenges the static view of culture held by many current LLMs. ChronoCulture introduces a benchmark to show that models often fail to capture the temporal shifts in cultural attitudes across different years, suggesting a need for models that can track change rather than just report snapshots.

The most significant development today concerns how we can reliably verify generated text, which is crucial because current text-to-image models often introduce subtle rendering errors that are invisible to standard recognition systems. TextVerifier aims to solve this by training a verifier on character Ideographic Description Sequences instead of single symbols, achieving an F1 score of 0.844 against benchmarks like TextPecker, which is significantly better than the performance of general vision-language models or OCR systems. This means we have a tool that can not only detect these structural flaws but also use that detection as a reward signal to correct them during training, showing it can actively reduce rendering errors.

This verification capability builds on work exploring representation alignment, where RECON proposes representation conditioning to improve semantics in later layers of diffusion transformer training, making the process significantly faster than other methods like SiT and REPA. This speed gain is important because it allows for quicker iteration on complex generative models. Furthermore, the diagnostic study of world-space human recovery under high camera dynamics reveals a critical failure mode when fast motion is involved; specifically, when camera and human estimates are combined at mismatched scales, the system suffers from drift that grows with displacement in broadcast dance videos. A closed-form correction derived here can estimate this mismatch and apply a consistent scale to all performers, stabilizing the world-space reconstruction for choreography modeling.

Another area of progress involves improving 3D reconstruction quality for medical imaging; where Should the Gaussians Go? proposes a reusable labelaware allocation front end that decides where a fixed budget of Gaussians should be placed based on anatomical labels and transfer functions, achieving gains in vessel-masked PSNR over uniform allocation. This approach is promising because it avoids per-patient optimization for the initial placement, offering consistent performance across different patients when initialized from population training data. Finally, the research on facial image watermarking highlights a structural gap in biometric systems: even when a face image remains visually faithful and watermark-valid, it can become biometrically unreliable under adversarial attacks that target recognition similarity.

Today's papers

The papers

Important terms

Catastrophic Forgetting
This is the problem where language models forget old knowledge while learning new things. The research focuses on finding simpler ways to adapt models without needing complex rehearsal buffers or auxiliary objectives.
Draft-Thinking
This lightweight framework helps models learn a compact working derivation for answers. It tackles cross-scale mismatches by progressively adapting reasoning granularity to the target model, reducing reasoning cost.
Genesis Steering
This technique controls LLM behavior during inference using contrastive examples from the model's own data. It guides the model toward desired outputs, improving safety and control.
DeLoop
This method combats repetitive loops in long tasks by using a self-supervised training approach. It learns to treat loops as negative examples to improve efficiency and success rates.