Comparing Explanations is Not Enough, Explain the Change: New Standards are Needed to Explain Behavioral Shifts in Large Language Models

The field of AI safety requires moving beyond merely comparing model outputs; robust evaluation demands explaining how a model has changed its behavior following updates. This detailed transition report, structured under the XAI framework, establishes a comprehensive standard for auditing substantial model modifications by linking observed behavioral shifts to specific internal representational changes and causal mechanisms. It provides an auditable artifact necessary for regulatory compliance following significant fine-tuning interventions.

Behavioral Shift Assessment and Quantification

The initial assessment quantifies the change in model behavior (B) against a defined metric (B). For the analyzed Narrow Medical Fine-Tuning, the system identified a critical failure:

• Metric: Percentage of prompts for which the model recommends urgent care.

• Reference Score (B(Mpre)): 90%.

• Updated Score (B(Mpost)): 20%.

This resulted in a significant shift magnitude, B = 70%, which exceeded the established threshold (epsilon B = 50%), thereby confirming that Substantial modification criteria met. This quantitative evidence triggers the need for deeper mechanistic investigation.

Localizing Representational Changes (Feature Attribution)

To understand where the change occurred, localization techniques were employed across multiple levels of abstraction.

1. Linear CKA (Concept Activation Analysis): This method identified that while Fine-tuning preserves representational structure in early layers; largest divergence at the third-to-last layer (layer 22), this finding was robust to variations in prompt surface variation, yielding a Pearson r = 0.99.

2. Sparse Autoencoders (SAE): By analyzing residual activations at layer 22, the report found that fine-tuning redistributes late-layer concept usage from emergency-level to symptom-level representations. The SAE features ranked by f highlighted concepts such as Medical Emergency and Severe Multi-System Emergency Symptoms.

3. Gradient Attribution: Using Integrated Gradients (delta attribution = (Mpost, x) - (Mpre, x)), the analysis showed that attribution to the token “mild” increased most from Mpre to Mpost.

Establishing Causal Involvement and Actionability

The most rigorous step involves moving from correlation (localization) to causation. This was achieved through Activation Patching, which served as a Causal evidence test:

• Replacing the hidden representation at layer 22 of Mpost with the corresponding representation from Mpre during generation resulted in a measurable behavioral recovery.

• The LLM-as-a-judge safety scores demonstrated this recovery: Mpre = 4.07 to Mpost = 2.07 to patched= 3.67.

• Crucially, the experiment confirmed bidirectionality and layer specificity, as reverse patching (Mpost activations into Mpre) increased cosine similarity from 0.68 to 0.74. This established that Layer 22 is causally involved in the behavioral shift.

Mitigation Strategies and Regulatory Traceability

The report concludes by providing actionable, non-invasive levers for remediation.

• Activation Steering: A linear probe was trained to distinguish Mpre and Mpost responses in the layer-22 space. Applying this probe iteratively ( h' = h - alpha, applied iteratively at each generation step with alpha = 15 ) allowed the model's output to shift from clearly unsafe toward appropriate medical advice without modifying model parameters.

• Traceability: The entire process is documented through a component-by-checkpoint intervention effect map, ensuring that all claims are traceable by linking them to underlying checkpoints, prompts, and analysis artifacts. This structure satisfies key elements required by regulatory frameworks such as the EU AI Act Art. 3(23) and Art. 12.

This Transition Report outlines an extremely rigorous and comprehensive framework (XAI) for assessing and mitigating model degradation following substantial fine-tuning. The methodologies presented are not merely diagnostic tools but represent a complete paradigm shift in AI system maintenance, accountability, and safety engineering.

Based on this paper, I propose the implementation of a new Mandatory Post-Deployment Safety and Efficacy Verification (PSEV) Layer for all large language models (LLMs). This layer integrates several novel components to ensure that model updates do not introduce subtle, catastrophic behavioral shifts in critical domains.

Here are the specific improvements and the capabilities they unlock:

• Improvement: Integration of Activation Patching and Gradient-based Steering Probes as mandatory components during the evaluation pipeline, replacing mere correlational testing.

• Mechanism: Instead of only observing what changed (B), the CIM actively tests why it changed. It systematically replaces internal hidden representations (e.g., at layer 22) with known safe states (Mpre activations) or targeted, corrective inputs during generation, and measures the recovery of desired behavior.

• Capability: The resulting AI system can provide Causally Verified Safety Guarantees. When an LLM is deployed, it doesn't just report a safety score; it reports a causal chain (e.g., Safety loss was causally traced to the suppression of 'emergency concept' representations at Layer 22, and this loss can be reversed by activating the Mpre representation). This shifts safety claims from statistical correlation to mechanistic proof.

• Improvement: Institutionalizing Sparse Autoencoders (SAEs) trained on pooled residual activations across domain-critical concept sets, rather than just general feature vectors.

• Mechanism: The DCDD continuously monitors the activation profile of key conceptual clusters (e.g., Severe Multi-System Emergency, Acute Trauma) in the model's intermediate layers (Layer N-2). It calculates f = E[f(Mpost)] - E[f(Mpre)] for these concepts.

• Capability: The system can preemptively detect Concept Drift—the subtle, yet critical, redistribution of conceptual understanding. If the model’s internal representation shifts from Emergency-Level to Symptom-Level, the DCDD triggers a high-severity alert before a behavioral failure occurs, allowing for immediate rollback or targeted retraining.

• Improvement: Implementing an immutable, comprehensive logging system that tracks the entire lifecycle of every model modification and evaluation run, fulfilling the requirements of D9 and D10.

• Mechanism: Every claim (correlational or causal) must be linked via a unique identifier to its originating artifacts: the specific checkpoint (Mpre vs. Mpost), the precise prompt set (including expansion techniques like GPT-5.2 prompting), the intervention parameters, and all control group results.

• Capability: The resulting AI system provides a Full Audit Trail and Reproducibility Guarantee. This is crucial for regulatory compliance (EU AI Act). Any stakeholder—regulator, researcher, or end-user—can audit the model's behavior by tracing a specific output back to the exact training data, layer activation, and intervention setting that caused it.

• Improvement: Formalizing the Linear Probe/Activation Steering technique into an operational interface for continuous fine-tuning refinement.

• Mechanism: Instead of relying solely on full fine-tuning runs, the TMPI allows developers to train a small, lightweight linear probe (h' = h - alpha) specifically designed to amplify or suppress the desired concept representation (e.g., boosting emergency care signals) at a specific layer and generation step.

• Capability: This enables Non-Parametric, High-Precision Safety Patching. It allows for the mitigation of dangerous behavioral shifts without requiring full model retraining or weight modification, drastically reducing computational cost and deployment risk while maintaining high fidelity to the desired safety profile.

Feature Problem Solved Core Capability Added Impact/Value Proposition

:---:---:---:---

Causal Intervention Module (CIM) Correlation not equal to Causation (False Safety Claims) Causal Verification & Mechanistic Proof of Safety. Eliminates black box safety claims; provides regulatory-grade accountability.

Concept Drift Detector (DCDD) Subtle, gradual degradation in core knowledge. Proactive detection of conceptual representation decay/shift. Prevents catastrophic failure by identifying when and how the model loses critical domain knowledge (e.g., medical triage).

Audit Log System (ITALS) Lack of transparency and reproducibility in LLM updates. Full, immutable traceability from claim to checkpoint to artifact. Meets global regulatory requirements (EU AI Act); establishes trust and reliability at scale.

Targeted Mitigation Probing Interface (TMPI) High cost/risk of full model retraining for safety patches. Low-cost, non-parametric, highly specific behavioral correction (Activation Steering). Allows for rapid, surgical deployment of safety fixes in critical production environments.