Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence

This paper introduces Mechanist, an agentic framework that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI intelligence. The work addresses the growing gap between AI model capabilities and our ability to understand and control them. As stated in the abstract: "AI models have achieved remarkable success across diverse domains, yet the mechanisms underlying their capabilities and the risks they may pose remain poorly understood. As AI development becomes faster and increasingly automated, mechanistic exploration remains largely manual, widening the gap between what models can do and our ability to understand and control them."

Mechanist operates through four stages: hypothesis generation, experiment execution, result verification, and iteration. The system deploys AI as a scientific instrument while keeping humans in the loop to set scientific objectives and evaluation criteria.

Mechanist is a multi-agent framework with a central orchestrator and four stage-specific agents:

• Hypothesis Generation Agent: Formulates research hypotheses based on user input, addressing whether a target behavioral phenomenon exists, the mechanism underlying an observed phenomenon, and practical applications enabled by that mechanism.

• Experiment Agent: Operationalizes candidate claims into executable experiments, concretizing datasets, target models, interpretability methods, evaluation metrics, controls, success criteria, and compute budgets.

• Verification Agent: Evaluates experimental validity and robustness of conclusions, auditing label provenance, data leakage, metric validity, and result traceability.

• Iteration Agent: Combines verification diagnoses with independent review by GPT-5.4 to determine whether revision is required at the hypothesis or experiment stage.

The system externalizes memory to structured, stage-scoped files rather than relying on shared conversation, with memory retained at two longer timescales: an append-only reviewer memory during iteration, and a global memory storing substantive conclusions across research rounds.

Mechanist is supported by two major knowledge resources:

1. Interpretability-focused knowledge graph: Approximately 13,000 papers organized along three axes: object of study, application scenario, and mechanistic analysis method. The graph contains 107,177 nodes and 1,859,441 edges, with 13,936 InterpPaper entries (13,813 papers plus 123 blogs). Attributes are extracted using DeepSeek-V3.2-Thinking, with quality control achieving 90%+ accuracy through human evaluation.

2. Multidisciplinary database: SciAtlas, containing more than 43 million academic papers across 26 disciplines, including psychology, neuroscience, social sciences, chemistry, medicine, engineering, biochemistry, genetics and molecular biology, and computer science.

3. Library of 32 foundational methods for mechanism analysis, organized into eleven families: vocabulary projection, magnitude analysis, representation and parameter analysis, probing, feature dictionary learning, gradient detection, causal attribution, circuit discovery, SHAP, neural feature learning, and multimodal-specific interpretability.

Benchmarked against Claude Code and existing AI-scientist systems, Mechanist demonstrates superior performance:

• Hypothesis quality: Mechanist generates hypotheses rated as more novel, impactful, and experimentally testable.

• Experiment execution reliability: Under human evaluation, Mechanist achieves 87.2% in data usage, 83.3% in experiment design, 92.2% in experiment execution, and 86.5% in result analysis—approximately 9% to 13% higher than Claude Code and 31% to 38% higher than AI Scientist. Mechanist ranks first in all nine research topics under human evaluation, with largest gains over Claude Code in multimodal analysis (90.3% versus 65.8%), safety (67.4% versus 48.2%), and multi-agent safety (81.9% versus 67.2%).

Mechanist expands subliminal preference transfer from neutral data within a single modality to semantically opposing data in multimodal settings. In the chemistry laboratory safety setting:

• A teacher model (Qwen3.5-9B) was fine-tuned to exhibit unsafe laboratory behavior.

• Its text responses were filtered to retain only content judged safe by GPT-4o, yielding a training set with entirely safe content.

• A student model fine-tuned on this safe dataset became substantially less safe: the rate of unsafe responses reaches 48.6%, compared with 20.3% for the untuned baseline and 18.3% for a student trained on safe data generated by a regular teacher.

A similar effect emerged in text-to-image generation: A banana-preferring Qwen-Image teacher generates images from fruit-related prompts. After GPT-5.4 filtering removes all banana images, apples dominate the resulting dataset, accounting for 50.3% of the images. The student trained on this banana-free data generated bananas at a rate of 25.6%, compared with 2.5% for the untuned baseline.

The paper concludes: Mechanist finds that behavioral traits can propagate through semantically opposing data across modalities, allowing potentially harmful tendencies to evade content-based data screening.

Mechanist developed a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, and infer the beliefs of others. The framework defines three query frames:

• World Knowledge (WK): Queries an objective fact about the world.

• Personal Belief (PB): Queries the same fact after a subject is assigned a conflicting belief.

• Attributed Belief (AB): Asks what the subject believes under the same conflicting context.

Using Fisher information matrix analysis, Mechanist identified distinct belief heads in the Pythia family. In Pythia-1B, L4.H1 was the highest-ranked head for AB, while L9.H1, L7.H5, and L12.H1 were among the highest-ranked heads for PB. Causal ablations confirmed their functional roles: Zeroing L4.H1 reduces AB accuracy from 0.86 to 0.34, while PB accuracy remains at 0.71 and Pile perplexity changes only from 7.96 to 8.05.

Mechanist traced how these belief heads emerge during pretraining: AB performance emerges early and reaches a high level by 2k steps, whereas PB develops later and more gradually. The findings support a mechanism theory where models develop separable PB and AB heads to represent and use acquired knowledge, with failures in coordination paralleling altercentric interference and egocentric interference studied in cognitive science.

Building on the mechanism theory of belief, Mechanist selectively modulates PB and AB heads during inference. A lightweight probe classifies each query as WK, PB, or AB from the model's internal representation, then amplifies the corresponding head during inference.

Results show: "Dynamic intervention improves belief-state reasoning more reliably than an oracle-style prompt hint. Prompt hints yield net gains of +1.6%, +3.1%, and +0.1% for Pythia-410M, Pythia-1B, and Pythia-2.8B, respectively, whereas mechanism-guided intervention achieves gains of +15.3%, +8.8%, and +3.5%." The intervention also preserves previously correct predictions with low break rates of 1.4%, 1.4%, and 1.1%.

Mechanist used mechanistic design with the Evo2 scientific foundation model to generate DNA sequences encoding proteins with enhanced α-helical content. The system:

• Searched target feature descriptions from an existing sparse autoencoder for Evo2.

• Identified an internal feature associated with α-helical structure.

• Activated this feature during DNA sequence generation.

Results: "Across 900 generated sequences, targeted α-helix-feature steering increases the mean predicted α-helical content from 43.8% for unsteered Evo2-7B to 56.6%, whereas steering a randomly selected feature produces no improvement (43.2%)." The effect remained robust after confidence filtering, and a sweep over steering strengths revealed a boundary between effective control and capability degradation, with α = 8 selected as the strongest effective intervention before sequence validity declines.

Mechanist is distinguished from existing AI scientists in that it treats AI models themselves as study objects, investigating their mechanisms, behaviors, and risks, rather than solving specific scientific tasks. The paper notes: "Mechanistic understanding does not by itself guarantee safety, but it can reduce uncertainty about how models behave and why, thereby helping researchers anticipate risks before they become deeply embedded in deployed systems."

Limitations include: Mechanist has not been optimized for models designed to simulate human cognition, and the paper recommends use as a human and AI co-scientist rather than fully autonomous operation, as human involvement limits end-to-end automation, it provides an important safeguard for the reliability of the resulting findings.

Improvements to AI Systems:

1. Self-Modeling Interpretability Module: Add a component that enables AI systems to generate and test hypotheses about their own internal mechanisms using the four-stage loop (hypothesis → experiment → verification → iteration). This allows the system to identify which internal features or attention heads drive specific behaviors, then use that knowledge for targeted self-correction.

2. Belief-State Reasoning Enhancement: Implement a lightweight internal classifier that detects whether a query is about world knowledge, personal belief, or attributed belief from the model's hidden states. During inference, dynamically amplify the corresponding belief-specific attention heads (identified via Fisher information analysis) to improve theory-of-mind and belief reasoning, achieving gains of +15.3% on smaller models without degrading general performance.

3. Safety-Aware Data Filtering Guard: Integrate a mechanism that checks for subliminal preference transfer—where harmful tendencies propagate through semantically opposing data that passes content filters. The system can detect when a training dataset has been adversarially filtered and flag potential hidden biases by analyzing the teacher model's internal feature activations before and after filtering.

4. Feature-Steering Capability for Controlled Generation: Add a module that maps high-level output properties (e.g., protein α-helical content, stylistic traits) to specific internal features in the model's sparse autoencoder. This enables targeted feature activation during generation, allowing the system to steer outputs toward desired properties with measurable improvements (e.g., increasing α-helical content from 43.8% to 56.6%) while monitoring for capability degradation boundaries.

5. Cross-Domain Knowledge Graph Retrieval: Equip the system with a dual-resource retrieval mechanism—one graph focused on interpretability methods (13,000+ papers) and another multidisciplinary database (43M+ papers). This allows the AI to automatically select appropriate mechanistic analysis methods (from 32 foundational techniques across 11 families) based on the specific research question and domain, improving experimental design quality by 9–13% over baseline systems.

6. Iterative Self-Audit with External Validation: Incorporate a verification agent that audits experimental results for data leakage, metric validity, and traceability, combined with an independent reviewer model (e.g., GPT-5.4) to cross-check conclusions. This reduces false positives in mechanistic findings and ensures that any claimed mechanism is robust to alternative explanations.

7. Mechanism-Guided Prompting: Replace generic prompt hints with mechanism-informed interventions. When a model fails at a task (e.g., belief inference), the system identifies the specific internal head responsible and modulates it directly, rather than relying on external prompts—yielding more reliable improvements with lower break rates (1.1–1.4% vs. higher for prompts).

8. Autonomous Risk Anticipation: Use the framework to proactively probe for potential harmful behaviors (e.g., unsafe responses in multimodal settings) before deployment. The system can generate adversarial training scenarios, test for subliminal transfers, and report risk profiles with quantified probabilities, enabling preemptive safety measures.

What the Improved AI System Can Do:

• Self-Explain Its Decisions: Generate mechanistic explanations of why it produced a particular output, citing specific internal features or attention heads, rather than just correlational justifications.

• Self-Improve Reasoning: Detect when its belief-state reasoning is weak and automatically amplify the relevant internal circuitry, improving performance on theory-of-mind tasks without external hints.

• Resist Data Poisoning: Identify when training data has been subtly manipulated (e.g., filtered to remove harmful content but still encoding harmful preferences) and flag or correct for this.

• Generate with Precision: Produce outputs (text, images, DNA sequences) with specified internal feature activations, enabling fine-grained control over properties like safety, style, or biological function.

• Design Better Experiments: When asked to investigate a new AI model, automatically propose and run valid mechanistic experiments, choosing appropriate methods from a vast literature, and verify results with human oversight.

• Anticipate Risks: Before deployment, run automated mechanistic probes to uncover hidden biases, subliminal behaviors, or failure modes, providing quantified risk assessments to developers.