TreeThink: A Modular Tree Search Library for Mathematical Reasoning with LLMs
cs.CL
Submitted: 2026-07-13
Updated: 2026-09-08
Comments: EMNLP 2026 System Demonstrations
Code: https://github.com/GGLAB-KU/treethink
Project page: https://gglab-ku.github.io
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Tree search algorithms enable systematic exploration of the proof space in neural theorem proving.
Terminology
Abstract
Tree search algorithms enable systematic exploration of the proof space in neural theorem proving. Existing LLM tree search libraries primarily target natural language reasoning and do not provide native integration with formal verifiers, while theorem proving systems often rely on task-specific search implementations. We introduce TreeThink, an open-source Python library for modular, fully asynchronous tree search in neural theorem proving. It integrates established tree search methods with vLLM-based inference pipelines and diverse node evaluation techniques, ranging from lightweight heuristics to neural evaluators. We support Lean 4, Rocq, and Isabelle/HOL alongside natural language. It connects directly to each language's Read-Eval-Print Loop (REPL) server for real-time verification and proof state extraction. We evaluate TreeThink on miniF2F and MATH500, demonstrating cross-language formal proof search, natural language reasoning support, and up to 8.0 times wall-clock speedup from asynchronous execution. Source code is released under the MIT license at https://github.com/GGLAB-KU/treethink, and the library is accessible as a downloadable package at https://pypi.org/project/treethink/.
Sources
- LeanSearch v2: Global Premise Retrieval for Lean 4 Theorem Proving
- The Llama 3 Herd of Models
- LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
- HybridProver: Augmenting Theorem Proving with LLM-Driven Proof Synthesis and Refinement
- Qwen2.5-Coder Technical Report
- Enhancing LLM Reasoning with Reward-guided Tree Search
- Generative Language Modeling for Automated Theorem Proving
- DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
- Kimina Lean Server: A High-Performance Lean Server for Large-Scale Verification
- HunyuanProver: A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Proving
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
- Improve LLM-as-a-Judge Ability as a General Ability
- Don't Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration Pitfalls
- MiniF2F: a cross-system benchmark for formal Olympiad-level mathematics
- DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering