LSR-Ben: A Logical and Scientific Reasoning Benchmark for Evaluating Process Reward Models

LSR-Ben introduces a comprehensive benchmark designed to evaluate Process Reward Models (PRMs) across both scientific and logical reasoning domains, addressing the critical limitation of existing benchmarks that are narrowly focused on mathematical reasoning. This work is significant because it provides a necessary tool for assessing PRM performance in diverse, real-world reasoning scenarios, aiming to foster future research that enhances the general reasoning capabilities of Large Language Models (LLMs).

Benchmark Design and Scope

The GR-Ben benchmark is specifically designed to assess PRMs’ ability to identify erroneous steps in broad reasoning scenarios. It is characterized by its Comprehensiveness of Reasoning Types, covering two major categories: scientific reasoning and logical reasoning, along with nine subdomains. This structure enables a comprehensive evaluation of PRMs on identifying reasoning errors. The benchmark includes 3600 data instances spread across these domains, with quality ensured by professional annotators and a double cross-validation process.

Reasoning Domains and Subdomains

The paper categorizes reasoning into two main groups: scientific reasoning (four subdomains—physics, chemistry, biology, and computer science) and logical reasoning (five subdomains—deductive reasoning, inductive reasoning, abductive reasoning, analogical reasoning, and mixed-form reasoning). The data collection for scientific tasks is drawn from the widely used benchmark MMLU-Pro. For logical tasks, problem instances are curated from datasets such as FOLIO (for deductive), MIRAGE (for inductive), CauseLogics (for abductive), Analobench (for analogical), and LogiQA2.0.

Error Classification

The benchmark systematically annotates not just erroneous reasoning steps but also corresponding error categories, enabling the evaluation of PRMs' weaknesses in identifying specific error types. The five main categories of reasoning errors are:

1. Knowledge-based errors: the employment of extra domain-specific knowledge, commonsense knowledge, or world knowledge that is incorrect during the reasoning process.

2. Factual errors: the adoption of facts that contradict the known information in the reasoning steps.

3. Computational errors: mathematical miscalculations are made in the course of the reasoning process.

4. Logical errors: when the conclusion in a reasoning step cannot be logically derived or inferred from the available information.

5. Others.

Data Curation and Annotation Quality

To ensure a balanced distribution between erroneous and correct solutions, a multi-stage data curation process is employed. First, LLMs are leveraged to generate corresponding solutions (reasoning processes). Second, solution reformatting is used to standardize the granularity of reasoning steps, which helps in aligning fragments with logically coherent steps. Third, human experts and professional annotation companies are engaged to identify the first erroneous step in the reasoning process and label its error type. Quality assurance involves a rigorous double cross-validation procedure where annotators review conflicting annotations, and solutions that fail to achieve consensus are discarded.

Evaluation Metrics

The primary metric used for comparison between PRMs and LLMs is the F1 score, which is derived by computing accuracies for erroneous and correct samples separately and then taking the harmonic mean of these two accuracy metrics. This metric is prioritized because it effectively balances the trade-off between being overly critical (i.e., over-identifying errors) and being incapable of identifying errors (i.e., failing to detect errors). Furthermore, error analysis focuses on calculating the False Negative (FN) rate and False Positive (FP) rate for each error type to determine the specific categories of errors that are more difficult to identify. Experimental results indicate a complementary capability relationship, where PRMs exhibit an inherent tendency to overlook errors compared to LLMs, while LLMs tend toward over-identification.

Model Comparison Findings

Extensive experiments on the GR-Ben dataset reveal several key findings. First, existing PRMs fail to identify errors in general reasoning domains, demonstrating that their performance on GR-Ben is "much lower (<20% in general) than that on ProcessBench. Second, compared to math-oriented PRMs, the non-mathoriented model (VersaPRM) shows deficiencies, particularly for deductive reasoning. Finally, the analysis of error types shows that PRMs are less adept at identifying knowledge-based errors, whereas LLMs exhibit poorer performance in detecting computational errors. The authors conclude that GR-Ben can foster future researches on PRMs for general domains, thereby further enhancing the reasoning capabilities of LLMs."

Limitations

The current evaluation is limited because only one PRM for general reasoning, VersaPRM, is publicly available. Furthermore, the original OpenPRM model cannot be retrieved due to server shutdowns. Future work will extend evaluations to include additional open-source PRMs if they are released.

Based on the provided scientific paper, GR-Ben: A General Reasoning Benchmark for Evaluating Process Reward Models, here are specific improvements that can be made to AI systems, categorized by their potential impact:

)1. Targeted PRM Development for General Reasoning Domains

The primary improvement is the transition from mathematical-only Process Reward Models (PRMs) to models capable of robust error detection across diverse cognitive domains.

• I can improve AI systems by training/fine-tuning PRMs specifically on the GR-Ben benchmark, which covers both scientific reasoning (physics, chemistry, biology, computer science) and logical reasoning (deductive, inductive, analogical reasoning).

• The improved PRM system will be able to accurately identify the earliest erroneous step in complex non-mathematical reasoning chains.

• Specifically: The system will excel at detecting Knowledge-based errors (e.g., using incorrect commonsense knowledge) and Factual errors (e.g., contradictory facts), which are currently identified as weak points in existing PRMs (Table 4).

)2. Enhanced LLM Reasoning Capabilities via Domain-Specific Prompting

The paper suggests that while LLMs show non-trivial error identification in general reasoning, their performance is highly dependent on the thinking mode employed.

• I can improve AI systems by implementing Slow Thinking or structured decomposition prompting (as shown in Table 5) for LLMs tackling complex reasoning tasks.

• The improved LLM will be able to better handle multi-step, abstract reasoning problems by decomposing the problem before generating a response, leading to higher accuracy in identifying computational and logical errors.

• Specifically: The system will exhibit superior performance on Logical Errors compared to its current state when tested on GR-Ben's logical subdomains (Table 7).

)3. Developing Error-Specific Detection Modules

The benchmark systematically annotates distinct error types (knowledge-based, factual, computational, logical).

• I can improve AI systems by developing modular error detection layers that are fine-tuned to recognize specific error patterns corresponding to these categories.

• The improved system will move beyond simply flagging an error and instead classify the nature of the flaw.

• Specifically: The system will be able to differentiate between a simple arithmetic mistake (Computational Error) and a flawed inference based on missing domain knowledge (Knowledge-based Error).

)4. Robust Solution Generation Pipelines

The research highlights that solution generation from diverse LLMs leads to inconsistent step granularity, which hinders annotation quality.

• I can improve the pipeline for generating reasoning processes by integrating a Solution Reformatting module (as described in Section 3.2) that standardizes the output structure of intermediate reasoning steps before they reach the error detection mechanism.

• The improved system will produce more uniform, logically coherent reasoning chains, making it significantly easier and more accurate for subsequent evaluation modules (both PRMs and LLMs) to analyze step-by-step correctness.

)5. Improved Decision Making Stance (Mitigating Conservatism)

The study notes that both PRMs and LLMs exhibit a tendency toward overlooking errors (lower accuracy on erroneous samples relative to correct ones).

• I can improve the core decision-making algorithm by implementing a mechanism that dynamically balances the trade-off between being overly critical and being incapable of identifying errors.

• The improved system will adopt a more balanced approach to error detection, reducing the tendency to ignore potential flaws due to conservatism.

• Specifically: The system will show reduced False Negative (FN) rates compared to its current state, meaning it is less likely to miss an actual erroneous step in favor of being overly cautious.