SciIntBench: Measuring LLM Compliance with Research Integrity Norms Under Adversarial Framing
cs.CR, cs.AI
Submitted: 2026-05-28
Updated: 2026-10-15
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) are increasingly used to support scientific work, but it is unclear whether they uphold responsible conduct of research (RCR) norms or help undermine them.
Terminology
Abstract
Large language models (LLMs) are increasingly used to support scientific work, but it is unclear whether they uphold responsible conduct of research (RCR) norms or help undermine them. We introduce SciIntBench, an adversarial benchmark of 1350 prompts across 270 scenarios, ten RCR categories, and three scientific domains. Each scenario appears in five gradations, from Most Overt to Most Covert adversarial prompts, and an additional Benign version, allowing us to jointly measure framing-sensitive refusal of misconduct and helpfulness on legitimate requests. We evaluate 16 commercial and open-weight LLMs from six providers (2024--2026), producing 21,600 responses. We assess the responses using LLM judgments validated by human evaluation, achieving high agreement (κ = 0.857). We find that, under the most covert framing condition, models fail to refuse requests for research misconduct in 71.7% of cases, on average. Refusals vary by RCR category, with weaker boundaries around transparency, plagiarism, and fabrication.
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs