SAFARI: An Industrial Benchmark for LLM-Assisted Hazard Analysis and Risk Assessment

arXiv:2609.20584 · cs.CL · Submitted 2026-09-17 · Read on arXiv

cs.CL

Submitted: 2026-09-17

Updated: 2026-09-17

Comments: Accepted at EMNLP 2026 Industry Track

Code: https://github.com/xixi47520-hash/HARA

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 28× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 7× in this paper · explained in 3 episodes
• Consistency — 6× in this paper · explained in 3 episodes
• Hazard Analysis and Risk Assessment — 5× in this paper
• Alignment — 3× in this paper · explained in 1 episode
• Chain-of-Thought Prompting — 3× in this paper · explained in 2 episodes
• IEEE Intelligent Vehicles Symposium — 3× in this paper
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Calibration — 2× in this paper · explained in 7 episodes
• Core — 2× in this paper · explained in 2 episodes
• Ontology — 2× in this paper · explained in 2 episodes
• Trace — 2× in this paper · explained in 2 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Adversarial Examples — 1× in this paper · explained in 2 episodes
• Affective Computing — 1× in this paper · explained in 1 episode
• Automotive Safety Integrity Level — 1× in this paper
• Causal Reasoning — 1× in this paper · explained in 2 episodes
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Evaluation Dimensions — 1× in this paper · explained in 3 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fault Tolerant Time Interval — 1× in this paper
• GRPO — 1× in this paper · explained in 6 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Hazard Logic Mapping — 1× in this paper
• LLM judge — 1× in this paper · explained in 2 episodes
• Physical Consequence Consistency — 1× in this paper
• Reasoning Traces — 1× in this paper · explained in 3 episodes
• Risk Coverage Completeness — 1× in this paper
• Safety Goal Alignment — 1× in this paper
• Safety-Aware Functional Automotive Risk Inference — 1× in this paper
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Do LLM Evaluators Prefer Themselves for a Reason?
• LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
• Engineering Safety Requirements for Autonomous Driving with Large Language Models
• When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks
• Herculean: An Agentic Benchmark for Financial Intelligence
• PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics
• Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling
• AudioProcessBench: Benchmark for Identifying Process Errors in Audio-Grounded Reasoning

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering