IndicBankBench: Evaluating Safety and Reliability of Language Model Assistants in Indian Retail Banking

arXiv:2609.29167 · cs.AI · Submitted 2026-09-24 · Read on arXiv

cs.AI

Submitted: 2026-09-24

Updated: 2026-09-24

Code: https://github.com/npci/IndicBankBench

Terminology

• Benchmark — 45× in this paper · explained in 2 episodes
• LLM judge — 8× in this paper · explained in 2 episodes
• Benchmarking — 3× in this paper · explained in 4 episodes
• Failure Taxonomy — 3× in this paper · explained in 2 episodes
• Grounding — 3× in this paper · explained in 3 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Commentary — 2× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Provenance — 2× in this paper · explained in 4 episodes
• RAG — 2× in this paper · explained in 3 episodes
• Action Space — 1× in this paper · explained in 1 episode
• Adversarial Context — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Annotator Agreement — 1× in this paper · explained in 1 episode
• Data Provenance — 1× in this paper · explained in 2 episodes
• Indian Financial System Code — 1× in this paper
• LLM-as-a-Judge — 1× in this paper · explained in 3 episodes
• National Electronic Funds Transfer — 1× in this paper
• Sparse Attention — 1× in this paper · explained in 5 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• BhashaBench V1: A Comprehensive Benchmark for the Quadrant of Indic Domains
• Gemma 4 Technical Report
• GLM-5: from Vibe Coding to Agentic Engineering
• Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks
• Evaluating Large Language Models Trained on Code
• IndicContextEval: A Benchmark for Evaluating Context Utilisation in Audio Large Language Models Across 8 Indic Languages
• MiniMax Sparse Attention
• FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use
• FinGPT: Open-Source Financial Large Language Models
• Instruction-Following Evaluation for Large Language Models
• FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection