BITEM at the NTCIR-19 R2C2 Task: Predicting Confidence from Agentic RAG Pipeline Signals

arXiv:2609.37993 · cs.CL, cs.AI, cs.IR · Submitted 2026-09-29 · Read on arXiv

cs.CL, cs.AI, cs.IR

Submitted: 2026-09-29

Updated: 2026-09-29

Project page: https://brev-rag-workshop.github.io/paper-02.pdf

Terminology

• Accuracy — 27× in this paper · explained in 1 episode
• Trace — 9× in this paper · explained in 2 episodes
• RAG — 8× in this paper · explained in 3 episodes
• Passage Retrieval — 6× in this paper
• Answering with Confidence — 4× in this paper
• Calibration — 4× in this paper · explained in 7 episodes
• Agentic Search — 3× in this paper · explained in 1 episode
• Aggregation — 3× in this paper · explained in 1 episode
• Benchmark — 2× in this paper · explained in 2 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• AUC — 1× in this paper · explained in 2 episodes
• Agentic Retrieval — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Knowledge Graph — 1× in this paper · explained in 7 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Verifiability — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
• Language Models (Mostly) Know What They Know
• Beyond Parallel Sampling: Diverse Query Initialization for Agentic Search
• Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests
• Passage Re-ranking with BERT
• Nomic Embed: Training a Reproducible Long Context Text Embedder
• VeriCite: Towards Reliable Citations in Retrieval-Augmented Generation via Rigorous Verification
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering