ExecRetrieval: Measuring the Functional-Correctness Gap in Code-Embedding Retrieval

arXiv:2609.01865 · cs.SE, cs.AI, cs.CL, cs.IR · Submitted 2026-09-01 · Read on arXiv

cs.SE, cs.AI, cs.CL, cs.IR

Submitted: 2026-09-01

Updated: 2026-09-01

Terminology

• Embeddings — 26× in this paper · explained in 2 episodes
• Benchmark — 25× in this paper · explained in 2 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Cosine Similarity — 3× in this paper · explained in 3 episodes
• Empirical Methods in Natural Language Processing — 3× in this paper
• Transformer — 3× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• In International Conference on Learning Representations — 2× in this paper
• Verification — 2× in this paper · explained in 2 episodes
• Agentic Loop — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Archetype — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Calibration — 1× in this paper · explained in 7 episodes
• Commentary — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• International Conference on Machine Learning — 1× in this paper
• Meeting of the Association for Computational Linguistics — 1× in this paper
• Normalized Discounted Cumulative Gain — 1× in this paper
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Program Synthesis with Large Language Models
• Evaluating Large Language Models Trained on Code
• CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
• Gemini Embedding: Generalizable Embeddings from Gemini
• Towards General Text Embeddings with Multi-stage Contrastive Learning
• Functional Consistency of LLM Code Embeddings: A Self-Evolving Data Synthesis Framework for Benchmarking
• Text and Code Embeddings by Contrastive Pre-Training
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties