Beyond Relevance: Structured Semantic Supervision for Product Search with LLM-Augmented Annotations
cs.IR, cs.CL
Submitted: 2026-09-20
Updated: 2026-09-20
Comments: 16 pages, 2 figures
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: E-commerce search requires distinguishing products that are merely related to a query from those that directly satisfy the user's shopping intent.
Terminology
Abstract
E-commerce search requires distinguishing products that are merely related to a query from those that directly satisfy the user's shopping intent. We augment query-product pairs with structured LLM-generated query and product attributes and human-validated relevance, explanations, and centrality judgments, and evaluate these signals using a simple dual-encoder retriever and MLP re-ranker. On an augmented subset of ESCI, a human-feature oracle reaches 0.9382 nDCG@10, while a human-free trained Q+P configuration reaches 0.9258. Synthetic approximations of the human signals reach 0.9150 overall but provide substantial gains for difficult, low-performing queries. Ablations show that most of the oracle improvement comes from post-edited explanations and annotator comments rather than the scalar centrality feature, suggesting that LLMs are most useful for exposing and approximating structured semantic supervision rather than replacing human judgment directly.
Sources
- Overview of the TREC 2023 Product Product Search Track
- Exploring the Viability of Synthetic Query Generation for Relevance Prediction
- Synthetic Data Powers Product Retrieval for Long-tail Knowledge-Intensive Queries in E-commerce Search
- Intent term selection and refinement in e-commerce queries
- Text Embeddings by Weakly-Supervised Contrastive Pre-training
- Passage Re-ranking with BERT
- Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation
- Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Search
- Qwen3 Technical Report
- A Semantic Alignment System for Multilingual Query-Product Retrieval
Related papers
- The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
- SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
- MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
- Right Family, Wrong Skill: Evaluating Risk Exposure in Agent Skill Retrieval
- UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG