Mine and Refine: Optimizing Graded Relevance in E-commerce Semantic Search Retrieval
cs.IR, cs.LG
Submitted: 2026-02-19
Updated: 2026-08-28
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Embedding-based retrieval (EBR) for large-scale e-commerce search faces three intertwined challenges: graded (non-binary) relevance where engagement signals are noisy and intent-varying while
Terminology
Abstract
Embedding-based retrieval (EBR) for large-scale e-commerce search faces three intertwined challenges: graded (non-binary) relevance where engagement signals are noisy and intent-varying while business relevance guidelines admit acceptable-but-not-exact matches, false negatives in hard sample mining, and unstable similarity score separability across relevance levels, the last of which complicates hybrid search score fusion and downstream ranking. We propose Mine and Refine, a two-stage contrastive training framework that addresses all three. A lightweight LLM, fine-tuned with engagement-driven audit, serves as a guideline-aligned scalable labeler throughout training. Stage 1 establishes a robust global embedding space via label-aware supervised contrastive learning; Stage 2 mines hard samples, re-annotates them with the LLM labeler to mitigate spurious negatives, and refines the model through a multi-level extension of circle loss that enforces margin-controlled separation across relevance levels. Deployed in production e-commerce search across multiple product verticals, the approach delivers statistically significant lifts in user engagement and gross order value, and substantially improves retrieval and end-to-end relevance metrics.
Sources
- OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Search
- Language-agnostic BERT Sentence Embedding
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Unsupervised Dense Information Retrieval with Contrastive Learning
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
- MTEB: Massive Text Embedding Benchmark
- GPT-4o System Card
- BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
- Text Embeddings by Weakly-Supervised Contrastive Pre-training
- An Embedding-Based Grocery Search Model at Instacart
- Multilingual Universal Sentence Encoder for Semantic Retrieval
Related papers
- The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
- SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
- MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
- Right Family, Wrong Skill: Evaluating Risk Exposure in Agent Skill Retrieval
- UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG