Rank-Reliable Teacher-Guided Fitness Approximation for Expensive Evolutionary Optimization: A TinyML Architecture Search Study

arXiv:2609.30553 · cs.AI, cs.LG · Submitted 2026-09-24 · Read on arXiv

cs.AI, cs.LG

Submitted: 2026-09-24

Updated: 2026-09-24

Terminology

• Fidelity — 51× in this paper · explained in 2 episodes
• Accuracy — 36× in this paper · explained in 1 episode
• Distillation — 36× in this paper · explained in 2 episodes
• Ablation — 12× in this paper · explained in 1 episode
• Knowledge Distillation — 9× in this paper · explained in 11 episodes
• Combinatorial Optimization — 8× in this paper · explained in 2 episodes
• Bayesian Optimization — 4× in this paper · explained in 3 episodes
• Trace — 4× in this paper · explained in 2 episodes
• Benchmark — 3× in this paper · explained in 2 episodes
• Teacher-Guided Learning — 3× in this paper
• Alignment — 2× in this paper · explained in 1 episode
• Pairwise Comparisons — 2× in this paper · explained in 2 episodes
• Quantization — 2× in this paper · explained in 12 episodes
• Systematic Bias — 2× in this paper · explained in 3 episodes
• Theoretical Guarantees — 2× in this paper · explained in 2 episodes
• Calibration — 1× in this paper · explained in 7 episodes
• Core — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Error Decomposition — 1× in this paper · explained in 3 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Monte Carlo dropout — 1× in this paper · explained in 2 episodes
• Multi-objective Optimization — 1× in this paper · explained in 4 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Resource Constraints — 1× in this paper · explained in 3 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Sample Efficiency — 1× in this paper · explained in 4 episodes
• Stochastic Gradient Descent (SGD) — 1× in this paper · explained in 5 episodes

Sources

• Distilling the Knowledge in a Neural Network
• Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"
• Progressive distillation induces an implicit curriculum
• SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Search
• Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection