How Weight Encoding Affects Language Model Placement and Performance on the Apple Neural Engine
cs.LG, cs.AR, cs.PF
Submitted: 2026-08-22
Updated: 2026-09-23
Code: https://github.com/shershah1024/ane-llm-measurements
Terminology
Sources
- Apple Neural Engine: Architecture, Programming, and Performance
- ANEForge: Python for direct computation on the Apple Neural Engine
- Orion: Characterizing and Programming Apple's Neural Engine for LLM Training and Inference
- Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUs
- T-MAN: Enabling End-to-End Low-Bit LLM Inference on NPUs via Unified Table Lookup
- Fast On-device LLM Inference with NPUs
- Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference
- RooflineBench: A Benchmarking Framework for On-Device LLMs via Roofline Analysis
- Pagoda: An Energy and Time Roofline Study for DNN Workloads on Edge Accelerators
- Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference
- ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inference
- Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference
- LLM Inference at the Edge: Mobile, NPU, and GPU Performance Efficiency Trade-offs Under Sustained Load
- LFM2 Technical Report
- STAR: Synthesis of Tailored Architectures
- Liquid Time-constant Networks
- ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
- MnasNet: Platform-Aware Neural Architecture Search for Mobile
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks