Emotion2Skill: Model-Internal Emotion Signals for Adaptive Skill Selection and Evolution

Emotion2Skill is a framework that extracts LLM-internal emotion vectors and incorporates them into both skill selection and skill evolution for skill-based LLM agents. The paper states: We propose Emotion2Skill, a framework that extracts LLM-internal emotion vectors and incorporates them into both skill selection and skill evolution.

The method works as follows: At each decision step, a 27-dimensional emotion state is extracted from the residual stream and mapped to a confidence-gated summary injected into the routing prompt. The emotion vectors are extracted using a contrastive-averaging procedure on Qwen3, where For each of the K=27 emotion concepts in the GoEmotions taxonomy, the model generates N=100 short stories with the target emotion. A PCA-based denoising step projects out top principal components of neutral activations, and the optimal extraction layer L* is selected by maximizing GoEmotions classification accuracy (L*=24 for Qwen3-8B with 37.2% accuracy; L*=26 for Qwen3-14B with 39.4% accuracy).

The emotion state is processed by a lightweight encoder: The encoder is a 3-layer MLP that produces an encoded representation rt = fθ(et) ∈ Rd'. This produces a template index over a bank of C=12 natural-language emotion-state descriptions and a scalar confidence score ct. A confidence-gating mechanism omits the emotion signal when ct < τ (default τ=0.3).

Beyond online selection, emotion trajectories are analyzed for abrupt internal-state shifts to pinpoint problematic skill invocations, guiding targeted SOP rewriting that replaces the coarse binary outcome signal of prior methods. The framework computes step-to-step emotion shift magnitude δt = 1 - e⊤t et-1 / (∥et∥ ∥et-1∥), and transition points are defined as steps where δt exceeds the episode mean by more than one standard deviation.

Results show: "On WebShop and ALFWorld, Emotion2Skill with Qwen3-8B improves over the Zero-Shot baseline by +26.9% success rate and +25.5% average success respectively, outperforming all baselines on both benchmarks with consistent gains on Qwen3-14B." Specifically, on Qwen3-8B, Emotion2Skill reaches 47.4% average success on ALFWorld and 29.7% Succ. on WebShop. On Qwen3-14B, it achieves 52.3% average success and 30.7% Succ.

The paper notes: "The per-task ALFWorld breakdown reveals that Emotion2Skill's advantage concentrates on tasks with high uncertainty and frequent error-recovery demands. Heat improves from 9.6% to 56.9% on the 8B model, a gain of 47.3 pp, while Pick2 rises from 4.4% to 31.3%, a gain of 26.9 pp."

Ablation studies show that removing emotion extraction causes the largest drop (-4.7 pp WS Succ), removing the emotion encoder costs-3.1 pp WS Succ and-5.5 pp ALF Avg, and removing emotion evolution costs-0.9 pp WS Succ and-3.1 pp ALF Avg.

Co-activation analysis reveals semantically coherent emotion–skill pairings: (i) curiosity+desire → ProductSearch; (ii) confusion+nervousness → QueryRephrase; (iii) approval+optimism → PurchaseConfirm; (iv) annoyance+disappointment → PriceCompare. The consistency rate is 76.5% (Cohen's κ=0.81 vs. human annotation).

Out-of-domain generalization on MATH and MBPP shows: Compared to a Zero-Shot baseline, Emotion2Skill achieves +14.4 pp on MATH, raising accuracy from 54.8% to 69.2%, and +11.8 pp on MBPP, raising pass@1 from 60.0% to 71.8%.

The paper concludes: These results establish LLM-internal emotion representations as an effective decision-level signal for orchestrating agent skill systems, extending their utility beyond interpretability and output steering. The code is available at https://github.com/BoHan-LIN04/Emotion2Skill.

Improvements to AI Systems:

1. Adaptive Error-Recovery via Emotion-Shift Detection: Implement a real-time monitor that tracks the agent’s internal emotion-state shift magnitude (δt) during task execution. When δt exceeds the episode mean by one standard deviation, the system automatically triggers a pre-defined recovery protocol—e.g., switching from a failing skill to a more exploratory one (like QueryRephrase) or re-evaluating the current subgoal. This improves robustness in high-uncertainty environments (e.g., ALFWorld’s Heat task, which jumped from 9.6% to 56.9% success) by catching failure states before they cascade.

2. Confidence-Gated Emotion Injection for Sparse Decision Making: Integrate the confidence score (ct) into the agent’s routing logic so that emotion-based prompts are only used when the internal state is reliably detected (ct ≥ 0.3). For low-confidence steps, the system falls back to a neutral, purely skill-based prompt. This reduces noise in decision-making and improves performance on tasks where emotional signals are ambiguous, as evidenced by the +26.9% success gain on WebShop over Zero-Shot.

3. Emotion-Conditioned Skill Evolution (SOP Rewriting): Replace binary success/failure signals with emotion-trajectory analysis to identify why a skill failed. When a transition point (δt spike) coincides with a failed skill invocation, the system rewrites the Standard Operating Procedure (SOP) for that skill to include alternative sub-steps that mitigate the detected emotional state (e.g., if confusion+nervousness precedes a QueryRephrase failure, add a clarification step). This yields +3.1 pp on ALFWorld average success and +0.9 pp on WebShop, enabling more nuanced skill refinement.

4. Cross-Domain Generalization via Emotion-Transferable Prompts: Use the extracted emotion vectors (e.g., curiosity+desire → ProductSearch) to pre-train a lightweight prompt adapter that can be applied to new, unseen domains. Since Emotion2Skill shows +14.4 pp on MATH and +11.8 pp on MBPP without task-specific tuning, this adapter can be deployed as a plug-in module for any LLM agent, improving performance on math and code generation tasks by injecting task-appropriate emotional context (e.g., confidence for problem-solving, curiosity for exploration).

5. Human-Aligned Skill Selection with Co-Activation Consistency: Leverage the 76.5% consistency rate (Cohen’s κ=0.81) between emotion–skill co-activations and human annotations to build a preference-learning layer. The improved system can rank candidate skills not just by predicted reward, but by emotional coherence with the current state, making the agent’s behavior more interpretable and trustworthy to human users, especially in interactive settings like customer service or tutoring.

What the Improved AI System Can Do:

• Self-correct in real time by detecting internal emotional distress (e.g., confusion, annoyance) and automatically switching to more suitable skills, leading to higher task completion rates in dynamic environments.

• Learn from emotional feedback to refine its own skill library, not just from success/failure but from why a step felt uncertain, enabling faster adaptation to novel tasks.

• Transfer emotional intelligence across domains (e.g., from shopping to math) without retraining, using a universal emotion-vector encoder.

• Explain its decisions to humans by mapping internal emotion states to natural-language descriptions (e.g., I feel confused, so I’m rephrasing the query), improving transparency and user trust.

• Operate reliably under uncertainty by suppressing emotion signals when they are noisy, avoiding over-reaction to false emotional cues.