Learning What Matters Now: Dynamic Preference Inference under Contextual Shifts

arXiv:2603.22813 · cs.AI · Submitted 2026-03-24 · Read on arXiv

cs.AI

Submitted: 2026-03-24

Updated: 2026-03-24

Comments: 10 pages, ICLR 2026 poster paper

Journal ref: The Fourteenth International Conference on Learning Representations (ICLR), 2026

Code: https://github.com/XianweiC/DPI

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: Humans often juggle multiple, sometimes conflicting objectives and shift their priorities as circumstances change, rather than following a fixed objective function.

Terminology

Abstract

Humans often juggle multiple, sometimes conflicting objectives and shift their priorities as circumstances change, rather than following a fixed objective function. In contrast, most computational decision-making and multi-objective RL methods assume static preference weights or a known scalar reward. In this work, we study sequential decision-making problem when these preference weights are unobserved latent variables that drift with context. Specifically, we propose Dynamic Preference Inference (DPI), a cognitively inspired framework in which an agent maintains a probabilistic belief over preference weights, updates this belief from recent interaction, and conditions its policy on inferred preferences. We instantiate DPI as a variational preference inference module trained jointly with a preference-conditioned actor-critic, using vector-valued returns as evidence about latent trade-offs. In queueing, maze, and multi-objective continuous-control environments with event-driven changes in objectives, DPI adapts its inferred preferences to new regimes and achieves higher post-shift performance than fixed-weight and heuristic envelope baselines.

Sources

Related papers