Cold-Start Active Preference Learning in Socio-Economic Domains

Active preference learning faces a significant hurdle known as the cold-start problem when no initial labeled data are available, which severely limits its performance in socio-economic domains. This work proposes a method that initiates learning with a self-supervised phase employing Principal Component Analysis (PCA) to generate initial pseudo-labels, followed by an active learning loop that strategically queries a simulated noisy oracle for labels, demonstrating superior performance compared to standard active learning strategies without prior information.

How it works

1. Data Preparation: The process begins with cleaning and preprocessing the raw input dataset through automatic (without supervision) and standard operations, including Feature Selection, Encoding Categorical Variables, and Handling Missing Values. This results in a cleaned dataset, denoted as D =

(xi, yi)n i=1, where yi is an associated scalar value from which preferences can be derived.

2. Warm-Up Phase: This phase bootstraps the learning process using the inherent structure of the cleaned data through PCA to generate surrogate preference labels. Specifically, it involves:

PCA for Trend Approximation and Surrogate Score Generation:

The first principal component (PC) is found by solving an optimization problem to find the weight vector w, which captures the primary axis of variation. The projection of each data point xi onto this component yields a scalar score ti: t = Xw, thus ti = xiw. This score ti serves as a surrogate value for generating initial preference labels.

3. Tuning the Strength of the Prior: To assess the fit, the paper analyzes the principal component reconstruction error, quantified by σ2r. The number of pairwise samples used for pre-training, Npre, is determined dynamically based on n, σ2r, and hyperparameters k and α: Npre = n · k / (1 + α · σ2r).

4. Residual-Based Pair Generation: The Npre pairs for model initialization are constructed using a weighted sampling strategy that prioritizes data points well-represented by PCA. The selection probability ps(xi) is inversely proportional to its residual ri: ps(xi) ∝ 1 / ri + ϵ, where ri = Xi − Xˆi2 is the Euclidean distance between the original point and its reconstruction. These pairs generate a PCA-labeled preference dataset, DPCA P =

Npre m=1.

5. Self-Supervised Model Initialization: An XGBoost binary classifier (XGBClassifier) is pre-trained using this PCA-labeled dataset DPCA P to yield the warmed-up model M0. This stage is self-supervised as no human-annotated labels are involved, minimizing the logistic loss L(θ) = − X Npre m=1 h lPCA uvm ln pθ(zuvm) + (1 − lPCA uvm) ln(1 − pθ(zuvm))i.

Warm-Start Active Learning

The final phase implements an iterative learning loop commencing with M0. This process proceeds in steps t (for t = 1, 2,..., Tmax):

1. Training Batch Request and Query Strategy Formulation: The model signals the sampler component to select a batch of Nb unlabeled pairs (xu, xv) from the pool. The sampler can use either random sampling or an uncertainty-based sampling approach, prioritizing pairs for which the current model Mt−1 exhibits high uncertainty.

2. Oracle Label Acquisition via Sampler-Expert Interaction: The selected batch is communicated to the simulated oracle. For each pair, the expert ascertains true underlying target values (yi and yj), and a Bradley-Terry (BT) model generates a (potentially noisy) preference label loracle uv, resulting in a newly oracle-labeled batch Bt = Nb k=1.

3. Incremental Model Update: The XGBoost model Mt−1 is updated using the new batch Bt to obtain Mt, leveraging XGBoost’s incremental training capability.

Oracle Simulation

To emulate human expert feedback, a simulated expert oracle generates realistic, potentially noisy preference labels. This is achieved through the Bradley-Terry (BT) model:

1. Probabilistic Preference Generation via Bradley-Terry Model: The BT model posits that the probability of one item i being preferred over another item j is Pr(xi ≻ xj yi, yj) = βi / (βi + βj), where βk is derived from the true target values yi and yj.

Here are the specific improvements that can be made to AI systems based on the proposed Cold-Start Active Preference Learning in Socio-Economic Domains framework, and what those improved systems could achieve:

The core contribution of this paper is a robust, computationally efficient method for preference learning in domains lacking initial labeled data. The following improvements focus on leveraging this framework to create more reliable, sample-efficient models:

1. mathbfEnhanced Cold-Start Robustness via PCA Warm-up (Self-Supervised Initialization):

The system can now initialize preference models using the first Principal Component Analysis (PCA) scores as surrogate labels, even when no expert input is available.

2. mathbfImproved Latent Structure Inference:

The improved AI system can infer an underlying, dominant latent construct (e.g., overall quality of life or socio-economic status) from raw data features without prior labeling. This allows the model to establish a strong inductive bias based on the data's intrinsic structure before any human labels are acquired.

3. mathbf Increased Sample Efficiency in Active Learning:

By using PCA-derived residuals to guide pair sampling, the active learning loop becomes highly targeted. The system will strategically query only those data pairs where the current model is most uncertain or where they lie far from the established principal component trend. This means a given annotation budget yields significantly more informative labels than random selection or standard uncertainty sampling alone.

4. mathbf Realistic Noise Modeling in Preference Elicitation:

The simulated oracle, powered by the Bradley-Terry (BT) model, allows the system to generate preference labels that mimic real-world expert fallibility and inconsistency (stochasticity). This moves beyond idealized binary labeling assumptions.

5. mathbf Adaptive Model Refinement via Incremental Training:

The framework utilizes XGBoost's incremental learning capability to iteratively update the model with each new oracle label. The system can continuously refine its preference predictions as it gathers data, ensuring that the final model is a cumulative representation of both initial structural knowledge (PCA) and specific expert feedback.

These improvements enable an AI system to perform the following functions:

1. mathbf Automated Preference Modeling in Data-Scarce Domains:

The system can build accurate preference models (utility functions or binary predicates) for complex socio-economic assessments (e.g., judging creditworthiness, market prices, or national happiness rankings) even when expert labeling is prohibitively expensive or impossible to start with.

2. mathbf High-Precision Ranking and Decision Support:

Because the model is initialized structurally sound (via PCA), it can provide reliable pairwise comparisons and relative rankings of alternatives in a specific domain (e.g., ranking football players by market price) with high accuracy, even when the training data is sparse.

3. mathbf Optimized Resource Allocation for Data Collection:

In scenarios where human expert time is limited, the system minimizes the number of required labels to reach a target performance threshold (Low-Data Regime). This drastically reduces annotation costs and time while maximizing the utility of every acquired label.

4. Retroactive Model Validation: The inclusion of comparison benchmarks (like GPT-based initialization) allows researchers to quantitatively assess how much performance gain is achieved by using this computationally efficient PCA warm-up versus more complex, state-of-the-art unsupervised methods, validating the trade-off between simplicity and performance.