Designing for the Next Click: Bandits for Real-Time Page Layout

arXiv:2608.29850 · cs.LG · Submitted 2026-08-30 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Designing for the Next Click: Bandits for Real-Time Page Layout".

Jane: The paper was written by Bhavtosh Rath, Harshith Narasimhamurthy, Bob Eisinger, Cole Stiegler, Adnan Awow et al. from Target Corporation.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: So, let's talk about the title and what it implies before we even look at the abstract; "Designing for the Next Click" suggests they are optimizing for immediate user action. Jane, how does this concept fit into a broad e-commerce context?

Jane: It fits because every second of a shopper’s time on a page is valuable, and by optimizing the layout to be more relevant, we can guide them toward that next crucial interaction. The title promises that they aren't just shuffling boxes around but are actively designing for the most likely path to purchase.

Lu: The "Bandits" part of the title suggests an inherent balance between exploration and exploitation, meaning we’re not just showing what works now, but we’re also willing to test things that might work better in the future. That is a really powerful way to manage uncertainty.

Meng: From a practical standpoint, it sounds like they are saying that the layout itself—the structure of the content—is an optimization problem solvable by AI, which is a huge shift in how we think about web design. How do they manage that balance practically?

Lalam: I think the title implies moving away from fixed paths; instead, it suggests that every single click is an opportunity for the platform to learn and for us as users to have a tailored experience based on what we might want next.

Summary: Tom: Now, let's look at the summary of "Designing for the Next Click: Bandits for Real-Time Page Layout" and focus on what they actually achieved in practice. Jane, the abstract mentions testing this system on entry product pages, which is a challenging environment.

Jane: It’s challenging because these users arrive without any history or login information, meaning we have very little to go on. But the summary shows that even in this low-information state, the AI can identify patterns and deliver results.

Lu: The positive lift in session-level performance is quite impressive, suggesting that even though the context is limited, the dynamic nature of their approach allows us to see a significant improvement over static rules.

Meng: I'm very focused on those numbers: a four point nine percent increase in click-through-rate and a one point five percent lift in cart adds—those are very concrete metrics that demonstrate real business impact from this AI system.

Lalam: The success there is remarkable because it suggests that the power of contextual bandits isn't just to predict what works, but to find effective patterns where human intuition might fail entirely ignore the lack of user data.

Improvements/Methodology: Tom: The system in "Designing for the Next Click: Bandits for Real-Time Page Layout" is quite sophisticated, so let’s break down how they implemented this complex logic. Jane, what is the core mechanism that allows them to balance exploration and exploitation?

Jane: They are using a contextual bandit model called LinUCB; it acts like an intelligent traffic cop that allows us to explore different layout options while still prioritizing the ones we know perform well. It’s not just picking one random thing.

Lu: The way they represent the page layout as a set of modular arms—allowing them to rank eighteen possible modules and then select the optimal seven for display—is truly clever because it breaks down a massive problem into manageable pieces.

Meng: I'm really interested in the technical requirements; maintaining a p95 latency of under 25ms while running this complex, dynamic bandit logic is a significant engineering feat that demands very efficient microservices.

Lalam: It feels like they are not just optimizing product placement but optimizing the entire presentation itself, making the experience feel like a continuous conversation between an AI and the customer.

Conclusion: Tom: We’ve seen how LinUCB works in practice, so let's quickly wrap up this discussion on "Designing for the Next Click: Bandits for Real-Time Page Layout." Jane, what is the ultimate takeaway for our listeners?

Jane: The paper shows that dynamic layout optimization is not only feasible at scale but also has a strong commercial potential to significantly improve engagement.

Lu: I think the major breakthrough is moving beyond optimizing individual components and actually optimizing the entire experience of composition itself, which is a huge conceptual leap.

Meng: From a practical standpoint, this means building robust systems that can handle massive traffic spikes while continuously improving their performance without constant manual tweaking by human staff.

Lalam: This provides a blueprint for how technology can better serve the diverse needs of all users, suggesting we are moving toward an era where digital platforms are truly responsive to human context.

Tom: That’s exactly right, Lalam; we're not just making things look prettier, we're making them actively respond to what the user intends to do next.

Lu: I'm excited to see how others will build on this foundation and explore the even more complex possibilities of full-page optimization.

Meng: And I think we can’t wait for practical implementations, seeing how this applies in the real world, adding another layer of excitement.

Lalam: We hope that this sets a wonderful precedent for better user-centric design across all digital platforms and serves as a model for how technology can serve the diverse needs of everyone.

Bhavtosh Rath, Harshith Narasimhamurthy, Bob Eisinger, Cole Stiegler, Adnan Awow, Amit Pande,

Target Corporation

cs.LG

Submitted: 2026-08-30

Updated: 2026-08-30

Comments: Accepted to The Web Conference 2026 (short paper track), but later withdrawn due to internal prioritization. Subsequently accepted to the Online & Adaptive Recommender Systems Workshop (held in conjunction with the 20th ACM Conference on Recommender Systems, RecSys 2026)

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 84/100

The gist: The paper introduces the application of contextual bandits to solve the complex problem of real-time page layout optimization in e-commerce environments, moving beyond simple module ranking to adapt

Key concepts

Contextual Bandits: This is a machine learning model used to balance exploration and exploitation. It allows the system to test new layout options (exploration) while prioritizing those that are known to perform well (exploitation), managing uncertainty effectively.
LinUCB: A specific contextual bandit model used in the study. It acts like an intelligent traffic cop, allowing the system to explore different layout options while still prioritizing high-performing ones. This ensures dynamic optimization is efficient and effective.
Dynamic Layout Optimization: The process of using AI to change a website's structure or presentation in real-time based on user intent. Instead of fixed paths, every click becomes an opportunity for the platform to learn and tailor the experience.

Terminology

Contextual Bandits — 18× in this paper · explained in 3 episodes
Regret — 7× in this paper · explained in 3 episodes
Personalization — 6× in this paper · explained in 2 episodes
Reward Function — 4× in this paper · explained in 2 episodes
Core — 3× in this paper · explained in 2 episodes
Order Conversion — 3× in this paper
Adaptive Layout — 2× in this paper · explained in 1 episode
Embeddings — 2× in this paper · explained in 2 episodes
Reinforcement Learning — 2× in this paper · explained in 4 episodes
Robustness — 2× in this paper · explained in 5 episodes
Adam — 1× in this paper · explained in 2 episodes
Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
Interpretability — 1× in this paper · explained in 15 episodes
Inverse Propensity Weighting — 1× in this paper
Q-learning — 1× in this paper · explained in 3 episodes
Semantic Similarity — 1× in this paper · explained in 2 episodes

Summary

The paper introduces the application of contextual bandits to solve the complex problem of real-time page layout optimization in e-commerce environments, moving beyond simple module ranking to adapt the entire user experience based on context. This approach allows e-commerce interfaces to continuously learn from user interactions and tailor layouts to user context, providing a scalable path toward adaptive page design that maximizes commercial value by optimizing the presentation of information modules.

Performance Measurement and Core Methodology

The system utilizes contextual bandits, specifically employing a LinUCB-driven variation, to optimize recommendation module layouts. Since rewards for unchosen arms are not observed in logged bandit data, the authors report a proxy measure called pseudo-regret. This proxy is computed using predicted expected rewards: ̂ t = a in A t t(x t, a) - t(x t, a t), where t is the predicted expected reward of an arm a given the context x t. Furthermore, the initial implementation employs Inverse Propensity Weighting (IPW) to partially correct for position bias, although the authors acknowledge that IPW is known to exhibit high variance when propensity scores are small.

Empirical Results and Business Impact

The experimental results demonstrate that the contextual bandit approach successfully translates higher user engagement into tangible commercial value. Figure 3(b) illustrates the daily percentage lift in add-to-carts per visitor (ATC/V), showing that LinUCB-driven variation consistently matched or outperformed the control across the experiment. The treatment maintained a modest but persistent edge, peaking at day 5—coinciding with increased site activity over the weekend. This pattern was replicated by other metrics, as "DPV & CTR also followed a similar pattern, indicating that higher engagement translated into incremental commercial value rather than short-term exploration noise." Overall, these results suggest that the adaptive layout design is highly practical and scalable.

Future Directions for Robustness and Scope Expansion

The authors identify several critical areas for future improvement to enhance the robustness and scope of the system. These planned advancements include:

Advanced Off-Policy Estimation: Moving beyond simple IPW, future iterations will investigate more robust off-policy estimators, such as ‘Self-Normalized Inverse Propensity Scoring’ and ‘Doubly Robust’ estimation, which are designed to provide lower-variance and more reliable reward estimates.
Multi-Objective Reward Functions: The current optimization is primarily focused on click-through rate (CTR). Future work will explore multi-objective reward functions that directly optimize downstream business metrics such as Add-to-Cart, Order Conversion, and Demand Per Visitor, thereby better aligning online learning with long-term business objectives.
Whole-Page Optimization: While the current architecture optimizes recommendation modules independently, the ultimate goal is to investigate whole-page optimization, where multiple page components are jointly optimized rather than ranking recommendation modules independently.

By continuously evolving both the model and its contextual features—such as incorporating richer behavioral signals—the system aims toward a comprehensive vision of adaptive layout optimization.

Improvements for AI systems

[Confidential Research Memo: System Architecture Upgrade]

To: Engineering Leads / Product Strategy

From: AI Research Division

Subject: Critical Architectural Upgrades for Contextual Bandit Deployment in E-commerce Optimization (Moving Beyond V1.0 Limitations)

The current framework is academically sound but carries significant deployment risks due to reliance on biased off-policy estimation and narrow objective functions. To achieve reliable, high-ROI production performance that justifies the infrastructure investment, we must implement the following three major architectural upgrades:

The Problem: The current use of Inverse Propensity Weighting (IPW) introduces unacceptable variance in off-policy evaluation, particularly when user actions are rare or the propensity score (pi(ax)) is low. This high variance makes reliable A/B testing and model updates infeasible at scale.

The Improvement: Replace the IPW estimator with a Doubly Robust (DR) Estimator.

What the Improved AI System Can Do:

Reliable Causal Estimation: The system will compute a low-variance, unbiased estimate of the expected treatment effect (E[R(a)x]) by combining two estimators: one based on propensity modeling and another based on direct outcome modeling (e.g., a regression model predicting reward).
Robust Model Updates: We can confidently estimate the true incremental value of any module or layout change, even when that change was rarely shown during the initial exploration phase (i.e., when pi(ax) is near zero). This drastically reduces the financial risk associated with deploying novel layouts.
Deployment Mechanism: The system must incorporate a dedicated Causal Inference Microservice layer that takes raw logs (x, a, r) and outputs the stabilized, counterfactual reward estimate DR.

Summary of Deliverables: The resulting system will transition from a simple module recommender to a Causal, Revenue-Optimizing, Full-Page Experience Generator, capable of providing statistically rigorous evidence for every layout change deployed.

Abstract

E-commerce platforms increasingly personalize user experiences through machine learning, yet page layout decisions remain dominated by static rules and manual curation. We present a scalable bandit-based system that optimizes product page layouts in real time while preserving human control over design intent. A contextual bandit model dynamically selects the most effective layout for each session using user, item, and category-level features. The system leverages a LinUCB-based policy to balance exploration and exploitation as it learns from live user interactions. The architecture is designed for seamless integration into large-scale web serving stacks, supporting low-latency inference and continuous model updates. The system was first tested on entry product pages. In online A/B deployments on a major retail platform, our approach achieved positive lifts in session-level performance metrics over a strong heuristic baseline. Our results demonstrate that contextual bandits can effectively optimize visual and structural aspects of product discovery for user engagement, providing a scalable path toward learning-to-design the web.