Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision

arXiv:2610.00897 · cs.RO · Submitted 2026-10-01 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa, and with me are Dev and Taro, guest researcher.

Dev: Today's paper: "Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision".

Rosa: Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision proposes an adaptive method to dynamically regulate supervisory capacity and allocate robot supervision tasks to multiple human operators based on their real-time…

Dev: First, who's behind it and why it matters.

Title and authors: Rosa: Before we get into the specifics of how this works, let's talk about the paper "Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision" and who came up with it. The authors are Seabin Lee, Sujeong Park, Nayoung Kim, Sungjin Park, Haechan Jung, and Changjoo Nam.

Dev: I think the title itself is very descriptive; it clearly tells us that the paper is focused on making task allocation smarter by considering how humans are actually performing under pressure in complex robotic supervision scenarios.

Taro: The author lineup suggests a strong blend of expertise here, especially with people involved in autonomy and control, which hints at a deep dive into both the high-level planning and the low-level execution aspects of this system.

Rosa: Indeed, they seem to have covered all the bases from perception to control synthesis, which is exactly what you need when you're building something that needs to be robust in complex operational settings.

Dev: It’s interesting how their focus is on integrating human factors directly into the allocation mechanism rather than treating it as an afterthought, which suggests a more holistic design approach.

Taro: I think the real value here is seeing how they manage the complexity of multiple interacting systems—robots, multiple humans, and changing task demands—all at once.

Rosa: That’s right; their aim is to ensure that when things get busy, the system doesn't just break down due to human overload.

Dev: So, while we know they're dealing with a complex problem space—like trajectory planning or power flow issues from other papers we’ve seen—this paper focuses specifically on the human side of that complexity.

Taro: It sets a good precedent for how autonomy systems should think about their human partners; it needs to be aware of the operator's cognitive limits during execution.

Rosa: And I'm thinking about the broader implications: if this works, it means we can deploy more complex, multi-robot missions that rely on human supervisors without having to rigidly limit how many robots a person can handle.

Dev: It shifts the design philosophy from setting hard caps based on assumed human capability to dynamically adjusting resources based on observed reality.

Taro: That dynamic capacity adjustment is what makes it relevant when the mission profile changes, which is always true in real-world deployment scenarios.

Rosa: So, we're moving toward a system that is inherently more flexible and less brittle than the previous models we’ve discussed.

Dev: And that flexibility needs to be backed up by solid performance metrics, which I assume they provide in the rest of their work on this topic.

The paper's summary: Rosa: Moving on to what this paper actually says about the method, "Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision," it proposes a system that dynamically adjusts supervisory capacity based on the real-time cognitive states of the operators.

Dev: So, in simpler terms, they are using a method called HAMA to constantly measure an operator's mental workload and fatigue and adjust how many tasks that person is allowed to supervise concurrently.

Taro: I see it as an intelligent feedback loop where the system monitors human performance indicators—like workload and physiological signals—and uses that data to decide whether to increase or decrease the operator's capacity at set time intervals.

Rosa: That’s right; they use a specific update rule, calculating something called s n,o based on changes in workload indicators and task success metrics to make those capacity adjustments.

Dev: It’s quite clever because it combines subjective reports from the operator, like NASA-TLX scores, with objective behavioral measures such as eye-tracking data and blink counts for fatigue.

Taro: That fusion of subjective and objective data is key; it suggests that relying on just one method would give you an incomplete picture of what the operator is actually experiencing mentally.

Rosa: Exactly; they use those indicators to estimate visual demand through Area-Of-Interest dwell time, which helps them scale the difficulty of a task based on how much attention the operator is focused on.

Dev: So, when allocating tasks, they don't just look at urgency; they factor in these estimated human factors and the current capacity estimates for each operator to make a better decision.

Taro: Their allocation strategy is greedy and online; it means whenever there's a task waiting and an operator has available capacity, the system assigns it immediately based on calculated priorities.

Rosa: And they use task prioritization scores that consider factors like robot type, urgency, and even geometric constraints for things like deadlock resolution.

Dev: I’m paying attention to how they quantify the difficulty of different task types using historical data on average Area-Of-Interest dwell times measured in previous sessions to adjust the workload weight.

Taro: That historical scaling is important because it allows the system to understand that some tasks are inherently more taxing for certain operators than others, which is a very nuanced piece of information.

Rosa: It sounds like they’ve built a framework that treats supervisory capacity not as a fixed number but as a flexible variable that can respond to the actual human condition.

Dev: That dynamic nature is what separates it from previous approaches where you just set a static limit and hope it works for every operator in every situation.

The paper's improvements: Rosa: Now let's look at what the authors suggest as the specific improvements they’ve introduced in "Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision." The main improvement is shifting from a fixed supervisory capacity model to a dynamic, real-time adaptive capacity model.

Dev: So, they are proposing an AI system that will dynamically estimate the cognitive workload and fatigue of each human operator in real time using a multi-modal fusion approach involving NASA-TLX scores, AOI dwell time from eye-tracking, and blink counts.

Taro: That's a significant step because it moves beyond relying on just one source of data by combining those modalities to get a more robust estimate of the human state.

Rosa: And then they propose updating the effective supervisory capacity for every operator at predefined time intervals using an update rule that balances perceived workload against task success and physiological signals.

Dev: That update rule, which they derived from combining relative changes in workload, successful tasks, and eye blinks to calculate s n,o, is what allows the system to decide whether to increment or decrement capacity based on those real-time measurements.

Taro: The allocation strategy is also improved by incorporating a greedy approach that prioritizes robots requiring intervention based on task type and urgency scores.

Rosa: Furthermore, they refine the scoring mechanism by scaling the workload assigned to a specific task type using historical data from previous sessions to better reflect the relative difficulty of different tasks for each operator.

Dev: They also suggest incorporating additional constraints management during high-stakes tasks like deadlock resolution, where they propose dynamically reassigning robots within a cluster to those on the convex hull boundary for escape routes.

Taro: That constraint management feature is particularly interesting because it suggests the AI can handle complex spatial reasoning during critical events, not just task assignment.

Rosa: Overall, these improvements aim to build an AI system that prevents operator overload and burnout while maintaining high overall team performance by optimizing task distribution based on actual cognitive states.

Dev: So, the goal is to ensure that the system adapts its resource allocation in real time based on what’s happening with the humans, not just sticking to pre-set limits.

Conclusion: Rosa: So, wrapping up our discussion on "Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision," we see that the paper successfully treats supervisory capacity as a controllable variable instead of a fixed constant, allowing it to adjust based on real-time human state indicators.

Dev: That dynamic regulation is what allows HAMA to maintain balanced mental workload and prevent overload by continuously monitoring the operators' cognitive states.

Taro: I think the most impactful part for me is seeing how this system manages situations where the world misbehaves; it’s not just about routine tasks, but handling unexpected failures with appropriate human intervention support.

Rosa: And I think if we look at the experimental results, they showed a significant main effect on task performance

F(one twenty-nine) = four point three six two, p = zero point zero four six: , meaning HAMA led to a higher number of successfully completed tasks than the baseline strategy tested against it.

Dev: I also noticed that the system managed to reduce blink counts by about sixteen percent, which suggests lower behavioral signs of fatigue in the operators under HAMA compared to whatever fixed-capacity method they compared it against.

Taro: That reduction in fatigue indicators is a strong signal; it means the system isn't just managing workload; it’s actively supporting operator well-being during demanding operations.

Rosa: It sounds like this paper provides a solid foundation for building more intelligent, resilient human-robot teams capable of sustaining high performance over longer missions.

Dev: I agree, and I think the ability to adapt dynamically based on real data is what makes this method a practical tool for real-world deployment scenarios.

Taro: It’s definitely a step toward autonomy that accounts for the human element as an active participant in task execution, not just a passive recipient of commands.

Seabin Lee, Sujeong Park, Nayoung Kim, Sungjin Park, Haechan Jung, Changjoo Nam

Sogang University

cs.RO

Submitted: 2026-10-01

Updated: 2026-10-01

Comments: 10 pages, 7 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 77/100

The gist: Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision proposes an adaptive method to dynamically regulate supervisory capacity and allocate robot supervision tasks to

Key concepts

Operator Capacity Update Rule
This rule determines if an operator's supervisory capacity should increase or decrease by calculating a metric (sn,o) based on the change in combined workload and relative changes in successful tasks and eye blinks compared to the previous session. A positive change suggests increasing capacity, while a negative change suggests decreasing it.
Combined Workload Indicator (qo)
This indicator merges two real-time metrics: the NASA-TLX score, which measures subjective mental workload, and the operator's subjective cognitive fatigue score. This single value provides a comprehensive snapshot of how much mental effort an operator is currently exerting during a session.
Task Prioritization Scoring
Tasks are allocated greedily using a priority score based on task type and urgency. High-priority tasks, like docking, receive higher scores than lower-priority ones, ensuring critical operations are handled first. Deadlock task scores also incorporate distance from goal locations to encourage balanced robot distribution.
Area-Of-Interest (AOI) Dwell Time
This objective signal measures how long an operator focuses on specific interface elements during a session. This data is used to estimate the visual demand and difficulty of different task types, allowing the system to scale workload assignments accurately based on previous performance.

Terminology

Summary

Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision proposes an adaptive method to dynamically regulate supervisory capacity and allocate robot supervision tasks to multiple human operators based on their real-time cognitive states. This approach is significant because it addresses the limitations of existing methods that assume fixed operator capacity, leading to unbalanced workloads and potential overload in multi-human multi-robot supervision scenarios.

The gist: HAMA dynamically regulates supervisory capacity and allocates tasks in a way that maintains balanced mental workload, prevents overload, and improves overall team performance by using a greedy strategy that minimizes estimated operator workloads with task prioritization.

How it works

The Human-factor-Aware Multi-robot Allocation (HAMA) method consists of two main components: an algorithm for updating the operator capacity from real-time measurements and an allocation method based on those capacity estimates. The initial capacity is determined from the measured cognitive ability of each operator through 2-back tests, and this capacity is updated at predefined time intervals called sessions.

The update rule relies on combining several real-time human state indicators:

  1. The combined workload indicator: Let wo and fo denote the NASA-TLX score and the subjective cognitive fatigue score of operator o measured in the current session, respectively. These are combined into a single workload indicator qo = wo + fo.

  2. Relative changes: The method computes relative changes for workload, successful tasks, and eye blinks with respect to previous session values (lines 3–5).

  3. The core update metric: Using these terms, we compute sn,o = −∆qo + ∆uo − ∆bo in line 6, which determines whether the supervision capacity of operator o should be incremented or decremented. If the change is positive ("If ∆sn,o > sprev n,o), the system increases capacity by one up to the system limit; if negative (If ∆sn,o < 0"), it decreases capacity by one down to one.

Task Allocation Strategy

HAMA allocates supervision requests greedily in an online manner whenever the task queue Q is non-empty and at least one available operator is assigned fewer tasks than their current capacity. The allocation prioritizes robots based on a priority score calculated from task type and urgency.

The scoring mechanism for tasks incorporates several factors:

  1. Task type and urgency are primary ordering criteria, with the maximum possible score of the docking task is higher than the deadlock task as delayed intervention can lead to critical battery depletion.

  2. Deadlock task scores are calculated based on the distance from the goal locations, encouraging wider robot distribution.

  3. The workload assigned to a specific task type is scaled using historical data: we estimate the relative difficulty of different task types for each operator using their average AOI dwell times measured in the previous session. For instance, if an operator exhibited 30 seconds for docking and 45 seconds for deadlock in the previous session, a ratio of 1: 1.5 is applied to scale the workload weight.

Human Factor Measurement and Indicators

HAMA estimates human factors by integrating subjective reports with objective behavioral and physiological signals to provide individualized operator capacities. The indicators used include:

  1. Mental Workload: Measured subjectively by the NASA-TLX score (evaluating mental demand, physical demand, temporal demand, perceived performance, effort, and frustration).

  2. Cognitive Fatigue: Measured via a single-item subjective rating and physiological signals like blink count, which is noted to be non-intrusive and effective in detecting fatigue in real time.

  3. Visual Demand: Estimated using Area-Of-Interest (AOI) dwell time, which reflects how long an operator focuses on specific interface elements, used to estimate visual demand and scale task difficulty.

Experimental Results

The user study compared HAMA against a baseline strategy that maintained a fixed capacity of four and disabled attention-based weighting. The results demonstrated significant improvements in performance metrics under the HAMA algorithm:

  1. Performance: A significant main effect of the algorithm was found for task performance [F(1, 29) = 4.362, p = 0.046], indicating that HAMA led to a higher number of successfully completed tasks than the baseline.

  2. Fatigue Reduction: HAMA reduced blink counts by about 16%, indicating lower behavioral signs of fatigue.

  3. Workload Stability: NASA-TLX scores showed no significant effect of the algorithm, suggesting that HAMA improved performance without increasing perceived workload, as the system often assigned fewer concurrent robots per operator than the baseline.

Discussion and Conclusion

HAMA successfully treats supervisory capacity as a controllable variable instead of a fixed constant, allowing it to adjust based on real-time human state indicators.

Improvements for AI systems

Here are specific improvements that can be made to AI systems, based on the proposed Human-factor-Aware Multi-robot Allocation (HAMA) method:

  1. The core improvement is transitioning from a fixed supervisory capacity model to a dynamic, real-time adaptive capacity model for human operators.

  2. The improved AI system will perform the following specific functions:

  3. Dynamically estimate the cognitive workload and fatigue of each human operator in real-time using a multi-modal fusion approach (NASA-TLX scores, AOI dwell time via eye-tracking, and blink count).

  4. Update the effective supervisory capacity (maximum concurrent tasks) for every operator at predefined temporal intervals based on a learned update rule that balances perceived workload against task success and physiological signals.

  5. Implement a greedy, priority-based task allocation strategy that assigns robots requiring intervention (deadlock resolution or manual docking) to the operator who currently has the lowest estimated relative workload, adjusted by the relative difficulty of different task types (e.g., weighting deadlock resolution based on geometric constraints).

  6. Enforce dynamic precedence and constraint management during high-stakes tasks like deadlock resolution by dynamically reassigning robots within a cluster to those on the convex hull boundary, ensuring outer robots are prioritized for escape routes.

The resulting improved AI system will be capable of:

  1. Prevent human operator overload and burnout in long-duration, high-demand supervisory roles.

  2. Maintain higher overall team performance (throughput) by optimizing task distribution based on actual operator cognitive states rather than fixed limits.

  3. Reduce negative behavioral signs of fatigue (e.g., decreased blink counts) among human supervisors, leading to more reliable and sustained operation over extended missions.

Abstract

We propose a human-factor-aware method of allocating robot supervision tasks to multiple human operators. In scenarios where multiple operators occasionally teleoperate multiple robots to help the robots overcome difficulties, the allocation of the supervisory control tasks to humans needs to consider the real-time cognitive states of individual operators. However, most existing methods assume fixed supervisory capacity per operator and overlook fluctuations in the human factors such as workload and fatigue. As a result, workload distribution can be unbalanced where some operators become overloaded while the others remain underused. Our method dynamically regulates supervisory capacity and allocates tasks in a way that maintains balanced mental workload, prevents overload, and improves overall team performance. The allocation method uses a greedy strategy that minimizes estimated operator workloads with task prioritization. Robots are assigned to operators by reflecting their current supervisory capacity where the required effort depends on the types of tasks. In the user study, the analysis across predefined time intervals shows that the proposed method consistently achieves higher performance and lower behavioral signs of fatigue compared to a baseline method that does not consider human factors. These results highlight adaptive capacity adjustment as an effective preventive mechanism for sustaining operator performance in long-duration, high-demand settings.

Sources

Related papers