SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking

arXiv:2609.06061 · cs.RO · Submitted 2026-09-05 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking".

Dev: Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: "it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint,

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: So we've seen how this method handles terminal navigation problems, and now I want to focus on what the authors actually claim in the summary of "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking." Essentially, they are describing a fix for a major problem called "feasibility collapse."

Dev: Feasibility collapse sounds like when the agent gets stuck because it prioritizes surviving over actually reaching its destination, which is a classic dilemma in survival-weighted objectives.

Taro: I think that happens when the constraints get very tight near the goal, and if you're outside the goal region, staying there is safer than trying to enter a zone where a small violation would kill you. That’s what they are describing as the failure mode of standard CaT.

Rosa: Right; so SCoCaT introduces something new—a per-step success indicator—to break that cycle, making the system actively work toward docking while still respecting those safety boundaries.

Dev: The paper formalizes this by defining the core of SCoCaT using an indicator called Isucc,t, which checks if the vehicle is within certain tolerances for position, orientation, velocity, and angular rate all at once.

Taro: That signal seems really clever because it’s not just checking one thing; it’s evaluating multiple docking pose tolerances simultaneously in real-time.

Rosa: It sounds like they are using a value critic, Vs, trained specifically against this success indicator to guide the policy gradient when the agent is near that goal corridor.

Dev: That means you don't need to change the fundamental termination mechanism itself; you just add this auxiliary signal into the learning process, which keeps things much simpler for deployment.

The paper's summary: Rosa: Moving on from how they fixed the collapse, let's talk about what specific improvements SCoCaT offers over previous methods and why that matters for future research in this area.

Dev: The paper highlights a few key advantages, one being that SCoCaT doesn't just fix the collapse; it actually improves task engagement. They showed on the CubeSat that SCoCaT reached zero point six one three declared success at zero point nine four five compliance, which they compared to unconstrained PPO’s success at ninety-seven percent of CaT’s compliance.

Taro: That comparison is telling because it shows a significant improvement in task completion rate—that sixty-six percent versus the unconstrained performance—while still maintaining high safety standards. That suggests the method isn't just a bandage; it’s actually better at achieving the mission objective.

Rosa: It sounds like they are showing that you can have both high corridor entry and sustained docking at once, which is exactly what we need for complex maneuvers in space exploration or intricate robotic assembly.

Dev: From a control loop perspective, the authors showed that their combined advantage scalarization breaks the hover equilibrium because the combined advantage becomes positive when Vs (sG) is greater than R over one minus gamma. That means it actively fights stagnation.

Taro: So, when things get difficult and the agent tries to hover indefinitely outside the goal region, this new scalarization term kicks in and pushes it back toward success. That’s a really interesting feature for handling unexpected world misbehavior.

The paper's improvements: Rosa: Alright, we're coming to the end of our discussion on "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking." To wrap up, I want to summarize the main implications and what this means for the field moving forward.

Dev: The paper really confirms that you can use termination-based RL effectively even when you have complex, tight safety constraints that get more restrictive as you approach a goal.

Taro: It implies that we don't have to sacrifice either safety or task performance when dealing with terminal navigation tasks; we can actually achieve both simultaneously with this structural fix.

Rosa: Exactly; the fact that they showed zero-shot sim-to-real transfer confirms that this isn't just a simulation trick, but a generalizable approach applicable across different physical embodiments and hardware.

Dev: It means we can trust these methods more when moving from lab tests to actual missions where you can’t afford to fail online during the critical final approach phase.

Taro: I just want to add that the limitations they noted, like the angular-velocity sim-to-real gap and the bimodal seed distribution on that CubeSat, are important for us because they show exactly where we still need more research before we can fully trust this across all hardware platforms.

Rosa: That’s a fair point; knowing those gaps is crucial so we know what to target next in our research. So, folks, "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking" is a significant piece of work that provides a principled resolution to feasibility collapse in termination-based constrained RL for terminal-navigation tasks.

Dev: It’s certainly a solid contribution to the area, and I think we should all be very optimistic about how this impacts our ability to deploy safer autonomous systems in constrained environments.

Taro: I agree; this paper gives us a much clearer path toward building more capable autonomy that doesn't just survive, but actively succeeds in complex physical interactions.

Rosa: Well, that’s all the time we have for today; thank you all so much for tuning in to discuss "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking." We'll see you on the next paper soon.

Conclusion: Rosa: So, to wrap things up on "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking," this paper essentially introduces a novel way to solve the feasibility collapse problem that plagues many termination-based RL methods in terminal navigation.

Dev: Yeah, it’s a clever augmentation using a per-step success indicator, Isucc,t, to keep the agent focused on actually reaching the goal while still respecting those crucial safety constraints during the final approach.

Taro: I think that ability to decouple task success from survival weighting is what makes it so powerful; it stops the system from just prioritizing "staying alive" over "getting there."

Rosa: It really does show how vital this structural change is for safety-critical deployments, especially when we're dealing with hardware like spacecraft where constraints tighten rapidly near the target.

Dev: From a control standpoint, that means we can design policies that actively fight stagnation rather than just passively waiting out a survival timer; I’m really interested in how stable those combined advantage scalarizations are during rapid maneuvers.

Taro: I agree; it shows we can engineer systems where the reward signal is perfectly balanced between reaching the goal and adhering to hard limits, which is something we need when the world misbehaves unpredictably.

Rosa: And that zero-shot sim-to-real transfer validation across different platforms really gives us confidence that this isn't just a lab result; it’s actually applicable hardware science.

Dev: It tells me we can probably deploy these kinds of constrained systems on smaller, resource-limited platforms because the complexity is managed by embedding the logic in the termination mechanism itself rather than needing massive runtime solvers.

Taro: It opens up possibilities for autonomous systems that need to operate reliably in real-world environments without having to run computationally intensive optimization routines during inference.

Rosa: That’s what I want to emphasize; this work on "SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking" provides a tangible path forward for building robust autonomy in constrained settings.

Dev: It's certainly a solid piece of research, and I think the implications for real-world deployment of these systems are quite substantial.

Taro: I agree; it gives us a much clearer framework for designing agents that can handle complex physical interactions with both safety and performance as primary objectives.

Rosa: And that’s all the time we have for today on this fascinating paper; next week, we’ll be taking a look at some work on finite-horizon approximations in LQ games.

SnT - Interdisciplinary Centre for Security, Reliability and Trust University of Luxembourg

cs.RO

Submitted: 2026-09-05

Updated: 2026-09-24

Comments: Accepted at CoRL 2026 Conference (https://www.corl.org/)

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 90/100

The gist: Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: "it avoids online optimization at inference, scales easily to many constraints via a single

Key concepts

Feasibility Collapse: This is a failure mode where an agent gets stuck because it prioritizes surviving over actually reaching its destination. It happens when constraints are very tight near the goal, causing the agent to stay outside the goal region for safety rather than attempting entry.
Isucc,t: This is a per-step success indicator used in SCoCaT that checks if the vehicle is within tolerances for position, orientation, velocity, and angular rate simultaneously. It evaluates multiple docking pose tolerances in real-time to guide the learning process.
Success Conditioned Constrained Reinforcement Learning: This type of reinforcement learning uses termination-based constraints where success is conditioned on meeting specific success criteria. SCoCaT modifies this by conditioning the termination mechanism on a per-step success indicator, allowing agents to actively work toward docking while respecting safety boundaries.
Zero-shot Sim-to-Real Transfer: This refers to the ability of the method to transfer effectively from simulation to real hardware without extensive retraining. The paper confirms this capability, suggesting the approach is generalizable across different physical embodiments and hardware platforms.

Terminology

Reinforcement Learning — 32× in this paper · explained in 4 episodes
Floating Platform — 27× in this paper
Sim-to-Real Transfer — 14× in this paper · explained in 2 episodes
Ablation — 11× in this paper · explained in 1 episode
Core — 5× in this paper · explained in 2 episodes
Task Completion Rate — 5× in this paper
Reward Shaping — 4× in this paper · explained in 2 episodes
Sim-to-Real Gap — 3× in this paper · explained in 6 episodes
Ablation study — 2× in this paper · explained in 1 episode
Decoupling — 2× in this paper · explained in 2 episodes
Deep Reinforcement Learning — 2× in this paper · explained in 2 episodes
Goal Corridor Occupancy — 2× in this paper
Ablation Studies — 1× in this paper · explained in 1 episode
Activation Function — 1× in this paper · explained in 1 episode
Actor — 1× in this paper · explained in 3 episodes
Adam — 1× in this paper · explained in 2 episodes
Adaptation — 1× in this paper · explained in 1 episode
Adaptive Weighting — 1× in this paper · explained in 1 episode
Aggregation — 1× in this paper · explained in 1 episode
Foundation Model — 1× in this paper · explained in 3 episodes
Foundation Models — 1× in this paper · explained in 7 episodes
Reward Hacking — 1× in this paper · explained in 8 episodes
Robustness — 1× in this paper · explained in 5 episodes
Zero-Shot Transfer — 1× in this paper · explained in 4 episodes

Summary

The authors formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this “feasibility collapse.” They propose Success-Conditioned Constraints as Terminations (SCoCaT), an extension of CaT that preserves its deployment-time and implementation simplicity. This is achieved by including a per-step success indicator independent from CaT’s survival weighting, which is an auxiliary signal. Specifically, the core of SCoCaT is defined by the per-step success indicator: "Isucc,t = 1[∥pt − pG ∥ < εp] · 1[eorient,t < εo] · 1[∥vt ∥ < εv] · 1[∥ωt < εω]" (Equation 3). This signal evaluates whether the vehicle currently satisfies the docking pose tolerances simultaneously. A value critic, Vs, is trained against this signal and provides a dense learning gradient at goal-corridor states without modifying the termination mechanism.

The methodology considers a Markov decision process M augmented with constraint functions: "The safety requirement is that each constraint be violated with bounded occupancy probability, ∀ i ∈ I, P(s,a)∼ρπγ [ci (s, a) > 0] ≤ ε̃i. This chance-constrained formulation is handled via CaT’s termination-based reformulation: JCaT (π) = Eτ ∼π γ(1 − δt′) r(st, at), t=0" (Equation 2).

The paper demonstrates the effectiveness of SCoCaT through empirical validation across two representative spacecraft platforms: a 6U-CubeSat spanning the mass and degree-of-freedom envelope of operational proximity operations, and a floating platform testbed for zero-shot sim-to-real transfer in our laboratory.

Key findings from the experiments include:

"On the Floating Platform, SCoCaT achieves a TCR of 100% and constraint compliance of 98.8%, versus CaT’s 0.4% TCR at a marginally higher compliance of 99.4%, demonstrating a 250× improvement in task engagement with no regression in safety."

"On the CubeSat, SCoCaT mitigates this collapse, reaching 0.613 declared success at 0.945 compliance, which is 66% of unconstrained PPO’s success at 97% of CaT’s compliance, with the residual compliance gap to CaT confined to line-of-sight."

The paper concludes that SCoCaT recovers high corridor entry and sustained docking while preserving constraint compliance and that Zero-shot sim-to-real transfer confirms the approach generalizes beyond simulation. The analysis shows that the combined advantage scalarization, where the policy gradient uses a single scalarized advantage, breaks the hover equilibrium because "the combined advantage is γ[Vs (sG) − R/(1 − γ)], which is positive once Vs (sG) > R/(1 − γ), thereby breaking the local optimality of the hovering policy. Furthermore, ablation studies show that The continuous signal floors at 0.67 rather than reaching 1, and a finite horizon H with hold length T tightens the requirement to R < (1 − γ T)/(1 − γ H−k), i.e. R < 0.64 in the worst case; our R ≤ 0.099 clears all three."

In summary, SCoCaT is established as a principled resolution to feasibility collapse in termination-based constrained RL for terminal-navigation tasks, and it validates its generality across different embodiments and task families through sim-to-real transfer. The limitations noted include the angular-velocity sim-to-real gap and the bimodal seed distribution observed on the 6U CubeSat. (The full summary is derived from pages 1–22.)

(Self-Correction/Refinement based on strict instruction: The request asks for just the summary and to Quote the relevant parts. The above text is a detailed extraction adhering to those rules.)

Final Output Generation:

The authors formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this “feasibility collapse.” They propose Success-Conditioned Constraints as Terminations (SCoCaT), an extension of CaT that preserves its deployment-time and implementation simplicity. This is achieved by including a per-step success indicator independent from CaT’s survival weighting, which is an auxiliary signal. Specifically, the core of SCoCaT is defined by the per-step success indicator: "Isucc,t = 1[∥pt − pG ∥ < εp] · 1[eorient,t < εo] · 1[∥vt < εv] · 1[∥ωt < εω]" (Equation 3). This signal evaluates whether the vehicle currently satisfies the docking pose tolerances simultaneously. A value critic, Vs, is trained against this signal and provides a dense learning gradient at goal-corridor states without modifying the termination mechanism.

(Self-Correction/Refinement: The request asks for just the summary and to Quote the relevant parts. The above text is a detailed extraction adhering to those rules.)

Improvements for AI systems

As a fastidious researcher, I have analyzed SCoCaT (Success Conditioned Constraints as Terminations) and its contributions. The core innovation is resolving feasibility collapse in terminal navigation tasks by decoupling safety constraint enforcement (via termination-based CaT) from the task completion objective using an auxiliary success critic.

Here are the specific, high-impact improvements to AI systems based on this research:

The ability to perform terminal navigation (e.g., spacecraft docking, robotic peg-in-hole insertion) while simultaneously guaranteeing a high degree of safety compliance under strict Size, Weight, and Power (SWaP) constraints.
The development of robust Sim-to-Real transfer capabilities for complex manipulation and proximity operations on physical hardware with limited computational resources (e.g., CubeSats).
Creation of Success-Conditioned control policies that actively seek and sustain goal configurations, rather than merely avoiding constraint violations when near the target region.

Specific AI System Capabilities Enabled:

A spacecraft autonomous system capable of executing a precise docking maneuver where the policy is inherently designed to enter and maintain the goal corridor (high Task Completion Rate) without violating critical velocity or angular rate constraints (high Constraint Compliance).
A robotic manipulator system capable of performing peg-in-hole insertion tasks in complex geometries, where the policy learns to satisfy geometric tolerances while avoiding collisions, even when reward shaping is sparse near the goal.
An autonomous drone/UAV system operating in cluttered environments (obstacle avoidance) that maintains precise proximity to a target while ensuring collision constraints are never violated during the final approach phase.
A reinforcement learning agent that can operate effectively on resource-constrained embedded platforms (like small satellites) by relying only on a single neural network forward pass for inference, as the safety logic is structurally embedded in the termination mechanism rather than requiring runtime QP solvers.
An advanced policy optimization framework that utilizes decoupled advantage streams—one tracking constraint satisfaction (via CaT) and another tracking goal achievement (via SCoCaT)—to ensure that the agent learns a trajectory that maximizes both metrics simultaneously, breaking the traditional trade-off where safety and reward are in direct competition.

Abstract

Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint, and is simpler to implement than commonly used Lagrangian methods. Instead of pricing violations through summed cost penalties, this approach makes violations structurally unprofitable by shortening the effective horizon for each violation. We identify a structural failure mode of this method class on terminal-navigation tasks: reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach. When the goal sits inside the region close to where the constraints become active, the survival-weighted objective makes dwelling outside the goal region strictly preferable to entering, producing high constraint compliance with low task completion. We formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this ``feasibility collapse''. We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance. Validation across two representative spacecraft platforms: a 6U-CubeSat spanning the mass and degree-of-freedom envelope of operational proximity operations, and a floating platform testbed for zero-shot sim-to-real transfer in our laboratory, supports the generality of these findings.