Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control

arXiv:2609.39594 · cs.RO · Submitted 2026-09-30 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control".

Dev: Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control (NEUPRO) proposes a neuro-symbolic framework that represents safety specifications as interpretable first-order logic rules, allowing for flexible,

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: So, we're diving into "Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control," which is quite a mouthful. Basically, this paper tackles the problem of making robot safety rules transparent instead of using those dense cost functions that are hard to understand.

Dev: Right, Rosa? It seems the core thesis is about representing safety requirements as interpretable first-order logic rules so we can get gradients flowing through them to ground learned features in human-understandable semantics.

Taro: From my side, I'm curious how this helps when the world misbehaves; what does this system actually do when it encounters something unexpected that violates a rule?

Rosa: Well, the paper suggests they propose "Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control" as NEUPRO, which uses a differentiable reasoner to learn reusable safety representations from human-specified safety knowledge. They claim this lets practitioners express task-related safety requirements as transparent symbolic rules.

Dev: And what's the mechanism behind making those gradients flow back into the Logical Predicate Model, or LPM, inferred from a visual foundation model like Grounding DINO? That part seems crucial for linking the symbols to the pixels.

Taro: I think that ability to map raw observations into grounded atoms based on those rules is what makes it powerful; it means the robot isn't just following a pre-set path, but actually reasoning about why an action might be unsafe in that specific scene.

Rosa: Exactly, and they show that this formulation allows the learned representation to be explicitly grounded in interpretable safety concepts rather than just fitting an opaque label. They suggest this approach enables object-level and task-level generalization without needing to retrain the whole system.

Dev: That sounds promising for deployment, but I have to ask about the practical side—Rosa, how long does this system actually run outside of a highly controlled lab environment? What are the latency concerns we need to watch out for?

Taro: That's a big question, Dev; if it relies on complex graph reasoning and image grounding models, I wonder how robust it is when things aren't perfectly labeled or when the scene changes rapidly during execution.

Paper summary: Rosa: The paper focuses on training the system using "differentiable rule evaluation as a structured supervision signal," which they use to match rule-level safety satisfaction, which is key for learning from partially annotated data. This suggests it's designed to be more flexible than methods that need perfect labeling across every single scene.

Dev: So, if we look at the architecture, the graph-based differentiable reasoner uses nodes like a Conjunction Node and a Disjunction Node to handle logical compositions of predicates within each mini-batch. That sounds like it's trying to manage the complexity of many rules efficiently in real time.

Taro: Managing that sparsity in the reasoning graph is important because if the graph gets too dense, we lose the interpretability we’re aiming for, and I worry about failure modes when the system has to make a tough choice between conflicting safety constraints.

Rosa: They handle literal polarity through an affine transformation mapping continuous valuations to a range of

sign, bias: values, which helps in representing both positive and negated literals within the logic structure. It's a clever way to encode that symbolic information into the continuous output of the LPM.

Dev: Speaking of those outputs, they produce a "rule-satisfaction matrix P" where each entry is predicted as the probability that an observation satisfies a specific rule Fi, which feeds into their masked binary cross-entropy loss. How does this probability translate directly into a reliable action decision for the robot?

Taro: I see it as providing an explicit explanation of safety violation; if the model predicts a low satisfaction probability for a specific rule, we have grounds to flag that action as potentially unsafe and investigate why.

Rosa: That's the bigger implication, Taro; moving away from black-box cost design means we get these explicit explanations about what the robot is thinking in terms of safety constraints, which is vital for building trust in deployed systems.

Paper summary: Dev: I'm still focused on the engineering reality; if we’re running this on a mobile platform, maintaining a low loop rate while performing this complex graph construction and evaluation needs to be really efficient.

Taro: The real-world impact could be in making robots safer in unstructured environments where pre-programmed rules can't cover everything, because the system learns to compose new safety logic from existing knowledge.

Rosa: That leads us nicely into what these authors suggest about the future work of Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control. They hint that they are pushing towards achieving robust performance on unseen objects and transferring this learned safety knowledge across different tasks without needing a complete retraining cycle.

Dev: If they can achieve that kind of generalization, it really shifts the burden from constant manual rule updating to having a system that can adapt its safety understanding on the fly. That would significantly reduce our maintenance overhead for new environments.

Taro: I agree; if the system maintains high accuracy when applied to unseen objects or different tasks, it means we build a safety foundation once, and it scales better across the entire robotics domain.

Rosa: So, to wrap up this discussion on Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control, we've seen how they use differentiable reasoners to link visual inputs to interpretable logic rules. The authors suggest that the real value lies in getting those explicit explanations of why a robot action is deemed safe or unsafe.

Dev: I think it’s important to remember that while they show strong performance in predicate grounding accuracy and task accuracy when composing predicates, we still need to figure out the long-term reliability and latency under high-speed real-time operational conditions.

Taro: We also need more data on how well this system handles truly novel situations where no pre-existing safety rules apply, which is a key area for future development in autonomy research.

Rosa: That's what we'll be looking at next; the implications of this work are huge because it moves us closer to having robots whose safety logic is as transparent and verifiable as human-written specifications.

Conclusion: Rosa: So, we've been looking at how this Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control paper works, and now we need to talk about what that title really means and who put it together.

Dev: Yeah, I'm thinking about the implications of having a system that can learn safety rules from raw visual data, Rosa. How does that translate into actual robot behavior in the real world?

Taro: From my side, I'm focused on what happens when things go wrong; if this system has to navigate a situation where no pre-defined rule applies, how does its reasoning hold up?

Rosa: It really shows a move toward giving robots safety logic that looks and feels more like human understanding. The authors put together the paper with some very smart people who are clearly pushing the boundaries of both machine learning and formal logic.

Dev: I saw the methodology involves a differentiable reasoner that lets gradients flow through symbolic rules, which is interesting for my control concerns because it suggests a way to supervise complex reasoning without needing perfect manual labels for every single scene.

Taro: That ability to ground learned features in those interpretable logic rules is what excites me most; it means we’re not just training a black box, but building something that has traceable safety constraints.

Rosa: Exactly, and the way they handle those continuous valuations from the Logical Predicate Model really lets us see the "soft truth values" of objects as they appear in the visual input. It’s a nice bridge between pixels and formal logic.

Dev: I'm still thinking about how this would perform under high-speed operation; we need to know if this reasoning graph construction keeps up with a fast loop rate on actual hardware, or if the latency becomes an issue during critical maneuvers.

Taro: That’s a valid concern for deployment; if the reasoning step adds too much delay, it defeats the purpose of real-time safety control in dynamic environments.

Rosa: So, we've seen how this framework lets us represent safety requirements as transparent FOL rules and ground those concepts from visual inputs, and now we're thinking about what this means for future robot autonomy.

Zihan Ye, Jiayi Liu, Puze Liu, Jiayun Li, Georgia Chalvatzaki, Jan Peters, Kristian Kersting

TU Darmstadt Department of Computer Science and Engineering (AIML group) · IAS group (Institute for Advanced Studies) · PEARL group (Program/Group) · Hessian AI Institute · DFKI Institute for Information Technology

cs.RO

Submitted: 2026-09-30

Updated: 2026-09-30

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control (NEUPRO) proposes a neuro-symbolic framework that represents safety specifications as interpretable first-order logic rules, allowing

Key concepts

Logical Predicate Model (LPM): This component maps raw images to continuous 'soft truth values' for grounded atoms. It uses Grounding DINO to extract object features and then outputs a vector of probabilities (between 0 and 1) indicating how true each relevant symbolic predicate is in the scene.
Graph-based Differentiable Reasoner: This structure dynamically builds a sparse graph for each image batch. It uses nodes like 'Conjunction Nodes' to calculate the probability of multiple conditions being true together, and 'Disjunction Nodes' to handle alternatives, allowing logic to be computed differentiably.
Masked Binary Cross-Entropy Loss: This is the training objective. The model is trained by predicting the probability that an observation satisfies a specific safety rule. It uses this loss to guide the neural network so it assigns high satisfaction scores to correct rules and low scores to unsafe ones.
Perceptual Symbol Grounding: This process connects visual inputs (pixels) with symbolic concepts (predicates). It ensures that the features extracted from an image are not just abstract numbers but are specifically mapped to meaningful, human-understandable terms like 'fragile' or 'near'.

Terminology

Grounding — 24× in this paper · explained in 3 episodes
Accuracy — 23× in this paper · explained in 1 episode
Reinforcement Learning — 15× in this paper · explained in 4 episodes
Generalization — 13× in this paper · explained in 13 episodes
Interpretability — 11× in this paper · explained in 15 episodes
First-Order Logic — 9× in this paper
Benchmark — 8× in this paper · explained in 2 episodes
Logical Predicate Model — 5× in this paper
Aggregation — 4× in this paper · explained in 1 episode
Graph Construction — 3× in this paper · explained in 3 episodes
World Model — 3× in this paper · explained in 7 episodes
Abstraction — 2× in this paper · explained in 1 episode
Core — 2× in this paper · explained in 2 episodes
Foundation Model — 2× in this paper · explained in 3 episodes
In-Context Learning — 2× in this paper · explained in 4 episodes
Vision-Language Model — 2× in this paper
Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
F1 Score — 1× in this paper · explained in 3 episodes
Failure Modes — 1× in this paper · explained in 3 episodes
GNNs — 1× in this paper · explained in 2 episodes
Graph Neural Networks (GNNs) — 1× in this paper · explained in 7 episodes
IEEE International Conference on Robotics and Automation — 1× in this paper
In-Context Learning (ICL) — 1× in this paper · explained in 5 episodes
Loss Function — 1× in this paper · explained in 2 episodes
Representation Learning — 1× in this paper · explained in 2 episodes
Robustness — 1× in this paper · explained in 5 episodes
Sparsity — 1× in this paper · explained in 4 episodes
Task Planning — 1× in this paper · explained in 2 episodes
Verification — 1× in this paper · explained in 2 episodes
Vision-Language Model (VLM) — 1× in this paper · explained in 3 episodes
Vision-language models — 1× in this paper · explained in 2 episodes

Summary

Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control (NEUPRO) proposes a neuro-symbolic framework that represents safety specifications as interpretable first-order logic rules, allowing for flexible, interpretable safety reasoning directly from visual inputs. This approach addresses the limitations of existing methods by enabling gradients to flow through symbolic rules to ground learned features in human-understandable semantics, thereby moving beyond opaque cost design toward reusable safety reasoning.

The gist

NEUPRO leverages a differentiable reasoner that can learn reusable safety representations from human-specified safety knowledge, allowing practitioners to express task-related safety requirements as transparent symbolic rules while enabling gradients to propagate through these rules to a feature extractor that maps raw observations to safety-relevant concepts.

Problem Formulation and Goal

The overall goal is to represent safety requirements as a set of interpretable First-Order Logic (FOL) rules denoted as a set of clauses, where each rule defines predicates over a vocabulary of predicates, constants, and variables. The core challenge is to learn how to ground these symbolic predicates from raw robot observations without requiring explicit labels for every predicate in every scene. Instead, the framework uses differentiable rule evaluation as a structured supervision signal, allowing gradients to guide the Logical Predicate Model (LPM) even when only rule-level labels or partially annotated predicates are available. The training objective is defined by a masked binary cross-entropy loss: NEUPRO is trained to match rule-level safety satisfaction, predicting the probability that an observation satisfies a safety rule, denoted as yˆt,i = P(Fi = True st; θ).

Perceptual Symbol Grounding

NEUPRO utilizes two main components: (i) a Logical Predicate Model (LPM) and (ii) a graph-based differentiable reasoner. The LPM maps raw images to soft truth values of grounded atoms, where the output is a continuous valuation vector, v = pθ(s) ∈ (0, 1)G. This process leverages Grounding DINO to extract object features from the raw image batch. Based on labeled rules associated with the image, a subset of features corresponding to terms in the rule is selected. The neural LPM then outputs these continuous valuations for all grounded atoms in the batch.

Reasoning Graph for NEUPRO

To manage memory efficiently, NEUPRO employs a graph-based differentiable reasoner that dynamically constructs a sparse reasoning graph for each mini-batch of active safety specifications. This graph contains three types of nodes: Grounding DINO MLP, which computes soft truth values; Conjunction Node, which computes the logical conjunction probability among multiple grounded atoms (e.g., fragile(cup0) ∧ near(cup0, table edge)); and Disjunction Node, which computes the logical disjunction probability among multiple applied instances (e.g., unsafe(cup0) ∨ unsafe(cup2)). Literal polarity is handled through an affine transformation: a positive literal uses [sign, bias] = [1, 0], while a negated literal uses [sign, bias] = [-1, 1], mapping valuation x to 1 − x.

Differentiable Aggregation and Learning Objective

The reasoning proceeds in two steps over the sparse graph. Step 1 involves aggregating grounded literals into conjunction nodes using a differentiable product t-norm: hconj = scatter prod (1, l, ic, dim size = Nc). Step 2 aggregates multiple conjunctions associated with the same rule head into a rule-level satisfaction probability using a differentiable Soft-OR aggregation: Hout = scatter softorγ (hconj, id, dim size = B × C), where softorγ is an approximation of the max function. The final output is reshaped into a rule-satisfaction matrix P, where each entry represents NEUPRO’s predicted probability that observation st satisfies safety rule Fi. The training objective is the masked binary cross-entropy loss: L(θ) = − ∑ t=1 ∑ i=1 Mt,ih yt,i log ˆyt,i + (1 − yt,i) log(1 − yˆt,i), which encourages high satisfaction probabilities for valid safety rules and low probabilities for violations.

Evaluation and Contributions

NEUPRO is evaluated on the REASON benchmark dataset. Experiments demonstrate that NEUPRO learn[s] safety-critical features that generalize across tasks, mitigating the interpretability limitations of conventional black-box cost formulations, and provides explicit explanations of safety violation. Key results show strong performance in predicate grounding accuracy (RQ1) and task accuracy when composing predicates through symbolic rules (RQ2). Furthermore, NEUPRO exhibits Object-level Generalization and Task-level Generalization, maintaining high accuracy when applied to unseen objects or transferred to different tasks, confirming that its learned representations are "

Improvements for AI systems

Here are the specific improvements that can be made to existing AI systems by implementing the NEUPRO framework, and what those improved systems will be able to do:

The ability to translate high-level, human-specified safety requirements (e.g., Do not point a knife at a human) directly into executable symbolic logic rules that guide perception.
The creation of robot policies that are not just optimized for reward, but are explicitly constrained by transparent, verifiable logical predicates derived from visual input (soft grounding).
The development of Safety-aware Perceptual Representations where the learned features are not opaque vectors but are mathematically grounded in human-understandable concepts (e.g., the system understands that sharpness and proximity to a person are the critical components of an unsafe state).
The capability for cross-task safety generalization, allowing a robot trained on one environment or object type to safely operate in another without requiring extensive retraining, provided they share the same underlying semantic safety predicates.
The generation of explicit, interpretable explanations for why a specific action was deemed unsafe (e.g., Unsafe because holding(robot, scissor) AND near(scissor, person)).

This improved AI system can perform:

Navigate complex human-centric environments with guaranteed safety against specific hazards defined by symbolic rules.
Perform manipulation tasks while maintaining explicit adherence to safety constraints derived from expert knowledge, moving beyond opaque cost functions.
Act as a Safety Auditor for autonomous systems, capable of reasoning about violations based on interpretable logical proofs rather than just black-box probability scores.
Adapt quickly to novel safety scenarios by reusing learned semantic concepts (e.g., the concept of sharpness or close proximity) across different objects and tasks.

Abstract

As robots are increasingly deployed in everyday environments, ensuring their safety has become a central challenge. Existing methods often encode safety requirements as opaque mathematical/logical formulations or dense cost functions. While effective in specific tasks, they remain difficult to interpret, tightly coupled to individual tasks, and offer limited insight into why a robot action is considered safe or unsafe. To address this limitation, we propose ``Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control'' (NEUPRO), which leverages a differentiable reasoner that can learn reusable safety representations from human-specified safety knowledge. NEUPRO allows practitioners to express task-related safety requirements as transparent symbolic rules, while enabling gradients to propagate through these rules to a feature extractor that maps raw observations to safety-relevant concepts. As a result, the learned feature extractor is (softly) grounded in human-understandable semantics, supports transparent constraint evaluation, and is transferable across tasks. By coupling interpretability with differentiability, NEUPRO moves beyond opaque cost design toward reusable safety reasoning. To evaluate NEUPRO's capability, we collect and release REASON, the first real robot benchmark dataset for interpretable robot safety specification. Experiments on REASON show that NEUPRO learns safety-critical features that generalize across tasks, mitigate the interpretability limitations of conventional black-box cost formulations, and provide explicit explanations of safety violation.