Towards Reasonable Concept Bottleneck Models

The gist: We propose Concept REAsoning Models (CREAM), a novel framework for Concept Bottleneck Models (CBMs) that explicitly encodes prior knowledge about concept-concept and concept-task relationships through a reasoning graph, enabling interpretable and effective prediction even with limited concepts.

How it works

The CREAM framework is designed to encode arbitrary types of C-C relationships such as mutual exclusivity, hierarchical associations, and/or correlations, as well as potentially sparse C→Y relationships through a reasoning graph G = (V,E) where V = C ∪ Y and E ⊆ V × V captures plausible (un)directed relationships The graph is partitioned into two subgraphs: the concept graph GC:= G[C] containing the C-C relationships, and the task graph GY:= G[Cdirect ∪ Y] for C→Y reasoning

Concept-Concept Reasoning

The concept graph’s GC adjacency matrix AC captures both hierarchical relationships and correlations between them, represented as asymmetric (AC (i, j) ̸= AC (j, i)) and symmetric (AC (i, j) = AC (j, i)) entries respectively This matrix can represent a Partially Directed Acyclic Graph (PDAG), combining hierarchical and correlated relationships

Concept-Task Reasoning

The C→Y relationships are encoded in GY using a task adjacency matrix AY ∈ 0, 1 K×L where AY (i, j) = (1 if (ci, yj) ∈ EGY, for ci ∈ Cdirect, yj ∈ Y; 0 otherwise CREAM does not require all concepts to connect directly to the task; indirect concepts Cindirect help predict other concepts within GC

Key Mechanisms and Components

CREAM is implemented by embedding plug-and-play components into any CBM variant, including a representation splitter, a concept-concept block, a regularized side-channel, and a concept-task block The backbone's output is split into two representations: the concept representation zC ∈ R dC K which serves as input to the concept-concept block, and an optional side-channel representation zY ∈ R z-dC K capturing information beyond the predefined concepts

Concept-Concept Block

This block enforces the C-C relationships encoded in AC using Structural Causal Models (SCMs) principles, where each concept depends only on its parents and its corresponding exogenous variables For mutually exclusive (mutex) concepts in GC, a softmax over the logits within each group is applied

Concept-Task Classifier

The final stage maps concept predictions to task logits while incorporating the side-channel in a controlled manner using StrNNs to enforce C→Y relationships expressed by AY The task prediction for class j is computed as yˆj = f([ˆcP aj, zˆYj]) where f: R K+L → R L, cˆP aj ⊆ Cdirect

Intervention and Interpretability

Ease of Interventions

Since only Cdirect is used for predictions, the total number of effective interventions is lowered from K to Cdirect Users can also identify which concept predictions led to incorrect task predictions by tracing the edges of GY and prioritize interventions on those specific concepts

Propagating Interventions

CREAM supports propagating interventions, where changing one concept can automatically adjust downstream concepts like “Tops”, ultimately affecting the task prediction, allowing Cindirect to affect the task This is achieved by inverting the concept activations to recover their corresponding exogenous variables

Concept Leakage Mitigation

The structured reasoning process through AC and AY constrains spurious or semantically invalid pathways, compelling the model to use intended relationships, thereby reducing leakage and improving interpretability CREAM is presented as the first soft CBM framework that is leakage-free

Evaluation and Findings

Experiments across FashionMNIST (iFMNIST), Complete FMNIST (cFMNIST), CUB, and CelebA demonstrate that CREAM achieves competitive task and concept performance, even under incomplete settings The two main components of CREAM offer orthogonal benefits: structured reasoning promotes interpretability, while the side-channel addresses limitations of incomplete or noisy concepts

Concept Channel Importance (CCI)

Concept Channel Importance (CCI) is based on Shapley Additive Global Explanations (SAGE), and we desire the importance of the concept channel to be greater or equal to the importance of the side channel, i.e., ϕc(vf) > ϕy(vf) A CCI value close to 1 indicates that the model relies primarily on the concept channel, reinforcing interpretability

Permutation Feature Importance (PFI)

Permutation Feature Importance (PFI) measures the significance of a feature by evaluating how much the model’s accuracy deteriorates when its values are randomly shuffled The PFI-based metrics also indicate that CREAM prioritizes the concept-channel instead of the side-channel

The framework's modular design supports diverse C-C relationships and concept representations, while the sparser C→Y reasoning facilitates interventions and interpretability Furthermore, our side-channel and regularization framework enables the effective use of CBMs, and thus interpretability in neural networks, even in regimes with very limited concept supervision. It is also observed that proper regularization of the side-channel (e.g., via dropout) is crucial to maintaining interpretability and intervenability in hybrid CBMs. Finally, we introduce Concept Channel Importance (CCI), a new model-agnostic metric for quantifying interpretability when predictions rely on auxiliary channels beyond the concept channel. It is also shown that group interventions on mutex concepts can help scale the intervention procedure.

Future Work and Limitations

CREAM requires prior domain knowledge to encode concept-concept and concept-task relationships, and future work could explore automated structure learning to infer these dependencies Improving the stability and reliability of intervention propagation via inversion presents an exciting avenue for future enhancement. Additionally, implementing adaptive and test-time dropout strategies, where the side-channel is dynamically leveraged based on concept prediction uncertainty, could improve robustness and interpretability while reducing the effort required for hyperparameter tuning Furthermore, the structure learned in the exogenous variables and the side-channel can be leveraged to guide the discovery of new concepts that are not captured by the current concept bottleneck.

Acknowledgments

We would like to thank Pablo M. Olmos and every member of the Probabilistic Machine Learning lab for their invaluable feedback, as well as the anonymous reviewers whose feedback helped us improve this manuscript. The work was funded in part by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – GRK 2853/1 “Neuroexplicit Models of Language, Vision, and Action” - project number 471607914. The paper is funded in part by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – GRK 2853/1 “Neuroexplicit Models of Language, Vision, and Action” - project number 471607914. The work was funded in part by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – GRK 2853/1 “Neuroexplicit Models of Language, Vision, and Action” - project number 471607914. The work was funded in part by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – GRK 2853/1 “Neuroexplicit Models of Language, Vision, and Action” - project number 471607914 <ref:2506.

1. Encoding Prior Knowledge in Reasoning CREAM models explicitly encode prior knowledge of C-C (concept-concept) and C→Y (concept-task) relationships through a reasoning graph (G), allowing practitioners to explicitly encode and extend their prior knowledge and beliefs about the concept-concept (C-C) and concept-task (C→Y) relationships within the model’s reasoning when making predictions.

2. Modeling Complex Concept Relationships The framework can architecturally encode arbitrary types of C-C relationships such as mutual exclusivity, hierarchical associations, and/or correlations, enabling the system to handle complex structural constraints beyond standard conditional independence assumptions.

3. Mitigating Concept Leakage by Design CREAM mitigates concept leakage by design because the reasoning structure of CREAM allows only for intended information flows, thus mitigating leakage by design, without needing hard concepts or introducing additional regularization.

4. Enabling Efficient Interventions The framework supports interventions that are more efficient than standard CBMs; specifically, Since only Cdirect is used for predictions, the total number of effective interventions is lowered from K to Cdirect, and it enables propagating interventions by leveraging invertibility.

5. Improving Interpretability via Metrics The system utilizes metrics like Concept Channel Importance (CCI) to quantify interpretability when predictions partially rely on the side-channel, aiming for the desired outcome where "CCI > 0.5, ensuring that predictions primarily rely on the concepts rather than being dominated by the side-channel."

6. Adapting to Concept Incompleteness CREAM enables CBMs to remain effective even in scenarios where only a limited number of concepts are available, as it uses a regularized side-channel to complement the potentially incomplete concept sets, achieving competitive task performance while encouraging predictions to be concept-grounded.

7. Scaling Interventions with Group Reasoning The framework supports group interventions on mutex concepts, allowing human experts to intervene on a group simultaneously, which leads to the same performance gains as individual interventions, but with less human effort by reducing the number of necessary interventions down to the number of directly connected concept mutex groups.