dOPT: Differentiating Conic Optimization via Geometric Reduction

dOPT introduces a solver-agnostic framework that differentiates through parametric convex conic programs by reducing them to an equality-constrained quadratic program, thereby enabling efficient gradient computation independently of the forward solver. This method is significant because it captures local first- and second-order conic geometry relevant to differentiation, remains well defined at singular configurations, and demonstrates favorable backward-pass scalability over existing differentiable conic optimization methods.

The Gist

dOPT constructs an explicit, first-order equivalent equality-constrained QP from a primal–dual solution and the local conic geometry, yielding a reduced problem from which solution derivatives can be efficiently computed with a single symmetric linear solve.

Geometric Reduction Principle

The core idea is to exploit the distinction between what is needed to solve a problem and what is needed to characterize the local sensitivity of its solution. The paper suggests that retain only the local constraint geometry that affects first-order sensitivity at the solution. This principle extends beyond linearly constrained problems by accounting for changes in the active normal direction which introduce a curvature term essential for recovering the correct solution derivative, especially at singular boundary configurations.

Construction of Reduced Form

Under standard regularity conditions (nondegeneracy and strict complementarity), Theorem 4.1 constructs a reduced problem that is first-order equivalent to the original conic problem. This reduction yields a quadratic program with only equality constraints, capturing precisely the degrees of freedom relevant for differentiation. The resulting differential system is solved by a single symmetric linear solve, which decouples optimization and differentiation, allowing mature black-box solvers to be used without differentiating through their algorithms.

Explicit Reductions for Conic Programs

dOPT provides explicit constructions for different classes of conic programs:

1. For NLPs, the reduction recovers the classical active-set reduction for nonlinear programs by setting the curvature matrix H to zero because the boundary of the nonpositive orthant is locally polyhedral.

2. For QPs, it recovers reductions derived from previous work (Magoon et al., 2025).

3. For SOCPs, it constructs matrices H and C based on active subsets: for non-apex blocks, they are given by H∗j = −s∗j∇2ϕ(y∗j), ∇ϕ(Gj (z∗))T and C∗ is the corresponding tangent space matrix. For apex blocks, the critical cone is trivial, leading to a well-defined reduction where no nonzero admissible direction along which curvature must be evaluated.

4. For SDPs, the construction involves an eigenbasis transformation and yields H as H∗ = −Z∗† ⊗ S∗ + S∗ ⊗ Z∗†, capturing the dual variation up to a normal residual.

Validation and Performance

Numerical experiments validate the computed gradients, showing favorable backward-pass scalability and substantial speedups over existing differentiable conic optimization methods as problem size increases. The framework is evaluated on synthetic SOCPs and SDPs, achieving gradient accuracy on the order of approximately 10−9 for SOCPs and 10−7 for SDPs when compared to analytic gradients from the envelope theorem. Furthermore, in an end-to-end learning benchmark involving a differentiable SDP layer, dOPT demonstrated that its backward pass is significantly faster than baselines like CVXPYLayers and FFOLayer across various problem sizes.

Key Geometric Objects

The framework relies on characterizing two matrices: C∗ (whose nullspace is the critical cone) and H∗ (which captures the corresponding normal variation). These define the constraints and curvature correction of the locally equivalent reduced formulation. The construction naturally accommodates singular configurations such as SOC apexes and PSD matrices with multiple zero eigenvalues, ensuring that the reduced problem remains well defined even when Z∗ has multiple zero eigenvalues. The critical cone, CK(y∗), is determined entirely by the local first-order cone geometry.

Solver Agnosticism

A key feature is its solver independence: Computing solution derivatives then requires a single symmetric linear solve, independently of the forward solver. This means that one may use any black-box solver to compute the solution of the original problem, and subsequently differentiate it solely through the reduced problem. The construction requires only the problem data and a computed primal–dual solution to define C∗ and H∗.

Limitations

Notable limitations include the restriction to convex problems, reliance on high-quality solutions, and limited exploitation of algebraic structure, especially for SDPs. Forward solves remain the main computational bottleneck that dOPT does not address. Future work is motivated by extensions to nonconvex problems, support for approximate solutions (e.g., from L2O), more effective use of structure, and weaker regularity assumptions. The framework's success at singular points depends on "strict complementarity and non-degeneracy.

As a fastidious researcher, I have analyzed the provided paper, dOPT: DIFFERENTIATING CONIC OPTIMIZATION VIA GEOMETRIC REDUCTION. The core innovation of this work is providing a solver-agnostic framework for efficiently computing gradients of optimal solutions to conic programs (NLP, QP, SOCP, SDP) by reducing the problem to an equality-constrained Quadratic Program (QP) based on local first-order conic geometry.

Here are the specific improvements and capabilities this framework enables in AI systems:

Core Improvements and Capabilities Enabled by dOPT:

Optimization of Differentiable Conic Constraints in Large-Scale Models:

The paper directly addresses the bottleneck of differentiating through complex, structured constraints (like those found in Second-Order Cone Programs (SOCPs) or Semidefinite Programs (SDPs)) during training. By reducing the problem to a first-order equivalent equality-constrained QP, dOPT allows for efficient gradient computation via a single symmetric linear solve.

The improved AI system can now:

• Train models governed by constraints that are inherently conic, such as those arising in robust control systems (where SOCPs are common) or in complex risk management/covariance estimation problems (SDPs).

• Achieve massive speedups over existing differentiable solvers (like CVXPYLayers or FFOLayer) during the backward pass, leading to significantly faster training cycles for large-scale neural networks integrated with these optimization layers.

Handling Singular and Non-Smooth Configurations Robustly:

A key strength of dOPT is its explicit construction of the critical cone and curvature correction matrix (Hstar). This framework is proven to remain well-defined even at singular configurations, such as Second-Order Cone apexes or Semidefinite Programs with multiple zero eigenvalues.

The improved AI system can now:

• Deploy optimization layers in scenarios where the optimal solution lies exactly at a singularity (e.g., when a covariance matrix becomes singular). Existing methods often fail or require complex approximations at these points; dOPT provides mathematically sound derivatives.

• Ensure numerical stability and gradient accuracy when training models that operate near boundary conditions defined by conic constraints that are not strictly smooth everywhere (e.g., in certain reinforcement learning reward functions or physical system constraints).

Solver Agnosticism for Flexible Infrastructure:

dOPT is a solver-agnostic framework; it takes a computed primal-dual solution from an arbitrary forward solver (like MOSEK or SCS) and constructs the reduced problem formulation independently of how that solution was obtained.

The improved AI system can now:

• Integrate with any existing black-box optimization library used in the forward pass without needing to modify its internal differentiation logic. This allows researchers to leverage the most efficient existing solvers for solving the complex primal problem while using dOPT solely for the backward differentiation step.

Efficient Gradient Computation via Linear Algebra:

The framework replaces complex implicit differentiation through nonlinear KKT systems with solving a single, symmetric linear system (Equation 6).

The improved AI system can now:

• Achieve superior backward-pass scalability, as the cost of computing the gradient becomes dominated by this single linear solve rather than iterative or complex implicit differentiation procedures. This is crucial for deep learning architectures where the backward pass must be extremely fast to enable large batch sizes and long training runs.

Accurate Sensitivity Analysis:

The framework preserves both the reference solution and its first-order sensitivity by explicitly accounting for the variation of the cone normal (via Hstar). This ensures that parameter perturbations are correctly mapped to solution changes, even in non-standard geometric settings.

The improved AI system can now:

• Perform rigorous sensitivity analysis on learned models or control policies parameterized by optimization variables, providing reliable estimates of how small changes in model parameters affect the resulting optimal behavior.

In summary, this paper enables the development of a class of AI systems that are not only capable of using complex conic constraints but can also compute their gradients with high accuracy and efficiency across a wide range of geometrically challenging scenarios where standard differentiation techniques fail or become prohibitively slow.