Interior-point proximal methods for nonsmooth optimization in Hilbert spaces with cone-ordered constraints

Interior-point methods are studied here for nonsmooth, nonconvex optimization problems in Hilbert spaces with cone-ordered constraints, providing a unified framework for both finite-dimensional and infinite-dimensional PDE-constrained optimization. The gist is: The total complexity is dominated by the final outer iterations because of the growth of the barrier curvature. This work establishes convergence to KKT points and derives total inner–outer complexity bounds for reaching an approximate KKT point, showing that logarithmic barriers outperform power barriers across various test cases.

Problem Formulation and Setting

The paper considers problems of the form minimize J (u):= F(u) + R(u) subject to H(u) ≤Z 0 in Z (1), where U is a Hilbert space, F is a smooth term, and R is a convex (possibly nonsmooth) term with a computable proximal mapping. The constraints are formulated using an order cone in a Banach lattice Z, covering finite-dimensional problems like sparse dictionary learning with componentwise constraints and infinite-dimensional problems such as PDE-constrained optimization with state constraints where Z = C(K).

Barrier Regularization and Inner Subproblem Analysis

Interior-point methods replace the original constraint with a barrier functional B, leading to subproblems of the form minimize u∈U Jν(u):= J (u) + νB(H(u)). Since the objective contains a nonsmooth term R, these subproblems are solved inexactly by a proximal-gradient method. The analysis focuses on logarithmic and power-type barriers, defined by scalar kernels such as ϕlog(t) = − log(−t) and ϕpow(−t) = (−t) − p. Key findings include:

1. The barrier functional B is convex and twice continuously Fréchet differentiable on the interior of the negative cone int(Z−).

2. The gradient of the barrier subproblem, ∇Qν(u), is Lipschitz continuous with constant Lsmooth(ν; a, b) (11).

3. The inner loop complexity for solving subproblem (9) is bounded by Nin(ν, ε) = O L2(mnm+1) smooth (ν)ε−2. For the logarithmic barrier, the smoothness constant scales as Lsmooth(νk) = O ν−1.

Outer Interior-Point Scheme and Convergence

The full interior-point scheme involves an outer loop that successively reduces the barrier parameter νk and inner tolerance εk. The analysis establishes several key properties for iterates generated by Algorithm 4.1:

1. Primal feasibility (i): uk ∈ U is strictly feasible for (1), i.e., H(uk) <Z 0.

2. Dual feasibility (ii): µk ∈ Z∗+ satisfies the approximate KKT condition dist (−∇F(uk) − H′(uk)∗µk, ∂R(uk)) ≤ εk.

3. Convergence to KKT points (Theorem 4.10): Under suitable constraint qualifications, the full inexact interior-point scheme converges to stationary points, and the sequence of multipliers satisfies approximate KKT conditions.

Total Complexity Bounds

The total complexity is bounded by Ntot(ϵ):= PNout(ϵ) k=0 Nin(νk, εk). The analysis distinguishes between barrier types:

1. Logarithmic Barrier: The complementarity residual rk = m(Ξ) νk, leading to an outer iteration bound of Nout(ϵ) = max ln(ε0/ε) ln(1/θε), ln(m(Ξ) ν0/ε) ln(1/θν).

2. Power Barrier (p > 1): The complementarity residual rk is bounded by a term that depends on the barrier parameter, allowing for an outer iteration bound of Nout(ϵ) = max [ln(ε0/ε), (p + 1) ln ν−1/(p+1) C/ε ln(1/θν)].

Verification and Applications

The abstract assumptions are verified for state-constrained semilinear elliptic optimal control problems, confirming that the framework applies to these infinite-dimensional settings. Numerical experiments validate the theoretical findings, showing that logarithmic-type barriers (logarithmic and log-like kernels) generally require fewer inner iterations than inverse or power barriers across all metrics. The total complexity is asymptotically dominated by the final outer iterations in both barrier regimes.

Conclusion

The paper develops a unified inexact interior-point framework for nonsmooth conic optimization, establishing convergence to KKT points and deriving total inner–outer complexity bounds. The key structural feature is the growth of barrier curvature as ν → 0, which necessitates that the total complexity is dominated by the final outer iterations. The framework successfully covers both logarithmic and power barriers in both finite-dimensional and infinite-dimensional settings.

As a fastidious researcher, I have analyzed this paper, Interior-Point Proximal Methods for Nonsmooth Optimization in Hilbert Spaces with Cone-Ordered Constraints. The core contribution is a unified framework for solving complex, nonsmooth optimization problems (combining smooth and nonsmooth terms) subject to conic constraints (modeled via order cones in Banach lattices), applicable to both finite-dimensional machine learning and infinite-dimensional PDE-constrained control.

Here are the specific improvements I can suggest for AI systems:

)

The paper provides a theoretically rigorous and complexity-aware method for solving optimization problems that traditional interior-point methods struggle with, specifically those involving nonsmooth objectives (like sparsity penalties, indicator functions of sparse sets) and conic constraints (like nonnegativity or box constraints).

AI Systems can solve highly complex, large-scale machine learning problems that involve both smooth data-fitting terms and highly sparse, structured regularization penalties (e.g., Lasso, Group Lasso) subject to nonnegativity or box constraints.

The improved system will be able to handle infinite-dimensional state constraints arising in continuous control problems (e.g., controlling the dynamics of a physical system governed by PDEs). This allows for the design of robust control policies where state variables are subject to pointwise bounds derived from physical or safety limits.

The AI can optimize complex models where the objective function is a combination of standard loss functions and nonsmooth regularizers (like those in sparse dictionary learning) under structural constraints defined by order cones (e.g., component-wise sparsity).

The system will exhibit superior convergence behavior in highly non-convex settings, specifically when the barrier parameter approaches zero, due to the analysis of curvature growth and total complexity bounds. This means the optimization process is guaranteed to reach a good enough approximate solution efficiently, even when the problem is difficult or nonconvex.

The framework provides a concrete computational roadmap (Algorithm 4.1) that balances inner-loop gradient approximations with outer barrier updates, leading to provable termination criteria for achieving an ε-approximate KKT point. This allows for the development of AI training pipelines that are both faster and more reliable than standard first-order methods in complex constrained spaces.

The system can be used to solve state-constrained optimal control problems (e.g., optimizing a system's trajectory while respecting constraints on its state variables, such as keeping temperature below a certain threshold). This is critical for real-world applications like robotics and autonomous systems where the state dynamics are governed by PDEs.

The analysis provides insights into the worst-case complexity of these methods, specifically showing that the total computational cost is dominated by the final outer iterations as parameters approach optimality. This allows researchers to design AI training schedules or control strategies optimized for minimizing total computational effort while maintaining high accuracy.

We study an inexact interior-point method for nonsmooth, possibly nonconvex optimization in a Hilbert space with inequality constraints ordered by a cone in a Banach lattice, with particular emphasis on infinite-dimensional state-constrained optimal control. The objective function is given by the sum of a smooth, possibly nonconvex term and a convex, possibly nonsmooth term with a computable proximal mapping. The constraints are formulated by means of an order cone in a Banach lattice. This setting covers finite-dimensional nonsmooth nonlinear problems with componentwise constraints as well as infinite-dimensional PDE-constrained optimization problems with pointwise state constraints. The method is based on barrier-regularized subproblems, which are solved inexactly by a proximal-gradient method. We consider logarithmic and power-type barriers and derive the differentiability and curvature estimates needed for the convergence and complexity analysis. Under suitable constraint qualifications and compactness assumptions, we establish approximate KKT conditions for the original problem and convergence of the inexact interior-point sequence. For logarithmic and power-type barriers, we derive complementarity estimates and outer iteration bounds; the corresponding power-barrier rates and total inner-outer complexity bounds are stated under explicit barrier-path and uniform smoothness assumptions. For convex problems, we obtain stronger convergence results. We apply the framework to state-constrained semilinear elliptic optimal control and sparse dictionary learning with nonlinear side constraints. Numerical experiments illustrate the proposed method.