On finite-horizon approximation of an infinite-horizon feedback Nash equilibrium in discrete-time LQ games

In infinite-horizon discrete-time linear-quadratic (LQ) dynamic games, computing feedback Nash equilibria (FNEs) is computationally challenging due to coupled Riccati equations involving high-dimensional matrices, numerous cross-product terms, and nonlinear algebraic structures. Motivated by this difficulty, the paper studies a finite-horizon strategy as an approximation for one of the infinite-horizon FNEs.

The proposed finite-horizon strategy is defined such that each player chooses an individual prediction horizon, say a player i has an individual prediction horizonT i. In the infinite-horizon game, at each stage, player i computes its control by envisioning an auxiliaryT i-stage game where the same set of players play and computing the unique FNE of that auxiliary game using a standard method, then implementing only the first-stage control.

The main results presented are:

1. Under suitable conditions, the total cost under these finite-horizon strategies converges to that under one of the infinite-horizon FNEs when all players’ prediction horizons tend to infinity: Our main result is, under suitable conditions, the total cost under these finite-horizon strategies converges to that under one of the infinite-horizon FNEs when all players’ prediction horizons tend to infinity.

2. An explicit cubic-polynomial upper bound on this cost gap is derived with respect to the distance between corresponding strategy matrices: Moreover, we derive an explicit cubicpolynomial upper bound on this cost gap with respect to the distance between the corresponding strategy matrices. This bound vanishes as all players’ prediction horizons tend to infinity.

3. The strategy is tractable and implementable, as it avoids the direct solution of coupled algebraic Riccati equations (CARE) of infinite-horizon LQ games: This strategy is tractable and implementable, as it avoids the direct solution of the coupled algebraic Riccati equations (CARE) of infinite-horizon LQ games.

The paper recalls standard results for discrete-time finite-horizon LQ games in i/o/s form, where player i aims to minimize a cost function involving output tracking errors and control effort. It introduces the notion of a finite-horizon strategy where each player chooses a fixed individual prediction horizon throughout the infinite-horizon game and implements only the first-stage action based on solving an auxiliary finite-horizon LQ game at each stage.

The theoretical analysis focuses on the zero-reference case, which is a fundamental special case of trajectory tracking. The paper establishes convergence properties under Assumption 1 (Invertibility Condition, Convergence Condition, and Stable Condition). Lemma 3 shows that for any given time t in the infinite-horizon game and any player i, lim Kti∗ (T) = K i∗, T →+∞ and lim Pti∗ (T) = P i∗. Moreover, the strategy set u it = K i x t i ∈ , 1 ≤ t ≤ T constitutes an FNE of the infinite-horizon game.

Theorem 4 characterizes the convergence property of total cost under these finite-horizon strategies: "In the infinite-horizon game, suppose all players adopt finitehorizon strategies with individual prediction horizons. That is, each player i chooses his/her individual prediction horizon T i and adopts the following strategy u it (x t) = K1i∗ (T i)x t at any stage t ∈ N+. The resulting total cost for player i under these finite-horizon strategies is denoted by J̃i (x1; T 1,…, T N):= J̃i (x1). Then we have lim J̃i (x1) = J i (x1) for any i ∈ , where T h = min T i. If max i∈‖K1i∗(T i) − K i∗‖2 < 1, then J̃i (x1) − J i (x1) ≤ 1/x squared θ i(ε) squared for any i ∈ , where θ i(ε) = θ i1 ε + θ i2 ε squared + θ i3 ε cubed. This bound tends to zero as all prediction horizons increase because by Lemma 3,‖K1i∗ (T i)−K i∗‖2 → 0 as T h = min T i → +∞, which implies ε → 0 as T h = min T i → +∞. Since θ i(ε) is a cubic polynomial in ε, the i∈ upper bound vanishes accordingly."

A numerical example with nonzero output reference trajectories is provided to illustrate the performance of the proposed finite-horizon strategy beyond this theoretical baseline. The simulation results show that the total costs J̃1 (x1)(T) and J̃2 (x1)(T) converge to the FNE costs J 1 (x1) and J 2 (x1) as T tends to infinity.

In conclusion, the finite-horizon strategy serves as an approximation to one of the FNEs in the infinite-horizon game with a quantifiable and diminishing error. The paper notes that An open question remains as to what parameter-based conditions guarantee the convergence of the iterative matrices generated by the discrete coupled Riccati difference equations.

The paper also includes Appendix A, which shows how i/o/s games can be rewritten as equivalent i/s LQ dynamic games, and Appendix B, C, and D present several proofs. The transformation involves defining an augmented state z t = t and rewriting the cost functional in terms of standard finite-horizon LQ game dynamics. The final cost function under the infinite-horizon FNE is given by J i (x1) = x T 2 1 / P i∗ x1. The total cost under the finite-horizon strategies is expressed as a complex expression involving summation terms related to the Riccati difference equations and strategy matrices. The convergence proof relies on bounding terms in equation (33), showing that as T→+∞, these bounds tend to zero.

The paper concludes by stating that lim J̃i (x1) − J i(x1) = 0, ∀i ∈ , T h → +∞. This is because lim ε = 0 by Lemma 3. Thus, we have T h → +∞ θ i(ε) = 0. The paper also includes a declaration regarding the use of generative AI for polishing language.

The relevant mathematical tools include:

- matrices:

- Riccati difference equations (3)-(7)

- Coupled algebraic Riccati equations (CARE)

- Feedback Nash equilibrium (FNE) definitions.

The paper's contribution is the introduction of a tractable and implementable finite-horizon strategy that avoids solving the coupled CARE, providing explicit performance guarantees. The final result is that the finite-horizon strategy approximates one of the FNEs in an infinite-horizon game with a quantifiable and diminishing error.

---

**(Self-Correction/Verification Note: The request asks for **

The provided text is extremely long and detailed, containing many mathematical derivations and definitions. The request demands a summary that is long and detailed while quoting relevant parts. I will synthesize the core narrative, focusing on the motivation, methodology (the strategy), the main theoretical results (convergence and bounds), and the conclusion.)

---

Summary of Scientific Paper

The paper investigates feedback Nash equilibria (FNEs) in discrete-time linear-quadratic (LQ) dynamic games with input/output/state dynamics. The central challenge addressed is that computing FNEs in infinite-horizon LQ games involves solving coupled Riccati equations, which are computationally demanding due to their high dimensionality and nonlinear algebraic structures.

Proposed Methodology: Finite-Horizon Strategy

To circumvent the computational difficulty of solving these coupled algebraic Riccati equations (CARE), the authors propose a tractable and implementable finite-horizon strategy. This strategy is inspired by model predictive control (MPC): Each player i looks T i steps ahead and implements only the first-stage action. Specifically, each player i chooses a fixed individual prediction horizon, denoted as T i, which is determined before the infinite-horizon game begins. At each stage t in the infinite-horizon game, player i computes its control by envisioning an auxiliary T i-stage game and implementing only the first-stage control derived from that auxiliary FNE.

Main Theoretical Contributions

The paper makes three main contributions:

1. Introduction of a tractable strategy: First, we introduce a tractable and implementable finite-horizon strategy, where each player is allowed to use an individual prediction horizon.

2. Convergence of Cost: "Second, when all players adopt the proposed finite-horizon strategies in the infinite-horizon game, we establish that, under suitable conditions, each player’s induced total cost converges to the total cost associated with the limiting FNE of the infinite-horizon game."

3. Explicit Error Bound: "Third, we derive an explicit cubic-polynomial upper bound on the cost difference, expressed in terms of the distance between the corresponding feedback strategy matrices, and show that this bound vanishes as all players’ prediction horizons tend to infinity."

Key Results and Proof Outline

The analysis is performed under Assumption 1 (Invertibility Condition, Convergence Condition, and Stable Condition), which ensures the well-defined nature of iterative matrices. Lemma 3 establishes that the limiting matrices of the discrete coupled Riccati difference equations converge to those of an FNE in the infinite-horizon game: lim K t i∗ (T) = K i∗, T →+∞ and lim P t i∗ (T) = P i∗. Moreover, the strategy set u it = K i x t i ∈ , 1 ≤ t ≤ T constitutes an FNE of the infinite-horizon game.

Theorem 4 quantifies the convergence property of total cost: "In the infinite-horizon game, suppose all players adopt finitehorizon strategies with individual prediction horizons. That is, each player i chooses his/her individual prediction horizon T i and adopts the following strategy u it (x t) = K1i∗ (T i)x t at any stage t ∈ N+. The resulting total cost for player i under these finite-horizon strategies is denoted by J̃i (x1; T 1,…, T N):= J̃i (x1). Then we have lim J̃i (x1) = J i (x1) for any i ∈ , where T h = min T i."

Crucially, the paper derives an explicit upper bound on the cost gap: "Let ε = max i∈ K1i∗(T i) − K i∗ squared denote the maximum distance between the player-specific first-stage FNE strategy matrices, computed from the respective auxiliary games induced by the prediction horizons (T 1, …, T N), and their limiting counterparts. If (∑ B j K j∗ + B j squared ε < 1, then J̃i (x1) − J i (x1) ≤ 1/x squared θ i(ε) squared for any i ∈ , where θ i(ε) = θ i1 ε + θ i2 ε squared + θ i3 ε cubed. This bound vanishes as all prediction horizons increase because by Lemma 3,‖K1i∗ (T i)−K i∗‖2 → 0 as T h = min T i → +∞, which implies ε → 0 as T h = min T i → +∞. Since θ i(ε) is a cubic polynomial in ε, the i∈ upper bound vanishes accordingly."

Numerical Illustration

A numerical simulation with a two-player game demonstrates the convergence: the total costs J̃1 (x1)(T) and J̃2 (x1)(T) converge to the FNE costs J 1 (x1) and J 2 (x1) as T tends to infinity. The figure shows that for any finite horizon T, the cost under the finite-horizon strategy converges to the limiting FNE cost as T increases.

Conclusion

The paper successfully demonstrates that a finite-horizon strategy approximates an infinite-horizon FNE with a quantifiable and diminishing error. The convergence is guaranteed as all prediction horizons tend to infinity: lim J̃i (x1) − J i(x1) = 0, ∀i ∈ , T h → +∞. While the theoretical convergence and performance bound are established under the zero-reference setting, a numerical example with nonzero output reference is presented to illustrate behavior beyond this baseline. The paper remains open to finding parameter-based conditions that guarantee the convergence of the iterative matrices generated by the discrete coupled Riccati difference equations.

Relevant Mathematical Context

The analysis relies on transforming i/o/s games into equivalent i/s LQ games using an augmented state z t = t, leading to a cost functional in standard finite-horizon LQ game form (Equation 15). The convergence proof involves bounding terms in equation (33) using properties of the spectral radius and norms of matrices, ultimately showing that the error term decays as T→+∞. The final cost gap is bounded by an expression involving theta i(ε), which goes to zero as ε → 0.

Declaration of Generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work the author(s) used ChatGPT in order to polish the language. It is noted that the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.

References Cited (Selected Examples)

[3] T. Basar, G. J. Olsder, Dynamic Noncooperative Game Theory, SIAM, 1998.

[5] B. Nortmann, A. Monti, M. Sassano, T. Mylvaganam, Nash equilibria for linear quadratic discrete-time dynamic games via iterative and data-driven algorithms, IEEE Trans. Autom. Control 69 (2024) 6561–6575.

[8] M. Siniscalchi, Structural rationality in dynamic games, Econometrica 90 (2022) 2437–2469.

[13] M. A. Nowak, K. Sigmund, Evolutionary dynamics of biological games, Science 303 (2004) 793–799.

[15] U. Sadana, P. V. Reddy, G. Zaccour, Feedback nash equilibria in differential games with impulse control, IEEE Trans. Autom. Control 68 (2023) 4523–4538.]

Final Summary Statement:

The paper introduces a finite-horizon strategy where each player uses an individual prediction horizon to approximate the infinite-horizon FNE of an LQ game. It proves that under suitable conditions, the total cost incurred by players using these strategies converges to the cost of the limiting infinite-horizon FNE as all prediction horizons tend to infinity, and it provides an explicit cubic-polynomial upper bound on this cost gap that vanishes with increasing prediction horizons. This strategy is tractable and avoids solving coupled CAREs. The convergence is rigorously established via iterative analysis of Riccati difference equations, leading to the conclusion that the finite-horizon approximation serves as a valid method for approximating FNEs in infinite-horizon LQ games with a quantifiable error that diminishes as the prediction horizon grows.

---

(End of Summary)

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work the author(s) used ChatGPT in order to polish the language. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.

Final Output:

In infinite-horizon discrete-time linear-quadratic (LQ) dynamic games, computing feedback Nash equilibria (FNEs) is computationally challenging due to coupled Riccati equations involving high-dimensional matrices, numerous cross-product terms, and nonlinear algebraic structures. Motivated by this difficulty, the paper studies a finite-horizon strategy as an approximation for one of the infinite-horizon FNEs.

The main results presented are:

1. Under suitable conditions, the total cost under these finite-horizon strategies converges to that under one of the infinite-horizon FNEs when all players’ prediction horizons tend to infinity: Our main result is, under suitable conditions, the total cost under these finite-horizon strategies converges to that under one of the infinite-horizon FNEs when all players’ prediction horizons tend to infinity.

2. An explicit cubic-polynomial upper bound on this cost gap is derived with respect to the distance between corresponding strategy matrices: Moreover, we derive an explicit cubicpolynomial upper bound on this cost gap with respect to the distance between the corresponding strategy matrices. This bound vanishes as all players’ prediction horizons tend to infinity.

3. The strategy is tractable and implementable, as it avoids the direct solution of coupled algebraic Riccati equations (CARE) of infinite-horizon LQ games: This strategy is tractable and implementable, as it avoids the direct solution of the coupled algebraic Riccati equations (CARE) of infinite-horizon LQ games.

The theoretical analysis focuses on the zero-reference case, which is a fundamental special case of trajectory tracking. The paper establishes convergence properties under Assumption 1 (Invertibility Condition, Convergence Condition, and Stable Condition). Lemma 3 shows that for any given time t in the infinite-horizon game and any player i, lim K t i∗ (T) = K i∗, T →+∞ and lim P t i∗ (T) = P i∗. Moreover, the strategy set u it = K i x t i ∈ , 1 ≤ t ≤ T constitutes an FNE of the infinite-horizon game.

A numerical simulation with a two-player game demonstrates the convergence: the total costs J̃1 (x1)(T) and J̃2 (x1)(T) converge to the FNE costs J 1 (x1) and J 2 (x1) as T tends to infinity. The figure shows that for any finite horizon T, the cost under the finite-horizon strategy converges to the limiting FNE cost as T increases.

---

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work the author(s) used ChatGPT in order to polish the language. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.

The proposed finite-horizon strategy is defined such that each player chooses an individual prediction horizon, say a player i has an individual prediction horizonT i. In the infinite-horizon game, at each stage, player i computes its control by envisioning an auxiliaryT i-stage game where the same set of players play and computing the unique FNE of that auxiliary game using a standard method, then implementing only the first-stage control.

The paper concludes by stating that lim J̃i (x1) − J i(x1) = 0, ∀i ∈ , T h → +∞. This is because "lim ε = 0 by Lemma 3. Thus

As a fastidious researcher, I have analyzed this paper titled On finite-horizon approximation of a feedback Nash equilibrium in LQ games. The core contribution is providing a computationally tractable method—the finite-horizon strategy—to approximate an infinite-horizon Feedback Nash Equilibrium (FNE) in discrete-time Linear Quadratic (LQ) dynamic games.

Based on the mathematical results presented, here are the specific improvements that can be made to AI systems and what those improved systems can achieve:

---

The paper's primary achievement is transforming a computationally intractable problem (solving coupled Algebraic Riccati Equations, CAREs) into a tractable one (solving finite-horizon Riccati difference equations) while guaranteeing convergence to the true infinite-horizon solution. This allows AI agents to make decisions in complex, multi-agent environments with explicit performance guarantees.

Here are the specific improvements and capabilities:

1. Improved Agent Control Policy Computation (Tractability):

2. Guaranteed Convergence of Multi-Agent Learning/Planning (Stability):

3. Explicit Performance Bounds for Suboptimal Play (Risk Management):

4. Model Agnostic Strategy Approximation (Generalizability)::

5. The proposed finite-horizon strategy replaces the need to solve high-dimensional, coupled algebraic Riccati equations with a sequence of standard, finite-horizon Riccati difference equations solved recursively (Algorithm 1).

6. Improved Agent Control Policy Computation: AI agents can compute their optimal control inputs using a tractable, iterative backward algorithm. Instead of requiring real-time solution of coupled non-linear algebraic systems at every time step, the agent only needs to solve a sequence of linear systems derived from the finite-horizon structure, making high-dimensional multi-agent coordination feasible in real-time.

7. Guaranteed Convergence of Multi-Agent Learning/Planning: The paper proves that if all agents adopt this finite-horizon strategy and their prediction horizons tend to infinity, their total incurred cost converges to the cost of the true infinite-horizon FNE (Theorem 4, Part 1). This provides a theoretical guarantee that an agent’s approximate policy will eventually align with the globally optimal equilibrium in the long run.

8. Explicit Performance Bounds for Suboptimal Play: The paper derives an explicit cubic polynomial upper bound on the cost gap between the finite-horizon strategy and the limiting FNE (Theorem 4, Part 2). This allows AI systems to quantify exactly how much performance is lost due to using a finite prediction horizon, enabling proactive tuning of prediction horizons to minimize error.

9. Model Agnostic Strategy Approximation: The framework is applied to LQ games with state/input/output (i/o/s) dynamics and quadratic cost functions. This means the system can be used effectively in real-world scenarios (like robotics or autonomous driving) where the exact dynamics might be imperfectly modeled, as long as they fit the linear i/o/s structure. The strategy approximates the FNE even when output references are present (non-zero reference trajectories), enhancing its applicability beyond simple zero-reference tracking.

In summary, this research allows AI systems to move from computationally paralyzed equilibrium finding to a robust, guaranteed approximation framework for decentralized decision-making in complex, multi-agent environments.