Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels

arXiv:2601.07622 · cs.IT, cs.SY, eess.SY, math.IT · Submitted 2026-01-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Clipped Affine Policy".

Dev: This paper introduces low-complexity, near-optimal online power control policies for battery-limited point-to-point energy harvesting communications over slow block-fading channels.

Rosa: First, who's behind it and why it matters.

Title and authors: Dev: To summarize what we're seeing here, the core contribution of this work is developing a linear-policy-based approximation for the relative-value function within the Bellman equation for this power control problem. This approximation then allows them to derive two specific policies, one that's optimistic and another that's robust, which are essentially clipped affine policies.

Rosa: I see how they use these affine forms, sigma(b, gamma) = theta zero + theta one b - theta two/gamma, to represent the control strategy based on the battery level b and the channel SNR coefficient gamma. They’ve made this approximation much simpler than other general value iteration methods or deep RL approaches that require massive amounts of data to learn.

Taro: I see how they use these affine forms— sigma(b, gamma) = theta zero + theta one b - theta two/gamma —to represent the control strategy based on the battery level b and the channel SNR coefficient gamma. This approximation is what makes it lower complexity than other general value iteration methods or deep RL approaches that require massive amounts of data to learn.

Dev: Precisely, and this approximation is what makes it lower complexity than other general value iteration methods or deep RL approaches that require massive amounts of data to learn; it’s a big deal for real-time systems. They show that for specific conditions, like independent and identically distributed energy arrivals and channel states, they can develop two families of schemes based on these policies respectively.

Rosa: And they've shown that for specific conditions, like independent and identically distributed energy arrivals and channel states, they can develop two families of schemes based on these policies respectively; this is a neat way to structure the solution before moving into the more complex adaptive learning parts.

The paper's summary: Rosa: What I find really interesting about the improvements is how they extend these base policies into adaptive reinforcement learning schemes that call themselves RCA-RL. This extension allows the system to learn the parameters of this linear policy rather than just using a fixed approximation.

Dev: That extension is where things get practical for online control, because it allows the system to learn the parameters of this linear policy rather than just using a fixed approximation; you can tune the control strategy based on what you actually observe in your specific deployment environment.

Taro: The paper shows that this adaptive RCA policy, when extended with contextual information like one-step energy lookahead or channel lookahead, performs very well. This means it’s not just about having a good policy; it's about making that policy adaptive by feeding it immediate future information, which is vital for systems where conditions change quickly.

Rosa: It's not just about having a good policy; it's about making that policy adaptive by feeding it immediate future information, which is vital for systems where conditions change quickly; that contextual awareness makes the whole system much more responsive to sudden changes in the environment.

Dev: And the results are quite compelling; they report that when you look at both charging and discharging constraints, the RCA-OLA-A and RCA-RL schemes only incur less than about one percent performance loss compared to the optimal policy across a range of scenarios. That small loss is what makes it viable for practical use.

Taro: That small loss is what makes it viable for practical use; achieving results within about one percent of the true optimum while maintaining low complexity is a significant achievement when you consider these systems operate in real-world conditions, not just idealized simulations.

The paper's improvements: Dev: So, to wrap up on this paper, what we have here is a framework that uses an analytically tractable approximation of the relative-value function to generate clipped affine policies and then adapts those policies using worst-case analysis principles for real-time operation. It’s a solid foundation for building control loops that need to be fast and reliable.

Rosa: It seems the main implication is that we can get near-optimal performance in battery-limited energy harvesting communications without needing the enormous computational resources that some generic model-free reinforcement learning methods demand; this makes it accessible for many embedded systems.

Taro: I think this paper has a lot of implications because it shows how to leverage domain knowledge about the problem structure—the MDP formulation—to create a more stable and effective control system than purely data-driven approaches. It’s about using physics and math to guide the learning process, which is much more reliable than just throwing raw data at a generic agent.

Dev: It's certainly a competitive building block for practical EH wireless communication systems, Rosa, especially when we need low latency and high reliability; the paper establishes RCA-RL as a very effective way forward.

Rosa: Agreed, it’s a promising direction for designing control loops that are both efficient and robust under the uncertainty of energy harvesting; we have to keep an eye on how they handle those lookahead scenarios next.

Taro: I just want to add that the paper's mention of exploiting temporal correlations with Markov energy arrivals in RCA-RL-M suggests this is really heading toward systems that can anticipate future events, which is crucial when you're operating autonomously and managing resources dynamically.

Dev: That anticipation capability, combined with the low complexity, makes this a solid piece of work for deployment considerations; we need to see if they can maintain that level of performance when the channel conditions become even more volatile.

Rosa: Well, it’s definitely a promising direction for designing control loops that are both efficient and robust under the uncertainty of energy harvesting; I'm looking forward to seeing how this plays out in real-world deployments next time.

Conclusion: Rosa: So, to wrap up, this paper on "Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels" shows us a way to achieve near-optimal performance in battery-limited energy harvesting communications using simple linear policies and robust analysis.

Dev: Exactly, Rosa; the core idea is taking that complex value function and approximating it with clipped affine forms, which keeps the computational load manageable for real-time control loops. Taro I think what really stands out is how they manage that uncertainty by having both an optimistic policy and a robust one derived from worst-case analysis, which gives us a better picture of system behavior when things go sideways. Rosa It really makes sense that they’re looking at both certainty equivalence and worst-case scenarios because in real life, you can never be one hundred percent sure about the noise or the energy input.

Dev: That dual structure is key; it suggests they aren't just betting on one set of assumptions about the energy arrivals or channel states. Taro And they've shown that this adaptive RCA policy, when extended with contextual information like one-step energy lookahead or channel lookahead, performs very well. Rosa It’s not just about having a good policy; it’s about making that policy adaptive by feeding it immediate future information, which is vital for systems where conditions change quickly.

Dev: And the results are quite compelling; they report that when you look at both charging and discharging constraints, the RCA-OLA-A and RCA-RL schemes only incur less than about one percent performance loss compared to the optimal policy in a range of scenarios. Taro I just want to add that the paper's mention of exploiting temporal correlations with Markov energy arrivals in RCA-RL-M suggests this is really heading toward systems that can anticipate future events, which is crucial when you're operating autonomously. Rosa That anticipation capability, combined with the low complexity, makes this a solid piece of work for deployment considerations.

Dev: That anticipation capability, combined with the low complexity, makes it a solid piece of work for deployment considerations; it means we can actually put this kind of intelligent power control on edge devices where resources are tight. Taro I agree; being able to anticipate those energy arrivals changes how the whole system reacts to sudden drops in harvested power. Rosa It’s definitely a promising direction for designing control loops that are both efficient and robust under the uncertainty of energy harvesting.

Dev: So, to wrap up on this paper, what we have here is a framework that uses an analytically tractable approximation of the relative-value function to generate clipped affine policies, and then adapts those policies using worst-case analysis principles for real-time operation. Rosa It seems the main implication is that we can get near-optimal performance in battery-limited energy harvesting communications without needing the enormous computational resources that some generic model-free reinforcement learning methods demand.

Taro: I think this paper has a lot of implications because it shows how to leverage domain knowledge about the problem structure—the MDP formulation—to create a more stable and effective control system than purely data-driven approaches. Dev It's certainly a competitive building block for practical EH wireless communication systems, Rosa, especially when we need low latency and high reliability. Rosa Agreed, it’s a promising direction for designing control loops that are both efficient and robust under the uncertainty of energy harvesting.

Dev: That anticipation capability, combined with the low complexity, makes this a solid piece of work for deployment considerations. Taro I agree; being able to anticipate those energy arrivals changes how the whole system reacts to sudden drops in harvested power. Rosa It’s definitely a promising direction for designing control loops that are both efficient and robust under the uncertainty of energy harvesting.

Dev: So, we've looked at the methodology and results of "Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels." Rosa It’s a solid piece of work that moves us closer to practical implementations in remote sensing and IoT networks.

Taro: I think we should keep an eye on how they extend this framework to even more complex, dynamic environments where the channel fading itself is highly correlated with the energy supply.

Sussex Artificial Intelligence Institute of Zhejiang Gongshang University · Zhejiang University

cs.IT, cs.SY, eess.SY, math.IT

Submitted: 2026-01-12

Updated: 2026-10-01

Comments: 29 pages, 15 figures, v1.1

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: This paper introduces low-complexity, near-optimal online power control policies for battery-limited point-to-point energy harvesting communications over slow block-fading channels.

Key concepts

Markov Decision Process (MDP)
The power control problem is modeled as an MDP where the state includes the battery level and channel SNR. The goal is to maximize long-term throughput by choosing optimal transmission power at each time step, balancing energy consumption against data rate.
Clipped Affine Policies
These are simple linear policies derived from approximating the value function. They take a specific form ($ heta_0 + heta_1b - heta_2/ar{ u}$) and represent two strategies: an optimistic policy based on certainty equivalence and a robust policy based on worst-case channel conditions.
Worst-Case Analysis (RCA)
The robust policy is derived using worst-case analysis. This approach ensures the system performs reliably even under the most challenging channel conditions, providing a conservative but stable strategy for power allocation.
Adaptive Reinforcement Learning (RCA-RL)
This scheme extends the robust policy by making its parameters adaptive. It learns these parameters using contextual information like energy lookahead or temporal correlations, allowing the policy to adjust dynamically for better performance across various scenarios.

Terminology

Summary

This paper introduces low-complexity, near-optimal online power control policies for battery-limited point-to-point energy harvesting communications over slow block-fading channels. The core contribution is a linear-policy approximation of the relative-value function in the Bellman equation, which leads to two fundamental parameterized clipped affine policies—an optimistic policy and a robust policy—and extends these to adaptive reinforcement learning schemes that achieve superior performance compared to generic model-free RL baselines.

Problem Formulation and MDP Modeling

The power control problem is formulated as a Markov Decision Process (MDP) where the state at time t is defined as St:= (Bt, Γt), representing the battery level and the channel SNR coefficient. The goal is to maximize the long-term expected throughput, G((Ut)∞ t=1). The system dynamics are governed by equations describing battery evolution: Bt+1 = (Bt - Ut)/ηd + ηc⟨Et⟩≤Ecmax ≤c, subject to constraints like the energy-causality constraint (Ut/ηd ≤ Bt) and the maximum-dischargeable-energy constraint (Ut ≤ Edmax). The reward function is defined by the data rate achieved: r(ΓtUt) = log(1 + ΓtUt).

Clipped Affine Policies

The paper derives a linear-policy-based approximation to the relative-value function, leading to two parameterized policies based on certainty equivalence and worst-case analysis. These are coined clipped affine policies, taking the form: σ(b, γ) = θ0 + θ1b − θ2/γ. The optimistic policy is derived from a certainty-equivalence-type approximation (OCA), while the robust policy is derived from worst-case analysis (RCA). These policies are shown to be batterylimited weighted directional waterfilling mechanism operating between adjacent time slots, an online counterpart of the directional waterfilling principle [25] in the offline setting.

Adaptive and Closed-Form Schemes

Building on these policies, two families of schemes are proposed: those based on OCA and those based on RCA. The adaptive RCA policy (RCA-RL) is extended to address four scenarios with contextual information: one-step energy lookahead, one-step channel lookahead, one-step joint energy-channel lookahead, and Markov energy arrivals. The best overall performance is achieved by the adaptive RCA policy based on the maximin optimal linear-policy slope approximation (RCA-OLA-A) and the RCA-RL scheme. Furthermore, the best closed-form policy is identified as the RCA policy based on the maximin optimal linear policy (RCA-OL).

Performance and Robustness Analysis

Extensive simulation results demonstrate that these proposed schemes provide a favorable tradeoff between computational complexity and performance. The adaptive RCA-RL scheme achieves less than 2% performance loss relative to the optimal policy across a range of scenarios. Specifically, the RCA-OLA-A and RCA-RL achieve less than about 1% performance loss under charging and discharging constraints. The paper highlights that the adaptive estimate of the DMCR (p) is highly stable, indicating that RCA-RL exhibits satisfactory stability, rapid convergence, and robust performance over a practical range of learning rates.

Contextual Information Extensions

The paper extends the schemes to handle contextual information by incorporating lookahead data. For one-step energy lookahead (RCA-RL-ELK), the DMCR estimate p is replaced by a conditional expectation variant, and for one-step channel lookahead (RCA-RL-CLK), the parameter γˆ can be estimated using a function of the next channel SNR coefficient. The scheme for Markov energy arrivals further exploits temporal correlation by using contextual information like the most recent energy arrival to predict p and q via binning functions and multilayer perceptrons (MLPs). The results show that RCA-RL-M and RCA-RL-MP significantly outperform generic model-free RL baselines, especially in exploiting temporal correlations.

Conclusion

The work establishes a general design approach for energy management in EH wireless communication systems: identifying an analytically tractable approximation of the relative-value function, applying worst-case analysis for current action optimization, and developing an adaptive scheme to learn the parameters of this approximation. The findings suggest that the RCA-RL scheme is more effective than generic model-free RL methods because it leverages domain knowledge about the underlying problem structure rather than relying solely on generic RL methods. The proposed power control schemes form a competitive and promising building block for practical EH wireless communication systems.

The gist

A linear-policy-based approximation to the relative-value function, two parameterized clipped affine policies, and their corresponding closed-form policies and adaptive schemes for power control in point-to-point EH wireless communication systems. The low complexity and high performance of these algorithms are demonstrated through comparative analysis in Tables I and IV and extensive simulation results in Sec. VI.

Improvements for AI systems

As a fastidious and diligent AI researcher, I have analyzed this paper, Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels. The core contribution is the development of low-complexity, near-optimal power control policies for battery-limited energy harvesting (EH) communications by approximating the relative-value function using linear policies.

The improvements derived from this paper are highly specific to systems where energy management and resource allocation under fading/stochastic conditions are critical.

Here are the specific improvements that can be made to AI systems, categorized by application:

) Specific Improvements for AI Systems:

(1) Acknowledge the necessity of a two-tier policy design: an optimistic policy (OCA), derived from certainty equivalence, and a robust policy (RCA), derived from worst-case analysis. This structure should be leveraged in any online decision-making system where uncertainty is high.

(2) Implement a linear-policy approximation for the relative-value function in Markov Decision Processes (MDPs) governing resource allocation problems (e.g., scheduling, power control). This approximation significantly reduces computational complexity compared to general value iteration or deep RL methods, making real-time deployment feasible on low-power edge devices.

(3) Develop an adaptive learning framework based on the worst-case analysis principle (RCA). Instead of relying solely on empirical data to estimate policy parameters, the system should be designed to maintain robustness against potential model inaccuracies by using worst-case estimates (like DMCR or MCR) as conservative bounds for control decisions.

(4) Integrate contextual information into RL/control policies through lookahead mechanisms. Specifically, design modules that allow the system to incorporate:

(a) One-step energy lookahead (predicting immediate energy arrival).

(b) One-step channel lookahead (predicting immediate channel state variation).

(c) Joint energy-channel lookahead.

(5) Design specialized RL extensions for temporal correlation in stochastic processes. For scenarios with Markovian or persistent energy arrivals, the system should use context-aware predictors (e.g., MLP predictors based on recent history) to estimate dynamic parameters (like the DMCR, 'p') used in policy selection, rather than relying on static estimates.

(6) Employ a hybrid learning architecture that combines analytical closed-form policies with reinforcement learning. The RL component should be reserved for parameter estimation (learning the linear policy slope 'q' and channel coefficient 'γˆ') rather than performing high-dimensional action search, thus achieving superior sample efficiency and stability compared to pure model-free methods like DQN or PPO.

(7) Utilize a Semi-Adaptive online learning scheme (like RCA-OL-SA) where parameters are updated based on observed energy and capacity dynamics, providing a balance between the speed of adaptation and the stability derived from worst-case analysis.

) What the Improved AI System Can Do:

The resulting improved AI system will be capable of performing highly efficient, robust online decision-making in environments characterized by intermittent energy supplies and fluctuating communication channels. Specifically, it can:

(1) Maximize throughput in battery-limited wireless communication links (e.g., IoT sensor networks or remote sensing) while ensuring the policy is near-optimal with minimal computational overhead (low complexity).

(2) Operate reliably under uncertainty by explicitly incorporating worst-case channel and energy scenarios, guaranteeing performance close to the theoretical optimum even when environmental statistics are unknown or poorly characterized.

(3) Adapt its control strategy in real-time by learning the most relevant system parameters (like the effective energy arrival rate, 'p') directly from observed battery dynamics, leading to faster convergence and better performance than static baseline controllers.

(4) Exhibit superior temporal correlation handling: When energy arrivals are not i.i.d., the system can use short-term history (contextual information) to make more informed decisions about power allocation, significantly boosting throughput compared to generic RL agents that treat all time steps independently.

(5) Handle dynamic constraints: The system can efficiently manage complex operational constraints, such as maximum charge/discharge limits per time slot, by clipping the derived affine policies directly (as shown in Remark 3), ensuring physical feasibility while optimizing performance.

Sources

Related papers