RNN(p) for Power Consumption Forecasting

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "RNN(p) for Power Consumption Forecasting".

Jane: The RNN(p) model introduces an elementary Recurrent Neural Network architecture that generalizes linear autoregressive models ARX(p) and demonstrates excellent forecasting accuracy for power consumption,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper called "RNN(p) for Power Consumption Forecasting" by Roberto Baviera and Pietro Manzoni. It’s basically introducing a new, simpler way to build recurrent neural networks that are specifically good for time series with seasonal patterns, like power consumption data.

Jane: A new architecture that generalizes those linear autoregressive models, ARX(p), so it's taking something established and making it nonlinear for forecasting energy use. What does this actually mean for us when we talk about the real world?

Tom: Well, the core idea is this RNN(p) model uses multiple Jordan feedbacks of order p to create a structured feedback loop, which is different from just relying on the previous hidden state like in Elman-type RNNs. This structure allows it to directly use lagged outputs yˆ(t−k) for k from one up to p <ref:2209.01378#pg4>.

Lu: What I find interesting is that this architecture, shown in Figure one maintains a high degree of interpretability because the feedback connections are clearly defined between the outputs and the hidden layer h(t) <ref:2209.01378#pg6>. It's easier to trace what's happening inside than some of those more complex gating mechanisms in LSTMs or GRUs <ref:2209.01378#pg2>.

Meng: From a deployment standpoint, simplicity is often good, but we have to talk about the cost of running these models. The paper lays out three main ways to train them—Real-Time Recurrent Learning, Backpropagation Through Time, and Adjoint Automatic Differentiation. That’s where things get interesting for us <ref:2209.01378#pg8>.

Jane: Right, so how do these training methods compare in terms of what it actually takes to compute? Tom, you mentioned the computational complexity earlier. What’s the big difference between those three approaches?

Tom: Yeah, well Real-Time Recurrent Learning has a time complexity of O(τ p y h w) and space complexity of O(p y w) <ref:2209.01378#pg10>. It’s efficient because it doesn't need to store all the previous inputs or hidden states, just the last p total derivatives <ref:2209.01378#pg10>.

Jane: And Backpropagation Through Time, BPTT, it seems like that would get much worse if p is bigger than one? What does that exponential increase actually look like on the page?

Tom: It's pretty wild when p is greater than one because it results in nested temporal dependencies leading to an exponential increase in computational workload as shown by the growth of nodes in the unrolled tree <ref:2209.01378#pg7>. The complexity is O(Sτ, p h w), where Sτ,p is defined as the sum of the first τ terms of the p-bonacci sequence <ref:2209.01378#pg8>.

Title and authors: Lu: That’s a significant bottleneck for long sequences. But then we have Adjoint Automatic Differentiation, AAD, which they claim is more efficient in time complexity with O(τ h w) and space complexity of O(τ (x + h + y)) <ref:2209.01378#pg10>.

Jane: So if I'm looking at the numbers, AAD seems like it’s the preferred way to compute gradients for these RNN(p) models, especially when p is larger than one. What are we actually doing with this architecture in practice?

Tom: They applied these models to two specific datasets: total load data for the New England region and net load data for the London area <ref:2209.01378#pg9>. The forecasting task is mid-term, using hourly resolution to predict one year ahead <ref:2209.01378#pg6>.

Meng: And they didn't just use raw data, did they? I remember seeing that preprocessing step <ref:2209.01378#pg9>.

Jane: Right, the preprocessing involved removing trend and seasonal components using a linear model fitted on in-sample data, including Fourier terms for the day of the year and dummy variables for days of the week or holidays <ref:2209.01378#pg9>. It sounds like they aren't just feeding it raw numbers, they’re trying to clean it up first.

Tom: After that, they trained these RNN(p) models using a moving window approach with a lookback period τ of forty-nine to capture both hourly and daily dependencies <ref:2209.01378#pg6>. The network output can be configured for point forecasting minimizing mean squared error, or probabilistic forecasting by minimizing Gaussian Negative LogLikelihood, NLL <ref:2209.01378#pg6>.

Lu: And they didn't just stop at training; they tested the structure itself by comparing different lag structures. They found that using the full multi-lag structure with p equals one two and twenty-four leads to substantial gains in predictive accuracy on both datasets <ref:2209.01378#pg9>.

Jane: Substantial gains are good, but what about the caveat? Did they find that you need that full multi-lag structure for every single case?

Tom: They did, and this is a key point. The inclusion of the full multi-lag structure p = one two and twenty-four reduces both MAPE and RMSE by nearly ten percent compared to using just the single-lag variant <ref:2209.01378#pg9>. That’s a real number showing the benefit of that structure.

Meng: For someone trying to put this into production, that reduction in error translates directly into operational savings, right? They mention something about economic implications for the London dataset <ref:2209.01378#pg9>.

Title and authors: Jane: Exactly, and they point out that transitioning from statistical models like ARX or SARIMAX to this RNN(p) framework could yield annual savings of several tens of millions of pounds due to reduced imbalance settlement costs <ref:2209.01378#pg9>. That’s a massive potential impact on the energy market.

Tom: And because they are doing probabilistic forecasting, they can actually use those predicted distributions to create risk-adjusted bidding strategies and pricing load-following derivatives in electricity markets <ref:2209.01378#pg9>. It moves beyond just giving a single number to give you a range of possibilities.

Lu: And I think the interpretability aspect is also important here, even if it's not the main focus of this specific paper on RNN(p) <ref:2209.01378#pg5>. They emphasize that this architecture provides clear insights that help practitioners understand model behavior <ref:2209.01378#pg5>.

Jane: So, to summarize, the main point of RNN(p) for Power Consumption Forecasting is that it’s a structured way to build recurrent networks that handles seasonal patterns better than simpler models and gives us a way to get better, more reliable forecasts with probabilistic outputs. What should we expect next in this research?

Tom: They seem focused on proving the accuracy across different lag structures, so they’ll keep pushing that comparison forward <ref:2209.01378#pg9>. It’s about showing that the structure itself is beneficial, not just whatever specific model you pick.

Meng: From a production standpoint, I'd be looking at how easily we can implement AAD when we have massive time series data streams <ref:2209.01378#pg10>. If the complexity stays linear with τ, that’s good for scaling up our inference pipeline.

Lalam: Based on what I'm seeing in the literature, this RNN(p) framework is a solid extension of ARX(p), making it a natural fit when you need to model those specific types of time dependencies <ref:2209.01378#pg6>. I’m quite sure this approach will be used again because it solves the problem of capturing those complex seasonal dynamics in energy data <ref:2209.01378#pg5>.

Jane: It sounds like a very practical tool for someone who only listens to the show—it gives you a clearer picture of how much better the numbers actually are when you include those extra lags and that probabilistic output. So, we’ve seen how RNN(p) tackles power consumption forecasting by using structured feedback loops to improve accuracy and give us risk-aware tools for energy markets.

Tom: That's right. We're wrapping up our look at the RNN(p) paper <ref:2209.01378#pg1>. We’ll be back next time when we look at how other models are trying to tackle similar sequential dependencies in the literature <ref:2209.01378#pg5>.

The paper's summary: Tom: So we just walked through how this RNN(p) model builds its structure using those multiple Jordan feedbacks to handle those lagged outputs for power consumption data, right?

Jane: Exactly, and what this paper is really summarizing is that they took a standard linear model called ARX(p) and made it nonlinear using this new recurrent setup. It’s basically taking an old tool and giving it a brain that can learn more complex patterns in the energy use.

Tom: Right, so the main takeaway from their summary is that this RNN(p) approach doesn't just fit the data; it captures these specific seasonal and temporal dependencies much better than those simpler models <ref:2209.01378#pg6>.

Jane: And they showed that when you use the full multi-lag structure, specifically p equals one, two, and twenty-four, you get a real jump in accuracy on both the New England and London datasets <ref:2209.01378#pg9>. That's a concrete number showing that adding those extra lags isn't just academic; it actually helps the model learn the right patterns.

Tom: I’m seeing that, and they also highlighted how this model handles data cleaning differently than before <ref:2209.01378#pg9>. They found out that if you remove the trend and seasonality using a simple linear model, performance degrades substantially across all lag configurations <ref:2209.01378#pg5>.

Jane: That means you can't just throw raw data at it and expect great results; you have to do that preprocessing step first, like fitting Fourier terms for the day of the year and dummy variables for holidays <ref:2209.01378#pg9>. It’s about giving the model a head start on what it should be looking at.

Tom: And they didn't just stop at getting a single best guess; they also focused on probabilistic forecasting, using Gaussian Negative LogLikelihood, NLL <ref:2209.01378#pg6>. This means instead of just giving you one number for tomorrow’s load, the model gives you a range of possibilities <ref:2209.01378#pg6>.

Jane: Which is huge for decision making, because that distribution lets you build risk-aware bidding strategies in electricity markets <ref:2209.01378#pg9>. It moves the forecasting from a simple prediction to a tool for managing uncertainty.

Tom: And if you look at the computational side, they showed that Adjoint Automatic Differentiation, or AAD, is the most efficient way to train this thing <ref:2209.01378#pg10>. It’s much faster than Backpropagation Through Time when p is bigger than one <ref:2209.01378#pg7>.

Jane: So, what does that mean for us in terms of deployment? It means we can train these more complex models without the training process taking forever to finish <ref:2209.01378#pg10>. We’re not looking at exponential growth in time complexity anymore, which is a big relief for scaling up our systems.

Tom: It’s about making sure the math actually scales with the data size, not just with the complexity of the structure itself <ref:2209.01378#pg10>. The authors are really stressing that this model is well-suited for capturing those nonlinear dynamics in real-world time series <ref:2209.01378#pg5>.

Jane: So, to wrap up this section, the paper’s summary boils down to using a structured recurrence that handles seasonality better and gives you a probabilistic output that helps with risk management. That sets us up perfectly for looking at how these models are actually being deployed in real energy markets next.

The paper's improvements: Tom: So we’ve talked about how they built the RNN(p) model and how they trained it using different math, right? Now let’s look at what they suggest is better than just that basic setup.

Jane: Right, so the authors point out a few specific things that make this RNN(p) architecture really strong when you apply it to energy forecasting <ref:2209.01378#pg5>.

Tom: They show that by using the full multi-lag structure with p equals one, two, and twenty-four, the predictive accuracy improves significantly on both the New England and London datasets <ref:2209.01378#pg9>. That’s a real reduction in error compared to just using a single lag setup.

Jane: That means for those energy companies trying to predict their load, they can get much more reliable numbers, which is the whole point of the research <ref:2209.01378#pg9>. And they also emphasize that this model handles raw data better than before <ref:2209.01378#pg5>.

Tom: They’re saying you can't just strip away all the seasonal and trend stuff using a simple linear model and expect good results; you need that explicit handling of seasonality to get optimal outcomes <ref:2209.01378#pg5>. That’s a necessary condition for success.

Jane: And beyond just accuracy, they focus on the output itself, which is the probabilistic forecast <ref:2209.01378#pg6>. This means the model spits out a Gaussian random variable with both a mean and a standard deviation <ref:2209.01378#pg6>.

Tom: That’s what we talked about before, but the improvement is in how it changes things for someone who only listens to the show. The probabilistic forecast lets you build risk-aware bidding strategies in day-ahead markets <ref:2209.01378#pg9>. It gives you a range of possibilities instead of just one guess.

Jane: Exactly, and that’s where the economic implications hit hard for the London dataset <ref:2209.01378#pg9>. It could lead to annual savings in the tens of millions of pounds because companies can manage their risk better with those distributions <ref:2209.01378#pg9>.

Tom: And on the training side, they’ve shown that using Adjoint Automatic Differentiation, or AAD, is the most efficient way to train this thing <ref:2209.01378#pg10>. It’s much faster than Backpropagation Through Time when p is bigger than one <ref:2209.01378#pg7>.

Jane: So, the main improvement here is making the whole process more practical; it’s not just about getting a fancy result, but doing it in a way that actually runs fast enough for real-world deployment <ref:2209.01378#pg10>.

Tom: The authors are really pushing that idea that this RNN(p) model is well-suited for capturing those nonlinear dynamics in real-world time series forecasting, which is what makes it so relevant for energy markets <ref:2209.01378#pg5>.

Jane: And while they show great performance, they do flag that the architecture provides clear insights into model behavior, which helps practitioners understand what the AI is actually doing <ref:2209.01378#pg5>.

Tom: It’s about giving a transparent architecture that’s also accurate, which is a difficult balance to strike in machine learning right now.

Conclusion: Tom: So to wrap up, we’ve seen how the RNN(p) for Power Consumption Forecasting model uses structured feedback loops to handle seasonal patterns better than simpler models and gives you a probabilistic output that helps with risk management.

Jane: Exactly, so this paper shows that by adding those specific lags and using a Gaussian distribution for the forecast, you can move from just getting a single prediction to having a tool for managing uncertainty <ref:2209.01378#pg9>.

Tom: And that probabilistic output is key because it lets you build risk-aware bidding strategies in electricity markets <ref:2209.01378#pg9>. It makes the AI useful for actual financial decisions.

Jane: It really does change things for someone who only listens to the show because it shows how a model can be used to create more robust and economically viable energy strategies <ref:2209.01378#pg9>.

Lu: I think what’s really significant is the way they frame this as a simple nonlinear extension of those linear ARX(p) models <ref:2209.01378#pg6>. It shows that you don't always need the most complex deep learning architecture to capture those specific time dependencies.

Tom: Right, and on the training side, they proved that Adjoint Automatic Differentiation is the most efficient way to train this thing <ref:2209.01378#pg10>. It’s much faster than Backpropagation Through Time when p is bigger than one <ref:2209.01378#pg7>.

Jane: So, for deployment, that means we can train these more complex models without the training process taking forever to finish <ref:2209.01378#pg10>. It’s a big win for scaling up our systems.

Meng: From a production standpoint, I'm interested in how easy it is to actually implement AAD when we have massive time series data streams <ref:2209.01378#pg10>. If the complexity stays linear with tau, that’s good for scaling up our inference pipeline.

Lalam: I'm very sure of what they found; this RNN(p) framework is a solid extension of ARX(p), making it a natural fit when you need to model those specific types of time dependencies <ref:2209.01378#pg6>. It’s proven to work well for these energy problems.

Lu: The authors are really stressing that this model is well-suited for capturing those nonlinear dynamics in real-world time series forecasting <ref:2209.01378#pg5>. That's the core idea they’re pushing.

Tom: So, to wrap up on the RNN(p) for Power Consumption Forecasting paper, we've seen how they built a structured recurrence that handles seasonality better and gives you a probabilistic output that helps with risk management.

Jane: It’s about using structured feedback loops to improve accuracy and give us risk-aware tools for energy markets <ref:2209.01378#pg5>.

Tom: We've talked about the numbers, the caveats, and what it changes for someone who only listens to the show.

Lu: I think what’s next is seeing how this structured approach compares against other recurrent models that handle these dependencies in literature <ref:2209.01378#pg5>.

Jane: It sounds like a very practical tool for someone who only listens to the show, giving them a clearer picture of how much better the numbers actually are when you include those extra lags and that probabilistic output.

Meng: I'd be looking at how easily we can implement AAD when we have massive time series data streams <ref:2209.01378#pg10>. If the complexity stays linear with tau, that’s good for scaling up our inference pipeline.

Lalam: Based on what I'm seeing in the literature, this RNN(p) framework is a solid extension of ARX(p), making it a natural fit when you need to model those specific types of time dependencies <ref:2209.01378#pg6>.

Tom: That's right. We're wrapping up our look at the RNN(p) for Power Consumption Forecasting paper <ref:2209.01378#pg1>. We’ll be back next time when we look at how other models are trying to tackle similar sequential dependencies in the literature <ref:2209.01378#pg5>.