Forecast Accuracy Is Not Trading Profit: Evolving Small Recurrent Networks for Stock Return Prediction

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Forecast Accuracy Is Not Trading Profit".

Jane: The gist The evolved networks rank first on both forecast accuracy and net trading performance,

Tom: First, who's behind it and why it matters.

Title and authors: Jane: We’ve been talking about the paper "Forecast Accuracy Is Not Trading Profit: Evolving Small Recurrent Networks for Stock Return Prediction," and it really shows that just being accurate on a prediction isn't the same as actually making money in trading.

Tom: Exactly. The title itself sets up this tension, focusing on the gap between a good forecast and a profitable trade execution when you factor in real-world costs.

Lu: The authors are comparing standard time series forecasting models—like ARIMA—against newer deep learning architectures, including recurrent networks evolved by neuroevolutionary search methods called EXAMM.

Meng: They are using this comparison to show that the way we evaluate these models needs to move beyond just looking at pointwise error and start looking at the full decision-making process.

Tom: So what’s the specific mechanism they focused on for improvement in "Forecast Accuracy Is Not Trading Profit: Evolving Small Recurrent Networks for Stock Return Prediction"?

Lu: They evolved deep recurrent connections that can span more than one time step directly, which lets a network read information from arbitrary past lags without having to pass everything through every intermediate state.

Jane: That’s a way of saying the model can focus its attention more directly on what matters in the temporal sequence rather than just processing everything sequentially.

Meng: They also introduced Algorithm one which is their daily long-short strategy, where they sort stocks by predicted return each trading day and then take equal-weight long positions in the top ten and short positions in the bottom ten.

Tom: That sorting mechanism is crucial because it directly links the prediction to a concrete action, moving beyond just a score.

Jane: And they mentioned that EXAMM’s entry rule re-enters the book only when every long-side prediction is positive and every short-side prediction is negative, which means their positions are held for several days rather than just one day.

Lu: It seems they found that this longer holding period helps them capitalize on the longer forecast horizons they achieved compared to other models.

Meng: From a deployment standpoint, this structure suggests we need to design our trading logic around these temporal rules, not just simple buy or sell signals.

The paper's summary: Tom: So what are the specific ways they improved things compared to existing approaches in "Forecast Accuracy Is Not Trading Profit: Evolving Small Recurrent Networks for Stock Return Prediction"?

Jane: They focused on evolving deep recurrent connections and edges that span more than one time step, which is a big part of how they changed the model structure.

Lu: They showed these recurrent edges let a network read information from arbitrary past lags directly, bypassing the need to propagate it through every intermediate state in the way standard gated architectures do, which has been shown to improve generalization.

Meng: They also introduced EXAMM, which is an asynchronous distributed search algorithm where a master process maintains the population and issues genomes on request while workers independently train genomes and report their fitness back without waiting on each other.

Tom: That asynchronous search mechanism is key because it allows for a massive search space to be explored efficiently, even though the total compute stays within eight point three to nine point nine core-hours across all configurations tested.

Jane: It sounds like they’re trying to find the best structure by letting the evolution happen in parallel rather than step-by-step.

Lu: They found that EXAMM evolved networks average sixty-six weights, and this search process was completed on CPU in under twenty minutes on thirty-two cores, which is quite fast for an architecture search.

Meng: And they also introduced Algorithm one again, the daily long-short strategy where stocks are sorted by predicted return and then positions are taken based on whether both long and short predictions meet their respective conditions.

Tom: So it seems the improvement isn't just in the structure of one model, but in how the system is designed to search for a structure that works across different time scales.

Jane: It really shows they weren't just tweaking existing models; they were evolving a whole new way to search for them.

The paper's improvements: Tom: So wrapping things up on "Forecast Accuracy Is Not Trading Profit: Evolving Small Recurrent Networks for Stock Return Prediction," the paper suggests that we shouldn't just focus on pointwise error when evaluating these systems.

Jane: It’s about looking at the whole process of prediction, from the initial ranking to actually holding positions and paying those transaction costs, which is where most of these models fall short.

Lu: The main implication is that these evolved networks can be discovered and deployed on commodity hardware, which isn't feasible for baseline architectures like Transformers.

Meng: Because they average sixty-six weights, fewer than any other model in the comparison and two to four orders of magnitude below the Transformers in terms of raw complexity, they execute in ten point eight two microseconds per prediction on a Raspberry Pi Zero.

Tom: That’s huge for production. You can put these models on embedded devices without needing massive computational power, which is what matters for real-time systems.

Jane: And the paper shows that EXAMM is the only model where accuracy, return, risk-adjusted return, and downside protection align at its smallest size.

Lu: It also opens the door for finding architectures where the parameter count is actually relevant to realized return, not just some abstract complexity metric.

Meng: So we’re left with EXAMM as the only model that achieves this alignment across those four metrics simultaneously at its smallest size.

Tom: I think so. It really shows a new direction for how we think about deploying these types of models in finance, focusing on finding something that works across accuracy and profit when costs are included.

Conclusion: Jane: So we’ve looked at "Forecast Accuracy Is Not Trading Profit: Evolving Small Recurrent Networks for Stock Return Prediction," and it really hammers home that just being accurate on a prediction isn't the same as actually making money in trading.

Tom: Exactly. It shows you have to look at the whole process, from picking a stock to holding it and paying those transaction costs, which is where most of these models fall short when they only focus on accuracy.

Lu: The authors are using neuroevolutionary methods like EXAMM to evolve recurrent connections that can span more than one time step directly, instead of having to pass everything through every single state transition.

Meng: And the search space they used was small enough that they found a model with sixty-six weights that actually runs on commodity hardware, which is what matters for us in production.

Tom: That’s a big deal. They found these evolved networks are two to four orders of magnitude below the Transformers in terms of raw complexity, but they still rank very highly across the board.

Jane: And this leads us to a bigger question: if these small models are so good, what does that say about the structure of financial markets compared to what we usually assume about deep learning models?

Lu: It suggests that sometimes simpler structures, like these evolved networks, can capture relevant temporal dependencies more efficiently than massive parameter counts.

Tom: And they found EXAMM is the only model whose alpha against the traded universe is positively distinguishable from zero, meaning it actually has a positive edge over just random noise or correlation.

Jane: That means it actually has a positive edge, not just some noise or random correlation that happens to look good on paper when you look at its information coefficient and Sharpe ratio across those four portfolios.

Meng: But they also pointed out that this model can rank the cross section well on average and still lose money once positions are formed and costs are charged.

Lalam: From my perspective, the paper proves that accuracy, return, risk-adjusted return, and downside protection align with the smallest model size.

Lu: It opens the door for finding architectures where the parameter count is actually relevant to realized return rather than just some abstract complexity metric.

Jane: So we're left with EXAMM as the only model that is best on all four metrics simultaneously at its smallest size.