How Far Can 5,500 Hours of Driving Take You? A Scaling Law Analysis of Video Diffusion Models

We present a systematic scaling-law study of video diffusion models trained from scratch on driving data: a family of models from 1M to 9B parameters, trained at different exposures on up to 5,500 hours of driving. Validation loss follows consistent power laws in both model size and training exposure, answering the questions that shape a training budget: whether compute is better spent on longer training or on a larger model, and whether more data is needed. Loss improves much faster with training exposure than with model size, making longer training the most effective way to improve a fixed model under limited compute. However, larger models continue to achieve lower asymptotic loss, so compute-optimal scaling still favors increasing model size when sufficient compute and data are available.

Guided by these laws, we train a 9B-parameter model, to our knowledge the largest video diffusion model trained from scratch on driving data: it sets a new open-source state of the art for driving video generation, as measured on nuScenes.

The study investigates how far a fixed dataset, 5,500 hours of driving video, can take a video diffusion model. To answer this, we need to know exactly where each unit of compute is best spent: is it better to build a larger model, or to train for more steps? Two differences separate our setting from these studies [on LLMs]: First, the optimization objective relies on denoising or flow matching rather than next-token prediction. Second, we operate in the data-constrained regime, unlike LLMs whose text corpora are practically unbounded. In Chinchilla-style scaling [13], growing the data axis means sourcing more unique tokens; here, the driving corpus is fixed at 5,500 hours: far larger than any public driving dataset, yet still small enough that repeated epochs over the same clips are unavoidable. Training exposure, the total number of samples seen, unique or repeated alike, thus becomes the primary scalable resource. We fit a simple asymptotic power law, L(x) = L0 + A x−α, to the validation loss along three complementary axes: model scaling (x:= N), training scaling (x:= D), and compute scaling (x:= C).

The study proceeds in two steps: first we fit the scaling laws, then we spend the budget where they point. The fits are accurate across more than 200 runs spanning 1.6M to 1.1B parameters, and three findings follow:

(1) The training-exposure exponent is far steeper than the model-size exponent, so longer training is the fastest way to improve a fixed model, while the asymptotic loss keeps falling with capacity, so compute-optimal budgets still favor larger models.

(2) The fixed corpus does not break the laws: within the fewer than five epochs we run, repeating the data behaves like fresh samples [20, 26], so the 5,500 hours are not yet the bottleneck. We hypothesize this is because flow matching resamples noise and timestep at every pass, unlike the fixed targets of autoregressive training.

(3) The laws extrapolate: a single 9B-parameter model, to our knowledge the largest open-source video diffusion model trained from scratch on driving data, reaches a validation loss predicted before training to within 3.6% of a law extrapolated 8× beyond the largest fitted model, the lowest loss among all our models.

The analysis is structured around three axes:

Model scaling (x=N):

We investigate how the validation loss scales with model size N under fixed training exposure (10M samples). The fitted law is:

L(N) = 0.0595 + 0.1041 · N−0.2125, where the fitted exponent is α N = 0.2125. Extrapolating to a 9B-parameter model trained with the same exposure (D ≈ 10 7 samples) predicts a validation loss of L(9B) ≈ 0.0746.

Training scaling (x=D):

We investigate how validation loss scales with training exposure D while keeping model size fixed. The fitted law is L(D) = L0 + AD−αD across model sizes, where the convergence exponent α D shows no systematic trend across scale (0.74 on average, spread 0.66–0.84). We estimate that training up to 1.5 × 10 7 samples captures most of the achievable gain: with the mean fitted coefficients of Fig. 3a (A ≈ 0.005, αD ≈ 0.74), the loss reduction attainable beyond this exposure is below 0.004.

Compute scaling (x=C):

We investigate scaling under a fixed compute training budget C (TFLOPs). The fitted law is:

L(C) = 0.0522 + 0.5955 · C−0.15443. Combining the predicted optimal compute allocation with the planned 9B architecture gives, for a model trained on approximately 10 7 samples (1.438 × 10 9 TFLOPs), an estimated validation loss of L(1.438 × 10 9) ≈ 0.0753.

Guided by the scaling laws, we train our 9B model and evaluate it against prominent open-source driving world models [7, 9]. The resulting model sets a new open-source state of the art for driving video generation on nuScenes. Validation loss at 9B scale is predicted to be 0.0753 before training, and after training up to 1.2 × 10 7 samples (approximately 2 epochs), the model reaches a validation loss of 0.0781, the lowest among all evaluated models, demonstrating continued improvements when scaling from 1B to 9B parameters. This closely matches the extrapolated prediction, with only a 3.6% relative error. The post-hoc fitted asymptotic loss (L 0 ≈ 0.0748) sits within 1% of the a-priori compute-law prediction of 0.0753, providing an independent confirmation of the extrapolated estimate.

The study concludes that across more than 200 training runs spanning 1.6M to 1.1B parameters, three findings emerge: (1) the training-exposure exponent is far steeper than the model-size exponent, so longer training is the fastest way to improve a fixed model, while compute-optimal budgets still favor larger models as capacity keeps lowering the asymptotic loss; (2) the fixed corpus does not break the laws: repeated data behaves like fresh samples, so the 5,500 hours are not yet the bottleneck; and (3) the laws extrapolate: a 9B-parameter model reaches a validation loss predicted before training to within 3.6% of a law extrapolated 8× beyond the largest fitted model. The main limitation is the scale gap between the models used for fitting and the flagship: intermediate scales (e.g., 3B–4B) would tighten extrapolated estimates.

The resulting model sets a new open-source state of the art for driving video generation, as measured on nuScenes. We further post-finetune it with pseudo-trajectory conditioning to enable action control and fair comparison with baselines, which all rely on trajectory conditioning. The 9B model reaches a loss comparable to the fitted asymptote of the 1.1B model (L 0=0.078, Fig. 3a), a level that the smaller 1.1B model would require more than an order of magnitude additional training exposure to approach, in contrast to the 9B model reaching this regime within 1.2 × 10 7 samples while remaining far from its own asymptote (Fig. 6). The post-training fine-tuning on nuScenes achieves the best performance on both image and video-based perceptual metrics compared to prior work like GEM and Vista, which use parameter-efficient adaptation.

The main limitation is the scale gap between the models used for fitting and the flagship: intermediate scales (e.g., 3B–4B) would tighten extrapolated estimates. The main limitation is that the scaling laws assume a uniform learning rate; since the 9B model required adjusting the learning rate during training for stability, a hyperparameter grid search could further tighten our predictions.

The study answers its title question: 5,500 hours of driving take a from-scratch video diffusion model to a new open-source state-of-the-art for driving video generation, as measured on nuScenes. The main limitation is the scale gap between the models used for fitting and the flagship: intermediate scales (e.g., 3B–4B) would tighten extrapolated estimates.

The authors hope these scaling laws provide a practical basis for training future driving world models. This work answers its title question: 5,500 hours of driving take a from-scratch video diffusion model to a new open-source state-of-the-art for driving video generation, as measured on nuScenes. Across more than 200 training runs spanning 1.6M to 1.1B parameters, three findings emerge: (1) the trainingexposure exponent is far steeper than the model-size exponent, so longer training is the fastest way to improve a fixed model, while compute-optimal budgets still favor larger models as capacity keeps lowering the asymptotic loss; (2) the fixed corpus does not break the laws: repeated data behaves like fresh samples, so the 5,500 hours are not yet the bottleneck; and (3) the laws extrapolate: a 9B-parameter model, to our knowledge the largest open-source video diffusion model trained from scratch on driving data, reaches a validation loss predicted before training to within 3.6% of a law extrapolated 8× beyond the largest fitted model, the lowest loss among all our models. The main limitation is the scale gap between the models used for fitting and the flagship: intermediate scales (e.g., 3B–4B) would tighten extrapolated estimates.

The resulting model sets a new open-source state of the art for driving video generation on nuScenes. We further post-finetune it with pseudo-trajectory conditioning to enable action control and fair comparison with baselines, which all rely on trajectory conditioning. The 9B model reaches a loss comparable to the fitted asymptote of the 1.1B model (L 0=0.078, Fig. 3a), a level that the smaller 1.1B model would require more than an order of magnitude additional training exposure to approach, in contrast to the 9B model reaching this regime within 1.2 × 10 7 samples while remaining far from its own asymptote (Fig. 6). The post-training fine-tuning on nuScenes achieves the best performance on both image and video-based perceptual metrics compared to prior work like GEM and Vista, which use parameter-efficient adaptation.

The study answers its title question: 5,500 hours of driving take a from-scratch video diffusion model to a new open-source state-of-the-art for driving video generation, as measured on nuScenes. Across more than 200 training runs spanning 1.6M to 1.1B parameters, three findings emerge: (1) the trainingexposure exponent is far steeper than the model-size exponent, so longer training is the fastest way to improve a fixed model, while compute-optimal budgets still favor larger models as capacity keeps lowering the asymptotic loss; (2) the fixed corpus does not break the laws: repeated data behaves like fresh samples, so the 5,500 hours are not yet the bottleneck; and (3) the laws extrapolate: a 9B-parameter model, to our knowledge the largest open-source video diffusion model trained from scratch on driving data, reaches a validation loss predicted before training to within 3.6% of a law extrapolated 8× beyond the largest fitted model, the lowest loss among all our models. The main limitation is the scale gap between the models used for fitting and the flagship: intermediate scales (e.g., 3B–4B) would tighten extrapolated estimates.

The study answers its title question: 5,500 hours of driving take a from-scratch video diffusion model to a new open-source state-of-the-art for driving video generation, as measured on nuScenes. Across more than 200 training runs spanning 1.6M to 1.1B parameters, three findings emerge: (1) the trainingexposure exponent is far steeper than the model-size exponent, so longer training is the fastest way to improve a fixed model, while compute-optimal budgets still favor larger models as capacity keeps lowering the

Here are the specific improvements that can be made to AI systems, based on the findings of this research:

1. The most effective way to improve a fixed video diffusion model under limited compute is by increasing the amount of training data (training exposure), rather than solely relying on increasing model size. The training-exposure exponent is significantly steeper than the model-size exponent, indicating that longer training steps yield faster improvements in loss reduction for a given budget.

2. For achieving lower asymptotic validation loss, computational resources should be allocated toward larger models, even under fixed data constraints, as larger capacities continue to drive the loss down asymptotically.

3. The fixed dataset size of 5,500 hours is not yet a bottleneck for diffusion models when training exposures are sufficient (i.e., more than a few epochs). The use of flow matching—which resamples noise and timesteps at every pass—allows repeated data to behave like fresh samples, suggesting that the model can effectively leverage the existing corpus without immediate saturation.

4. A compute-optimal training budget is achieved by jointly scaling model size and training exposure. The compute scaling law suggests that for a target large model (like 9B), an optimal training exposure of approximately 107 samples (about 2 epochs) is sufficient to reach a validation loss close to the theoretical minimum predicted by extrapolating the scaling laws.

5. The derived models, especially the trained 9B parameter model, set a new open-source state-of-the-art for driving video generation on benchmarks like nuScenes.

6. The resulting 9B model can generate high-fidelity driving videos (as measured by FID and FVD metrics) that exhibit superior temporal coherence and long-range motion consistency compared to smaller models, especially when augmented with trajectory conditioning.

This improved AI system can perform the following:

1. Generate highly realistic, physically consistent video sequences of autonomous driving scenarios from scratch using only 5,500 hours of driving footage.

2. Produce world models capable of predicting future driving scenes and actions by leveraging learned representations from this fixed corpus, making it robust for simulation and planning tasks.

3. Perform sophisticated trajectory control and scenario generation by incorporating pseudo-trajectories (pseudo-trajectories extracted via OccAny) directly into the generation process, allowing the AI to generate videos conditioned on specific maneuvers (e.g., hard braking or sharp turns).

4. Serve as a competitive, open-source foundation model for driving video generation that surpasses existing proprietary models in terms of perceptual quality and temporal fidelity across various driving modalities.