When gas left the map: Evergreen on a model going stale

Building a good model is no longer the hard part. A capable data scientist, or an AI assistant, can produce a strong forecaster in an afternoon. The hard part starts the day it ships, because the world it learned from starts to move. This is the story of that happening to two of our own models, and of what JITM Evergreen did about it.

The setup

Morrowatt forecasts tomorrow's hourly power prices for Germany and Spain. We trained both day-ahead models on data up to 1 August 2026, then replayed 1 August to 19 October through JITM one delivery day at a time: a forecast each morning, the real auction price posted back as the outcome, and a drift check every Monday. Everything ran in JITM for real; the only thing we replayed was the calendar. One caveat matters: each day used the final published wind, solar and demand forecasts, and some of those are not out yet at 11:00 the day before, when a live forecast is made. Live accuracy would be somewhat worse. Both sides of every comparison below saw exactly the same inputs, so the comparisons themselves are fair.

Caught on a Monday

On 31 August, Evergreen flagged Germany: forecast error had been running about 20% above the model's own first weeks in production, and not as a one-off. Spain followed on 28 September. In both cases the alarm was about a sustained change, not a single bad day, which is the difference between a signal and noise.

Why: the gas price had left the map

Gas plants set much of the price of power in Europe. In training, gas rarely went above €65 per MWh. By mid-September it was above €75, and by mid-October €87. Gas was the leading suspect, but being out of range is evidence, not proof, so we tested it directly. Holding everything else at the real October values and changing only the gas price, from €45 to €87.50, moved the original model's forecast by about €1. It had learned to read the market through recent prices, not through gas itself. As gas pushed prices up, those recent prices climbed to the top of what the model had seen in training, and a tree-based model cannot extrapolate past that. Its forecasts were capped: around €121 for Germany in a week when the real average was €184. Nothing broke. The world moved, and the model could not follow.

A fair fight

When drift fires, Evergreen first sets aside the newest real outcomes as a test window. It then trains a challenger on the original data plus all the earlier outcomes, so the test window stays unseen. Both models are scored on it, and the champion is judged on the forecasts it actually served, not a re-run. For this replay we fixed the promotion rule before we started: promote if, and only if, the challenger's error is lower on that window. Both challengers won.

With and without Evergreen

To measure what Evergreen was actually worth, we trained a second copy of each original model and never updated it. JITM's training is deterministic, so each copy made exactly the same forecasts as its original right up to the first promotion. Every difference after that is down to Evergreen. These numbers come from the days after each promotion, which played no part in training or choosing the challenger. From the promotion to 19 October, error was 20% lower in Germany, better on 47 of 50 days, and 16% lower in Spain, better on 18 of 22 days.

What Evergreen could not fix on its own

Evergreen kept both models in step with a moving market, but it could not make them see the future. On 19 October, with gas at €87.50, both still missed the day by close to €70 per MWh. Retraining helps a model catch up with prices it has now seen. It cannot teach a model to reason about prices it has never seen. That needs a better design, and the drift evidence showed us where to look.

What we learned about Evergreen

Running this replay made Evergreen better. We fixed a bug that hid challenger results, made challengers always learn from every outcome they are given, and changed drift checks to judge a model against its own early record in production rather than its training scores, which tend to flatter. We also found two things to improve next. A model's early record should cover at least a full week. And when a whole market gets more volatile, a rising error is not always a model getting worse, so drift should be judged on skill relative to a simple benchmark, not on raw error alone.

Why this is the point of JITM

An AI assistant can build you a model in an afternoon. What it does not do is close the loop between a forecast and what actually happened, day after day: keep every forecast, grade it against the outcome, notice when the world moves, train a challenger on the new reality, judge it fairly against what the champion really served, and leave evidence for every decision. Evergreen will not make a model right forever. It keeps it measured and improving after it ships, and it shows its working.

For the geeks

Day-ahead hourly prices for DE-LU and ES; features known by 11:00 the day before (load, wind and solar forecasts, recent prices, TTF gas), using final-vintage ENTSO-E forecasts. Champions trained on 31,391 hourly rows each (January 2023 to July 2026), grouped by delivery day. Drift fires when error over the latest outcomes is more than 15% above the model's first 100 live outcomes and a Page-Hinkley test confirms the shift is sustained. Germany's challenger learned from 540 outcomes and was scored on 180 held-out ones; Spain's from 1,044 and 348. The control is a second model trained on the same upload with the same settings and never sent an outcome; its daily error matched the original champion's to the cent on every day before the first promotion. After promotion: Germany 35.3 vs 44.2 €/MWh mean daily MAE over 50 days, Spain 28.4 vs 33.6 over 22 days. A same-hour-last-week forecast scored 60.7 and 50.1 over the same periods. Gas probe: the never-updated model on real inputs for 13 to 19 October, with only the gas price varied from €45 to €87.50, one request per level; mean forecast moved from 120.1 to 120.8 in Germany and 135.0 to 136.7 in Spain.