Guides
Was Last Month's Forecast Any Good? Bias and Error Need Different Fixes
A miss tells you the forecast was wrong. Only the sign of the miss tells you whether the fix belongs in the forecast or in the buffer.
You can't score a forecast you didn't write down
A forecast review has one hard requirement, and it isn't a metric. It's a number that was fixed before the outcome existed.
Rebuild last month's forecast today and you get a model that has already seen last month's sales. It will look accurate and teach you nothing — what you're scoring is no longer a prediction. The forecast worth grading is the one that sized the order.
So freeze three things at decision time: the number as it stood, the grain, and the window. Grain means variant — a product-level total can be right while every variant under it is wrong.
Match the lag to the decision, too. If you commit eight weeks before the stock sells, the eight-week-out forecast is the one that cost you money — a one-week-out refresh measures a forecast nobody bought against.
Two columns, not one: keep the sign in one, drop it in the other
One definition: miss = actual minus forecast. Positive means you under-forecast — actual came in above the figure you used; negative means you over-forecast. Reviews go wrong by collapsing that column into a single average.
Keep two columns instead. The first keeps the sign and averages it: that's bias, the net direction of your misses. The second drops the sign and averages the magnitude: that's error, how far off you typically are regardless of direction. They are independent: a variant can have a small average absolute miss and still lean the same way every month, or sit dead center on average while swinging wildly period to period.
bias = average( actual − forecast )
error (per variant) = average | actual − forecast |
catalog error (WAPE) = Σ | actual − forecast | ÷ Σ actual
Error is per variant, measured in units; WAPE is the volume-weighted roll-up across the whole catalog — total absolute miss over total actual units.
One trap when rolling up across a mixed catalog: percentage error explodes on slow movers and is undefined when actual sales are zero. A variant that sells one when you forecast two is off by 100%, drowning out a fast mover that missed by 3%. The catalog roll-up is the same idea weighted by volume — total absolute miss over total actual units — so the SKUs that move your money dominate.
Suppose two variants are reviewed over six periods. Variant A's misses run +12, +9, +14, +8, +11, +10 — every one positive. Its average signed miss is about +10.7 units and its average absolute miss is the same 10.7 units, because the signs never disagree. Variant B's misses run +22, −18, +15, −24, +19, −14. Its average signed miss is exactly 0, and its average absolute miss is about 18.7 units — nearly twice A's.
By headline accuracy B looks like the worse forecast, and it is the noisier one. But B has no lean: the misses cancel, and the fix is a bigger buffer. A is the one with a forecast defect — it sits under actual demand every single period, and no amount of safety stock corrects that. (Illustrative numbers.)
Read the sign column before you read the size column
Under an unbiased forecast, each period's miss is close to a coin flip on sign. That makes the sign column the fastest diagnostic you have.
Count the last six periods and look at the signs. Three or four each way is what a centered forecast looks like. Five on one side is a lean worth watching; six is hard to explain by chance — something is pulling the forecast one way. The formal version is the tracking signal, cumulative signed error divided by average absolute deviation, but the run of signs gets you there by eye.
| What you're looking at | Bias (one-sided miss) | Error (noisy miss) |
|---|---|---|
| Sign column | Same side, month after month | Flips roughly half the time |
| Average signed miss | Clearly above or below zero | Near zero |
| Average absolute miss | Can be small and still be a problem | The number that matters |
| What it costs | Steady drift — dead stock or chronic shortfall | Swings around the right level |
| What to change | An input to the forecast | The size of the buffer |
| What not to do | Don't buffer it — you'll fund the lean forever | Don't keep retuning the model — the center is right |
Bias means change an input. Error means change the buffer.
A one-sided miss — all positives, or all negatives — is a defect in an input, with a short list of usual causes. Biased low, forecasting under actual demand: history censored by stockouts, where days you had nothing to sell were recorded as days nobody wanted it; a trend not yet caught; a horizon shorter than your lead time. Biased high: a promotional week sitting in the baseline as if it were normal, returns never netted out, last season's peak carried into a period that has none.
An unbiased but noisy miss is a different animal: nothing in the forecast needs fixing — the center is right, and demand genuinely varies that much. The response is on the inventory side: size the buffer against the spread of the misses rather than the average, and shorten the review cycle so you react sooner instead of predicting better.
The anti-pattern is treating the first case with the second's remedy. Stacking safety stock on a forecast that leans low buries the lean in inventory you then fund every cycle. Bias is systematic error with a cause; buffers exist for variation that has none. Correct the input, then buffer what's left.
Making it a standing monthly review
None of this needs a new discipline — just one comparison on a schedule.
Assisty forecasts what each product — and each variant — will sell over the horizon you choose and flags what runs out. You see predicted sales per product or per variant over the period you pick, because buying decisions happen at variant level. That's the forecast side of the comparison.
The actuals side is a report. Ask for sales, inventory, forecasting, replenishment, lost-revenue or stock-availability reports — by product, variant, category, vendor, location, customer, or over time, from the last 7 days to year-over-year. Atlas queries your synced store data and answers with a live table.
Record the forecast figure yourself when you commit the order — that saved number is what the review scores.
Once the review names a cause, the fixes sit in the same place. Censored history is what the stockout exclusion is for — days you were out of stock can be excluded, so the forecast reflects real demand, not the days you couldn't sell. A suspected trend lean can be priced first: ask "what if sales trend up 20%?" and the forecast and the reorder needs update in front of you. When you can't name the cause, ask why demand is changing — Atlas explains the drivers instead of shrugging a number at you.
| Review step | What Assisty provides |
|---|---|
| Forecast side | Predicted sales per product or per variant over the period you pick — buying decisions happen at variant level, so do the forecasts (Forecasting) |
| Actuals side | Sales, inventory, forecasting, replenishment, lost-revenue or stock-availability reports — by product, variant, category, vendor, location, customer, or over time, from the last 7 days to year-over-year (AI Report Builder) |
| Scoring | Yours to record — write down the forecast figure when you commit the order; that saved number is what the review scores |
| Fix a low bias | Days you were out of stock can be excluded, so the forecast reflects real demand — not the days you couldn't sell (Forecasting) |
| Fix a trend lean | "What if sales trend up 20%?" — scenarios apply instantly; the forecast and the reorder needs update in front of you (Forecasting) |
| Diagnose the cause | Ask why demand is changing and Atlas explains the drivers instead of shrugging a number at you (Forecasting) |
| Act on what it finds | At-risk items carry straight into the Smart Buying Table, so seeing a risk and fixing it are the same motion (Smart Buying Table) |
| Make it standing | Save it to My Reports and schedule email delivery — daily, weekly, or monthly (AI Report Builder) |
Put the actuals report on a monthly schedule and add the column you saved as each one arrives. Six months of that and the sign column answers a question no single month can.