How to measure forecast accuracy when demand is intermittent

Why the usual metric fails

Mean absolute percentage error divides by actual demand. For an item that sold zero units last week, that division is undefined, and most tools quietly skip the row or clamp it. On a catalogue where half the SKU-locations sell fewer than one unit a week, MAPE ends up measuring the fast movers only, and reporting an accuracy number that has nothing to do with the items that fill the warehouse.

The second failure is subtler. Percentage error treats a one-unit miss on a two-unit item as a fifty percent error, and a one-unit miss on a hundred-unit item as one percent. Planners learn to ignore the metric on slow items, because everything looks terrible. Once the metric is ignored, it stops steering anything.

What to use instead

Scaled error, not percentage error

Mean absolute scaled error (MASE) divides the forecast error by the error of a naive forecast on the same history. A MASE below one means you beat naive; above one means the naive forecast would have done better. It is defined for zero-demand periods, it is comparable across items with wildly different volumes, and it answers the question a planner actually has: is this forecast better than what I would have guessed?

Bias, tracked separately

Accuracy metrics hide direction. A forecast that is consistently ten percent high and one that is consistently ten percent low have the same MAPE and very different consequences: one builds stock, the other builds stock-outs. Report bias as a scaled signed error alongside the accuracy figure. On intermittent items, bias is usually the number that explains the inventory position.

Pinball loss for probabilistic forecasts

If the forecast is a range rather than a single number, which it should be for intermittent items, score the range. Pinball loss evaluates each quantile: it penalises a P90 that is exceeded more than ten percent of the time, and a P90 that is never approached. It is the only common metric that rewards a forecast for being honest about uncertainty, which is the whole point of forecasting slow items.

Measure at the level decisions are made

A forecast that is accurate at the national weekly level and wrong at every store-day is useless for replenishment and fine for financial planning. Decide which decision the forecast feeds, then measure at that grain and horizon. For replenishment that is usually SKU-location, at the lead time horizon, on the periods where an order could have been placed.

What changes when you switch

Teams that move from MAPE to scaled error and bias on intermittent items typically find two things. First, the items that looked worst were not: a two-unit item with a one-unit miss is doing fine. Second, a handful of items with modest percentage error carry most of the excess stock, because their bias was invisible. That reordering of attention is worth more than any model change.

Practical setup

• Classify items by demand pattern (smooth, intermittent, lumpy, erratic) before choosing a metric. One metric per class.
• Report MASE and scaled bias per class, per horizon, weekly. Put it on a board the planners open.
• For probabilistic forecasts, report coverage of the P10 to P90 band alongside pinball loss. A band that covers eighty percent of outcomes is calibrated.
• Keep MAPE for the fast-moving top of the catalogue if finance is used to it. Do not let it be the only number.