modelspread

Methods and data

How ModelSpread measures forecast error

Methodology version 2026.09
Published accuracy archive: 2024-08-02 through 2026-08-05
Latest derived dataset: 2026-08-12

ModelSpread archives National Blend of Models forecasts and compares them with observations after each forecast window has ended. The published local accuracy pages use raw NBM median temperature guidance initialized at 12Z.

National Blend of Models

The NBM combines guidance from multiple numerical weather models with statistical processing to produce nationally consistent forecast guidance. ModelSpread verifies the archived raw NBM median and percentile fields. It does not score the final official National Weather Service forecast or commercial weather applications.

Typical miss

Typical miss is mean absolute error: the average distance between the forecast and observation, regardless of whether the forecast was too warm or too cool. High and low temperatures are evaluated separately and grouped by forecast day.

The two elements use different 18-hour windows against the 12Z initialization. Forecast-day 1 ends 42 hours after initialization for the high and 30 hours after initialization for the low. High and low results therefore describe the same numbered forecast day but are not the same exact lead; this 12-hour difference applies at every displayed forecast day.

Usually warmer or cooler

Signed bias preserves the direction of each error. A positive value means forecasts averaged warmer than observed; a negative value means they averaged cooler. Small positive and negative errors can cancel, so bias and typical miss answer different questions.

Monthly results

Monthly performance is grouped by the calendar month in which the daily forecast window ended. The current public archive covers two years and includes multiple NBM model eras. It is retrospective evidence, not a promise of future accuracy.

Calibrated likely ranges

Temperature ranges apply regional, seasonal, lead-time, and raw-width adjustments estimated from the previous year of forecast errors, so each season is corrected with errors from the same season a year earlier. Precipitation estimates use a separate archive of 24-hour precipitation forecasts and observations. Current calibration products are experimental estimates awaiting independent prospective evaluation.

How the calibrated temperature range was tested

The temperature product targets 80% coverage: over many comparable forecasts, approximately four out of five observed highs or lows should fall inside the displayed likely range. Coverage alone is insufficient because an extremely wide range could contain almost every outcome while providing little useful information. Evaluation therefore also records average width, bias, median-forecast error, and interval score. Interval score rewards narrower ranges and adds a penalty when an observation falls outside.

The retrospective audit uses rolling monthly tests. Each month from August 2025 through June 2026 was evaluated using a calibration fitted to the preceding year of verified forecasts, the same trailing-year fit used for the live product. The audit contains 330 evaluation dates and 9,962,181 station forecasts. Each date receives equal weight; individual stations provide geographic coverage, but nearby stations are not independent measurements.

The raw NBM P10–P90 range covered 73.49% of outcomes and averaged 11.89°F wide. Regional calibration covered 80.42% and averaged 13.77°F wide. A simpler national calibration covered 80.43% with a 14.12°F average width. The regional method was closer to the 80% target than the raw range in 10 of 11 monthly tests, had a better interval score in 10, and had lower median-forecast error in 8. Its aggregate interval score, 19.32, was better than both the raw range (19.81) and the national method (19.62), and its average bias was +0.01°F compared with +0.24°F for raw guidance.

July 1–27, 2026 was examined during development, so it is not an independent validation period. In those 27 dates, calibrated coverage was 83.54%, compared with 79.54% raw. Bias improved from +0.85°F to −0.09°F, interval score improved from 14.02 to 13.72, and median-forecast error fell from 2.98°F to 2.94°F, while average width increased from 9.37°F to 10.31°F.

The live calibration is refit on the most recent year of verified forecasts, currently August 9, 2025 through August 8, 2026, and is used for no more than one year after that period ends. The year spans two NBM versions: version 5 forecasts cover only May through early August 2026, so cool-season adjustments currently come from the preceding version.

Validation status: retrospective evaluation is complete. Prospective evaluation of the current method begins with the September 30, 2026 forecast run and requires at least 60 eligible forecast dates after the method was frozen. No eligible prospective result is yet present in the published verification archive. Until those results accumulate, the displayed range remains an experimental estimate of uncertainty.

How the calibrated precipitation outlook was tested

Precipitation calibration was tested in 22 rolling monthly windows from December 1, 2024 through September 13, 2026, covering 641 forecast dates and 7,076,022 location cases. Each test month is fitted from the preceding 120 days, using only forecasts whose 24-hour verifying window ended before the test month began. Over the full record, calibration improved Brier score from 0.06761 to 0.06645. It improved Brier score in 21 of 22 months and interval score in 19 of 22; May 2025 was worse on both measures, and June and July 2025 were worse on interval score.

The product’s guardrails and typical-error estimates use the most recent 12 windows, October 1, 2025 through September 13, 2026: 344 forecast dates and 3,820,069 location cases. In those months the raw probability of more than 0.25 inches averaged 13.55%, compared with a 10.96% verifying rate. Calibration reduced the mean probability to 10.99% and improved Brier score from 0.06476 to 0.06338, a 2.1% reduction. Brier and interval scores improved in each of the 12 months. Probabilities below 40% moved close to verifying frequencies, while probabilities above 50% remained somewhat high.

The raw 24-hour amount range averaged 0.320 inches wide with an interval score of 0.470 inches. The calibrated range averaged 0.231 inches wide and had an interval score of 0.447 inches, 4.9% lower. The average error of the median amount forecast increased slightly from 0.0838 to 0.0857 inches. On cases whose verifying estimate exceeded 0.25 inches, that error decreased from 0.4878 to 0.4802 inches.

The amount range is not an 80% interval. Although it is fitted toward 80% coverage, it contained the verifying amount in 91.6% of cases, compared with 93.4% for the raw P10–P90 range. Misses were one-sided: 0.6% of outcomes fell below the calibrated range and 7.7% above it, largely because dry outcomes fall inside ranges that begin at zero. The forecast page therefore calls it a typical range, one that held the observed amount on about 9 in 10 past days.

The live calibration is a separate fit to May 5 through September 13, 2026, the NBM version 5 period, and is used for forecasts through September 13, 2027. That fit spans only late spring to early autumn, and August and September 2026 are also inside the test record, so the rolling results describe the method rather than independently testing the live fit. Its cool-season performance on version 5 forecasts has not yet been measured. When calibrated guidance is unavailable, the forecast page shows raw NBM percentiles labeled as not calibrated. Precipitation for a ZIP code is sampled at the NBM grid point of its reporting station, which can be up to 20 miles from the ZIP center; showers and thunderstorms can differ over that distance. These are retrospective rolling results; they are not a completed prospective test of forecasts issued after the method was frozen. The research metrics artifact records the source database hashes and report analysis tables.

Location representation

ZIP codes use an eligible reporting station within 20 miles, generally the nearest station. San Francisco and Denver use closely located named-city airport stations so their pages represent a distinct local record rather than repeating a neighboring city’s station. Every local accuracy page identifies the station and distance used so users can judge how well it represents their location.

Sources and provenance

Limits

Corrections

Methodology or data concerns can be sent through the contact page. Include the city, station, affected metric, and page URL. Material corrections should update the methodology version and derived-data date shown above.