The Lessons Ledger

Every rule below was bought with a graded miss. They are binding on future reports: a forecast that violates one must say so explicitly and justify it. Newest first. This page only grows.

These rules are enforced in code, not memory (added 2026-08-18, same day as L4–L7): the graded record lives in a machine-readable ledger (prediction_grades.csv); a calibration engine (calibration.py) recomputes coverage and derives mandatory minimum range widths from measured error every weekly run; a survey-anchored estimator (production_estimate.py) fails the pipeline if the operative production estimate leaves the print-±-max-revision window; and the demand indicators we under-weighted (7-state drought index, weekly pasture condition, a cash-price benchmark ledger) are now collected as first-class data series. The test suite fails the build on violations. Current calibration floors, straight from the misses: price predictions ±$65, production ±30%, volume ±57% — until measured coverage earns narrower.

L7 — Calibrate interval width to measured coverage, not to confidence. (2026-08-18)

The miss: our pre-registered ranges contained the outcome 3 times out of 10. Ranges that miss 70% of the time are roughly half as wide as honesty requires. The rule: prediction ranges are set from measured error (the vintage backtest: ±$25–30 on winter months; the release-grading record), not from how sure the narrative feels. If the last cycle's coverage was below ~70%, the next cycle's ranges widen mechanically — no discretion.

L6 — When misses share a sign across cycles, it's bias, not bad luck. (2026-08-18)

The miss: three consecutive reports where every price surprise broke bullish (June $225 contract → July cash rally → August $275 delivered / $350–380 stable lots). Each time we treated it as a one-off. The rule: two consecutive same-signed misses on a series triggers a named bias review in the next report. Current standing bias: we under-weight demand. Demand-side indicators (pasture condition %, neighbor-state DSCI, feeder cattle margins, KS–CO spread) are now first-class forecast inputs, not color.

L5 — Drought indices are demand proxies for irrigated crops, not supply proxies. (2026-08-18)

The miss: we read "worst water year on record" as "the alfalfa crop fails." USDA's survey says CO alfalfa −4.8% while winter wheat went −67% and corn −24%. Drought kills dryland; wells and senior rights keep irrigated alfalfa alive — what drought actually does to the alfalfa price is destroy pasture and grass hay, which turns every ruminant into an alfalfa buyer. The rule: supply estimates for irrigated crops must be built from water deliveries by region weighted by production share, with explicit credit for groundwater. DSCI/USDM feed the demand model.

L4 — News coverage samples the disaster, not the state. (2026-08-18 — the big one)

The miss: our 1.5M-ton (−36%) production estimate was assembled from canal shutdowns, fallowing headlines, and worst-on-record stories in the regions where the story was dramatic. USDA's farmer survey printed 2,244k (−4.8%). For our number to be right, the quiet 57% of the state would have had to fail too — and nobody writes articles about fields that got water. The rule: statewide estimates anchor on the survey base rate and the maximum historical revision. The plausible-final window is [Aug print − 12%, Aug print] (2022 is the record revision). An estimate outside that window requires extraordinary, quantified evidence — not vibes, not clippings. Our own estimate is now graded against this: 1.5M (frozen) vs ~1.95M (rev1), settled Jan 12, 2027.

L3 — Load-bearing negative claims need two independent verification methods. (2026-08-08)

The miss: a broken shell tool returned empty search results and we published "USDA dropped hay from the August report." The files contained the tables all along. Retracted same day. The rule: any "X is absent / X stopped happening" claim ships only after two independent checks (different tool, different method).

L2 — Measure markets from cash prints, never from lagging state averages. (2026-08-08)

The miss: July's "Kansas caps SE CO at ~$155–170 landed" used a stale NASS state average ($123) while Kansas cash grinding hay traded $200–210. The cap was off by ~$60–65 landed. The rule: floors and ceilings are computed from AMS cash trades plus current freight (the market_parity.py table), refreshed every cycle. NASS monthly averages are trend confirmation only — they lag 4–8 weeks.

L1 — Don't forecast fuel. Mark it to market. (2026-07-11)

The miss: two consecutive directional misses on diesel (April: missed the fall; July: called it "falling" at $4.58, it went to $5.35). The rule: every report uses the current posted EIA weekly price. No diesel opinions.

Why this page exists: on 2026-08-18 the project's pre-registered predictions were graded against the August USDA releases and scored 3/10 on intervals, with the central production thesis missing by 28%. The full grading record is here; the reports that own each miss are in the reports archive. We keep the misses public because the alternative — quietly getting better without admitting what was wrong — is how forecasting products lie.