Checking our crowd forecast: how close it came to what happened

Publishing a forecast and never scoring it is the lazy option, so we scored ours against measured data from queue-times.com. As an ordering it holds up. Adjacent levels it cannot separate. The awkward half is in here too.

The short answer: a rank correlation of 0.599. That is against measured data from queue-times.com — 334 days from 2024-09-01 to 2025-07-31, and 335 days from 2025-10-01 to 2026-08-31, counting only days where both parks reported. Scored the same way, "just avoid weekends and public holidays" comes out at 0.114, and adding the right weekday order reaches 0.223. Use it to narrow a shortlist of dates. Do not use it to lock in a single one.

Measured levels drift between years, so each day is converted to its position within its own year (0 = the quietest day, 100 = the busiest) before comparing. What we calibrate against measurement is the order of the day types, and only that. The spacing between levels — how many steps apart they are — is a value we set by hand, so we do not publish ratios between levels. What we compare against is queue length, never attendance.

The ordering holds. Individual days do not

Ranking days by our level against their measured rank gives 0.599. Closer to 1 means the order matches. As an ordering it works. As a way to call a specific day, it does not.

On the window the weekday coefficients came from (2025-10-01 to 2026-08-31) it reads 0.557; on the other window (2024-09-01 to 2025-07-31), 0.642. The seasonal coefficients were set by looking at both windows, so the second one is not fully held out either.

Just avoiding weekends barely works at all?

For comparison we scored the simple reading the same way. "Weekends and holidays are busy, weekdays are quiet" comes out at 0.114. Adding the right weekday order reaches 0.223.

The weekday does carry signal, but it stops at 0.223. Adding the season and the shape of the holiday runs takes it to 0.599. The figure above shows where each forecast level actually landed.

The misses are not evenly spread

Here is the awkward part. The figure above contains two inversions. Days we called "Fairly easy" landed higher than days we called "Normal", and the same happens between "Fairly busy" and "Busy".

Adjacent levels are not distinguishable. Comparing levels that are far apart — "Easy" against "Busy" — works. Choosing a date on a one-level difference does not. Level 1 of the ten never actually occurs, so in practice there are nine.

Why we do not claim a 90% hit rate

Some crowd-calendar sites publish high hit rates. We are not disputing their numbers, but a hit rate means whatever its definition says it means. Count "within one level" as a hit, or round to five levels first, and the same forecast can multiply its score.

So we publish a rank correlation and the baselines to compare it against. We do not publish ratios between levels either, because only the order is calibrated — the spacing is ours. Use it to shortlist dates, not to commit to one.

How to use this forecast

Now that the misses are mapped, here is the usable part. Group the levels into three bands and their measured positions are 30 for the bottom band, 53 for the middle, 78 for the top. Between bands the ordering holds cleanly.

So: shortlist three to five candidate dates and take the ones in the bottom band. Which of those is the single best, this forecast cannot tell you. Settle that on room rates or your own schedule instead.

Last updated: 2026-09-01