How much should you trust a prediction?

A point prediction without uncertainty is like a deploy without monitoring. Prediction intervals tell you how much to trust a number, and whether the uncertainty is tight enough to act on.

“The model predicts 1,000 visitors tomorrow.”

It sounds like an answer. It’s really half of one. Is that almost certainly between 990 and 1,010, or somewhere between 600 and 1,400, your guess is as good as mine? Those are wildly different situations, and they call for different decisions — provision tightly and move on, or build in a buffer, a fallback, and a plan for being wrong. The single number can’t tell you which world you’re standing in.

A point prediction with no uncertainty attached is a deploy with no monitoring. You’ve done the work and shipped the thing, but you’ve left yourself no way to know whether it’s going well — and, worse, no way to know how worried to be.

Half the answer is the range

A prediction interval is the other half: it wraps the best guess in a range that owns up to how much the model doesn’t know. “Between 938 and 1,062 visitors, with 95% probability” is a different kind of statement from “1,000.” You get the centre — the best guess; the width — how much doubt is left; and the coverage — how often the truth is meant to land inside. The centre is the part everyone quotes, mine included for longer than I’d care to total up. The other two are where the useful information hides.

import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(42)

days = np.arange(1, 31)
lam = 1000
visitors = rng.poisson(lam=lam, size=30)

# For a Poisson with large λ, the normal approximation gives a prediction interval
lower = lam - 1.96 * np.sqrt(lam)
upper = lam + 1.96 * np.sqrt(lam)

fig, ax = plt.subplots(figsize=(10, 5))
fig.patch.set_alpha(0)
ax.patch.set_alpha(0)

ax.scatter(days, visitors, color='#0072B2', alpha=0.6, s=30, zorder=3)
ax.axhline(lam, color='#E69F00', linewidth=2, linestyle='--', label=f'Point prediction ({lam})')
ax.fill_between(days, lower, upper, color='#E69F00', alpha=0.20,
                label=f'95% prediction interval ({lower:.0f}–{upper:.0f})')

ax.set_xlabel('Day')
ax.set_ylabel('Visitors')
ax.set_title('A prediction is incomplete without its interval')
ax.set_xlim(0.5, 30.5)
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.yaxis.grid(True, linestyle=':', alpha=0.4, color='grey')
ax.set_axisbelow(True)
ax.legend(loc='upper right', framealpha=0.0)
plt.tight_layout()
plt.show()

Figure 1: A point prediction (amber dashed line) says where the centre is. The prediction interval (shaded band) says how much to trust it. Wider intervals mean more uncertainty, and more honest communication about the limits of the model.

The shaded band is the interval. Most days land inside it; a few don’t — which is exactly what a 95% interval is promising, not a sign it’s broken. That last part took some getting used to. I spent a while reading every point outside the band as evidence the model needed more work, when a band nothing ever escaped would have been the thing worth worrying about. It isn’t a guarantee. It’s a calibrated admission of doubt, which is a more honest thing for a model to hand you than a lone confident number.

Same guess, very different worlds

The width carries something the centre can’t: whether you actually know enough to act.

Picture two models forecasting next month’s server bill. Model A says £8,200–£8,800. Model B says £5,000–£12,000. They might share the identical point prediction — £8,500 — and still be telling you completely different things. A is saying I’m fairly sure. B is saying I genuinely don’t know. How much to reserve, whether to pre-buy capacity, whether to go and collect more data first — all of that rides on the width, and none of it on the centre they agree about.

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))
fig.patch.set_alpha(0)

for ax, lo, hi, title in [(ax1, 8200, 8800, 'Model A: narrow interval'),
                           (ax2, 5000, 12000, 'Model B: wide interval')]:
    ax.patch.set_alpha(0)
    centre = 8500
    ax.barh(0, hi - lo, left=lo, height=0.4, color='#E69F00', alpha=0.3)
    ax.plot(centre, 0, 'o', color='#E69F00', markersize=10, zorder=3)
    ax.set_title(title)
    ax.set_xlabel('Predicted monthly cost (£)')
    ax.set_yticks([])
    ax.set_xlim(3000, 14000)
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)
    ax.spines['left'].set_visible(False)
    ax.axvline(centre, color='#E69F00', linewidth=1.5, linestyle='--', alpha=0.5)
    ax.text(lo, 0.3, f'£{lo:,}', fontsize=8, ha='center', color='#D55E00')
    ax.text(hi, 0.3, f'£{hi:,}', fontsize=8, ha='center', color='#D55E00')

plt.tight_layout()
plt.show()

Figure 2: Two models with the same point prediction but very different intervals. Model A (left) gives you enough precision to act. Model B (right) is telling you it doesn’t know enough yet; that honesty is more useful than a false sense of certainty.

A wide interval isn’t the model failing. It’s the model being honest about what it can’t see yet — and that honesty is the part you can act on. It tells you to go and find a better feature, gather more data, or design the decision so it survives the whole range rather than betting the result on a single estimate.

It shrinks, then it stops

This is the other side of the line between reducible and irreducible error. Give the model something genuinely useful — a better feature, more data, a structure that actually fits — and the interval tightens: more of the variation is explained, so less is left to doubt.

But it never closes to a point. There’s a floor, set by the randomness that lives in the process itself, and no amount of modelling gets you under it. Knowing roughly where that floor sits is one of the quietly most useful things a model can tell you — it’s the line between effort that pays and effort that’s chasing a precision the world doesn’t contain.

models = [
    ('Baseline model', 1000, 120),
    ('+ day-of-week', 1000, 80),
    ('+ seasonal + marketing', 1000, 55),
]

fig, ax = plt.subplots(figsize=(10, 4))
fig.patch.set_alpha(0)
ax.patch.set_alpha(0)

irreducible = 30  # floor

for i, (label, centre, half_width) in enumerate(models):
    ax.barh(i, 2 * half_width, left=centre - half_width, height=0.5,
            color='#0072B2', alpha=0.3 + 0.2 * i)
    ax.plot(centre, i, 'o', color='#E69F00', markersize=8, zorder=3)
    ax.text(centre - half_width - 5, i, f'{centre - half_width}', fontsize=8,
            ha='right', va='center', color='#D55E00')
    ax.text(centre + half_width + 5, i, f'{centre + half_width}', fontsize=8,
            ha='left', va='center', color='#D55E00')

# Irreducible floor
ax.barh(len(models) + 0.3, 2 * irreducible, left=1000 - irreducible, height=0.3,
        color='#D55E00', alpha=0.3)
ax.text(1000, len(models) + 0.3, 'Irreducible floor', fontsize=8,
        ha='center', va='center', color='#D55E00')

ax.set_yticks(range(len(models)))
ax.set_yticklabels([m[0] for m in models])
ax.set_xlabel('Predicted visitors')
ax.set_title('Better models narrow the interval — but a floor remains')
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.xaxis.grid(True, linestyle=':', alpha=0.4, color='grey')
ax.set_axisbelow(True)
plt.tight_layout()
plt.show()

Figure 3: As models improve, prediction intervals narrow — but they never reach zero. The remaining width is irreducible uncertainty. Knowing where this floor sits tells you when to stop modelling and start making decisions.

Each new feature tends to buy a little less than the last, until you reach the point where what’s left isn’t missing information but genuine noise. That’s the floor. Recognising it is what stops you grinding away at a fourth decimal place that was never on offer.

The width is the decision

All of which lands on the thing you were going to do anyway: decide something.

If tomorrow’s traffic interval is 950–1,050, you provision for 1,050 and get on with your day. If it’s 700–1,300, the same centre demands a different plan entirely — autoscaling, a buffer, a fallback. Same point prediction, opposite decisions, and the only thing that changed was the width. Which is why a forecast handed to whoever signs off the budget should never arrive as a lone number: strip the interval off and you’ve quietly made the decision for them — and hidden that you did.

If that logic feels familiar, it should — you already refuse the single number somewhere else. Nobody reports a service’s latency as “40ms”. They report p50, p95, p99, because everyone learned at some point that the mean is the least interesting fact about a response-time distribution and the tail is where the pagers live. A prediction interval is that same refusal pointed at a model’s output: here’s the middle, here’s the spread, here’s what to plan for at the edges. The habit was already there. It just had a different name on the dashboard.

Once that clicks, a bare point prediction starts to look like a dashboard showing only the average. Technically true. Practically not enough to act on.

What I haven’t worked out is what to do when someone asks for the number and visibly doesn’t want the range. Handing over an interval where a single figure was expected reads as evasion, or as not knowing the answer, and I have folded and quoted the centre more than once rather than have that conversation. The spread was always the useful part. Getting it listened to is a separate problem, and not a statistical one.

---

This article is part of a series drawn from Thinking in Uncertainty, a book that teaches data science to experienced software engineers.