Honest estimates: designing AI products that can say "insufficient data"

Every prediction product faces the same temptation: always show a number. A number looks confident, fills the screen and makes a good demo. But a number produced without enough evidence is worse than no number at all, because people act on it.

Products that estimate values — land prices, energy bills, market conditions, time to learn a skill — earn trust by being clear about where each number comes from and by refusing to guess when they should.

Keep different numbers apart

Many domains have several numbers that sound alike but mean very different things. In land and property, for example:

• the asking price is what a seller hopes to receive,
• the transaction price is what comparable land actually sold for,
• the guideline value is an administrative figure set by a government authority.

Blending these into one “price” hides exactly the information a buyer needs. Show them separately, each with its source and date. An older guideline value and last month’s comparable sale are both useful — but only when the user can see which is which.

Prefer ranges to points

A single figure implies precision the data rarely supports. A range built from comparable evidence communicates both the central estimate and the uncertainty around it. When the range is wide, that width is itself useful information.

Set an evidence threshold — and enforce it

Decide in advance how much evidence an estimate needs: how many comparable records, how recent and how similar. Below that threshold the product should say so plainly:

Insufficient data for a reliable estimate.

This is not a failure state to hide. It is one of the most valuable things the product can say, because it stops users from anchoring on a fabricated number.

Backtest before you publish

Before a model’s output appears in a product, test it against history: hold out past records, estimate them with the data that was available at the time, and measure the error. Publish predictions only if the backtest justifies it, and keep the evaluation with the code so it can be re-run when data or logic changes. If the backtest does not justify a forecast, ship the tool without one. Calculators, comparisons and well-labelled evidence are still useful.

Label assumptions in calculators

Many “AI” features are really calculators with assumptions: loan repayments, energy costs, return scenarios. Let users choose the assumptions — growth rate, tariff, interest rate — and show them next to the result. When a value is an assumption rather than a fact, say so.

State the boundaries

Good estimation products are explicit about what they are not:

• not an official valuation or government record,
• not affiliated with the authorities whose data they reference,
• not financial advice.

These statements are not legal decoration. They help users decide how much weight to give the output.

Why it matters for AI products

Language models make it easy to produce fluent, confident text about anything. That makes honest uncertainty a differentiator. A product that shows its sources, labels its assumptions and declines when evidence is thin will be trusted when it does give an answer.