AI in real estate, signal vs noise
The Missing Failure State
6 min read
Feed any valuation tool a thin comparable set. Three sales instead of six, none matching the subject on bedrooms, sizes 40% apart. A figure comes back, to the dollar, with a confidence label beside it.
Try it with an automated estimate, a CMA product, or a spreadsheet an agent built themselves. Same result every time, for the same reason. None of them has a way to decline.
A confidence label is not a refusal
The objection to this is that the tools do report confidence, and they do. A band, a rating, sometimes a set of factors underneath it.
But read what the label describes. It describes the output. Here is a number and here is how much to trust it, which still leaves a number on the screen and a person holding it. Nothing in that structure produces the other answer, the one that says these sales will not carry a recommendation and you should go and find another.
The distinction matters because of what happens next. A wide band reads as caution and gets ignored, because the figure at the centre of it is what the seller asked for. A LOW rating still ships. There is no state in which the tool declines to be useful, so the agent makes that call alone, on a document that already looks finished.
Where the missing state would have caught something
The gap is visible in the arithmetic rather than in anyone's judgment.
Where the nearest sale is 25% off the subject on size, the size adjustment is carrying the number rather than the sales. Where not one comparable matches the bedroom count, every value in the grid is bedroom-adjusted and the adjustment is doing work the evidence was supposed to do. Where two pricing methods land 15% apart, one is describing a different house. And where adjusted values still spread 30% after adjustment, the adjustment has failed at its only job.
Each of those is computable. None of them requires a model, a judgment, or a market read. They are relationships between numbers the tool already holds at the moment it produces the report.
So the information exists. What is missing is any obligation to act on it.
| Signal already computed | What it means | Typical response |
|---|---|---|
| Nearest comparable 25% off on size | The adjustment is carrying the number | Report issued |
| No comparable matches bedroom count | Every grid value is bedroom-adjusted | Report issued |
| Two methods 15% apart | One describes a different house | Report issued |
| Adjusted values 30% apart after adjustment | The sales are not comparable | Report issued |
Why nobody built the refusal
Partly because it is unpleasant to sell. A product that sometimes declines to produce its main output is a harder demonstration than one that always works, and the demonstration is where these products are bought.
Partly because the purchaser is rarely the user. Where an MLS or brokerage buys for its members, the evaluation runs on adoption and consistency, and a tool that occasionally tells an agent to find another sale scores badly on both. And the failure is invisible either way: if a listing sits for ninety days, the explanations available are the market, the condition, the photographs, the season. Nobody goes back to the comparable set.
What the state would have to do to be worth anything
Two things, and the second is where it gets difficult.
It has to fire on arithmetic rather than a model's opinion, or it becomes another confidence label with a sterner tone. The thresholds have to be stated, so an agent can disagree with them and know what they are disagreeing with, which is the same standard that should apply to judging any valuation tool.
And it has to name the specific fault rather than emit a warning. A tool that says confidence is low has told the agent something they can already see. A tool that says no comparable is within 25% of this house on size, and the closest is 1,740 square feet against 2,340, has told them what to go and fix. The difference between those two sentences is the difference between a caveat and a piece of work.
This is buildable rather than theoretical. It is in CMAflow, where two signals firing at high weight change what the pre-appointment screen says: it stops offering answers to expected objections, states the report should not be sent, and lists which relationships failed and by how much. That is one implementation, it is early, and its thresholds are arguable rather than settled. The point is not that it is right. The point is that a refusal state is an ordinary thing to build and almost nobody has bothered.
Can a valuation tool tell you when its own estimate is unreliable?
Most cannot, in any actionable sense. They report a confidence band or rating, which describes the output rather than deciding whether the output should exist, so a figure appears regardless. The relationships that would justify declining, comparable size gaps, bedroom mismatch, disagreement between methods, are computable at the moment of production. Almost no product acts on them.
What does a low confidence rating on a CMA mean?
Usually that the comparable values disagree with each other after adjustment. It is a statement about dispersion, not about whether the analysis should be presented. A low rating on a report that still ships leaves the decision with the agent, on a document that already looks finished, which is where most of these ratings quietly go to die.
How do you know a comparable set is too thin to price from?
Check four relationships. Whether any sale is within roughly 25% of the subject on size. Whether any matches the bedroom count. Whether a second pricing method lands near the first. And whether the adjusted values sit close together once adjustments are applied. If adjustment has not narrowed the spread, it has not made the sales comparable.
The report that should not have been sent
Context Blindness™ is usually described as a tool missing what an agent knows. There is a version underneath it that is harder to see, where the tool holds every relevant fact, computes them correctly, and has no mechanism for concluding that they do not add up to an answer.
The number arrives anyway. It is precise, it is labelled, and it is presented to somebody making a decision worth several hundred thousand dollars to them. Somewhere in that process a system knew the evidence was thin, and the only place that knowledge had to go was a range on a page nobody reads as a warning.
This article is general information and analysis, not financial, lending, or appraisal advice. Verify any home value with a licensed professional before acting.
The Independent Agent
Substack | Spotify | CMAflow FAQ | YouTube | Free CMA | Home valuation | Insights | Blog