The deletion leaves no trace

A comparable selection rule that is correct in one property type will quietly destroy a valuation in another, and the report it produces will pass every internal check. That is the failure worth naming, because it does not look like a failure. It looks like a clean document with a different number in it.

The case is my own. A rule shipped on a Tuesday and by Wednesday it had taken $57,000 off a house in Columbus.

Why a comparable selection rule breaks when it moves between property types

Pricing a two bedroom condominium against one bedroom sales in the same tower is close to meaningless. In a high rise the bedroom count is not a feature of the home so much as the name of the product. A one bedroom and a two bedroom on the same floor are marketed to different buyers, financed differently, and trade in separate pools. Match the bedroom count first and the grid improves immediately. The evidence for that came from a Seattle tower where the previous behaviour had produced a figure that could not be defended in front of a seller.

Then the same rule ran on a three bedroom house on an ordinary street, where the best evidence available was the subject property's own neighbours. Same floor plan, one block away, sold recently, with one extra bedroom. The rule read the bedroom count, found a mismatch, and reached instead for three bedroom sales scattered across the suburb that had closed in July. Older, further away, different streets, and every one of them a weaker comparison than the house two doors down.

Candidate set	What it offered	What the rule did
Neighbours, same floor plan, one block away, sold recently, one extra bedroom	Identical construction, identical street, current pricing	Removed, bedroom count did not match
Three bedroom sales across the suburb, closed in July	Matching bedroom count and little else	Used, the rule was satisfied

What the Selling Guide requires when comparables come from outside the neighborhood

The Fannie Mae Selling Guide is explicit on this point. Section B4-1.3-08, dated 4 June 2025, states that sale activity from within the neighborhood is the best indicator of value, and that the appraiser is responsible for determining which comparables are the best and most appropriate for the assignment. Where a comparable comes from a competing neighborhood, the appraiser must indicate that and address the differences that exist, and must provide an explanation as to why those specific sales were used.

Read that against what the rule did. It replaced sale activity from within the neighborhood with sale activity from outside it, on a single attribute, and it left no explanation because none was required of it. The standard assigns the selection to a person who has to justify it. Software that performs the same selection silently has taken the responsibility without taking the obligation that came with it.

Why a filter is the wrong shape for comparable selection

The defect was not the rule. It was the shape of the rule. A filter answers one question, in or out. A ranking says prefer this kind of evidence, then that kind, and when something sits lower in the order, use it anyway with the difference adjusted and the reason printed beside it. The same floor plan one block away is close to the strongest evidence a residential grid can hold, and no bedroom rule should have the authority to remove it. It should only be able to move it down the list.

Operation	Shown in the finished report	Reader can override it	What survives for review
Adjustment applied to a comparable	Yes, the dollar figure with its printed reason	Yes, the row is on the page	The figure, the reason, and somewhere to disagree
Comparable removed by a selection rule	No	No, the row was never rendered	Nothing

That distinction matters more than it sounds, because of what each one leaves behind. An adjustment a reader disagrees with is visible: the figure is on the page, the reason is next to it, and the disagreement has somewhere to happen. A sale that was never shown to the reader leaves nothing. There is no line to argue with, no row to override, and no way to tell from the finished document that the decision was ever made. Deletion is the one operation in a valuation that erases the evidence of itself, which is why it is the operation a formula should be least trusted with. The point was made from the other direction in comp selection is the whole game, and the consumer version of the same question is covered in why your agent left some sales out.

How a rule that is true in one market becomes an error in another

There is a second failure underneath the first, and it is the more common one. The Seattle fix was real evidence, correctly read. It was also evidence about towers. Taking a truth learned in one property type and shipping it as a rule that applies everywhere is the ordinary way software degrades without anybody noticing. Nobody writes a bad rule deliberately. They write a rule that holds in the place it was learned, then apply it in a place where the same words describe a different object. A bedroom in a high rise and a bedroom in a 1960s ranch are not the same thing, and the field that records them carries no marker saying which one is in hand. The field itself is where Context Blindness™ starts. A record that stores a bedroom count stores a number, not the thing the number means, and a rule reading that number has no way to ask which kind of market produced it.

Why a wrong valuation can pass every internal check

The report that carried the lower number was internally clean. Every adjustment traced to a real sale. The written analysis matched the computed figure. The confidence level agreed with the dispersion of the adjusted values, because those values did cluster, which is exactly what a tidy set of the wrong comparables produces. CMAflow runs a verification pass before every save that checks the prose against the computed figures, and it passed, correctly, because consistency was all it was ever able to measure.

That is the limit of self verification and it is worth stating plainly rather than discovering later. A system checking its own output can confirm that the parts agree with each other. It cannot confirm that the inputs were the right inputs. The failure state that matters is not the one that throws an error, as argued in the missing failure state, but the one that returns a well formed answer to a question nobody checked.

What catches an error the system cannot see in itself

The answer is memory, and it has to be specific rather than general. Not a suite of invented test cases written to demonstrate that the system works, which is a different exercise serving a different purpose. A corpus of real past failures: the Seattle tower, the Columbus street, the estate sale that should have anchored nobody, every property where the output was defensible on its face and wrong underneath. Every change is replayed against the whole set before it ships. If a change makes any of them worse, it does not go out.

The economics of that are worth being honest about. It is slow, it grows monotonically, and it produces nothing a buyer can see in a demonstration. The survey of best practices for CMAs and listing presentations that Giant Steps Advisors ran this year with Lone Wolf Technologies, across 2,165 US professionals, found 83.2% at least somewhat likely to use tools that assist with the CMA, and among the named uses was automating the adjustment calculation. That is the slot this rule was sitting in. Anything placed there inherits the property of being believed, because the output arrives looking like arithmetic rather than like a judgement somebody made on a Tuesday afternoon and shipped on the Wednesday.

Why would a valuation model exclude the closest comparable sale?

Because most selection rules are written as filters rather than as rankings. A filter asks one question, in or out, and a rule that requires a matching bedroom count will remove a sale that differs by one bedroom even when that sale is the same floor plan one block away. The rule is doing what it was told. The problem is that it was given the authority to delete evidence rather than the authority to rank it.

What does the Fannie Mae Selling Guide say about comparables from the subject neighborhood?

Section B4-1.3-08, dated 4 June 2025, states that sale activity from within the neighborhood is the best indicator of value, and that the appraiser is responsible for determining which comparables are the best and most appropriate for the assignment. Where comparables come from a competing neighborhood, the appraiser must indicate that and address the differences. The judgement sits with a person, and the reasoning is required to be visible.

How do you catch a pricing error when the report looks correct?

Not by checking the report against itself. Internal verification confirms that the prose, the figures and the confidence level agree with each other, which a tidy set of wrong comparables will satisfy. The practical answer is a corpus of past failures that every change is replayed against, so a rule that would reintroduce a known error cannot ship, whatever the current report looks like.

The Columbus house is in the corpus permanently now, which means no future bedroom rule can pass without stepping over it. The next one is already in there too, sitting quietly in a property nobody has looked at in months, waiting for a rule that has not been written yet.

For the organisations holding the sales history rather than the software, there is a version of this worth taking seriously. A closed file records the price. What it does not record is which comparables were considered and set aside, on what reasoning, and whether that reasoning later proved correct. A record that carries its own corrections is a different class of asset from one that carries only outcomes, and nothing in the way the industry currently stores a transaction preserves the working.

---

The Independent Agent
Substack | Spotify | CMAflow FAQ | YouTube | Free CMA | Insights | Blog