Our Evidence Standard: Rule, Record, Assessment
Short answer: We are about to start documenting what happens in collision claims, not just what the law requires. This page sets the rules we will follow, published before we hold a single record, so they can be used against us.
Why this is published before the evidence, not after
Everything on this site so far is what the law requires. Statutes, regulations, insurance department bulletins, OEM procedures. That material is quotable, dated, and traceable to a government server, and we grade our confidence in every row of it in public.
What comes next is different in kind: what documents show actually happened. Estimates, denial letters, supplements, correspondence, complaints, enforcement actions. That material concerns named private parties, arrives from people with a stake in the outcome, and can be wrong in ways a statute cannot.
Anyone can promise rigour after publishing an accusation. Committing in advance to the things we will refuse to say, in a place where anyone can hold us to it, is the only version of this worth doing. So this page exists before the first record does.
If you work in this industry, this page is an invitation to attack. Every hole found in it now is a hole that never becomes a correction. If you are an appraiser, an adjuster, a repairer, or an attorney and something below is naive, wrong, or unworkable in practice, we would rather hear it from you than discover it.
The three layers, and the wall between them
| Layer | What it is | The question it answers |
|---|---|---|
| 1. The Rule | Statute, regulation, bulletin, order, court decision, OEM procedure, policy language | What is required, of whom, and when? |
| 2. The Record | A specific document: an estimate, a denial, a communication, on a date | What does this document show? |
| 3. The Assessment | A named reviewer's judgement of how a record relates to a rule | How do these two relate, and according to whom? |
An incident never changes the law, and the law never proves the incident happened. A rule is true whether or not anyone follows it. A document is what it is whether or not it turns out to be lawful. Layers 1 and 2 are established separately, by different methods, and neither is evidence for the other. Only Layer 3 connects them, and Layer 3 is opinion with a name on it.
These are three separate records in three separate tables. Not three columns of one table.
The five things an assessment can say
- CONSISTENT. The documented conduct appears consistent with the requirement.
- QUESTION RAISED. There is a document and there is a rule, and not yet enough to say how they relate.
- POTENTIAL CONFLICT. The document appears inconsistent with a specific requirement, subject to facts or legal interpretation not yet resolved.
- OFFICIAL FINDING. A regulator, court, or other authorised body has found a violation. Nothing but that body's own record supports this state.
- DISPUTED. Both sides have evidence or interpretation and it is unresolved.
"We do not know yet" is a publishable result. It is QUESTION RAISED, and it will be the honest state for most things most of the time. A system that can only report findings will eventually manufacture them.
The four claims we will never make
Each of these is enforced by a deploy gate, and each has a stated falsifier. If you can show us in breach of one, we will correct it in the open, the way we correct everything else.
1. No rate without a denominator. We will never publish a percentage, a frequency, or a count that implies one, without stating what it is a count of and where that denominator came from. Forty-seven documented instances out of forty-seven submissions is not a rate. It is forty-seven documents. You have caught us if: any figure implying frequency appears without its denominator on the same page.
2. No violation without an official finding. Only a regulator or a court determines that law was broken. We can publish what a rule requires, what a document shows, and that the two appear to conflict. We cannot say someone broke the law, and neither can our reviewers. You have caught us if: the words "violation", "illegal" or "unlawful" attach to a named party in any state other than OFFICIAL FINDING.
3. No named party without a primary document we obtained ourselves. A company, shop or individual is named only where we hold the underlying document and have read it. Not a summary, not a description, not somebody's account of it. You have caught us if: a named private party appears with no primary document behind the record.
4. No assessment without a named reviewer and their interest. Every assessment carries who made the judgement and where they sit in the industry. An attorney who represents repairers assessing a repairer's dispute is not disqualified. It is disclosed, on the page, beside the assessment. You have caught us if: a published assessment has an anonymous or uncredentialed reviewer.
The thing most people get wrong about this kind of data
People who send evidence are people with a grievance. No amount of care in intake produces a representative sample, because the sample selects itself at the source. This is not a quality problem to manage. It is a permanent property of voluntary reporting, and it does not improve with volume.
So our records can honestly support "here are N documented instances, and here is what each one shows." They can never support "this occurs X% of the time."
We do not intend to rely on discipline for that. The database carries no aggregate view, no count endpoint and no percentage column on observations. Producing a rate requires a separate denominators table with its own cited source. Where no denominator exists, the query that would produce a rate cannot be written. This is the same approach as the checks that gate every page on this site: we do not remember not to publish the wrong number, the system declines to compute it.
Assessments are superseded, never edited
When an attorney corrects a reading, or a court rules, the assessment gets a new entry and the old one is marked superseded. The original stays readable. The record it rested on is never rewritten, and neither is the rule.
That is the difference between a record and a story. A record shows what was believed, by whom, and when it changed. A story shows only the current version and asks you to trust it about the rest. Our corrections log already works this way, including the entries where we were wrong.
If we ever take submissions: leads in, primary sources out
A submission would be a pointer telling us where to look. It would never itself be the published evidence. Nothing reaches the record layer until we have independently obtained the underlying document.
Anonymous submissions of unverifiable material, aggregated into a count, is the fastest available way to destroy what the rule layer has earned. One planted document would do it, and the parties with a motive to plant one are not short of resources.
Where we are honest about the limits
We are not a law firm and this is not legal advice. We publish what a source says and where to read it. What it means for a specific claim, on a specific vehicle, under a specific policy, is a question for a lawyer.
We do not have decades in this industry. We have the research infrastructure and the discipline to source things properly. We are actively looking for collision professionals, appraisers, claims professionals and attorneys who can tell us where the data misleads, where an interpretation is naive, and which problems are actually worth measuring.
No shop, insurer, association, sponsor or expert decides what the evidence says. Advice, access and correction are welcome from every direction. Editorial control is not for sale, and if that ever changes this page will say so before anything else does.
Sources
- This page states our own methodology and commits us to it. The internal version that governs the repository is
docs/EVIDENCE-STANDARD.md, and it is enforced at deploy time rather than left to memory. - How the rule layer is sourced and graded: how our data is verified
- The confidence ladder in practice, including the rows where we could not reach the text: body shop licensing compared across all 51 jurisdictions
General consumer information: not legal, insurance, or financial advice. Requirements, coverage, and practices vary by state, policy, and manufacturer.
Where this fits
Each link says what it is for. We add one only when a reader on this page has a real reason to need that page next.
- How This Site Is Written, and What the Machine Is Never Allowed to Do (who applies this standard, and the four things the machine is forbidden to do)