When the Detector Disagrees With the Label

Every false positive is two hypotheses at once: the model is wrong, or the label is wrong. Detection evaluation assumes the first. Dataset improvement has to consider the second.

This is a deep dive on Axis A — label integrity from Active Learning Needs Orthogonal Axes. That post argued each kind of intervention deserves its own evidence and its own queue. Axis A is the one that never asks for new data. Its candidate universe is the labeled pool only; its job is to find annotations that may be wrong or incomplete and route them to a reviewer. The question it answers is narrow: is an existing annotation wrong or incomplete?

The idea: TIDE was built to diagnose the model — to break mAP into interpretable error types. Flip the assumption — trust the model, doubt the label — and the same error types become a ranked list of annotation suspects. A detector disagreement is a suspicion, never proof. The output is a label-audit queue, never an automatic mutation of the dataset.

TIDE in one paragraph

TIDE matches each prediction to ground truth by IoU and assigns it an error type. Two thresholds do the work: a foreground threshold (IoU ≥ 0.5, a real match) and a background threshold (IoU < 0.1, a match to nothing). Between them sit the interesting cases — right class but loose box, right box but wrong class, a second box on an object already found. In a normal evaluation each of these is charged against the model. Here, every one of them is also a place where the annotation could be the thing that is wrong.

The flip

The same signal reads two ways depending on which side you trust:

Model-eval lens: a high-confidence box on background (fp_bg) is a hallucination — penalize precision.
Dataset lens: the same box may be a real object that nobody annotated — a missing-label suspect.

Axis A runs the dataset lens. It does not decide which hypothesis is true; it surfaces the disagreement, attaches the evidence, and lets a human adjudicate. What changes versus model evaluation is only the default suspicion.

The labeling errors it catches

Each subtype is triggered by a TIDE error type from classify_errors, where a model prediction disagrees with the ground truth:

Subtype	TIDE trigger	What it means
Missing label	`fp_bg` — high-confidence box, IoU ≈ 0 with any GT	Model found a real object nobody annotated (or a hallucination)
Wrong class	`fp_cls` — high-IoU box, high confidence, different class	Box is right, label is wrong (class confusion)
Inaccurate box	`fp_loc` — right class, marginal IoU (loose / tight / shifted)	Object is labeled but the box geometry is off
Duplicate annotation	`fp_dup` — two boxes for one object	Redundant / overlapping GT
Invalid / ambiguous	mixed signals, low agreement	Annotation can't be trusted, or the class is genuinely unclear

What does not go to Axis A

The recall-side TIDE types are not label-integrity evidence. A genuine miss (fn_miss) and a below-threshold miss (fn_thr) are model failures, not annotation failures — the label is presumably fine; the detector did not find it or found it too weakly. Those route elsewhere:

fn_miss on a thinly covered slice → Axis B (coverage scarcity);
fn_thr, or misses near a confusable boundary → Axis C (decision-boundary information);
misses despite dense coverage → Axis E (model / pipeline limitation — resolution, tiling, threshold, NMS).

Keeping these off Axis A is the whole point of separating the axes: a recall failure should never quietly become a request to re-annotate data that was annotated correctly.

From error type to audit queue

A raw list of TIDE disagreements is not yet a queue. Three things turn it into one:

Rank by how surprising the disagreement is — a very-high-confidence fp_bg on a clean, well-covered slice is a far stronger missing-label suspect than a borderline one near the background threshold.
Attach the evidence, not just the verdict — the prediction, its confidence, the IoU to the nearest GT, the conflicting classes, and the crop. A reviewer should decide in seconds, not re-investigate from scratch.
Group by subtype — missing-label review, class correction, and box tightening are different reviewer tasks with different speeds and different acceptance rates. Mixing them slows all three.

Why it stays a suspicion

The discipline that makes Axis A safe is refusing to act on the model's word alone. A detector can be wrong; a label can be wrong; sometimes both are partially wrong. A vision-language model's description of the crop is not proof either — it is one more evidence source that helps prioritize, not a verdict. So Axis A proposes; a human disposes. Every item should carry its provenance — which trigger fired, what evidence supported it, what the reviewer decided, and whether the correction actually reduced error on the relevant slice afterward.

Measured the right way, this axis pays for itself: reviewer acceptance rate, corrected annotations per review hour, agreement between reviewers, and the drop in annotation-induced error after correction. None of those require collecting a single new image — which is exactly why label integrity earns its own axis instead of hiding inside a global active-learning score.

This is a deep dive on one axis of Active Learning Needs Orthogonal Axes. The error-type names (fp_bg, fp_cls, fp_loc, fp_dup, fn_miss, fn_thr) follow a TIDE-style classifier; thresholds shown are TIDE's defaults.