Amodal Counting: count what detectors can't see

Picture a platform camera watching a boarding queue at rush hour, or a checkpoint camera counting people through a gate. As the crowd thickens, people start standing behind each other: a shoulder here, a partial torso there, someone almost fully hidden by the person directly in front. A standard object detector draws a box around every person it can actually see, and the count of boxes becomes the crowd number that everything downstream, capacity alerts, density thresholds, dashboards, treats as the truth. The moment two people meaningfully overlap in the camera's flat 2D view, the person behind becomes partially or fully invisible to the detector. The detector is not malfunctioning; the pixels it would need to see that person simply are not there.

This kind of undercount does not announce itself. The raw number looks confident and precise, no error bars, no caveat, and it is tempting to treat it as ground truth. But the gap between what is drawn and what is actually present grows exactly when it matters most: the denser the crowd, the more occlusion, and the more the raw count silently underestimates. A safety or capacity system built on that bare number alone will systematically under-react during the busiest, highest-risk moments, and it will do so quietly, with no signal that anything is wrong.

Training a bigger or more accurate detector does not remove this problem, it only pushes the same failure to a higher density, because occlusion is a geometric fact of a flat projection, not a gap in model capacity. What is needed instead is a way to reason about which detections were even possible to make, given how much of each object was actually visible, use that to correct the count, and state honestly how much to trust the correction. That is what Amodal Counting does.

1. Visibility from box geometry. Visibility of a detection is the fraction of its box not covered by occluders, not covered by other detections in front of it, and not truncated by the image border. A missed occluding object can only ever make this look less occluded than reality, so the bias runs one direction, which keeps the downstream correction conservative by construction.

2. Calibrated detectability curve. A logistic curve fit by maximum likelihood from labeled (visibility, was-detected) pairs, carrying its own calibrated support: a query outside the range actually seen during fitting is clamped and flagged extrapolation_limited, never silently extrapolated. A multivariate variant extends this to box scale, local crowd density, and image y-position, not just geometric visibility, when those features are genuinely predictive.

3. Horvitz-Thompson interval counting. Each detection is weighted 1 / p(detect); weights sum to the corrected estimate. Three honesty gates are in the code, not just the docs: the corrected count can never drop below the raw detection count (unless the two-sided estimator below is deliberately enabled), every weight is capped so a single deep-occlusion detection cannot conjure a crowd, and any detection outside calibrated support flags the whole estimate.

4. Conformal interval calibration. The analytic variance is a textbook approximation and is measured to under-cover; a calibration pass on labeled scenes fits a multiplier so the declared coverage claim is empirical, not assumed.

5. Two-sided estimator. Standard reweighting can only push a count up. A second fitted model, the probability a detection is real versus a tiling duplicate or artifact, allows the correction to also push down, weighting each detection by p_true / p_detect instead of 1 / p_detect. This is the change that lets the estimator address a detector whose error is a two-sided mix of undercount and overcount, at the cost of dropping the never-below-raw-count floor by design, documented in code as an intentional, narrow relaxation.

6. Track-based occlusion persistence. A track that vanishes mid-frame away from an exit region is treated as probably occluded, with a survival probability that decays over hidden time, fit from observed reappearance-gap statistics, not hand-picked.

A worked example, with numbers from this session's run

The bench --seed 0 run reproduced in this session fit, at crowding 0.3, a detectability curve with k=7.5703 and v0=0.3347 from 720 calibration samples. Under that curve, a detection that is 90% visible has p(detect)=0.986 and contributes a weight of 1.014, essentially just itself. A detection that is 60% visible has p(detect)=0.882 and contributes 1.134. A detection that is only 30% visible has p(detect)=0.435 and contributes 2.300: the estimator infers that for every such barely-visible object the detector did catch, roughly 1.3 similar ones were missed entirely. Summing these weights across the scene, instead of counting boxes, is the whole correction. The conformal calibration step then fit z_scale=8.2022 at this crowding level, an honest admission that the textbook variance formula was too optimistic by roughly 8x on these worlds, and the reported interval is widened accordingly before any coverage claim is made.

Diagram reflects the one-directional path (steps 1 to 4). The two-sided estimator (mechanism step 5, above) swaps step 3's weight for p_true / p_detect, using a second precision model fit alongside the detectability curve; see the real-data results below for when each path wins.

Each example below was generated by running the real amodal-counting visibility-estimation, detectability-curve calibration, and Horvitz-Thompson interval-counting code against a synthetic scene, offline, once, with the same private codebase whose evidence trail this page cites. Red boxes are ground truth (hidden from the algorithm, darker means less visible); blue boxes are what the simulated detector reported.

Command run fresh in this session: PYTHONPATH=src python -m amodal.cli bench --seed 0. The detectability curve and interval scale are calibrated on 60 training worlds and evaluated fresh on 40 held-out worlds per crowding level; no number here is computed on data the calibration saw.