DIMED · DIagnosing MEtric Depth

What the Averages Hide

Monocular Metric Depth Models Still Fail in Everyday Scenes, Mostly on Scale

Hualie Jiang1, Yujie Luo1,2, Ziyang Song1, Ximeng Li1, Zhiqiang Lou1, Rui Xu1†

1Antigravity Tech, Insta360    2Nanyang Technological University

How far away is this table?

An empty table seen from a robot-mounted camera

A camera looks down at an empty table. Guess the typical distance from the camera to the visible surface.

1.00 m

Same table, same model: objects can fix the scale, but not always

MoGe-2 depth ground-truth depth
⇆
ground truthMoGe-2 (with K)
depth colour scale

Empty table. Drag: left = ground truth, right = MoGe-2 (given the true intrinsics). MoGe-2 places the table 3.5× too far (2.63 m for a true 0.76 m, δ1 0.00); all 18 model evaluations fail on this image.

The geometry is right in all three: what the model lacks is a reliable cue for how big the scene is.

0.875 yet 1 in 13The best of 18 evaluations has mean δ1 0.875, yet still fails on 7.7% of images (metric δ1 < 0.5).
98.3%of those failures pass once a single global scale is corrected; 78.8% have intact structure and are wrong only in scale.
~40%of the top models' error falls on the 7.3% of images that most of the 11 models fail, against 14% to 18% for the same number of random images; they lie on HAMMER (4 of its 7 scenes) and DIODE.

Why it matters

A robot that grasps, an AR app that places furniture, or a SLAM system initialised from one frame needs metres, not relative depth. A depth map with the right shape but 3.5× the wrong scale puts the cup out of reach. Dataset averages hide this: a model can score well on average while a fixed set of everyday views, such as an empty table or a blank wall, defeats it every time.

Most models fail on the same images

Pick an image and a model. Ground truth and prediction share one colour scale. White: no ground truth, or sky / invalid region as predicted by the model itself (MoGe-2, MoGe-3, MA-PointMap, FoundationGeo, DA3METRIC output such a mask; the other models do not). In almost all of these failures the shape is right and only the scale is off.

input image
ground-truth depth
predicted depth
depth colour scale
They fail in the same direction. On 66% of the images that at least 6 of the 11 models fail, every failing model errs the same way. In the examples here, the HAMMER tables and most indoor views are placed too far, and the open DIODE outdoor views with sky mostly too near.
A pass is not always a cue. MA-DepthMap gets most HAMMER tables right, but not because it reads the scene better: on HAMMER its depth is consistently about 30% nearer than that of the other models, which cancels their shared "too far" error. On the frames with objects it fails more often than the others (35% vs. a median of 19%).
Objects in view are not enough. In the second group a person can judge the size from furniture, monitors or buildings, yet 6 to 10 of the 11 models still miss it, again mostly in one direction.

Where each model fails

|

The averages look similar across models; the failures do not. Most of them sit on DIODE and HAMMER, and on HAMMER almost every model errs in the same direction.

What averages do not show

Evaluations average a metric score and a scale-aligned score per dataset. DIMED keeps, for every image, the metric δ1, the scale-aligned δ1 and the alignment scale s*, and reports:

Per-image metric error against the alignment scale
Per image, metric accuracy is decided mainly by whether s* falls inside [0.8, 1.25] (shaded); errors in structure are small.

Main results

18 model evaluations on the 7 everyday-scene benchmarks used by MoGe-2 (4,406 images), datasets weighted equally. Click a column to sort.

↑/↓: higher/lower is better; →0: closer to zero is better. Bias = mean(ln(1/s*)), positive = depth overestimated. Dispersion = std(ln s*) within a dataset.

Try it on your model. Evaluate it with our code, then split its errors by our per-image shared-failure labels, without re-running the 11 baselines:

git clone https://github.com/antigravity-tech/DIMED && cd DIMED/eval
bash run.sh <your_model> 0
python ../analysis/stratify_new_model.py ../results/shared_failure_labels.csv <results>/main/<run>

Image content moves the scale

Change in scale error when objects or printed patterns are added
(a) On paired HAMMER frames, adding objects reduces the scale error of 10 of 11 models, also when the scale is solved on the table and walls alone. (b) TUM RGB-D sequences with printed posters (separate recordings) go with underestimated depth.

Intrinsics and model size help, but not enough

Effect of ground-truth intrinsics
Ground-truth intrinsics raise metric δ1 for every model that can run without them. Even with intrinsics and its largest public size, every architecture still fails on at least 8% of images.

What to do with this

Using metric depth? Do not trust the absolute scale on views without objects of known size (empty tables, blank walls). One known length, or one scale per sequence, fixes most failures, because the shape is usually right.
Building a model? Report the failure rate and the bias of ln s* next to the mean, and check your errors against our shared-failure labels: they show whether you fixed the hard images or only improved the rest.
Supplying intrinsics? They raise metric δ1, but they are not an upper bound: for models that solve the focal length after the network they lower the scale-aligned δ1, and on HAMMER some models (including the ray-input UniK3D) lose metric accuracy with them.

Code and data

Citation

@article{jiang2026averages,
  title   = {What the Averages Hide: Monocular Metric Depth Models Still Fail in Everyday Scenes, Mostly on Scale},
  author  = {Jiang, Hualie and Luo, Yujie and Song, Ziyang and Li, Ximeng and Lou, Zhiqiang and Xu, Rui},
  journal = {arXiv preprint arXiv:2611.20518},
  year    = {2026}
}