Please be sure to provide your full legal name, date of birth, and full organization name with all corporate identifiers. Avoid the use of acronyms and special characters. Failure to follow these instructions may prevent you from accessing this model and others on Hugging Face. You will not have the ability to edit this form after submission, so please ensure all information is accurate.
The information you provide will be collected, stored, processed and shared in accordance with the Meta Privacy Policy.
Log in or Sign Up to review the conditions and access this model content.
Left-Right-Consistent Iterative Stereo Matching via Cross-Injection
Checkpoints for cross-injection (CI) (WACV 2027): a bolt-on extension that makes an iterative stereo matcher predict left and right disparities that agree with each other. Paper: arXiv:2611.09458. Code: facebookresearch/stereo_crossinjection.
At every refinement step, each view's pass is given the per-pixel signed left-right inconsistency with the other view, masked by co-visibility, as an extra input through weights that start at zero (a zero-initialised encoder for S2M2, zero-padded input channels of the finest GRU for FoundationStereo). These checkpoints are the paper's "+ CI" fine-tunes of two public stereo matchers.
Files
| model | file | base checkpoint | layout |
|---|---|---|---|
| FoundationStereo + CI | foundationstereo_ci.pth (+ foundationstereo_ci_cfg.yaml) |
FoundationStereo ViT-L | {"model": ...}, as upstream FoundationStereo |
| S2M2-XL + CI | s2m2_xl_ci.pth |
S2M2-XL (CH384NTR3.pth) |
{"state_dict": ...}, as upstream S2M2 |
Each file holds the upstream model's parameters plus the cross-injection
weights (the three encoder tensors for S2M2; the two widened input channels of
gru04 for FoundationStereo) and loads strictly into the adapters in the code
repository with crossinj.io.load_checkpoint. They contain weights only (no
optimizer state) and load with torch.load(..., weights_only=True).
The FoundationStereo file leaves out FoundationStereo's frozen DepthAnythingV2
ViT-L backbone, which fine-tuning did not change. The adapter loads it
separately, from the public
DepthAnythingV2-Large
weights, when the model is constructed; load_checkpoint accepts its absence,
whereas a plain strict load_state_dict would not. foundationstereo_ci_cfg.yaml
is for reference only. The files need the code repository's adapters; upstream's own
demo scripts cannot load them.
Usage
The repository is gated: request access on this page (requests are approved
manually), then log in with the approved account. The code installs the hf
command-line tool:
git clone --recursive https://github.com/facebookresearch/stereo_crossinjection
cd stereo_crossinjection
conda env create -f environment.yml && conda activate crossinj
pip install -e ".[foundationstereo]"
hf auth login # or set HF_TOKEN
hf download facebook/stereo_crossinjection --local-dir checkpoints
environment.yml installs CPU builds of torch; see the code repository's
README for CUDA builds, CPU runtimes and scripts/demo.py, which runs either
model on your own stereo pair. In Python, crossinj.hub.download_checkpoint
fetches a checkpoint into the Hugging Face cache instead.
import torch
from crossinj.adapters.s2m2 import CrossInjectionS2M2
from crossinj.io import load_checkpoint, load_image
device = "cuda" if torch.cuda.is_available() else "cpu"
model = CrossInjectionS2M2(model_size="XL")
load_checkpoint(model, "checkpoints/s2m2_xl_ci.pth")
model = model.to(device).eval()
left = load_image("left.png", device=device) # (1, 3, H, W) float in [0, 255]
right = load_image("right.png", device=device) # any size; the adapter pads internally
with torch.inference_mode():
disp_left, disp_right = model(left, right) # each (1, 1, H, W), in pixels
import torch
from crossinj.adapters.foundation_stereo import (
CrossInjectionFoundationStereo, foundation_stereo_cfg)
from crossinj.io import load_checkpoint, load_image
device = "cuda" if torch.cuda.is_available() else "cpu"
# Loads the frozen DepthAnythingV2-Large backbone (downloaded on first use).
model = CrossInjectionFoundationStereo(foundation_stereo_cfg())
load_checkpoint(model, "checkpoints/foundationstereo_ci.pth")
model = model.to(device).eval()
left = load_image("left.png", device=device)
right = load_image("right.png", device=device)
with torch.inference_mode():
disp_left, disp_right = model(left, right, iters=32)
Results
From the paper (Tab. 2). Error rates in % (lower is better). BP-N: bad-N px error; D1: KITTI D1; LR-BP-1: left-right disagreement above 1 px (no ground truth needed).
| model | Middlebury-H BP-2 | Middlebury-H LR-BP-1 | ETH3D BP-1 | ETH3D LR-BP-1 | KITTI-12 D1 | KITTI-12 LR-BP-1 | KITTI-15 D1 | KITTI-15 LR-BP-1 | Booster-H BP-2 | Booster-H LR-BP-1 |
|---|---|---|---|---|---|---|---|---|---|---|
| FoundationStereo | 2.77 | 9.89 | 0.65 | 5.59 | 2.73 | 12.20 | 2.88 | 10.23 | 10.70 | 8.84 |
| FoundationStereo + CI | 2.68 | 9.03 | 0.61 | 3.26 | 2.49 | 7.09 | 2.69 | 5.34 | 9.24 | 3.60 |
| S2M2-XL | 2.98 | 9.58 | 0.64 | 5.05 | 4.12 | 16.26 | 2.97 | 8.56 | 7.00 | 8.31 |
| S2M2-XL + CI | 2.81 | 9.05 | 0.53 | 3.21 | 2.00 | 7.58 | 2.24 | 5.38 | 6.53 | 3.92 |
The paper also reports a matched fine-tune without cross-injection for each base model, which separates the effect of CI from that of the extra training. "-H" marks evaluation at half resolution.
Licence
These weights are released under the FAIR Noncommercial Research License (LICENSE). They are fine-tuned from third-party checkpoints, whose own non-commercial, research-only terms also apply:
foundationstereo_ci.pthis a derivative work of FoundationStereo and is also subject to FoundationStereo's NVIDIA licence, a complete copy of which is in LICENSE-FoundationStereo.s2m2_xl_ci.pthis a modified version of the S2M2-XL weights, which are under CC BY-NC 4.0.
DepthAnythingV2-Large, which the FoundationStereo model downloads separately, is under CC BY-NC 4.0.
Citation
@article{knott2026crossinjection,
title = {Left-Right-Consistent Iterative Stereo Matching via Cross-Injection},
author = {Knott, Manuel and Luvizon, Diogo and
Scharstein, Daniel and Vansteenkiste, Elias},
journal = {arXiv preprint arXiv:2611.09458},
year = {2026}
}
Please also cite the base model: FoundationStereo (Wen et al., CVPR 2025) or S2M2 (Min et al., ICCV 2025).