Please be sure to provide your full legal name, date of birth, and full organization name with all corporate identifiers. Avoid the use of acronyms and special characters. Failure to follow these instructions may prevent you from accessing this model and others on Hugging Face. You will not have the ability to edit this form after submission, so please ensure all information is accurate.

The information you provide will be collected, stored, processed and shared in accordance with the Meta Privacy Policy.

Log in or Sign Up to review the conditions and access this model content.

Left-Right-Consistent Iterative Stereo Matching via Cross-Injection

Checkpoints for cross-injection (CI) (WACV 2027): a bolt-on extension that makes an iterative stereo matcher predict left and right disparities that agree with each other. Paper: arXiv:2611.09458. Code: facebookresearch/stereo_crossinjection.

At every refinement step, each view's pass is given the per-pixel signed left-right inconsistency with the other view, masked by co-visibility, as an extra input through weights that start at zero (a zero-initialised encoder for S2M2, zero-padded input channels of the finest GRU for FoundationStereo). These checkpoints are the paper's "+ CI" fine-tunes of two public stereo matchers.

Files

model file base checkpoint layout
FoundationStereo + CI foundationstereo_ci.pth (+ foundationstereo_ci_cfg.yaml) FoundationStereo ViT-L {"model": ...}, as upstream FoundationStereo
S2M2-XL + CI s2m2_xl_ci.pth S2M2-XL (CH384NTR3.pth) {"state_dict": ...}, as upstream S2M2

Each file holds the upstream model's parameters plus the cross-injection weights (the three encoder tensors for S2M2; the two widened input channels of gru04 for FoundationStereo) and loads strictly into the adapters in the code repository with crossinj.io.load_checkpoint. They contain weights only (no optimizer state) and load with torch.load(..., weights_only=True).

The FoundationStereo file leaves out FoundationStereo's frozen DepthAnythingV2 ViT-L backbone, which fine-tuning did not change. The adapter loads it separately, from the public DepthAnythingV2-Large weights, when the model is constructed; load_checkpoint accepts its absence, whereas a plain strict load_state_dict would not. foundationstereo_ci_cfg.yaml is for reference only. The files need the code repository's adapters; upstream's own demo scripts cannot load them.

Usage

The repository is gated: request access on this page (requests are approved manually), then log in with the approved account. The code installs the hf command-line tool:

git clone --recursive https://github.com/facebookresearch/stereo_crossinjection
cd stereo_crossinjection
conda env create -f environment.yml && conda activate crossinj
pip install -e ".[foundationstereo]"
hf auth login                  # or set HF_TOKEN
hf download facebook/stereo_crossinjection --local-dir checkpoints

environment.yml installs CPU builds of torch; see the code repository's README for CUDA builds, CPU runtimes and scripts/demo.py, which runs either model on your own stereo pair. In Python, crossinj.hub.download_checkpoint fetches a checkpoint into the Hugging Face cache instead.

import torch
from crossinj.adapters.s2m2 import CrossInjectionS2M2
from crossinj.io import load_checkpoint, load_image

device = "cuda" if torch.cuda.is_available() else "cpu"
model = CrossInjectionS2M2(model_size="XL")
load_checkpoint(model, "checkpoints/s2m2_xl_ci.pth")
model = model.to(device).eval()

left = load_image("left.png", device=device)    # (1, 3, H, W) float in [0, 255]
right = load_image("right.png", device=device)  # any size; the adapter pads internally
with torch.inference_mode():
    disp_left, disp_right = model(left, right)  # each (1, 1, H, W), in pixels
import torch
from crossinj.adapters.foundation_stereo import (
    CrossInjectionFoundationStereo, foundation_stereo_cfg)
from crossinj.io import load_checkpoint, load_image

device = "cuda" if torch.cuda.is_available() else "cpu"
# Loads the frozen DepthAnythingV2-Large backbone (downloaded on first use).
model = CrossInjectionFoundationStereo(foundation_stereo_cfg())
load_checkpoint(model, "checkpoints/foundationstereo_ci.pth")
model = model.to(device).eval()

left = load_image("left.png", device=device)
right = load_image("right.png", device=device)
with torch.inference_mode():
    disp_left, disp_right = model(left, right, iters=32)

Results

From the paper (Tab. 2). Error rates in % (lower is better). BP-N: bad-N px error; D1: KITTI D1; LR-BP-1: left-right disagreement above 1 px (no ground truth needed).

model Middlebury-H BP-2 Middlebury-H LR-BP-1 ETH3D BP-1 ETH3D LR-BP-1 KITTI-12 D1 KITTI-12 LR-BP-1 KITTI-15 D1 KITTI-15 LR-BP-1 Booster-H BP-2 Booster-H LR-BP-1
FoundationStereo 2.77 9.89 0.65 5.59 2.73 12.20 2.88 10.23 10.70 8.84
FoundationStereo + CI 2.68 9.03 0.61 3.26 2.49 7.09 2.69 5.34 9.24 3.60
S2M2-XL 2.98 9.58 0.64 5.05 4.12 16.26 2.97 8.56 7.00 8.31
S2M2-XL + CI 2.81 9.05 0.53 3.21 2.00 7.58 2.24 5.38 6.53 3.92

The paper also reports a matched fine-tune without cross-injection for each base model, which separates the effect of CI from that of the extra training. "-H" marks evaluation at half resolution.

Licence

These weights are released under the FAIR Noncommercial Research License (LICENSE). They are fine-tuned from third-party checkpoints, whose own non-commercial, research-only terms also apply:

DepthAnythingV2-Large, which the FoundationStereo model downloads separately, is under CC BY-NC 4.0.

Citation

@article{knott2026crossinjection,
  title   = {Left-Right-Consistent Iterative Stereo Matching via Cross-Injection},
  author  = {Knott, Manuel and Luvizon, Diogo and
             Scharstein, Daniel and Vansteenkiste, Elias},
  journal = {arXiv preprint arXiv:2611.09458},
  year    = {2026}
}

Please also cite the base model: FoundationStereo (Wen et al., CVPR 2025) or S2M2 (Min et al., ICCV 2025).

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Paper for facebook/stereo_crossinjection