Offline Base

Nemotron 3.5 Bolt + Nemotron 3.5 Parse: a two-stage document reader

Packaged and tested by Offline Base: private AI that runs on hardware you own.

An all-NVIDIA way to read documents and answer questions about them:

  1. Nemotron 3.5 Parse (0.63B) reads a page image and writes it out as Markdown, with tables as HTML, using NVIDIA's own postprocessing.
  2. Nemotron 3.5 Bolt E4B (8B total, 4B effective) answers questions from that Markdown.

Parse is a separate encoder-decoder model (RADIO vision encoder with an mBART-style decoder), not a projector, so it cannot be loaded into Bolt's llama.cpp process. The two run as two stages. If you want one model and one process, see the sister repo OfflineBase/Nemotron-3.5-Bolt-E4B-Vision-GGUF, which runs Bolt with Gemma 4 E4B's vision projector (unofficial: NVIDIA lists Bolt's input as text) and measures it on a Jetson Orin Nano.

Community packaging. Not made, reviewed or endorsed by NVIDIA. This repo holds code and documentation; both models are downloaded from NVIDIA's own repositories at pinned revisions.

Models used

Stage Model Parameters Runtime Source (pinned revision) License
1. Read the page Nemotron 3.5 Parse 0.63B PyTorch + transformers 5.14.1 (trust_remote_code) nvidia/Nemotron-3.5-Parse @ 62cb1e4 OpenMDW-1.1
2. Answer Nemotron 3.5 Bolt E4B, Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf (6.18 GB, SHA-256 c8af2b8aโ€ฆ9612) 8B total, 4B effective llama.cpp llama-server nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB @ 1db54d5 OpenMDW-1.1

Quick start

pip install -r requirements.txt                        # Python 3.12
llama-server -m Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf -c 8192 -ngl 99 -fa on --jinja --reasoning off   # stage 2

python3 pipeline.py page.png                           # print the page as Markdown
python3 pipeline.py page.png -q "What is the security deposit?"   # parse, then ask Bolt

pipeline.py picks CUDA, then Apple silicon, then CPU for Parse (--device overrides). On a Jetson Orin Nano, serve Bolt with run/serve-bolt-jetson-orin-nano.sh.

Use cases

  • Structured extraction. Parse keeps tables as tables (HTML in the Markdown), so statements, invoices and inventories can go straight into a spreadsheet or database, and Bolt answers questions over the result.
  • Building a searchable archive. Batch-convert a backlog of scanned pages on a workstation GPU into clean, layout-aware Markdown chunks for retrieval, then serve questions with Bolt on a small edge box.
  • An all-NVIDIA stack. Both stages are NVIDIA Nemotron models under the same OpenMDW license, which matters for some partners and procurement rules.
  • Layout-heavy documents. Forms, multi-part pages and pages with figures, where reading order and structure matter as much as the words.

Not a fit (yet): an 8 GB edge board running everything in one process, since Parse needs PyTorch and has not been tested on a Jetson. For that, use the single-model sister repo.

How Parse read our test pages

The same 50 made-up pages used for the sister repo (10 document types ร— clean, scan, heavy scan, phone photo, low-res; 305 key facts). Parse ran on a Mac M4 Max (PyTorch, Apple GPU, bf16) with NVIDIA's default prompt and postprocessing.

Engine Key facts read exactly 95% interval Lines exact Invented numbers Table rows intact Seconds per page
Nemotron 3.5 Parse 97.0% 94.5 to 98.4 94.3% 5 (1 on readable pages) 85/85 7.4 (Mac)
Bolt + Gemma 4 E4B projector, for comparison 98.0% 95.8 to 99.1 96.4% 15 (0 on readable pages) 85/85 13.0 (Jetson)

By condition, Parse found 100% of key facts on clean and low-res pages, 98.4% on scans and phone photos, and 88.5% on heavy scans. Its outputs are structured Markdown with real tables, which suits retrieval and citation. Reviewers who checked every miss against the page image found three real misreads, including a 4-digit gate code written as Japanese characters, and one dropped address line on a heavy-scan lease. Every output and the audit are in eval/.

What has and has not been tested

  • Tested: Parse on 50 pages (Mac, NVIDIA's prompt and postprocessing, exactly the code path in pipeline.py); Bolt on Jetson Orin Nano boards as a chat and question-answering model.
  • Not yet tested: the two stages run end to end through pipeline.py; Parse on a Jetson (it needs PyTorch for JetPack 7, which we have not verified); Parse's memory use beside Bolt on an 8 GB board.
  • Memory: on Apple silicon, Parse's GPU memory peaked near 12 GB in our run. Plan for that on shared machines.

License

Both models are NVIDIA's, under the OpenMDW License Agreement, version 1.1; Parse's tokenizer is under MIT (see NVIDIA's repo). The code and documentation here are Apache 2.0. See NOTICE.md.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for OfflineBase/Nemotron-3.5-Bolt-Parse-Pipeline