cronos3k commited on
Commit
917ab65
·
verified ·
1 Parent(s): 0843040

Add torchvision (Qwen2-VL video sub-processor dep); switch default reasoning model to E4B (works on ZeroGPU CUDA emulation)

Browse files
README.md CHANGED
@@ -81,12 +81,12 @@ Alternative reasoning models — set `REASONING_MODEL_ID` to switch:
81
 
82
  | `REASONING_MODEL_ID` override | Approx VRAM | Recommended slice |
83
  |---|---|---|
84
- | `nvidia/Gemma-4-26B-A4B-NVFP4` *(default)* | ~16 GB (NVFP4) | `large` (48 GB) |
85
  | `google/gemma-4-26B-A4B-it` | ~52 GB (bf16) | `xlarge` (96 GB) |
86
  | `google/gemma-4-31B-it` | ~62 GB (bf16) | `xlarge` |
87
- | `google/gemma-4-E4B-it` | ~8 GB (bf16) | `large` — smaller / faster |
88
  | `openai/gpt-oss-20b` | ~16 GB (MXFP4) | `large` — needs Hopper+ |
89
- | `RedHatAI/gemma-4-26B-A4B-it-NVFP4` | ~16 GB (NVFP4) | `large` community quant |
90
 
91
  ## Configuration
92
 
 
81
 
82
  | `REASONING_MODEL_ID` override | Approx VRAM | Recommended slice |
83
  |---|---|---|
84
+ | `google/gemma-4-E4B-it` *(default)* | ~8 GB text / ~16 GB multimodal (bf16) | `large` (48 GB) — places cleanly through ZeroGPU CUDA emulation |
85
  | `google/gemma-4-26B-A4B-it` | ~52 GB (bf16) | `xlarge` (96 GB) |
86
  | `google/gemma-4-31B-it` | ~62 GB (bf16) | `xlarge` |
87
+ | `RedHatAI/gemma-4-26B-A4B-it-NVFP4` | ~16 GB (compressed-tensors NVFP4) | `large` — but module-level NVFP4 unpacking hangs through CUDA emulation; needs lazy-load fix |
88
  | `openai/gpt-oss-20b` | ~16 GB (MXFP4) | `large` — needs Hopper+ |
89
+ | `nvidia/Gemma-4-26B-A4B-NVFP4` | ~16 GB (modelopt) | requires `nvidia-modelopt`, not native to transformers |
90
 
91
  ## Configuration
92
 
legal_doc_redteam/reasoning_review.py CHANGED
@@ -22,7 +22,7 @@ from __future__ import annotations
22
  import json
23
  from typing import Any, Callable
24
 
25
- DEFAULT_REASONING_MODEL = "nvidia/Gemma-4-26B-A4B-NVFP4"
26
 
27
  SYSTEM_INSTRUCTIONS = (
28
  "You are the Document Integrity Verifier — a defensive auditor that "
 
22
  import json
23
  from typing import Any, Callable
24
 
25
+ DEFAULT_REASONING_MODEL = "google/gemma-4-E4B-it"
26
 
27
  SYSTEM_INSTRUCTIONS = (
28
  "You are the Document Integrity Verifier — a defensive auditor that "
requirements.txt CHANGED
@@ -9,6 +9,10 @@ accelerate>=0.34
9
  kernels>=0.4,<0.15
10
  compressed-tensors>=0.7
11
  torch>=2.8
 
 
 
 
12
  rapidocr-onnxruntime>=1.4
13
  onnxruntime>=1.18
14
  beautifulsoup4>=4.12
 
9
  kernels>=0.4,<0.15
10
  compressed-tensors>=0.7
11
  torch>=2.8
12
+ # Qwen2-VL family (Nanonets-OCR-s) auto-loads a video sub-processor that
13
+ # requires torchvision even if we only feed it images. Without it, the VLM
14
+ # OCR backend errors out at AutoProcessor.from_pretrained import time.
15
+ torchvision
16
  rapidocr-onnxruntime>=1.4
17
  onnxruntime>=1.18
18
  beautifulsoup4>=4.12