Simplify the model card

README.md

@@ -35,12 +35,7 @@ tags:
   </div>
 </div>
 
-This is the image detector of [SAM 3.1](https://huggingface.co/facebook/sam3.1) by Meta, converted to the transformers `Sam3Model` format and stored as BF16 safetensors. <br>
-Give it an image and a short text prompt such as "sofa" or "red lamp", and it returns a mask, a box and a score for every matching instance.
-- Loads with stock transformers (`Sam3Model` + `Sam3Processor`, any release that ships SAM 3). Meta publishes SAM 3.1 only as the original multiplex checkpoint (`sam3.1_multiplex.pt`, 3.5 GB) and transformers has no SAM 3.1 class; this repository is the detector alone in the existing SAM 3 layout, 1.68 GB.
-- Powers "Mask from text" in [Unsloth Studio](https://github.com/unslothai/unsloth/): type `sofa, red lamp, pillow:2` under the inpaint or localized-edit mask painter and the matching objects become an editable mask.
-- Same masks as the fp32 conversion and as [facebook/sam3](https://huggingface.co/facebook/sam3) (IoU of at least 0.99 on every test prompt, see [Checks](#checks)).
-- Also ships pre-quantized detectors (FP8, INT8 and INT8-ConvRot, 0.99 to 1.05 GB instead of 1.68 GB, about 0.6 GB less GPU memory). INT8-ConvRot is the closest to BF16 and is what Unsloth Studio loads by default on NVIDIA GPUs; see [Files](#files) and [Checks](#checks).
+SAM 3.1 by Meta in the transformers `Sam3Model` format: give it an image and a text prompt such as "sofa" and it returns a mask, a box and a score for every match. It powers "Mask from text" in [Unsloth Studio](https://github.com/unslothai/unsloth/).
 
 | Prompts: sofa, red lamp, pillow, potted plant, rug |
 |---|
@@ -50,33 +45,17 @@ Give it an image and a short text prompt such as "sofa" or "red lamp", and it re
 |---|
 | <img src="samples/street.png" width="900"> |
 
-Each colour is one text prompt (the count is the number of instances found at score 0.5); generated with this repository in BF16, threshold 0.5.
+| File | Size | Mask IoU vs BF16 (mean / min) |
+|---|---|---|
+| `model.safetensors` (BF16) | 1.68 GB | 1.0000 / 1.0000 |
+| `sam3.1-INT8-ConvRot.safetensors` | 1.05 GB | 0.9984 / 0.9913 |
+| `sam3.1-INT8.safetensors` | 0.99 GB | 0.9976 / 0.9829 |
+| `sam3.1-FP8.safetensors` | 1.05 GB | 0.9971 / 0.9805 |
 
-## Files
-
-| File | Size | Contents | Mask IoU vs BF16, mean / min | Score delta vs BF16, mean / max | Studio |
-|---|---|---|---|---|---|
-| `model.safetensors` | 1.68 GB | `Sam3Model` weights, BF16, 840M parameters | 1 / 1 | 0 / 0 | CPU, Apple Silicon, `UNSLOTH_SAM3_PRECISION=bf16` |
-| `sam3.1-INT8-ConvRot.safetensors` | 1.05 GB | INT8 per-channel weight with ConvRot rotation (block Hadamard, group 256), dynamic per-token INT8 activation; first 8 ViT blocks BF16 | 0.9984 / 0.9913 | 0.0012 / 0.0111 | default on NVIDIA GPUs (Ampere and newer) |
-| `sam3.1-INT8.safetensors` | 0.99 GB | INT8 per-channel weight, dynamic per-token INT8 activation; first 4 ViT blocks BF16 | 0.9976 / 0.9829 | 0.0015 / 0.0146 | `UNSLOTH_SAM3_INT8_CONVROT=0` |
-| `sam3.1-FP8.safetensors` | 1.05 GB | FP8 e4m3, per-row weight scale, dynamic per-token activation; first 8 ViT blocks BF16 | 0.9971 / 0.9805 | 0.0023 / 0.0485 | `UNSLOTH_SAM3_PRECISION=fp8` only |
-| `config.json` | | `Sam3Model` config |
-| `processor_config.json` | | `Sam3Processor` (1008x1008 input) |
-| `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `vocab.json`, `merges.txt` | | text tokenizer, unchanged from `facebook/sam3.1` |
-| `LICENSE` | | the SAM License, unchanged |
-
-The quantized files replace `model.safetensors` only; they use the same configs and tokenizer. In all three the ViT and the text encoder are quantized and every 256-wide DETR, geometry, mask-decoder and scoring layer stays BF16. Mask IoU and score deltas are over 53 text prompts, see [Checks](#checks). No quantized file changed which objects were found: no kept-count change, no false positive, no detection of an absent concept.
+53 text prompts on 6 images; 1.0 means identical masks. Unsloth Studio uses INT8-ConvRot on NVIDIA GPUs and BF16 elsewhere.
 
 ## Usage
 
-### Unsloth Studio
-
-Open Images, pick Inpaint (or the mask or paint mode of a localized edit), load a source image, type the objects in "Mask from text" and press Find. `:N` keeps up to N instances of a concept (`pillow:2`), and "Add to current mask" adds to what is already painted. Studio downloads this repository on first use, runs it next to the image model, and unloads it when the GPU is needed elsewhere.
-
-Studio fetches one weights file with the configs and tokenizer: `sam3.1-INT8-ConvRot.safetensors` on an NVIDIA GPU (Ampere or newer), `model.safetensors` on a CPU or a Mac. `UNSLOTH_SAM3_PRECISION` (`auto`, `bf16`, `fp8`, `int8`) overrides the choice; `auto` only picks a file that passed the accuracy gate below. When a mask has to run on the CPU (the GPU is busy) and only a quantized file is on disk, its weights are dequantized for the CPU.
-
-### transformers
-
 ```python
 import torch
 from PIL import Image
@@ -95,71 +74,6 @@ results = processor.post_process_instance_segmentation(
 print(results["scores"], results["boxes"], results["masks"].shape)
 ```
 
-One prompt per call; run the model once per concept and combine the masks for several.
-
-### Quantized files outside Studio
-
-The quantized files are safetensors in torchao's layout (`tensor_names` and a per-tensor entry in the header; torchao 0.18 or newer) and load into a stock `Sam3Model` with `strict=True`:
-
-```python
-import json, torch
-from accelerate import init_empty_weights
-from safetensors import safe_open
-from safetensors.torch import load_file
-from torchao.prototype.safetensors.safetensors_support import unflatten_tensor_state_dict
-from transformers import Sam3Config, Sam3Model
-
-path = "sam3.1-INT8.safetensors"  # or sam3.1-FP8.safetensors
-with safe_open(path, "pt") as f:
-    header = f.metadata()
-state, _ = unflatten_tensor_state_dict(load_file(path), header)
-with init_empty_weights(include_buffers=False):
-    model = Sam3Model(Sam3Config.from_pretrained("unsloth/sam3.1"))
-model.load_state_dict(state, strict=True, assign=True)
-model = model.to("cuda").eval()
-```
-
-`sam3.1-INT8-ConvRot.safetensors` stores the listed Linears' weights in a rotated basis (`activation_rotation_fqns` in `unsloth_metadata`): their inputs must be multiplied blockwise by the same normalized Hadamard matrix (group 256) before the matmul, which Unsloth Studio does. The INT8 and FP8 files need nothing extra.
-
-## Checks
-
-### Quantized files
-
-53 text prompts over six test images (the two above and four more: kitchen, desk, park, market), including nine concepts that are not in the picture and twelve with a `:N` count, run through Unsloth Studio's own Mask from text path on an NVIDIA B200, score threshold 0.5. Each file is compared with this BF16 model. Two BF16 runs are bit-identical, so the yardstick is what BF16 itself already moves against the fp32 conversion, and a file passes when every number is within twice that.
-
-| | Mask IoU, mean / min | Score delta, mean / max | Presence delta, max | Kept-count changes | False positives | Gate |
-|---|---|---|---|---|---|---|
-| BF16 vs the fp32 conversion (yardstick) | 0.9984 / 0.9878 | 0.0017 / 0.0125 | 0.0056 | 0 | 0 | |
-| Tolerance (2x) | 0.9968 / 0.9755 | 0.0034 / 0.0249 | 0.0113 | 0 | 0 | |
-| `sam3.1-INT8-ConvRot.safetensors` | 0.9984 / 0.9913 | 0.0012 / 0.0111 | 0.0049 | 0 | 0 | pass |
-| `sam3.1-INT8.safetensors` | 0.9976 / 0.9829 | 0.0015 / 0.0146 | 0.0088 | 0 | 0 | pass |
-| `sam3.1-FP8.safetensors` | 0.9971 / 0.9805 | 0.0023 / 0.0485 | 0.0059 | 0 | 0 | fail (max score delta) |
-
-Mask IoU is per prompt. For a prompt with a count where the last kept instance and the next one score within 0.02 of each other (`orange:5` among some thirty oranges), which of the near-equal instances is kept is a coin flip at any precision, the fp32 conversion included, so those prompts compare the masks of every instance above the threshold.
-
-Which layers stay BF16 came from a sensitivity scan: one group of layers quantized at a time, the rest BF16, same prompts. The first ViT blocks moved the masks the most for every scheme, the text encoder and the last ViT blocks the least, and the 256-wide DETR and mask-decoder layers moved them as much as half the ViT for 4% of the bytes. Each file keeps the fewest early ViT blocks in BF16 that brought it inside the tolerance (FP8: the closest it came).
-
-On one B200 shared with other jobs (median of five calls, five concepts per call): BF16 1.62 GB of weights on the GPU, 1.82 GB peak, 0.35 s per call; INT8-ConvRot 1.02 GB, 1.29 GB peak, 0.77 s; INT8 0.96 GB, 1.24 GB peak, 0.76 s; FP8 1.02 GB, 1.38 GB peak, 0.52 s. The quantized files trade speed for memory and download size: without a compiled graph, quantizing every activation costs more than the 8-bit matmuls save at this model size.
-
-### BF16 model
-
-15 text prompts over two test images, BF16 on an NVIDIA GPU, score threshold 0.5:
-
-| Comparison | Min mask IoU | Mean mask IoU | Max score delta | Kept-count mismatches |
-|---|---|---|---|---|
-| This BF16 model vs the fp32 conversion run in fp32 | 0.9975 | 0.9991 | 0.0044 | 0 |
-| This BF16 model vs `facebook/sam3` in BF16 | 0.9928 | 0.9975 | 0.0074 | 0 |
-
-About 2.1 GB of GPU memory at peak for one 1008x1008 image; the weights take 1.6 GB.
-
-## How it was made
-
-- The `detector.*` weights of `sam3.1_multiplex.pt` were renamed with transformers' own SAM 3 conversion script (`convert_sam3_to_hf.py`: its key mapping, QKV split, position-embedding and text-projection handling). The same procedure applied to `facebook/sam3`'s `sam3.pt` reproduces the published `facebook/sam3` `model.safetensors` exactly (all 1468 tensors, max abs difference 0).
-- The video tracker and its neck convolutions are not included; this is the image model only.
-- SAM 3.1 has no weights for the fourth FPN level of the vision neck (`vision_encoder.neck.fpn_layers.3`, 4 small tensors). They are kept from `facebook/sam3`. The detector does not read that level: zeroing it leaves every mask and score bit-identical.
-- The fp32 conversion was cast to BF16. The tokenizer files are the unchanged `facebook/sam3.1` files.
-- The quantized files were made from `model.safetensors` with the same torchao configuration Unsloth Studio uses for its pre-quantized image and video models (FP8: per-row weight scale with dynamic per-token activation; INT8: per-channel weight with dynamic per-token activation; ConvRot: the weights of every quantized Linear whose input width 256 divides are rotated by a block Hadamard matrix before quantization, 265 of 289). Linears narrower than 512 stay BF16. Loading a file reproduces the in-memory quantization of the BF16 weights bit for bit.
-
 ---
 
 # SAM 3.1