Vinci maker’s mark and wordmark with Vero

Vinci-MLE-8B-1.0-GGUF

Vinci MLE 8B β€” machine-learning engineering research.

Open-weight ML-engineering research in GGUF: evidence-based diagnosis, bounded changes, and justified no-change or stop decisions.

Canadian-developed by SimpleDirect Β· Apache-2.0 Β· Research baseline

Available now: F16, Q8_0, Q5_K_M, and Q4_K_M. Each variant has recorded generation and tool-call-format smoke checks; see runtime validation for their scope.

This is the GGUF distribution of Vinci MLE 8B 1.0, an open-weight research model trained around evidence-based diagnosis, bounded workspace changes, and justified no-change or stop decisions. Its source model is a fine-tuned derivative of IBM Granite 4.1 8B.

Model at a glance

Property Value
Developer SimpleDirect, Canada
Source model simpledirect/Vinci-MLE-8B-1.0
Original parent ibm-granite/granite-4.1-8b
Distribution format GGUF
Available model files F16, Q8_0, Q5_K_M, Q4_K_M
Conversion runtime llama.cpp, revision 6f4f53f2
Validation scope CUDA full-offload runtime smoke; no minimum-memory certification
License Apache-2.0, with NOTICE

ML engineering specialization

The source model was trained on 91 trajectories across 64 task roots in eight ML-engineering families: data duplication and quality; experiment design and interpretation; failed resume; memory, batching and gradient accumulation; metric/evaluator defects; NaNs and divergence; packaging, serving and reproducibility; and tokenizer/model configuration.

Trained response Intended behavior
Act Make a bounded change when evidence supports a defect and intervention
No Change Preserve a correct setup and explain the evidence
Stop / Needs Evidence Identify missing evidence or a blocker instead of inventing a repair

These describe the source model's training scope, not measured success rates. Both source-model sizes consumed the same tokenized examples but used different training recipes. See the source card for the 8B-specific recipe, selective-supervision design, and development-selection limitations.

Files and choosing a variant

File Bytes GB GiB
Q4_K_M 5,347,914,336 5.35 4.98
Q5_K_M 6,253,884,000 6.25 5.82
Q8_0 9,345,610,336 9.35 8.70
F16 17,587,417,696 17.59 16.38

Sizes were read from the Hugging Face API on September 28, 2026 and matched to GGUF-RUNTIME.json. File size is not total RAM or VRAM use: allow additional memory for context, the KV cache, and runtime overhead.

Q4_K_M is the smallest download in this set; Q5_K_M and Q8_0 offer progressively larger representations. F16 is the floating-point GGUF conversion. Choose according to available memory and your own task validation; no quality ranking between these variants has been established here. The BF16 conversion master mentioned in the runtime record is an intermediate artifact, not an additional downloadable variant.

Quick start

Install huggingface_hub and transformers, and authenticate with hf auth login if repository access requires it. Use a working CUDA build of llama.cpp at the tested revision with llama-completion on your PATH and enough GPU memory for full offload.

The example downloads Q4_K_M at a fixed repository revision, checks its SHA-256, renders one of the smoke prompts using the source tokenizer's chat template, and invokes the same completion interface and decoding settings used in the recorded checks. It downloads only the selected GGUF variant; source tokenizer loading does not download the BF16 weight shards.

import hashlib
import subprocess

from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

model_path = hf_hub_download(
    repo_id="simpledirect/Vinci-MLE-8B-1.0-GGUF",
    filename="Vinci-MLE-8B-1.0-Q4_K_M.gguf",
    revision="4c8abbe3e2c45cae749d7b9a865ca86c0bdc98b4",
    local_dir="./vinci-mle-8b-gguf",
)
with open(model_path, "rb") as model_file:
    actual_sha256 = hashlib.file_digest(model_file, "sha256").hexdigest()
if actual_sha256 != "87313e7e349fa9e59a49cf73ec1e41b5764c1e3b5642ac0543229df460efc798":
    raise RuntimeError("GGUF checksum mismatch; download the pinned file again.")

tokenizer = AutoTokenizer.from_pretrained(
    "simpledirect/Vinci-MLE-8B-1.0",
    revision="75256cf293770b6439b5ca52b98d43aa7687294d",
    trust_remote_code=False,
)
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain in two sentences what a learning-rate warmup does."}],
    tokenize=False,
    add_generation_prompt=True,
)
subprocess.run(
    [
        "llama-completion", "-m", model_path, "-p", prompt,
        "-n", "48", "--temp", "0", "--top-k", "1", "--seed", "42",
        "-no-cnv", "--no-display-prompt", "-ngl", "99", "--no-mmap",
    ],
    check=True,
)

Use Python 3.11 or later for hashlib.file_digest. This is a reference integration of the recorded smoke procedure, not a new run on your hardware. The example requests full GPU offload; CPU-only use, partial offload, other backends, and graphical front ends were not validated by these checks. The source model's Transformers/vLLM weight-loading instructions apply to the BF16 repository, not these GGUF files.

For another variant, change the filename and expected hash together using SHA256SUMS. When verifying a complete snapshot, run sha256sum -c SHA256SUMS (macOS: shasum -a 256 -c SHA256SUMS) inside its directory. The single-file example above verifies only its selected GGUF.

Runtime validation

The uploaded runtime record reports the following for each distributed variant:

Variant Nonempty generation outputs Nonzero generation exit codes Detected NaN/error log lines Tool-call format
Q4_K_M 6/6 0 0 Parsed read_file call
Q5_K_M 6/6 0 0 Parsed read_file call
Q8_0 6/6 0 0 Parsed read_file call
F16 6/6 0 0 Parsed read_file call

The checks used six fixed prompts with a 48-token generation budget, plus one tool-format prompt with a 96-token budget per variant. Decoding was greedy (temperature 0, top-k 1, seed 42), with CUDA full offload and memory mapping disabled. Prompts were rendered with the staged source tokenizer; embedded chat templates matched the staged template in every distributed file.

These are runtime and format checks, not capability evaluations. The tool check parsed a JSON call and checked the name read_file; it did not validate the arguments, execute the tool, or establish reliable agent behavior. The short generation checks do not measure answer correctness, throughput, minimum memory, or long-context capability.

Additional conversion checks

The first eight re-tokenized output tokens agreed between Q8_0 and F16 on all six generation prompts. This narrow agreement does not establish complete-output equivalence or quality retention against BF16.

An independent 8B conversion produced byte-identical files for the distributed variants and the intermediate BF16 master. This checks conversion reproducibility, not deterministic generation across runtimes.

Evaluation and limitations

Vinci MLE 1.0 has no valid held-out ML-engineering task score. The source card reports broad parent-capability comparisons separately from MLE training scope, including unfavorable results and methodology limitations.

GGUF runtime smoke checks are reported above. Task-level capability retention against the BF16 source has not been measured for these variants. The BF16 card’s benchmark scores are not GGUF measurements; no benchmark parity or MLE-task improvement is claimed.

The source model's evaluation scope, limitations, and contamination audit remain relevant. Applications must implement and validate their own tools, sandbox, permissions, and workspace; model weights alone are not a complete Vinci MLE agent harness.

Source and conversion provenance

Record Value
BF16 source artifact reference 75256cf293770b6439b5ca52b98d43aa7687294d
GGUF artifact snapshot 4c8abbe3e2c45cae749d7b9a865ca86c0bdc98b4
Conversion tool llama.cpp 6f4f53f2b7da54fcdbbecaaa734337c337ad6176
F16 conversion Direct conversion from the staged BF16 Safetensors with convert_hf_to_gguf.py --outtype f16
Quantization input BF16 GGUF master converted from the staged Safetensors
Quantization variants llama-quantize with Q8_0, Q5_K_M, or Q4_K_M; no importance matrix supplied
Artifact checksums SHA256SUMS
Runtime checks GGUF-RUNTIME.json
Task-level comparison with BF16 Not measured

The conversion workflow checked the staged source files against the release artifact pins before conversion. The BF16 source reference identifies those model artifacts; the GGUF snapshot pins the uploaded conversions and their original runtime record. Later card-only changes do not alter those files. Source-model provenance is available in the BF16 repository’s PROVENANCE.json, also referenced by the accompanying source NOTICE.

License and attribution

The source model is licensed under Apache-2.0; its LICENSE and NOTICE are included here. IBM developed Granite; SimpleDirect developed the Vinci MLE fine-tune. No affiliation with or endorsement by IBM is implied.

BF16 source Β· MLE 30B GGUF Β· Vinci models

Downloads last month
110
GGUF
Model size
9B params
Architecture
granite
Hardware compatibility
Log In to add your hardware

4-bit

5-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for simpledirect/Vinci-MLE-8B-1.0-GGUF

Quantized
(1)
this model

Collection including simpledirect/Vinci-MLE-8B-1.0-GGUF