Jarvis Intent Router โ€” v1.0.2

Jarvis Intent Router (t5-small, int8 ONNX) converts a spoken command into a compact JSON list of intents. It is the fast, local router in Jarvis; the app executes only schema-valid plans after its normalizer and SafetyGuard.

This is a fine-tuned derivative of google-t5/t5-small, with a full-precision trainable checkpoint and an int8 quantized ONNX export of those fine-tuned weights. The model tree records the fine-tuning lineage; quantization describes the included deployment artifacts. Continued training used the Jarvis dataset through the project's RL pipeline (supervised continuation, not PPO/RLHF).

Repository: chitesh12545/jarvis-intent-t5-small
Dataset: chitesh12545/jarvis-intent-dataset
Visibility: Public
Release date: 2026-11-06

Selected release and broader evaluation

v1.0.2 selects Run 6. Against published v1.0.1, established exact parsing improves from 166/229 (72.5%) to 222/229 (96.9%), and the newer 550 comparison cases improve from 298/550 (54.2%) to 475/550 (86.4%). Shuffle is 12/12, volume 28/28, and all 33 actual tab/project commands pass. Native search changes from 16/16 to 15/16. These are offline intent-and-argument scores, not live app success rates. The owner accepted the remaining generalization limitations.

See full benchmark comparison and machine-readable results.

At a glance

Field Value
Base architecture t5-small encoder-decoder
Parameters 60.5M
Quantization int8 ONNX (encoder + merged decoder)
Model size 205.7 MB (202.5 MB ONNX)
Max source / target tokens 128 / 448
Intent schema version 5
Primary evaluation split test (45 examples)
Exact match 95.56%
Parse valid 100.00%
Latency p50 / p95 109.2 ms / 218.2 ms
Standalone RL rounds completed 0
License Apache License 2.0

What it does

Input is prefixed with parse: , for example:

parse: open chatgpt in brave and play lofi on spotify

Output is canonical JSON for the intent schema:

[{"name":"url.open_in","args":{"url":"https://chatgpt.com","browser":"Brave Browser"}},
 {"name":"media.play_playlist","args":{"name":"lofi"}}]

The model does not execute commands, call APIs, or answer questions. The app validates the plan against intent_schema.json, normalizes arguments, and applies SafetyGuard before execution.

Architecture

  • t5-small encoder-decoder, originally 60.5M parameters, continued from the existing Jarvis intent checkpoint (same architecture and vocabulary; the model does not grow between rounds).
  • int8 dynamic quantization of the encoder and merged decoder for ONNX Runtime.
  • Maximum source length 128; maximum target length
  • The served artifact is the quantized ONNX pair plus tokenizer/config files in this repository. model_meta.json records the schema and training provenance.

Training

Field Value
Started (UTC) 2026-11-06T08:55:20.590348+00:00
Training examples 4944
Dev examples 272
Dataset shards 6
Epochs 3
Batch / gradient accumulation 4 / 4
Learning rate 3e-05
Seed 13

The training set is hand-authored and deduplicated command/intent pairs across the app's capabilities. The model is fine-tuned from the previous intent checkpoint, not trained from scratch. Targets are canonicalized in schema order; conflicting labels are excluded. The linked dataset repository is the current corpus; the exact shard set used by this checkpoint is recorded in model_meta.json and hashed in metrics.json.

RL / reward-weighted post-training

  • Owner-declared standalone rounds completed: 0
  • Harvested pairs matching the recorded run hash: not verified for this run
  • Repeat factor in the current training run: 3

Run 6 is continued supervised fine-tuning on selected new synthetic shards with token-weighted intent loss. No harvested pairs, PPO or RLHF were used in this run.

This release is continued supervised fine-tuning in the projectโ€™s RL pipeline, not PPO or RLHF. Run 6 uses selected new synthetic shards, no harvested execution pairs, and a token-weighted intent loss. Six earlier continuation rounds include the preceding 070โ€“072 and 073 rounds and campaign runs 1โ€“4; Run 6 is the seventh stage in that lineage. Runs 5 and 7 were alternative experiments and are not ancestors of these weights.

Benchmarks

Split Examples Exact match Parse valid p50 p95
native_search 16 93.75% 100.00% 60.7 ms 68.1 ms
valid 45 95.56% 100.00% 83.0 ms 158.2 ms
test 45 95.56% 100.00% 109.2 ms 218.2 ms
novel 45 97.78% 100.00% 81.3 ms 222.0 ms
shuffle 12 100.00% 100.00% 60.6 ms 61.0 ms
volume 28 100.00% 100.00% 70.8 ms 129.4 ms
workspace 38 97.37% 100.00% 63.7 ms 118.9 ms

These are recorded historical results from the run's evaluation splits, not a fresh benchmark of the staged release. exact_match is the fraction of plans that match gold JSON exactly; parse_valid in this release requires full intent-schema validation. Latency covers model invocation, not speech recognition or application execution. These v1.0.2 measurements use the native Apple Silicon CPU and the actual ONNX serving decoder; timings are not a guarantee for another device. Sanitization can change published dataset rows, so these scores are not scores on the modified public corpus.

Previous version comparison

Metric Previous This release Delta
Exact match 93.33% 95.56% +2.22 pp
p50 latency 116.2 ms 109.2 ms -7.0 ms
p95 latency 229.9 ms 218.2 ms -11.7 ms

The baseline is the previous checkpoint recorded in the run's evaluation.json. A first Hugging Face release may have no earlier remote release; the local baseline is still shown when available. These deltas compare training checkpoints, not successive Hub packaging releases. Release v1.0.1 adds Apache-2.0 licensing and the trainable checkpoint to the same inference weights published in v1.0.0; it does not claim a new training run or accuracy improvement over v1.0.0.

Release gate

Status: OWNER ACCEPTED; strict generalization criteria not fully met

Gate Measured Requirement Result
promotion_passed True true PASS
test exact match 95.56% >= 90.00% PASS
parse valid 100.00% >= 99.00% PASS
p50 latency 109.2 ms <= 200.0 ms PASS
p95 latency 218.2 ms <= 250.0 ms PASS
exact match vs previous +2.22 pp >= +0.00 pp PASS
export/run provenance True export and selected run metadata must match PASS
recorded evaluation True export and selected run aggregates must match PASS
evaluation promotion True true PASS
finite metric ranges True finite rates in [0,1], nonnegative latency and positive counts PASS
general-command regression 130 vs 125 aggregate valid/test/novel must not regress PASS
valid regression 0.9555555555555556 each general suite must match or exceed baseline PASS
test regression 0.9555555555555556 each general suite must match or exceed baseline PASS
novel regression 0.9777777777777777 each general suite must match or exceed baseline PASS
native_search exact match 93.75% >= 90% when evaluated PASS
shuffle exact match 100.00% >= 90% when evaluated PASS
volume exact match 100.00% >= 90% when evaluated PASS
workspace exact match 97.37% >= 90% when evaluated PASS
inference artifacts True all required serving artifacts must exist PASS
generalization evidence integrity True unchanged audit reports and matching quantized weights PASS
generalization audit False passing frozen audits, unchanged evidence, and matching quantized weights FAIL
owner acceptance binding True explicit decision bound to version, weights, checkpoint and unchanged evidence PASS

The release gate preserves every strict check above. The owner explicitly accepted Run 6 for functional use despite unmet strict generalization criteria; that decision is bound to this release version, checkpoint, quantized weights and unchanged evidence. It does not waive artifact, provenance or privacy checks. metrics.json retains strict_passed: false and the accepted quality tradeoffs. Inspection flags cannot enable publication.

Usage

The app loads these files from backend/models/intent-t5-small/. For a direct ONNX Runtime smoke test, use the quantized encoder and merged decoder with the tokenizer files in this repository. The input contract is:

parse: <natural-language command>

Do not feed user text without the parse: prefix, and do not treat malformed JSON as executable.

The included intent_schema.json describes the current dataset schema (version 5). This checkpoint's recorded training schema is version 5; newer labels are not evidence it was retrained.

Continue fine-tuning

model.safetensors contains the matching full-precision fine-tuned checkpoint. Use it for further training; the quantized ONNX pair is intended for inference. The application downloads only inference assets and license files, so the training checkpoint does not increase its download size.

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

repo = "chitesh12545/jarvis-intent-t5-small"
tokenizer = AutoTokenizer.from_pretrained(repo, revision="v1.0.2")
model = AutoModelForSeq2SeqLM.from_pretrained(repo, revision="v1.0.2")
# Continue with Seq2SeqTrainer and your command/intent pairs.

Prefix inputs with parse: ; targets are canonical JSON arrays. Keep the source and target limits shown above, preserve brace tokens, and evaluate your own held-out data before deployment. This release does not include personal logs or optimizer state. Training under Apache-2.0 remains subject to its license terms.

Files

File Size
BENCHMARKS.md 5.6 KB
LICENSE 11.4 KB
NOTICE 0.5 KB
added_tokens.json 0.0 KB
benchmarks.json 8.2 KB
config.json 1.5 KB
decoder_model_merged_quantized.onnx 167.0 MB
encoder_model_quantized.onnx 35.5 MB
generation_config.json 0.1 KB
intent_schema.json 5.3 KB
metrics.json 40.9 KB
model.safetensors 242.0 MB
model_meta.json 2.1 KB
privacy_report.json 0.4 KB
special_tokens_map.json 2.5 KB
spiece.model 791.0 KB
tokenizer.json 2.4 MB
tokenizer_config.json 21.2 KB

Limitations

  • English-first command routing; performance on other languages is not claimed.
  • The model can be wrong on compound, ambiguous, or previously unseen commands; the app's fast rules, normalizer, LLM fallback, and SafetyGuard remain required.
  • Browser examples are intentionally scoped to Brave and Chrome in the current training shards.
  • Quantized latency varies with CPU/GPU, ONNX Runtime version, and load.
  • It is a router, not an assistant: it must not answer or execute the command.

License

The fine-tuned checkpoint, quantized ONNX exports, and accompanying release materials are licensed under Apache License 2.0. See LICENSE and NOTICE. The upstream google-t5/t5-small base is also Apache-2.0.

Privacy and data

The release pipeline redacts personal paths, volume names, email addresses, phone numbers, postal addresses, secrets, and machine/user identifiers before writing staging files. SentencePiece training-only paths and obsolete self-test metadata are removed; vocabulary, scores and runtime normalization are preserved. It scans the sanitized copies again and aborts on any residual high-risk match. Automatic pattern detection is not a proof of anonymity; staged text requires owner review. Binary artifacts are screened for local path markers and known private identifiers, but this does not establish that trained weights cannot memorize data. No raw router logs, audio, Keychain data, or private user records are included.

Category Redactions
tokenizer_training_metadata 17

Provenance

  • Release version: 1.0.2
  • Model repo: chitesh12545/jarvis-intent-t5-small
  • Dataset repo: chitesh12545/jarvis-intent-dataset
  • Schema: intent_schema.json version 5
  • Dataset manifest SHA-256: 71aa42e80bc1c8f9
  • Initial weights SHA-256: a42b7a85b4208504
  • Reward-pairs SHA-256: n/a
  • Prior releases remain available through their immutable Hugging Face version tags.

This card is generated by owner-local release tooling; metrics.json in this repository contains the machine-readable numbers.

Downloads last month
123
Safetensors
Model size
60.5M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for chitesh12545/jarvis-intent-t5-small

Finetuned
(2,345)
this model