Set merged 12-Oct-2026 Humaneness Voice Small as default

README.md

@@ -19,15 +19,23 @@ model-index:
 
 # Humaneness Voice Small
 
-Humaneness Voice Small is an experimental English/German text-to-speech and voice-acting model. This repository contains **ten stage-local checkpoints from one continuous S1→S10 training run** plus a later S3-based full-parameter, long-reference continuation. It also contains training and inference source, run statistics, and a guide to the exact prompt surfaces. It is not a standard `AutoModel.from_pretrained()` checkpoint: it uses a Qwen3-0.6B semantic backbone, a MOSS-style local Talker, and the separately loaded [MOSS Audio Tokenizer v2](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-v2). Read the inference instructions before loading weights.
+## Current default: Humaneness Voice Small (12-10-26)
 
-## Current default: S3 Ref15 Full FT (9 October 2026)
+**[Humaneness Voice Small (12-10-26)](checkpoints/HVS-Small-2026-10-12/model_bf16.pt)** is the current, complete BF16 checkpoint. It is the Real+Gemini full-parameter model after two more Gemini acting epochs with a Rank-64 semantic attention LoRA, **merged into the base weights**. You do not need to load an adapter. The Talker weights come from Real+Gemini and were not changed by this final LoRA. `python code/infer.py --stage default` loads this checkpoint; `--stage S3-Ref15-FullFT` still selects the preceding default. All original S1–S10 and Ref15 checkpoints remain available.
 
-The new **reference-conditioned default checkpoint** is [`checkpoints/S3-Ref15-FullFT/model_bf16.pt`](checkpoints/S3-Ref15-FullFT/model_bf16.pt), the final **step 26,774** of a full-parameter continuation from the **original S3 step 32,337**. The user chose it because it sounded acoustically preferable in listening. The updated `code/infer.py` loads it when `--stage` is omitted or set to `default`. The ten original stage checkpoints are unchanged; `--stage S3` explicitly loads the original S3 and is still the comparison to try for **reference-free** generation. For the new default, supply a clean, distinct 5–15-second reference WAV; every accepted retraining example had a reference.
+The user selected this default by listening, particularly for natural/organic acting. Audition the [matched five-checkpoint acting grid](https://laion-humaneness-voice-small-four-checkpoint-acting.static.hf.space/) with seven emotions, five voices, 12-second references and no-reference cases. The grid audio was generated with the unmerged LoRA; the merged export is checked for structural compatibility and should be numerically close, but no blinded perceptual or universal metric win is claimed. For lineage, exact training recipe, conditioning and caveats, see the [12 October release note](docs/HVS_SMALL_2026_10_12.md).
+
+For speaker identity, start with a **distinct, clean 5–15-second reference**. The acting continuation also saw pseudonym-conditioned examples; use its companion acting inference path for pseudonym-only prompts. The older `code/infer.py` remains the general single-clip example and now resolves `default` to the merged weight file. The historical prompt and benchmark sections below describe earlier S1–S10 experiments and should not be mistaken for a fresh evaluation of this new default.
+
+Humaneness Voice Small is an experimental English/German text-to-speech and voice-acting model. This repository contains **ten stage-local checkpoints from one continuous S1→S10 training run** plus an S3-based long-reference continuation, Gemini/real-speech acting continuations, and the merged 12 October default. It also contains training and inference source, run statistics, and a guide to the exact prompt surfaces. It is not a standard `AutoModel.from_pretrained()` checkpoint: it uses a Qwen3-0.6B semantic backbone, a MOSS-style local Talker, and the separately loaded [MOSS Audio Tokenizer v2](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-v2). Read the inference instructions before loading weights.
+
+## Previous default: S3 Ref15 Full FT (9 October 2026)
+
+The previous **reference-conditioned default checkpoint** is [`checkpoints/S3-Ref15-FullFT/model_bf16.pt`](checkpoints/S3-Ref15-FullFT/model_bf16.pt), the final **step 26,774** of a full-parameter continuation from the **original S3 step 32,337**. The user chose it because it sounded acoustically preferable in listening. Use `code/infer.py --stage S3-Ref15-FullFT` to load it explicitly. The ten original stage checkpoints are unchanged; `--stage S3` explicitly loads the original S3 and is still the comparison to try for **reference-free** generation. For this Ref15 checkpoint, supply a clean, distinct 5–15-second reference WAV; every accepted retraining example had a reference.
 
 The key fix is the historical **2.96-second reference crop**: the original training loader and earlier example inference code used at most 37 × 80-ms codec frames even though the stored reference sequences were longer. This continuation used up to **187 frames / 14.96 seconds**, with a 55/20/20/5 family of script, historical structured, caption-transcript, and transcript-only prompts. It fully updated the active semantic backbone and Talker after a conservative target-text repair gate; it is **not** the Rank-128 LoRA and did not start from that adapter. There were **3,255,217 accepted training targets**, each presented once with its distinct reference. Read the detailed [data selection, repair, training and evaluation report](docs/S3_REF15_FULL_FT.md) and the [checkpoint metadata](checkpoints/S3-Ref15-FullFT/metadata.json) before reusing it.
 
-This is a **listening-selected acoustic default**, not an objectively established all-round winner. In a matched 50-case non-speaker-disjoint test, final-step WER was 0.096 versus 0.088 for original S3; Orange cosine to the supplied reference was 0.707 versus 0.745, while cosine to the intended target improved from 0.658 to 0.734. Step 25,000 had better measured WER and target cosine than the final step. These learned/ASR proxies and the user's listening impression answer different questions; further blinded and speaker-disjoint evaluation is needed. No optimizer state is uploaded for this continuation.
+This was a **listening-selected acoustic default**, not an objectively established all-round winner. In a matched 50-case non-speaker-disjoint test, final-step WER was 0.096 versus 0.088 for original S3; Orange cosine to the supplied reference was 0.707 versus 0.745, while cosine to the intended target improved from 0.658 to 0.734. Step 25,000 had better measured WER and target cosine than the final step. These learned/ASR proxies and the user's listening impression answer different questions; further blinded and speaker-disjoint evaluation is needed. No optimizer state is uploaded for this continuation.
 
 The semantic backbone was **initialized from pretrained Qwen3-0.6B**, not from random weights. The **112,764,928-parameter Talker and bridge were initialized fresh** for this run. This corrects the loose description “trained from scratch” sometimes applied to earlier experiments. The Qwen hidden width is 1,024 and the local Talker width is 2,560. One semantic frame spans 80 ms; the Talker predicts **12 RVQ codebook indices per frame**, or 150 codebook indices per second. It does not predict 32 codebooks per frame. The codec reconstructs 48-kHz audio. The full state is approximately 715 million parameters, including the frozen, unused score conditioner; the model-only exports are BF16 PyTorch state dictionaries.
 
@@ -48,7 +56,7 @@ The `checkpoints/S*/model_bf16.pt` files are the **lowest independent held-out v
 | S9 | tier1 | 1,439 | 41,878 | 4.7198 | 0.4540 |
 | S10 | tier0 | 578 | 41,936 | 4.7278 | 0.4572 |
 
-¹ These are **training presentations**, not unique source-audio hours. Multi-format prompts repeat some recordings. The ladder tiers are nested and paired with sidecar material; do not add these hours to estimate unique audio. ² Target-aware automatic reward on 1,651 shared valid acting cells per stage, not a human MOS. The S3–S10 paired difference is +0.00557 with a 95% challenge-bootstrap CI of −0.00701 to +0.01899: **the benchmark does not establish S3 as significantly better than S10**. Among the **original ladder stages**, S3 was a reasonable provisional choice; inspect the same prompts in S3/S10 before choosing for a production voice. The later Ref15 Full FT is now the reference-conditioned default by listening choice, under the caveats above. S4 has the best held-out validation loss. All underlying values and intervals are in `stats/acting_findings.json` and `stats/acting_analysis.json.gz`.
+¹ These are **training presentations**, not unique source-audio hours. Multi-format prompts repeat some recordings. The ladder tiers are nested and paired with sidecar material; do not add these hours to estimate unique audio. ² Target-aware automatic reward on 1,651 shared valid acting cells per stage, not a human MOS. The S3–S10 paired difference is +0.00557 with a 95% challenge-bootstrap CI of −0.00701 to +0.01899: **the benchmark does not establish S3 as significantly better than S10**. Among the **original ladder stages**, S3 was a reasonable provisional choice; inspect the same prompts in S3/S10 before choosing for a production voice. The later Ref15 Full FT was the previous reference-conditioned default by listening choice, under the caveats above. S4 has the best held-out validation loss. All underlying values and intervals are in `stats/acting_findings.json` and `stats/acting_analysis.json.gz`.
 
 The run used 32 JUPITER Booster nodes × 4 GPUs/node, batch 4,096, **one** 2,097-step linear warmup followed by one cosine decay across all 41,936 updates—**no stage LR restarts**. Peak LRs were 8e-5 for the semantic backbone and 2.4e-4 for the Talker, ending at 5% of peak. The source `code/plan_caption_production_32nodes.json` records the exact historical manifests and hashes; its absolute JUPITER paths are provenance, not portable defaults.
 
@@ -139,7 +147,7 @@ Throughput was measured on an **RTX 3090**, using S3 at revision `5de86032771c37
 - `code/continuous_train.py`, `code/train.sbatch`, `code/plan_caption_production_32nodes.json` and adjacent modules: the historical JUPITER source and training contract. The raw plan refers to local manifests and intentionally fails its hash checks if edited; regenerate plans and manifests for new data. `code/infer.py` is the standalone example for the exported weights. The code bundles MOSS export modules, tokenizer configuration and score schema; the audio codec is fetched separately from its upstream Hugging Face repository.
 - `stats/training_steps.jsonl`: every logged training-step scalar line across all four Slurm invocations. `stats/invocation_summaries.json`, `stats/run_configuration.json`, `stats/stage_winners.json`, `stats/validation_loss_full.tar.gz`, `stats/acting_findings.json`, and `stats/acting_analysis.json.gz` hold the remaining scalar summaries, all held-out checkpoint-sweep records, and the public acting aggregate. `stats/CONTENTS.json` gives hashes.
 
-Example using the **new reference-conditioned default** (requires a CUDA environment with PyTorch, Transformers, NumPy, SoundFile, Torchaudio, Safetensors, and permission to download the separate codec):
+Example using the **current merged default with a reference** (requires a CUDA environment with PyTorch, Transformers, NumPy, SoundFile, Torchaudio, Safetensors, and permission to download the separate codec):
 
 ```bash
 python code/infer.py --stage default --reference-wav reference.wav --language en --frames 75 \
@@ -147,7 +155,7 @@ python code/infer.py --stage default --reference-wav reference.wav --language en
   --text 'I cannot believe we made it.' --output example.wav
 ```
 
-Do not use the target recording as its own reference. For reference-free inference, compare `--stage S3`; the new default was trained only with references. **Correction (8 October 2026):** an earlier public `code/infer.py` rejected any reference over 37 MOSS codec frames (2.96 s). This was an inference-code limit as well as the historical S1–S10 training-loader crop, not an architectural limit. The updated example accepts a configurable reference ceiling (15 s default, 20 s maximum), reports a warning below 5 s and no longer silently blocks longer audio. The new Full FT was trained with up to 14.96 s; using longer than 15 s at inference is out of distribution and warned about. The original S1–S10 weights were trained with at most 2.96 s references, so longer-reference behavior from an original stage is experimental, not a guaranteed improvement. The separate [S3 Ref15 Rank-128 LoRA](https://huggingface.co/laion/Humaneness-Voice-Small-Ref15-LoRA-r128) is a different, frozen-backbone comparison. The example is a source-level starting point; the public listening Space is the already-audited playback path. This repository does **not** contain the large original SFT-3 model weights or original Qwen checkpoint separately, because the exported Humaneness Voice Small states include every trained model weight required for inference.
+Do not use the target recording as its own reference. For reference-free inference, compare `--stage S3` as a historical control; the current merged default also saw pseudonym and no-reference acting conditions, while the preceding Ref15-only stage was trained with references. **Correction (8 October 2026):** an earlier public `code/infer.py` rejected any reference over 37 MOSS codec frames (2.96 s). This was an inference-code limit as well as the historical S1–S10 training-loader crop, not an architectural limit. The updated example accepts a configurable reference ceiling (15 s default, 20 s maximum), reports a warning below 5 s and no longer silently blocks longer audio. The new Full FT was trained with up to 14.96 s; using longer than 15 s at inference is out of distribution and warned about. The original S1–S10 weights were trained with at most 2.96 s references, so longer-reference behavior from an original stage is experimental, not a guaranteed improvement. The separate [S3 Ref15 Rank-128 LoRA](https://huggingface.co/laion/Humaneness-Voice-Small-Ref15-LoRA-r128) is a different, frozen-backbone comparison. The example is a source-level starting point; the public listening Space is the already-audited playback path. This repository does **not** contain the large original SFT-3 model weights or original Qwen checkpoint separately, because the exported Humaneness Voice Small states include every trained model weight required for inference.
 
 ## Training data, license and responsible use
 
@@ -167,7 +175,7 @@ Both collections use the S3 checkpoint at step 32,337; consult each adapter card
 <!-- hvs-small-dpo-default-study-2026-10-04 -->
 ## Default DPO-adapter recommendation (matched speaker/emotion study)
 
-For **the historical S3 DPO comparison, use the unadapted S3 checkpoint rather than activating a DPO LoRA by default**. This is an adapter-level recommendation, **not** a reversal of the newer Ref15 Full FT checkpoint default above; the DPO LoRAs were trained against S3 and should not be assumed compatible or beneficial on Ref15 Full FT. The historical S3 prompt/CFG recommendations above also refer to S3, not a completed Ref15 tuning sweep. The [matched generated-audio study
+For **the historical S3 DPO comparison, use the unadapted S3 checkpoint rather than activating a DPO LoRA by default**. This is an adapter-level recommendation, **not** a reversal of the newer 12 October merged checkpoint default above; the DPO LoRAs were trained against S3 and should not be assumed compatible or beneficial on Ref15 Full FT. The historical S3 prompt/CFG recommendations above also refer to S3, not a completed Ref15 tuning sweep. The [matched generated-audio study
 with references and listening examples](https://laion-humaneness-voice-small-acting-eval-space.static.hf.space/dpo_lora_speaker_emotion_2026-10-02.html) compared the same 12
 English/German acting challenges, prompts and seeds for each condition (432
 outputs per condition, including 198 reference-conditioned outputs). Its
@@ -197,7 +205,7 @@ exact prompts, paired audio players and methodological limits.
 
 **None for general use on the historical S3 base:** unadapted S3 achieved
 lower WER than any DPO LoRA in that matched study. The repository-wide
-reference-conditioned checkpoint default is now Ref15 Full FT, which was
+reference-conditioned checkpoint default was Ref15 Full FT before the 12 October release, which was
 evaluated under a different 50-case protocol; this DPO result does not rank it.
 If an application explicitly prioritizes similarity to a provided speaker
 reference, the **best-tested optional DPO LoRA at this date** is

checkpoints/HVS-Small-2026-10-12/metadata.json

@@ -0,0 +1,23 @@
+{
+  "title": "Humaneness Voice Small (12-10-26)",
+  "checkpoint": "HVS-Small-2026-10-12",
+  "format": "complete adapter-free BF16 PyTorch state_dict",
+  "base": "/e/scratch/laionize/schuhmann1/acting_challenges_finetune/small/real_gemini_training/checkpoints/step-00015238/model.safetensors",
+  "base_sha256": "b1ad2bcbe94a5760c37b4e57a752a894db0033adc28fc62cb885d532cf4e46f2",
+  "adapter": "/e/scratch/laionize/schuhmann1/acting_challenges_finetune/small/gemini_lora_r64_2ep_v1/training/adapter.safetensors",
+  "adapter_sha256": "3c9e01bc448b0732785240a64dd47f8ea7a7b4d09106b0d24a208e35677a85fe",
+  "model_file": "model_bf16.pt",
+  "model_bytes": 1431106131,
+  "model_sha256": "45f1de4bcc62bef038afc812ac8681b3e6507d9d97e4c18d27518f5a5a346bd4",
+  "rank": 64,
+  "alpha": 64,
+  "merged_modules": 112,
+  "target_modules": "Qwen3 semantic attention q_proj, k_proj, v_proj, o_proj",
+  "source_step": 15238,
+  "adapter_step": 2298,
+  "optimizer_state_included": false,
+  "merge_validation": "112 nonzero finite deltas; saved state strict-loads without adapter; prompt-only generation canary passed",
+  "generation_canary_frames": 24,
+  "delta_max_abs": 0.039547283202409744,
+  "lineage": "S3 Ref15 Full FT -> Gemini acting full FT -> Real+Gemini full FT -> two-epoch Gemini Rank-64 LoRA -> merged export"
+}

checkpoints/default.json

@@ -1,10 +1,13 @@
 {
-  "checkpoint": "S3-Ref15-FullFT",
-  "model_file": "checkpoints/S3-Ref15-FullFT/model_bf16.pt",
-  "metadata_file": "checkpoints/S3-Ref15-FullFT/metadata.json",
-  "chosen_on": "2026-10-09",
-  "selection_basis": "user listening preference for acoustic quality",
-  "reference_conditioned": true,
-  "trained_reference_max_seconds": 14.96,
+  "checkpoint": "HVS-Small-2026-10-12",
+  "display_name": "Humaneness Voice Small (12-10-26)",
+  "model_file": "checkpoints/HVS-Small-2026-10-12/model_bf16.pt",
+  "metadata_file": "checkpoints/HVS-Small-2026-10-12/metadata.json",
+  "chosen_on": "2026-10-12",
+  "selection_basis": "user listening preference on matched acting grid",
+  "merged_adapter": true,
+  "reference_or_alias_conditioning": true,
+  "reference_max_seconds_recommended": 15,
+  "previous_default": "S3-Ref15-FullFT",
   "reference_free_fallback": "S3"
 }

code/acting_2026_10_12/lora_layers.py

@@ -0,0 +1,49 @@
+"""Rank-configurable Qwen3 semantic q/k/v/o LoRA for Humaneness Voice Small.
+
+Training used rank=alpha=64 and adapted 28 × 4 linear projections. The model
+loader and data mappings remain described in the release note; this is only
+the adapter layer definition, not a standalone training pipeline.
+"""
+from __future__ import annotations
+
+import math
+import torch
+from torch import nn
+
+
+class LoRALinear(nn.Module):
+    def __init__(self, base: nn.Linear, rank: int = 64, alpha: int = 64):
+        super().__init__()
+        self.base = base
+        for parameter in self.base.parameters():
+            parameter.requires_grad_(False)
+        self.a = nn.Linear(base.in_features, rank, bias=False, dtype=torch.float32,
+                           device=base.weight.device)
+        self.b = nn.Linear(rank, base.out_features, bias=False, dtype=torch.float32,
+                           device=base.weight.device)
+        nn.init.kaiming_uniform_(self.a.weight, a=math.sqrt(5))
+        nn.init.zeros_(self.b.weight)
+        self.scale = alpha / rank
+        self.enabled = True
+
+    def forward(self, x):
+        result = self.base(x)
+        if self.enabled:
+            result = result + self.scale * self.b(self.a(x.float())).to(result.dtype)
+        return result
+
+
+def attach(model, rank: int = 64, alpha: int = 64):
+    suffixes = ('.self_attn.q_proj', '.self_attn.k_proj',
+                '.self_attn.v_proj', '.self_attn.o_proj')
+    wrapped = []
+    for name, module in list(model.named_modules()):
+        if not name.startswith('transformer.layers.') or not name.endswith(suffixes):
+            continue
+        assert isinstance(module, nn.Linear), name
+        parent, leaf = name.rsplit('.', 1)
+        adapted = LoRALinear(module, rank=rank, alpha=alpha)
+        setattr(model.get_submodule(parent), leaf, adapted)
+        wrapped.append((name, adapted))
+    assert len(wrapped) == 112, len(wrapped)
+    return wrapped

code/acting_2026_10_12/merge_lora.py

@@ -0,0 +1,135 @@
+#!/usr/bin/env python3
+"""Merge the finished Gemini Rank-64 semantic LoRA into Real+Gemini Small.
+
+Run on a JUPITER Booster node. The export is a complete BF16 PyTorch state
+dict in the same format as the original Humaneness Voice Small checkpoints.
+No optimizer state or source checkpoint is changed.
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+from pathlib import Path
+import sys
+
+import torch
+from safetensors.torch import load_file
+
+HOME = Path('/e/home/jusers/schuhmann1/jupiter')
+DATA = Path('/e/scratch/laionize/schuhmann1/acting_challenges_finetune')
+SOURCE = DATA / 'small/real_gemini_training/checkpoints/step-00015238/model.safetensors'
+ADAPTER = DATA / 'small/gemini_lora_r64_2ep_v1/training/adapter.safetensors'
+OUT = DATA / 'small/hf_release_2026_10_12/checkpoints/HVS-Small-2026-10-12'
+sys.path[:0] = [str(DATA), str(HOME / 'hvs_ref_only_r128_v1/public_release')]
+import train_fullft as ft
+from lora_layers import attach
+
+
+def sha(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open('rb') as stream:
+        for block in iter(lambda: stream.read(8 << 20), b''):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def main() -> None:
+    torch.set_num_threads(8)
+    assert torch.cuda.is_available(), 'Run merge on a GPU compute node'
+    assert SOURCE.is_file() and ADAPTER.is_file()
+    done = json.loads((ADAPTER.parent / 'DONE.json').read_text())
+    assert done['status'] == 'COMPLETE', done
+    OUT.mkdir(parents=True, exist_ok=True)
+    target = OUT / 'model_bf16.pt'
+    metadata = OUT / 'metadata.json'
+    if target.exists() or metadata.exists():
+        raise FileExistsError('Release export already exists; inspect it, do not overwrite it')
+
+    model, config = ft.load_model('small')
+    source = load_file(str(SOURCE), device='cpu')
+    assert source and all(name.startswith('base.') for name in source)
+    missing, unexpected = model.load_state_dict(
+        {name[5:]: value for name, value in source.items()}, strict=False)
+    tied = {'text_lm_head.weight'} | {f'audio_lm_heads.{i}.weight' for i in range(12)}
+    assert not unexpected and set(missing) <= tied, (missing, unexpected)
+    model.tie_weights()
+    del source
+    wrapped = attach(model, rank=64, alpha=64)
+    assert len(wrapped) == 112
+    adapter = load_file(str(ADAPTER), device='cpu')
+    expected = {name + suffix for name, _ in wrapped for suffix in ('.a.weight', '.b.weight')}
+    assert set(adapter) == expected, (len(adapter), len(expected))
+
+    changes = []
+    with torch.no_grad():
+        for name, module in wrapped:
+            a = adapter[name + '.a.weight'].to('cuda', dtype=torch.float32)
+            b = adapter[name + '.b.weight'].to('cuda', dtype=torch.float32)
+            old = module.base.weight.to('cuda', dtype=torch.float32)
+            delta = (b @ a) * module.scale
+            assert torch.isfinite(delta).all() and torch.count_nonzero(delta) > 0, name
+            merged = (old + delta).to('cpu', dtype=torch.bfloat16)
+            module.base.weight.copy_(merged.to(module.base.weight.dtype))
+            parent, leaf = name.rsplit('.', 1)
+            setattr(model.get_submodule(parent), leaf, module.base)
+            changes.append({'name': name, 'max_abs_delta': float(delta.abs().max().item())})
+            if len(changes) % 28 == 0:
+                print(json.dumps({'event': 'merged', 'modules': len(changes)}), flush=True)
+    assert len(changes) == 112
+    assert not any('.a.weight' in key or '.b.weight' in key for key in model.state_dict())
+    model = model.to('cpu', dtype=torch.bfloat16).eval()
+    state = model.state_dict()
+    assert state and all(value.dtype == torch.bfloat16 for value in state.values()
+                         if value.is_floating_point())
+    temp = target.with_suffix('.pt.partial')
+    torch.save(state, temp)
+    del state
+    # The saved artifact must load strictly as an ordinary, adapter-free model.
+    check = torch.load(temp, map_location='cpu', weights_only=True)
+    model.load_state_dict(check, strict=True)
+    del check
+    # Exact public-grid prompt path, without loading or writing loose audio.
+    from packing import generated_audio
+    processor, _ = ft.load_processor('small')
+    user = processor.build_user_message(
+        text='Hello there.',
+        instruction='GENERAL: warm, natural speech.\nSCRIPT:\n(warm, conversational) Hello there.',
+        tokens=24, reference=None, language='English')
+    ids = processor([[user]], mode='generation')['input_ids'][0].long().unsqueeze(0).to('cuda')
+    mask = torch.ones(ids.shape[:2], dtype=torch.long, device='cuda')
+    model = model.to('cuda', dtype=torch.bfloat16).eval()
+    torch.manual_seed(20261012)
+    with torch.inference_mode(), torch.autocast('cuda', dtype=torch.bfloat16):
+        generated = model.generate(
+            input_ids=ids, attention_mask=mask, max_new_frames=45,
+            do_sample=True, audio_temperature=1.0, audio_top_p=.95,
+            audio_top_k=50, audio_repetition_penalty=1.0, use_kv_cache=True)
+        codes = generated_audio(generated, config)
+    assert len(codes) >= 2 and codes.shape[-1] == 12, codes.shape
+    del generated
+    os.replace(temp, target)
+    record = {
+        'title': 'Humaneness Voice Small (12-10-26)',
+        'checkpoint': 'HVS-Small-2026-10-12',
+        'format': 'complete adapter-free BF16 PyTorch state_dict',
+        'base': str(SOURCE), 'base_sha256': sha(SOURCE),
+        'adapter': str(ADAPTER), 'adapter_sha256': sha(ADAPTER),
+        'model_file': target.name, 'model_bytes': target.stat().st_size,
+        'model_sha256': sha(target), 'rank': 64, 'alpha': 64,
+        'merged_modules': len(changes),
+        'target_modules': 'Qwen3 semantic attention q_proj, k_proj, v_proj, o_proj',
+        'source_step': 15238, 'adapter_step': 2298,
+        'optimizer_state_included': False,
+        'merge_validation': '112 nonzero finite deltas; saved state strict-loads without adapter; prompt-only generation canary passed',
+        'generation_canary_frames': len(codes),
+        'delta_max_abs': max(row['max_abs_delta'] for row in changes),
+        'lineage': 'S3 Ref15 Full FT -> Gemini acting full FT -> Real+Gemini full FT -> two-epoch Gemini Rank-64 LoRA -> merged export',
+    }
+    metadata.write_text(json.dumps(record, indent=2) + '\n')
+    print(json.dumps({'status': 'EXPORTED', 'output': str(target),
+                      'bytes': target.stat().st_size, 'sha256': record['model_sha256']}), flush=True)
+
+
+if __name__ == '__main__':
+    main()

code/acting_2026_10_12/train_lora.py

@@ -0,0 +1,244 @@
+#!/usr/bin/env python3
+"""Two Gemini-only epochs of q/k/v/o Rank-64 LoRA above Real+Gemini Small.
+
+Each target is presented once per epoch. Its conditioning alternates between
+same-voice reference audio and pseudonym, so both are seen exactly once over
+two epochs. The three frozen prompt views are balanced by UID; no loose audio
+files are produced. Training and resume checkpoints include optimizer/RNG.
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import math
+import os
+from pathlib import Path
+import signal
+import sys
+import time
+
+import numpy as np
+import torch
+from safetensors.torch import load_file, save_file
+
+HOME = Path('/e/home/jusers/schuhmann1/jupiter')
+DATA = Path('/e/scratch/laionize/schuhmann1/acting_challenges_finetune')
+BASE = DATA / 'small/real_gemini_training/checkpoints/step-00015238/model.safetensors'
+OUT = DATA / 'small/gemini_lora_r64_2ep_v1'
+sys.path[:0] = [str(DATA), str(HOME / 'hvs_ref_only_r128_v1/public_release')]
+import train_fullft as ft
+from acting_data import ActingData, pack_example
+from lora_layers import attach
+
+RANK = 64
+PEAK_LR = 1.0e-4
+SEED = 20261012
+ACCUM = 2
+STOP = False
+
+
+def _signal(_number, _frame):
+    global STOP
+    STOP = True
+
+
+def atomic_json(path, data):
+    temp = path.with_suffix(path.suffix + '.partial')
+    temp.write_text(json.dumps(data, indent=2, sort_keys=True) + '\n')
+    os.replace(temp, path)
+
+
+def load_base():
+    model, config = ft.load_model('small')
+    wrapped = load_file(str(BASE), device='cpu')
+    assert wrapped and all(k.startswith('base.') for k in wrapped)
+    state = {k[5:]: v for k, v in wrapped.items()}
+    missing, unexpected = model.load_state_dict(state, strict=False)
+    allowed = {'text_lm_head.weight'} | {f'audio_lm_heads.{i}.weight' for i in range(12)}
+    assert not unexpected and set(missing) <= allowed, (missing, unexpected)
+    model.tie_weights()
+    del wrapped, state
+    for p in model.parameters():
+        p.requires_grad_(False)
+    adapted = attach(model, rank=RANK, alpha=RANK)
+    assert len(adapted) == 112
+    # Some Gemini targets exceed 1,900 codec frames. Checkpoint semantic
+    # decoder activations as in the proven full-FT path; a frozen backbone
+    # still needs its input activations to compute LoRA gradients.
+    assert ft.enable_grad_ckpt(model) >= 20
+    return model, config
+
+
+def view(data, uid, epoch, index):
+    # Both reference and alias paths occur exactly once for every UID across
+    # the two epochs, while the 3 GENERAL/SCRIPT views stay balanced.
+    cond = 'reference' if (index + epoch) % 2 == 0 else 'alias'
+    prompt_epoch = 6 + epoch + sum(uid.encode()) % 3
+    return data.example_condition(uid, prompt_epoch, cond)
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--smoke-steps', type=int, default=0)
+    args = parser.parse_args()
+    assert BASE.is_file() and (BASE.parent / 'READY.json').is_file()
+    assert json.loads((DATA / 'PROMPTS_READY.json').read_text())['status'] == 'READY'
+    from accelerate import Accelerator, DistributedDataParallelKwargs
+    from accelerate.utils import set_seed
+    accelerator = Accelerator(gradient_accumulation_steps=ACCUM, mixed_precision='bf16',
+                              kwargs_handlers=[DistributedDataParallelKwargs(
+                                  find_unused_parameters=False, gradient_as_bucket_view=True)])
+    world, rank = accelerator.num_processes, accelerator.process_index
+    assert world == (4 if args.smoke_steps else 16), world
+    set_seed(SEED, device_specific=True)
+    torch.set_num_threads(4)
+    signal.signal(signal.SIGUSR1, _signal)
+    signal.signal(signal.SIGTERM, _signal)
+    data = ActingData('train')
+    validation = ActingData('validation')
+    proc, _ = ft.load_processor('small')
+    model, config = load_base()
+    parameters = [(n, p) for n, p in model.named_parameters() if p.requires_grad]
+    assert len(parameters) == 224 and all(n.endswith(('.a.weight', '.b.weight')) for n, _ in parameters)
+    optimizer = torch.optim.AdamW([p for _, p in parameters], lr=PEAK_LR,
+                                  betas=(.9, .95), weight_decay=0., foreach=False)
+    wrapper, optimizer = accelerator.prepare(ft.TrainStep(model), optimizer)
+    result = OUT / ('smoke' if args.smoke_steps else 'training')
+    result.mkdir(parents=True, exist_ok=True)
+    n = len(data.ids)
+    longest_index = max(range(n), key=lambda i: data.by_id[data.ids[i]][0][2])
+    steps_per_epoch = math.ceil(n / (world * ACCUM))
+    total = steps_per_epoch * 2
+    contract = {'kind': 'humaneness-small-real-gemini-plus-gemini-lora-r64-two-epochs-v1',
+                'base': str(BASE), 'base_bytes': BASE.stat().st_size,
+                'assignment_sha256': ft.file_sha(DATA / 'acting_challenges_three_view_v2.sqlite'),
+                'prompt_cache_sha256': ft.file_sha(DATA / 'prompt_cache.sqlite'),
+                'rank': RANK, 'alpha': RANK, 'target': 'semantic Qwen3 attention q,k,v,o only',
+                'adapted_linear_modules': 112, 'trainable_parameters': sum(p.numel() for _, p in parameters),
+                'dataset_rows_per_epoch': n, 'world': world, 'accum': ACCUM,
+                'steps_per_epoch': steps_per_epoch, 'total_steps': total,
+                'peak_lr': PEAK_LR, 'warmup_fraction': .05, 'floor_factor': .1,
+                'seed': SEED, 'conditioning': 'each UID once reference and once alias over two epochs',
+                'prompt_views': 'balanced rotating GENERAL/SCRIPT views and original 10% full caption dropout'}
+    digest = hashlib.sha256(json.dumps(contract, sort_keys=True).encode()).hexdigest()
+    if accelerator.is_main_process:
+        atomic_json(result / 'RUN_CONTRACT.json', {**contract, 'contract_sha256': digest})
+    accelerator.wait_for_everyone()
+    ckpts = sorted((result / 'checkpoints').glob('step-*/READY.json'))
+    step = 0
+    if ckpts:
+        latest = ckpts[-1]
+        meta = json.loads(latest.read_text())
+        assert meta['contract_sha256'] == digest
+        accelerator.load_state(str(latest.parent))
+        step = int(meta['step'])
+    if accelerator.is_main_process:
+        print(json.dumps({'event': 'start', 'world': world, 'step': step, 'total': total,
+                          'steps_per_epoch': steps_per_epoch, 'parameters': contract['trainable_parameters']}), flush=True)
+    begun = time.monotonic()
+    last_save = begun
+    orders = [np.random.default_rng(SEED + e).permutation(n) for e in range(2)]
+    warm = max(1, round(total * .05))
+
+    def save(current, reason):
+        path = result / 'checkpoints' / f'step-{current:08d}'
+        temp = result / 'checkpoints' / f'step-{current:08d}.partial'
+        conflict = torch.tensor([int(accelerator.is_main_process and (path.exists() or temp.exists()))],
+                                dtype=torch.int32, device=accelerator.device)
+        if world > 1:
+            torch.distributed.broadcast(conflict, src=0)
+        if conflict.item():
+            raise FileExistsError(path)
+        accelerator.save_state(str(temp))
+        accelerator.wait_for_everyone()
+        if accelerator.is_main_process:
+            atomic_json(temp / 'READY.json', {'step': current, 'reason': reason,
+                                             'contract_sha256': digest,
+                                             'optimizer_and_rng_saved': True})
+            os.replace(temp, path)
+            print(json.dumps({'event': 'checkpoint', 'step': current, 'path': str(path),
+                              'reason': reason}), flush=True)
+        accelerator.wait_for_everyone()
+
+    for current in range(step + 1, total + 1):
+        epoch, offset = divmod(current - 1, steps_per_epoch)
+        valid = min(world * ACCUM, n - offset * world * ACCUM)
+        assert valid > 0
+        if current <= warm:
+            factor = current / warm
+        else:
+            frac = min(1., (current - warm) / max(1, total - warm))
+            factor = .1 + .9 * .5 * (1 + math.cos(math.pi * frac))
+        optimizer.param_groups[0]['lr'] = PEAK_LR * factor
+        last_loss = None
+        for micro in range(ACCUM):
+            position = (offset * ACCUM + micro) * world + rank
+            active = position < n
+            index = int(orders[epoch][position]) if active else 0
+            if args.smoke_steps and current == 2 and micro == 0 and rank == 0:
+                index = longest_index  # Exercise worst-case attention memory.
+            uid = data.ids[index]
+            row = view(data, uid, epoch, index)
+            batch = {k: v.to(accelerator.device, non_blocking=True)
+                     for k, v in pack_example(proc, config, row).items()}
+            with accelerator.accumulate(wrapper):
+                loss = wrapper(batch['input_ids'], batch['attention_mask'], batch['labels'])
+                if not bool(torch.isfinite(loss).item()):
+                    raise FloatingPointError((current, uid, 'loss'))
+                accelerator.backward(loss * (world * ACCUM / valid if active else 0.))
+                if accelerator.sync_gradients:
+                    norm = accelerator.clip_grad_norm_(wrapper.parameters(), 1.)
+                    if not bool(torch.isfinite(norm).item()):
+                        raise FloatingPointError((current, uid, 'norm'))
+                optimizer.step()
+                optimizer.zero_grad(set_to_none=True)
+            last_loss = loss.detach().float()
+        if current <= 3 or current % 20 == 0 or current == total:
+            mean_loss = float(accelerator.gather(last_loss.reshape(1)).mean())
+            if accelerator.is_main_process:
+                event = {'event': 'step', 'step': current, 'total': total,
+                         'epoch': epoch+1, 'loss_last_micro': mean_loss,
+                         'lr': optimizer.param_groups[0]['lr'],
+                         'elapsed_s': round(time.monotonic()-begun, 1),
+                         'peak_gpu_gb': round(torch.cuda.max_memory_allocated()/1e9, 2)}
+                with (result / 'steps.jsonl').open('a') as log:
+                    log.write(json.dumps(event) + '\n')
+                print(json.dumps(event), flush=True)
+        names = [None, 'smoke', 'final', 'epoch', 'periodic', 'runtime', 'signal']
+        reason = ('smoke' if args.smoke_steps and current >= args.smoke_steps else
+                  'final' if current == total else 'epoch' if current % steps_per_epoch == 0 else
+                  'periodic' if time.monotonic()-last_save >= 1800 else
+                  'runtime' if time.monotonic()-begun >= 33000 else
+                  'signal' if STOP else None) if accelerator.is_main_process else None
+        code = torch.tensor([names.index(reason)], device=accelerator.device, dtype=torch.int32)
+        if world > 1:
+            torch.distributed.broadcast(code, src=0)
+        reason = names[int(code.item())]
+        if reason:
+            save(current, reason)
+            last_save = time.monotonic()
+        if args.smoke_steps and current >= args.smoke_steps:
+            return
+        if reason in ('runtime', 'signal'):
+            sys.exit(75)
+    accelerator.wait_for_everyone()
+    if accelerator.is_main_process:
+        bare = accelerator.unwrap_model(wrapper).base
+        adapter = {name.replace('._checkpoint_wrapped_module', ''):
+                   p.detach().float().cpu().contiguous()
+                   for name, p in bare.named_parameters() if p.requires_grad}
+        assert len(adapter) == len(parameters)
+        temp = result / 'adapter.safetensors.partial'
+        save_file(adapter, str(temp), metadata={'format': 'hvs-gemini-lora-r64-2ep-v1',
+                                               'contract_sha256': digest})
+        os.replace(temp, result / 'adapter.safetensors')
+        atomic_json(result / 'DONE.json', {'status': 'COMPLETE', 'step': total,
+                                          'adapter': str(result / 'adapter.safetensors'),
+                                          'contract_sha256': digest})
+        print(json.dumps({'event': 'done', 'step': total,
+                          'adapter': str(result / 'adapter.safetensors')}), flush=True)
+
+
+if __name__ == '__main__':
+    main()

code/infer.py

@@ -19,9 +19,9 @@ sys.path.insert(0, str(ROOT / 'code'))
 
 def main() -> None:
     parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument('--stage', choices=['default', 'S3-Ref15-FullFT'] +
+    parser.add_argument('--stage', choices=['default', 'HVS-Small-2026-10-12', 'S3-Ref15-FullFT'] +
                         [f'S{i}' for i in range(1, 11)], default='default',
-                        help='default = S3 Ref15 Full FT; S1–S10 retain the original ladder weights')
+                        help='default = Humaneness Voice Small (12-10-26); original stages remain available')
     parser.add_argument('--prompt', required=True, help='Literal GENERAL/SCRIPT, CAPTION/TRANSCRIPT or TRANSCRIPT text')
     parser.add_argument('--text', required=True, help='The exact spoken transcript')
     parser.add_argument('--frames', type=int, required=True, help='Frame budget at 12.5 frames/second')
@@ -36,7 +36,7 @@ def main() -> None:
     if not 0 < args.reference_max_seconds <= 20:
         parser.error('--reference-max-seconds must be in (0, 20]')
     if args.stage in ('default', 'S3-Ref15-FullFT') and args.reference_max_seconds > 15:
-        print('Warning: Ref15 Full FT trained with at most 14.96 s of reference; '
+        print('Warning: these continuations trained with at most about 15 s of reference; '
               'longer inference references are out of distribution', file=sys.stderr)
 
     import numpy as np
@@ -57,9 +57,9 @@ def main() -> None:
     moss_small.load_qwen_backbone = lambda model, log=print: None
     schema = json.loads((ROOT / 'assets/score_schema.json').read_text())
     model, config = build_fresh(schema, log=lambda _: None)
-    checkpoint_name = 'S3-Ref15-FullFT' if args.stage == 'default' else args.stage
+    checkpoint_name = 'HVS-Small-2026-10-12' if args.stage == 'default' else args.stage
     if checkpoint_name == 'S3-Ref15-FullFT' and args.reference_wav is None:
-        print('Warning: the Ref15 default was tuned only with reference audio; '
+        print('Warning: the Ref15 checkpoint was tuned only with reference audio; '
               'for reference-free inference compare --stage S3', file=sys.stderr)
     state = torch.load(ROOT / 'checkpoints' / checkpoint_name / 'model_bf16.pt',
                        map_location='cpu', weights_only=True)

docs/HVS_SMALL_2026_10_12.md

@@ -0,0 +1,19 @@
+# Humaneness Voice Small (12-10-26)
+
+This is the current, adapter-free Humaneness Voice Small default. The complete model state is `checkpoints/HVS-Small-2026-10-12/model_bf16.pt`; `checkpoints/default.json` and `code/infer.py` resolve `default` to it. The previous S3 Ref15 Full FT and the original ten ladder checkpoints are retained, not overwritten. This release is under the repository's CC BY 4.0 license; upstream codec and foundation-model dependencies retain their own terms.
+
+## What was trained
+
+The lineage is: original S3 ladder checkpoint → full-parameter Ref15 retraining with longer, distinct speaker references → three Gemini acting epochs → real-speech/Gemini full-parameter continuation → two Gemini-only epochs of a Rank-64 LoRA on the last continuation. The final LoRA trained **18,350,080 parameters** in **112 Qwen3 semantic q/k/v/o attention projections** (28 layers × 4 projections). The MOSS-style Talker was not updated by these final two LoRA epochs. The Real+Gemini parent is step 15,238; the LoRA completed step 2,298. Its training contract used 36,754 Gemini rows per epoch, a 5% warmup, peak learning rate 1e-4, and a cosine-like decay to 10% of peak. Every target appeared once per epoch; across two epochs it was conditioned once on another same-voice reference and once on a voice pseudonym. Three GENERAL/SCRIPT views rotated, with the existing 10% full-caption dropout. This checkpoint contains weights only, **not optimizer or RNG state**.
+
+The export added each LoRA delta to the Real+Gemini base projection in FP32, cast the result to BF16, removed all adapter modules, then saved a complete ordinary PyTorch state dict. All 112 deltas were finite and nonzero, and the saved export strictly loaded without an adapter. The SHA-256 and exact local source hashes are in `checkpoints/HVS-Small-2026-10-12/metadata.json`. The final BF16 rounding may differ slightly from running the adapter unmerged; the public grid was generated from the unmerged adapter at the same training step.
+
+## Listening and use
+
+The [public, matched acting grid](https://laion-humaneness-voice-small-four-checkpoint-acting.static.hf.space/) compares original S10, Ref15, Gemini acting, Real+Gemini and this final stage using the same rows, references, prompts, seeds and decoding settings. Its 105 conditions include 35 natural/spontaneous prompt cases and 70 original reference/no-reference cases; audio is on-demand MP3. It is a listening aid, not a blinded preference test. The checkpoint was chosen by user listening; no claim is made that it improves WER, speaker similarity or every emotion across a representative benchmark. Different weights can take different stochastic paths despite matched seeds.
+
+Use the model repository's `code/infer.py --stage default` for a reference-conditioned single-clip example. A clean, distinct 5–15 s reference is recommended; do not use the target audio as its own reference. The model was also trained with pseudonym conditioning, but the historical general inference example is not the exact alias-demo pathway. For the exact pseudonym/reference prompt construction used in the grid, inspect the linked demo's manifests and the acting-code release. A `GENERAL:` / `SCRIPT:` instruction with exact spoken `--text` is the principal acting format of the final two-epoch continuation. Longer references, unseen names and out-of-domain languages are not guaranteed.
+
+## Reproducibility and limits
+
+The LoRA training source and merge recipe are under `code/acting_2026_10_12/`. The exact original training depends on the separate Gemini acting dataset and JUPITER-side mappings described by the source; it is not reproduced from weights alone. The model remains experimental speech synthesis. Evaluate identity rights, consent, linguistic performance and potential deception before use. Do not impersonate people.