# Microduck: a reliable 1.5 m hill climb

Our selected policy climbs the Arena's **8° hill in 1.360 seconds, with 5/5 finishes**. The official Arena result exactly matches the local evaluator. It comes from our fastest sprint actor, adapted to the actual hill course with a fresh critic. Subsequent experiments are documented below; they have not replaced this selected policy.

[Watch the real MuJoCo replay](../media/hill-slope-runner.mp4) · [Replay metadata](../media/hill-slope-runner.json) · [Diagnostic report](../records/hill-climb-1m5/DIAGNOSIS.md)

![Actual MuJoCo hill climb, seed 3](../media/hill-slope-runner-poster.png)

The video shows **seed 3, whose timed finish is 1.34 s**. The five-seed mean is 1.360 s. It labels the 0.50 s untimed settle, plays the actual race at native speed, then replays that same trajectory at half speed. It is 1280×720, 16:9, 50 fps, 7.48 s long including annotated holds. The final public clip restores decorative meshes in a separate visual-only model, replaying the same exact Arena states on CPU. The recorded timed and settle trajectories are byte-for-byte equal to the official `run_once` trace. This is simulation footage, not generated imagery or a hardware demonstration.

## What the Arena measures

The installed `arena/events/hill-climb-1m5.toml` defines a 1.5 m lane, measured **along** a smooth 8° slope. The lane is 0.8 m wide, with 0.15 m walls. The plane and walls rotate by pitch −8°, so the forward/uphill direction is `[cos(8°), 0, sin(8°)]`. Progress is the dot product of trunk position with that direction, relative to the position **after** settling.

The robot begins in the Arena's `STAND` pose, upright in world coordinates. Its initial position is the plane normal times 0.12 m: `[-0.0167007721, 0, 0.1188321682]`. Joint positions vary by ±0.01 rad and heading by ±2°. The policy runs for 0.50 s before timing begins; this is an available run-up, not a passive pause.

MuJoCo uses a 0.005 s physics timestep, four substeps per control step, 50 Hz policy decisions, the full-collision legs rig, BAM XL330 torque actuation, implicitfast integration, and a 10/20 contact solver. Supply voltage is nominal 7.35 V and voltage-drop gain 0.1. Command delay remains uniformly sampled from 3–6 physics updates; other training randomizations take nominal values in the Arena.

The policy receives 61 observations: body gyro (3), projected gravity (3), joint positions relative to default pose (14), joint velocities one control step old (14), previous action (14), and commands (13). It returns 14 position-target actions. The Arena provides straight forward speed, zero lateral/yaw commands, and zero head/body commands; it does not provide lane position, heading, or slope directly.

Every speed from 0.15 to 1.00 m/s in 0.05 increments is tried on seed 1. Three finalists are evaluated on seeds 1–5. The score is their best mean finish time. A fall or timeout contributes **60 s**, making reliability decisive. A fall means projected gravity z exceeds −0.5 for 15 consecutive timed control steps (0.30 s). Fall takes precedence over finish when both occur together.

## Transfer baselines and licenses

| Policy, scored unchanged on the hill | Score | Finishes | Selected Arena vx |
|---|---:|---:|---:|
| Our sprint D1 launchboost, checkpoint 900 | 48.256 s | 1/5 | 0.15 |
| Original Hannes running policy | 12.964 s | 4/5 | 0.65 |
| Pollen hill reference | 5.880 s | 5/5 | 0.55 |
| Selected hill H1b, checkpoint 500 | **1.360 s** | **5/5** | 0.15 |

The original actor is [HannesVonEssen/microduck-running](https://huggingface.co/HannesVonEssen/microduck-running/tree/9f45af1f4b04252f3aa1f4c056fa7fa0d4b35e4c), pinned to revision `9f45af1f4b04252f3aa1f4c056fa7fa0d4b35e4c`. Its model card explicitly declares **Apache-2.0**, verified and retained in `logs/hannes_license/README.md`. Its SHA256 is `007707dd7779b2756ded67c58b2e9f94fe5071794a48c2b5a20d5f8d841efbeb`.

Pollen's downloadable hill reference card has no explicit model license in the inspected card. We used it only as a benchmark and did not reuse its weights. Public availability alone does not establish a reuse license. All selected-policy actor weights descend from the explicitly licensed Hannes source.

The selected ONNX embeds **raw actor vx=2.6**, with lateral/yaw commands forced to zero, before its unchanged observation normalizer. Consequently the Arena's reported 0.15 m/s sweep value does not mean the gait was trained to run at 0.15 m/s; the exported wrapper intentionally ignores that external forward command. The original Hannes actor has no such override and genuinely responds to its incoming speed command.

## Selected checkpoint and official result

The immutable selected artifacts are:

- `best/hill1_h1b_actorhill/model_500.onnx`
- `best/hill1_h1b_actorhill/model_500.pt`
- Convenience aliases `policy.onnx` and `checkpoint.pt` in that directory.
- `best/hill1_h1b_actorhill/selection.json` contains the exact local record and training parent.

ONNX SHA256: `f52e49b6f3fa1d9122811274e9bf490e2ca0cbf54ca47282fbebe28743b429fc`.

Full PPO checkpoint SHA256: `ac17eb2a1c360adfe7798244a1c345d37decab79f2884fc5c8adb36d9be07258`.

| Seed | Timed finish |
|---|---:|
| 1 | 1.40 s |
| 2 | 1.44 s |
| 3 | 1.34 s |
| 4 | 1.32 s |
| 5 | 1.30 s |
| Mean | **1.360 s** |

Root published the immutable checkpoint as [Kyry10/microduck-hill-climb-1m5-v1](https://huggingface.co/Kyry10/microduck-hill-climb-1m5-v1/tree/c2c850352701f33dd3c82c0a9c61e9d69b46d72b), revision `c2c850352701f33dd3c82c0a9c61e9d69b46d72b`. Official run `01M4TE87G4C9VY63V1JA6C` scored **1.360 s, 5/5**, matching local exactly. The old `v1` name is an existing entry; subsequent public names should describe the event/gait rather than use version suffixes. Only root handles publication.

## Training lineage and exact H1b recipe

The actor lineage is:

`Apache-2.0 Hannes running → sprint race adaptation → B3 full-PPO polish → D1 full-PPO launchboost → H1b hill actor adaptation`.

D1 checkpoint 900 was our 1.084 s, 5/5 sprint best when hill adaptation began. Hill H1b loaded that actor and its **exact frozen normalizer**, then created a fresh hill critic and empty optimizer. It did not resume the flat sprint critic. H1b ran 512 environments, PPO learning rate `1e-5`, exploration std initialized at `0.1`, forward-velocity reward weight 5, action-rate penalty −0.01, a finish bonus 10, fall cost −20, weak lateral and yaw-rate penalties, and checkpoint scoring every 100 updates. Head/body commands remained zero.

The first H1 run exposed contact-buffer overflow inherited from the sprint allocation. H1b raised allocation to 128 contacts and 1024 constraints per world, without changing physical geometry. Its training commit is **`242f7d43588d1a59d5e575e92c5ca3c6b8d2aab1`**, branch `kyry-hill-climb-1m5`. The selected checkpoint is iteration 500; later 600/700/800 each finished only 2/5, triggering the three-low rule. The last checkpoint was not the best.

This is the exact historical training command, run from the isolated event challenge clone at that commit:

```bash
export PYTHONPATH="$PWD/src"
export CUDA_VISIBLE_DEVICES=0
export XDG_CACHE_HOME=$MICRODUCK_ROOT/events/hill-climb-1m5/cache
export HF_HOME="$XDG_CACHE_HOME/hf"
export WARP_CACHE_PATH="$XDG_CACHE_HOME/warp"
export OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=1

.venv/bin/python scripts/finetune_sprint.py \
  --run-name h1b_actorhill \
  --parent-checkpoint $MICRODUCK_ROOT/events/sprint-2m/best/autonomous_d1_launchboost/checkpoint.pt \
  --num-envs 512 --iterations 1001 --save-interval 100 \
  --stage hill1 --target-score 0.90
```

At this commit, raw vx2.6, LR1e-5 and std0.1 are driver defaults. No `--full-resume` is supplied: actor-only loading is deliberate for the slope transfer. The original command is retained in `h1.sh`; each run also retains `command.json`, `provenance.json`, `parent.txt`, exact environment/agent YAML, TensorBoard events, checkpoints, exported ONNX and per-checkpoint scoring argv/log/records. These live in `microduck-challenges/logs/rsl_rl/sprint_2m_fast/h1b_actorhill/`.

The shared environment is read-only. Do not run `uv sync`, install packages into the shared venv, edit another event, or allow editable imports to resolve to the sprint clone. Always set `PYTHONPATH` to this clone's `src` and give the event its own caches. GPU retraining varies across machines and runs; reproducing an artifact's score is distinct from reproducing its training path bit-for-bit.

## Local scoring and source download commands

Every hill/roller CPU scorer uses the shared lock to maintain the two-process CPU cap:

```bash
cd $MICRODUCK_ROOT/events/hill-climb-1m5/microduck-challenges
export PYTHONPATH="$PWD/src" CUDA_VISIBLE_DEVICES=''
export OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1

flock $MICRODUCK_ROOT/resources/agent-scorer.lock \
  .venv/bin/arena score ../best/hill1_h1b_actorhill/model_500.onnx \
  --event hill-climb-1m5 --store ../logs/reproduce_best500

.venv/bin/hf download HannesVonEssen/microduck-running README.md \
  --revision 9f45af1f4b04252f3aa1f4c056fa7fa0d4b35e4c \
  --local-dir ../logs/hannes_license

.venv/bin/hf download pollen-robotics/microduck-hill-climb-1m5-reference \
  policy.onnx README.md --local-dir ../logs/reference
```

The full baseline commands are retained in `baseline.sh`. Its first score is D1 sprint, followed by the unchanged original Hannes actor, then Pollen hill reference. Each result has its full sweep and five-seed record in `logs/baseline_*`.

## What failed, and what the traces show

**H2 lower-LR polish** resumed H1b500 with its critic/Adam state, lowered LR to `3e-6`, froze inherited exploration std, and scored every 50. It did not improve the selected policy. Checkpoints 550/600/650 each finished 1/5, so the run stopped at 650.

**H3 original Hannes transfer** used the original licensed actor, frozen normalizer, fresh hill critic/optimizer, and raw vx0.65—the best command in its hill baseline. CPU checks verified the source had no command override, actor conversion parity, unchanged normalization denominator, and exact command-wrapper parity. H3 reached 4/5 at 200 but degraded: 400 finished 2/5, 500 and 600 finished 1/5. It stopped at 600.

The H3 checkpoint 600 diagnosis is particularly useful. Only seed 5 finished, in **1.14 s**. Seeds1/2 hit walls and fell at1.38/1.46 s. Seeds3/4 spent537/270 control steps contacting a wall before falling. All settle phases stayed below14° tilt. These failures were chiefly associated with lateral drift and wall stalls, not a fall during the run-up.

For reliable H1b500, settle-end uphill speed is1.028–1.215 m/s; peak100 ms mean speed is1.199–1.386 m/s. Timed tilt stays14.4–29.1°, comfortably below the fall threshold. Four seeds never contact a wall; seed 5 contacts a wall for only two control steps and finishes1.30 s. The best gait mainly needs more uphill speed, but its lateral margin is thin (maximum excursion0.366 m).

Trace commands are in `diagnose.sh`; raw NPZ traces and JSON summaries are in `recon/h1b500/` and `recon/h3_600/`. No actor input, control target or simulation state was altered for diagnosis.

## Arena alignment audit and the fall-clock repair

Compile-only CPU comparisons confirm slope sign/quaternion, start position, wall positions/sizes, nominal floor/foot/wall friction, physics timestep, solver10/20 and servo delay3–6. The policy uses gyro/gravity in the body frame, a one-control-step joint-velocity delay, no observation noise, and the original frozen normalizer. H3 raw0.65 is passed to the normalizer, not confused with normalized0.65: its source forward-command mean is1.1553453 and denominator0.59477854.

The audit found a real termination mismatch. Earlier training counted the fall-hold counter **during the untimed settle**; Arena starts that counter after settling. The corrected training condition is:

```python
timed = env.episode_length_buf > 25
tilted = (projected_gravity_z > -0.5) & timed
counter = torch.where(tilted, counter + 1, 0)
```

CPU boundary checks establish zero through step25, count1 at26, count14/no fall at39, and count15/fall at40. Repeated evaluation within a control step does not double-count; an upright step resets the counter. The finish origin is correctly captured after step25.

There is also a smaller structural actuation difference: narrowing training voltage range to nominal7.35 V reduces its compiled force ceiling to±0.956892Nm, while Arena retains the original8.2 V ceiling,±1.067553Nm. Normal current-limited torque is about 0.6405Nm, so this is unlikely to dominate ordinary gait failures; extreme velocity/PWM clipping is a caveat. Foot contact priority differs, but floor/foot friction and contact stiffness match. The audit records these facts rather than claiming perfect physical identity. See `recon/alignment.json` and `scripts/audit_hill_alignment.py`.

## H4 and H5: continuation status

**H4 settle-clock correction**, commit `916941d5353aa8d026d74313e5e1a153cd52dd87`, changed only that counter timing. It resumed immutableH1b500 actor/critic/Adam, preserved learned trainable std, retained LR1e-5/rawvx2.6/rewards and scored every 100. Its first checkpoint 500 finished5/5 at1.368 s, slightly slower than the selected policy. Checkpoints1200/1300/1400 each finished1/5; H4 stopped at1400 without a new best.

**H5 spawn-heading reward**, commit `534aeb7fe3fed10b61033c8da222f395164dbc4d`, is running as of this guide's2026-10-13 snapshot. It resumes H1b500 with the corrected settle clock, and adds only `heading_hold` relative to spawn yaw, std0.4, weight1.0. Actor/critic dimensions, LR1e-5, rawvx2.6, learned trainable std, nominal course and other rewards stay unchanged. It scores every 100. Its first checkpoint 500 finished3/5; no new reliable candidate has been selected. Check the current `results.csv` and `LOG.md` for subsequent results.

The exact continuation commands are retained in `h4.sh`/`h5.sh`, with configuration in `microduck-challenges/experiments/h4_settleclock.json` and `h5_headinghold.json`. For H5:

```bash
.venv/bin/python scripts/finetune_sprint.py \
  --run-name h5_headinghold \
  --parent-checkpoint $MICRODUCK_ROOT/events/hill-climb-1m5/best/hill1_h1b_actorhill/model_500.pt \
  --full-resume --preserve-std --command-vx 2.6 --heading-weight 1.0 \
  --num-envs 512 --iterations 1001 --lr 1e-5 --save-interval 100 \
  --best-score 1.36 --stage hill5 --target-score 0.90
```

Only a strictly faster **5/5** result may replace `CANDIDATE.md`. Three successive checkpoints below3/5 stop a run; roughly 1000 updates without reliable improvement also stop it. We preserve immutable best artifacts, score nearby checkpoints, and avoid assuming reward curves or the last checkpoint predict the Arena result. The previous position-based lane-hold reward was refuted; H5 instead prices cumulative yaw error, allowing steering back toward the original heading.

## Reproduce the video on CPU

The recording script is `microduck-challenges/scripts/record_hill.py`; its exact environment and command are retained in `media/record_command.sh`. It sets `CUDA_VISIBLE_DEVICES=''`, `MUJOCO_GL=osmesa`, uses a local Mesa library cache, and acquires the same scorer lock before running. Rendering changes only visibility/colors/camera, never physical geometry or action targets.

```bash
bash $MICRODUCK_ROOT/events/hill-climb-1m5/media/record_command.sh
```

OSMesa was absent from the system. Libraries were downloaded/extracted into this event's cache, with no system or shared-venv installation. The working library was Ubuntu `libosmesa6`25.1.7-1ubuntu2~24.04.2 (system LLVM20 satisfies its dependency). To recreate the local rendering cache:

```bash
mkdir -p ../cache/osmesa
cd ../cache/osmesa
apt-get download libosmesa6=25.1.7-1ubuntu2~24.04.2
dpkg-deb -x libosmesa6_25.1.7-1ubuntu2~24.04.2_amd64.deb .
```

`media/hill-climb-seed3-1.34s.json` retains model SHA, seed, native and half-speed playback information, exact trace-parity result, dimensions, fps and ffmpeg arguments. `ffprobe` verifies1280×720,50/1 fps,8.520 s. Training uses its assigned single GPU slot throughout; video uses CPU rendering and no additional training process.