NerveNet crawler policies

An independent NerveNet-style graph policy and capacity-matched MLP baseline, trained with PPO on a custom three-module MuJoCo crawler. This release contains the validation-selected checkpoint for each of five training seeds per policy, the matching implementation, evaluation results, and illustrative rollouts.

Source and full study · Original NerveNet paper

The original authors' code was not used. This is a fixed-morphology locomotion study, not a reproduction of the paper's morphology or disability transfer results.

Matched MLP Graph policy
Matched MLP rollout Graph rollout

Both clips show training seed 0 on held-out episode seed 30,000 for ten simulated seconds. They illustrate one episode; the statistics below cover all five runs.

Evaluation

Both policies receive the same 15 × 17 observation matrix and use the same critic and PPO settings. The graph actor uses width 64, four message-passing rounds, mean aggregation, and node-type-specific GRU updates. The MLP actor has two 151-unit hidden layers. Active actor parameter counts are 63,617 and 63,432, respectively; the graph's unused jointless-body GRU is excluded from that count.

Training seeds are 0–4, with a requested budget of up to 1.3M environment steps per run. Checkpoints are selected by mean reward on ten fixed validation episodes (seeds 10,000–10,009), then tested on fifty separate episodes (30,000–30,049). Selection is per training run, so released checkpoints can precede the final budget. The 1.3M extension followed an inspected 1M report and used fresh test seeds; both budget reports and the recorded extension are included.

Held-out metric Matched MLP Graph
Forward distance 16.04 ± 1.57 m 24.59 ± 1.58 m
Episode reward 783.15 ± 77.35 1213.79 ± 78.31
Absolute lateral distance 3.09 ± 1.12 m 1.96 ± 1.17 m
Actions near saturation 65.1% ± 6.2% 49.4% ± 4.6%
Actuated joint speed 8.55 ± 0.62 rad/s 8.54 ± 0.21 rad/s

Values are mean ± standard deviation across training seeds, not confidence intervals. The graph policy traveled 53.3% farther on average in this study. Median recorded training time per seed was 83.3 minutes for graph versus 8.6 minutes for matched MLP on the development machine; these are machine-specific.

Run a checkpoint

Download the complete model repository so its custom policy classes are available:

hf download Hasan-Yousef/nervenet-crawler-policies --local-dir nervenet-crawler-policies
cd nervenet-crawler-policies
python3.14 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python run_policy.py --policy graph --training-seed 0 --episode-seed 30000
python run_policy.py --policy matched --training-seed 0 --episode-seed 30000
python verify_release.py

The hf command is provided by huggingface_hub. Development and release verification use Python 3.14, MuJoCo 3.14.0, PyTorch 2.14.0, Gymnasium 1.3.0, and Stable-Baselines3 2.9.0 on macOS. Other platforms are not verified here. CPU inference requires no interactive viewer or GPU.

For direct loading, run from the downloaded directory:

from stable_baselines3 import PPO
from nervenet.policy_loading import create_policy_environment

env = create_policy_environment("graph", module_count=3, control_cost_weight=0.05)
model = PPO.load("checkpoints/graph-seed0.zip", env=env, device="cpu")
obs, _ = env.reset(seed=30000)
action, _ = model.predict(obs, deterministic=True)
obs, reward, terminated, truncated, info = env.step(action)
env.close()

Environment and contents

The crawler has three torso modules, six two-segment legs, twelve actuators, and two passive spine hinges. Control steps are 0.02 seconds and episodes last ten seconds. Reward is forward velocity minus 0.05 * mean(action**2). The model builder and environment configuration are included, rather than relying on a separately installed robot asset.

  • checkpoints/: ten complete SB3 PPO checkpoints, including optimizer state.
  • nervenet/: source snapshot needed for the custom policy and environment.
  • run_policy.py: deterministic episode example for either policy and every seed.
  • verify_release.py: hash checks and archived-episode reproduction for all ten policies.
  • release.json: checkpoint selection, SHA-256 hashes, and software provenance.
  • results/crawler-1m-v1/: reports, episode-level evaluations, learning curves, training-session provenance, and GIFs.
  • docs/: architecture and experiment documentation.

These are standard SB3 ZIP checkpoints, not Transformers models or Safetensors. SB3 loading uses serialized Python objects; load checkpoints only from a source you trust. There is no hosted inference endpoint in this release.

Intended use and limitations

Use for studying graph policies, inspecting learned crawler locomotion, and reproducing the included evaluations. The released PPO policies are specific to the three-module observation/action spaces. No zero-shot morphology transfer, disability transfer, real-robot deployment, or general graph-policy superiority is established. Large actions and fast joint motion remain common; greater distance does not establish a natural or energy-efficient gait.

No license has yet been designated for this project's original code and weights; this release does not grant an explicit reuse license. Dependency licenses remain those of their respective projects.

Downloads last month
51
Video Preview
loading