Deploy WhatMoves app

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+app/assets/sources/penguins.mp4 filter=lfs diff=lfs merge=lfs -text
+app/assets/sources/tiger.mp4 filter=lfs diff=lfs merge=lfs -text
+app/assets/targets/bear_and_dog.png filter=lfs diff=lfs merge=lfs -text
+docs/images/scene_composition.png filter=lfs diff=lfs merge=lfs -text
+docs/images/title.png filter=lfs diff=lfs merge=lfs -text
+docs/videos/app_demo.mp4 filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,11 @@
+__pycache__/
+*.py[cod]
+.venv/
+.env
+*.egg-info/
+build/
+dist/
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.DS_Store

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 CompVis - Computer Vision and Learning LMU Munich
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,14 +1,275 @@
 ---
-title: WhatMoves
-emoji: 👁
-colorFrom: red
-colorTo: yellow
+title: What Moves?
+emoji: 🎬
+colorFrom: blue
+colorTo: purple
 sdk: gradio
-sdk_version: 6.26.0
-python_version: '3.12'
+sdk_version: 5.49.1
+python_version: "3.12"
 app_file: app.py
 pinned: false
-license: mit
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+<p align="center">
+  <img src="docs/images/what_moves.png" width="35%" alt="WhatMoves logo">
+</p>
+
+<p align="center">
+  <a href="https://compvis.github.io/WhatMoves/"><img src="https://img.shields.io/badge/Project-Page-blue" alt="Project Page"></a>
+  <a href="https://arxiv.org/"><img src="https://img.shields.io/badge/arXiv-paper-b31b1b" alt="Paper"></a>
+  <a href="https://huggingface.co/CompVis/WhatMoves"><img src="https://img.shields.io/badge/%F0%9F%A4%97-weights-yellow" alt="Weights"></a>
+</p>
+
+<h2 align="center">
+  Localized Motion Representations for Compositional Scene Control
+</h2>
+<div align="center"> 
+  <a href="https://ffundel.de/" target="_blank">Frank Fundel</a><sup>*</sup> · 
+  <a href="https://www.linkedin.com/in/malek-ben-alaya/" target="_blank">Malek Ben Alaya</a><sup>*</sup> · 
+  <a href="https://www.linkedin.com/in/thomas-ressler-494758133/" target="_blank">Thomas Ressler-Antal</a><sup>*</sup>
+  <br>
+  <a href="https://stefan-baumann.eu/" target="_blank">Stefan A. Baumann</a> · 
+  <a href="https://ommer-lab.com/people/ommer/" target="_blank">Björn Ommer</a>
+</div>
+<p align="center"> 
+  <b>CompVis @ LMU Munich, MCML</b>
+  <br/>
+  <i>* equal contribution</i>
+  <br/>
+  ECCV 2026
+</p>
+
+
+<p align="center">
+  <img src="docs/images/title.png" width="100%" alt="WhatMoves overview">
+</p>
+
+
+## 💡 TL;DR
+
+Existing motion representations often entangle the dynamics of multiple entities, while isolating objects through cropping removes important scene context. **WhatMoves learns promptable, localized motion representations directly from full videos**, capturing the motion of user-selected regions while preserving their surrounding context. These representations enable object-level motion transfer, compositional scene control, and localized action recognition.
+
+
+## 🛠️ Installation
+
+The tested configuration is Python 3.11/3.12, PyTorch 2.8, and CUDA 12.8. Create an
+environment, install the PyTorch build appropriate for your CUDA installation,
+and then choose the smallest dependency set you need.
+
+```bash
+git clone https://github.com/CompVis/WhatMoves.git
+cd WhatMoves
+
+python3.12 -m venv .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip setuptools wheel packaging ninja
+
+# Example for CUDA 12.8; use the matching official PyTorch index for your host.
+pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128
+```
+
+Install one of the following:
+
+```bash
+# WhatMoves encoder only
+pip install -r requirements-core.txt
+
+# Encoder + Wan motion transfer
+pip install -r requirements-wan.txt --no-build-isolation
+
+# Complete app, including the Hugging Face Spaces transport
+SAM2_BUILD_CUDA=0 pip install -r requirements.txt --no-build-isolation
+
+# Optional Wan acceleration (requires a compatible wheel or a CUDA toolkit)
+pip install flash-attn==2.8.3 --no-build-isolation
+```
+
+The SAM2 CUDA extension is not needed for the app's image-prompt workflow. The
+Wan path runs without FlashAttention through PyTorch SDPA. FlashAttention is
+recommended for speed and memory use, but is kept optional because building it
+from source requires `nvcc` and a compatible compiler toolchain.
+
+
+## 🧩 Scene Composition
+WhatMoves can be used to **compose the motion of an entire scene from individual source motions**. Select an object or region in one or more source videos, choose the corresponding regions in a target image, and transfer each motion independently to its target. This makes it possible to animate different entities with motions taken from different videos while preserving the appearance and composition of the target scene.
+
+<p align="center">
+  <img src="docs/images/scene_composition.png" width="100%" alt="Scene composition with localized motion transfer">
+</p>
+
+Our released adapter integrates WhatMoves with [Wan2.2 I2V-A14B](https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B). The required Wan runtime is included in `wan/`, so no separate Wan repository is needed, and the base model weights are downloaded automatically on first use.
+
+```python
+import torch
+
+model = torch.hub.load(
+    "CompVis/WhatMoves",
+    "wan",
+    device="cuda",
+    trust_repo=True,
+)
+
+# target_image:  [B, H, W, 3], floating point in [-1, 1]
+# source_video:  [B, T, Hs, Ws, 3], floating point in [-1, 1]
+# source_mask:   [B, K, Hs, Ws], bool
+# target_mask:   [B, K, H, W], bool
+
+video = model.sample(
+    "A concise description of the target video",
+    target_image,
+    source_videos=[source_video],
+    source_masks=[source_mask],
+    target_masks=[target_mask],
+    num_frames=25,
+    num_inference_steps=40,
+    seed=42,
+)
+
+# video: [B, 25, H, W, 3] in [-1, 1]
+```
+
+Within each source video, the `K` selected source regions are matched positionally to the `K` regions in the corresponding target mask. To **compose a scene from multiple motions**, simply provide additional source videos and masks—their motions can then be transferred independently to different parts of the target scene.
+
+Source masks may also be provided as temporal `[B, T, K, Hs, Ws]` mask sequences. If a static source mask was drawn on a frame other than the first one, you can provide the corresponding frame through `source_content_images` so that the content embedding is extracted from the image the mask actually refers to:
+
+```python
+# If source_mask was drawn on source frame j:
+video = model.sample(
+    prompt,
+    target_image,
+    source_videos=[source_video],
+    source_masks=[source_mask],
+    source_content_images=[source_video[:, j]],
+    target_masks=[target_mask],
+)
+```
+
+Source and output videos do not need to have the same duration: WhatMoves aligns the extracted motion representations to the requested output timeline.
+
+Target dimensions must be divisible by 16, and the number of output frames must follow `4n + 1`, with at least nine frames for motion transfer. The released adapter was trained on **25-frame videos at 8 fps**, so we recommend staying close to this duration and frame rate for best results. The interactive app automatically resamples uploaded videos to 8 fps and previews the exact sequence used for inference.
+
+If you already have a local copy of the Wan2.2 base model, you can load it directly:
+
+```python
+model = torch.hub.load(
+    "CompVis/WhatMoves",
+    "wan",
+    wan_checkpoint="/path/to/Wan2.2-I2V-A14B",
+    device="cuda",
+    trust_repo=True,
+)
+```
+
+Prefer a visual, no-code workflow? You can try the same scene composition pipeline directly in the **interactive app** below.
+
+### 💾 Resource requirements
+
+Wan2.2 I2V-A14B requires substantial GPU memory and disk space. The base model occupies about **112 GiB** on disk, with the released WhatMoves and adapter checkpoints adding roughly **3 GiB**. We recommend keeping at least **125 GiB** available for the Hugging Face cache.
+
+The current single-GPU implementation keeps both 14B Wan experts in GPU memory. A one-source, 25-frame generation at 480 × 704 resolution uses approximately **67.6 GiB** of peak allocated memory on an 80 GB A100. A 40 GB GPU is therefore not sufficient for the current implementation. CPU offloading, multi-GPU inference, and sequential expert offloading are not yet supported.
+
+
+## 🖥️ Interactive App
+Want to try WhatMoves without writing any code? The interactive app provides a simple visual workflow for **selecting motion from source videos and composing it into a target scene**.
+
+
+https://github.com/user-attachments/assets/09ca005a-bc85-4eba-8f6c-c159c2611994
+
+
+You can upload one or more source videos, select the entities whose motion you want to reuse, choose the corresponding regions in a target image, and generate the resulting video directly from the interface. SAM2-assisted prompting makes region selection fast and interactive, while color-coded mappings help keep track of which source motion is assigned to which target object.
+
+The app also includes a ready-to-use example and supports multiple source videos and target images, frame-accurate source masking, temporal video previews, and the available Wan guidance settings.
+
+Launch it with:
+
+```bash
+python -m app
+```
+
+By default, the app is available at `127.0.0.1:7860`. For remote machines, Slurm setups, SSH tunneling, and additional configuration details, see [app/README.md](app/README.md).
+
+
+## 🎯 Extract Localized Motion
+
+Interested in the **motion representations themselves**? The scene composition model above uses the WhatMoves encoder internally, but you can also load and use the encoder independently—without running Wan or any generative model.
+
+Given a video and one or more spatial masks, the encoder extracts a separate localized motion representation for each selected region while retaining the context of the full scene. Torch Hub automatically downloads the released checkpoint from [Hugging Face](https://huggingface.co/CompVis/WhatMoves), verifies its SHA-256 checksum, and caches it locally.
+
+
+```python
+import torch
+
+encoder = torch.hub.load(
+    "CompVis/WhatMoves",
+    "what_moves",
+    device="cuda",
+    dtype=torch.bfloat16,
+    trust_repo=True,
+)
+
+# video: [B, T, H, W, 3] floating-point RGB in [-1, 1]
+# masks: [B, K, H, W] boolean regions in video's first frame
+motion = encoder.encode_motion(video, masks)          # [B, T-3, K, 384]
+content = encoder.encode_content(video[:, 0], masks)  # [B, K, 512]
+```
+
+The encoder accepts arbitrary spatial input sizes and internally resizes them
+to 256 × 256. It operates on eight-frame windows and emits five motion tokens
+per window, producing `T - 3` aligned output steps for a `T`-frame video. The
+released checkpoint was trained on video sampled at **6 fps**. While the input 
+frame rate can be chosen freely, we recommend using a sampling rate close to the training frame rate.
+
+You can also omit the masks entirely to extract **global motion representations** that capture the dynamics of the full scene:
+
+
+```python
+global_motion = encoder.encode_motion(video)  # [B, T-3, 384]
+```
+
+### 🎞️ Temporal masks for long videos
+
+For longer videos, you can also provide a **frame-aligned sequence of masks** with shape `[B, T, K, H, W]`. This allows WhatMoves to periodically update the content embeddings used to extract motion, instead of relying on a mask from a single reference frame throughout the entire video.
+
+This can be useful when an entity changes substantially in appearance, pose, or structure over time, such that its representation in the first frame is no longer a good match later in the sequence.
+
+```python
+motion, content = encoder.encode_motion(
+    video,
+    temporal_masks,
+    return_content=True,
+)
+
+# motion:  [B, T-3, K, 384]
+# content: [B, T-3, K, 512], aligned with motion
+```
+
+For each extraction window, WhatMoves updates the content embedding using the first available mask at or after the start of that window. If no valid mask is available for a region within a window, the most recent content embedding is reused. Each region therefore needs at least one valid mask in the first extraction window. `frame_chunk_size` and `window_chunk_size` can additionally be used to reduce peak memory usage. Both specify per-video chunk sizes, resulting in an effective forward batch size of `B × chunk_size`.
+
+
+
+## 🙏 Acknowledgements
+
+WhatMoves builds on a fantastic ecosystem of open-source research. We thank the **DINOv2**, **Wan2.2**, **Hugging Face Diffusers**, and **SAM2** teams for making their work publicly available and enabling projects like this one.
+
+You can also check out our previous work, [DisMo](https://github.com/CompVis/DisMo), which introduced **global, content-disentangled motion representations for open-world motion transfer**.
+
+This repository contains reduced and modified code from [DINOv2](https://github.com/facebookresearch/dinov2) and [Wan2.2](https://github.com/Wan-Video/Wan2.2), both released under the Apache License 2.0. The released model artifacts likewise contain a mixture of original and upstream-derived components.
+
+For the exact licensing and attribution details, please see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md), [wan/LICENSE.txt](wan/LICENSE.txt), and the Hugging Face model card. The root MIT license does not replace the applicable upstream terms.
+
+
+## 📝 Citation
+
+If you find **WhatMoves** useful in your research, please consider citing our work:
+
+```bibtex
+@inproceedings{fundel2026whatmoves,
+  title     = {What Moves? Localized Motion Representations for Compositional Scene Control},
+  author    = {Fundel, Frank and Ben Alaya, Malek and Ressler-Antal, Thomas and Baumann, Stefan Andreas and Ommer, Bjorn},
+  booktitle = {European Conference on Computer Vision (ECCV)},
+  year      = {2026}
+}
+```
+
+We would also love to hear about projects that build on WhatMoves or use the learned representations in new settings.
+

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,46 @@
+# Third-party notices
+
+WhatMoves contains reduced or vendored components from the projects below.
+The root MIT license applies to original WhatMoves code; it does not replace
+the licenses or copyright notices of these components.
+
+## DINOv2
+
+`what_moves/dinov2.py` is a reduced and modified implementation of
+[DINOv2](https://github.com/facebookresearch/dinov2) by Meta Platforms, Inc.,
+based on commit `7764ea0f912e53c92e82eb78a2a1631e92725fc8`.
+
+DINOv2 code and the pretrained DINOv2 parameters incorporated into the
+WhatMoves checkpoint are licensed under the Apache License 2.0. A copy of that
+license is included at `wan/LICENSE.txt`. The reduced implementation identifies
+its modifications in the source header.
+
+## Wan2.2
+
+The `wan/` package is a reduced runtime from
+[Wan2.2](https://github.com/Wan-Video/Wan2.2) by the Alibaba Wan Team, based on
+commit `388807310646ed5f318a99f8e8d9ad28c5b65373`. It is licensed under the
+Apache License 2.0; see `wan/LICENSE.txt`. Every modified vendored source file
+carries an explicit modification notice.
+
+The released checkpoints combine original WhatMoves parameters with components
+derived from Apache-2.0-licensed DINOv2 or designed to operate as deltas over
+Apache-2.0-licensed Wan2.2. See the WhatMoves Hugging Face model card for the
+per-file breakdown. The repository's root MIT license applies only to original
+WhatMoves material; all upstream notices and conditions remain in force.
+
+## Hugging Face Transformers and Diffusers
+
+The upstream Wan runtime's `wan/modules/t5.py` is derived from
+[Transformers](https://github.com/huggingface/transformers), and
+`wan/utils/fm_solvers_unipc.py` is derived from
+[Diffusers](https://github.com/huggingface/diffusers). Both projects are
+licensed under Apache License 2.0. Their provenance comments are retained in
+the corresponding files; the bundled license text is at `wan/LICENSE.txt`.
+
+## SAM2
+
+The optional interactive app installs
+[SAM2](https://github.com/facebookresearch/sam2) as an external dependency at a
+pinned revision. SAM2 is not copied into this repository and is licensed under
+Apache License 2.0.

app.py

@@ -0,0 +1,20 @@
+"""Hugging Face Spaces entrypoint for the WhatMoves demo."""
+
+from __future__ import annotations
+
+import os
+
+import uvicorn
+
+from app.gradio_app import create_gradio_application
+
+
+application = create_gradio_application()
+
+
+if __name__ == "__main__":
+    uvicorn.run(
+        application,
+        host=os.environ.get("WHATMOVES_APP_HOST", "0.0.0.0"),
+        port=int(os.environ.get("PORT", os.environ.get("WHATMOVES_APP_PORT", "7860"))),
+    )

app/README.md

@@ -0,0 +1,139 @@
+# WhatMoves interactive app
+
+The app is a build-free browser interface for localized motion transfer.
+FastAPI hosts the unchanged custom HTML, CSS, JavaScript, and streaming media;
+named Gradio endpoints carry browser operations and integrate GPU calls with
+Hugging Face ZeroGPU. One in-process scheduler still serializes SAM2 and Wan
+work.
+
+Install the complete application from the repository root using Python 3.12. Model weights are downloaded on first use and are not stored in this repository:
+
+```bash
+SAM2_BUILD_CUDA=0 pip install -r requirements.txt --no-build-isolation
+```
+
+## Run locally
+
+From the repository root:
+
+```bash
+source .venv/bin/activate
+python -m app
+```
+
+Open <http://localhost:7860>. The default bind address is loopback-only. Use
+`--port PORT` to choose another port.
+
+## Run on Slurm and connect over SSH
+
+Allocate one sufficiently large GPU. Substitute your cluster's partition and
+GPU resource syntax:
+
+```bash
+srun -p GPU_PARTITION --gres=gpu:GPU_TYPE:1 --time=04:00:00 \
+  --job-name=whatmoves-app --pty bash
+```
+
+On the allocated GPU node, run:
+
+```bash
+source /path/to/WhatMoves/.venv/bin/activate
+cd /path/to/WhatMoves
+
+export PYTHONNOUSERSITE=1
+export PYTHONPATH="$PWD"
+
+# Optional: reuse an existing official Wan2.2 base-model directory.
+export WHATMOVES_WAN_CHECKPOINT=/path/to/Wan2.2-I2V-A14B
+
+export WHATMOVES_APP_PORT=$(python -c \
+  'import socket; s=socket.socket(); s.bind(("", 0)); print(s.getsockname()[1]); s.close()')
+
+echo "GPU node: $(hostname)"
+echo "Remote port: $WHATMOVES_APP_PORT"
+python -m app --host 0.0.0.0 --port "$WHATMOVES_APP_PORT"
+```
+
+Leave that process running. On your laptop, substitute the printed values and
+your normal SSH login host or alias:
+
+```bash
+ssh -N -L 7860:GPU_NODE:REMOTE_PORT LOGIN_HOST
+```
+
+Then open <http://localhost:7860>. If local port 7860 is occupied, change only
+the first port, for example `-L 7861:GPU_NODE:REMOTE_PORT`, and open
+<http://localhost:7861>.
+
+Each user should request a separate GPU, start a separate app process, and use
+their own remote and laptop ports. Do not add Uvicorn workers: model and session
+state are intentionally process-local, and the Wan runtime is not reentrant.
+
+## Workflow
+
+The app opens with a small, fully masked and mapped synthetic example. You can
+generate from it immediately or remove it and upload your own media.
+
+1. Upload one or more source videos on the left and target images on the right.
+   Each rail keeps its uploads independent; generation uses only the selected
+   target image.
+2. Source videos are timestamp-resampled to the adapter's 8 fps training
+   cadence. Play them or scrub the filmstrip to inspect the exact frames used
+   by inference. The two handles select the source interval.
+3. Select any source frame, then choose **Start masking**. New uploads enter
+   masking mode automatically. Left-click adds a positive SAM2 point,
+   right-click adds a negative point, and left-drag adds a box. Hovering shows
+   a transient proposal without changing the committed prompt. Choose
+   **Add mask** to store the committed proposal at that exact frame.
+4. Colored source-mask previews remain visible at their timeline positions;
+   selecting one jumps back to its frame. Drag a colored source mask onto a
+   grey target mask, or click the target mask and choose a source from its menu.
+5. Choose a prompt and generate. Honest stage progress is shown, including the
+   exact sampling step.
+
+On ordinary local and Slurm deployments, SAM2 image features are queued as soon
+as media is uploaded. On ZeroGPU they are prepared lazily inside the first
+decorated mask request, because CUDA exists only for the lifetime of that
+request. Source features are cached per selected frame in a bounded LRU cache.
+Wan loads lazily on the first generation. A
+generation uses an immutable snapshot of the selected target, source intervals,
+masks, and settings present when **Generate video** was pressed; you may
+continue editing while it runs. The completed video is still shown and marked
+**previous inputs** if those inputs changed meanwhile.
+
+## Lifecycle and privacy
+
+- Runtime data is stored in a mode-`0700` directory under
+  `/tmp/whatmoves-app-<job>-<pid>` by default; media and output files use mode
+  `0600`.
+- Target upload files are deleted immediately after decoding. Uploaded source
+  videos are replaced by their canonical 8 fps session copy. Removing a source
+  or target drops its RGB arrays, masks, drafts, pending work, mappings, cached
+  SAM2 features, and session files. Bundled app assets are copied into a
+  session and are never modified.
+- Uncommitted mask drafts are cancelled when their source is left and after a
+  browser reload, because their point and box prompts exist only in that page.
+- Only the latest successful generated video is retained. Generation input
+  links are removed as soon as their job finishes.
+- Browser sessions send a lightweight heartbeat and expire after two idle
+  hours by default. Expiration or graceful server shutdown releases all
+  associated memory and files.
+- The server has no authentication. Keep it on loopback or a trusted compute
+  network and access it through SSH; do not expose it directly to the internet.
+
+The most useful environment overrides are:
+
+| Variable | Default | Purpose |
+|---|---:|---|
+| `WHATMOVES_WAN_CHECKPOINT` | download through Hugging Face | Local Wan2.2 base directory |
+| `WHATMOVES_APP_TMP_DIR` | `/tmp/whatmoves-app-<job>-<pid>` | Runtime-data root |
+| `WHATMOVES_APP_SESSION_TTL` | `7200` | Idle-session lifetime in seconds |
+| `WHATMOVES_APP_MAX_SOURCES` | `8` | Maximum source videos per session |
+| `WHATMOVES_APP_MAX_TARGETS` | `8` | Maximum target images per session |
+| `WHATMOVES_APP_MAX_SAM_FRAMES` | `8` | Maximum cached per-frame SAM2 predictors |
+| `WHATMOVES_APP_MAX_GPU_TASKS` | `32` | Maximum queued/running GPU tasks |
+| `WHATMOVES_APP_HOST` | `127.0.0.1` | Server bind address |
+| `WHATMOVES_APP_PORT` | `7860` | Server port |
+
+Use `--reload` only for frontend development. It is not intended for model
+inference or production sessions.

app/__init__.py

@@ -0,0 +1,5 @@
+"""WhatMoves interactive web application."""
+
+from .server import create_app
+
+__all__ = ["create_app"]

app/__main__.py

@@ -0,0 +1,36 @@
+"""Run the application with ``python -m app``."""
+
+from __future__ import annotations
+
+import argparse
+import os
+
+import uvicorn
+
+from .gradio_app import create_gradio_application
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="WhatMoves interactive demo")
+    parser.add_argument(
+        "--host", default=os.environ.get("WHATMOVES_APP_HOST", "127.0.0.1")
+    )
+    parser.add_argument(
+        "--port",
+        type=int,
+        default=int(os.environ.get("WHATMOVES_APP_PORT", "7860")),
+    )
+    parser.add_argument(
+        "--reload", action="store_true", help="Reload after source changes"
+    )
+    args = parser.parse_args()
+    uvicorn.run(
+        create_gradio_application(),
+        host=args.host,
+        port=args.port,
+        reload=args.reload,
+    )
+
+
+if __name__ == "__main__":
+    main()

app/assets/examples.json

@@ -0,0 +1,64 @@
+{
+  "version": 1,
+  "sources": [
+    {
+      "id": "tiger",
+      "name": "Tiger",
+      "video": "sources/tiger.mp4",
+      "trim": [0, 24],
+      "masks": [
+        {
+          "id": "tiger",
+          "frame": 0,
+          "mask": "masks/tiger_f00.png",
+          "color": "#E69F00"
+        }
+      ]
+    },
+    {
+      "id": "penguins",
+      "name": "Penguins",
+      "video": "sources/penguins.mp4",
+      "trim": [0, 24],
+      "masks": [
+        {
+          "id": "center-penguin",
+          "frame": 0,
+          "mask": "masks/center_penguin_f00.png",
+          "color": "#56B4E9"
+        }
+      ]
+    }
+  ],
+  "targets": [
+    {
+      "id": "bear-dog",
+      "name": "Bear and dog",
+      "image": "targets/bear_and_dog.png",
+      "masks": [
+        {
+          "id": "bear",
+          "mask": "masks/target_bear.png"
+        },
+        {
+          "id": "dog",
+          "mask": "masks/target_dog.png"
+        }
+      ]
+    }
+  ],
+  "mappings": [
+    {
+      "source": "tiger",
+      "source_mask": "tiger",
+      "target": "bear-dog",
+      "target_mask": "bear"
+    },
+    {
+      "source": "penguins",
+      "source_mask": "center-penguin",
+      "target": "bear-dog",
+      "target_mask": "dog"
+    }
+  ]
+}

app/assets/sources/penguins.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac0358344a10f49ced74cfe336e9f7c63c82acb423ff95bb802291da3e6c1482
+size 452943

app/assets/sources/tiger.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f99394c930814dcc574409c42088852653bec39356d9a816ad94f7d23658e6f3
+size 856651

app/config.py

@@ -0,0 +1,52 @@
+"""Application configuration."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+import os
+from pathlib import Path
+
+DEFAULT_VARIANT = "gated_static_step600000"
+
+
+def _optional_path(name: str) -> Path | None:
+    value = os.environ.get(name)
+    return Path(value).expanduser() if value else None
+
+
+def _runtime_root() -> Path:
+    configured = os.environ.get("WHATMOVES_APP_TMP_DIR")
+    if configured:
+        return Path(configured).expanduser()
+    job = os.environ.get("SLURM_JOB_ID", "local")
+    suffix = f"{job}-{os.getpid()}"
+    return Path("/tmp") / f"whatmoves-app-{suffix}"
+
+
+def _positive_int(name: str, default: int) -> int:
+    value = int(os.environ.get(name, default))
+    if value < 1:
+        raise ValueError(f"{name} must be positive")
+    return value
+
+
+@dataclass(frozen=True)
+class AppConfig:
+    model_variant: str = DEFAULT_VARIANT
+    wan_checkpoint: Path | None = _optional_path("WHATMOVES_WAN_CHECKPOINT")
+    sam_model_id: str = "facebook/sam2-hiera-large"
+    device: str = "cuda"
+    runtime_root: Path = _runtime_root()
+    output_height: int = 480
+    output_width: int = 704
+    output_frames: int = 25
+    output_fps: int = 8
+    source_fps: int = 8
+    max_image_side: int = 1600
+    max_upload_bytes: int = 2 * 1024**3
+    max_sources: int = _positive_int("WHATMOVES_APP_MAX_SOURCES", 8)
+    max_targets: int = _positive_int("WHATMOVES_APP_MAX_TARGETS", 8)
+    max_sam_frames: int = _positive_int("WHATMOVES_APP_MAX_SAM_FRAMES", 8)
+    max_gpu_tasks: int = _positive_int("WHATMOVES_APP_MAX_GPU_TASKS", 32)
+    session_ttl_seconds: int = _positive_int("WHATMOVES_APP_SESSION_TTL", 7200)
+    cleanup_interval_seconds: int = _positive_int("WHATMOVES_APP_CLEANUP_INTERVAL", 60)

app/examples.py

@@ -0,0 +1,135 @@
+"""Load the small, redistributable examples bundled with the release."""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+import shutil
+
+import numpy as np
+from PIL import Image
+
+from .media import decode_video_frame, probe_video, read_image
+from .state import MaskRecord, Session, SourceAsset, TargetAsset, new_id
+
+
+def _asset_path(root: Path, relative: str) -> Path:
+    path = (root / relative).resolve()
+    if root.resolve() not in path.parents or not path.is_file():
+        raise ValueError(f"Invalid example asset: {relative}")
+    return path
+
+
+def _mask(path: Path, shape: tuple[int, int]) -> np.ndarray:
+    with Image.open(path) as opened:
+        image = opened.convert("L")
+        if image.size != (shape[1], shape[0]):
+            image = image.resize((shape[1], shape[0]), Image.Resampling.NEAREST)
+        mask = np.asarray(image) > 127
+    if not mask.any():
+        raise ValueError(f"Example mask is empty: {path.name}")
+    return mask
+
+
+def load_examples(
+    session: Session,
+    root: Path,
+    *,
+    source_fps: int,
+    max_sources: int,
+    max_targets: int,
+    max_image_side: int,
+) -> None:
+    """Copy manifest-defined examples into one disposable browser session."""
+    manifest_path = root / "examples.json"
+    if not manifest_path.is_file():
+        return
+    manifest = json.loads(manifest_path.read_text())
+    if manifest.get("version") != 1:
+        raise ValueError("Unsupported examples manifest version")
+
+    source_refs: dict[str, tuple[SourceAsset, dict[str, str]]] = {}
+    for record in manifest.get("sources", [])[:max_sources]:
+        source_id = new_id("source")
+        source_path = _asset_path(root, record["video"])
+        runtime_path = session.directory / f"{source_id}.mp4"
+        shutil.copyfile(source_path, runtime_path)
+        runtime_path.chmod(0o600)
+        fps, frame_count = probe_video(runtime_path)
+        if frame_count is None or frame_count < 1:
+            raise ValueError(f"Example video has no known frames: {source_path.name}")
+        if abs(fps - source_fps) > 0.05:
+            raise ValueError(f"Example video must be encoded at {source_fps} fps")
+        trim = record.get("trim", [0, frame_count - 1])
+        trim_start = int(trim[0])
+        trim_end = int(trim[1])
+        if not 0 <= trim_start <= trim_end < frame_count:
+            raise ValueError(f"Invalid example trim for {source_path.name}")
+        frame = decode_video_frame(runtime_path, trim_start, max_image_side)
+        source = SourceAsset(
+            id=source_id,
+            name=str(record.get("name", source_path.stem)),
+            video_path=runtime_path,
+            frame=frame,
+            fps=float(source_fps),
+            frame_count=frame_count,
+            original_fps=float(source_fps),
+            current_frame=trim_start,
+            trim_start=trim_start,
+            trim_end=trim_end,
+            sam_status="idle",
+            example=True,
+        )
+        mask_refs = {}
+        for mask_record in record.get("masks", []):
+            frame_index = int(mask_record["frame"])
+            if not 0 <= frame_index < frame_count:
+                raise ValueError("Example source mask frame is out of range")
+            mask_id = new_id("mask")
+            allocated = session.allocate_color()
+            color = str(mask_record.get("color") or allocated)
+            source.masks[mask_id] = MaskRecord(
+                mask_id,
+                _mask(_asset_path(root, mask_record["mask"]), frame.shape[:2]),
+                color=color,
+                frame_index=frame_index,
+            )
+            mask_refs[str(mask_record["id"])] = mask_id
+            source.mask_revision += 1
+        session.sources[source.id] = source
+        source_refs[str(record["id"])] = (source, mask_refs)
+
+    target_refs: dict[str, tuple[TargetAsset, dict[str, str]]] = {}
+    for record in manifest.get("targets", [])[:max_targets]:
+        target_id = new_id("target")
+        image_path = _asset_path(root, record["image"])
+        image = read_image(image_path, max_image_side)
+        session.target_revision += 1
+        target = TargetAsset(
+            id=target_id,
+            name=str(record.get("name", image_path.stem)),
+            image=image,
+            revision=session.target_revision,
+            sam_status="idle",
+            example=True,
+        )
+        mask_refs = {}
+        for mask_record in record.get("masks", []):
+            mask_id = new_id("mask")
+            target.masks[mask_id] = MaskRecord(
+                mask_id,
+                _mask(_asset_path(root, mask_record["mask"]), image.shape[:2]),
+            )
+            mask_refs[str(mask_record["id"])] = mask_id
+            target.mask_revision += 1
+        session.targets[target.id] = target
+        target_refs[str(record["id"])] = (target, mask_refs)
+
+    for mapping in manifest.get("mappings", []):
+        source, source_masks = source_refs[str(mapping["source"])]
+        target, target_masks = target_refs[str(mapping["target"])]
+        target_mask = target.masks[target_masks[str(mapping["target_mask"])]]
+        target_mask.source_id = source.id
+        target_mask.source_mask_id = source_masks[str(mapping["source_mask"])]
+        target.mask_revision += 1
+    session.selected_target_id = next(iter(session.targets), None)

app/gradio_app.py

@@ -0,0 +1,209 @@
+"""Gradio transport for the unchanged WhatMoves browser experience.
+
+The public page and read-only media routes remain on the FastAPI application.
+All browser mutations and GPU jobs travel through named Gradio endpoints, so
+Hugging Face ZeroGPU can allocate hardware around the actual CUDA lifetime.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import os
+from pathlib import Path
+import time
+from typing import Any
+
+# Gradio reads its temp root during import. Use a process-private directory so
+# multi-user machines never collide on a shared /tmp/gradio directory.
+if "GRADIO_TEMP_DIR" not in os.environ:
+    _runtime_temp = os.environ.get("WHATMOVES_APP_TMP_DIR")
+    _gradio_temp = (
+        Path(_runtime_temp) / "gradio"
+        if _runtime_temp
+        else Path("/tmp") / f"whatmoves-gradio-{os.getpid()}"
+    )
+    _gradio_temp.mkdir(parents=True, exist_ok=True, mode=0o700)
+    os.environ["GRADIO_TEMP_DIR"] = str(_gradio_temp)
+os.environ.setdefault("MPLBACKEND", "Agg")
+
+import gradio as gr
+import httpx
+import spaces
+
+from .server import Runtime, create_app
+
+
+def _upload_path(value: Any) -> Path | None:
+    if value is None:
+        return None
+    if isinstance(value, (str, os.PathLike)):
+        return Path(value)
+    if isinstance(value, dict):
+        candidate = value.get("path") or value.get("name")
+        return Path(candidate) if candidate else None
+    candidate = getattr(value, "name", None)
+    return Path(candidate) if candidate else None
+
+
+class GradioBridge:
+    def __init__(self, service, runtime: Runtime) -> None:
+        self.service = service
+        self.runtime = runtime
+
+    async def _request(
+        self,
+        method: str,
+        path: str,
+        body: str | None,
+        upload: Any,
+    ) -> dict[str, Any]:
+        method = str(method or "GET").upper()
+        path = str(path or "")
+        if not path.startswith("/api/") or "://" in path:
+            return {"ok": False, "status": 400, "data": "Invalid API path"}
+
+        kwargs: dict[str, Any] = {}
+        upload_path = _upload_path(upload)
+        handle = None
+        try:
+            if upload_path is not None:
+                handle = upload_path.open("rb")
+                kwargs["files"] = {
+                    "file": (upload_path.name, handle, "application/octet-stream")
+                }
+            elif body:
+                kwargs["content"] = body.encode("utf-8")
+                kwargs["headers"] = {"content-type": "application/json"}
+
+            transport = httpx.ASGITransport(app=self.service)
+            async with httpx.AsyncClient(
+                transport=transport,
+                base_url="http://whatmoves.internal",
+            ) as client:
+                response = await client.request(method, path, **kwargs)
+        except Exception as error:
+            return {"ok": False, "status": 500, "data": str(error)}
+        finally:
+            if handle is not None:
+                handle.close()
+
+        if not response.content:
+            data: Any = None
+        else:
+            try:
+                data = response.json()
+            except ValueError:
+                data = response.text
+        return {"ok": response.is_success, "status": response.status_code, "data": data}
+
+    def request(
+        self,
+        method: str,
+        path: str,
+        body: str | None = None,
+        upload: Any = None,
+    ) -> dict[str, Any]:
+        return asyncio.run(self._request(method, path, body, upload))
+
+    def request_json(
+        self,
+        method: str,
+        path: str,
+        body: str | None = None,
+        upload: Any = None,
+    ) -> str:
+        return json.dumps(self.request(method, path, body, upload), separators=(",", ":"))
+
+    def generate(
+        self,
+        method: str,
+        path: str,
+        body: str | None = None,
+        upload: Any = None,
+        progress=gr.Progress(),
+    ) -> dict[str, Any]:
+        result = self.request(method, path, body, upload)
+        if not result["ok"]:
+            return result
+        task_id = (result.get("data") or {}).get("task_id")
+        if not task_id:
+            return result
+
+        while True:
+            record = self.runtime.scheduler.get(task_id)
+            if record is None:
+                return {"ok": False, "status": 404, "data": "Generation task disappeared"}
+            if record.progress_total and record.progress_current is not None:
+                progress(
+                    (record.progress_current, record.progress_total),
+                    desc=record.stage,
+                )
+            else:
+                progress(0, desc=record.stage)
+            if record.status not in {"queued", "running"}:
+                return result
+            time.sleep(0.2)
+
+    def generate_json(
+        self,
+        method: str,
+        path: str,
+        body: str | None = None,
+        upload: Any = None,
+        progress=gr.Progress(),
+    ) -> str:
+        return json.dumps(
+            self.generate(method, path, body, upload, progress),
+            separators=(",", ":"),
+        )
+
+
+def create_gradio_application():
+    runtime = Runtime(eager_embeddings=False)
+    service = create_app(runtime)
+    bridge = GradioBridge(service, runtime)
+
+    @spaces.GPU(size="large", duration=120)
+    def sam_request(method, path, body=None, upload=None):
+        return bridge.request_json(method, path, body, upload)
+
+    @spaces.GPU(size="xlarge", duration=300)
+    def generation_request(method, path, body=None, upload=None, progress=gr.Progress()):
+        return bridge.generate_json(method, path, body, upload, progress)
+
+    with gr.Blocks(title="What Moves?") as transport:
+        method = gr.Textbox(visible=False)
+        path = gr.Textbox(visible=False)
+        body = gr.Textbox(visible=False)
+        upload = gr.File(type="filepath", visible=False)
+        output = gr.Textbox(visible=False)
+        cpu_trigger = gr.Button(visible=False)
+        sam_trigger = gr.Button(visible=False)
+        generation_trigger = gr.Button(visible=False)
+
+        inputs = [method, path, body, upload]
+        cpu_trigger.click(
+            bridge.request_json,
+            inputs=inputs,
+            outputs=output,
+            api_name="api",
+            queue=False,
+        )
+        sam_trigger.click(
+            sam_request,
+            inputs=inputs,
+            outputs=output,
+            api_name="sam_api",
+            concurrency_limit=1,
+        )
+        generation_trigger.click(
+            generation_request,
+            inputs=inputs,
+            outputs=output,
+            api_name="generation_api",
+            concurrency_limit=1,
+        )
+
+    transport.queue()
+    return gr.mount_gradio_app(service, transport, path="/gradio")

app/inference.py

@@ -0,0 +1,220 @@
+"""Wan input preparation and Torch Hub model loading."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+import threading
+from typing import Any, Callable
+
+import numpy as np
+
+from .config import AppConfig
+from .media import decode_video_range, write_video
+
+
+@dataclass(frozen=True)
+class SourceSnapshot:
+    video_path: Path
+    trim_start: int
+    trim_end: int
+    source_masks: tuple[np.ndarray, ...]
+    target_masks: tuple[np.ndarray, ...]
+    reference_frames: tuple[int, ...]
+
+
+@dataclass(frozen=True)
+class GenerationSnapshot:
+    target_image: np.ndarray
+    sources: tuple[SourceSnapshot, ...]
+    prompt: str
+    negative_prompt: str | None
+    steps: int
+    seed: int
+    guidance_mode: str
+    text_guidance_scale: float
+    motion_guidance_scale: float
+    lora_scale: float
+    output_path: Path
+
+
+class WanService:
+    def __init__(self, config: AppConfig) -> None:
+        self.config = config
+        self._model = None
+        self._status = "idle"
+        self._lock = threading.RLock()
+
+    @property
+    def status(self) -> str:
+        with self._lock:
+            return self._status
+
+    def _set_status(self, status: str) -> None:
+        with self._lock:
+            self._status = status
+
+    def _load_model(self):
+        """Load the release through the same Torch Hub API exposed to users."""
+        if self._model is not None:
+            return self._model
+        self._set_status("loading")
+        try:
+            import torch
+
+            kwargs = {
+                "variant": self.config.model_variant,
+                "device": self.config.device,
+            }
+            if self.config.wan_checkpoint is not None:
+                kwargs["wan_checkpoint"] = str(self.config.wan_checkpoint)
+            self._model = torch.hub.load(
+                str(Path(__file__).resolve().parents[1]),
+                "wan",
+                source="local",
+                **kwargs,
+            )
+        except Exception:
+            self._set_status("error")
+            raise
+        self._set_status("ready")
+        return self._model
+
+    def close(self) -> None:
+        with self._lock:
+            self._model = None
+            self._status = "idle"
+        if self.config.device.startswith("cuda"):
+            import torch
+
+            torch.cuda.empty_cache()
+
+    def generate(
+        self,
+        request: GenerationSnapshot,
+        progress_callback: Callable[[str, int | None, int | None], None] | None = None,
+    ) -> dict[str, Any]:
+        import torch
+        import torch.nn.functional as F
+
+        def report(stage, current=None, total=None):
+            if progress_callback is not None:
+                progress_callback(stage, current, total)
+
+        height = self.config.output_height
+        width = self.config.output_width
+        frames = self.config.output_frames
+
+        report("Preparation")
+        target = torch.from_numpy(request.target_image.copy())
+        target = (
+            F.interpolate(
+                target.permute(2, 0, 1)[None].float(),
+                size=(height, width),
+                mode="bilinear",
+                antialias=True,
+            )
+            .permute(0, 2, 3, 1)
+            .div(127.5)
+            .sub(1)
+        )
+
+        source_videos = []
+        source_masks = []
+        target_masks = []
+        source_content_images = []
+        for source in request.sources:
+            decoded = decode_video_range(
+                source.video_path, source.trim_start, source.trim_end
+            )
+            video = torch.from_numpy(decoded.copy()).permute(0, 3, 1, 2).float()
+            video = (
+                F.interpolate(
+                    video,
+                    size=(height, width),
+                    mode="bilinear",
+                    antialias=True,
+                )
+                .permute(0, 2, 3, 1)
+                .div(127.5)
+                .sub(1)[None]
+            )
+            grouped: dict[int, list[int]] = {}
+            for index, reference in enumerate(source.reference_frames):
+                grouped.setdefault(reference, []).append(index)
+            for reference, indices in grouped.items():
+                local_reference = reference - source.trim_start
+                if not 0 <= local_reference < decoded.shape[0]:
+                    raise ValueError("A source mask lies outside its selected interval")
+                source_mask = torch.from_numpy(
+                    np.stack([source.source_masks[index] for index in indices])
+                ).float()[None]
+                source_mask = F.interpolate(
+                    source_mask, size=(height, width), mode="nearest"
+                ).bool()
+                target_mask = torch.from_numpy(
+                    np.stack([source.target_masks[index] for index in indices])
+                ).float()[None]
+                target_mask = F.interpolate(
+                    target_mask, size=(height, width), mode="nearest"
+                ).bool()
+                reference_image = video[:, local_reference]
+                source_videos.append(video)
+                source_masks.append(source_mask)
+                target_masks.append(target_mask)
+                source_content_images.append(reference_image)
+
+        # Decode and validate every user input before paying the cost of loading
+        # the two Wan experts. This also makes bad frame ranges fail quickly.
+        if self._model is None:
+            report("Model loading")
+        model = self._load_model()
+        kwargs = {
+            "prompt": request.prompt,
+            "target_image": target,
+            "negative_prompt": request.negative_prompt,
+            "num_frames": frames,
+            "num_inference_steps": request.steps,
+            "text_guidance_scale": request.text_guidance_scale,
+            "motion_guidance_scale": request.motion_guidance_scale,
+            "guidance_mode": request.guidance_mode,
+            "lora_scale": request.lora_scale,
+            "seed": request.seed,
+        }
+        if request.guidance_mode != "base_cfg":
+            kwargs.update(
+                source_videos=source_videos,
+                source_masks=source_masks,
+                target_masks=target_masks,
+                source_content_images=source_content_images,
+            )
+        self._set_status("generating")
+        try:
+            with torch.inference_mode():
+                generated = model.sample(
+                    **kwargs,
+                    progress_callback=progress_callback,
+                )
+            video = (
+                generated[0]
+                .detach()
+                .float()
+                .cpu()
+                .clamp(-1, 1)
+                .add(1)
+                .mul(127.5)
+                .round()
+                .byte()
+                .numpy()
+            )
+            report("Finishing")
+            write_video(request.output_path, video, fps=self.config.output_fps)
+            return {
+                "output_name": request.output_path.name,
+                "frames": int(video.shape[0]),
+                "width": int(video.shape[2]),
+                "height": int(video.shape[1]),
+                "fps": self.config.output_fps,
+            }
+        finally:
+            self._set_status("ready")

app/media.py

@@ -0,0 +1,293 @@
+"""Small image, mask, and video I/O helpers."""
+
+from __future__ import annotations
+
+from fractions import Fraction
+from io import BytesIO
+from pathlib import Path
+
+import av
+import numpy as np
+from PIL import Image, ImageOps
+
+
+def read_image(path: Path, max_side: int = 1600) -> np.ndarray:
+    with Image.open(path) as opened:
+        image = ImageOps.exif_transpose(opened).convert("RGB")
+        if max(image.size) > max_side:
+            image.thumbnail((max_side, max_side), Image.Resampling.LANCZOS)
+        return np.asarray(image).copy()
+
+
+def png_bytes(image: np.ndarray) -> bytes:
+    buffer = BytesIO()
+    mode = "L" if image.ndim == 2 else "RGB"
+    Image.fromarray(image, mode=mode).save(buffer, format="PNG", optimize=True)
+    return buffer.getvalue()
+
+
+def jpeg_bytes(image: np.ndarray, quality: int = 82) -> bytes:
+    buffer = BytesIO()
+    Image.fromarray(np.asarray(image, dtype=np.uint8), mode="RGB").save(
+        buffer,
+        format="JPEG",
+        quality=int(quality),
+        optimize=True,
+    )
+    return buffer.getvalue()
+
+
+def mask_png_bytes(mask: np.ndarray) -> bytes:
+    return png_bytes(np.asarray(mask, dtype=np.uint8) * 255)
+
+
+def _rate(stream) -> float:
+    value = stream.average_rate or stream.guessed_rate
+    try:
+        rate = float(value)
+    except (TypeError, ValueError, ZeroDivisionError):
+        rate = 0.0
+    return rate if np.isfinite(rate) and rate > 0 else 8.0
+
+
+def probe_video(path: Path) -> tuple[float, int | None]:
+    with av.open(str(path)) as container:
+        stream = container.streams.video[0]
+        fps = _rate(stream)
+        count = int(stream.frames) if stream.frames else None
+        if (
+            count is None
+            and stream.duration is not None
+            and stream.time_base is not None
+        ):
+            count = max(1, int(round(float(stream.duration * stream.time_base) * fps)))
+        return fps, count
+
+
+def decode_video_frame(path: Path, index: int, max_side: int = 1600) -> np.ndarray:
+    index = max(0, int(index))
+    with av.open(str(path)) as container:
+        stream = container.streams.video[0]
+        fps = _rate(stream)
+        if index and stream.time_base is not None:
+            timestamp = int((index / fps) / float(stream.time_base))
+            container.seek(timestamp, stream=stream, any_frame=False, backward=True)
+        closest = None
+        closest_distance = float("inf")
+        for decoded_index, frame in enumerate(container.decode(stream)):
+            if frame.pts is not None and stream.time_base is not None:
+                frame_index = round(float(frame.pts * stream.time_base) * fps)
+            else:
+                frame_index = decoded_index
+            distance = abs(frame_index - index)
+            if distance < closest_distance:
+                closest = frame
+                closest_distance = distance
+            if frame_index < index:
+                continue
+            frame = closest
+            image = Image.fromarray(frame.to_ndarray(format="rgb24"))
+            if max(image.size) > max_side:
+                image.thumbnail((max_side, max_side), Image.Resampling.LANCZOS)
+            return np.asarray(image).copy()
+    raise ValueError(f"Video has no frame {index}")
+
+
+def _resized_rgb(array: np.ndarray, max_side: int, size=None) -> np.ndarray:
+    image = Image.fromarray(array, mode="RGB")
+    if size is None:
+        if max(image.size) > max_side:
+            image.thumbnail((max_side, max_side), Image.Resampling.LANCZOS)
+        width = max(2, image.width - image.width % 2)
+        height = max(2, image.height - image.height % 2)
+        size = (width, height)
+    if image.size != size:
+        image = image.resize(size, Image.Resampling.LANCZOS)
+    return np.asarray(image).copy()
+
+
+def resample_video(
+    source_path: Path,
+    output_path: Path,
+    fps: int = 8,
+    max_side: int = 1600,
+) -> tuple[float, int, np.ndarray]:
+    """Create the exact, nearest-timestamp source sequence shown by the app."""
+    target_fps = int(fps)
+    if target_fps < 1:
+        raise ValueError("Target FPS must be positive")
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    first_output = None
+    output_count = 0
+    try:
+        with av.open(str(source_path)) as input_container:
+            input_stream = input_container.streams.video[0]
+            original_fps = _rate(input_stream)
+            with av.open(str(output_path), mode="w") as output_container:
+                try:
+                    output_stream = output_container.add_stream(
+                        "libx264", rate=Fraction(target_fps, 1)
+                    )
+                except av.codec.codec.UnknownCodecError:
+                    output_stream = output_container.add_stream(
+                        "mpeg4", rate=Fraction(target_fps, 1)
+                    )
+                output_stream.pix_fmt = "yuv420p"
+                output_stream.options = {"crf": "18", "preset": "veryfast"}
+                size = None
+                previous = None
+                previous_time = 0.0
+                first_time = None
+                next_time = 0.0
+                source_index = 0
+
+                def emit(array):
+                    nonlocal first_output, output_count, size
+                    resized = _resized_rgb(array, max_side, size)
+                    if size is None:
+                        size = (int(resized.shape[1]), int(resized.shape[0]))
+                        output_stream.width, output_stream.height = size
+                    if first_output is None:
+                        first_output = resized.copy()
+                    encoded = av.VideoFrame.from_ndarray(resized, format="rgb24")
+                    for packet in output_stream.encode(encoded):
+                        output_container.mux(packet)
+                    output_count += 1
+
+                for frame in input_container.decode(input_stream):
+                    raw_time = frame.time
+                    if raw_time is None:
+                        raw_time = source_index / original_fps
+                    raw_time = float(raw_time)
+                    if first_time is None:
+                        first_time = raw_time
+                    timestamp = max(0.0, raw_time - first_time)
+                    if previous is not None:
+                        timestamp = max(previous_time, timestamp)
+                    array = frame.to_ndarray(format="rgb24")
+                    if previous is not None:
+                        boundary = (previous_time + timestamp) / 2
+                        while next_time < boundary:
+                            emit(previous)
+                            next_time += 1 / target_fps
+                    previous = array
+                    previous_time = timestamp
+                    source_index += 1
+                if previous is None:
+                    raise ValueError("Video contains no decodable frames")
+                end_time = previous_time + 0.5 / original_fps
+                while next_time < end_time or output_count == 0:
+                    emit(previous)
+                    next_time += 1 / target_fps
+                for packet in output_stream.encode():
+                    output_container.mux(packet)
+        output_path.chmod(0o600)
+        return original_fps, output_count, first_output
+    except Exception:
+        output_path.unlink(missing_ok=True)
+        raise
+
+
+def decode_video_range(path: Path, start: int, end: int) -> np.ndarray:
+    start = max(0, int(start))
+    end = int(end)
+    if end < start:
+        raise ValueError("Video range end precedes its start")
+    wanted = end - start + 1
+    frames = []
+    with av.open(str(path)) as container:
+        stream = container.streams.video[0]
+        for index, frame in enumerate(container.decode(stream)):
+            if index < start:
+                continue
+            if index > end:
+                break
+            frames.append(frame.to_ndarray(format="rgb24"))
+    if len(frames) != wanted:
+        raise ValueError(
+            f"Source contains only {len(frames)} of frames {start} through {end}"
+        )
+    return np.stack(frames)
+
+
+def decode_sampled_video(path: Path, start: int, stride: int, count: int) -> np.ndarray:
+    start = max(0, int(start))
+    stride = max(1, int(stride))
+    wanted = {start + stride * index: index for index in range(int(count))}
+    frames: list[np.ndarray | None] = [None] * int(count)
+    final_index = max(wanted)
+    with av.open(str(path)) as container:
+        stream = container.streams.video[0]
+        for frame_index, frame in enumerate(container.decode(stream)):
+            slot = wanted.get(frame_index)
+            if slot is not None:
+                frames[slot] = frame.to_ndarray(format="rgb24")
+            if frame_index >= final_index:
+                break
+    available = sum(frame is not None for frame in frames)
+    if available != count:
+        raise ValueError(
+            f"Source contains only {available} of the requested {count} sampled frames "
+            f"(start={start}, stride={stride})"
+        )
+    return np.stack(frames)
+
+
+def _hex_rgb(color: str) -> tuple[int, int, int]:
+    value = color.lstrip("#")
+    return tuple(int(value[index : index + 2], 16) for index in (0, 2, 4))
+
+
+def composite_masks(
+    image: np.ndarray,
+    masks: list[tuple[np.ndarray, str]],
+    *,
+    max_size: tuple[int, int] | None = None,
+) -> np.ndarray:
+    output = image.astype(np.float32).copy()
+    for mask, color in masks:
+        mask = np.asarray(mask, dtype=bool)
+        if mask.shape != output.shape[:2]:
+            mask = np.asarray(
+                Image.fromarray(mask.astype(np.uint8), mode="L").resize(
+                    (output.shape[1], output.shape[0]),
+                    Image.Resampling.NEAREST,
+                ),
+                dtype=bool,
+            )
+        rgb = np.asarray(_hex_rgb(color), dtype=np.float32)
+        output[mask] = output[mask] * 0.67 + rgb * 0.33
+        edge = mask & ~(
+            np.roll(mask, 1, 0)
+            & np.roll(mask, -1, 0)
+            & np.roll(mask, 1, 1)
+            & np.roll(mask, -1, 1)
+        )
+        output[edge] = rgb
+    rendered = Image.fromarray(np.clip(output, 0, 255).astype(np.uint8))
+    if max_size is not None:
+        rendered.thumbnail(max_size, Image.Resampling.LANCZOS)
+    return np.asarray(rendered).copy()
+
+
+def write_video(path: Path, frames: np.ndarray, fps: int = 8) -> None:
+    frames = np.asarray(frames, dtype=np.uint8)
+    if frames.ndim != 4 or frames.shape[-1] != 3:
+        raise ValueError("Video frames must be [T,H,W,3] uint8")
+    path.parent.mkdir(parents=True, exist_ok=True)
+    with av.open(str(path), mode="w") as container:
+        try:
+            stream = container.add_stream("libx264", rate=Fraction(int(fps), 1))
+        except av.codec.codec.UnknownCodecError:
+            stream = container.add_stream("mpeg4", rate=Fraction(int(fps), 1))
+        stream.width = int(frames.shape[2])
+        stream.height = int(frames.shape[1])
+        stream.pix_fmt = "yuv420p"
+        stream.options = {"crf": "18"}
+        for array in frames:
+            frame = av.VideoFrame.from_ndarray(array, format="rgb24")
+            for packet in stream.encode(frame):
+                container.mux(packet)
+        for packet in stream.encode():
+            container.mux(packet)
+    path.chmod(0o600)

app/requirements.txt

@@ -0,0 +1,14 @@
+-r ../requirements-wan.txt
+torchvision==0.23.0
+tqdm==4.67.1
+hydra-core==1.3.2
+iopath==0.1.10
+fastapi==0.115.12
+uvicorn[standard]==0.34.0
+python-multipart==0.0.20
+httpx==0.28.1
+gradio==5.49.1
+spaces==0.51.3
+av==15.0.0
+pillow==11.3.0
+SAM-2 @ git+https://github.com/facebookresearch/sam2.git@2b90b9f5ceec907a1c18123530e92e794ad901a4

app/sam_service.py

@@ -0,0 +1,115 @@
+"""Lazy shared SAM2 model with one cached predictor per uploaded asset."""
+
+from __future__ import annotations
+
+from contextlib import nullcontext
+from collections import OrderedDict
+import threading
+
+import numpy as np
+
+
+class SamService:
+    def __init__(
+        self, model_id: str, device: str = "cuda", max_predictors: int = 8
+    ) -> None:
+        self.model_id = model_id
+        self.device = device
+        self._model = None
+        self.max_predictors = max(1, int(max_predictors))
+        self._predictors: OrderedDict[str, object] = OrderedDict()
+        self._lock = threading.RLock()
+
+    def has(self, key: str) -> bool:
+        with self._lock:
+            present = key in self._predictors
+            if present:
+                self._predictors.move_to_end(key)
+            return present
+
+    def prepare(self, key: str, image: np.ndarray) -> None:
+        with self._lock:
+            if key in self._predictors:
+                return
+            from sam2.sam2_image_predictor import SAM2ImagePredictor
+
+            if self._model is None:
+                predictor = SAM2ImagePredictor.from_pretrained(
+                    self.model_id, device=self.device
+                )
+                self._model = predictor.model
+            else:
+                predictor = SAM2ImagePredictor(self._model)
+            with self._autocast():
+                predictor.set_image(np.asarray(image, dtype=np.uint8))
+            self._predictors[key] = predictor
+            self._predictors.move_to_end(key)
+            while len(self._predictors) > self.max_predictors:
+                _, evicted = self._predictors.popitem(last=False)
+                evicted.reset_predictor()
+
+    def predict(
+        self,
+        key: str,
+        image: np.ndarray,
+        positive: list[list[float]],
+        negative: list[list[float]],
+        box: list[float] | None,
+    ) -> tuple[np.ndarray, float]:
+        with self._lock:
+            self.prepare(key, image)
+            predictor = self._predictors[key]
+            self._predictors.move_to_end(key)
+            points = positive + negative
+            point_coords = None
+            point_labels = None
+            if points:
+                height, width = image.shape[:2]
+                point_coords = np.asarray(
+                    [[point[0] * width, point[1] * height] for point in points],
+                    dtype=np.float32,
+                )
+                point_labels = np.asarray(
+                    [1] * len(positive) + [0] * len(negative), dtype=np.int32
+                )
+            pixel_box = None
+            if box is not None:
+                height, width = image.shape[:2]
+                pixel_box = np.asarray(
+                    [
+                        box[0] * width,
+                        box[1] * height,
+                        box[2] * width,
+                        box[3] * height,
+                    ],
+                    dtype=np.float32,
+                )
+            with self._autocast():
+                masks, scores, _ = predictor.predict(
+                    point_coords=point_coords,
+                    point_labels=point_labels,
+                    box=pixel_box,
+                    multimask_output=True,
+                )
+            best = int(np.argmax(scores))
+            return np.asarray(masks[best], dtype=bool), float(scores[best])
+
+    def release(self, prefix: str) -> None:
+        with self._lock:
+            for key in [key for key in self._predictors if key.startswith(prefix)]:
+                predictor = self._predictors.pop(key)
+                predictor.reset_predictor()
+
+    def close(self) -> None:
+        with self._lock:
+            for predictor in self._predictors.values():
+                predictor.reset_predictor()
+            self._predictors.clear()
+            self._model = None
+
+    def _autocast(self):
+        if not self.device.startswith("cuda"):
+            return nullcontext()
+        import torch
+
+        return torch.autocast("cuda", dtype=torch.bfloat16)

app/scheduler.py

@@ -0,0 +1,298 @@
+"""A bounded serial scheduler for stateful GPU inference."""
+
+from __future__ import annotations
+
+from concurrent.futures import Future
+from dataclasses import dataclass, field, replace
+import itertools
+import queue
+import threading
+import time
+import traceback
+from typing import Any, Callable
+from uuid import uuid4
+
+
+class Superseded(RuntimeError):
+    """Raised when queued work is replaced or its owner is removed."""
+
+
+class QueueFull(RuntimeError):
+    """Raised when the bounded GPU queue has no available slot."""
+
+
+@dataclass(order=True)
+class _QueuedTask:
+    priority: int
+    order: int
+    task_id: str = field(compare=False)
+    function: Callable[[], Any] | None = field(compare=False)
+    future: Future = field(compare=False)
+    key: str | None = field(compare=False, default=None)
+    owner: str | None = field(compare=False, default=None)
+    retain_record: bool = field(compare=False, default=False)
+
+
+@dataclass
+class TaskRecord:
+    id: str
+    label: str
+    status: str = "queued"
+    result: Any = None
+    error: str | None = None
+    created_at: float = field(default_factory=time.time)
+    started_at: float | None = None
+    finished_at: float | None = None
+    stage: str = "Queued"
+    progress_current: int | None = None
+    progress_total: int | None = None
+    message: str | None = None
+
+    def as_dict(self) -> dict[str, Any]:
+        return {
+            "id": self.id,
+            "label": self.label,
+            "status": self.status,
+            "result": self.result,
+            "error": self.error,
+            "created_at": self.created_at,
+            "started_at": self.started_at,
+            "finished_at": self.finished_at,
+            "stage": self.stage,
+            "progress_current": self.progress_current,
+            "progress_total": self.progress_total,
+            "message": self.message,
+        }
+
+
+class GpuScheduler:
+    """Run GPU work serially and release task payloads as soon as possible."""
+
+    def __init__(self, max_pending: int = 32) -> None:
+        if max_pending < 1:
+            raise ValueError("max_pending must be positive")
+        self.max_pending = int(max_pending)
+        self._queue: queue.PriorityQueue[_QueuedTask] = queue.PriorityQueue()
+        self._counter = itertools.count()
+        self._lock = threading.RLock()
+        self._latest_by_key: dict[str, str] = {}
+        self._queued: dict[str, _QueuedTask] = {}
+        self._records: dict[str, TaskRecord] = {}
+        self._closed = False
+        self._thread = threading.Thread(
+            target=self._run,
+            name="whatmoves-gpu",
+            daemon=True,
+        )
+        self._thread.start()
+
+    def submit(
+        self,
+        function: Callable[[], Any],
+        *,
+        priority: int = 10,
+        key: str | None = None,
+        owner: str | None = None,
+        label: str = "GPU task",
+        retain_record: bool = False,
+        task_id: str | None = None,
+    ) -> tuple[str, Future]:
+        future: Future = Future()
+        task_id = task_id or uuid4().hex
+        cancelled: tuple[Future, str] | None = None
+        with self._lock:
+            if self._closed:
+                raise RuntimeError("GPU scheduler is closed")
+            if task_id in self._records:
+                raise ValueError(f"Duplicate GPU task id: {task_id}")
+            if key is not None:
+                previous_id = self._latest_by_key.get(key)
+                if previous_id is not None:
+                    cancelled = self._cancel_locked(
+                        previous_id,
+                        f"Superseded by newer task: {label}",
+                    )
+            pending = sum(
+                record.status in {"queued", "running"}
+                for record in self._records.values()
+            )
+            if pending >= self.max_pending:
+                raise QueueFull(f"GPU queue is full ({self.max_pending} pending tasks)")
+            record = TaskRecord(task_id, label)
+            task = _QueuedTask(
+                int(priority),
+                next(self._counter),
+                task_id,
+                function,
+                future,
+                key,
+                owner,
+                bool(retain_record),
+            )
+            self._records[task_id] = record
+            self._queued[task_id] = task
+            if key is not None:
+                self._latest_by_key[key] = task_id
+            self._queue.put(task)
+        if cancelled is not None:
+            cancelled_future, reason = cancelled
+            if not cancelled_future.done():
+                cancelled_future.set_exception(Superseded(reason))
+        return task_id, future
+
+    def update_progress(
+        self,
+        task_id: str,
+        stage: str,
+        current: int | None = None,
+        total: int | None = None,
+        message: str | None = None,
+    ) -> None:
+        """Publish honest stage progress from the worker that owns a task."""
+        with self._lock:
+            record = self._records.get(task_id)
+            if record is None or record.status not in {"queued", "running"}:
+                return
+            record.stage = str(stage)
+            record.progress_current = None if current is None else int(current)
+            record.progress_total = None if total is None else int(total)
+            record.message = message
+
+    def get(self, task_id: str) -> TaskRecord | None:
+        with self._lock:
+            record = self._records.get(task_id)
+            return replace(record) if record is not None else None
+
+    def discard(self, task_id: str) -> None:
+        """Forget a terminal retained record and its result."""
+        with self._lock:
+            record = self._records.get(task_id)
+            if record is not None and record.status not in {"queued", "running"}:
+                self._records.pop(task_id, None)
+
+    def cancel_prefix(self, prefix: str) -> None:
+        """Cancel queued tasks whose coalescing key starts with ``prefix``."""
+        cancelled: list[tuple[Future, str]] = []
+        with self._lock:
+            for task_id, task in tuple(self._queued.items()):
+                if task.key is not None and task.key.startswith(prefix):
+                    item = self._cancel_locked(task_id, "Owning asset was removed")
+                    if item is not None:
+                        cancelled.append(item)
+        for future, reason in cancelled:
+            if not future.done():
+                future.set_exception(Superseded(reason))
+
+    def cancel_owner(self, owner: str) -> None:
+        """Cancel all queued tasks belonging to one browser session."""
+        cancelled: list[tuple[Future, str]] = []
+        with self._lock:
+            for task_id, task in tuple(self._queued.items()):
+                if task.owner == owner:
+                    item = self._cancel_locked(task_id, "Owning session expired")
+                    if item is not None:
+                        cancelled.append(item)
+        for future, reason in cancelled:
+            if not future.done():
+                future.set_exception(Superseded(reason))
+
+    def close(self) -> None:
+        cancelled: list[tuple[Future, str]] = []
+        with self._lock:
+            if self._closed:
+                return
+            self._closed = True
+            for task_id in tuple(self._queued):
+                item = self._cancel_locked(task_id, "GPU scheduler is closing")
+                if item is not None:
+                    cancelled.append(item)
+            sentinel = _QueuedTask(
+                -(10**9),
+                next(self._counter),
+                "__close__",
+                None,
+                Future(),
+            )
+            self._queue.put(sentinel)
+        for future, reason in cancelled:
+            if not future.done():
+                future.set_exception(Superseded(reason))
+        self._thread.join()
+        with self._lock:
+            with self._queue.mutex:
+                self._queue.queue.clear()
+            self._queued.clear()
+            self._latest_by_key.clear()
+            self._records.clear()
+
+    def _cancel_locked(
+        self,
+        task_id: str,
+        reason: str,
+    ) -> tuple[Future, str] | None:
+        record = self._records.get(task_id)
+        task = self._queued.get(task_id)
+        if record is None or task is None or record.status != "queued":
+            return None
+        task.function = None
+        record.status = "superseded"
+        record.error = reason
+        record.finished_at = time.time()
+        if task.key is not None and self._latest_by_key.get(task.key) == task_id:
+            self._latest_by_key.pop(task.key, None)
+        return task.future, reason
+
+    def _run(self) -> None:
+        while True:
+            task = self._queue.get()
+            if task.task_id == "__close__":
+                del task
+                return
+            with self._lock:
+                self._queued.pop(task.task_id, None)
+                record = self._records.get(task.task_id)
+                function = task.function
+                task.function = None
+                if record is None or function is None:
+                    if not task.retain_record:
+                        self._records.pop(task.task_id, None)
+                    del task
+                    continue
+                record.status = "running"
+                record.started_at = time.time()
+                if record.stage == "Queued":
+                    record.stage = "Starting"
+            try:
+                result = function()
+            except Exception as error:  # worker errors must never kill the queue
+                traceback.print_exc()
+                with self._lock:
+                    record.status = "failed"
+                    record.error = str(error)
+                    record.finished_at = time.time()
+                    if (
+                        task.key is not None
+                        and self._latest_by_key.get(task.key) == task.task_id
+                    ):
+                        self._latest_by_key.pop(task.key, None)
+                if not task.future.done():
+                    task.future.set_exception(error)
+            else:
+                with self._lock:
+                    record.status = "complete"
+                    record.result = result
+                    record.finished_at = time.time()
+                    if (
+                        task.key is not None
+                        and self._latest_by_key.get(task.key) == task.task_id
+                    ):
+                        self._latest_by_key.pop(task.key, None)
+                if not task.future.done():
+                    task.future.set_result(result)
+                del result
+            finally:
+                del function
+                with self._lock:
+                    if not task.retain_record:
+                        self._records.pop(task.task_id, None)
+                del task

app/server.py

@@ -0,0 +1,1275 @@
+"""FastAPI server for the WhatMoves annotation and generation workspace."""
+
+from __future__ import annotations
+
+import asyncio
+import base64
+from concurrent.futures import ThreadPoolExecutor
+from contextlib import asynccontextmanager
+import os
+from pathlib import Path
+import shutil
+import threading
+from typing import Literal
+
+from fastapi import FastAPI, File, HTTPException, UploadFile
+from fastapi.responses import FileResponse, HTMLResponse, Response
+from fastapi.staticfiles import StaticFiles
+from pydantic import BaseModel, Field
+
+from .config import AppConfig
+from .examples import load_examples
+from .inference import GenerationSnapshot, SourceSnapshot, WanService
+from .media import (
+    composite_masks,
+    decode_video_frame,
+    jpeg_bytes,
+    mask_png_bytes,
+    png_bytes,
+    read_image,
+    resample_video,
+)
+from .sam_service import SamService
+from .scheduler import GpuScheduler, QueueFull, Superseded
+from .state import (
+    Draft,
+    MaskRecord,
+    Session,
+    SessionStore,
+    SourceAsset,
+    TargetAsset,
+    clear_excluded_source_mappings,
+    clear_source_mappings,
+    new_id,
+    session_payload,
+    source_mask_lookup,
+)
+
+STATIC_ROOT = Path(__file__).resolve().parent / "static"
+ASSETS_ROOT = Path(__file__).resolve().parent / "assets"
+
+
+class PromptRequest(BaseModel):
+    asset_revision: int
+    prompt_revision: int
+    positive: list[list[float]] = Field(default_factory=list)
+    negative: list[list[float]] = Field(default_factory=list)
+    box: list[float] | None = None
+    frame_index: int | None = Field(default=None, ge=0)
+    transient: bool = False
+
+
+class CommitRequest(BaseModel):
+    asset_revision: int
+    prompt_revision: int
+    frame_index: int | None = Field(default=None, ge=0)
+
+
+class SourceSettings(BaseModel):
+    current_frame: int | None = Field(default=None, ge=0)
+    trim_start: int | None = Field(default=None, ge=0)
+    trim_end: int | None = Field(default=None, ge=0)
+
+
+class MappingRequest(BaseModel):
+    source_id: str | None = None
+    source_mask_id: str | None = None
+
+
+class GenerateRequest(BaseModel):
+    prompt: str = Field(default="", max_length=4096)
+    negative_prompt: str | None = Field(default=None, max_length=4096)
+    steps: int = Field(default=40, ge=1, le=100)
+    seed: int = Field(default=42, ge=-(2**63), le=2**63 - 1)
+    guidance_mode: Literal[
+        "base_cfg",
+        "text_cfg",
+        "joint_cfg",
+        "motion_cfg",
+        "additive_cfg",
+        "factorized_cfg",
+    ] = "text_cfg"
+    text_guidance_scale: float = Field(default=3.5, allow_inf_nan=False)
+    motion_guidance_scale: float = Field(default=1.0, allow_inf_nan=False)
+    lora_scale: float = Field(default=1.0, allow_inf_nan=False)
+
+
+class Runtime:
+    def __init__(
+        self,
+        config: AppConfig | None = None,
+        *,
+        eager_embeddings: bool = True,
+    ) -> None:
+        self.config = config or AppConfig()
+        self.eager_embeddings = eager_embeddings
+        self.store = SessionStore(self.config.runtime_root)
+        self.cpu = ThreadPoolExecutor(max_workers=4, thread_name_prefix="whatmoves-cpu")
+        self.scheduler = GpuScheduler(max_pending=self.config.max_gpu_tasks)
+        self.sam = SamService(
+            self.config.sam_model_id,
+            self.config.device,
+            max_predictors=self.config.max_sam_frames,
+        )
+        self.wan = WanService(self.config)
+        self._closing = threading.Event()
+        self._cleanup_thread = threading.Thread(
+            target=self._cleanup_sessions,
+            name="whatmoves-cleanup",
+            daemon=True,
+        )
+        self._cleanup_thread.start()
+
+    def close(self) -> None:
+        if self._closing.is_set():
+            return
+        self._closing.set()
+        self._cleanup_thread.join()
+        sessions = self.store.pop_all()
+        for session in sessions:
+            self.dispose_session(session)
+        self.scheduler.close()
+        for session in sessions:
+            _remove_session_directory(self.config.runtime_root, session.directory)
+        self.sam.close()
+        self.wan.close()
+        self.cpu.shutdown(wait=False, cancel_futures=True)
+        try:
+            self.config.runtime_root.rmdir()
+        except OSError:
+            pass
+
+    def dispose_session(self, session: Session) -> None:
+        """Release every live resource owned by one browser session."""
+        with session.lock:
+            if session.closed:
+                return
+            session.closed = True
+            targets = list(session.targets.values())
+            sources = list(session.sources.values())
+            latest_task = session.latest_generation
+            failed_task = session.failed_generation
+            latest_output = session.latest_output
+            active_task = session.active_generation
+            session.targets.clear()
+            session.selected_target_id = None
+            session.sources.clear()
+            session.latest_generation = None
+            session.failed_generation = None
+            session.latest_output = None
+        self.scheduler.cancel_owner(session.id)
+        self.sam.release(f"{session.id}:")
+        for target in targets:
+            _clear_asset(target)
+        for source in sources:
+            _clear_asset(source)
+            source.video_path.unlink(missing_ok=True)
+        if latest_output is not None:
+            latest_output.unlink(missing_ok=True)
+        if latest_task is not None and latest_task != active_task:
+            self.scheduler.discard(latest_task)
+        if failed_task is not None and failed_task != active_task:
+            self.scheduler.discard(failed_task)
+        active = self.scheduler.get(active_task) if active_task is not None else None
+        if active is None or active.status != "running":
+            if active_task is not None:
+                self.scheduler.discard(active_task)
+            _remove_session_directory(self.config.runtime_root, session.directory)
+
+    def _cleanup_sessions(self) -> None:
+        while not self._closing.wait(self.config.cleanup_interval_seconds):
+            for session in self.store.pop_expired(self.config.session_ttl_seconds):
+                self.dispose_session(session)
+
+
+async def _run_cpu(runtime: Runtime, function, *args, **kwargs):
+    # CPU-only is the lightweight test/development mode; production CUDA runs
+    # media work in the dedicated pool so the HTTP event loop stays responsive.
+    if runtime.config.device == "cpu":
+        return function(*args, **kwargs)
+    return await asyncio.wrap_future(runtime.cpu.submit(function, *args, **kwargs))
+
+
+def _session(runtime: Runtime, session_id: str) -> Session:
+    try:
+        return runtime.store.get(session_id)
+    except KeyError as error:
+        raise HTTPException(404, str(error)) from error
+
+
+def _remove_session_directory(root: Path, directory: Path) -> None:
+    """Remove exactly one validated session directory."""
+    root = root.resolve()
+    directory = directory.resolve()
+    if directory.parent != root or not directory.name:
+        raise RuntimeError(f"Refusing to remove invalid session path: {directory}")
+    shutil.rmtree(directory, ignore_errors=True)
+
+
+def _clear_asset(asset: TargetAsset | SourceAsset) -> None:
+    asset.draft = None
+    asset.prediction_token = None
+    asset.hover_prediction_token = None
+    if isinstance(asset, SourceAsset):
+        asset.frame_selection_token = None
+    asset.masks.clear()
+
+
+def _sam_prefix(session_id: str, kind: str, asset_id: str) -> str:
+    return f"{session_id}:{kind}:{asset_id}:"
+
+
+def _release_asset(runtime: Runtime, session_id: str, kind: str, asset_id: str) -> None:
+    prefix = _sam_prefix(session_id, kind, asset_id)
+    runtime.scheduler.cancel_prefix(f"sam:{prefix}")
+    runtime.sam.release(prefix)
+
+
+def _asset(session: Session, kind: str, asset_id: str):
+    if kind == "target":
+        try:
+            return session.targets[asset_id]
+        except KeyError as error:
+            raise HTTPException(404, f"Unknown target {asset_id}") from error
+    if kind == "source":
+        try:
+            return session.sources[asset_id]
+        except KeyError as error:
+            raise HTTPException(404, f"Unknown source {asset_id}") from error
+    raise HTTPException(404, f"Unknown asset kind {kind}")
+
+
+def _sam_key(
+    session_id: str,
+    kind: str,
+    asset_id: str,
+    revision: int,
+    frame_index: int | None = None,
+) -> str:
+    frame = "image" if frame_index is None else f"frame-{frame_index}"
+    return f"{session_id}:{kind}:{asset_id}:{revision}:{frame}"
+
+
+def _store_upload_sync(upload: UploadFile, path: Path, limit: int) -> None:
+    written = 0
+    try:
+        descriptor = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
+        with os.fdopen(descriptor, "wb") as output:
+            while chunk := upload.file.read(1024 * 1024):
+                written += len(chunk)
+                if written > limit:
+                    raise HTTPException(413, "Upload is too large")
+                output.write(chunk)
+    except Exception:
+        path.unlink(missing_ok=True)
+        raise
+
+
+def _queue_embedding(
+    runtime: Runtime,
+    session: Session,
+    kind: str,
+    asset_id: str,
+    revision: int,
+    frame_index: int | None = None,
+) -> None:
+    # ZeroGPU only exposes a real CUDA device while a @spaces.GPU function is
+    # active. In Gradio transport mode SAM prepares its embedding lazily inside
+    # the decorated prediction call instead of starting background CUDA work.
+    if not runtime.eager_embeddings:
+        with session.lock:
+            try:
+                asset = _asset(session, kind, asset_id)
+            except HTTPException:
+                return
+            if asset.revision == revision and (
+                kind == "target" or asset.current_frame == frame_index
+            ):
+                asset.sam_status = "ready"
+                asset.sam_error = None
+        return
+
+    key = _sam_key(session.id, kind, asset_id, revision, frame_index)
+
+    if runtime.sam.has(key):
+        with session.lock:
+            try:
+                asset = _asset(session, kind, asset_id)
+            except HTTPException:
+                return
+            if asset.revision == revision and (
+                kind == "target" or asset.current_frame == frame_index
+            ):
+                asset.sam_status = "ready"
+                asset.sam_error = None
+        return
+
+    def prepare():
+        with session.lock:
+            try:
+                current = _asset(session, kind, asset_id)
+            except HTTPException:
+                return None
+            if (
+                session.closed
+                or current.revision != revision
+                or (kind == "source" and current.current_frame != frame_index)
+            ):
+                return None
+            image = current.image if kind == "target" else current.frame
+        try:
+            runtime.sam.prepare(key, image)
+        except Exception as error:
+            with session.lock:
+                try:
+                    asset = _asset(session, kind, asset_id)
+                except HTTPException:
+                    return None
+                if asset.revision == revision and (
+                    kind == "target" or asset.current_frame == frame_index
+                ):
+                    asset.sam_status = "error"
+                    asset.sam_error = str(error)
+            raise
+        keep = False
+        with session.lock:
+            try:
+                asset = _asset(session, kind, asset_id)
+            except HTTPException:
+                asset = None
+            if (
+                asset is not None
+                and asset.revision == revision
+                and (kind == "target" or asset.current_frame == frame_index)
+            ):
+                asset.sam_status = "ready"
+                asset.sam_error = None
+                keep = True
+        if not keep:
+            runtime.sam.release(key)
+        return None
+
+    try:
+        runtime.scheduler.submit(
+            prepare,
+            priority=20,
+            key=f"sam:{key}:embedding",
+            owner=session.id,
+            label=f"Prepare {kind} mask features",
+        )
+    except QueueFull as error:
+        with session.lock:
+            try:
+                asset = _asset(session, kind, asset_id)
+            except HTTPException:
+                return
+            if asset.revision == revision:
+                asset.sam_status = "error"
+                asset.sam_error = str(error)
+
+
+def _state(runtime: Runtime, session: Session):
+    with session.lock:
+        payload = session_payload(session)
+    payload["wan_status"] = runtime.wan.status
+    payload["defaults"] = {
+        "frames": runtime.config.output_frames,
+        "width": runtime.config.output_width,
+        "height": runtime.config.output_height,
+        "fps": runtime.config.output_fps,
+        "source_fps": runtime.config.source_fps,
+        "minimum_source_frames": 8,
+    }
+    return payload
+
+
+def _validate_prompts(request: PromptRequest) -> None:
+    if not request.positive and not request.negative and request.box is None:
+        raise HTTPException(400, "At least one point or box is required")
+    for point in request.positive + request.negative:
+        if len(point) != 2 or not all(0 <= float(value) <= 1 for value in point):
+            raise HTTPException(400, "Prompt points must be normalized [x,y] pairs")
+    if request.box is not None:
+        if len(request.box) != 4 or not all(
+            0 <= float(value) <= 1 for value in request.box
+        ):
+            raise HTTPException(400, "Box must be normalized [x0,y0,x1,y1]")
+        if request.box[0] >= request.box[2] or request.box[1] >= request.box[3]:
+            raise HTTPException(400, "Box must have positive width and height")
+
+
+def _generation_snapshot(
+    runtime: Runtime, session: Session, request: GenerateRequest
+) -> GenerationSnapshot:
+    target = session.selected_target
+    if target is None:
+        raise HTTPException(400, "Upload a target image first")
+    grouped: dict[str, list[tuple[MaskRecord, MaskRecord]]] = {}
+    for target_mask in target.masks.values():
+        if target_mask.source_id is None or target_mask.source_mask_id is None:
+            continue
+        try:
+            source_mask = source_mask_lookup(
+                session, target_mask.source_id, target_mask.source_mask_id
+            )
+        except KeyError:
+            continue
+        grouped.setdefault(target_mask.source_id, []).append((source_mask, target_mask))
+    if request.guidance_mode == "base_cfg":
+        grouped.clear()
+    elif not grouped:
+        raise HTTPException(400, "Map at least one source mask to a target mask")
+
+    run_id = new_id("run")
+    output_path = session.directory / f"generation_{run_id}.mp4"
+    sources = []
+    try:
+        for index, (source_id, pairs) in enumerate(grouped.items()):
+            source = session.sources[source_id]
+            selected_frames = source.trim_end - source.trim_start + 1
+            if selected_frames < 8:
+                raise HTTPException(
+                    400,
+                    f"{source.name} needs at least 8 selected 8-fps frames",
+                )
+            if any(
+                mask.frame_index is None
+                or not source.trim_start <= mask.frame_index <= source.trim_end
+                for mask, _ in pairs
+            ):
+                raise HTTPException(409, "A mapped source mask lies outside its trim")
+            private_path = session.directory / (
+                f"input_{run_id}_{index}{source.video_path.suffix.lower()}"
+            )
+            os.link(source.video_path, private_path)
+            sources.append(
+                SourceSnapshot(
+                    video_path=private_path,
+                    trim_start=source.trim_start,
+                    trim_end=source.trim_end,
+                    source_masks=tuple(pair[0].data.copy() for pair in pairs),
+                    target_masks=tuple(pair[1].data.copy() for pair in pairs),
+                    reference_frames=tuple(pair[0].frame_index for pair in pairs),
+                )
+            )
+    except Exception:
+        for source in sources:
+            source.video_path.unlink(missing_ok=True)
+        raise
+    return GenerationSnapshot(
+        target_image=target.image.copy(),
+        sources=tuple(sources),
+        prompt=request.prompt.strip(),
+        negative_prompt=(request.negative_prompt or "").strip() or None,
+        steps=request.steps,
+        seed=request.seed,
+        guidance_mode=request.guidance_mode,
+        text_guidance_scale=request.text_guidance_scale,
+        motion_guidance_scale=request.motion_guidance_scale,
+        lora_scale=request.lora_scale,
+        output_path=output_path,
+    )
+
+
+def create_app(runtime: Runtime | None = None) -> FastAPI:
+    runtime = runtime or Runtime()
+
+    @asynccontextmanager
+    async def lifespan(_: FastAPI):
+        yield
+        runtime.close()
+
+    application = FastAPI(title="WhatMoves", lifespan=lifespan)
+    application.state.runtime = runtime
+    application.mount("/static", StaticFiles(directory=STATIC_ROOT), name="static")
+
+    @application.get("/", response_class=HTMLResponse)
+    async def index():
+        return (STATIC_ROOT / "index.html").read_text()
+
+    @application.get("/api/health")
+    async def health():
+        return {"status": "ok", "wan_status": runtime.wan.status}
+
+    @application.post("/api/sessions")
+    async def create_session():
+        session = runtime.store.create()
+        try:
+            await _run_cpu(
+                runtime,
+                load_examples,
+                session,
+                ASSETS_ROOT,
+                source_fps=runtime.config.source_fps,
+                max_sources=runtime.config.max_sources,
+                max_targets=runtime.config.max_targets,
+                max_image_side=runtime.config.max_image_side,
+            )
+        except Exception:
+            runtime.store.pop(session.id)
+            runtime.dispose_session(session)
+            raise
+        source = next(iter(session.sources.values()), None)
+        target = session.selected_target
+        if source is not None:
+            source.sam_status = "queued"
+            _queue_embedding(
+                runtime,
+                session,
+                "source",
+                source.id,
+                source.revision,
+                source.current_frame,
+            )
+        if target is not None:
+            target.sam_status = "queued"
+            _queue_embedding(runtime, session, "target", target.id, target.revision)
+        return _state(runtime, session)
+
+    @application.get("/api/sessions/{session_id}")
+    async def get_state(session_id: str):
+        return _state(runtime, _session(runtime, session_id))
+
+    @application.post("/api/sessions/{session_id}/targets")
+    async def upload_target(session_id: str, file: UploadFile = File(...)):
+        session = _session(runtime, session_id)
+        with session.lock:
+            if len(session.targets) >= runtime.config.max_targets:
+                raise HTTPException(
+                    409,
+                    f"A session may contain at most {runtime.config.max_targets} targets",
+                )
+        target_id = new_id("target")
+        suffix = Path(file.filename or "target.png").suffix.lower() or ".png"
+        path = session.directory / f"{target_id}{suffix}"
+        await _run_cpu(
+            runtime, _store_upload_sync, file, path, runtime.config.max_upload_bytes
+        )
+        try:
+            image = await _run_cpu(
+                runtime, read_image, path, runtime.config.max_image_side
+            )
+        except Exception as error:
+            raise HTTPException(
+                400, f"Could not decode target image: {error}"
+            ) from error
+        finally:
+            path.unlink(missing_ok=True)
+        with session.lock:
+            if session.closed:
+                raise HTTPException(410, "The browser session has expired")
+            if len(session.targets) >= runtime.config.max_targets:
+                raise HTTPException(
+                    409,
+                    f"A session may contain at most {runtime.config.max_targets} targets",
+                )
+            session.target_revision += 1
+            target = TargetAsset(
+                id=target_id,
+                name=Path(file.filename or "Target image").name,
+                image=image,
+                revision=session.target_revision,
+            )
+            session.targets[target.id] = target
+            session.selected_target_id = target.id
+            session.touch()
+        _queue_embedding(runtime, session, "target", target.id, target.revision)
+        return _state(runtime, session)
+
+    @application.put("/api/sessions/{session_id}/targets/{target_id}/selection")
+    async def select_target(session_id: str, target_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            target = _asset(session, "target", target_id)
+            session.selected_target_id = target.id
+            if not runtime.sam.has(
+                _sam_key(session.id, "target", target.id, target.revision)
+            ):
+                target.sam_status = "queued"
+            session.touch()
+        _queue_embedding(runtime, session, "target", target.id, target.revision)
+        return _state(runtime, session)
+
+    @application.delete("/api/sessions/{session_id}/targets/{target_id}")
+    async def remove_target(session_id: str, target_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            target = _asset(session, "target", target_id)
+            del session.targets[target_id]
+            if session.selected_target_id == target_id:
+                session.selected_target_id = next(iter(session.targets), None)
+            session.touch()
+        _release_asset(runtime, session.id, "target", target_id)
+        _clear_asset(target)
+        return _state(runtime, session)
+
+    @application.post("/api/sessions/{session_id}/sources")
+    async def upload_source(session_id: str, file: UploadFile = File(...)):
+        session = _session(runtime, session_id)
+        with session.lock:
+            if len(session.sources) >= runtime.config.max_sources:
+                raise HTTPException(
+                    409,
+                    f"A session may contain at most {runtime.config.max_sources} sources",
+                )
+        suffix = Path(file.filename or "source.mp4").suffix.lower() or ".mp4"
+        source_id = new_id("source")
+        upload_path = session.directory / f"upload_{source_id}{suffix}"
+        path = session.directory / f"{source_id}.mp4"
+        await _run_cpu(
+            runtime,
+            _store_upload_sync,
+            file,
+            upload_path,
+            runtime.config.max_upload_bytes,
+        )
+        try:
+            original_fps, frame_count, frame = await _run_cpu(
+                runtime,
+                resample_video,
+                upload_path,
+                path,
+                runtime.config.source_fps,
+                runtime.config.max_image_side,
+            )
+        except Exception as error:
+            path.unlink(missing_ok=True)
+            raise HTTPException(
+                400, f"Could not decode source video: {error}"
+            ) from error
+        finally:
+            upload_path.unlink(missing_ok=True)
+        source = SourceAsset(
+            id=source_id,
+            name=Path(file.filename or "Source video").name,
+            video_path=path,
+            frame=frame,
+            fps=float(runtime.config.source_fps),
+            frame_count=frame_count,
+            original_fps=original_fps,
+            trim_end=min(frame_count - 1, runtime.config.output_frames - 1),
+        )
+        with session.lock:
+            if session.closed:
+                path.unlink(missing_ok=True)
+                raise HTTPException(410, "The browser session has expired")
+            if len(session.sources) >= runtime.config.max_sources:
+                path.unlink(missing_ok=True)
+                raise HTTPException(
+                    409,
+                    f"A session may contain at most {runtime.config.max_sources} sources",
+                )
+            session.sources[source.id] = source
+            session.touch()
+        _queue_embedding(
+            runtime,
+            session,
+            "source",
+            source.id,
+            source.revision,
+            source.current_frame,
+        )
+        return _state(runtime, session)
+
+    @application.patch("/api/sessions/{session_id}/sources/{source_id}")
+    async def update_source(session_id: str, source_id: str, settings: SourceSettings):
+        session = _session(runtime, session_id)
+        with session.lock:
+            source = _asset(session, "source", source_id)
+            trim_start = (
+                source.trim_start
+                if settings.trim_start is None
+                else settings.trim_start
+            )
+            trim_end = (
+                source.trim_end if settings.trim_end is None else settings.trim_end
+            )
+            if not 0 <= trim_start <= trim_end < source.frame_count:
+                raise HTTPException(400, "Invalid source trim interval")
+            if source.frame_count >= 8 and trim_end - trim_start + 1 < 8:
+                raise HTTPException(400, "Select at least eight 8-fps frames")
+            new_frame = (
+                source.current_frame
+                if settings.current_frame is None
+                else settings.current_frame
+            )
+            new_frame = min(source.frame_count - 1, max(0, new_frame))
+            path = source.video_path
+            frame_changed = new_frame != source.current_frame
+            trim_changed = (
+                trim_start != source.trim_start or trim_end != source.trim_end
+            )
+            trim_requested = (
+                settings.trim_start is not None or settings.trim_end is not None
+            )
+            selection_token = new_id("frame") if frame_changed else None
+            if frame_changed:
+                source.frame_selection_token = selection_token
+        frame = None
+        if frame_changed:
+            try:
+                frame = await _run_cpu(
+                    runtime,
+                    decode_video_frame,
+                    path,
+                    new_frame,
+                    runtime.config.max_image_side,
+                )
+            except Exception as error:
+                raise HTTPException(400, str(error)) from error
+        with session.lock:
+            source = _asset(session, "source", source_id)
+            if frame_changed and source.frame_selection_token != selection_token:
+                raise HTTPException(409, "Superseded by a newer frame selection")
+            if trim_requested:
+                source.trim_start = trim_start
+                source.trim_end = trim_end
+            if trim_requested and trim_changed:
+                clear_excluded_source_mappings(session, source)
+            if frame_changed:
+                source.current_frame = new_frame
+                source.frame = frame
+                source.frame_selection_token = None
+                source.draft = None
+                source.prediction_token = None
+                source.hover_prediction_token = None
+                source.sam_status = "queued"
+                source.sam_error = None
+            if trim_requested and trim_changed:
+                session.touch()
+            revision = source.revision
+        if frame_changed:
+            runtime.scheduler.cancel_prefix(
+                f"sam:{_sam_prefix(session.id, 'source', source_id)}"
+            )
+            _queue_embedding(
+                runtime,
+                session,
+                "source",
+                source_id,
+                revision,
+                new_frame,
+            )
+        return _state(runtime, session)
+
+    @application.delete("/api/sessions/{session_id}/sources/{source_id}")
+    async def remove_source(session_id: str, source_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            if source_id not in session.sources:
+                raise HTTPException(404, f"Unknown source {source_id}")
+            source = session.sources.pop(source_id)
+            clear_source_mappings(session, source_id)
+            session.touch()
+        _release_asset(runtime, session.id, "source", source_id)
+        _clear_asset(source)
+        source.video_path.unlink(missing_ok=True)
+        return _state(runtime, session)
+
+    @application.post("/api/sessions/{session_id}/assets/{kind}/{asset_id}/predict")
+    async def predict_mask(
+        session_id: str, kind: str, asset_id: str, request: PromptRequest
+    ):
+        _validate_prompts(request)
+        session = _session(runtime, session_id)
+        with session.lock:
+            asset = _asset(session, kind, asset_id)
+            if asset.revision != request.asset_revision:
+                raise HTTPException(409, "The underlying media has changed")
+            frame_index = asset.current_frame if kind == "source" else None
+            if kind == "source" and request.frame_index != frame_index:
+                raise HTTPException(409, "The selected source frame has changed")
+            key = _sam_key(
+                session.id,
+                kind,
+                asset_id,
+                asset.revision,
+                frame_index,
+            )
+            prediction_token = new_id("prediction")
+            token_name = (
+                "hover_prediction_token" if request.transient else "prediction_token"
+            )
+            if not request.transient:
+                asset.draft = None
+                asset.hover_prediction_token = None
+            setattr(asset, token_name, prediction_token)
+
+        def run_prediction():
+            with session.lock:
+                try:
+                    current = _asset(session, kind, asset_id)
+                except HTTPException as error:
+                    raise Superseded("Media removed during mask prediction") from error
+                if (
+                    session.closed
+                    or current.revision != request.asset_revision
+                    or (kind == "source" and current.current_frame != frame_index)
+                    or getattr(current, token_name) != prediction_token
+                ):
+                    raise Superseded("Media changed during mask prediction")
+                image = current.image if kind == "target" else current.frame
+            mask, score = runtime.sam.predict(
+                key,
+                image,
+                request.positive,
+                request.negative,
+                request.box,
+            )
+            with session.lock:
+                try:
+                    current = _asset(session, kind, asset_id)
+                except HTTPException as error:
+                    raise Superseded("Media removed during mask prediction") from error
+                if (
+                    current.revision != request.asset_revision
+                    or (kind == "source" and current.current_frame != frame_index)
+                    or getattr(current, token_name) != prediction_token
+                ):
+                    raise Superseded("Media changed during mask prediction")
+                current.sam_status = "ready"
+                current.sam_error = None
+                if not request.transient:
+                    current.draft = Draft(
+                        mask,
+                        request.asset_revision,
+                        request.prompt_revision,
+                        frame_index,
+                    )
+            return mask, score
+
+        try:
+            if runtime.config.device == "cpu":
+                mask, score = run_prediction()
+            else:
+                try:
+                    if not request.transient:
+                        runtime.scheduler.cancel_prefix(
+                            f"sam:{_sam_prefix(session.id, kind, asset_id)}"
+                            "prediction:hover"
+                        )
+                    _, future = runtime.scheduler.submit(
+                        run_prediction,
+                        priority=30 if request.transient else 0,
+                        key=(
+                            f"sam:{_sam_prefix(session.id, kind, asset_id)}"
+                            f"prediction:{'hover' if request.transient else 'commit'}"
+                        ),
+                        owner=session.id,
+                        label="Update mask draft",
+                    )
+                except QueueFull as error:
+                    raise HTTPException(503, str(error)) from error
+                mask, score = await asyncio.wrap_future(future)
+        except Superseded as error:
+            raise HTTPException(409, str(error)) from error
+        except Exception as error:
+            raise HTTPException(500, f"SAM prediction failed: {error}") from error
+        return {
+            "asset_revision": request.asset_revision,
+            "prompt_revision": request.prompt_revision,
+            "score": score,
+            "transient": request.transient,
+            "mask": base64.b64encode(mask_png_bytes(mask)).decode("ascii"),
+        }
+
+    @application.delete("/api/sessions/{session_id}/assets/{kind}/{asset_id}/draft")
+    async def clear_draft(
+        session_id: str,
+        kind: str,
+        asset_id: str,
+        revision: int,
+    ):
+        session = _session(runtime, session_id)
+        runtime.scheduler.cancel_prefix(
+            f"sam:{_sam_prefix(session.id, kind, asset_id)}prediction"
+        )
+        with session.lock:
+            asset = _asset(session, kind, asset_id)
+            if asset.revision == revision:
+                asset.draft = None
+                asset.prediction_token = None
+                asset.hover_prediction_token = None
+        return Response(status_code=204)
+
+    @application.post("/api/sessions/{session_id}/assets/{kind}/{asset_id}/masks")
+    async def commit_mask(
+        session_id: str, kind: str, asset_id: str, request: CommitRequest
+    ):
+        session = _session(runtime, session_id)
+        with session.lock:
+            asset = _asset(session, kind, asset_id)
+            draft = asset.draft
+            if (
+                draft is None
+                or draft.asset_revision != request.asset_revision
+                or draft.prompt_revision != request.prompt_revision
+                or draft.frame_index != request.frame_index
+            ):
+                raise HTTPException(409, "The mask draft is no longer current")
+            if not draft.data.any():
+                raise HTTPException(400, "Cannot add an empty mask")
+            mask_id = new_id("mask")
+            color = session.allocate_color() if kind == "source" else None
+            asset.masks[mask_id] = MaskRecord(
+                mask_id,
+                draft.data.copy(),
+                color=color,
+                frame_index=draft.frame_index if kind == "source" else None,
+            )
+            asset.draft = None
+            asset.prediction_token = None
+            asset.hover_prediction_token = None
+            asset.mask_revision += 1
+            session.touch()
+        return _state(runtime, session)
+
+    @application.delete(
+        "/api/sessions/{session_id}/assets/{kind}/{asset_id}/masks/{mask_id}"
+    )
+    async def remove_mask(session_id: str, kind: str, asset_id: str, mask_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            asset = _asset(session, kind, asset_id)
+            if mask_id not in asset.masks:
+                raise HTTPException(404, f"Unknown mask {mask_id}")
+            del asset.masks[mask_id]
+            asset.mask_revision += 1
+            if kind == "source":
+                clear_source_mappings(session, asset_id, mask_id)
+            session.touch()
+        return _state(runtime, session)
+
+    @application.put(
+        "/api/sessions/{session_id}/targets/{target_id}/masks/{target_mask_id}/mapping"
+    )
+    async def map_mask(
+        session_id: str,
+        target_id: str,
+        target_mask_id: str,
+        request: MappingRequest,
+    ):
+        session = _session(runtime, session_id)
+        with session.lock:
+            target = _asset(session, "target", target_id)
+            if target_mask_id not in target.masks:
+                raise HTTPException(404, f"Unknown target mask {target_mask_id}")
+            target_mask = target.masks[target_mask_id]
+            if request.source_id is None and request.source_mask_id is None:
+                target_mask.source_id = None
+                target_mask.source_mask_id = None
+            elif request.source_id is None or request.source_mask_id is None:
+                raise HTTPException(400, "Both source and source mask are required")
+            else:
+                try:
+                    source_mask_lookup(
+                        session, request.source_id, request.source_mask_id
+                    )
+                except KeyError as error:
+                    raise HTTPException(404, str(error)) from error
+                target_mask.source_id = request.source_id
+                target_mask.source_mask_id = request.source_mask_id
+            target.mask_revision += 1
+            session.touch()
+        return _state(runtime, session)
+
+    @application.post("/api/sessions/{session_id}/generate")
+    async def generate(session_id: str, request: GenerateRequest):
+        session = _session(runtime, session_id)
+        with session.lock:
+            if session.closed:
+                raise HTTPException(410, "The browser session has expired")
+            if session.active_generation is not None:
+                active = runtime.scheduler.get(session.active_generation)
+                if active is not None and active.status in {"queued", "running"}:
+                    raise HTTPException(409, "This session is already generating")
+                session.active_generation = None
+            snapshot = _generation_snapshot(runtime, session, request)
+            input_revision = session.input_revision
+            previous_failure = session.failed_generation
+            session.failed_generation = None
+            if previous_failure is not None:
+                runtime.scheduler.discard(previous_failure)
+            output_path = snapshot.output_path
+            private_inputs = tuple(source.video_path for source in snapshot.sources)
+            task_id = new_id("task")
+
+            def report_progress(stage, current=None, total=None):
+                runtime.scheduler.update_progress(
+                    task_id,
+                    stage,
+                    current,
+                    total,
+                )
+
+            def run_generation():
+                try:
+                    result = runtime.wan.generate(snapshot, report_progress)
+                    result["video_url"] = (
+                        f"/api/sessions/{session.id}/outputs/{result['output_name']}"
+                    )
+                    with session.lock:
+                        result["inputs_changed"] = (
+                            session.input_revision != input_revision
+                        )
+                    return result
+                finally:
+                    for path in private_inputs:
+                        path.unlink(missing_ok=True)
+
+            try:
+                task_id, future = runtime.scheduler.submit(
+                    run_generation,
+                    priority=10,
+                    key=f"generation:{session.id}",
+                    owner=session.id,
+                    label="Generate video",
+                    retain_record=True,
+                    task_id=task_id,
+                )
+            except (QueueFull, RuntimeError) as error:
+                for path in private_inputs:
+                    path.unlink(missing_ok=True)
+                output_path.unlink(missing_ok=True)
+                raise HTTPException(503, str(error)) from error
+            session.active_generation = task_id
+
+        def generation_finished(done):
+            try:
+                done.result()
+            except Exception:
+                succeeded = False
+            else:
+                succeeded = True
+
+            old_task = None
+            old_output = None
+            with session.lock:
+                if session.active_generation == task_id:
+                    session.active_generation = None
+                closed = session.closed
+                if succeeded and not closed:
+                    old_task = session.latest_generation
+                    old_output = session.latest_output
+                    session.latest_generation = task_id
+                    session.latest_output = output_path
+                elif not closed:
+                    old_task = session.failed_generation
+                    session.failed_generation = task_id
+            if old_task is not None and old_task != task_id:
+                runtime.scheduler.discard(old_task)
+            if old_output is not None and old_output != output_path:
+                old_output.unlink(missing_ok=True)
+            if not succeeded or closed:
+                output_path.unlink(missing_ok=True)
+            if closed:
+                runtime.scheduler.discard(task_id)
+                _remove_session_directory(
+                    runtime.config.runtime_root,
+                    session.directory,
+                )
+
+        future.add_done_callback(generation_finished)
+        return {"task_id": task_id}
+
+    @application.delete("/api/sessions/{session_id}")
+    async def delete_session(session_id: str):
+        session = runtime.store.pop(session_id)
+        if session is None:
+            raise HTTPException(404, f"Unknown session {session_id}")
+        runtime.dispose_session(session)
+        return Response(status_code=204)
+
+    @application.get("/api/tasks/{task_id}")
+    async def task_status(task_id: str):
+        record = runtime.scheduler.get(task_id)
+        if record is None:
+            raise HTTPException(404, f"Unknown task {task_id}")
+        return record.as_dict()
+
+    @application.get("/api/sessions/{session_id}/targets/{target_id}/image")
+    async def target_image(session_id: str, target_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            content = png_bytes(_asset(session, "target", target_id).image)
+        return Response(
+            content,
+            media_type="image/png",
+            headers={"Cache-Control": "no-store"},
+        )
+
+    @application.get("/api/sessions/{session_id}/targets/{target_id}/thumbnail.png")
+    async def target_thumbnail(session_id: str, target_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            target = _asset(session, "target", target_id)
+            masks = []
+            for mask in target.masks.values():
+                color = "#a8afb9"
+                if mask.source_id and mask.source_mask_id:
+                    try:
+                        color = source_mask_lookup(
+                            session, mask.source_id, mask.source_mask_id
+                        ).color
+                    except KeyError:
+                        pass
+                masks.append((mask.data, color))
+            image = composite_masks(target.image, masks, max_size=(180, 120))
+        return Response(
+            png_bytes(image),
+            media_type="image/png",
+            headers={"Cache-Control": "no-store"},
+        )
+
+    @application.get(
+        "/api/sessions/{session_id}/targets/{target_id}/masks/{mask_id}.png"
+    )
+    async def target_mask_image(session_id: str, target_id: str, mask_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            target = _asset(session, "target", target_id)
+            if mask_id not in target.masks:
+                raise HTTPException(404, f"Unknown target mask {mask_id}")
+            content = mask_png_bytes(target.masks[mask_id].data)
+        return Response(
+            content,
+            media_type="image/png",
+            headers={"Cache-Control": "no-store"},
+        )
+
+    @application.get("/api/sessions/{session_id}/sources/{source_id}/frame.png")
+    async def source_frame(session_id: str, source_id: str, frame: int | None = None):
+        session = _session(runtime, session_id)
+        with session.lock:
+            source = _asset(session, "source", source_id)
+            requested = source.current_frame if frame is None else frame
+            if not 0 <= requested < source.frame_count:
+                raise HTTPException(404, f"Unknown frame {requested}")
+            path = source.video_path
+            cached = source.frame.copy() if requested == source.current_frame else None
+        image = cached
+        if image is None:
+            image = await _run_cpu(
+                runtime,
+                decode_video_frame,
+                path,
+                requested,
+                runtime.config.max_image_side,
+            )
+        return Response(
+            png_bytes(image),
+            media_type="image/png",
+            headers={"Cache-Control": "no-store"},
+        )
+
+    @application.get("/api/sessions/{session_id}/sources/{source_id}/video")
+    async def source_video(session_id: str, source_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            path = _asset(session, "source", source_id).video_path
+        if not path.is_file():
+            raise HTTPException(404, "Source video is unavailable")
+        return FileResponse(
+            path,
+            media_type="video/mp4",
+            headers={"Cache-Control": "no-store", "Accept-Ranges": "bytes"},
+        )
+
+    @application.get(
+        "/api/sessions/{session_id}/sources/{source_id}/frames/{frame_index}.jpg"
+    )
+    async def source_timeline_frame(session_id: str, source_id: str, frame_index: int):
+        session = _session(runtime, session_id)
+        with session.lock:
+            source = _asset(session, "source", source_id)
+            if not 0 <= frame_index < source.frame_count:
+                raise HTTPException(404, f"Unknown frame {frame_index}")
+            path = source.video_path
+        image = await _run_cpu(
+            runtime,
+            decode_video_frame,
+            path,
+            frame_index,
+            256,
+        )
+        return Response(
+            jpeg_bytes(image),
+            media_type="image/jpeg",
+            headers={"Cache-Control": "private, max-age=3600"},
+        )
+
+    @application.get("/api/sessions/{session_id}/sources/{source_id}/thumbnail.png")
+    async def source_thumbnail(session_id: str, source_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            source = _asset(session, "source", source_id)
+            frame_index = source.trim_start
+            path = source.video_path
+            masks = [
+                (mask.data.copy(), mask.color)
+                for mask in source.masks.values()
+                if mask.frame_index == frame_index
+            ]
+        frame = await _run_cpu(runtime, decode_video_frame, path, frame_index, 320)
+        image = composite_masks(frame, masks, max_size=(180, 120))
+        return Response(
+            png_bytes(image),
+            media_type="image/png",
+            headers={"Cache-Control": "no-store"},
+        )
+
+    @application.get(
+        "/api/sessions/{session_id}/sources/{source_id}/masks/{mask_id}.png"
+    )
+    async def source_mask_image(session_id: str, source_id: str, mask_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            source = _asset(session, "source", source_id)
+            if mask_id not in source.masks:
+                raise HTTPException(404, f"Unknown mask {mask_id}")
+            content = mask_png_bytes(source.masks[mask_id].data)
+        return Response(
+            content,
+            media_type="image/png",
+            headers={"Cache-Control": "no-store"},
+        )
+
+    @application.get(
+        "/api/sessions/{session_id}/sources/{source_id}/masks/{mask_id}/preview.png"
+    )
+    async def source_mask_preview(session_id: str, source_id: str, mask_id: str):
+        session = _session(runtime, session_id)
+        with session.lock:
+            source = _asset(session, "source", source_id)
+            if mask_id not in source.masks:
+                raise HTTPException(404, f"Unknown mask {mask_id}")
+            mask = source.masks[mask_id]
+            frame_index = mask.frame_index
+            path = source.video_path
+            mask_data = mask.data.copy()
+            color = mask.color
+        if frame_index is None:
+            raise HTTPException(409, "Source mask has no reference frame")
+        frame = await _run_cpu(runtime, decode_video_frame, path, frame_index, 320)
+        image = composite_masks(frame, [(mask_data, color)], max_size=(160, 100))
+        return Response(
+            png_bytes(image),
+            media_type="image/png",
+            headers={"Cache-Control": "no-store"},
+        )
+
+    @application.get("/api/sessions/{session_id}/outputs/{name}")
+    async def output_video(session_id: str, name: str):
+        session = _session(runtime, session_id)
+        safe_name = Path(name).name
+        with session.lock:
+            path = session.latest_output
+        if safe_name != name or path is None or path.name != name or not path.is_file():
+            raise HTTPException(404, "Unknown output")
+        return FileResponse(
+            path,
+            media_type="video/mp4",
+            headers={"Cache-Control": "no-store"},
+        )
+
+    return application

app/state.py

@@ -0,0 +1,314 @@
+"""In-memory browser-session state."""
+
+from __future__ import annotations
+
+import colorsys
+from collections import OrderedDict
+from dataclasses import dataclass, field
+from pathlib import Path
+import threading
+import time
+from typing import Any
+from uuid import uuid4
+
+import numpy as np
+
+MASK_COLORS = (
+    "#E69F00",
+    "#56B4E9",
+    "#009E73",
+    "#F0E442",
+    "#0072B2",
+    "#D55E00",
+    "#CC79A7",
+    "#7A6FF0",
+    "#32B8A6",
+    "#EF6C96",
+    "#8AAE3D",
+    "#C98932",
+)
+
+
+@dataclass
+class MaskRecord:
+    id: str
+    data: np.ndarray
+    color: str | None = None
+    source_id: str | None = None
+    source_mask_id: str | None = None
+    frame_index: int | None = None
+
+
+@dataclass
+class Draft:
+    data: np.ndarray
+    asset_revision: int
+    prompt_revision: int
+    frame_index: int | None = None
+
+
+@dataclass
+class TargetAsset:
+    id: str
+    name: str
+    image: np.ndarray
+    revision: int = 1
+    sam_status: str = "queued"
+    sam_error: str | None = None
+    masks: OrderedDict[str, MaskRecord] = field(default_factory=OrderedDict)
+    draft: Draft | None = None
+    prediction_token: str | None = None
+    hover_prediction_token: str | None = None
+    mask_revision: int = 0
+    example: bool = False
+
+
+@dataclass
+class SourceAsset:
+    id: str
+    name: str
+    video_path: Path
+    frame: np.ndarray
+    fps: float
+    frame_count: int
+    original_fps: float
+    current_frame: int = 0
+    trim_start: int = 0
+    trim_end: int = 0
+    revision: int = 1
+    sam_status: str = "queued"
+    sam_error: str | None = None
+    masks: OrderedDict[str, MaskRecord] = field(default_factory=OrderedDict)
+    draft: Draft | None = None
+    prediction_token: str | None = None
+    hover_prediction_token: str | None = None
+    frame_selection_token: str | None = None
+    mask_revision: int = 0
+    example: bool = False
+
+
+@dataclass
+class Session:
+    id: str
+    directory: Path
+    targets: OrderedDict[str, TargetAsset] = field(default_factory=OrderedDict)
+    selected_target_id: str | None = None
+    sources: OrderedDict[str, SourceAsset] = field(default_factory=OrderedDict)
+    next_color: int = 0
+    target_revision: int = 0
+    active_generation: str | None = None
+    latest_generation: str | None = None
+    failed_generation: str | None = None
+    latest_output: Path | None = None
+    input_revision: int = 0
+    created_at: float = field(default_factory=time.time)
+    accessed_at: float = field(default_factory=time.time)
+    closed: bool = False
+    lock: threading.RLock = field(default_factory=threading.RLock, repr=False)
+
+    @property
+    def selected_target(self) -> TargetAsset | None:
+        if self.selected_target_id is None:
+            return None
+        return self.targets.get(self.selected_target_id)
+
+    def allocate_color(self) -> str:
+        if self.next_color < len(MASK_COLORS):
+            color = MASK_COLORS[self.next_color]
+        else:
+            hue = (0.13 + self.next_color * 0.61803398875) % 1.0
+            rgb = colorsys.hsv_to_rgb(hue, 0.68, 0.94)
+            color = "#" + "".join(f"{round(channel * 255):02X}" for channel in rgb)
+        self.next_color += 1
+        return color
+
+    def touch(self) -> None:
+        self.input_revision += 1
+        self.accessed_at = time.time()
+
+    def access(self) -> None:
+        self.accessed_at = time.time()
+
+
+class SessionStore:
+    def __init__(self, root: Path) -> None:
+        self.root = root
+        self.root.mkdir(parents=True, exist_ok=True, mode=0o700)
+        self.root.chmod(0o700)
+        self._sessions: dict[str, Session] = {}
+        self._lock = threading.RLock()
+
+    def create(self) -> Session:
+        session_id = uuid4().hex
+        directory = self.root / session_id
+        directory.mkdir(parents=True, exist_ok=False, mode=0o700)
+        session = Session(session_id, directory)
+        with self._lock:
+            self._sessions[session_id] = session
+        return session
+
+    def get(self, session_id: str) -> Session:
+        with self._lock:
+            try:
+                session = self._sessions[session_id]
+            except KeyError as error:
+                raise KeyError(f"Unknown session {session_id}") from error
+        session.access()
+        return session
+
+    def pop(self, session_id: str) -> Session | None:
+        with self._lock:
+            return self._sessions.pop(session_id, None)
+
+    def pop_expired(self, ttl_seconds: float) -> list[Session]:
+        cutoff = time.time() - float(ttl_seconds)
+        with self._lock:
+            expired = [
+                session
+                for session in self._sessions.values()
+                if session.accessed_at < cutoff
+            ]
+            for session in expired:
+                self._sessions.pop(session.id, None)
+        return expired
+
+    def pop_all(self) -> list[Session]:
+        with self._lock:
+            sessions = list(self._sessions.values())
+            self._sessions.clear()
+        return sessions
+
+
+def new_id(prefix: str) -> str:
+    return f"{prefix}_{uuid4().hex[:10]}"
+
+
+def clear_source_mappings(
+    session: Session, source_id: str, mask_id: str | None = None
+) -> None:
+    for target in session.targets.values():
+        changed = False
+        for target_mask in target.masks.values():
+            if target_mask.source_id != source_id:
+                continue
+            if mask_id is None or target_mask.source_mask_id == mask_id:
+                target_mask.source_id = None
+                target_mask.source_mask_id = None
+                changed = True
+        if changed:
+            target.mask_revision += 1
+
+
+def clear_excluded_source_mappings(session: Session, source: SourceAsset) -> int:
+    """Clear mappings whose source observation falls outside the trim."""
+    excluded = {
+        mask.id
+        for mask in source.masks.values()
+        if mask.frame_index is None
+        or not source.trim_start <= mask.frame_index <= source.trim_end
+    }
+    changed = 0
+    for target in session.targets.values():
+        target_changed = False
+        for mask in target.masks.values():
+            if mask.source_id == source.id and mask.source_mask_id in excluded:
+                mask.source_id = None
+                mask.source_mask_id = None
+                changed += 1
+                target_changed = True
+        if target_changed:
+            target.mask_revision += 1
+    return changed
+
+
+def source_mask_lookup(session: Session, source_id: str, mask_id: str) -> MaskRecord:
+    try:
+        return session.sources[source_id].masks[mask_id]
+    except KeyError as error:
+        raise KeyError(f"Unknown source mask {source_id}/{mask_id}") from error
+
+
+def session_payload(session: Session) -> dict[str, Any]:
+    def mask_payload(mask: MaskRecord, url: str, preview_url: str | None = None):
+        payload = {
+            "id": mask.id,
+            "color": mask.color,
+            "source_id": mask.source_id,
+            "source_mask_id": mask.source_mask_id,
+            "frame_index": mask.frame_index,
+            "url": url,
+        }
+        if preview_url:
+            payload["preview_url"] = preview_url
+        return payload
+
+    targets = []
+    for target in session.targets.values():
+        base = f"/api/sessions/{session.id}/targets/{target.id}"
+        targets.append(
+            {
+                "id": target.id,
+                "name": target.name,
+                "revision": target.revision,
+                "width": int(target.image.shape[1]),
+                "height": int(target.image.shape[0]),
+                "sam_status": target.sam_status,
+                "sam_error": target.sam_error,
+                "example": target.example,
+                "image_url": f"{base}/image?v={target.revision}",
+                "thumbnail_url": (
+                    f"{base}/thumbnail.png?v={target.revision}-{target.mask_revision}"
+                ),
+                "masks": [
+                    mask_payload(
+                        mask,
+                        f"{base}/masks/{mask.id}.png?v={target.mask_revision}",
+                    )
+                    for mask in target.masks.values()
+                ],
+            }
+        )
+
+    sources = []
+    for source in session.sources.values():
+        base = f"/api/sessions/{session.id}/sources/{source.id}"
+        sources.append(
+            {
+                "id": source.id,
+                "name": source.name,
+                "revision": source.revision,
+                "width": int(source.frame.shape[1]),
+                "height": int(source.frame.shape[0]),
+                "fps": source.fps,
+                "original_fps": source.original_fps,
+                "frame_count": source.frame_count,
+                "current_frame": source.current_frame,
+                "trim_start": source.trim_start,
+                "trim_end": source.trim_end,
+                "sam_status": source.sam_status,
+                "sam_error": source.sam_error,
+                "example": source.example,
+                "image_url": (
+                    f"{base}/frame.png?frame={source.current_frame}&v={source.revision}"
+                ),
+                "video_url": f"{base}/video?v={source.revision}",
+                "thumbnail_url": f"{base}/thumbnail.png?v={source.revision}-{source.mask_revision}",
+                "masks": [
+                    mask_payload(
+                        mask,
+                        f"{base}/masks/{mask.id}.png?v={source.mask_revision}",
+                        f"{base}/masks/{mask.id}/preview.png?v={source.mask_revision}",
+                    )
+                    for mask in source.masks.values()
+                ],
+            }
+        )
+    return {
+        "id": session.id,
+        "targets": targets,
+        "selected_target_id": session.selected_target_id,
+        "sources": sources,
+        "active_generation": session.active_generation,
+        "latest_generation": session.latest_generation,
+        "input_revision": session.input_revision,
+    }

app/static/app.js

@@ -0,0 +1,1389 @@
+import { MaskCanvas } from "/static/mask-canvas.js?v=20260904h";
+import { VideoTimeline } from "/static/video-timeline.js?v=20260904h";
+import { Client, handle_file } from "/static/gradio-client.js?v=20260904j";
+
+const $ = (id) => document.getElementById(id);
+const elements = {
+  startOver: $("startOver"),
+  sourceRail: $("sourceRail"),
+  targetRail: $("targetRail"),
+  sourceFile: $("sourceFile"),
+  targetFile: $("targetFile"),
+  removeSource: $("removeSource"),
+  removeTarget: $("removeTarget"),
+  sourceRailUpload: $("sourceRailUpload"),
+  sourceEmpty: $("sourceEmpty"),
+  targetEmpty: $("targetEmpty"),
+  sourceTools: $("sourceTools"),
+  targetTools: $("targetTools"),
+  sourceMaskMode: $("sourceMaskMode"),
+  targetMaskMode: $("targetMaskMode"),
+  sourceModeBadge: $("sourceModeBadge"),
+  targetModeBadge: $("targetModeBadge"),
+  sourceCanvasStatus: $("sourceCanvasStatus"),
+  targetCanvasStatus: $("targetCanvasStatus"),
+  sourceLoader: $("sourceLoader"),
+  targetLoader: $("targetLoader"),
+  sourceAddMask: $("sourceAddMask"),
+  targetAddMask: $("targetAddMask"),
+  sourceCancelDraft: $("sourceCancelDraft"),
+  targetCancelDraft: $("targetCancelDraft"),
+  sourceSettings: $("sourceSettings"),
+  sourceVideo: $("sourceVideo"),
+  sourcePlay: $("sourcePlay"),
+  sourceTime: $("sourceTime"),
+  sourceTimeline: $("sourceTimeline"),
+  contextMenu: $("contextMenu"),
+  dragGhost: $("dragGhost"),
+  mappingHint: $("mappingHint"),
+  toast: $("toast"),
+  prompt: $("prompt"),
+  negativePrompt: $("negativePrompt"),
+  guidanceMode: $("guidanceMode"),
+  steps: $("steps"),
+  seed: $("seed"),
+  textGuidance: $("textGuidance"),
+  motionGuidance: $("motionGuidance"),
+  loraScale: $("loraScale"),
+  generateButton: $("generateButton"),
+  generateButtonProgress: $("generateButtonProgress"),
+  generateButtonLabel: $("generateButtonLabel"),
+  generateButtonCount: $("generateButtonCount"),
+  generateHint: $("generateHint"),
+  generatedRail: $("generatedRail"),
+  resultPanel: $("resultPanel"),
+  resultVideo: $("resultVideo"),
+  resultLoader: $("resultLoader"),
+  resultMeta: $("resultMeta"),
+  resultClose: $("resultClose"),
+};
+
+let sessionState = null;
+let selectedSourceId = null;
+let toastTimer = null;
+let pollTimer = null;
+let activeGeneration = null;
+let activeGenerationForm = null;
+let activeGenerationStatus = null;
+let frameController = null;
+let draggedMaskId = null;
+let mappingHintTimer = null;
+let exampleTutorialState = "pending";
+let exampleTutorialScheduled = false;
+let formTargetId = null;
+let settingsDirty = false;
+let settingsSaveTimer = null;
+let targetSettingsSessionId = null;
+let targetSettings = {};
+let generatedOutputs = [];
+const predictionControllers = { source: null, target: null };
+const hoverControllers = { source: null, target: null };
+const hoverTimers = { source: null, target: null };
+const loadingReasons = { source: new Set(), target: new Set(), result: new Set() };
+
+function setMediaLoading(kind, reason, active) {
+  const reasons = loadingReasons[kind];
+  const loader = kind === "source"
+    ? elements.sourceLoader
+    : kind === "target"
+      ? elements.targetLoader
+      : elements.resultLoader;
+  if (active) reasons.add(reason);
+  else reasons.delete(reason);
+  loader.classList.toggle("hidden", reasons.size === 0);
+}
+
+let gradioClientPromise = null;
+
+function getGradioClient() {
+  if (!gradioClientPromise) {
+    gradioClientPromise = Client.connect(new URL("/gradio", window.location.origin).href, {
+      events: ["data", "status"],
+    }).catch((error) => {
+      gradioClientPromise = null;
+      throw error;
+    });
+  }
+  return gradioClientPromise;
+}
+
+async function request(url, options = {}) {
+  const method = options.method || "GET";
+  const signal = options.signal;
+  if (signal?.aborted) throw new DOMException("Request aborted", "AbortError");
+
+  let body = null;
+  let upload = null;
+  if (options.body instanceof FormData) {
+    const file = options.body.get("file");
+    if (file instanceof File) upload = handle_file(file);
+  } else if (typeof options.body === "string") {
+    body = options.body;
+  }
+
+  const endpoint = url.includes("/predict")
+    ? "/sam_api"
+    : url.split("?", 1)[0].endsWith("/generate")
+      ? "/generation_api"
+      : "/api";
+  const client = await getGradioClient();
+  const inputs = [method, url, body, upload];
+  if (endpoint === "/api") {
+    const result = await client.predict(endpoint, inputs);
+    let envelope = result.data?.[0] ?? null;
+    if (typeof envelope === "string") {
+      try { envelope = JSON.parse(envelope); } catch (_) { /* Keep the server value for the error below. */ }
+    }
+    if (!envelope?.ok) {
+      const payload = envelope?.data;
+      const message = payload?.detail || payload || `Request failed (${envelope?.status || "unknown"})`;
+      const error = new Error(message);
+      error.status = envelope?.status;
+      throw error;
+    }
+    return envelope.data;
+  }
+  const submission = client.submit(endpoint, inputs);
+  const abort = () => submission.cancel();
+  signal?.addEventListener("abort", abort, { once: true });
+
+  let envelope = null;
+  try {
+    for await (const message of submission) {
+      if (signal?.aborted) throw new DOMException("Request aborted", "AbortError");
+      if (message.type === "status") {
+        options.onGradioStatus?.(message);
+        if (message.stage === "error") {
+          throw new Error(typeof message.message === "string" ? message.message : "Gradio request failed");
+        }
+      } else if (message.type === "data") {
+        envelope = message.data?.[0] ?? null;
+      }
+    }
+  } finally {
+    signal?.removeEventListener("abort", abort);
+  }
+
+  if (typeof envelope === "string") {
+    try { envelope = JSON.parse(envelope); } catch (_) { /* Gradio may already return an object. */ }
+  }
+  if (!envelope?.ok) {
+    const payload = envelope?.data;
+    const message = payload?.detail || payload || `Request failed (${envelope?.status || "unknown"})`;
+    const error = new Error(message);
+    error.status = envelope?.status;
+    throw error;
+  }
+  return envelope.data;
+}
+
+function jsonOptions(value, method = "POST") {
+  return {
+    method,
+    headers: { "Content-Type": "application/json" },
+    body: JSON.stringify(value),
+  };
+}
+
+function showToast(message) {
+  clearTimeout(toastTimer);
+  elements.toast.textContent = message;
+  elements.toast.classList.remove("hidden");
+  toastTimer = setTimeout(() => elements.toast.classList.add("hidden"), 4200);
+}
+
+function canvasStatus(kind, message = "", error = false) {
+  const element = kind === "source" ? elements.sourceCanvasStatus : elements.targetCanvasStatus;
+  element.textContent = message;
+  element.style.color = error ? "#ff9a9c" : "";
+  element.classList.toggle("hidden", !message);
+}
+
+function modeChanged(kind, active, pinned) {
+  const badge = kind === "source" ? elements.sourceModeBadge : elements.targetModeBadge;
+  const button = kind === "source" ? elements.sourceMaskMode : elements.targetMaskMode;
+  badge.classList.toggle("hidden", !active);
+  button.classList.toggle("active", pinned);
+  button.textContent = pinned ? "Done masking" : "Start masking";
+}
+
+function draftChanged(kind, ready) {
+  const add = kind === "source" ? elements.sourceAddMask : elements.targetAddMask;
+  const cancel = kind === "source" ? elements.sourceCancelDraft : elements.targetCancelDraft;
+  add.classList.toggle("hidden", !ready);
+  cancel.classList.toggle("hidden", !ready);
+}
+
+const sourceCanvas = new MaskCanvas({
+  host: $("sourceCanvasHost"),
+  canvas: $("sourceCanvas"),
+  role: "source",
+  callbacks: {
+    onError: (error) => showToast(error.message),
+    onLoading: (active) => setMediaLoading("source", "canvas", active),
+    onPromptsChanged: (payload) => predictMask("source", payload),
+    onHoverPrompt: (payload) => scheduleHover("source", payload),
+    onDraftState: (ready) => draftChanged("source", ready),
+    onModeChange: (active, pinned) => modeChanged("source", active, pinned),
+    onMaskClick: (masks, x, y) => openSourceMenu(masks, x, y),
+    onMappingDrag: (phase, mask, x, y) => mappingDrag(phase, mask, x, y),
+  },
+});
+
+const targetCanvas = new MaskCanvas({
+  host: $("targetCanvasHost"),
+  canvas: $("targetCanvas"),
+  role: "target",
+  callbacks: {
+    onError: (error) => showToast(error.message),
+    onLoading: (active) => setMediaLoading("target", "canvas", active),
+    onPromptsChanged: (payload) => predictMask("target", payload),
+    onHoverPrompt: (payload) => scheduleHover("target", payload),
+    onDraftState: (ready) => draftChanged("target", ready),
+    onModeChange: (active, pinned) => modeChanged("target", active, pinned),
+    onMaskClick: (masks, x, y) => openTargetMenu(masks, x, y),
+  },
+});
+
+const sourceTimeline = new VideoTimeline({
+  root: elements.sourceTimeline,
+  video: elements.sourceVideo,
+  playButton: elements.sourcePlay,
+  time: elements.sourceTime,
+  callbacks: {
+    onSeek: (frame, final) => selectSourceFrame(frame, final),
+    onTrim: (start, end) => updateSourceTrim(start, end),
+    onMaskClick: () => sourceCanvas.setPinned(false),
+    onLoading: (active) => setMediaLoading(
+      "source",
+      "video",
+      active && $("sourceCanvasHost").classList.contains("playing"),
+    ),
+    onPlaybackChange: (playing) => {
+      $("sourceCanvasHost").classList.toggle("playing", playing);
+      if (!playing) setMediaLoading("source", "video", false);
+      if (playing) {
+        sourceCanvas.setPinned(false);
+        clearCanvasDraft("source");
+      }
+    },
+  },
+});
+
+["loadstart", "waiting", "seeking"].forEach((eventName) => {
+  elements.resultVideo.addEventListener(eventName, () => setMediaLoading("result", "video", true));
+});
+["loadeddata", "canplay", "playing", "seeked", "emptied", "error"].forEach((eventName) => {
+  elements.resultVideo.addEventListener(eventName, () => setMediaLoading("result", "video", false));
+});
+
+function selectedSource() {
+  return sessionState?.sources.find((source) => source.id === selectedSourceId) || null;
+}
+
+function selectedTarget() {
+  return sessionState?.targets.find((target) => target.id === sessionState.selected_target_id) || null;
+}
+
+function sourceMaskEntries() {
+  const entries = [];
+  (sessionState?.sources || []).forEach((source, sourceIndex) => {
+    source.masks.filter((mask) => mask.frame_index >= source.trim_start && mask.frame_index <= source.trim_end).forEach((mask, maskIndex) => {
+      entries.push({
+        ...mask,
+        source,
+        sourceLabel: `S${sourceIndex + 1}`,
+        maskLabel: `M${maskIndex + 1}`,
+      });
+    });
+  });
+  return entries;
+}
+
+function mappedColor(targetMask) {
+  const found = sourceMaskEntries().find(
+    (mask) => mask.source.id === targetMask.source_id && mask.id === targetMask.source_mask_id,
+  );
+  return found?.color || "#a8afb9";
+}
+
+function isExampleTutorial(target = selectedTarget()) {
+  return Boolean(
+    target?.example &&
+    sessionState?.sources.some((source) => source.example) &&
+    target.masks.some((mask) => mask.source_id && mask.source_mask_id)
+  );
+}
+
+function generationSettingsPayload() {
+  const numberValue = (element, fallback, integer = false) => {
+    const value = integer ? Number.parseInt(element.value, 10) : Number.parseFloat(element.value);
+    return Number.isFinite(value) ? value : fallback;
+  };
+  return {
+    prompt: elements.prompt.value,
+    negative_prompt: elements.negativePrompt.value || null,
+    steps: numberValue(elements.steps, 40, true),
+    seed: numberValue(elements.seed, 42, true),
+    guidance_mode: elements.guidanceMode.value,
+    text_guidance_scale: numberValue(elements.textGuidance, 3.5),
+    motion_guidance_scale: numberValue(elements.motionGuidance, 1.0),
+    lora_scale: numberValue(elements.loraScale, 1.0),
+  };
+}
+
+function defaultGenerationSettings() {
+  return {
+    prompt: "A bear and a dog together in the forest.",
+    negative_prompt: null,
+    steps: 40,
+    seed: 42,
+    guidance_mode: "text_cfg",
+    text_guidance_scale: 3.5,
+    motion_guidance_scale: 1.0,
+    lora_scale: 1.0,
+  };
+}
+
+function targetSettingsStorageKey(sessionId = targetSettingsSessionId) {
+  return `whatmoves-target-settings-${sessionId}`;
+}
+
+function loadTargetSettings(sessionId) {
+  targetSettingsSessionId = sessionId;
+  try {
+    targetSettings = JSON.parse(sessionStorage.getItem(targetSettingsStorageKey(sessionId)) || "{}");
+  } catch (_) {
+    targetSettings = {};
+  }
+}
+
+function persistTargetSettings() {
+  if (!targetSettingsSessionId) return;
+  sessionStorage.setItem(targetSettingsStorageKey(), JSON.stringify(targetSettings));
+}
+
+function restoreGenerationSettings(target) {
+  if (!target) return;
+  const settings = targetSettings[target.id] || defaultGenerationSettings();
+  elements.prompt.value = settings.prompt ?? "";
+  elements.negativePrompt.value = settings.negative_prompt ?? "";
+  elements.steps.value = settings.steps ?? 40;
+  elements.seed.value = settings.seed ?? 42;
+  elements.guidanceMode.value = settings.guidance_mode ?? "text_cfg";
+  elements.textGuidance.value = settings.text_guidance_scale ?? 3.5;
+  elements.motionGuidance.value = settings.motion_guidance_scale ?? 1.0;
+  elements.loraScale.value = settings.lora_scale ?? 1.0;
+  settingsDirty = false;
+}
+
+async function flushTargetSettings() {
+  clearTimeout(settingsSaveTimer);
+  settingsSaveTimer = null;
+  if (!settingsDirty || !sessionState || !formTargetId) return true;
+  targetSettings[formTargetId] = generationSettingsPayload();
+  persistTargetSettings();
+  settingsDirty = false;
+  return true;
+}
+
+function scheduleTargetSettingsSave() {
+  if (!formTargetId) return;
+  settingsDirty = true;
+  clearTimeout(settingsSaveTimer);
+  settingsSaveTimer = setTimeout(flushTargetSettings, 450);
+}
+
+async function applyState(next) {
+  if (targetSettingsSessionId !== next.id) loadTargetSettings(next.id);
+  sessionState = next;
+  if (!activeGeneration && next.active_generation) {
+    activeGeneration = next.active_generation;
+    activeGenerationForm = null;
+    activeGenerationStatus = "queued";
+    pollGeneration(activeGeneration);
+  }
+  if (!next.sources.some((source) => source.id === selectedSourceId)) {
+    selectedSourceId = next.sources[0]?.id || null;
+  }
+  renderSourceRail();
+  renderTargetRail();
+  renderGeneratedRail();
+
+  const source = selectedSource();
+  elements.sourceTools.classList.toggle("hidden", !source);
+  elements.sourceSettings.classList.toggle("hidden", !source);
+  elements.removeSource.classList.toggle("hidden", !source);
+  elements.sourceEmpty.classList.toggle("hidden", Boolean(source));
+  if (source) {
+    source.session_id = next.id;
+    const masks = source.masks.filter((mask) => mask.frame_index === source.current_frame)
+      .map((mask) => ({ ...mask, displayColor: mask.color }));
+    await sourceCanvas.setAsset(source, masks);
+    sourceTimeline.setSource(source);
+    assetStatus("source", source);
+  } else {
+    await sourceCanvas.setAsset(null);
+    sourceTimeline.setSource(null);
+    canvasStatus("source");
+  }
+
+  const target = selectedTarget();
+  if (target?.id !== formTargetId) {
+    formTargetId = target?.id || null;
+    restoreGenerationSettings(target);
+  }
+  elements.targetTools.classList.toggle("hidden", !target);
+  elements.removeTarget.classList.toggle("hidden", !target);
+  elements.targetEmpty.classList.toggle("hidden", Boolean(target));
+  if (target) {
+    const hideExampleAssignments = exampleTutorialState !== "complete" && isExampleTutorial(target);
+    const masks = target.masks.map((mask) => ({
+      ...mask,
+      displayColor: hideExampleAssignments ? "#a8afb9" : mappedColor(mask),
+    }));
+    await targetCanvas.setAsset(target, masks);
+    assetStatus("target", target);
+  } else {
+    await targetCanvas.setAsset(null);
+    canvasStatus("target");
+  }
+
+  if (!source || (elements.sourceVideo.paused && !sourceTimeline.drag)) {
+    $("sourceCanvasHost").classList.remove("playing");
+  }
+
+  renderModelStatus();
+  updateGenerateState();
+  scheduleStatePoll();
+  scheduleMappingHint();
+}
+
+function renderModelStatus() {
+  // Model state is reflected by the generation control and the page content.
+}
+
+function assetStatus(kind, asset) {
+  if (asset.sam_status === "queued") canvasStatus(kind, "Preparing mask features…");
+  else if (asset.sam_status === "error") canvasStatus(kind, asset.sam_error || "SAM failed", true);
+  else canvasStatus(kind);
+}
+
+function renderSourceRail() {
+  elements.sourceRail.replaceChildren();
+  sessionState.sources.forEach((source, index) => {
+    const button = document.createElement("button");
+    button.className = `source-tile${source.id === selectedSourceId ? " selected" : ""}`;
+    button.setAttribute("aria-label", `Select source video ${index + 1}`);
+    const image = document.createElement("img");
+    image.src = source.thumbnail_url;
+    image.alt = "";
+    button.append(image);
+    button.addEventListener("click", async () => {
+      if (source.id === selectedSourceId) return;
+      await clearCanvasDraft("source");
+      selectedSourceId = source.id;
+      await applyState(sessionState);
+    });
+    elements.sourceRail.append(button);
+  });
+  const upload = document.createElement("button");
+  upload.className = "source-upload-tile";
+  upload.title = "Upload source video";
+  upload.textContent = "+";
+  upload.addEventListener("click", () => elements.sourceFile.click());
+  elements.sourceRail.append(upload);
+}
+
+function renderTargetRail() {
+  elements.targetRail.replaceChildren();
+  sessionState.targets.forEach((target, index) => {
+    const button = document.createElement("button");
+    button.className = `target-tile${target.id === sessionState.selected_target_id ? " selected" : ""}`;
+    button.setAttribute("aria-label", `Select start frame ${index + 1}`);
+    const image = document.createElement("img");
+    image.src = target.thumbnail_url;
+    image.alt = "";
+    button.append(image);
+    button.addEventListener("click", async () => {
+      if (target.id === sessionState.selected_target_id) return;
+      if (!await flushTargetSettings()) return;
+      await clearCanvasDraft("target");
+      try {
+        await applyState(await request(
+          `/api/sessions/${sessionState.id}/targets/${target.id}/selection`,
+          { method: "PUT" },
+        ));
+      } catch (error) {
+        showToast(error.message);
+      }
+    });
+    elements.targetRail.append(button);
+  });
+  const upload = document.createElement("button");
+  upload.className = "target-upload-tile";
+  upload.title = "Upload target image";
+  upload.textContent = "+";
+  upload.addEventListener("click", () => elements.targetFile.click());
+  elements.targetRail.append(upload);
+}
+
+function outputUrl(output) {
+  if (output.local_url) return output.local_url;
+  return `${output.video_url}?v=${encodeURIComponent(output.task_id || output.output_name)}`;
+}
+
+async function rememberGeneratedOutput(output) {
+  let record = { ...output };
+  try {
+    const response = await fetch(outputUrl(output));
+    if (!response.ok) throw new Error(`Could not cache output (${response.status})`);
+    record.local_url = URL.createObjectURL(await response.blob());
+  } catch (_) {
+    // The current server URL remains usable until a later generation replaces it.
+  }
+  generatedOutputs.push(record);
+  while (generatedOutputs.length > 8) {
+    const expired = generatedOutputs.shift();
+    if (expired.local_url) URL.revokeObjectURL(expired.local_url);
+  }
+  renderGeneratedRail();
+  return record;
+}
+
+function clearGeneratedOutputs() {
+  generatedOutputs.forEach((output) => {
+    if (output.local_url) URL.revokeObjectURL(output.local_url);
+  });
+  generatedOutputs = [];
+  renderGeneratedRail();
+}
+
+function showGeneratedOutput(output) {
+  if (!output?.video_url) return;
+  elements.resultVideo.pause();
+  elements.resultVideo.src = outputUrl(output);
+  elements.resultMeta.textContent = output.frames
+    ? `${output.frames} frames · ${output.width}×${output.height} · ${output.fps} fps`
+    : "";
+  elements.resultPanel.classList.remove("hidden");
+  elements.resultVideo.load();
+  elements.resultVideo.play().catch(() => {});
+}
+
+function renderGeneratedRail() {
+  const outputs = generatedOutputs;
+  elements.generatedRail.replaceChildren();
+  elements.generatedRail.classList.toggle("hidden", outputs.length === 0);
+  [...outputs].reverse().forEach((output, index) => {
+    const button = document.createElement("button");
+    button.className = "generated-tile";
+    button.setAttribute("aria-label", `Open generated video ${index + 1}`);
+    const video = document.createElement("video");
+    video.src = outputUrl(output);
+    video.muted = true;
+    video.loop = true;
+    video.playsInline = true;
+    video.preload = "metadata";
+    button.append(video);
+    button.addEventListener("click", () => showGeneratedOutput(output));
+    elements.generatedRail.append(button);
+  });
+}
+
+function scheduleStatePoll() {
+  clearTimeout(pollTimer);
+  const preparing = sessionState?.targets.some((target) => target.sam_status === "queued") ||
+    sessionState?.sources.some((source) => source.sam_status === "queued") ||
+    ["loading", "generating"].includes(sessionState?.wan_status) ||
+    Boolean(activeGeneration || sessionState?.active_generation);
+  const delay = preparing ? 900 : (document.hidden ? 60000 : 30000);
+  pollTimer = setTimeout(async () => {
+    try {
+      await applyState(await request(`/api/sessions/${sessionState.id}`));
+    } catch (_) {
+      scheduleStatePoll();
+    }
+  }, delay);
+}
+
+async function upload(kind, file) {
+  if (!file || !sessionState) return;
+  const body = new FormData();
+  body.append("file", file);
+  canvasStatus(kind, "Uploading…");
+  try {
+    predictionControllers[kind]?.abort();
+    const endpoint = kind === "target" ? "targets" : "sources";
+    const next = await request(`/api/sessions/${sessionState.id}/${endpoint}`, { method: "POST", body });
+    if (kind === "source") {
+      await clearCanvasDraft("source");
+      selectedSourceId = next.sources.at(-1)?.id || selectedSourceId;
+    }
+    if (kind === "target") {
+      targetCanvas.clearDraft();
+    }
+    await applyState(next);
+    if (kind === "source") sourceCanvas.setPinned(true);
+    if (kind === "target") targetCanvas.setPinned(true);
+  } catch (error) {
+    canvasStatus(kind);
+    showToast(error.message);
+  }
+}
+
+async function predictMask(kind, payload) {
+  const assetId = kind === "target" ? selectedTarget()?.id : selectedSourceId;
+  if (!assetId) return;
+  hoverControllers[kind]?.abort();
+  clearTimeout(hoverTimers[kind]);
+  payload.transient = false;
+  predictionControllers[kind]?.abort();
+  const controller = new AbortController();
+  predictionControllers[kind] = controller;
+  canvasStatus(kind, "Updating mask…");
+  try {
+    const result = await request(
+      `/api/sessions/${sessionState.id}/assets/${kind}/${assetId}/predict`,
+      { ...jsonOptions(payload), signal: controller.signal },
+    );
+    const canvas = kind === "source" ? sourceCanvas : targetCanvas;
+    await canvas.setDraft(result.mask, result.prompt_revision);
+    canvasStatus(kind);
+  } catch (error) {
+    if (error.name === "AbortError" || error.status === 409) return;
+    canvasStatus(kind, error.message, true);
+    showToast(error.message);
+  }
+}
+
+function scheduleHover(kind, payload) {
+  clearTimeout(hoverTimers[kind]);
+  hoverControllers[kind]?.abort();
+  hoverControllers[kind] = null;
+  const canvas = kind === "source" ? sourceCanvas : targetCanvas;
+  if (!payload) {
+    canvas.clearHoverDraft();
+    return;
+  }
+  hoverTimers[kind] = setTimeout(async () => {
+    const assetId = kind === "target" ? selectedTarget()?.id : selectedSourceId;
+    if (!assetId) return;
+    const controller = new AbortController();
+    hoverControllers[kind] = controller;
+    try {
+      const result = await request(
+        `/api/sessions/${sessionState.id}/assets/${kind}/${assetId}/predict`,
+        { ...jsonOptions(payload), signal: controller.signal },
+      );
+      await canvas.setHoverDraft(result.mask, result.prompt_revision);
+    } catch (error) {
+      if (error.name !== "AbortError" && error.status !== 409) showToast(error.message);
+    }
+  }, 85);
+}
+
+async function commitMask(kind) {
+  const canvas = kind === "source" ? sourceCanvas : targetCanvas;
+  const assetId = kind === "target" ? selectedTarget()?.id : selectedSourceId;
+  const payload = canvas.promptPayload();
+  try {
+    const next = await request(
+      `/api/sessions/${sessionState.id}/assets/${kind}/${assetId}/masks`,
+      jsonOptions({
+        asset_revision: payload.asset_revision,
+        prompt_revision: payload.prompt_revision,
+        frame_index: payload.frame_index,
+      }),
+    );
+    canvas.clearDraft();
+    await applyState(next);
+  } catch (error) {
+    showToast(error.message);
+  }
+}
+
+async function clearCanvasDraft(kind) {
+  const canvas = kind === "source" ? sourceCanvas : targetCanvas;
+  const asset = canvas.asset;
+  predictionControllers[kind]?.abort();
+  hoverControllers[kind]?.abort();
+  clearTimeout(hoverTimers[kind]);
+  predictionControllers[kind] = null;
+  canvas.clearDraft();
+  if (!asset || !sessionState) return;
+  try {
+    await request(
+      `/api/sessions/${sessionState.id}/assets/${kind}/${asset.id}/draft?revision=${asset.revision}`,
+      { method: "DELETE" },
+    );
+  } catch (error) {
+    if (error.status !== 404 && error.status !== 409) showToast(error.message);
+  }
+}
+
+async function discardUnrecoverableDrafts(state) {
+  const assets = [];
+  state.targets.forEach((target) => assets.push(["target", target.id, target.revision]));
+  state.sources.forEach((source) => assets.push(["source", source.id, source.revision]));
+  await Promise.allSettled(assets.map(([kind, assetId, revision]) => request(
+    `/api/sessions/${state.id}/assets/${kind}/${assetId}/draft?revision=${revision}`,
+    { method: "DELETE" },
+  )));
+}
+
+function menuButton(label, action, { danger = false, color = null, image = null, detail = null } = {}) {
+  const button = document.createElement("button");
+  button.className = `menu-item${danger ? " danger" : ""}`;
+  if (image) {
+    const preview = document.createElement("img");
+    preview.src = image;
+    preview.alt = "";
+    button.append(preview);
+  }
+  if (color) {
+    const dot = document.createElement("span");
+    dot.className = "color-dot";
+    dot.style.background = color;
+    button.append(dot);
+  }
+  const copy = document.createElement("span");
+  copy.className = "menu-copy";
+  const main = document.createElement("span");
+  main.textContent = label;
+  copy.append(main);
+  if (detail) {
+    const small = document.createElement("small");
+    small.textContent = detail;
+    copy.append(small);
+  }
+  button.append(copy);
+  button.addEventListener("click", async (event) => {
+    event.stopPropagation();
+    hideMenu();
+    await action();
+  });
+  return button;
+}
+
+function menuLabel(text) {
+  const label = document.createElement("div");
+  label.className = "menu-label";
+  label.textContent = text;
+  return label;
+}
+
+function divider() {
+  const value = document.createElement("div");
+  value.className = "menu-divider";
+  return value;
+}
+
+function showMenu(children, x, y) {
+  elements.contextMenu.replaceChildren(...children);
+  elements.contextMenu.classList.remove("hidden");
+  const width = elements.contextMenu.offsetWidth;
+  const height = elements.contextMenu.offsetHeight;
+  elements.contextMenu.style.left = `${Math.min(x + 6, window.innerWidth - width - 8)}px`;
+  elements.contextMenu.style.top = `${Math.min(y + 6, window.innerHeight - height - 8)}px`;
+}
+
+function hideMenu() {
+  elements.contextMenu.classList.add("hidden");
+}
+
+function openSourceMenu(masks, x, y) {
+  const source = selectedSource();
+  if (!source) return;
+  const children = [menuLabel(masks.length > 1 ? "Masks here" : "Source mask")];
+  masks.forEach((mask) => {
+    const index = source.masks.findIndex((value) => value.id === mask.id);
+    children.push(menuButton(
+      masks.length > 1 ? `Remove S${sessionState.sources.indexOf(source) + 1} · M${index + 1}` : "Remove mask",
+      () => removeMask("source", source.id, mask.id),
+      { danger: true, color: mask.color },
+    ));
+  });
+  showMenu(children, x, y);
+}
+
+function openTargetMenu(masks, x, y, selected = null) {
+  const target = selectedTarget();
+  if (!target) return;
+  const mask = selected || masks[0];
+  const targetIndex = target.masks.findIndex((value) => value.id === mask.id);
+  const children = [];
+  if (masks.length > 1 && !selected) {
+    children.push(menuLabel("Target masks here"));
+    masks.forEach((candidate) => {
+      const index = target.masks.findIndex((value) => value.id === candidate.id);
+      children.push(menuButton(`Target mask ${index + 1}`, () => openTargetMenu(masks, x, y, candidate), {
+        color: candidate.displayColor,
+      }));
+    });
+    children.push(divider());
+  }
+  children.push(menuLabel(`Target mask ${targetIndex + 1}`));
+  children.push(menuButton("Remove mask", () => removeMask("target", target.id, mask.id), { danger: true }));
+  const stored = target.masks.find((value) => value.id === mask.id);
+  if (stored?.source_id) {
+    children.push(menuButton("Clear motion", () => setMapping(mask.id, null, null)));
+  }
+  const sources = sourceMaskEntries();
+  if (sources.length) {
+    children.push(divider(), menuLabel("Use motion from"));
+    sources.forEach((entry) => {
+      children.push(menuButton(
+        `${entry.sourceLabel} · ${entry.maskLabel}`,
+        () => setMapping(mask.id, entry.source.id, entry.id),
+        { color: entry.color, image: entry.preview_url, detail: entry.source.name },
+      ));
+    });
+  }
+  showMenu(children, x, y);
+}
+
+async function removeMask(kind, assetId, maskId) {
+  try {
+    const next = await request(
+      `/api/sessions/${sessionState.id}/assets/${kind}/${assetId}/masks/${maskId}`,
+      { method: "DELETE" },
+    );
+    await applyState(next);
+  } catch (error) {
+    showToast(error.message);
+  }
+}
+
+async function setMapping(targetMaskId, sourceId, sourceMaskId) {
+  const target = selectedTarget();
+  if (!target) return false;
+  try {
+    const next = await request(
+      `/api/sessions/${sessionState.id}/targets/${target.id}/masks/${targetMaskId}/mapping`,
+      jsonOptions({ source_id: sourceId, source_mask_id: sourceMaskId }, "PUT"),
+    );
+    targetCanvas.setDropHover(null);
+    await applyState(next);
+    return true;
+  } catch (error) {
+    showToast(error.message);
+    return false;
+  }
+}
+
+function maskShape(element, mask, color) {
+  element.replaceChildren();
+  if (!mask?.alpha) return false;
+  let minX = mask.alpha.width;
+  let minY = mask.alpha.height;
+  let maxX = -1;
+  let maxY = -1;
+  for (let y = 0; y < mask.alpha.height; y += 1) {
+    for (let x = 0; x < mask.alpha.width; x += 1) {
+      if (!mask.pixels[y * mask.alpha.width + x]) continue;
+      minX = Math.min(minX, x);
+      minY = Math.min(minY, y);
+      maxX = Math.max(maxX, x);
+      maxY = Math.max(maxY, y);
+    }
+  }
+  if (maxX < 0 || maxY < 0) return false;
+  const width = maxX - minX + 1;
+  const height = maxY - minY + 1;
+  const raster = document.createElement("canvas");
+  raster.width = width;
+  raster.height = height;
+  const context = raster.getContext("2d");
+  context.fillStyle = color || mask.color || "#25d5f4";
+  context.fillRect(0, 0, raster.width, raster.height);
+  context.globalCompositeOperation = "destination-in";
+  context.drawImage(mask.alpha, minX, minY, width, height, 0, 0, width, height);
+  const image = document.createElement("img");
+  image.src = raster.toDataURL("image/png");
+  image.alt = "";
+  element.append(image);
+  return true;
+}
+
+function hideDragGhost() {
+  elements.dragGhost.classList.add("hidden");
+  elements.dragGhost.replaceChildren();
+  draggedMaskId = null;
+}
+
+function playMappingDrop(target, color, x, y) {
+  const pop = document.createElement("span");
+  pop.className = "mapping-drop-pop";
+  pop.style.left = `${x}px`;
+  pop.style.top = `${y}px`;
+  if (!maskShape(pop, target, color)) return;
+  document.body.append(pop);
+  pop.addEventListener("animationend", () => pop.remove(), { once: true });
+}
+
+function scheduleMappingHint() {
+  if (exampleTutorialState === "complete" || exampleTutorialScheduled) return;
+  const target = selectedTarget();
+  const source = selectedSource();
+  if (!isExampleTutorial(target) || !source) return;
+  const targetRecord = target.masks.find((value) =>
+    value.source_id === source.id && sourceCanvas.getMask(value.source_mask_id)
+  );
+  const sourceMask = targetRecord && sourceCanvas.getMask(targetRecord.source_mask_id);
+  const targetMask = targetRecord && targetCanvas.getMask(targetRecord.id);
+  if (!targetRecord || !sourceMask || !targetMask) return;
+  exampleTutorialScheduled = true;
+  mappingHintTimer = setTimeout(() => {
+    const start = sourceCanvas.maskClientCenter(sourceMask.id);
+    const end = targetCanvas.maskClientCenter(targetMask.id);
+    if (!start || !end) {
+      exampleTutorialScheduled = false;
+      return;
+    }
+    const token = document.createElement("span");
+    token.className = "mapping-hint-token";
+    token.style.left = `${start.x}px`;
+    token.style.top = `${start.y}px`;
+    token.style.setProperty("--hint-x", `${end.x - start.x}px`);
+    token.style.setProperty("--hint-y", `${end.y - start.y}px`);
+    maskShape(token, sourceMask, sourceMask.color);
+    const pointer = document.createElement("span");
+    pointer.className = "mapping-hint-pointer";
+    pointer.innerHTML = '<svg viewBox="0 0 24 28" aria-hidden="true"><path d="M3 2v20l5.2-5.1 3.6 8.1 4.1-1.9-3.7-7.8H20L3 2Z"></path></svg>';
+    const copy = document.createElement("span");
+    copy.className = "mapping-hint-copy";
+    copy.textContent = "Drag a source mask onto a target mask";
+    elements.mappingHint.replaceChildren(token, pointer, copy);
+    pointer.style.left = `${start.x + 8}px`;
+    pointer.style.top = `${start.y + 8}px`;
+    pointer.style.setProperty("--hint-x", `${end.x - start.x}px`);
+    pointer.style.setProperty("--hint-y", `${end.y - start.y}px`);
+    elements.mappingHint.classList.remove("hidden");
+    token.addEventListener("animationend", () => {
+      playMappingDrop(targetMask, sourceMask.color, end.x, end.y);
+      exampleTutorialState = "complete";
+      elements.mappingHint.classList.add("hidden");
+      elements.mappingHint.replaceChildren();
+      const currentTarget = selectedTarget();
+      if (currentTarget?.id === target.id) {
+        const masks = currentTarget.masks.map((mask) => ({
+          ...mask,
+          displayColor: mappedColor(mask),
+        }));
+        targetCanvas.setAsset(currentTarget, masks);
+      }
+    }, { once: true });
+  }, 850);
+}
+
+function mappingDrag(phase, mask, x, y) {
+  if (phase === "cancel") {
+    hideDragGhost();
+    targetCanvas.setDropHover(null);
+    return;
+  }
+  if (phase === "move") {
+    if (draggedMaskId !== mask.id) {
+      elements.dragGhost.replaceChildren();
+      const shape = document.createElement("span");
+      shape.className = "drag-mask-shape";
+      maskShape(shape, mask, mask.color);
+      elements.dragGhost.append(shape);
+      draggedMaskId = mask.id;
+    }
+    elements.dragGhost.style.left = `${x - 8}px`;
+    elements.dragGhost.style.top = `${y - 8}px`;
+    elements.dragGhost.classList.remove("hidden");
+    const target = targetCanvas.masksAtClient(x, y)[0] || null;
+    targetCanvas.setDropHover(target?.id || null);
+    return;
+  }
+  const target = targetCanvas.masksAtClient(x, y)[0] || null;
+  hideDragGhost();
+  targetCanvas.setDropHover(null);
+  if (target) {
+    playMappingDrop(target, mask.color, x, y);
+    setMapping(target.id, selectedSourceId, mask.id);
+  }
+}
+
+async function selectSourceFrame(frame, final) {
+  const source = selectedSource();
+  if (!source || !final) {
+    if (source) $("sourceCanvasHost").classList.add("playing");
+    return true;
+  }
+  const selected = Math.max(0, Math.min(source.frame_count - 1, Math.round(frame)));
+  $("sourceCanvasHost").classList.add("playing");
+  frameController?.abort();
+  frameController = new AbortController();
+  try {
+    canvasStatus("source", "Preparing selected frame…");
+    const next = await request(
+      `/api/sessions/${sessionState.id}/sources/${source.id}`,
+      { ...jsonOptions({ current_frame: selected }, "PATCH"), signal: frameController.signal },
+    );
+    await applyState(next);
+    return true;
+  } catch (error) {
+    if (error.name === "AbortError" || error.status === 409) return false;
+    showToast(error.message);
+    await applyState(sessionState);
+    return false;
+  }
+}
+
+async function updateSourceTrim(start, end) {
+  const source = selectedSource();
+  if (!source) return false;
+  const excluded = source.masks.filter((mask) =>
+    (mask.frame_index < start || mask.frame_index > end) &&
+    sessionState.targets.some((target) => target.masks.some((value) =>
+      value.source_id === source.id && value.source_mask_id === mask.id,
+    )),
+  );
+  if (excluded.length && !window.confirm(
+    `This trim excludes ${excluded.length} mapped source mask${excluded.length === 1 ? "" : "s"} and will clear those mappings. Continue?`,
+  )) return false;
+  try {
+    const next = await request(
+      `/api/sessions/${sessionState.id}/sources/${source.id}`,
+      jsonOptions({ trim_start: start, trim_end: end }, "PATCH"),
+    );
+    await applyState(next);
+    return true;
+  } catch (error) {
+    showToast(error.message);
+    await applyState(sessionState);
+    return false;
+  }
+}
+
+function updateGenerateState() {
+  const mode = elements.guidanceMode.value;
+  const target = selectedTarget();
+  const mapped = target?.masks.some((mask) => mask.source_id && mask.source_mask_id);
+  const ready = Boolean(target && (mode === "base_cfg" || mapped));
+  if (activeGeneration) {
+    elements.generateButton.disabled = true;
+    elements.generateHint.textContent = "GPU work continues in the background; you can keep editing inputs.";
+    return;
+  }
+  elements.generateButton.disabled = !ready;
+  elements.generateHint.textContent = !target
+    ? "Upload a target image first."
+    : mode === "base_cfg"
+      ? "Base Wan ignores source masks."
+      : mapped
+        ? ""
+        : "Create source and target masks, then drag a source mask onto a target mask.";
+}
+
+function setGenerateButtonStatus(label, count = "") {
+  elements.generateButtonLabel.textContent = label;
+  elements.generateButtonCount.textContent = count;
+}
+
+async function generate() {
+  if (activeGeneration) return;
+  const formSignature = generationFormSignature();
+  const payload = {
+    prompt: elements.prompt.value,
+    negative_prompt: elements.negativePrompt.value || null,
+    steps: Number.parseInt(elements.steps.value, 10),
+    seed: Number.parseInt(elements.seed.value, 10),
+    guidance_mode: elements.guidanceMode.value,
+    text_guidance_scale: Number.parseFloat(elements.textGuidance.value),
+    motion_guidance_scale: Number.parseFloat(elements.motionGuidance.value),
+    lora_scale: Number.parseFloat(elements.loraScale.value),
+  };
+  try {
+    activeGeneration = "gradio-pending";
+    activeGenerationForm = formSignature;
+    activeGenerationStatus = "queued";
+    elements.generateButton.disabled = true;
+    updateProgress({ stage: "Queued for GPU", progress_current: null, progress_total: null });
+    const result = await request(
+      `/api/sessions/${sessionState.id}/generate`,
+      {
+        ...jsonOptions(payload),
+        onGradioStatus: (message) => {
+          const progress = message.progress_data?.[0];
+          if (progress?.desc) {
+            updateProgress({
+              stage: progress.desc,
+              progress_current: progress.index,
+              progress_total: progress.length,
+            });
+          } else if (message.stage === "pending") {
+            const position = Number.isFinite(message.position) ? ` · ${message.position + 1} ahead` : "";
+            updateProgress({ stage: `Queued for GPU${position}`, progress_current: null, progress_total: null });
+          } else if (message.stage === "generating") {
+            updateProgress({ stage: "Starting generation", progress_current: null, progress_total: null });
+          }
+        },
+      },
+    );
+    activeGeneration = result.task_id;
+    activeGenerationForm = formSignature;
+    activeGenerationStatus = "queued";
+    elements.generateButton.disabled = true;
+    updateProgress({ stage: "Queued", progress_current: null, progress_total: null });
+    elements.generateHint.textContent = "GPU work continues in the background; you can keep editing inputs.";
+    renderModelStatus();
+    pollGeneration(result.task_id);
+  } catch (error) {
+    activeGeneration = null;
+    activeGenerationForm = null;
+    activeGenerationStatus = null;
+    updateProgress(null);
+    updateGenerateState();
+    showToast(error.message);
+  }
+}
+
+function updateProgress(task = null) {
+  if (!task) {
+    elements.generateButton.classList.remove("generating");
+    elements.generateButtonProgress.classList.remove("indeterminate");
+    elements.generateButtonProgress.style.width = "0";
+    setGenerateButtonStatus("Generate video");
+    return;
+  }
+  elements.generateButton.classList.add("generating");
+  const stage = task.stage || (task.status === "queued" ? "Queued" : "Working");
+  const determinate = Number.isFinite(task.progress_current) && Number.isFinite(task.progress_total) && task.progress_total > 0;
+  setGenerateButtonStatus(stage, determinate ? `${task.progress_current} / ${task.progress_total}` : "");
+  elements.generateButtonProgress.classList.toggle("indeterminate", !determinate);
+  if (determinate) {
+    elements.generateButtonProgress.style.width = `${Math.max(0, Math.min(100, task.progress_current / task.progress_total * 100))}%`;
+  } else {
+    elements.generateButtonProgress.style.width = "";
+  }
+}
+
+async function pollGeneration(taskId) {
+  if (activeGeneration !== taskId) return;
+  try {
+    const task = await request(`/api/tasks/${taskId}`);
+    updateProgress(task);
+    if (task.status === "queued") {
+      activeGenerationStatus = "queued";
+    } else if (task.status === "running") {
+      activeGenerationStatus = "running";
+    } else if (task.status === "complete") {
+      const formChanged = activeGenerationForm !== null &&
+        activeGenerationForm !== generationFormSignature();
+      activeGeneration = null;
+      activeGenerationForm = null;
+      activeGenerationStatus = null;
+      updateProgress(null);
+      const previousInputs = task.result.inputs_changed || formChanged;
+      const rememberedOutput = await rememberGeneratedOutput(task.result);
+      showGeneratedOutput(rememberedOutput);
+      if (previousInputs) {
+        showToast("Generation finished from the inputs captured when you clicked Generate.");
+      }
+      try {
+        await applyState(await request(`/api/sessions/${sessionState.id}`));
+      } catch (_) {
+        renderModelStatus();
+        updateGenerateState();
+      }
+      return;
+    } else if (["failed", "superseded"].includes(task.status)) {
+      activeGeneration = null;
+      activeGenerationForm = null;
+      activeGenerationStatus = null;
+      updateProgress(null);
+      showToast(task.error || "Generation failed");
+      renderModelStatus();
+      updateGenerateState();
+      return;
+    }
+    renderModelStatus();
+  } catch (error) {
+    activeGeneration = null;
+    activeGenerationForm = null;
+    activeGenerationStatus = null;
+    updateProgress(null);
+    showToast(error.message);
+    renderModelStatus();
+    updateGenerateState();
+    return;
+  }
+  setTimeout(() => pollGeneration(taskId), 1000);
+}
+
+function generationFormSignature() {
+  return JSON.stringify([
+    elements.prompt.value,
+    elements.negativePrompt.value,
+    elements.guidanceMode.value,
+    elements.steps.value,
+    elements.seed.value,
+    elements.textGuidance.value,
+    elements.motionGuidance.value,
+    elements.loraScale.value,
+  ]);
+}
+
+elements.sourceRailUpload.addEventListener("click", () => elements.sourceFile.click());
+elements.sourceEmpty.addEventListener("click", () => elements.sourceFile.click());
+elements.sourceFile.addEventListener("change", () => {
+  upload("source", elements.sourceFile.files[0]);
+  elements.sourceFile.value = "";
+});
+elements.targetEmpty.addEventListener("click", () => elements.targetFile.click());
+elements.targetFile.addEventListener("change", () => {
+  const file = elements.targetFile.files[0];
+  elements.targetFile.value = "";
+  flushTargetSettings().then((saved) => {
+    if (saved) upload("target", file);
+  });
+});
+elements.removeSource.addEventListener("click", async () => {
+  const source = selectedSource();
+  if (!source) return;
+  predictionControllers.source?.abort();
+  try {
+    const next = await request(`/api/sessions/${sessionState.id}/sources/${source.id}`, { method: "DELETE" });
+    await applyState(next);
+  } catch (error) {
+    showToast(error.message);
+  }
+});
+elements.removeTarget.addEventListener("click", async () => {
+  const target = selectedTarget();
+  if (!target) return;
+  predictionControllers.target?.abort();
+  try {
+    const next = await request(`/api/sessions/${sessionState.id}/targets/${target.id}`, { method: "DELETE" });
+    await applyState(next);
+  } catch (error) {
+    showToast(error.message);
+  }
+});
+
+elements.resultClose.addEventListener("click", () => {
+  elements.resultVideo.pause();
+  setMediaLoading("result", "video", false);
+  elements.resultPanel.classList.add("hidden");
+});
+
+elements.startOver.addEventListener("click", async () => {
+  if (!sessionState || !window.confirm("Remove the current media and start over?")) return;
+  predictionControllers.source?.abort();
+  predictionControllers.target?.abort();
+  activeGeneration = null;
+  activeGenerationForm = null;
+  activeGenerationStatus = null;
+  try {
+    const oldSettingsKey = targetSettingsStorageKey(sessionState.id);
+    await request(`/api/sessions/${sessionState.id}`, { method: "DELETE" });
+    let next = await request("/api/sessions", { method: "POST" });
+    for (const source of [...next.sources]) {
+      next = await request(`/api/sessions/${next.id}/sources/${source.id}`, { method: "DELETE" });
+    }
+    for (const target of [...next.targets]) {
+      next = await request(`/api/sessions/${next.id}/targets/${target.id}`, { method: "DELETE" });
+    }
+    sessionStorage.setItem("whatmoves-session", next.id);
+    sessionStorage.removeItem(oldSettingsKey);
+    selectedSourceId = null;
+    formTargetId = null;
+    settingsDirty = false;
+    targetSettingsSessionId = null;
+    targetSettings = {};
+    exampleTutorialState = "complete";
+    exampleTutorialScheduled = false;
+    clearGeneratedOutputs();
+    elements.resultVideo.pause();
+    elements.resultVideo.removeAttribute("src");
+    setMediaLoading("result", "video", false);
+    elements.resultPanel.classList.add("hidden");
+    updateProgress(null);
+    await applyState(next);
+  } catch (error) {
+    showToast(error.message);
+  }
+});
+
+elements.sourceMaskMode.addEventListener("click", async () => {
+  if (sourceCanvas.pinned) {
+    sourceCanvas.setPinned(false);
+    return;
+  }
+  sourceTimeline.pause();
+  if (await selectSourceFrame(sourceTimeline.currentFrame, true)) {
+    sourceCanvas.setPinned(true);
+  }
+});
+elements.targetMaskMode.addEventListener("click", () => targetCanvas.togglePinned());
+elements.sourceAddMask.addEventListener("click", () => commitMask("source"));
+elements.targetAddMask.addEventListener("click", () => commitMask("target"));
+elements.sourceCancelDraft.addEventListener("click", () => clearCanvasDraft("source"));
+elements.targetCancelDraft.addEventListener("click", () => clearCanvasDraft("target"));
+[
+  elements.prompt,
+  elements.negativePrompt,
+  elements.guidanceMode,
+  elements.steps,
+  elements.seed,
+  elements.textGuidance,
+  elements.motionGuidance,
+  elements.loraScale,
+].forEach((element) => {
+  element.addEventListener("input", scheduleTargetSettingsSave);
+  element.addEventListener("change", scheduleTargetSettingsSave);
+});
+elements.guidanceMode.addEventListener("change", updateGenerateState);
+elements.generateButton.addEventListener("click", generate);
+
+window.addEventListener("keydown", (event) => {
+  if (event.key === "Escape") {
+    hideMenu();
+    if (sourceCanvas.isMasking()) clearCanvasDraft("source");
+    if (targetCanvas.isMasking()) clearCanvasDraft("target");
+  }
+});
+document.addEventListener("pointerdown", (event) => {
+  if (!elements.contextMenu.contains(event.target)) hideMenu();
+});
+
+async function start() {
+  try {
+    let initial = null;
+    let resumed = false;
+    const savedSession = sessionStorage.getItem("whatmoves-session");
+    if (savedSession) {
+      try {
+        initial = await request(`/api/sessions/${savedSession}`);
+        resumed = true;
+      } catch (error) {
+        if (error.status !== 404) throw error;
+        sessionStorage.removeItem("whatmoves-session");
+      }
+    }
+    if (!initial) {
+      initial = await request("/api/sessions", { method: "POST" });
+      sessionStorage.setItem("whatmoves-session", initial.id);
+    }
+    if (resumed) await discardUnrecoverableDrafts(initial);
+    await applyState(initial);
+    if (!activeGeneration && initial.latest_generation) {
+      activeGeneration = initial.latest_generation;
+      activeGenerationForm = null;
+      activeGenerationStatus = "queued";
+      updateGenerateState();
+      pollGeneration(activeGeneration);
+    }
+  } catch (error) {
+    const message = `Could not start the app: ${error.message}`;
+    elements.generateHint.textContent = message;
+    showToast(message);
+  }
+}
+
+start();

app/static/gradio-client.js

@@ -0,0 +1,3 @@
+var Mt=Object.create;var je=Object.defineProperty;var Gt=Object.getOwnPropertyDescriptor;var jt=Object.getOwnPropertyNames;var Ft=Object.getPrototypeOf,qt=Object.prototype.hasOwnProperty;var Bt=(e,t)=>()=>{try{return t||e((t={exports:{}}).exports,t),t.exports}catch(n){throw t=0,n}};var zt=(e,t,n,s)=>{if(t&&typeof t=="object"||typeof t=="function")for(let r of jt(t))!qt.call(e,r)&&r!==n&&je(e,r,{get:()=>t[r],enumerable:!(s=Gt(t,r))||s.enumerable});return e};var Jt=(e,t,n)=>(n=e!=null?Mt(Ft(e)):{},zt(t||!e||!e.__esModule?je(n,"default",{value:e,enumerable:!0}):n,e));var Pt=Bt((ss,At)=>{"use strict";At.exports=function(){throw new Error("ws does not work in the browser. Browser clients must use the native WebSocket object")}});var ye=new Intl.Collator(0,{numeric:1}).compare;function re(e,t,n){return e=e.split("."),t=t.split("."),ye(e[0],t[0])||ye(e[1],t[1])||(t[2]=t.slice(2).join("."),n=/[.-]/.test(e[2]=e.slice(2).join(".")),n==/[.-]/.test(t[2])?ye(e[2],t[2]):n?-1:1)}var Fe="host";var oe="queue/data",qe="queue/join",we="upload",Be="login",ze="config",Je="info",We="runtime",He="sleeptime",Ve="heartbeat",Ke="component_server",Ze="reset",Qe="cancel";var Xe="https://gradio-space-api-fetcher-v2.hf.space/api";var ie="This application is currently busy. Please try again. ",k="Connection errored out. ",N="Could not resolve app config. ",Ye="Could not get space status. ",et="Could not get API info. ",V="Space metadata could not be loaded. ",tt="Invalid URL. A full URL path is required.",nt="Not authorized to access this space. ",be="Invalid credentials. Could not login. ",st="Login credentials are required to access this space.",rt="File system access is only available in Node.js environments",ve="Root URL not found in client config",ot="Error uploading file";function Ee(e,t,n){return t.startsWith("http://")||t.startsWith("https://")?n?e:t:e+t}async function Se(e,t,n){try{return(await(await fetch(`https://huggingface.co/api/spaces/${e}/jwt`,{headers:{Authorization:`Bearer ${t}`,...n?{Cookie:n}:{}}})).json()).token||!1}catch{return!1}}function it(e){let t={};return e.forEach(({api_name:n,id:s})=>{n&&(t[n]=s)}),t}async function at(e){let t=this.options.hf_token?{Authorization:`Bearer ${this.options.hf_token}`}:{};if(t["Content-Type"]="application/json",typeof window<"u"&&window.gradio_config&&location.origin!=="http://localhost:9876"&&!window.gradio_config.dev_mode){let n=window.gradio_config.root,s=window.gradio_config,r=Ee(e,s.root,!1);return s.root=r,{...s,path:n}}else if(e){let n=ce(e,ze),s=await this.fetch(n,{headers:t,credentials:"include"});if(s?.status===401&&!this.options.auth)throw new Error(st);if(s?.status===401&&this.options.auth)throw new Error(be);if(s?.status===200){let r=await s.json();return r.path=r.path??"",r.root=e,r.dependencies?.forEach((o,i)=>{o.id===void 0&&(o.id=i)}),r}else if(s?.status===401)throw new Error(nt);throw new Error(N)}throw new Error(N)}async function pt(){let{http_protocol:e,host:t}=await j(this.app_reference,this.options.hf_token);try{if(this.options.auth){let n=await Re(e,t,this.options.auth,this.fetch,this.options.hf_token);n&&this.set_cookies(n)}}catch(n){throw Error(n.message)}}async function Re(e,t,n,s,r){let o=new FormData;o.append("username",n?.[0]),o.append("password",n?.[1]);let i={};r&&(i.Authorization=`Bearer ${r}`);let a=await s(`${e}//${t}/${Be}`,{headers:i,method:"POST",body:o,credentials:"include"});if(a.status===200)return a.headers.get("set-cookie");throw a.status===401?new Error(be):new Error(V)}function ae(e){if(e.startsWith("http")){let{protocol:t,host:n,pathname:s}=new URL(e);return{ws_protocol:t==="https:"?"wss":"ws",http_protocol:t,host:n+(s!=="/"?s:"")}}else if(e.startsWith("file:"))return{ws_protocol:"ws",http_protocol:"http:",host:"lite.local"};return{ws_protocol:"wss",http_protocol:"https:",host:new URL(e).host}}var pe=e=>{let t=[];return e.split(/,(?=\s*[^\s=;]+=[^\s=;]+)/).forEach(s=>{let[r,o]=s.split(";")[0].split("=");r&&o&&t.push(`${r.trim()}=${o.trim()}`)}),t};var se=/^[a-zA-Z0-9_\-\.]+\/[a-zA-Z0-9_\-\.]+$/,Wt=/.*hf\.space\/{0,1}.*$/;async function j(e,t){let n={};t&&(n.Authorization=`Bearer ${t}`);let s=e.trim().replace(/\/$/,"");if(se.test(s))try{let o=(await(await fetch(`https://huggingface.co/api/spaces/${s}/${Fe}`,{headers:n})).json()).host;return{space_id:e,...ae(o)}}catch{throw new Error(V)}if(Wt.test(s)){let{ws_protocol:r,http_protocol:o,host:i}=ae(s);return{space_id:i.split("/")[0].replace(".hf.space",""),ws_protocol:r,http_protocol:o,host:i}}return{space_id:!1,...ae(s)}}var ce=(...e)=>{try{return e.reduce((t,n)=>(t=t.replace(/\/+$/,""),n=n.replace(/^\/+/,""),new URL(n,t+"/").toString()))}catch{throw new Error(tt)}};function ct(e,t,n){let s={named_endpoints:{},unnamed_endpoints:{}};return Object.keys(e).forEach(r=>{(r==="named_endpoints"||r==="unnamed_endpoints")&&(s[r]={},Object.entries(e[r]).forEach(([o,{parameters:i,returns:a}])=>{let p=t.dependencies.find(l=>l.api_name===o||l.api_name===o.replace("/",""))?.id||n[o.replace("/","")]||-1,c=p!==-1?t.dependencies.find(l=>l.id==p)?.types:{generator:!1,cancel:!1};if(p!==-1&&t.dependencies.find(l=>l.id==p)?.inputs?.length!==i.length){let l=t.dependencies.find(g=>g.id==p).inputs.map(g=>t.components.find(y=>y.id===g)?.type);try{l.forEach((g,y)=>{if(g==="state"){let D={component:"state",example:null,parameter_default:null,parameter_has_default:!0,parameter_name:null,hidden:!0};i.splice(y,0,D)}})}catch(g){console.error(g)}}let d=(l,g,y,D)=>({...l,description:Vt(l?.type,y),type:Ht(l?.type,g,y,D)||""});s[r][o]={parameters:i.map(l=>d(l,l?.component,l?.serializer,"parameter")),returns:a.map(l=>d(l,l?.component,l?.serializer,"return")),type:c}}))}),s}function Ht(e,t,n,s){if(t==="Api")return e.type;switch(e?.type){case"string":return"string";case"boolean":return"boolean";case"number":return"number"}if(n==="JSONSerializable"||n==="StringSerializable")return"any";if(n==="ListStringSerializable")return"string[]";if(t==="Image")return s==="parameter"?"Blob | File | Buffer":"string";if(n==="FileSerializable")return e?.type==="array"?s==="parameter"?"(Blob | File | Buffer)[]":"{ name: string; data: string; size?: number; is_file?: boolean; orig_name?: string}[]":s==="parameter"?"Blob | File | Buffer":"{ name: string; data: string; size?: number; is_file?: boolean; orig_name?: string}";if(n==="GallerySerializable")return s==="parameter"?"[(Blob | File | Buffer), (string | null)][]":"[{ name: string; data: string; size?: number; is_file?: boolean; orig_name?: string}, (string | null))][]"}function Vt(e,t){return t==="GallerySerializable"?"array of [file, label] tuples":t==="ListStringSerializable"?"array of strings":t==="FileSerializable"?"array of files or single file":e?.description}function le(e,t){switch(e.msg){case"send_data":return{type:"data"};case"send_hash":return{type:"hash"};case"queue_full":return{type:"update",status:{queue:!0,message:ie,stage:"error",code:e.code,success:e.success}};case"heartbeat":return{type:"heartbeat"};case"unexpected_error":return{type:"unexpected_error",status:{queue:!0,message:e.message,stage:"error",success:!1}};case"estimation":return{type:"update",status:{queue:!0,stage:t||"pending",code:e.code,size:e.queue_size,position:e.rank,eta:e.rank_eta,success:e.success}};case"progress":return{type:"update",status:{queue:!0,stage:"pending",code:e.code,progress_data:e.progress_data,success:e.success}};case"log":return{type:"log",data:e};case"process_generating":return{type:"generating",status:{queue:!0,message:e.success?null:e.output.error,stage:e.success?"generating":"error",code:e.code,progress_data:e.progress_data,eta:e.average_duration,changed_state_ids:e.success?e.output.changed_state_ids:void 0},data:e.success?e.output:null};case"process_streaming":return{type:"streaming",status:{queue:!0,message:e.output.error,stage:"streaming",time_limit:e.time_limit,code:e.code,progress_data:e.progress_data,eta:e.eta},data:e.output};case"process_completed":return"error"in e.output?{type:"update",status:{queue:!0,title:e.output.title,message:e.output.error,visible:e.output.visible,duration:e.output.duration,stage:"error",code:e.code,success:e.success}}:{type:"complete",status:{queue:!0,message:e.success?void 0:e.output.error,stage:e.success?"complete":"error",code:e.code,progress_data:e.progress_data,changed_state_ids:e.success?e.output.changed_state_ids:void 0},data:e.success?e.output:null};case"process_starts":return{type:"update",status:{queue:!0,stage:"pending",code:e.code,size:e.rank,position:0,success:e.success,eta:e.eta},original_msg:"process_starts"}}return{type:"none",status:{stage:"error",queue:!0}}}var lt=(e=[],t)=>{let n=t?t.parameters:[];if(Array.isArray(e))return e.length>n.length&&console.warn("Too many arguments provided for the endpoint."),e;let s=[],r=Object.keys(e);return n.forEach((o,i)=>{if(e.hasOwnProperty(o.parameter_name))s[i]=e[o.parameter_name];else if(o.parameter_has_default)s[i]=o.parameter_default;else throw new Error(`No value provided for required parameter: ${o.parameter_name}`)}),r.forEach(o=>{if(!n.some(i=>i.parameter_name===o))throw new Error(`Parameter \`${o}\` is not a valid keyword argument. Please refer to the API for usage.`)}),s.forEach((o,i)=>{if(o===void 0&&!n[i].parameter_has_default)throw new Error(`No value provided for required parameter: ${n[i].parameter_name}`)}),s};async function ut(){if(this.api_info)return this.api_info;let{hf_token:e}=this.options,{config:t}=this,n={"Content-Type":"application/json"};if(e&&(n.Authorization=`Bearer ${e}`),!!t)try{let s,r;if(typeof window<"u"&&window.gradio_api_info)r=window.gradio_api_info;else{if(re(t?.version||"2.0.0","3.30")<0)s=await this.fetch(Xe,{method:"POST",body:JSON.stringify({serialize:!1,config:JSON.stringify(t)}),headers:n,credentials:"include"});else{let o=ce(t.root,this.api_prefix,Je);s=await this.fetch(o,{headers:n,credentials:"include"})}if(!s.ok)throw new Error(k);r=await s.json()}return"api"in r&&(r=r.api),r.named_endpoints["/predict"]&&!r.unnamed_endpoints[0]&&(r.unnamed_endpoints[0]=r.named_endpoints["/predict"]),ct(r,t,this.api_map)}catch(s){""+s.message}}async function xe(e,t,n){let s={};this?.options?.hf_token&&(s.Authorization=`Bearer ${this.options.hf_token}`);let r=1e3,o=[],i;for(let a=0;a<t.length;a+=r){let p=t.slice(a,a+r),c=new FormData;p.forEach(l=>{c.append("files",l)});try{let l=n?`${e}${this.api_prefix}/${we}?upload_id=${n}`:`${e}${this.api_prefix}/${we}`;i=await this.fetch(l,{method:"POST",body:c,headers:s,credentials:"include"})}catch(l){throw new Error(k+l.message)}if(!i.ok){let l=await i.text();return{error:`HTTP ${i.status}: ${l}`}}let d=await i.json();d&&o.push(...d)}return{files:o}}async function Ce(e,t,n,s){let r=(Array.isArray(e)?e:[e]).map(i=>i.blob),o=r.filter(i=>i.size>(s??1/0));if(o.length)throw new Error(`File size exceeds the maximum allowed size of ${s} bytes: ${o.map(i=>i.name).join(", ")}`);return await Promise.all(await this.upload_files(t,r,n).then(async i=>{if(i.error)throw new Error(i.error);return i.files?i.files.map((a,p)=>new B({...e[p],path:a,url:`${t}${this.api_prefix}/file=${a}`})):[]}))}var B=class{path;url;orig_name;size;blob;is_stream;mime_type;alt_text;b64;meta={_type:"gradio.FileData"};constructor({path:t,url:n,orig_name:s,size:r,blob:o,is_stream:i,mime_type:a,alt_text:p,b64:c}){this.path=t,this.url=n,this.orig_name=s,this.size=r,this.blob=n?void 0:o,this.is_stream=i,this.mime_type=a,this.alt_text=p,this.b64=c}};var K=class{type;command;meta;fileData;constructor(t,n){this.type="command",this.command=t,this.meta=n}};var Kt=typeof process<"u"&&process.versions&&process.versions.node;function ke(e,t,n){for(;n.length>1;){let r=n.shift();if(typeof r=="string"||typeof r=="number")e=e[r];else throw new Error("Invalid key type")}let s=n.shift();if(typeof s=="string"||typeof s=="number")e[s]=t;else throw new Error("Invalid key type")}async function ue(e,t=void 0,n=[],s=!1,r=void 0){if(Array.isArray(e)){let o=[];return await Promise.all(e.map(async(i,a)=>{let p=n.slice();p.push(String(a));let c=await ue(e[a],s?r?.parameters[a]?.component||void 0:t,p,!1,r);o=o.concat(c)})),o}else{if(globalThis.Buffer&&e instanceof globalThis.Buffer||e instanceof Blob)return[{path:n,blob:new Blob([e]),type:t}];if(typeof e=="object"&&e!==null){let o=[];for(let i of Object.keys(e)){let a=[...n,i],p=e[i];o=o.concat(await ue(p,void 0,a,!1,r))}return o}}return[]}function dt(e,t){let n=t?.dependencies?.find(s=>s.id==e)?.queue;return n!=null?!n:!t.enable_queue}function ft(e,t){return new Promise((n,s)=>{let r=new MessageChannel;r.port1.onmessage=(({data:o})=>{r.port1.close(),n(o)}),window.parent.postMessage(e,t,[r.port2])})}function mt(e){if(typeof e=="string"){if(e.startsWith("http://")||e.startsWith("https://"))return{path:e,url:e,orig_name:e.split("/").pop()??"unknown",meta:{_type:"gradio.FileData"}};if(Kt)return new K("upload_file",{path:e,name:e,orig_path:e})}else{if(typeof File<"u"&&e instanceof File)return new Blob([e]);if(e instanceof Buffer)return new Blob([e]);if(e instanceof Blob)return e}throw new Error("Invalid input: must be a URL, File, Blob, or Buffer object.")}function Z(e,t,n,s,r=!1){if(s==="input"&&!r)throw new Error("Invalid code path. Cannot skip state inputs for input.");if(s==="output"&&r)return e;let o=[],i=0,a=s==="input"?t.inputs:t.outputs;for(let p=0;p<a.length;p++){let c=a[p];if(n.find(l=>l.id===c)?.type==="state"){if(r)if(e.length===a.length){let l=e[i];o.push(l),i++}else o.push(null);else{i++;continue}continue}else{let l=e[i];o.push(l),i++}}return o}async function _t(e,t,n){let s=this;await Zt(s,t);let r=await ue(t,void 0,[],!0,n);return(await Promise.all(r.map(async({path:i,blob:a,type:p})=>{if(!a)return{path:i,type:p};let c=await s.upload_files(e,[a]),d=c.files&&c.files[0];return{path:i,file_url:d,type:p,name:typeof File<"u"&&a instanceof File?a?.name:void 0}}))).forEach(({path:i,file_url:a,type:p,name:c})=>{if(p==="Gallery")ke(t,a,i);else if(a){let d=new B({path:a,orig_name:c});ke(t,d,i)}}),t}async function Zt(e,t){if(!(e.config?.root||e.config?.root_url))throw new Error(ve);await gt(e,t)}async function gt(e,t,n=[]){for(let s in t)t[s]instanceof K?await Qt(e,t,s):typeof t[s]=="object"&&t[s]!==null&&await gt(e,t[s],[...n,s])}async function Qt(e,t,n){let s=t[n],r=e.config?.root||e.config?.root_url;if(!r)throw new Error(ve);try{let o,i;if(typeof process<"u"&&process.versions&&process.versions.node){let d=await import("fs/promises");i=(await import("path")).resolve(process.cwd(),s.meta.path),o=await d.readFile(i)}else throw new Error(rt);let a=new Blob([o],{type:"application/octet-stream"}),p=await e.upload_files(r,[a]),c=p.files&&p.files[0];if(c){let d=new B({path:c,orig_name:s.meta.name||""});t[n]=d}}catch(o){console.error(ot,o)}}async function ht(e,t,n){let s={"Content-Type":"application/json"};this.options.hf_token&&(s.Authorization=`Bearer ${this.options.hf_token}`);try{var r=await this.fetch(e,{method:"POST",body:JSON.stringify(t),headers:{...s,...n},credentials:"include"})}catch{return[{error:k},500]}let o,i;try{o=await r.json(),i=r.status}catch(a){o={error:`Could not parse server response: ${a}`},i=500}return[o,i]}async function Ae(e,t={}){let n=!1,s=!1,r;if(!this.config)throw new Error("Could not resolve app config");if(typeof e=="number")r=this.config.dependencies.find(o=>o.id==e);else{let o=e.replace(/^\//,"");r=this.config.dependencies.find(i=>i.id==this.api_map[o])}return new Promise(async(o,i)=>{let a=this.submit(e,t,null,null,!0),p;for await(let c of a)c.type==="data"&&(s&&o(p),n=!0,p=c),c.type==="status"&&(c.stage==="error"&&i(c),c.stage==="complete"&&(s=!0,n&&o(p)))})}async function Q(e,t,n){let s=t==="subdomain"?`https://huggingface.co/api/spaces/by-subdomain/${e}`:`https://huggingface.co/api/spaces/${e}`,r,o;try{if(r=await fetch(s),o=r.status,o!==200)throw new Error;r=await r.json()}catch{n({status:"error",load_status:"error",message:Ye,detail:"NOT_FOUND"});return}if(!r||o!==200)return;let{runtime:{stage:i},id:a}=r;switch(i){case"STOPPED":case"SLEEPING":n({status:"sleeping",load_status:"pending",message:"Space is asleep. Waking it up...",detail:i}),setTimeout(()=>{Q(e,t,n)},1e3);break;case"PAUSED":n({status:"paused",load_status:"error",message:"This space has been paused by the author. If you would like to try this demo, consider duplicating the space.",detail:i,discussions_enabled:await yt(a)});break;case"RUNNING":case"RUNNING_BUILDING":n({status:"running",load_status:"complete",message:"Space is running.",detail:i});break;case"BUILDING":n({status:"building",load_status:"pending",message:"Space is building...",detail:i}),setTimeout(()=>{Q(e,t,n)},1e3);break;case"APP_STARTING":n({status:"starting",load_status:"pending",message:"Space is starting...",detail:i}),setTimeout(()=>{Q(e,t,n)},1e3);break;default:n({status:"space_error",load_status:"error",message:"This space is experiencing an issue.",detail:i,discussions_enabled:await yt(a)});break}}var Pe=async(e,t)=>{let n=0,s=12,r=5e3;return new Promise(o=>{Q(e,se.test(e)?"space_name":"subdomain",i=>{t(i),i.status==="running"||i.status==="error"||i.status==="paused"||i.status==="space_error"?o():(i.status==="sleeping"||i.status==="building")&&(n<s?(n++,setTimeout(()=>{Pe(e,t).then(o)},r)):o())})})},Xt=/^(?=[^]*\b[dD]iscussions{0,1}\b)(?=[^]*\b[dD]isabled\b)[^]*$/;async function yt(e){try{let t=await fetch(`https://huggingface.co/api/spaces/${e}/discussions`,{method:"HEAD"}),n=t.headers.get("x-error-message");return!(!t.ok||n&&Xt.test(n))}catch{return!1}}async function wt(e,t){let n={};t&&(n.Authorization=`Bearer ${t}`);try{let s=await fetch(`https://huggingface.co/api/spaces/${e}/${We}`,{headers:n});if(s.status!==200)throw new Error("Space hardware could not be obtained.");let{hardware:r}=await s.json();return r.current}catch(s){throw new Error(s.message)}}async function bt(e,t,n){let s={};n&&(s.Authorization=`Bearer ${n}`);let r={seconds:t};try{let o=await fetch(`https://huggingface.co/api/spaces/${e}/${He}`,{method:"POST",headers:{"Content-Type":"application/json",...s},body:JSON.stringify(r)});if(o.status!==200)throw new Error("Could not set sleep timeout on duplicated Space. Please visit *ADD HF LINK TO SETTINGS* to set a timeout manually to reduce billing charges.");return await o.json()}catch(o){throw new Error(o.message)}}var De=["cpu-basic","cpu-upgrade","cpu-xl","t4-small","t4-medium","a10g-small","a10g-large","a10g-largex2","a10g-largex4","a100-large","zero-a10g","h100","h100x8"];async function vt(e,t){let{hf_token:n,private:s,hardware:r,timeout:o,auth:i}=t;if(r&&!De.includes(r))throw new Error(`Invalid hardware type provided. Valid types are: ${De.map(w=>`"${w}"`).join(",")}.`);let{http_protocol:a,host:p}=await j(e,n),c=null;if(i){let w=await Re(a,p,i,fetch);w&&(c=pe(w))}let d={Authorization:`Bearer ${n}`,"Content-Type":"application/json",...c?{Cookie:c.join("; ")}:{}},l=(await(await fetch("https://huggingface.co/api/whoami-v2",{headers:d})).json()).name,g=e.split("/")[1],y={repository:`${l}/${g}`};s&&(y.private=!0);let D;try{r||(D=await wt(e,n))}catch(w){throw Error(V+w.message)}let Y=r||D||"cpu-basic";y.hardware=Y;try{let w=await fetch(`https://huggingface.co/api/spaces/${e}/duplicate`,{method:"POST",headers:d,body:JSON.stringify(y)});if(w.status===409)try{return await z.connect(`${l}/${g}`,t)}catch(L){throw console.error("Failed to connect Client instance:",L),L}else if(w.status!==200)throw new Error(w.statusText);let ee=await w.json();return await bt(`${l}/${g}`,o||300,n),await z.connect(Yt(ee.url),t)}catch(w){throw new Error(w)}}function Yt(e){let t=/https:\/\/huggingface.co\/spaces\/([^/]+\/[^/]+)/,n=e.match(t);if(n)return n[1]}var de=class extends TransformStream{#e="";constructor(t={allowCR:!1}){super({transform:(n,s)=>{for(n=this.#e+n;;){let r=n.indexOf(`
+`),o=t.allowCR?n.indexOf("\r"):-1;if(o!==-1&&o!==n.length-1&&(r===-1||r-1>o)){s.enqueue(n.slice(0,o)),n=n.slice(o+1);continue}if(r===-1)break;let i=n[r-1]==="\r"?r-1:r;s.enqueue(n.slice(0,i)),n=n.slice(r+1)}this.#e=n},flush:n=>{if(this.#e==="")return;let s=t.allowCR&&this.#e.endsWith("\r")?this.#e.slice(0,-1):this.#e;n.enqueue(s)}})}};function Et(e){let t=new TextDecoderStream,n=new de({allowCR:!0});return e.pipeThrough(t).pipeThrough(n)}function St(e){let n=/[:]\s*/.exec(e),s=n&&n.index;if(s)return[e.substring(0,s),e.substring(s+n[0].length)]}function Ie(e,t,n){e.get(t)||e.set(t,n)}async function*tn(e,t){if(!e.body)return;let n=Et(e.body),s,r=n.getReader(),o;for(;;){if(t&&t.aborted)return r.cancel();if(s=await r.read(),s.done)return;if(!s.value){o&&(yield o),o=void 0;continue}let[i,a]=St(s.value)||[];i==="data"?(o||={},o[i]=o[i]?o[i]+`
+`+a:a):i==="event"?(o||={},o[i]=a):i==="id"?(o||={},o[i]=String(+a)===a?+a:a):i==="retry"&&(o||={},o[i]=+a||void 0)}}async function Rt(e,t){let n=new Request(e,t);Ie(n.headers,"Accept","text/event-stream"),Ie(n.headers,"Content-Type","application/json");let s=await fetch(n);if(!s.ok)throw s;return tn(s,n.signal)}async function xt(){let{event_callbacks:e,unclosed_events:t,pending_stream_messages:n,stream_status:s,config:r,jwt:o}=this,i=this;if(!r)throw new Error("Could not resolve app config");s.open=!0;let a=null,p=new URLSearchParams({session_hash:this.session_hash}).toString(),c=new URL(`${r.root}${this.api_prefix}/${oe}?${p}`);if(o&&c.searchParams.set("__sign",o),a=this.stream(c),!a){console.warn("Cannot connect to SSE endpoint: "+c.toString());return}a.onmessage=async function(d){let l=JSON.parse(d.data);if(l.msg==="close_stream"){X(s,i.abort_controller);return}let g=l.event_id;if(!g)await Promise.all(Object.keys(e).map(y=>e[y](l)));else if(e[g]&&r){l.msg==="process_completed"&&["sse","sse_v1","sse_v2","sse_v2.1","sse_v3"].includes(r.protocol)&&t.delete(g);let y=e[g];typeof window<"u"&&typeof document<"u"?setTimeout(y,0,l):y(l)}else n[g]||(n[g]=[]),n[g].push(l)},a.onerror=async function(){await Promise.all(Object.keys(e).map(d=>e[d]({msg:"unexpected_error",message:k})))}}function X(e,t){e&&(e.open=!1,t?.abort())}function Ct(e,t,n){!e[t]?(e[t]=[],n.data.forEach((r,o)=>{e[t][o]=r})):n.data.forEach((r,o)=>{let i=nn(e[t][o],r);e[t][o]=i,n.data[o]=i})}function nn(e,t){return t.forEach(([n,s,r])=>{e=sn(e,s,n,r)}),e}function sn(e,t,n,s){if(t.length===0){if(n==="replace")return s;if(n==="append")return e+s;throw new Error(`Unsupported action: ${n}`)}let r=e;for(let i=0;i<t.length-1;i++)r=r[t[i]];let o=t[t.length-1];switch(n){case"replace":r[o]=s;break;case"append":r[o]+=s;break;case"add":Array.isArray(r)?r.splice(Number(o),0,s):r[o]=s;break;case"delete":Array.isArray(r)?r.splice(Number(o),1):delete r[o];break;default:throw new Error(`Unknown action: ${n}`)}return e}function kt(e,t={}){let n={close:()=>{console.warn("Method not implemented.")},onerror:null,onmessage:null,onopen:null,readyState:0,url:e.toString(),withCredentials:!1,CONNECTING:0,OPEN:1,CLOSED:2,addEventListener:()=>{throw new Error("Method not implemented.")},dispatchEvent:()=>{throw new Error("Method not implemented.")},removeEventListener:()=>{throw new Error("Method not implemented.")}};return Rt(e,t).then(async s=>{n.readyState=n.OPEN;try{for await(let r of s)n.onmessage&&n.onmessage(r);n.readyState=n.CLOSED}catch(r){n.onerror&&n.onerror(r),n.readyState=n.CLOSED}}).catch(s=>{console.error(s),n.onerror&&n.onerror(s),n.readyState=n.CLOSED}),n}function Oe(e,t={},n,s,r){try{let m=function(_){(r||Ot[_.type])&&$t(_)},G=function(){for(me=!0;ne.length>0;)ne.shift()({value:void 0,done:!0})},Ue=function(_){me||(ne.length>0?ne.shift()(_):_e.push(_))},Lt=function(_){Ue(rn(_)),G()},$t=function(_){Ue({value:_,done:!1})},ge=function(){return _e.length>0?Promise.resolve(_e.shift()):me?Promise.resolve({value:void 0,done:!0}):new Promise(_=>ne.push(_))},{hf_token:o}=this.options,{fetch:i,app_reference:a,config:p,session_hash:c,api_info:d,api_map:l,stream_status:g,pending_stream_messages:y,pending_diff_streams:D,event_callbacks:Y,unclosed_events:w,post_data:ee,options:L,api_prefix:J}=this,Te=this;if(!d)throw new Error("No API found");if(!p)throw new Error("Could not resolve app config");let{fn_index:u,endpoint_info:Ne,dependency:W}=on(d,e,l,p),Dt=lt(t,Ne),S,H,$=p.protocol??"ws",Le="",It=()=>Le,f=typeof e=="number"?"/predict":e,te,b=null,R=!1,fe={},F=typeof window<"u"&&typeof document<"u"?new URLSearchParams(window.location.search).toString():"",Ot=L?.events?.reduce((_,U)=>(_[U]=!0,_),{})||{};async function Tt(){let _={stage:"complete",queue:!1,time:new Date};R=_,m({..._,type:"status",endpoint:f,fn_index:u});let U={},q={};$==="ws"?(S&&S.readyState===0?S.addEventListener("open",()=>{S.close()}):S.close(),U={fn_index:u,session_hash:c}):(X(g,Te.abort_controller),G(),U={event_id:b},q={event_id:b,session_hash:c,fn_index:u});try{if(!p)throw new Error("Could not resolve app config");"event_id"in q&&await i(`${p.root}${J}/${Qe}`,{headers:{"Content-Type":"application/json"},method:"POST",body:JSON.stringify(q)}),await i(`${p.root}${J}/${Ze}`,{headers:{"Content-Type":"application/json"},method:"POST",body:JSON.stringify(U)})}catch{console.warn("The `/reset` endpoint could not be called. Subsequent endpoint results may be unreliable.")}}let Nt=async _=>{await this._resolve_hearbeat(_)};async function $e(_){if(!p)return;let U=_.render_id;p.components=[...p.components.filter(E=>E.props.rendered_in!==U),..._.components],p.dependencies=[...p.dependencies.filter(E=>E.rendered_in!==U),..._.dependencies];let q=p.components.some(E=>E.type==="state"),h=p.dependencies.some(E=>E.targets.some(I=>I[1]==="unload"));p.connect_heartbeat=q||h,await Nt(p),m({type:"render",data:_,endpoint:f,fn_index:u})}this.handle_blob(p.root,Dt,Ne).then(async _=>{if(te={data:Z(_,W,p.components,"input",!0)||[],event_data:n,fn_index:u,trigger_id:s},dt(u,p))m({type:"status",endpoint:f,stage:"pending",queue:!1,fn_index:u,time:new Date}),ee(`${p.root}${J}/run${f.startsWith("/")?f:`/${f}`}${F?"?"+F:""}`,{...te,session_hash:c}).then(([h,E])=>{let I=h.data;E==200?(m({type:"data",endpoint:f,fn_index:u,data:Z(I,W,p.components,"output",L.with_null_state),time:new Date,event_data:n,trigger_id:s}),h.render_config&&$e(h.render_config),m({type:"status",endpoint:f,fn_index:u,stage:"complete",eta:h.average_duration,queue:!1,time:new Date})):m({type:"status",stage:"error",endpoint:f,fn_index:u,message:h.error,queue:!1,time:new Date})}).catch(h=>{m({type:"status",stage:"error",message:h.message,endpoint:f,fn_index:u,queue:!1,time:new Date})});else if($=="ws"){let{ws_protocol:h,host:E}=await j(a,o);m({type:"status",stage:"pending",queue:!0,endpoint:f,fn_index:u,time:new Date});let I=new URL(`${h}://${Ee(E,p.path,!0)}/queue/join${F?"?"+F:""}`);this.jwt&&I.searchParams.set("__sign",this.jwt),S=new WebSocket(I),S.onclose=A=>{A.wasClean||m({type:"status",stage:"error",broken:!0,message:k,queue:!0,endpoint:f,fn_index:u,time:new Date})},S.onmessage=function(A){let O=JSON.parse(A.data),{type:v,status:x,data:C}=le(O,fe[u]);if(v==="update"&&x&&!R)m({type:"status",endpoint:f,fn_index:u,time:new Date,...x}),x.stage==="error"&&S.close();else if(v==="hash"){S.send(JSON.stringify({fn_index:u,session_hash:c}));return}else v==="data"?S.send(JSON.stringify({...te,session_hash:c})):v==="complete"?R=x:v==="log"?m({type:"log",title:C.title,log:C.log,level:C.level,endpoint:f,duration:C.duration,visible:C.visible,fn_index:u}):v==="generating"&&m({type:"status",time:new Date,...x,stage:x?.stage,queue:!0,endpoint:f,fn_index:u});C&&(m({type:"data",time:new Date,data:Z(C.data,W,p.components,"output",L.with_null_state),endpoint:f,fn_index:u,event_data:n,trigger_id:s}),R&&(m({type:"status",time:new Date,...R,stage:x?.stage,queue:!0,endpoint:f,fn_index:u}),S.close()))},re(p.version||"2.0.0","3.6")<0&&addEventListener("open",()=>S.send(JSON.stringify({hash:c})))}else if($=="sse"){m({type:"status",stage:"pending",queue:!0,endpoint:f,fn_index:u,time:new Date});var q=new URLSearchParams({fn_index:u.toString(),session_hash:c}).toString();let h=new URL(`${p.root}${J}/${oe}?${F?F+"&":""}${q}`);if(this.jwt&&h.searchParams.set("__sign",this.jwt),H=this.stream(h),!H)return Promise.reject(new Error("Cannot connect to SSE endpoint: "+h.toString()));H.onmessage=async function(E){let I=JSON.parse(E.data),{type:A,status:O,data:v}=le(I,fe[u]);if(A==="update"&&O&&!R)m({type:"status",endpoint:f,fn_index:u,time:new Date,...O}),O.stage==="error"&&(H?.close(),G());else if(A==="data"){let[x,C]=await ee(`${p.root}${J}/queue/data`,{...te,session_hash:c,event_id:b});C!==200&&(m({type:"status",stage:"error",message:k,queue:!0,endpoint:f,fn_index:u,time:new Date}),H?.close(),G())}else A==="complete"?R=O:A==="log"?m({type:"log",title:v.title,log:v.log,level:v.level,endpoint:f,duration:v.duration,visible:v.visible,fn_index:u}):(A==="generating"||A==="streaming")&&m({type:"status",time:new Date,...O,stage:O?.stage,queue:!0,endpoint:f,fn_index:u});v&&(m({type:"data",time:new Date,data:Z(v.data,W,p.components,"output",L.with_null_state),endpoint:f,fn_index:u,event_data:n,trigger_id:s}),R&&(m({type:"status",time:new Date,...R,stage:O?.stage,queue:!0,endpoint:f,fn_index:u}),H?.close(),G()))}}else if($=="sse_v1"||$=="sse_v2"||$=="sse_v2.1"||$=="sse_v3"){m({type:"status",stage:"pending",queue:!0,endpoint:f,fn_index:u,time:new Date});let h="";typeof window<"u"&&typeof document<"u"&&(h=window?.location?.hostname);let I=h.includes(".dev.")?`https://moon-${h.split(".")[1]}.dev.spaces.huggingface.tech`:"https://huggingface.co";(typeof window<"u"&&typeof document<"u"&&window.parent!=window&&window.supports_zerogpu_headers?ft("zerogpu-headers",I):Promise.resolve(null)).then(x=>ee(`${p.root}${J}/${qe}?${F}`,{...te,session_hash:c},x)).then(async([x,C])=>{if(C===503)m({type:"status",stage:"error",message:ie,queue:!0,endpoint:f,fn_index:u,time:new Date});else if(C!==200)m({type:"status",stage:"error",message:k,queue:!0,endpoint:f,fn_index:u,time:new Date});else{b=x.event_id,Le=b;let Ge=async function(he){try{let{type:M,status:T,data:P,original_msg:Ut}=le(he,fe[u]);if(M=="heartbeat")return;if(M==="update"&&T&&!R)m({type:"status",endpoint:f,fn_index:u,time:new Date,original_msg:Ut,...T});else if(M==="complete")R=T;else if(M=="unexpected_error")console.error("Unexpected error",T?.message),m({type:"status",stage:"error",message:T?.message||"An Unexpected Error Occurred!",queue:!0,endpoint:f,fn_index:u,time:new Date});else if(M==="log"){m({type:"log",title:P.title,log:P.log,level:P.level,endpoint:f,duration:P.duration,visible:P.visible,fn_index:u});return}else(M==="generating"||M==="streaming")&&(m({type:"status",time:new Date,...T,stage:T?.stage,queue:!0,endpoint:f,fn_index:u}),P&&W.connection!=="stream"&&["sse_v2","sse_v2.1","sse_v3"].includes($)&&Ct(D,b,P));P&&(m({type:"data",time:new Date,data:Z(P.data,W,p.components,"output",L.with_null_state),endpoint:f,fn_index:u}),P.render_config&&await $e(P.render_config),R&&(m({type:"status",time:new Date,...R,stage:T?.stage,queue:!0,endpoint:f,fn_index:u}),G())),(T?.stage==="complete"||T?.stage==="error")&&(Y[b]&&delete Y[b],b in D&&delete D[b])}catch(M){console.error("Unexpected client exception",M),m({type:"status",stage:"error",message:"An Unexpected Error Occurred!",queue:!0,endpoint:f,fn_index:u,time:new Date}),["sse_v2","sse_v2.1","sse_v3"].includes($)&&(X(g,Te.abort_controller),g.open=!1,G())}};b in y&&(y[b].forEach(he=>Ge(he)),delete y[b]),Y[b]=Ge,w.add(b),g.open||await this.open_stream()}})}});let me=!1,_e=[],ne=[],Me={[Symbol.asyncIterator]:()=>Me,next:ge,throw:async _=>(Lt(_),ge()),return:async()=>(G(),ge()),cancel:Tt,event_id:It};return Me}catch(o){throw console.error("Submit function encountered an error:",o),o}}function rn(e){return{then:(t,n)=>n(e)}}function on(e,t,n,s){let r,o,i;if(typeof t=="number")r=t,o=e.unnamed_endpoints[r],i=s.dependencies.find(a=>a.id==t);else{let a=t.replace(/^\//,"");r=n[a],o=e.named_endpoints[t.trim()],i=s.dependencies.find(p=>p.id==n[a])}if(typeof r!="number")throw new Error("There is no endpoint matching that name of fn_index matching that number.");return{fn_index:r,endpoint_info:o,dependency:i}}var z=class{app_reference;options;config;api_prefix="";api_info;api_map={};session_hash=Math.random().toString(36).substring(2);jwt=!1;last_status={};cookies=null;stream_status={open:!1};closed=!1;pending_stream_messages={};pending_diff_streams={};event_callbacks={};unclosed_events=new Set;heartbeat_event=null;abort_controller=null;stream_instance=null;current_payload;ws_map={};get_url_config(t=null){if(!this.config)throw new Error(N);t===null&&(t=window.location.href);let n=i=>i.replace(/^\/+|\/+$/g,""),s=n(new URL(this.config.root).pathname),r=n(new URL(t).pathname),o;return r.startsWith(s)?o=n(r.substring(s.length)):o="",this.get_page_config(o)}get_page_config(t){if(!this.config)throw new Error(N);let n=this.config;return t in n.page||(t=""),{...n,current_page:t,layout:n.page[t].layout,components:n.components.filter(s=>n.page[t].components.includes(s.id)),dependencies:this.config.dependencies.filter(s=>n.page[t].dependencies.includes(s.id))}}fetch(t,n){let s=new Headers(n?.headers||{});if(this&&this.cookies&&s.append("Cookie",this.cookies),this&&this.options.headers)for(let r in this.options.headers)s.append(r,this.options.headers[r]);return fetch(t,{...n,headers:s})}stream(t){let n=new Headers;if(this&&this.cookies&&n.append("Cookie",this.cookies),this&&this.options.headers)for(let s in this.options.headers)n.append(s,this.options.headers[s]);return this.abort_controller=new AbortController,this.stream_instance=kt(t.toString(),{credentials:"include",headers:n,signal:this.abort_controller.signal}),this.stream_instance}view_api;upload_files;upload;handle_blob;post_data;submit;predict;open_stream;resolve_config;resolve_cookies;constructor(t,n={events:["data"]}){this.app_reference=t,n.events||(n.events=["data"]),this.options=n,this.current_payload={},this.view_api=ut.bind(this),this.upload_files=xe.bind(this),this.handle_blob=_t.bind(this),this.post_data=ht.bind(this),this.submit=Oe.bind(this),this.predict=Ae.bind(this),this.open_stream=xt.bind(this),this.resolve_config=at.bind(this),this.resolve_cookies=pt.bind(this),this.upload=Ce.bind(this),this.fetch=this.fetch.bind(this),this.handle_space_success=this.handle_space_success.bind(this),this.stream=this.stream.bind(this)}async init(){if((typeof window>"u"||!("WebSocket"in window))&&!global.WebSocket){let t=await Promise.resolve().then(()=>Jt(Pt(),1));global.WebSocket=t.WebSocket}this.options.auth&&await this.resolve_cookies(),await this._resolve_config().then(({config:t})=>this._resolve_hearbeat(t)),this.api_info=await this.view_api(),this.api_map=it(this.config?.dependencies||[])}async _resolve_hearbeat(t){if(t&&(this.config=t,this.api_prefix=t.api_prefix||"",this.config&&this.config.connect_heartbeat&&this.config.space_id&&this.options.hf_token&&(this.jwt=await Se(this.config.space_id,this.options.hf_token,this.cookies))),t.space_id&&this.options.hf_token&&(this.jwt=await Se(t.space_id,this.options.hf_token)),this.config&&this.config.connect_heartbeat){let n=new URL(`${this.config.root}${this.api_prefix}/${Ve}/${this.session_hash}`);this.jwt&&n.searchParams.set("__sign",this.jwt),this.heartbeat_event||(this.heartbeat_event=this.stream(n))}}static async connect(t,n={events:["data"]}){let s=new this(t,n);return await s.init(),s}close(){this.closed=!0,X(this.stream_status,this.abort_controller)}set_current_payload(t){this.current_payload=t}static async duplicate(t,n={events:["data"]}){return vt(t,n)}async _resolve_config(){let{http_protocol:t,host:n,space_id:s}=await j(this.app_reference,this.options.hf_token),{status_callback:r}=this.options;s&&r&&await Pe(s,r);let o;try{if(o=await this.resolve_config(`${t}//${n}`),!o)throw new Error(N);return this.config_success(o)}catch(i){if(s&&r)Q(s,se.test(s)?"space_name":"subdomain",this.handle_space_success);else throw r&&r({status:"error",message:"Could not load this space.",load_status:"error",detail:"NOT_FOUND"}),Error(i)}}async config_success(t){if(this.config=t,this.api_prefix=t.api_prefix||"",typeof window<"u"&&typeof document<"u"&&window.location.protocol==="https:"&&(this.config.root=this.config.root.replace("http://","https://")),this.config.auth_required)return this.prepare_return_obj();try{this.api_info=await this.view_api()}catch(n){console.error(et+n.message)}return this.prepare_return_obj()}async handle_space_success(t){if(!this)throw new Error(N);let{status_callback:n}=this.options;if(n&&n(t),t.status==="running")try{if(this.config=await this._resolve_config(),this.api_prefix=this?.config?.api_prefix||"",!this.config)throw new Error(N);return await this.config_success(this.config)}catch(s){throw n&&n({status:"error",message:"Could not load this space.",load_status:"error",detail:"NOT_FOUND"}),s}}async component_server(t,n,s){if(!this.config)throw new Error(N);let r={},{hf_token:o}=this.options,{session_hash:i}=this;o&&(r.Authorization=`Bearer ${this.options.hf_token}`);let a,p=this.config.components.find(d=>d.id===t);p?.props?.root_url?a=p.props.root_url:a=this.config.root;let c;if("binary"in s){c=new FormData;for(let d in s.data)d!=="binary"&&c.append(d,s.data[d]);c.set("component_id",t.toString()),c.set("fn_name",n),c.set("session_hash",i)}else c=JSON.stringify({data:s,component_id:t,fn_name:n,session_hash:i}),r["Content-Type"]="application/json";o&&(r.Authorization=`Bearer ${o}`);try{let d=await this.fetch(`${a}${this.api_prefix}/${Ke}/`,{method:"POST",body:c,headers:r,credentials:"include"});if(!d.ok)throw new Error("Could not connect to component server: "+d.statusText);return await d.json()}catch(d){console.warn(d)}}set_cookies(t){this.cookies=pe(t).join("; ")}prepare_return_obj(){return{config:this.config,predict:this.predict,submit:this.submit,view_api:this.view_api,component_server:this.component_server}}async connect_ws(t){return new Promise((n,s)=>{let r;try{r=new WebSocket(t)}catch{this.ws_map[t]="failed";return}r.onopen=()=>{n()},r.onerror=o=>{console.error("WebSocket error:",o),this.close_ws(t),this.ws_map[t]="failed",n()},r.onclose=()=>{delete this.ws_map[t],this.ws_map[t]="failed"},r.onmessage=o=>{},this.ws_map[t]=r})}async send_ws_message(t,n){t in this.ws_map||await this.connect_ws(t);let s=this.ws_map[t];s instanceof WebSocket?s.send(JSON.stringify(n)):this.post_data(t,n)}async close_ws(t){if(t in this.ws_map){let n=this.ws_map[t];n instanceof WebSocket&&(n.close(),delete this.ws_map[t])}}};export{z as Client,mt as handle_file};

app/static/index.html

@@ -0,0 +1,190 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <meta name="color-scheme" content="light" />
+    <meta
+      name="description"
+      content="Interactive demo for What Moves? Localized Motion Representations for Compositional Scene Control."
+    />
+    <title>What Moves? — Interactive Demo</title>
+    <link rel="icon" type="image/svg+xml" href="/static/favicon.svg?v=20260904h" />
+    <link rel="stylesheet" href="/static/styles.css?v=20260904h" />
+  </head>
+  <body>
+    <header class="page-header">
+      <div class="app-brand">
+        <div>
+          <h1><em>What Moves?</em></h1>
+          <p>Transfer localized motion from videos to a still image.</p>
+        </div>
+      </div>
+      <div class="header-actions">
+        <button id="startOver" class="start-over-button" title="Remove all current media and start over">
+          <span class="broom-icon" aria-hidden="true"></span>
+          <span>Start Over</span>
+        </button>
+      </div>
+    </header>
+
+    <main>
+      <div class="workspace">
+        <section class="panel source-panel">
+          <div class="panel-heading">
+            <div>
+              <span class="eyebrow">Motion</span>
+              <h2>Source videos</h2>
+            </div>
+            <button id="removeSource" class="danger-quiet hidden">Remove</button>
+          </div>
+
+          <div class="source-workspace">
+            <aside id="sourceRail" class="source-rail" aria-label="Uploaded source videos">
+              <button id="sourceRailUpload" class="source-upload-tile" title="Upload source video">+</button>
+            </aside>
+            <div class="source-main">
+              <div id="sourceCanvasHost" class="canvas-host">
+                <video id="sourceVideo" muted playsinline></video>
+                <canvas id="sourceCanvas"></canvas>
+                <button id="sourceEmpty" class="empty-state">
+                  <span class="empty-icon">↥</span>
+                  <strong>Upload a source video</strong>
+                  <small>It will be sampled at 8 fps and appear here.</small>
+                </button>
+                <div id="sourceCanvasStatus" class="canvas-status hidden"></div>
+                <div id="sourceModeBadge" class="mode-badge hidden">Masking</div>
+                <div id="sourceLoader" class="media-loader hidden" role="status" aria-label="Loading source media"><span class="loading-spinner"></span></div>
+              </div>
+            <div id="sourceTools" class="canvas-tools hidden">
+              <button id="sourceMaskMode" class="tool-button">Start masking</button>
+              <span class="shortcut-hint">Click foreground · right-click background · drag a box</span>
+              <div class="tool-spacer"></div>
+              <button id="sourcePlay" class="tool-button playback-button" aria-label="Play" title="Play">
+                <svg class="playback-icon" viewBox="0 0 24 24" aria-hidden="true">
+                  <path d="M8 5v14l11-7z"></path>
+                </svg>
+              </button>
+              <button id="sourceCancelDraft" class="text-button hidden">Cancel</button>
+              <button id="sourceAddMask" class="primary-small hidden">Add mask</button>
+            </div>
+            <div id="sourceSettings" class="source-settings hidden">
+              <span id="sourceTime" class="source-time hidden" aria-hidden="true"></span>
+              <div id="sourceTimeline" class="video-timeline hidden" aria-label="Source video timeline">
+                  <div class="timeline-filmstrip"></div>
+                  <div class="timeline-selection"></div>
+                  <div class="timeline-before"></div>
+                  <div class="timeline-after"></div>
+                  <div class="timeline-masks"></div>
+                  <div class="timeline-playhead"></div>
+                  <div class="trim-handle left" data-handle="left"></div>
+                  <div class="trim-handle right" data-handle="right"></div>
+                </div>
+              </div>
+            </div>
+          </div>
+        </section>
+
+        <div class="target-stack">
+          <section class="panel target-panel">
+            <div class="panel-heading">
+              <div>
+                <span class="eyebrow">Appearance</span>
+                <h2>Target image</h2>
+              </div>
+              <div class="panel-actions">
+                <button id="removeTarget" class="danger-quiet hidden">Remove</button>
+              </div>
+            </div>
+            <div class="target-workspace">
+              <div class="target-main">
+                <div id="targetCanvasHost" class="canvas-host target-host">
+                  <canvas id="targetCanvas"></canvas>
+                  <button id="targetEmpty" class="empty-state">
+                    <span class="empty-icon">↥</span>
+                    <strong>Upload a target image</strong>
+                    <small>Create grey regions, then drag colored source masks onto them.</small>
+                  </button>
+                  <div id="targetCanvasStatus" class="canvas-status hidden"></div>
+                  <div id="targetModeBadge" class="mode-badge hidden">Masking</div>
+                  <div id="targetLoader" class="media-loader hidden" role="status" aria-label="Loading target image"><span class="loading-spinner"></span></div>
+                </div>
+                <div id="targetTools" class="canvas-tools hidden">
+                  <button id="targetMaskMode" class="tool-button">Start masking</button>
+                  <span class="shortcut-hint">Click foreground · right-click background · drag a box</span>
+                  <div class="tool-spacer"></div>
+                  <button id="targetCancelDraft" class="text-button hidden">Cancel</button>
+                  <button id="targetAddMask" class="primary-small hidden">Add mask</button>
+                </div>
+              </div>
+              <aside id="targetRail" class="target-rail" aria-label="Uploaded target images"></aside>
+            </div>
+          </section>
+
+          <section class="generate-card">
+            <label for="prompt" class="prompt-label">Describe the target video</label>
+            <textarea id="prompt" rows="2" placeholder="Describe how the bear and dog should move">A bear and a dog together in the forest.</textarea>
+            <details id="advancedSettings">
+              <summary>Advanced settings</summary>
+              <div class="advanced-grid">
+                <label>Guidance
+                  <select id="guidanceMode">
+                    <option value="text_cfg">Text CFG</option>
+                    <option value="joint_cfg">Joint CFG</option>
+                    <option value="motion_cfg">Motion CFG</option>
+                    <option value="additive_cfg">Additive CFG</option>
+                    <option value="factorized_cfg">Factorized CFG</option>
+                    <option value="base_cfg">Base Wan (no motion)</option>
+                  </select>
+                </label>
+                <label>Steps <input id="steps" type="number" min="1" max="100" value="40" /></label>
+                <label>Seed <input id="seed" type="number" value="42" /></label>
+                <label>Text guidance <input id="textGuidance" type="number" step="0.1" value="3.5" /></label>
+                <label>Motion guidance <input id="motionGuidance" type="number" step="0.1" value="1.0" /></label>
+                <label>LoRA scale <input id="loraScale" type="number" step="0.1" value="1.0" /></label>
+                <label class="negative-prompt">Negative prompt
+                  <input id="negativePrompt" type="text" placeholder="Use the Wan default" />
+                </label>
+              </div>
+            </details>
+            <button id="generateButton" class="generate-button">
+              <span id="generateButtonProgress" class="generate-button-progress"></span>
+              <span class="generate-button-content">
+                <span id="generateButtonLabel">Generate video</span>
+                <span id="generateButtonCount"></span>
+              </span>
+            </button>
+            <div id="generatedRail" class="generated-rail hidden" aria-label="Generated videos"></div>
+            <p id="generateHint" class="generate-hint">Upload media and create a motion mapping.</p>
+          </section>
+        </div>
+      </div>
+
+      <section id="resultPanel" class="result-panel hidden">
+        <div class="result-heading">
+          <div>
+            <span class="eyebrow">Result</span>
+            <h2>Generated video</h2>
+          </div>
+          <div class="result-actions">
+            <span id="resultMeta"></span>
+            <button id="resultClose" class="icon-button result-close" aria-label="Close generated video" title="Close">×</button>
+          </div>
+        </div>
+        <div class="result-media">
+          <video id="resultVideo" controls loop playsinline></video>
+          <div id="resultLoader" class="media-loader hidden" role="status" aria-label="Loading generated video"><span class="loading-spinner"></span></div>
+        </div>
+      </section>
+    </main>
+
+    <input id="sourceFile" type="file" accept="video/*" hidden />
+    <input id="targetFile" type="file" accept="image/*" hidden />
+    <div id="contextMenu" class="context-menu hidden"></div>
+    <div id="dragGhost" class="drag-ghost hidden"></div>
+    <div id="mappingHint" class="mapping-hint hidden" aria-hidden="true"></div>
+    <div id="toast" class="toast hidden"></div>
+
+    <script type="module" src="/static/app.js?v=20260904l"></script>
+  </body>
+</html>

app/static/mask-canvas.js

@@ -0,0 +1,562 @@
+const GREY = "#a8afb9";
+
+function makeCanvas(width, height) {
+  const canvas = document.createElement("canvas");
+  canvas.width = width;
+  canvas.height = height;
+  return canvas;
+}
+
+async function bitmapFromUrl(url) {
+  const response = await fetch(url);
+  if (!response.ok) throw new Error(`Could not load image (${response.status})`);
+  return createImageBitmap(await response.blob());
+}
+
+function boundaryDistance(pixels, width, height) {
+  const infinity = 65535;
+  const distance = new Uint16Array(width * height);
+  distance.fill(infinity);
+  for (let y = 0; y < height; y += 1) {
+    for (let x = 0; x < width; x += 1) {
+      const index = y * width + x;
+      const value = pixels[index];
+      const boundary = (value && (x === 0 || y === 0 || x === width - 1 || y === height - 1)) ||
+        (x && pixels[index - 1] !== value) ||
+        (x + 1 < width && pixels[index + 1] !== value) ||
+        (y && pixels[index - width] !== value) ||
+        (y + 1 < height && pixels[index + width] !== value);
+      if (boundary) distance[index] = 0;
+    }
+  }
+  for (let y = 0; y < height; y += 1) {
+    for (let x = 0; x < width; x += 1) {
+      const index = y * width + x;
+      let best = distance[index];
+      if (x) best = Math.min(best, distance[index - 1] + 3);
+      if (y) best = Math.min(best, distance[index - width] + 3);
+      if (x && y) best = Math.min(best, distance[index - width - 1] + 4);
+      if (x + 1 < width && y) best = Math.min(best, distance[index - width + 1] + 4);
+      distance[index] = best;
+    }
+  }
+  for (let y = height - 1; y >= 0; y -= 1) {
+    for (let x = width - 1; x >= 0; x -= 1) {
+      const index = y * width + x;
+      let best = distance[index];
+      if (x + 1 < width) best = Math.min(best, distance[index + 1] + 3);
+      if (y + 1 < height) best = Math.min(best, distance[index + width] + 3);
+      if (x + 1 < width && y + 1 < height) best = Math.min(best, distance[index + width + 1] + 4);
+      if (x && y + 1 < height) best = Math.min(best, distance[index + width - 1] + 4);
+      distance[index] = best;
+    }
+  }
+  return distance;
+}
+
+async function loadMask(url, width, height, record) {
+  const bitmap = await bitmapFromUrl(url);
+  const raster = makeCanvas(width, height);
+  const rasterContext = raster.getContext("2d", { willReadFrequently: true });
+  rasterContext.drawImage(bitmap, 0, 0, width, height);
+  bitmap.close();
+  const rgba = rasterContext.getImageData(0, 0, width, height).data;
+  const pixels = new Uint8Array(width * height);
+  for (let index = 0; index < pixels.length; index += 1) {
+    pixels[index] = rgba[index * 4] > 127 ? 1 : 0;
+  }
+
+  const alpha = makeCanvas(width, height);
+  const alphaContext = alpha.getContext("2d");
+  const alphaImage = alphaContext.createImageData(width, height);
+  for (let y = 0; y < height; y += 1) {
+    for (let x = 0; x < width; x += 1) {
+      const index = y * width + x;
+      if (!pixels[index]) continue;
+      const rgbaIndex = index * 4;
+      alphaImage.data[rgbaIndex] = 255;
+      alphaImage.data[rgbaIndex + 1] = 255;
+      alphaImage.data[rgbaIndex + 2] = 255;
+      alphaImage.data[rgbaIndex + 3] = 255;
+    }
+  }
+  alphaContext.putImageData(alphaImage, 0, 0);
+  return {
+    ...record,
+    pixels,
+    alpha,
+    edgeDistance: boundaryDistance(pixels, width, height),
+    edgeCanvases: new Map(),
+  };
+}
+
+export class MaskCanvas {
+  constructor({ host, canvas, role, callbacks = {} }) {
+    this.host = host;
+    this.canvas = canvas;
+    this.context = canvas.getContext("2d");
+    this.role = role;
+    this.callbacks = callbacks;
+    this.asset = null;
+    this.assetSignature = "";
+    this.base = null;
+    this.masks = [];
+    this.draft = null;
+    this.hoverDraft = null;
+    this.hoverPoint = null;
+    this.positive = [];
+    this.negative = [];
+    this.box = null;
+    this.promptRevision = 0;
+    this.hovered = null;
+    this.dropHovered = null;
+    this.pinned = false;
+    this.pointerInside = false;
+    this.pointerDown = null;
+    this.loadToken = 0;
+    this.scratch = makeCanvas(1, 1);
+
+    canvas.addEventListener("pointerenter", () => {
+      this.pointerInside = true;
+      this._modeChanged();
+    });
+    canvas.addEventListener("pointerleave", () => {
+      this.pointerInside = false;
+      if (!this.pointerDown) this.hovered = null;
+      this.clearHoverDraft();
+      this.hoverPoint = null;
+      this.callbacks.onHoverPrompt?.(null);
+      this._modeChanged();
+      this.render();
+    });
+    canvas.addEventListener("pointerdown", (event) => this._pointerDown(event));
+    canvas.addEventListener("pointermove", (event) => this._pointerMove(event));
+    canvas.addEventListener("pointerup", (event) => this._pointerUp(event));
+    canvas.addEventListener("pointercancel", () => this._cancelPointer());
+    canvas.addEventListener("contextmenu", (event) => this._contextMenu(event));
+    this.resizeObserver = new ResizeObserver(() => this._resizeDisplay());
+    this.resizeObserver.observe(host);
+  }
+
+  async setAsset(asset, masks = []) {
+    if (!asset) {
+      this.loadToken += 1;
+      this.asset = null;
+      this.assetSignature = "";
+      if (this.base) this.base.close();
+      this.base = null;
+      this.masks = [];
+      this.clearDraft();
+      this.host.classList.remove("has-image");
+      this.callbacks.onLoading?.(false);
+      return;
+    }
+    const signature = JSON.stringify([
+      asset.image_url,
+      masks.map((mask) => [mask.id, mask.url, mask.displayColor]),
+    ]);
+    if (signature === this.assetSignature) return;
+    const mediaChanged = !this.asset || this.asset.image_url !== asset.image_url;
+    const token = ++this.loadToken;
+    this.callbacks.onLoading?.(true);
+    try {
+      const base = mediaChanged ? await bitmapFromUrl(asset.image_url) : this.base;
+      if (token !== this.loadToken) {
+        if (mediaChanged) base.close();
+        return;
+      }
+      if (mediaChanged) {
+        if (this.base) this.base.close();
+        this.base = base;
+        this.canvas.width = base.width;
+        this.canvas.height = base.height;
+        this.scratch = makeCanvas(base.width, base.height);
+        this.clearDraft();
+      }
+      const loadedMasks = await Promise.all(
+        masks.map((mask) => loadMask(mask.url, this.canvas.width, this.canvas.height, mask)),
+      );
+      if (token !== this.loadToken) return;
+      this.asset = asset;
+      this.assetSignature = signature;
+      this.masks = loadedMasks;
+      this.host.classList.add("has-image");
+      this._resizeDisplay();
+      this.render();
+    } catch (error) {
+      this.callbacks.onError?.(error);
+    } finally {
+      if (token === this.loadToken) this.callbacks.onLoading?.(false);
+    }
+  }
+
+  togglePinned() {
+    return this.setPinned(!this.pinned);
+  }
+
+  setPinned(value) {
+    this.pinned = Boolean(value);
+    if (!this.pinned) {
+      this.clearHoverDraft();
+      this.hoverPoint = null;
+      this.callbacks.onHoverPrompt?.(null);
+    }
+    this._modeChanged();
+    this.render();
+    return this.pinned;
+  }
+
+  isMasking() {
+    return Boolean(this.asset && this.pinned);
+  }
+
+  promptPayload() {
+    return {
+      asset_revision: this.asset?.revision,
+      frame_index: this.role === "source" ? this.asset?.current_frame : null,
+      prompt_revision: this.promptRevision,
+      positive: this.positive,
+      negative: this.negative,
+      box: this.box,
+    };
+  }
+
+  async setDraft(base64, promptRevision) {
+    if (promptRevision !== this.promptRevision || !this.asset) return false;
+    const record = { id: "draft", url: `data:image/png;base64,${base64}` };
+    const draft = await loadMask(record.url, this.canvas.width, this.canvas.height, record);
+    if (promptRevision !== this.promptRevision) return false;
+    this.draft = draft;
+    this.callbacks.onDraftState?.(true);
+    this.render();
+    return true;
+  }
+
+  async setHoverDraft(base64, promptRevision) {
+    if (promptRevision !== this.promptRevision || !this.asset || !this.isMasking()) return false;
+    const record = { id: "hover-draft", url: `data:image/png;base64,${base64}` };
+    const draft = await loadMask(record.url, this.canvas.width, this.canvas.height, record);
+    if (promptRevision !== this.promptRevision || !this.isMasking()) return false;
+    this.hoverDraft = draft;
+    this.render();
+    return true;
+  }
+
+  clearHoverDraft() {
+    if (!this.hoverDraft) return;
+    this.hoverDraft = null;
+    this.hoverPoint = null;
+    this.render();
+  }
+
+  clearDraft() {
+    this.positive = [];
+    this.negative = [];
+    this.box = null;
+    this.draft = null;
+    this.hoverDraft = null;
+    this.promptRevision += 1;
+    this.callbacks.onDraftState?.(false);
+    this.render();
+  }
+
+  masksAtClient(clientX, clientY) {
+    const point = this._point({ clientX, clientY });
+    if (!point) return [];
+    return this.masks
+      .filter((mask) => mask.pixels[Math.floor(point.y) * this.canvas.width + Math.floor(point.x)])
+      .reverse();
+  }
+
+  setDropHover(maskId) {
+    if (this.dropHovered === maskId) return;
+    this.dropHovered = maskId;
+    this.render();
+  }
+
+  getMask(maskId) {
+    return this.masks.find((value) => value.id === maskId) || null;
+  }
+
+  maskClientCenter(maskId) {
+    const mask = this.masks.find((value) => value.id === maskId);
+    if (!mask || !this.canvas.width || !this.canvas.height) return null;
+    let minX = this.canvas.width;
+    let minY = this.canvas.height;
+    let maxX = -1;
+    let maxY = -1;
+    for (let y = 0; y < this.canvas.height; y += 1) {
+      for (let x = 0; x < this.canvas.width; x += 1) {
+        if (!mask.pixels[y * this.canvas.width + x]) continue;
+        minX = Math.min(minX, x);
+        minY = Math.min(minY, y);
+        maxX = Math.max(maxX, x);
+        maxY = Math.max(maxY, y);
+      }
+    }
+    if (maxX < 0 || maxY < 0) return null;
+    const rect = this.canvas.getBoundingClientRect();
+    return {
+      x: rect.left + ((minX + maxX) / 2 / this.canvas.width) * rect.width,
+      y: rect.top + ((minY + maxY) / 2 / this.canvas.height) * rect.height,
+    };
+  }
+
+  render() {
+    if (!this.base) return;
+    const context = this.context;
+    context.clearRect(0, 0, this.canvas.width, this.canvas.height);
+    context.drawImage(this.base, 0, 0, this.canvas.width, this.canvas.height);
+    for (const mask of this.masks) {
+      const color = mask.displayColor || mask.color || GREY;
+      this._drawTint(mask.alpha, color, 0.28);
+      const hovered = mask.id === this.hovered;
+      const dropping = mask.id === this.dropHovered;
+      this._drawEdge(mask, dropping ? "#ffffff" : color, hovered || dropping ? 4 : 2);
+    }
+    const preview = this.hoverDraft || this.draft;
+    if (preview && this.isMasking()) {
+      this._drawTint(preview.alpha, "#ffffff", this.hoverDraft ? 0.16 : 0.2);
+      this._drawEdge(preview, "#ffffff", 2);
+    }
+    if (this.isMasking()) this._drawPrompts();
+  }
+
+  _drawTint(alpha, color, opacity) {
+    const scratch = this.scratch;
+    const scratchContext = scratch.getContext("2d");
+    scratchContext.clearRect(0, 0, scratch.width, scratch.height);
+    scratchContext.globalCompositeOperation = "source-over";
+    scratchContext.fillStyle = color;
+    scratchContext.fillRect(0, 0, scratch.width, scratch.height);
+    scratchContext.globalCompositeOperation = "destination-in";
+    scratchContext.drawImage(alpha, 0, 0);
+    scratchContext.globalCompositeOperation = "source-over";
+    this.context.save();
+    this.context.globalAlpha = opacity;
+    this.context.drawImage(scratch, 0, 0);
+    this.context.restore();
+  }
+
+  _drawEdge(mask, color, cssRadius) {
+    const rect = this.canvas.getBoundingClientRect();
+    const radius = Math.max(1, Math.round(cssRadius * this.canvas.width / Math.max(rect.width, 1)));
+    let edge = mask.edgeCanvases.get(radius);
+    if (!edge) {
+      edge = makeCanvas(this.canvas.width, this.canvas.height);
+      const edgeContext = edge.getContext("2d");
+      const image = edgeContext.createImageData(edge.width, edge.height);
+      const threshold = radius * 3;
+      for (let index = 0; index < mask.edgeDistance.length; index += 1) {
+        if (mask.edgeDistance[index] > threshold) continue;
+        const rgba = index * 4;
+        image.data[rgba] = 255;
+        image.data[rgba + 1] = 255;
+        image.data[rgba + 2] = 255;
+        image.data[rgba + 3] = 255;
+      }
+      edgeContext.putImageData(image, 0, 0);
+      mask.edgeCanvases.set(radius, edge);
+    }
+    const scratchContext = this.scratch.getContext("2d");
+    scratchContext.clearRect(0, 0, this.scratch.width, this.scratch.height);
+    scratchContext.globalCompositeOperation = "source-over";
+    scratchContext.fillStyle = color;
+    scratchContext.fillRect(0, 0, this.scratch.width, this.scratch.height);
+    scratchContext.globalCompositeOperation = "destination-in";
+    scratchContext.drawImage(edge, 0, 0);
+    scratchContext.globalCompositeOperation = "source-over";
+    this.context.drawImage(this.scratch, 0, 0);
+  }
+
+  _drawPrompts() {
+    const context = this.context;
+    const rect = this.canvas.getBoundingClientRect();
+    const scale = this.canvas.width / Math.max(rect.width, 1);
+    const radius = Math.max(5, 5 * scale);
+    const drawPoint = (point, fill, sign) => {
+      const x = point[0] * this.canvas.width;
+      const y = point[1] * this.canvas.height;
+      context.beginPath();
+      context.arc(x, y, radius, 0, Math.PI * 2);
+      context.fillStyle = fill;
+      context.fill();
+      context.lineWidth = Math.max(2, 1.5 * scale);
+      context.strokeStyle = "white";
+      context.stroke();
+      context.beginPath();
+      context.moveTo(x - radius * 0.45, y);
+      context.lineTo(x + radius * 0.45, y);
+      if (sign === "+") {
+        context.moveTo(x, y - radius * 0.45);
+        context.lineTo(x, y + radius * 0.45);
+      }
+      context.strokeStyle = "white";
+      context.lineWidth = Math.max(1.5, scale);
+      context.stroke();
+    };
+    this.positive.forEach((point) => drawPoint(point, "#29b889", "+"));
+    this.negative.forEach((point) => drawPoint(point, "#df5b61", "−"));
+    if (this.hoverPoint) drawPoint(this.hoverPoint, "#28c997", "+");
+    const box = this.pointerDown?.mode === "mask" && this.pointerDown.dragging
+      ? this._normalizedBox(this.pointerDown.start, this.pointerDown.current)
+      : this.box;
+    if (box) {
+      context.strokeStyle = "rgba(255,255,255,.95)";
+      context.lineWidth = Math.max(2, 2 * scale);
+      context.setLineDash([7 * scale, 5 * scale]);
+      context.strokeRect(
+        box[0] * this.canvas.width,
+        box[1] * this.canvas.height,
+        (box[2] - box[0]) * this.canvas.width,
+        (box[3] - box[1]) * this.canvas.height,
+      );
+      context.setLineDash([]);
+    }
+  }
+
+  _point(event) {
+    if (!this.base) return null;
+    const rect = this.canvas.getBoundingClientRect();
+    if (!rect.width || !rect.height) return null;
+    const x = (event.clientX - rect.left) / rect.width * this.canvas.width;
+    const y = (event.clientY - rect.top) / rect.height * this.canvas.height;
+    if (x < 0 || y < 0 || x >= this.canvas.width || y >= this.canvas.height) return null;
+    return { x, y, clientX: event.clientX, clientY: event.clientY };
+  }
+
+  _normalized(point) {
+    return [
+      Math.min(1, Math.max(0, point.x / this.canvas.width)),
+      Math.min(1, Math.max(0, point.y / this.canvas.height)),
+    ];
+  }
+
+  _normalizedBox(first, second) {
+    const a = this._normalized(first);
+    const b = this._normalized(second);
+    return [Math.min(a[0], b[0]), Math.min(a[1], b[1]), Math.max(a[0], b[0]), Math.max(a[1], b[1])];
+  }
+
+  _pointerDown(event) {
+    if (!this.asset || event.button !== 0) return;
+    const point = this._point(event);
+    if (!point) return;
+    const mode = this.isMasking() ? "mask" : "normal";
+    if (mode === "mask") {
+      this.clearHoverDraft();
+      this.hoverPoint = null;
+      this.callbacks.onHoverPrompt?.(null);
+    }
+    const hit = this.masksAtClient(event.clientX, event.clientY)[0] || null;
+    this.pointerDown = { start: point, current: point, mode, hit, dragging: false };
+    this.canvas.setPointerCapture(event.pointerId);
+    event.preventDefault();
+  }
+
+  _pointerMove(event) {
+    const point = this._point(event);
+    if (this.pointerDown) {
+      const clientPoint = point || { clientX: event.clientX, clientY: event.clientY };
+      const distance = Math.hypot(clientPoint.clientX - this.pointerDown.start.clientX, clientPoint.clientY - this.pointerDown.start.clientY);
+      if (distance > 5) this.pointerDown.dragging = true;
+      if (this.pointerDown.mode === "normal" && this.pointerDown.dragging && this.role === "source" && this.pointerDown.hit) {
+        this.callbacks.onMappingDrag?.("move", this.pointerDown.hit, event.clientX, event.clientY);
+      } else if (point) {
+        this.pointerDown.current = point;
+      }
+      this.render();
+      return;
+    }
+    if (this.isMasking() && point) {
+      this.hovered = null;
+      this.hoverPoint = this._normalized(point);
+      const payload = this.promptPayload();
+      payload.positive = [...payload.positive, this.hoverPoint];
+      payload.transient = true;
+      this.callbacks.onHoverPrompt?.(payload, point.clientX, point.clientY);
+      this.render();
+      return;
+    }
+    const next = this.masksAtClient(event.clientX, event.clientY)[0]?.id || null;
+    this.hoverPoint = null;
+    if (next !== this.hovered) {
+      this.hovered = next;
+      this._modeChanged();
+      this.render();
+    }
+  }
+
+  _pointerUp(event) {
+    if (!this.pointerDown || event.button !== 0) return;
+    const interaction = this.pointerDown;
+    const point = this._point(event) || interaction.current;
+    this.pointerDown = null;
+    if (interaction.mode === "mask") {
+      if (interaction.dragging) {
+        const box = this._normalizedBox(interaction.start, point);
+        if ((box[2] - box[0]) * this.canvas.width > 3 && (box[3] - box[1]) * this.canvas.height > 3) {
+          this.box = box;
+          this._promptsChanged();
+        }
+      } else {
+        this.positive.push(this._normalized(point));
+        this._promptsChanged();
+      }
+    } else if (interaction.dragging && this.role === "source" && interaction.hit) {
+      this.callbacks.onMappingDrag?.("drop", interaction.hit, event.clientX, event.clientY);
+    } else {
+      const hits = this.masksAtClient(event.clientX, event.clientY);
+      if (hits.length) this.callbacks.onMaskClick?.(hits, event.clientX, event.clientY);
+    }
+    this.render();
+  }
+
+  _contextMenu(event) {
+    if (!this.isMasking() || !this.asset) return;
+    event.preventDefault();
+    const point = this._point(event);
+    if (!point) return;
+    this.negative.push(this._normalized(point));
+    this._promptsChanged();
+  }
+
+  _cancelPointer() {
+    if (this.pointerDown?.mode === "normal" && this.pointerDown.hit) {
+      this.callbacks.onMappingDrag?.("cancel", this.pointerDown.hit, 0, 0);
+    }
+    this.pointerDown = null;
+    this.render();
+  }
+
+  _promptsChanged() {
+    this.promptRevision += 1;
+    this.draft = null;
+    this.hoverDraft = null;
+    this.hoverPoint = null;
+    this.callbacks.onHoverPrompt?.(null);
+    this.callbacks.onDraftState?.(false);
+    this.callbacks.onPromptsChanged?.(this.promptPayload());
+    this.render();
+  }
+
+  _modeChanged() {
+    const active = this.isMasking();
+    if (!active) {
+      this.hoverDraft = null;
+      this.hoverPoint = null;
+    }
+    this.canvas.style.cursor = active ? "crosshair" : (this.hovered ? "pointer" : "default");
+    this.callbacks.onModeChange?.(active, this.pinned);
+  }
+
+  _resizeDisplay() {
+    if (!this.base) return;
+    const width = this.host.clientWidth;
+    const height = this.host.clientHeight;
+    const scale = Math.min(width / this.canvas.width, height / this.canvas.height);
+    this.canvas.style.width = `${Math.max(1, this.canvas.width * scale)}px`;
+    this.canvas.style.height = `${Math.max(1, this.canvas.height * scale)}px`;
+    this.render();
+  }
+}

app/static/styles.css

@@ -0,0 +1,1130 @@
+:root {
+  color-scheme: dark;
+  --background: #0d0f13;
+  --panel: #15181e;
+  --panel-raised: #1b1f27;
+  --line: #292e38;
+  --line-strong: #3a4250;
+  --text: #f2f4f7;
+  --muted: #969eaa;
+  --accent: #8ba5ff;
+  --accent-strong: #6f8ff7;
+  --danger: #ff7f82;
+  --radius: 18px;
+  font-family: Inter, ui-sans-serif, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
+}
+
+* { box-sizing: border-box; }
+
+body {
+  margin: 0;
+  min-height: 100vh;
+  background:
+    radial-gradient(circle at 80% -20%, rgba(80, 105, 170, 0.13), transparent 38rem),
+    var(--background);
+  color: var(--text);
+}
+
+button, input, textarea, select { font: inherit; }
+button { color: inherit; }
+
+.page-header, main {
+  width: min(1600px, calc(100% - 48px));
+  margin-inline: auto;
+}
+
+.page-header {
+  display: flex;
+  align-items: flex-end;
+  justify-content: space-between;
+  padding: 34px 0 22px;
+}
+
+h1, h2, p { margin: 0; }
+h1 { font-size: 25px; letter-spacing: -0.04em; }
+h2 { font-size: 16px; font-weight: 640; }
+.page-header p { margin-top: 5px; color: var(--muted); font-size: 13px; }
+
+.status-pill {
+  padding: 7px 11px;
+  border: 1px solid var(--line);
+  border-radius: 999px;
+  color: var(--muted);
+  font-size: 12px;
+  background: rgba(21, 24, 30, 0.8);
+}
+
+.workspace {
+  display: grid;
+  grid-template-columns: minmax(0, 1.14fr) minmax(380px, 0.86fr);
+  gap: 18px;
+  align-items: start;
+}
+
+.panel, .generate-card, .result-panel {
+  border: 1px solid var(--line);
+  background: rgba(21, 24, 30, 0.94);
+  border-radius: var(--radius);
+  box-shadow: 0 16px 50px rgba(0, 0, 0, 0.16);
+}
+
+.panel { padding: 16px; }
+.panel-heading, .result-heading {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  margin-bottom: 13px;
+}
+.panel-actions { display: flex; align-items: center; gap: 6px; }
+.panel-actions .danger-quiet { margin-left: 0; }
+
+.eyebrow {
+  display: block;
+  color: var(--muted);
+  font-size: 10px;
+  font-weight: 700;
+  letter-spacing: 0.12em;
+  text-transform: uppercase;
+  margin-bottom: 3px;
+}
+
+.icon-button, .quiet-button, .tool-button, .text-button, .primary-small, .danger-quiet {
+  border: 1px solid var(--line);
+  background: var(--panel-raised);
+  border-radius: 9px;
+  cursor: pointer;
+  transition: border-color 120ms, background 120ms, transform 120ms;
+}
+
+.icon-button:hover, .quiet-button:hover, .tool-button:hover, .text-button:hover,
+.primary-small:hover { border-color: var(--line-strong); background: #222731; }
+.icon-button:active, .quiet-button:active, .tool-button:active { transform: scale(0.97); }
+.icon-button { width: 31px; height: 31px; font-size: 20px; line-height: 1; }
+.quiet-button, .tool-button, .text-button, .primary-small, .danger-quiet { padding: 7px 10px; font-size: 12px; }
+.primary-small { background: var(--accent-strong); border-color: var(--accent-strong); color: white; }
+.tool-button.active { background: rgba(111, 143, 247, 0.2); border-color: var(--accent); }
+.danger-quiet { margin-left: auto; color: var(--danger); background: transparent; }
+.danger-quiet:hover { background: rgba(255, 127, 130, 0.08); }
+
+.source-workspace { display: grid; grid-template-columns: 92px minmax(0, 1fr); gap: 12px; }
+.source-rail {
+  display: flex;
+  flex-direction: column;
+  gap: 8px;
+  max-height: 620px;
+  overflow-y: auto;
+  scrollbar-width: thin;
+}
+
+.source-tile, .source-upload-tile, .target-tile, .target-upload-tile {
+  position: relative;
+  flex: 0 0 68px;
+  width: 100%;
+  padding: 0;
+  overflow: hidden;
+  border: 1px solid var(--line);
+  border-radius: 11px;
+  background: #0f1116;
+  cursor: pointer;
+}
+.source-tile img, .target-tile img { width: 100%; height: 100%; object-fit: cover; display: block; }
+.source-tile.selected, .target-tile.selected { border-color: var(--accent); box-shadow: inset 0 0 0 1px var(--accent); }
+.source-tile .tile-label, .target-tile .tile-label {
+  position: absolute; inset: auto 4px 4px;
+  overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
+  padding: 3px 5px; border-radius: 5px;
+  font-size: 9px; text-align: left;
+  background: rgba(0, 0, 0, 0.7);
+}
+.source-upload-tile, .target-upload-tile { font-size: 24px; color: var(--muted); border-style: dashed; }
+.source-upload-tile:hover, .target-upload-tile:hover { color: var(--text); border-color: var(--line-strong); }
+
+.canvas-host {
+  position: relative;
+  display: grid;
+  place-items: center;
+  width: 100%;
+  min-height: 430px;
+  max-height: 650px;
+  overflow: hidden;
+  border-radius: 13px;
+  background: #090b0e;
+  border: 1px solid #20242c;
+}
+.target-host { min-height: 350px; }
+.canvas-host canvas {
+  display: none;
+  max-height: 650px;
+  touch-action: none;
+}
+.canvas-host video {
+  display: none;
+  width: 100%;
+  height: 100%;
+  max-height: 650px;
+  object-fit: contain;
+  background: black;
+}
+.canvas-host.playing video { display: block; }
+.canvas-host.has-image canvas { display: block; }
+.canvas-host.has-image.playing canvas { display: none; }
+
+.empty-state {
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  gap: 6px;
+  border: 0;
+  color: var(--muted);
+  background: transparent;
+  cursor: pointer;
+  padding: 40px;
+}
+.canvas-host.has-image .empty-state { display: none; }
+.empty-state strong { color: #d9dde4; font-size: 14px; font-weight: 600; }
+.empty-state small { max-width: 270px; line-height: 1.45; }
+.empty-icon { font-size: 26px; color: var(--accent); }
+
+.canvas-status, .mode-badge {
+  position: absolute;
+  top: 10px;
+  padding: 6px 9px;
+  border-radius: 7px;
+  font-size: 11px;
+  backdrop-filter: blur(10px);
+}
+.canvas-status { left: 10px; background: rgba(11, 13, 17, 0.76); color: #c4cad4; }
+.mode-badge { right: 10px; background: rgba(111, 143, 247, 0.88); color: white; }
+
+.canvas-tools {
+  display: flex;
+  align-items: center;
+  gap: 7px;
+  min-height: 39px;
+  padding-top: 8px;
+}
+.shortcut-hint { color: var(--muted); font-size: 10px; line-height: 1.3; }
+.tool-spacer { flex: 1; }
+
+.source-settings {
+  display: flex;
+  flex-direction: column;
+  gap: 7px;
+  padding-top: 5px;
+  border-top: 1px solid transparent;
+}
+.source-settings label, .advanced-grid label {
+  display: flex;
+  flex-direction: column;
+  gap: 5px;
+  color: var(--muted);
+  font-size: 10px;
+  font-weight: 620;
+}
+.source-settings input { width: 90px; }
+.source-meta { color: var(--muted); font-size: 10px; padding-bottom: 8px; }
+
+.playback-row { display: flex; align-items: center; gap: 9px; min-height: 34px; }
+.playback-button {
+  flex: 0 0 34px;
+  width: 34px;
+  height: 34px;
+  padding: 0;
+}
+.playback-icon {
+  display: block;
+  width: 15px;
+  height: 15px;
+  fill: currentColor;
+}
+.playback-row .source-meta { margin-left: auto; padding: 0; }
+.playback-row .danger-quiet { margin-left: 0; }
+.source-time { color: #cbd1dc; font-size: 10px; font-variant-numeric: tabular-nums; }
+.video-timeline {
+  position: relative;
+  height: 66px;
+  overflow: hidden;
+  border: 1px solid var(--line);
+  border-radius: 10px;
+  background: #090b0e;
+  cursor: ew-resize;
+  touch-action: none;
+  user-select: none;
+}
+.timeline-filmstrip { position: absolute; inset: 0; display: flex; justify-content: space-between; gap: 3px; }
+.timeline-thumbnail { position: relative; flex: 0 0 auto; height: 100%; overflow: hidden; }
+.timeline-thumbnail img { width: 100%; height: 100%; object-fit: cover; opacity: .84; }
+.timeline-selection { position: absolute; top: 0; bottom: 0; border-block: 2px solid var(--accent); pointer-events: none; }
+.timeline-before, .timeline-after { position: absolute; top: 0; bottom: 0; background: rgba(4,5,7,.67); pointer-events: none; }
+.trim-handle {
+  position: absolute; z-index: 12; top: 0; bottom: 0; width: 10px;
+  border: 0; background: var(--accent); transform: translateX(-50%); cursor: col-resize;
+}
+.trim-handle::after { content: ""; position: absolute; inset: 24px 3px; border-left: 1px solid white; border-right: 1px solid white; opacity: .8; }
+.timeline-playhead { position: absolute; z-index: 7; top: 0; bottom: 0; width: 2px; background: white; transform: translateX(-1px); pointer-events: none; box-shadow: 0 0 0 1px rgba(0,0,0,.3); }
+.timeline-playhead::before { content: ""; position: absolute; left: -4px; top: 0; border: 5px solid transparent; border-top-color: white; }
+.timeline-masks { position: absolute; z-index: 9; inset: 0; pointer-events: none; }
+.timeline-mask-preview {
+  position: absolute; width: 38px; height: 27px;
+  padding: 0; overflow: hidden; border: 2px solid; border-radius: 5px;
+  background: black; transform: translateX(-50%); pointer-events: auto; cursor: pointer;
+  box-shadow: 0 2px 8px rgba(0,0,0,.5);
+}
+.timeline-mask-preview img { display: block; width: 100%; height: 100%; object-fit: cover; }
+.timeline-mask-preview.excluded { opacity: .42; filter: grayscale(.45); }
+
+.target-workspace { display: grid; grid-template-columns: minmax(0, 1fr) 76px; gap: 10px; }
+.target-main { min-width: 0; }
+.target-rail {
+  display: flex; flex-direction: column; gap: 8px; max-height: 500px;
+  overflow-y: auto; scrollbar-width: thin;
+}
+.target-tile, .target-upload-tile { flex-basis: 62px; }
+
+input, textarea, select {
+  border: 1px solid var(--line);
+  color: var(--text);
+  background: #101318;
+  border-radius: 9px;
+  outline: none;
+  transition: border-color 120ms, box-shadow 120ms;
+}
+input:focus, textarea:focus, select:focus {
+  border-color: var(--accent-strong);
+  box-shadow: 0 0 0 3px rgba(111, 143, 247, 0.12);
+}
+input, select { height: 34px; padding: 0 9px; }
+textarea { width: 100%; padding: 10px 11px; resize: vertical; min-height: 62px; line-height: 1.4; }
+
+.target-stack { display: flex; flex-direction: column; gap: 18px; }
+.generate-card { padding: 16px; }
+.prompt-label { display: block; margin-bottom: 7px; font-size: 12px; font-weight: 620; }
+details { margin-top: 9px; }
+summary { color: var(--muted); font-size: 11px; cursor: pointer; user-select: none; }
+.advanced-grid {
+  display: grid;
+  grid-template-columns: repeat(3, minmax(0, 1fr));
+  gap: 10px;
+  margin-top: 11px;
+  padding: 12px;
+  background: #111419;
+  border: 1px solid var(--line);
+  border-radius: 11px;
+}
+.negative-prompt { grid-column: 1 / -1; }
+.generate-button {
+  position: relative;
+  overflow: hidden;
+  width: 100%;
+  height: 50px;
+  margin-top: 13px;
+  border: 0;
+  border-radius: 12px;
+  color: white;
+  background: linear-gradient(135deg, #6d8ef8, #876fe9);
+  font-size: 14px;
+  font-weight: 700;
+  cursor: pointer;
+  box-shadow: 0 10px 28px rgba(93, 111, 218, 0.23);
+  transition: filter 120ms, transform 120ms;
+}
+.generate-button-content { position: relative; z-index: 1; display: inline-flex; align-items: center; justify-content: center; gap: 8px; }
+.generate-button-progress {
+  position: absolute;
+  inset: 0 auto 0 0;
+  width: 0;
+  background: rgba(255, 255, 255, 0.18);
+  transition: width 180ms linear;
+}
+.generate-button-progress.indeterminate {
+  width: 36%;
+  animation: progress-slide 1.1s ease-in-out infinite;
+}
+.generate-button.generating:disabled { opacity: 1; filter: none; }
+.generate-button:hover { filter: brightness(1.08); }
+.generate-button:active { transform: translateY(1px); }
+.generate-button:disabled { cursor: wait; filter: grayscale(.45); opacity: .72; }
+.generate-hint { margin-top: 8px; min-height: 15px; color: var(--muted); font-size: 10px; text-align: center; }
+.generated-rail { display: flex; gap: 8px; margin-top: 10px; padding: 2px 1px 3px; overflow-x: auto; scrollbar-width: thin; }
+.generated-tile { flex: 0 0 74px; width: 74px; height: 52px; padding: 0; overflow: hidden; border: 1px solid var(--line); border-radius: 9px; background: #101319; cursor: pointer; }
+.generated-tile:hover { border-color: var(--accent); }
+.generated-tile video { display: block; width: 100%; height: 100%; object-fit: cover; }
+@keyframes progress-slide { from { transform: translateX(-110%); } to { transform: translateX(310%); } }
+
+.result-panel { margin: 18px 0 50px; padding: 16px; }
+.result-heading span:last-child { color: var(--muted); font-size: 11px; }
+.result-panel video { display: block; width: min(100%, 1000px); margin: auto; border-radius: 12px; background: black; }
+
+.context-menu {
+  position: fixed;
+  z-index: 100;
+  min-width: 210px;
+  max-height: min(430px, calc(100vh - 20px));
+  overflow-y: auto;
+  padding: 6px;
+  border: 1px solid var(--line-strong);
+  border-radius: 12px;
+  background: rgba(26, 30, 37, 0.98);
+  box-shadow: 0 18px 55px rgba(0, 0, 0, .45);
+  backdrop-filter: blur(18px);
+}
+.menu-label { padding: 8px 9px 5px; color: var(--muted); font-size: 9px; font-weight: 700; text-transform: uppercase; letter-spacing: .1em; }
+.menu-divider { height: 1px; background: var(--line); margin: 5px; }
+.menu-item {
+  display: flex;
+  align-items: center;
+  gap: 9px;
+  width: 100%;
+  min-height: 34px;
+  padding: 7px 9px;
+  border: 0;
+  border-radius: 8px;
+  color: var(--text);
+  text-align: left;
+  background: transparent;
+  cursor: pointer;
+}
+.menu-item:hover { background: #252a34; }
+.menu-item.danger { color: var(--danger); }
+.menu-item img { width: 58px; height: 38px; object-fit: cover; border-radius: 5px; }
+.color-dot { flex: 0 0 10px; width: 10px; height: 10px; border-radius: 50%; }
+.menu-copy { min-width: 0; display: flex; flex-direction: column; gap: 2px; }
+.menu-copy small { color: var(--muted); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; max-width: 130px; }
+
+.drag-ghost {
+  position: fixed;
+  z-index: 120;
+  pointer-events: none;
+  display: flex;
+  align-items: center;
+  gap: 7px;
+  padding: 7px 10px;
+  border: 1px solid rgba(255,255,255,.2);
+  border-radius: 999px;
+  background: rgba(20, 23, 29, .92);
+  box-shadow: 0 10px 30px rgba(0,0,0,.35);
+  font-size: 11px;
+  transform: translate(12px, 12px);
+}
+
+.toast {
+  position: fixed;
+  z-index: 200;
+  left: 50%; bottom: 24px;
+  transform: translateX(-50%);
+  max-width: min(520px, calc(100% - 32px));
+  padding: 10px 13px;
+  border: 1px solid var(--line-strong);
+  border-radius: 10px;
+  background: #232831;
+  box-shadow: 0 14px 40px rgba(0,0,0,.4);
+  font-size: 12px;
+}
+
+.hidden { display: none !important; }
+
+@media (max-width: 980px) {
+  .workspace { grid-template-columns: 1fr; }
+  .source-panel { order: 1; }
+  .target-stack { order: 2; }
+  .canvas-host { min-height: 330px; }
+}
+
+@media (max-width: 620px) {
+  .page-header, main { width: min(100% - 22px, 1600px); }
+  .page-header { padding-top: 22px; }
+  .page-header p, .status-pill { display: none; }
+  .panel, .generate-card, .result-panel { border-radius: 14px; }
+  .source-workspace { grid-template-columns: 68px minmax(0, 1fr); }
+  .source-tile, .source-upload-tile { flex-basis: 58px; }
+  .canvas-host, .target-host { min-height: 260px; }
+  .shortcut-hint { display: none; }
+  .source-settings { flex-wrap: wrap; }
+  .target-workspace { grid-template-columns: minmax(0, 1fr) 60px; }
+  .advanced-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); }
+}
+
+/* What Moves? project-page theme */
+:root {
+  color-scheme: light;
+  --background: #ffffff;
+  --panel: rgba(255, 255, 255, 0.88);
+  --panel-raised: #f7f8fb;
+  --line: #e4e6ed;
+  --line-strong: #c7ccd8;
+  --text: #26272b;
+  --muted: #777b84;
+  --accent: #25d5f4;
+  --accent-strong: #1688b9;
+  --magenta: #df58f5;
+  --blue: #73a3ff;
+  --danger: #c94d57;
+  --radius: 22px;
+  --shadow-cyan: 0 18px 52px rgba(37, 213, 244, 0.2);
+  --shadow-magenta: 0 18px 52px rgba(223, 88, 245, 0.18);
+}
+
+body {
+  background:
+    linear-gradient(120deg, rgba(37, 213, 244, 0.09), transparent 28%),
+    linear-gradient(260deg, rgba(223, 88, 245, 0.08), transparent 24%),
+    var(--background);
+}
+
+.page-header {
+  position: sticky;
+  top: 0;
+  z-index: 50;
+  align-items: center;
+  width: 100%;
+  max-width: none;
+  padding: 13px clamp(18px, 4vw, 56px);
+  border-bottom: 1px solid rgba(228, 230, 237, 0.8);
+  background: rgba(255, 255, 255, 0.76);
+  backdrop-filter: blur(18px);
+  -webkit-backdrop-filter: blur(18px);
+}
+
+.page-header > * { margin-inline: 0; }
+
+.start-over-button {
+  display: inline-flex;
+  align-items: center;
+  gap: 7px;
+  padding: 8px 12px;
+  border: 1px solid var(--line);
+  border-radius: 999px;
+  color: var(--accent-strong);
+  background: rgba(255, 255, 255, 0.82);
+  font-size: 12px;
+  font-weight: 680;
+  cursor: pointer;
+  transition: border-color 120ms, background 120ms, transform 120ms;
+}
+.start-over-button:hover { border-color: var(--line-strong); background: white; }
+.start-over-button:active { transform: translateY(1px); }
+.header-actions { display: flex; flex-direction: column; align-items: flex-end; gap: 3px; }
+.broom-icon {
+  width: 19px;
+  height: 19px;
+  background: currentColor;
+  -webkit-mask: url("/static/broom-flaticon.png") center / contain no-repeat;
+  mask: url("/static/broom-flaticon.png") center / contain no-repeat;
+}
+.app-brand {
+  display: flex;
+  align-items: center;
+  gap: 13px;
+}
+
+.app-kicker {
+  display: block;
+  margin-bottom: 4px;
+  color: var(--accent-strong);
+  font-size: 9px;
+  font-weight: 850;
+  letter-spacing: 0.13em;
+  text-transform: uppercase;
+}
+
+h1 {
+  font-family: Georgia, "Times New Roman", serif;
+  font-size: clamp(28px, 3vw, 42px);
+  line-height: 0.95;
+  letter-spacing: -0.02em;
+  background: linear-gradient(105deg, var(--accent) 4%, var(--blue) 42%, var(--magenta) 82%);
+  -webkit-background-clip: text;
+  background-clip: text;
+  color: transparent;
+}
+
+h2 {
+  font-size: 18px;
+  font-weight: 720;
+  letter-spacing: -0.02em;
+}
+
+.page-header p {
+  margin-top: 5px;
+  color: var(--muted);
+  font-size: 12px;
+}
+
+main { padding-top: 28px; }
+
+.status-pill {
+  padding: 0;
+  border: 0;
+  border-radius: 0;
+  color: var(--muted);
+  font-weight: 400;
+  background: transparent;
+  box-shadow: none;
+}
+
+.panel,
+.generate-card,
+.result-panel {
+  border-color: var(--line);
+  background: var(--panel);
+  box-shadow: 0 20px 64px rgba(32, 36, 46, 0.1);
+  backdrop-filter: blur(16px);
+  -webkit-backdrop-filter: blur(16px);
+}
+
+.source-panel { box-shadow: var(--shadow-cyan); }
+.target-panel { box-shadow: var(--shadow-magenta); }
+
+.eyebrow { color: var(--accent-strong); }
+
+.icon-button,
+.quiet-button,
+.tool-button,
+.text-button,
+.primary-small,
+.danger-quiet {
+  border-color: var(--line);
+  border-radius: 999px;
+  background: rgba(255, 255, 255, 0.88);
+  box-shadow: 0 8px 20px rgba(34, 38, 48, 0.06);
+}
+
+.icon-button:hover,
+.quiet-button:hover,
+.tool-button:hover,
+.text-button:hover,
+.primary-small:hover {
+  border-color: rgba(37, 213, 244, 0.55);
+  background: #ffffff;
+  color: var(--accent-strong);
+  box-shadow: var(--shadow-cyan);
+}
+
+.primary-small {
+  border-color: transparent;
+  color: #ffffff;
+  background: linear-gradient(135deg, var(--accent-strong), var(--blue));
+}
+
+.tool-button.active {
+  border-color: var(--accent);
+  color: var(--accent-strong);
+  background: rgba(37, 213, 244, 0.13);
+}
+
+.danger-quiet { color: var(--danger); background: transparent; box-shadow: none; }
+.danger-quiet:hover { border-color: rgba(201, 77, 87, 0.25); background: rgba(201, 77, 87, 0.08); }
+
+.source-tile,
+.source-upload-tile,
+.target-tile,
+.target-upload-tile { background: #f1f3f7; }
+
+.source-tile.selected,
+.target-tile.selected {
+  border-color: var(--accent);
+  box-shadow: 0 0 0 2px rgba(37, 213, 244, 0.2), var(--shadow-cyan);
+}
+
+.source-upload-tile:hover,
+.target-upload-tile:hover {
+  border-color: var(--accent);
+  color: var(--accent-strong);
+  background: rgba(37, 213, 244, 0.07);
+}
+
+.canvas-host {
+  border-color: #dfe3eb;
+  border-radius: 16px;
+  background:
+    linear-gradient(135deg, rgba(37, 213, 244, 0.06), transparent 34%),
+    #11141a;
+}
+
+.canvas-host:not(.has-image) {
+  border-color: var(--line-strong);
+  border-style: dashed;
+  background:
+    linear-gradient(135deg, rgba(37, 213, 244, 0.07), transparent 42%),
+    linear-gradient(315deg, rgba(223, 88, 245, 0.06), transparent 38%),
+    #f7f8fb;
+}
+
+.canvas-host:not(.has-image) .empty-state { color: var(--muted); }
+.canvas-host:not(.has-image) .empty-state strong { color: var(--text); font-weight: 650; }
+
+.mode-badge {
+  color: #173145;
+  font-weight: 750;
+  background: linear-gradient(135deg, rgba(37, 213, 244, 0.92), rgba(115, 163, 255, 0.92));
+}
+
+.source-time { color: #555b66; }
+
+input,
+textarea,
+select {
+  border-color: var(--line);
+  border-radius: 11px;
+  color: var(--text);
+  background: #ffffff;
+}
+
+input:focus,
+textarea:focus,
+select:focus {
+  border-color: var(--accent);
+  box-shadow: 0 0 0 3px rgba(37, 213, 244, 0.14);
+}
+
+.generate-card {
+  background:
+    linear-gradient(135deg, rgba(37, 213, 244, 0.07), transparent 36%),
+    linear-gradient(315deg, rgba(223, 88, 245, 0.07), transparent 30%),
+    var(--panel);
+}
+
+.advanced-grid { background: rgba(247, 248, 251, 0.86); }
+
+.generate-button {
+  background: linear-gradient(105deg, var(--accent), var(--blue) 48%, var(--magenta));
+  box-shadow: 0 14px 34px rgba(115, 163, 255, 0.28);
+}
+
+.progress-copy { color: #555b66; }
+.progress-track { background: #e9ecf2; }
+.progress-bar { background: linear-gradient(90deg, var(--accent), var(--magenta)); }
+
+.context-menu {
+  background: rgba(255, 255, 255, 0.97);
+  box-shadow: 0 18px 55px rgba(34, 38, 48, 0.18);
+}
+
+.menu-item:hover { background: rgba(37, 213, 244, 0.08); }
+
+.drag-ghost {
+  border: 0;
+  background: transparent;
+  box-shadow: none;
+}
+
+.toast {
+  background: rgba(255, 255, 255, 0.96);
+  box-shadow: 0 14px 40px rgba(34, 38, 48, 0.18);
+}
+
+/* Small, shape-preserving mask feedback for mapping gestures. */
+.drag-ghost {
+  width: 38px;
+  height: 38px;
+  padding: 0;
+  border-radius: 0;
+  transform: translate(-72%, -72%);
+  opacity: 0.94;
+  filter: drop-shadow(0 3px 5px rgba(31, 42, 55, 0.28));
+  animation: mask-pickup 150ms cubic-bezier(.2, 1.4, .4, 1) both;
+}
+
+.drag-mask-shape,
+.mapping-hint-token,
+.mapping-drop-pop { display: block; overflow: visible; }
+.drag-mask-shape { width: 100%; height: 100%; }
+.drag-mask-shape img,
+.mapping-hint-token img,
+.mapping-drop-pop img { display: block; width: 100%; height: 100%; object-fit: contain; }
+
+@keyframes mask-pickup {
+  from { opacity: 0; transform: translate(-72%, -72%) scale(.2); }
+  to { opacity: .94; transform: translate(-72%, -72%) scale(1); }
+}
+
+.mapping-drop-pop {
+  position: fixed;
+  z-index: 119;
+  width: 48px;
+  height: 48px;
+  pointer-events: none;
+  filter: drop-shadow(0 3px 6px rgba(31, 42, 55, 0.24));
+  animation: mask-drop 440ms cubic-bezier(.2, 1.3, .35, 1) both;
+}
+
+@keyframes mask-drop {
+  0% { opacity: 0; transform: translate(-50%, -50%) scale(.18); }
+  52% { opacity: .95; transform: translate(-50%, -50%) scale(1.24); }
+  100% { opacity: 0; transform: translate(-50%, -50%) scale(.88); }
+}
+
+.mapping-hint {
+  position: fixed;
+  inset: 0;
+  z-index: 115;
+  pointer-events: none;
+}
+
+.mapping-hint-token {
+  position: absolute;
+  width: 44px;
+  height: 44px;
+  filter: drop-shadow(0 3px 5px rgba(31, 42, 55, 0.3));
+  animation: mapping-hint-drag 2.45s cubic-bezier(.55, 0, .3, 1) both;
+}
+
+.mapping-hint-pointer {
+  position: absolute;
+  width: 24px;
+  height: 28px;
+  animation: mapping-hint-drag 2.45s cubic-bezier(.55, 0, .3, 1) both;
+}
+.mapping-hint-pointer svg { display: block; width: 100%; height: 100%; overflow: visible; }
+.mapping-hint-pointer path { fill: white; stroke: #26323f; stroke-width: 1.7; stroke-linejoin: round; filter: drop-shadow(0 2px 2px rgba(31, 42, 55, .24)); }
+
+.mapping-hint-copy {
+  position: fixed;
+  left: 50%;
+  bottom: 26px;
+  padding: 8px 12px;
+  border: 1px solid var(--line);
+  border-radius: 999px;
+  color: var(--text);
+  background: rgba(255, 255, 255, .94);
+  box-shadow: 0 10px 28px rgba(34, 38, 48, .14);
+  font-size: 11px;
+  font-weight: 680;
+  transform: translateX(-50%);
+  animation: mapping-hint-copy 2.45s ease both;
+}
+
+@keyframes mapping-hint-drag {
+  0%, 12% { opacity: 0; transform: translate(-50%, -50%) scale(.2); }
+  24% { opacity: .95; transform: translate(-50%, -50%) scale(1); }
+  72% { opacity: .95; transform: translate(calc(var(--hint-x) - 50%), calc(var(--hint-y) - 50%)) scale(1); }
+  86% { opacity: 1; transform: translate(calc(var(--hint-x) - 50%), calc(var(--hint-y) - 50%)) scale(1.3); }
+  100% { opacity: 0; transform: translate(calc(var(--hint-x) - 50%), calc(var(--hint-y) - 50%)) scale(.72); }
+}
+
+@keyframes mapping-hint-copy {
+  0%, 8%, 92%, 100% { opacity: 0; }
+  18%, 82% { opacity: 1; }
+}
+
+/* Complete light-mode treatment, including transient and media surfaces. */
+.canvas-host,
+.canvas-host video,
+.video-timeline,
+.timeline-mask-preview,
+.generated-tile,
+.result-panel video {
+  background: #f3f5f8;
+}
+
+.canvas-host { border-color: #dfe3eb; }
+.timeline-before,
+.timeline-after { background: rgba(255, 255, 255, .66); }
+.timeline-playhead { background: var(--accent-strong); box-shadow: 0 0 0 1px rgba(255, 255, 255, .85); }
+.timeline-playhead::before { border-top-color: var(--accent-strong); }
+.timeline-mask-preview { box-shadow: 0 2px 8px rgba(34, 38, 48, .2); }
+.canvas-status { color: var(--muted); background: rgba(255, 255, 255, .9); }
+.context-menu { color: var(--text); background: rgba(255, 255, 255, .98); }
+.menu-item:hover { background: #f1f4f8; }
+.toast { color: var(--text); background: rgba(255, 255, 255, .98); }
+
+.media-loader {
+  position: absolute;
+  inset: 0;
+  z-index: 24;
+  display: grid;
+  place-items: center;
+  pointer-events: none;
+  border-radius: inherit;
+  background: rgba(247, 248, 251, .58);
+  backdrop-filter: blur(1.5px);
+  -webkit-backdrop-filter: blur(1.5px);
+}
+
+.loading-spinner {
+  width: 34px;
+  height: 34px;
+  border: 3px solid rgba(115, 163, 255, .2);
+  border-top-color: var(--accent);
+  border-right-color: var(--magenta);
+  border-radius: 50%;
+  box-shadow: 0 5px 18px rgba(115, 163, 255, .18);
+  animation: media-spin .72s linear infinite;
+}
+
+@keyframes media-spin { to { transform: rotate(360deg); } }
+
+.result-media {
+  position: relative;
+  flex: 1 1 auto;
+  display: grid;
+  min-width: 0;
+  min-height: 0;
+  place-items: center;
+  overflow: hidden;
+  border-radius: 12px;
+}
+
+.result-media video {
+  width: 100%;
+  height: 100%;
+  min-height: 0;
+  object-fit: contain;
+}
+
+@media (prefers-reduced-motion: reduce) {
+  .drag-ghost,
+  .mapping-drop-pop,
+  .mapping-hint-token,
+  .mapping-hint-pointer,
+  .mapping-hint-copy { animation-duration: 1ms; }
+  .loading-spinner { animation: none; }
+}
+
+@media (max-width: 620px) {
+  .page-header {
+    width: 100%;
+    padding: 11px;
+  }
+
+  .app-kicker { display: none; }
+}
+
+/* Viewport-fitting application shell */
+html,
+body {
+  width: 100%;
+  height: 100%;
+  min-height: 0;
+  overflow: hidden;
+}
+
+body {
+  display: flex;
+  flex-direction: column;
+}
+
+.page-header {
+  flex: 0 0 auto;
+  min-height: 0;
+}
+
+main {
+  display: flex;
+  flex: 1 1 auto;
+  min-height: 0;
+  padding-top: clamp(8px, 2vh, 20px);
+  padding-bottom: clamp(8px, 2vh, 20px);
+}
+
+.workspace {
+  width: 100%;
+  height: 100%;
+  min-height: 0;
+  align-items: stretch;
+  gap: clamp(8px, 1.2vw, 18px);
+}
+
+.panel,
+.generate-card,
+.target-stack,
+.source-workspace,
+.source-main,
+.target-workspace,
+.target-main {
+  min-width: 0;
+  min-height: 0;
+}
+
+.source-panel,
+.target-panel {
+  display: flex;
+  flex-direction: column;
+  overflow: hidden;
+}
+
+.source-workspace,
+.target-workspace {
+  flex: 1 1 auto;
+  overflow: hidden;
+}
+
+.source-main,
+.target-main {
+  display: flex;
+  flex-direction: column;
+}
+
+.canvas-host {
+  flex: 1 1 auto;
+  width: 100%;
+  height: auto;
+  min-height: 0;
+  max-height: none;
+}
+
+.canvas-host canvas,
+.canvas-host video {
+  max-width: 100%;
+  max-height: 100%;
+}
+
+.source-rail,
+.target-rail {
+  max-height: 100%;
+  min-height: 0;
+}
+
+.target-stack {
+  display: grid;
+  grid-template-rows: minmax(0, 1fr) auto;
+  height: 100%;
+}
+
+.result-panel:not(.hidden) {
+  position: fixed;
+  top: clamp(72px, 12vh, 110px);
+  left: 50%;
+  width: min(620px, calc(100vw - 28px));
+  height: min(460px, calc(100vh - 96px));
+  transform: translateX(-50%);
+  z-index: 70;
+  display: flex;
+  flex-direction: column;
+  margin: 0;
+  overflow: hidden;
+  box-shadow: 0 28px 90px rgba(32, 36, 46, 0.24), var(--shadow-magenta);
+}
+
+.result-actions { display: flex; align-items: center; gap: 10px; }
+.result-close { width: 28px; height: 28px; padding: 0; font-size: 19px; line-height: 1; }
+.result-actions > span { color: var(--muted); font-size: 11px; }
+
+.result-panel video {
+  flex: 1 1 auto;
+  width: auto;
+  max-width: 100%;
+  min-height: 0;
+  object-fit: contain;
+}
+
+button,
+.status-pill,
+.canvas-status,
+.mode-badge,
+.source-tile .tile-label,
+.target-tile .tile-label {
+  align-items: center;
+  justify-content: center;
+  text-align: center;
+}
+
+button,
+.status-pill,
+.canvas-status,
+.mode-badge {
+  display: inline-flex;
+}
+
+button {
+  line-height: 1.15;
+}
+
+.empty-state {
+  display: flex;
+  flex-direction: column;
+}
+
+.source-tile,
+.target-tile {
+  display: block;
+}
+
+.menu-item {
+  justify-content: flex-start;
+  text-align: left;
+}
+
+.panel-heading,
+.result-heading {
+  flex: 0 0 auto;
+}
+
+@media (max-height: 820px) {
+  .page-header { padding-block: 8px; }
+  h1 { font-size: clamp(25px, 2.4vw, 34px); }
+  .page-header p { margin-top: 2px; }
+  .panel { padding: 12px; }
+  .panel-heading, .result-heading { margin-bottom: 8px; }
+  .canvas-tools { min-height: 32px; padding-top: 5px; }
+  .target-stack { gap: 9px; }
+  .generate-card { padding: 11px 12px; }
+  .generate-button { height: 40px; margin-top: 8px; }
+  textarea { min-height: 44px; padding-block: 7px; }
+  details { margin-top: 5px; }
+  .generate-hint { margin-top: 5px; }
+}
+
+@media (max-width: 980px) {
+  .workspace {
+    grid-template-columns: 1fr;
+    grid-template-rows: minmax(0, 1.05fr) minmax(0, 0.95fr);
+  }
+
+  .source-panel,
+  .target-stack {
+    min-height: 0;
+  }
+
+  .canvas-host,
+  .target-host {
+    min-height: 0;
+  }
+}
+
+@media (max-width: 620px) {
+  main {
+    width: calc(100% - 14px);
+    padding-block: 7px;
+  }
+
+  .workspace { gap: 7px; }
+  .panel, .generate-card { padding: 9px; border-radius: 15px; }
+  .panel-heading { margin-bottom: 6px; }
+  h2 { font-size: 15px; }
+  .source-workspace { grid-template-columns: 64px minmax(0, 1fr); gap: 7px; }
+  .target-workspace { grid-template-columns: minmax(0, 1fr) 54px; gap: 7px; }
+  .source-tile, .source-upload-tile { flex-basis: 54px; }
+  .target-tile, .target-upload-tile { flex-basis: 48px; }
+  .canvas-host, .target-host { min-height: 0; border-radius: 11px; }
+  .generate-button { height: 36px; }
+  .prompt-label { margin-bottom: 4px; }
+  textarea { min-height: 38px; }
+  .generate-hint { display: none; }
+}
+
+@media (max-height: 650px) {
+  .page-header p,
+  .app-kicker,
+  .shortcut-hint,
+  .generate-hint { display: none; }
+
+  .page-header { padding-block: 5px; }
+  h1 { font-size: 25px; }
+  main { padding-block: 6px; }
+
+  .workspace {
+    grid-template-columns: minmax(0, 1.1fr) minmax(300px, 0.9fr);
+    grid-template-rows: 1fr;
+  }
+
+  .panel { padding: 9px; }
+  .panel-heading { margin-bottom: 5px; }
+  .eyebrow { margin-bottom: 1px; }
+  .canvas-tools { min-height: 29px; }
+  .source-settings { gap: 3px; padding-top: 2px; }
+  .video-timeline { height: 48px; }
+  .generate-card { padding: 8px 10px; }
+  .generate-button { height: 34px; }
+  textarea { min-height: 34px; }
+}

app/static/video-timeline.js

@@ -0,0 +1,284 @@
+function clamp(value, minimum, maximum) {
+  return Math.min(maximum, Math.max(minimum, value));
+}
+
+function timeLabel(seconds) {
+  const minutes = Math.floor(seconds / 60);
+  const remainder = Math.max(0, seconds - minutes * 60);
+  return `${minutes}:${remainder.toFixed(1).padStart(4, "0")}`;
+}
+
+export class VideoTimeline {
+  constructor({ root, video, playButton, time, callbacks = {} }) {
+    this.root = root;
+    this.video = video;
+    this.playButton = playButton;
+    this.time = time;
+    this.callbacks = callbacks;
+    this.filmstrip = root.querySelector(".timeline-filmstrip");
+    this.maskLayer = root.querySelector(".timeline-masks");
+    this.playhead = root.querySelector(".timeline-playhead");
+    this.selection = root.querySelector(".timeline-selection");
+    this.before = root.querySelector(".timeline-before");
+    this.after = root.querySelector(".timeline-after");
+    this.leftHandle = root.querySelector('[data-handle="left"]');
+    this.rightHandle = root.querySelector('[data-handle="right"]');
+    this.source = null;
+    this.currentFrame = 0;
+    this.trimStart = 0;
+    this.trimEnd = 0;
+    this.drag = null;
+    this.animation = null;
+    this.thumbnailSignature = "";
+
+    playButton.addEventListener("click", () => this.togglePlayback());
+    root.addEventListener("pointerdown", (event) => this._pointerDown(event));
+    video.addEventListener("play", () => this._playbackChanged(true));
+    video.addEventListener("pause", () => this._playbackChanged(false));
+    video.addEventListener("ended", () => this._loop());
+    ["loadstart", "waiting", "seeking"].forEach((eventName) => {
+      video.addEventListener(eventName, () => this.callbacks.onLoading?.(true));
+    });
+    ["loadeddata", "canplay", "playing", "seeked", "emptied", "error"].forEach((eventName) => {
+      video.addEventListener(eventName, () => this.callbacks.onLoading?.(false));
+    });
+    this.resizeObserver = new ResizeObserver(() => this._renderThumbnails());
+    this.resizeObserver.observe(root);
+  }
+
+  setSource(source) {
+    const sameMedia = Boolean(
+      source && this.source && source.id === this.source.id && source.revision === this.source.revision
+    );
+    this.source = source;
+    this.root.classList.toggle("hidden", !source);
+    if (!source) {
+      this.callbacks.onLoading?.(false);
+      this.video.pause();
+      this.video.removeAttribute("src");
+      this.filmstrip.replaceChildren();
+      this.maskLayer.replaceChildren();
+      return;
+    }
+    this.trimStart = source.trim_start;
+    this.trimEnd = source.trim_end;
+    const absoluteUrl = new URL(source.video_url, window.location.href).href;
+    if (this.video.src !== absoluteUrl) {
+      this.video.pause();
+      this.video.src = source.video_url;
+      this.video.load();
+    }
+    if (!sameMedia || (this.video.paused && !this.drag)) {
+      this.currentFrame = source.current_frame;
+      this._setVideoTime(this.currentFrame / source.fps);
+    }
+    this._renderThumbnails();
+    this._renderMasks();
+    this._renderGeometry();
+  }
+
+  togglePlayback() {
+    if (!this.source) return;
+    if (!this.video.paused) {
+      this.video.pause();
+      this.callbacks.onSeek?.(this.currentFrame, true);
+      return;
+    }
+    if (this.currentFrame < this.trimStart || this.currentFrame >= this.trimEnd) {
+      this._seekLocal(this.trimStart);
+    }
+    this.video.play().catch(() => {});
+  }
+
+  pause() {
+    this.video.pause();
+  }
+
+  jumpToMask(mask) {
+    this.pause();
+    this._seekLocal(mask.frame_index);
+    this.callbacks.onSeek?.(mask.frame_index, true);
+  }
+
+  _playbackChanged(playing) {
+    this.playButton.innerHTML = playing
+      ? '<svg class="playback-icon" viewBox="0 0 24 24" aria-hidden="true"><path d="M6 5h4v14H6zM14 5h4v14h-4z"></path></svg>'
+      : '<svg class="playback-icon" viewBox="0 0 24 24" aria-hidden="true"><path d="M8 5v14l11-7z"></path></svg>';
+    this.playButton.setAttribute("aria-label", playing ? "Pause" : "Play");
+    this.playButton.title = playing ? "Pause" : "Play";
+    this.callbacks.onPlaybackChange?.(playing);
+    cancelAnimationFrame(this.animation);
+    if (playing) this._tick();
+  }
+
+  _tick() {
+    if (!this.source || this.video.paused) return;
+    const frame = clamp(
+      Math.round(this.video.currentTime * this.source.fps),
+      this.trimStart,
+      this.trimEnd,
+    );
+    this.currentFrame = frame;
+    if (this.video.currentTime >= (this.trimEnd + 0.75) / this.source.fps) {
+      this._loop();
+    } else {
+      this._renderGeometry();
+    }
+    this.animation = requestAnimationFrame(() => this._tick());
+  }
+
+  _loop() {
+    if (!this.source) return;
+    this._seekLocal(this.trimStart);
+    if (!this.video.paused) this.video.play().catch(() => {});
+  }
+
+  _seekLocal(frame) {
+    if (!this.source) return;
+    this.currentFrame = clamp(Math.round(frame), 0, this.source.frame_count - 1);
+    this._setVideoTime(this.currentFrame / this.source.fps);
+    this._renderGeometry();
+  }
+
+  _setVideoTime(seconds) {
+    const apply = () => {
+      try { this.video.currentTime = seconds; } catch (_) {}
+    };
+    if (this.video.readyState >= 1) apply();
+    else this.video.addEventListener("loadedmetadata", apply, { once: true });
+  }
+
+  _frameAt(clientX) {
+    const rect = this.root.getBoundingClientRect();
+    const ratio = clamp((clientX - rect.left) / Math.max(rect.width, 1), 0, 1);
+    return Math.round(ratio * Math.max(this.source.frame_count - 1, 0));
+  }
+
+  _pointerDown(event) {
+    if (!this.source || event.button !== 0 || event.target.closest(".timeline-mask-preview")) return;
+    const handle = event.target.closest(".trim-handle")?.dataset.handle;
+    this.pause();
+    this.drag = {
+      mode: handle || "playhead",
+      pointerId: event.pointerId,
+      originalStart: this.trimStart,
+      originalEnd: this.trimEnd,
+    };
+    this.root.setPointerCapture(event.pointerId);
+    this.root.addEventListener("pointermove", this._boundMove ||= (value) => this._pointerMove(value));
+    this.root.addEventListener("pointerup", this._boundUp ||= (value) => this._pointerUp(value));
+    this.root.addEventListener("pointercancel", this._boundUp);
+    this._pointerMove(event);
+    event.preventDefault();
+  }
+
+  _pointerMove(event) {
+    if (!this.drag || !this.source) return;
+    const frame = this._frameAt(event.clientX);
+    const minimum = Math.min(8, this.source.frame_count);
+    if (this.drag.mode === "left") {
+      this.trimStart = clamp(frame, 0, this.trimEnd - minimum + 1);
+      this._seekLocal(Math.max(this.currentFrame, this.trimStart));
+    } else if (this.drag.mode === "right") {
+      this.trimEnd = clamp(frame, this.trimStart + minimum - 1, this.source.frame_count - 1);
+      this._seekLocal(Math.min(this.currentFrame, this.trimEnd));
+    } else {
+      this._seekLocal(frame);
+      this.callbacks.onSeek?.(this.currentFrame, false);
+    }
+    this._renderGeometry();
+  }
+
+  async _pointerUp(event) {
+    if (!this.drag || !this.source) return;
+    const drag = this.drag;
+    this.drag = null;
+    try { this.root.releasePointerCapture(event.pointerId); } catch (_) {}
+    this.root.removeEventListener("pointermove", this._boundMove);
+    this.root.removeEventListener("pointerup", this._boundUp);
+    this.root.removeEventListener("pointercancel", this._boundUp);
+    if (drag.mode === "playhead") {
+      await this.callbacks.onSeek?.(this.currentFrame, true);
+    } else {
+      const accepted = await this.callbacks.onTrim?.(this.trimStart, this.trimEnd);
+      if (accepted === false) {
+        this.trimStart = drag.originalStart;
+        this.trimEnd = drag.originalEnd;
+        this._seekLocal(clamp(this.currentFrame, this.trimStart, this.trimEnd));
+      }
+    }
+    this._renderGeometry();
+  }
+
+  _ratio(frame) {
+    return this.source.frame_count <= 1 ? 0 : frame / (this.source.frame_count - 1);
+  }
+
+  _renderGeometry() {
+    if (!this.source) return;
+    const start = this._ratio(this.trimStart) * 100;
+    const end = this._ratio(this.trimEnd) * 100;
+    const current = this._ratio(this.currentFrame) * 100;
+    this.selection.style.left = `${start}%`;
+    this.selection.style.width = `${Math.max(0, end - start)}%`;
+    this.before.style.width = `${start}%`;
+    this.after.style.left = `${end}%`;
+    this.after.style.width = `${100 - end}%`;
+    this.leftHandle.style.left = `${start}%`;
+    this.rightHandle.style.left = `${end}%`;
+    this.playhead.style.left = `${current}%`;
+    this.time.textContent = `${timeLabel(this.currentFrame / this.source.fps)}  ·  frame ${this.currentFrame + 1}/${this.source.frame_count}  ·  ${this.trimEnd - this.trimStart + 1} selected`;
+  }
+
+  _renderThumbnails() {
+    if (!this.source || !this.root.clientWidth) return;
+    const height = 50;
+    const width = Math.max(48, height * this.source.width / this.source.height);
+    const count = Math.max(1, Math.floor(this.root.clientWidth / (width + 3)));
+    const indices = [];
+    for (let index = 0; index < Math.min(count, this.source.frame_count); index += 1) {
+      const denominator = Math.max(Math.min(count, this.source.frame_count) - 1, 1);
+      indices.push(Math.round(index * (this.source.frame_count - 1) / denominator));
+    }
+    const signature = `${this.source.id}:${this.source.revision}:${indices.join(",")}`;
+    if (signature === this.thumbnailSignature) return;
+    this.thumbnailSignature = signature;
+    this.filmstrip.replaceChildren(...indices.map((frame) => {
+      const item = document.createElement("div");
+      item.className = "timeline-thumbnail";
+      item.style.width = `${width}px`;
+      const image = document.createElement("img");
+      image.src = `/api/sessions/${this.source.session_id}/sources/${this.source.id}/frames/${frame}.jpg`;
+      image.alt = "";
+      item.append(image);
+      return item;
+    }));
+  }
+
+  _renderMasks() {
+    if (!this.source) return;
+    this.maskLayer.replaceChildren(...this.source.masks.map((mask, index) => {
+      const button = document.createElement("button");
+      button.className = "timeline-mask-preview";
+      if (mask.frame_index < this.trimStart || mask.frame_index > this.trimEnd) {
+        button.classList.add("excluded");
+      }
+      button.style.left = `${this._ratio(mask.frame_index) * 100}%`;
+      button.style.borderColor = mask.color;
+      button.style.top = `${3 + (index % 3) * 5}px`;
+      const excluded = button.classList.contains("excluded") ? " · outside selected interval" : "";
+      button.title = `Mask ${index + 1} · ${timeLabel(mask.frame_index / this.source.fps)}${excluded}`;
+      const image = document.createElement("img");
+      image.src = mask.preview_url;
+      image.alt = "";
+      button.append(image);
+      button.addEventListener("pointerdown", (event) => event.stopPropagation());
+      button.addEventListener("click", (event) => {
+        event.stopPropagation();
+        this.jumpToMask(mask);
+        this.callbacks.onMaskClick?.(mask);
+      });
+      return button;
+    }));
+  }
+}

docs/videos/app_demo.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db914ecc898ea2100affa5a57709342f62a7dce1806f770e368a8e060cfb209c
+size 1952239

hubconf.py

@@ -0,0 +1,311 @@
+"""PyTorch Hub entrypoints for WhatMoves and its Wan motion adapter."""
+
+from __future__ import annotations
+
+import hashlib as _hashlib
+import json as _json
+from pathlib import PurePosixPath as _PurePosixPath
+
+import torch as _torch
+
+dependencies = ["torch", "einops", "huggingface_hub"]
+
+_WEIGHTS_REPO = "CompVis/WhatMoves"
+_WEIGHTS_REVISION = "31820f6dbaa3f4f535bdb472d44db2c0bb03349c"
+_WAN_BASE_MODEL_ID = "Wan-AI/Wan2.2-I2V-A14B"
+_WAN_BASE_REVISION = "206a9ee1b7bfaaf8f7e4d81335650533490646a3"
+_DEFAULT_VARIANT = "gated_static_step600000"
+_VARIANTS = {
+    _DEFAULT_VARIANT: {
+        "what_moves": "what_moves/gated_static_step600000.pt",
+        "metadata": "wan/gated_static_step600000/metadata.json",
+    }
+}
+_WAN_ALLOW_PATTERNS = (
+    "high_noise_model/*",
+    "low_noise_model/*",
+    "Wan2.1_VAE.pth",
+    "models_t5_umt5-xxl-enc-bf16.pth",
+    "google/umt5-xxl/*",
+)
+
+
+def _variant_files(variant: str) -> dict[str, str]:
+    try:
+        return _VARIANTS[variant]
+    except KeyError as error:
+        choices = ", ".join(sorted(_VARIANTS))
+        raise ValueError(
+            f"Unknown WhatMoves variant {variant!r}; choose: {choices}"
+        ) from error
+
+
+def _download_file(
+    filename: str,
+    *,
+    repo_id: str,
+    revision: str | None,
+    cache_dir: str | None,
+    token: bool | str | None,
+    force_download: bool,
+    local_files_only: bool,
+) -> str:
+    from huggingface_hub import hf_hub_download
+
+    return hf_hub_download(
+        repo_id=repo_id,
+        filename=filename,
+        revision=revision,
+        cache_dir=cache_dir,
+        token=token,
+        force_download=force_download,
+        local_files_only=local_files_only,
+    )
+
+
+def _download_kwargs(
+    *,
+    repo_id: str,
+    revision: str | None,
+    cache_dir: str | None,
+    hf_token: bool | str | None,
+    force_download: bool,
+    local_files_only: bool,
+) -> dict[str, object]:
+    return {
+        "repo_id": repo_id,
+        "revision": revision,
+        "cache_dir": cache_dir,
+        "token": hf_token,
+        "force_download": force_download,
+        "local_files_only": local_files_only,
+    }
+
+
+def _release_revision(repo_id: str, revision: str | None) -> str | None:
+    """Pin official release downloads while leaving custom repositories alone."""
+    if revision is None and repo_id == _WEIGHTS_REPO:
+        return _WEIGHTS_REVISION
+    return revision
+
+
+def _metadata(
+    variant: str,
+    *,
+    repo_id: str,
+    revision: str | None,
+    cache_dir: str | None,
+    hf_token: bool | str | None,
+    force_download: bool,
+    local_files_only: bool,
+) -> dict[str, object]:
+    files = _variant_files(variant)
+    path = _download_file(
+        files["metadata"],
+        **_download_kwargs(
+            repo_id=repo_id,
+            revision=revision,
+            cache_dir=cache_dir,
+            hf_token=hf_token,
+            force_download=force_download,
+            local_files_only=local_files_only,
+        ),
+    )
+    with open(path, encoding="utf-8") as file:
+        metadata = _json.load(file)
+    required = {
+        "format": "whatmoves_wan_motion_adapter",
+        "format_version": 1,
+        "variant": variant,
+    }
+    mismatched = {
+        name: metadata.get(name)
+        for name, expected in required.items()
+        if metadata.get(name) != expected
+    }
+    if mismatched:
+        raise ValueError(f"Invalid Wan adapter metadata in {path}: {mismatched}")
+    return metadata
+
+
+def _metadata_path(metadata: dict[str, object], name: str) -> str:
+    value = metadata.get(name)
+    if not isinstance(value, str):
+        raise ValueError(f"Wan adapter metadata has no string {name!r}")
+    path = _PurePosixPath(value)
+    if path.is_absolute() or ".." in path.parts:
+        raise ValueError(f"Wan adapter metadata has unsafe {name!r}: {value!r}")
+    return value
+
+
+def _verify_download(
+    path: str,
+    filename: str,
+    metadata: dict[str, object],
+) -> None:
+    hashes = metadata.get("sha256")
+    expected = hashes.get(filename) if isinstance(hashes, dict) else None
+    if not isinstance(expected, str) or len(expected) != 64:
+        raise ValueError(f"Wan adapter metadata has no SHA-256 for {filename!r}")
+    digest = _hashlib.sha256()
+    with open(path, "rb") as file:
+        for chunk in iter(lambda: file.read(8 * 1024 * 1024), b""):
+            digest.update(chunk)
+    actual = digest.hexdigest()
+    if actual != expected:
+        raise RuntimeError(
+            f"Checksum mismatch for {filename}: expected {expected}, got {actual}"
+        )
+
+
+def what_moves(
+    *,
+    variant: str = _DEFAULT_VARIANT,
+    checkpoint: str | None = None,
+    device: str | _torch.device = "cpu",
+    dtype: _torch.dtype | None = None,
+    repo_id: str = _WEIGHTS_REPO,
+    revision: str | None = None,
+    cache_dir: str | None = None,
+    hf_token: bool | str | None = None,
+    force_download: bool = False,
+    local_files_only: bool = False,
+    verify_checksum: bool = True,
+    **model_kwargs,
+):
+    """Load the released localized-motion encoder.
+
+    By default the checkpoint is downloaded from ``CompVis/WhatMoves`` and
+    cached by Hugging Face. Pass ``checkpoint`` to use a local file instead.
+    Private repositories work after ``hf auth login`` or with ``HF_TOKEN``.
+    """
+    _variant_files(variant)
+    if checkpoint is None:
+        revision = _release_revision(repo_id, revision)
+        metadata = _metadata(
+            variant,
+            repo_id=repo_id,
+            revision=revision,
+            cache_dir=cache_dir,
+            hf_token=hf_token,
+            force_download=force_download,
+            local_files_only=local_files_only,
+        )
+        filename = _metadata_path(metadata, "what_moves_checkpoint")
+        checkpoint = _download_file(
+            filename,
+            **_download_kwargs(
+                repo_id=repo_id,
+                revision=revision,
+                cache_dir=cache_dir,
+                hf_token=hf_token,
+                force_download=force_download,
+                local_files_only=local_files_only,
+            ),
+        )
+        if verify_checksum:
+            _verify_download(checkpoint, filename, metadata)
+
+    from what_moves import WhatMoves as _WhatMoves
+
+    return _WhatMoves.from_pretrained(
+        checkpoint,
+        device=device,
+        dtype=dtype,
+        **model_kwargs,
+    )
+
+
+def wan(
+    *,
+    variant: str = _DEFAULT_VARIANT,
+    wan_checkpoint: str | None = None,
+    what_moves_checkpoint: str | None = None,
+    adapter_checkpoint: str | None = None,
+    device: str | _torch.device = "cuda",
+    dtype: _torch.dtype = _torch.bfloat16,
+    repo_id: str = _WEIGHTS_REPO,
+    revision: str | None = None,
+    base_model_id: str | None = None,
+    base_revision: str | None = None,
+    cache_dir: str | None = None,
+    hf_token: bool | str | None = None,
+    force_download: bool = False,
+    local_files_only: bool = False,
+    verify_checksum: bool = True,
+    **model_kwargs,
+):
+    """Load Wan2.2 I2V-A14B with the matching WhatMoves adapter.
+
+    Release weights and metadata come from ``CompVis/WhatMoves``. The official
+    Wan base snapshot is downloaded automatically unless ``wan_checkpoint`` is
+    supplied. Any checkpoint argument can independently override its download.
+    """
+    _variant_files(variant)
+    revision = _release_revision(repo_id, revision)
+    metadata = None
+    if (
+        what_moves_checkpoint is None
+        or adapter_checkpoint is None
+        or (wan_checkpoint is None and base_model_id is None)
+    ):
+        metadata = _metadata(
+            variant,
+            repo_id=repo_id,
+            revision=revision,
+            cache_dir=cache_dir,
+            hf_token=hf_token,
+            force_download=force_download,
+            local_files_only=local_files_only,
+        )
+    download_kwargs = _download_kwargs(
+        repo_id=repo_id,
+        revision=revision,
+        cache_dir=cache_dir,
+        hf_token=hf_token,
+        force_download=force_download,
+        local_files_only=local_files_only,
+    )
+    if what_moves_checkpoint is None:
+        filename = _metadata_path(metadata, "what_moves_checkpoint")
+        what_moves_checkpoint = _download_file(
+            filename,
+            **download_kwargs,
+        )
+        if verify_checksum:
+            _verify_download(what_moves_checkpoint, filename, metadata)
+    if adapter_checkpoint is None:
+        filename = _metadata_path(metadata, "adapter_checkpoint")
+        adapter_checkpoint = _download_file(
+            filename,
+            **download_kwargs,
+        )
+        if verify_checksum:
+            _verify_download(adapter_checkpoint, filename, metadata)
+    if wan_checkpoint is None:
+        if base_model_id is None:
+            base_model_id = _metadata_path(metadata, "base_model_id")
+        if base_revision is None and base_model_id == _WAN_BASE_MODEL_ID:
+            base_revision = _WAN_BASE_REVISION
+        from huggingface_hub import snapshot_download
+
+        wan_checkpoint = snapshot_download(
+            repo_id=base_model_id,
+            revision=base_revision,
+            cache_dir=cache_dir,
+            token=hf_token,
+            force_download=force_download,
+            local_files_only=local_files_only,
+            allow_patterns=list(_WAN_ALLOW_PATTERNS),
+        )
+
+    from what_moves import load_wan_model as _load_wan_model
+
+    return _load_wan_model(
+        wan_checkpoint,
+        what_moves_checkpoint,
+        adapter_checkpoint,
+        device=device,
+        dtype=dtype,
+        **model_kwargs,
+    )

requirements-core.txt

@@ -0,0 +1,5 @@
+# Minimal dependencies for the WhatMoves motion encoder.
+torch==2.8.0
+einops==0.8.1
+huggingface-hub==0.36.2
+numpy==1.26.4

requirements-wan.txt

@@ -0,0 +1,10 @@
+# Additional dependencies for Wan motion transfer.
+-r requirements-core.txt
+peft==0.17.0
+diffusers @ git+https://github.com/huggingface/diffusers.git@7ea065c5070a5278259e6f1effa9dccea232e62a
+transformers==4.57.1
+tokenizers==0.22.1
+accelerate==1.10.0
+safetensors==0.5.3
+ftfy==6.3.1
+regex==2024.11.6

requirements.txt

@@ -0,0 +1,2 @@
+# Complete, Hugging Face Spaces-ready WhatMoves installation.
+-r app/requirements.txt

wan/LICENSE.txt

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

wan/README.md

@@ -0,0 +1,13 @@
+# Vendored Wan2.2 runtime
+
+This directory contains the minimal inference runtime needed by the WhatMoves
+Wan I2V-A14B adapter. The source files were reduced from the official
+[Wan2.2 repository](https://github.com/Wan-Video/Wan2.2) at
+commit `388807310646ed5f318a99f8e8d9ad28c5b65373`.
+
+Wan2.2 is distributed under the Apache License 2.0; see `LICENSE.txt` in this
+directory. Changes are limited to omitting unused pipelines and model variants,
+removing unused imports, and using PyTorch's restricted tensor-only checkpoint
+loader and current autocast API. Attention is routed through Wan's existing
+PyTorch-SDPA fallback when FlashAttention is unavailable. Modified files are
+marked in their headers.

wan/__init__.py

@@ -0,0 +1,4 @@
+# Copyright 2024-2025 The Alibaba Wan Team Authors. All rights reserved.
+# Modified by the WhatMoves authors: removed unused pipeline imports.
+
+"""Minimal vendored runtime from the official Wan2.2 repository."""

wan/modules/__init__.py

@@ -0,0 +1,4 @@
+# Copyright 2024-2025 The Alibaba Wan Team Authors. All rights reserved.
+# Modified by the WhatMoves authors: removed eager re-exports.
+
+"""Wan2.2 model, text encoder, attention, and VAE modules."""

wan/modules/attention.py

@@ -0,0 +1,179 @@
+# Copyright 2024-2025 The Alibaba Wan Team Authors. All rights reserved.
+import torch
+
+try:
+    import flash_attn_interface
+    FLASH_ATTN_3_AVAILABLE = True
+except ModuleNotFoundError:
+    FLASH_ATTN_3_AVAILABLE = False
+
+try:
+    import flash_attn
+    FLASH_ATTN_2_AVAILABLE = True
+except ModuleNotFoundError:
+    FLASH_ATTN_2_AVAILABLE = False
+
+import warnings
+
+__all__ = [
+    'flash_attention',
+    'attention',
+]
+
+
+def flash_attention(
+    q,
+    k,
+    v,
+    q_lens=None,
+    k_lens=None,
+    dropout_p=0.,
+    softmax_scale=None,
+    q_scale=None,
+    causal=False,
+    window_size=(-1, -1),
+    deterministic=False,
+    dtype=torch.bfloat16,
+    version=None,
+):
+    """
+    q:              [B, Lq, Nq, C1].
+    k:              [B, Lk, Nk, C1].
+    v:              [B, Lk, Nk, C2]. Nq must be divisible by Nk.
+    q_lens:         [B].
+    k_lens:         [B].
+    dropout_p:      float. Dropout probability.
+    softmax_scale:  float. The scaling of QK^T before applying softmax.
+    causal:         bool. Whether to apply causal attention mask.
+    window_size:    (left right). If not (-1, -1), apply sliding window local attention.
+    deterministic:  bool. If True, slightly slower and uses more memory.
+    dtype:          torch.dtype. Apply when dtype of q/k/v is not float16/bfloat16.
+    """
+    half_dtypes = (torch.float16, torch.bfloat16)
+    assert dtype in half_dtypes
+    assert q.device.type == 'cuda' and q.size(-1) <= 256
+
+    # params
+    b, lq, lk, out_dtype = q.size(0), q.size(1), k.size(1), q.dtype
+
+    def half(x):
+        return x if x.dtype in half_dtypes else x.to(dtype)
+
+    # preprocess query
+    if q_lens is None:
+        q = half(q.flatten(0, 1))
+        q_lens = torch.tensor(
+            [lq] * b, dtype=torch.int32).to(
+                device=q.device, non_blocking=True)
+    else:
+        q = half(torch.cat([u[:v] for u, v in zip(q, q_lens)]))
+
+    # preprocess key, value
+    if k_lens is None:
+        k = half(k.flatten(0, 1))
+        v = half(v.flatten(0, 1))
+        k_lens = torch.tensor(
+            [lk] * b, dtype=torch.int32).to(
+                device=k.device, non_blocking=True)
+    else:
+        k = half(torch.cat([u[:v] for u, v in zip(k, k_lens)]))
+        v = half(torch.cat([u[:v] for u, v in zip(v, k_lens)]))
+
+    q = q.to(v.dtype)
+    k = k.to(v.dtype)
+
+    if q_scale is not None:
+        q = q * q_scale
+
+    if version is not None and version == 3 and not FLASH_ATTN_3_AVAILABLE:
+        warnings.warn(
+            'Flash attention 3 is not available, use flash attention 2 instead.'
+        )
+
+    # apply attention
+    if (version is None or version == 3) and FLASH_ATTN_3_AVAILABLE:
+        # Note: dropout_p, window_size are not supported in FA3 now.
+        x = flash_attn_interface.flash_attn_varlen_func(
+            q=q,
+            k=k,
+            v=v,
+            cu_seqlens_q=torch.cat([q_lens.new_zeros([1]), q_lens]).cumsum(
+                0, dtype=torch.int32).to(q.device, non_blocking=True),
+            cu_seqlens_k=torch.cat([k_lens.new_zeros([1]), k_lens]).cumsum(
+                0, dtype=torch.int32).to(q.device, non_blocking=True),
+            seqused_q=None,
+            seqused_k=None,
+            max_seqlen_q=lq,
+            max_seqlen_k=lk,
+            softmax_scale=softmax_scale,
+            causal=causal,
+            deterministic=deterministic)[0].unflatten(0, (b, lq))
+    else:
+        assert FLASH_ATTN_2_AVAILABLE
+        x = flash_attn.flash_attn_varlen_func(
+            q=q,
+            k=k,
+            v=v,
+            cu_seqlens_q=torch.cat([q_lens.new_zeros([1]), q_lens]).cumsum(
+                0, dtype=torch.int32).to(q.device, non_blocking=True),
+            cu_seqlens_k=torch.cat([k_lens.new_zeros([1]), k_lens]).cumsum(
+                0, dtype=torch.int32).to(q.device, non_blocking=True),
+            max_seqlen_q=lq,
+            max_seqlen_k=lk,
+            dropout_p=dropout_p,
+            softmax_scale=softmax_scale,
+            causal=causal,
+            window_size=window_size,
+            deterministic=deterministic).unflatten(0, (b, lq))
+
+    # output
+    return x.type(out_dtype)
+
+
+def attention(
+    q,
+    k,
+    v,
+    q_lens=None,
+    k_lens=None,
+    dropout_p=0.,
+    softmax_scale=None,
+    q_scale=None,
+    causal=False,
+    window_size=(-1, -1),
+    deterministic=False,
+    dtype=torch.bfloat16,
+    fa_version=None,
+):
+    if FLASH_ATTN_2_AVAILABLE or FLASH_ATTN_3_AVAILABLE:
+        return flash_attention(
+            q=q,
+            k=k,
+            v=v,
+            q_lens=q_lens,
+            k_lens=k_lens,
+            dropout_p=dropout_p,
+            softmax_scale=softmax_scale,
+            q_scale=q_scale,
+            causal=causal,
+            window_size=window_size,
+            deterministic=deterministic,
+            dtype=dtype,
+            version=fa_version,
+        )
+    else:
+        if q_lens is not None or k_lens is not None:
+            warnings.warn(
+                'Padding mask is disabled when using scaled_dot_product_attention. It can have a significant impact on performance.'
+            )
+        attn_mask = None
+
+        q = q.transpose(1, 2).to(dtype)
+        k = k.transpose(1, 2).to(dtype)
+        v = v.transpose(1, 2).to(dtype)
+
+        out = torch.nn.functional.scaled_dot_product_attention(
+            q, k, v, attn_mask=attn_mask, is_causal=causal, dropout_p=dropout_p)
+
+        out = out.transpose(1, 2).contiguous()
+        return out

wan/modules/model.py

@@ -0,0 +1,548 @@
+# Copyright 2024-2025 The Alibaba Wan Team Authors. All rights reserved.
+# Modified by the WhatMoves authors: routed attention through the upstream
+# PyTorch-SDPA fallback.
+import math
+
+import torch
+import torch.nn as nn
+from diffusers.configuration_utils import ConfigMixin, register_to_config
+from diffusers.models.modeling_utils import ModelMixin
+
+from .attention import attention
+
+__all__ = ['WanModel']
+
+
+def sinusoidal_embedding_1d(dim, position):
+    # preprocess
+    assert dim % 2 == 0
+    half = dim // 2
+    position = position.type(torch.float64)
+
+    # calculation
+    sinusoid = torch.outer(
+        position, torch.pow(10000, -torch.arange(half).to(position).div(half)))
+    x = torch.cat([torch.cos(sinusoid), torch.sin(sinusoid)], dim=1)
+    return x
+
+
+@torch.amp.autocast('cuda', enabled=False)
+def rope_params(max_seq_len, dim, theta=10000):
+    assert dim % 2 == 0
+    freqs = torch.outer(
+        torch.arange(max_seq_len),
+        1.0 / torch.pow(theta,
+                        torch.arange(0, dim, 2).to(torch.float64).div(dim)))
+    freqs = torch.polar(torch.ones_like(freqs), freqs)
+    return freqs
+
+
+@torch.amp.autocast('cuda', enabled=False)
+def rope_apply(x, grid_sizes, freqs):
+    n, c = x.size(2), x.size(3) // 2
+
+    # split freqs
+    freqs = freqs.split([c - 2 * (c // 3), c // 3, c // 3], dim=1)
+
+    # loop over samples
+    output = []
+    for i, (f, h, w) in enumerate(grid_sizes.tolist()):
+        seq_len = f * h * w
+
+        # precompute multipliers
+        x_i = torch.view_as_complex(x[i, :seq_len].to(torch.float64).reshape(
+            seq_len, n, -1, 2))
+        freqs_i = torch.cat([
+            freqs[0][:f].view(f, 1, 1, -1).expand(f, h, w, -1),
+            freqs[1][:h].view(1, h, 1, -1).expand(f, h, w, -1),
+            freqs[2][:w].view(1, 1, w, -1).expand(f, h, w, -1)
+        ],
+                            dim=-1).reshape(seq_len, 1, -1)
+
+        # apply rotary embedding
+        x_i = torch.view_as_real(x_i * freqs_i).flatten(2)
+        x_i = torch.cat([x_i, x[i, seq_len:]])
+
+        # append to collection
+        output.append(x_i)
+    return torch.stack(output).float()
+
+
+class WanRMSNorm(nn.Module):
+
+    def __init__(self, dim, eps=1e-5):
+        super().__init__()
+        self.dim = dim
+        self.eps = eps
+        self.weight = nn.Parameter(torch.ones(dim))
+
+    def forward(self, x):
+        r"""
+        Args:
+            x(Tensor): Shape [B, L, C]
+        """
+        return self._norm(x.float()).type_as(x) * self.weight
+
+    def _norm(self, x):
+        return x * torch.rsqrt(x.pow(2).mean(dim=-1, keepdim=True) + self.eps)
+
+
+class WanLayerNorm(nn.LayerNorm):
+
+    def __init__(self, dim, eps=1e-6, elementwise_affine=False):
+        super().__init__(dim, elementwise_affine=elementwise_affine, eps=eps)
+
+    def forward(self, x):
+        r"""
+        Args:
+            x(Tensor): Shape [B, L, C]
+        """
+        return super().forward(x.float()).type_as(x)
+
+
+class WanSelfAttention(nn.Module):
+
+    def __init__(self,
+                 dim,
+                 num_heads,
+                 window_size=(-1, -1),
+                 qk_norm=True,
+                 eps=1e-6):
+        assert dim % num_heads == 0
+        super().__init__()
+        self.dim = dim
+        self.num_heads = num_heads
+        self.head_dim = dim // num_heads
+        self.window_size = window_size
+        self.qk_norm = qk_norm
+        self.eps = eps
+
+        # layers
+        self.q = nn.Linear(dim, dim)
+        self.k = nn.Linear(dim, dim)
+        self.v = nn.Linear(dim, dim)
+        self.o = nn.Linear(dim, dim)
+        self.norm_q = WanRMSNorm(dim, eps=eps) if qk_norm else nn.Identity()
+        self.norm_k = WanRMSNorm(dim, eps=eps) if qk_norm else nn.Identity()
+
+    def forward(self, x, seq_lens, grid_sizes, freqs):
+        r"""
+        Args:
+            x(Tensor): Shape [B, L, num_heads, C / num_heads]
+            seq_lens(Tensor): Shape [B]
+            grid_sizes(Tensor): Shape [B, 3], the second dimension contains (F, H, W)
+            freqs(Tensor): Rope freqs, shape [1024, C / num_heads / 2]
+        """
+        b, s, n, d = *x.shape[:2], self.num_heads, self.head_dim
+
+        # query, key, value function
+        def qkv_fn(x):
+            q = self.norm_q(self.q(x)).view(b, s, n, d)
+            k = self.norm_k(self.k(x)).view(b, s, n, d)
+            v = self.v(x).view(b, s, n, d)
+            return q, k, v
+
+        q, k, v = qkv_fn(x)
+
+        x = attention(
+            q=rope_apply(q, grid_sizes, freqs),
+            k=rope_apply(k, grid_sizes, freqs),
+            v=v,
+            k_lens=seq_lens,
+            window_size=self.window_size)
+
+        # output
+        x = x.flatten(2)
+        x = self.o(x)
+        return x
+
+
+class WanCrossAttention(WanSelfAttention):
+
+    def forward(self, x, context, context_lens):
+        r"""
+        Args:
+            x(Tensor): Shape [B, L1, C]
+            context(Tensor): Shape [B, L2, C]
+            context_lens(Tensor): Shape [B]
+        """
+        b, n, d = x.size(0), self.num_heads, self.head_dim
+
+        # compute query, key, value
+        q = self.norm_q(self.q(x)).view(b, -1, n, d)
+        k = self.norm_k(self.k(context)).view(b, -1, n, d)
+        v = self.v(context).view(b, -1, n, d)
+
+        # compute attention
+        x = attention(q, k, v, k_lens=context_lens)
+
+        # output
+        x = x.flatten(2)
+        x = self.o(x)
+        return x
+
+
+class WanAttentionBlock(nn.Module):
+
+    def __init__(self,
+                 dim,
+                 ffn_dim,
+                 num_heads,
+                 window_size=(-1, -1),
+                 qk_norm=True,
+                 cross_attn_norm=False,
+                 eps=1e-6):
+        super().__init__()
+        self.dim = dim
+        self.ffn_dim = ffn_dim
+        self.num_heads = num_heads
+        self.window_size = window_size
+        self.qk_norm = qk_norm
+        self.cross_attn_norm = cross_attn_norm
+        self.eps = eps
+
+        # layers
+        self.norm1 = WanLayerNorm(dim, eps)
+        self.self_attn = WanSelfAttention(dim, num_heads, window_size, qk_norm,
+                                          eps)
+        self.norm3 = WanLayerNorm(
+            dim, eps,
+            elementwise_affine=True) if cross_attn_norm else nn.Identity()
+        self.cross_attn = WanCrossAttention(dim, num_heads, (-1, -1), qk_norm,
+                                            eps)
+        self.norm2 = WanLayerNorm(dim, eps)
+        self.ffn = nn.Sequential(
+            nn.Linear(dim, ffn_dim), nn.GELU(approximate='tanh'),
+            nn.Linear(ffn_dim, dim))
+
+        # modulation
+        self.modulation = nn.Parameter(torch.randn(1, 6, dim) / dim**0.5)
+
+    def forward(
+        self,
+        x,
+        e,
+        seq_lens,
+        grid_sizes,
+        freqs,
+        context,
+        context_lens,
+    ):
+        r"""
+        Args:
+            x(Tensor): Shape [B, L, C]
+            e(Tensor): Shape [B, L1, 6, C]
+            seq_lens(Tensor): Shape [B], length of each sequence in batch
+            grid_sizes(Tensor): Shape [B, 3], the second dimension contains (F, H, W)
+            freqs(Tensor): Rope freqs, shape [1024, C / num_heads / 2]
+        """
+        assert e.dtype == torch.float32
+        with torch.amp.autocast('cuda', dtype=torch.float32):
+            e = (self.modulation.unsqueeze(0) + e).chunk(6, dim=2)
+        assert e[0].dtype == torch.float32
+
+        # self-attention
+        y = self.self_attn(
+            self.norm1(x).float() * (1 + e[1].squeeze(2)) + e[0].squeeze(2),
+            seq_lens, grid_sizes, freqs)
+        with torch.amp.autocast('cuda', dtype=torch.float32):
+            x = x + y * e[2].squeeze(2)
+
+        # cross-attention & ffn function
+        def cross_attn_ffn(x, context, context_lens, e):
+            x = x + self.cross_attn(self.norm3(x), context, context_lens)
+            y = self.ffn(
+                self.norm2(x).float() * (1 + e[4].squeeze(2)) + e[3].squeeze(2))
+            with torch.amp.autocast('cuda', dtype=torch.float32):
+                x = x + y * e[5].squeeze(2)
+            return x
+
+        x = cross_attn_ffn(x, context, context_lens, e)
+        return x
+
+
+class Head(nn.Module):
+
+    def __init__(self, dim, out_dim, patch_size, eps=1e-6):
+        super().__init__()
+        self.dim = dim
+        self.out_dim = out_dim
+        self.patch_size = patch_size
+        self.eps = eps
+
+        # layers
+        out_dim = math.prod(patch_size) * out_dim
+        self.norm = WanLayerNorm(dim, eps)
+        self.head = nn.Linear(dim, out_dim)
+
+        # modulation
+        self.modulation = nn.Parameter(torch.randn(1, 2, dim) / dim**0.5)
+
+    def forward(self, x, e):
+        r"""
+        Args:
+            x(Tensor): Shape [B, L1, C]
+            e(Tensor): Shape [B, L1, C]
+        """
+        assert e.dtype == torch.float32
+        with torch.amp.autocast('cuda', dtype=torch.float32):
+            e = (self.modulation.unsqueeze(0) + e.unsqueeze(2)).chunk(2, dim=2)
+            x = (
+                self.head(
+                    self.norm(x) * (1 + e[1].squeeze(2)) + e[0].squeeze(2)))
+        return x
+
+
+class WanModel(ModelMixin, ConfigMixin):
+    r"""
+    Wan diffusion backbone supporting both text-to-video and image-to-video.
+    """
+
+    ignore_for_config = [
+        'patch_size', 'cross_attn_norm', 'qk_norm', 'text_dim', 'window_size'
+    ]
+    _no_split_modules = ['WanAttentionBlock']
+
+    @register_to_config
+    def __init__(self,
+                 model_type='t2v',
+                 patch_size=(1, 2, 2),
+                 text_len=512,
+                 in_dim=16,
+                 dim=2048,
+                 ffn_dim=8192,
+                 freq_dim=256,
+                 text_dim=4096,
+                 out_dim=16,
+                 num_heads=16,
+                 num_layers=32,
+                 window_size=(-1, -1),
+                 qk_norm=True,
+                 cross_attn_norm=True,
+                 eps=1e-6):
+        r"""
+        Initialize the diffusion model backbone.
+
+        Args:
+            model_type (`str`, *optional*, defaults to 't2v'):
+                Model variant - 't2v' (text-to-video) or 'i2v' (image-to-video)
+            patch_size (`tuple`, *optional*, defaults to (1, 2, 2)):
+                3D patch dimensions for video embedding (t_patch, h_patch, w_patch)
+            text_len (`int`, *optional*, defaults to 512):
+                Fixed length for text embeddings
+            in_dim (`int`, *optional*, defaults to 16):
+                Input video channels (C_in)
+            dim (`int`, *optional*, defaults to 2048):
+                Hidden dimension of the transformer
+            ffn_dim (`int`, *optional*, defaults to 8192):
+                Intermediate dimension in feed-forward network
+            freq_dim (`int`, *optional*, defaults to 256):
+                Dimension for sinusoidal time embeddings
+            text_dim (`int`, *optional*, defaults to 4096):
+                Input dimension for text embeddings
+            out_dim (`int`, *optional*, defaults to 16):
+                Output video channels (C_out)
+            num_heads (`int`, *optional*, defaults to 16):
+                Number of attention heads
+            num_layers (`int`, *optional*, defaults to 32):
+                Number of transformer blocks
+            window_size (`tuple`, *optional*, defaults to (-1, -1)):
+                Window size for local attention (-1 indicates global attention)
+            qk_norm (`bool`, *optional*, defaults to True):
+                Enable query/key normalization
+            cross_attn_norm (`bool`, *optional*, defaults to False):
+                Enable cross-attention normalization
+            eps (`float`, *optional*, defaults to 1e-6):
+                Epsilon value for normalization layers
+        """
+
+        super().__init__()
+
+        assert model_type in ['t2v', 'i2v', 'ti2v']
+        self.model_type = model_type
+
+        self.patch_size = patch_size
+        self.text_len = text_len
+        self.in_dim = in_dim
+        self.dim = dim
+        self.ffn_dim = ffn_dim
+        self.freq_dim = freq_dim
+        self.text_dim = text_dim
+        self.out_dim = out_dim
+        self.num_heads = num_heads
+        self.num_layers = num_layers
+        self.window_size = window_size
+        self.qk_norm = qk_norm
+        self.cross_attn_norm = cross_attn_norm
+        self.eps = eps
+
+        # embeddings
+        self.patch_embedding = nn.Conv3d(
+            in_dim, dim, kernel_size=patch_size, stride=patch_size)
+        self.text_embedding = nn.Sequential(
+            nn.Linear(text_dim, dim), nn.GELU(approximate='tanh'),
+            nn.Linear(dim, dim))
+
+        self.time_embedding = nn.Sequential(
+            nn.Linear(freq_dim, dim), nn.SiLU(), nn.Linear(dim, dim))
+        self.time_projection = nn.Sequential(nn.SiLU(), nn.Linear(dim, dim * 6))
+
+        # blocks
+        self.blocks = nn.ModuleList([
+            WanAttentionBlock(dim, ffn_dim, num_heads, window_size, qk_norm,
+                              cross_attn_norm, eps) for _ in range(num_layers)
+        ])
+
+        # head
+        self.head = Head(dim, out_dim, patch_size, eps)
+
+        # buffers (don't use register_buffer otherwise dtype will be changed in to())
+        assert (dim % num_heads) == 0 and (dim // num_heads) % 2 == 0
+        d = dim // num_heads
+        self.freqs = torch.cat([
+            rope_params(1024, d - 4 * (d // 6)),
+            rope_params(1024, 2 * (d // 6)),
+            rope_params(1024, 2 * (d // 6))
+        ],
+                               dim=1)
+
+        # initialize weights
+        self.init_weights()
+
+    def forward(
+        self,
+        x,
+        t,
+        context,
+        seq_len,
+        y=None,
+    ):
+        r"""
+        Forward pass through the diffusion model
+
+        Args:
+            x (List[Tensor]):
+                List of input video tensors, each with shape [C_in, F, H, W]
+            t (Tensor):
+                Diffusion timesteps tensor of shape [B]
+            context (List[Tensor]):
+                List of text embeddings each with shape [L, C]
+            seq_len (`int`):
+                Maximum sequence length for positional encoding
+            y (List[Tensor], *optional*):
+                Conditional video inputs for image-to-video mode, same shape as x
+
+        Returns:
+            List[Tensor]:
+                List of denoised video tensors with original input shapes [C_out, F, H / 8, W / 8]
+        """
+        if self.model_type == 'i2v':
+            assert y is not None
+        # params
+        device = self.patch_embedding.weight.device
+        if self.freqs.device != device:
+            self.freqs = self.freqs.to(device)
+
+        if y is not None:
+            x = [torch.cat([u, v], dim=0) for u, v in zip(x, y)]
+
+        # embeddings
+        x = [self.patch_embedding(u.unsqueeze(0)) for u in x]
+        grid_sizes = torch.stack(
+            [torch.tensor(u.shape[2:], dtype=torch.long) for u in x])
+        x = [u.flatten(2).transpose(1, 2) for u in x]
+        seq_lens = torch.tensor([u.size(1) for u in x], dtype=torch.long)
+        assert seq_lens.max() <= seq_len
+        x = torch.cat([
+            torch.cat([u, u.new_zeros(1, seq_len - u.size(1), u.size(2))],
+                      dim=1) for u in x
+        ])
+
+        # time embeddings
+        if t.dim() == 1:
+            t = t.expand(t.size(0), seq_len)
+        with torch.amp.autocast('cuda', dtype=torch.float32):
+            bt = t.size(0)
+            t = t.flatten()
+            e = self.time_embedding(
+                sinusoidal_embedding_1d(self.freq_dim,
+                                        t).unflatten(0, (bt, seq_len)).float())
+            e0 = self.time_projection(e).unflatten(2, (6, self.dim))
+            assert e.dtype == torch.float32 and e0.dtype == torch.float32
+
+        # context
+        context_lens = None
+        context = self.text_embedding(
+            torch.stack([
+                torch.cat(
+                    [u, u.new_zeros(self.text_len - u.size(0), u.size(1))])
+                for u in context
+            ]))
+
+        # arguments
+        kwargs = dict(
+            e=e0,
+            seq_lens=seq_lens,
+            grid_sizes=grid_sizes,
+            freqs=self.freqs,
+            context=context,
+            context_lens=context_lens)
+
+        for block in self.blocks:
+            x = block(x, **kwargs)
+
+        # head
+        x = self.head(x, e)
+
+        # unpatchify
+        x = self.unpatchify(x, grid_sizes)
+        return [u.float() for u in x]
+
+    def unpatchify(self, x, grid_sizes):
+        r"""
+        Reconstruct video tensors from patch embeddings.
+
+        Args:
+            x (List[Tensor]):
+                List of patchified features, each with shape [L, C_out * prod(patch_size)]
+            grid_sizes (Tensor):
+                Original spatial-temporal grid dimensions before patching,
+                    shape [B, 3] (3 dimensions correspond to F_patches, H_patches, W_patches)
+
+        Returns:
+            List[Tensor]:
+                Reconstructed video tensors with shape [C_out, F, H / 8, W / 8]
+        """
+
+        c = self.out_dim
+        out = []
+        for u, v in zip(x, grid_sizes.tolist()):
+            u = u[:math.prod(v)].view(*v, *self.patch_size, c)
+            u = torch.einsum('fhwpqrc->cfphqwr', u)
+            u = u.reshape(c, *[i * j for i, j in zip(v, self.patch_size)])
+            out.append(u)
+        return out
+
+    def init_weights(self):
+        r"""
+        Initialize model parameters using Xavier initialization.
+        """
+
+        # basic init
+        for m in self.modules():
+            if isinstance(m, nn.Linear):
+                nn.init.xavier_uniform_(m.weight)
+                if m.bias is not None:
+                    nn.init.zeros_(m.bias)
+
+        # init embeddings
+        nn.init.xavier_uniform_(self.patch_embedding.weight.flatten(1))
+        for m in self.text_embedding.modules():
+            if isinstance(m, nn.Linear):
+                nn.init.normal_(m.weight, std=.02)
+        for m in self.time_embedding.modules():
+            if isinstance(m, nn.Linear):
+                nn.init.normal_(m.weight, std=.02)
+
+        # init output layer
+        nn.init.zeros_(self.head.head.weight)

wan/modules/t5.py

@@ -0,0 +1,515 @@
+# Modified from transformers.models.t5.modeling_t5
+# Copyright 2024-2025 The Alibaba Wan Team Authors. All rights reserved.
+# Modified by the WhatMoves authors: load weights in restricted tensor-only mode.
+import logging
+import math
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from .tokenizers import HuggingfaceTokenizer
+
+__all__ = [
+    'T5Model',
+    'T5Encoder',
+    'T5Decoder',
+    'T5EncoderModel',
+]
+
+
+def fp16_clamp(x):
+    if x.dtype == torch.float16 and torch.isinf(x).any():
+        clamp = torch.finfo(x.dtype).max - 1000
+        x = torch.clamp(x, min=-clamp, max=clamp)
+    return x
+
+
+def init_weights(m):
+    if isinstance(m, T5LayerNorm):
+        nn.init.ones_(m.weight)
+    elif isinstance(m, T5Model):
+        nn.init.normal_(m.token_embedding.weight, std=1.0)
+    elif isinstance(m, T5FeedForward):
+        nn.init.normal_(m.gate[0].weight, std=m.dim**-0.5)
+        nn.init.normal_(m.fc1.weight, std=m.dim**-0.5)
+        nn.init.normal_(m.fc2.weight, std=m.dim_ffn**-0.5)
+    elif isinstance(m, T5Attention):
+        nn.init.normal_(m.q.weight, std=(m.dim * m.dim_attn)**-0.5)
+        nn.init.normal_(m.k.weight, std=m.dim**-0.5)
+        nn.init.normal_(m.v.weight, std=m.dim**-0.5)
+        nn.init.normal_(m.o.weight, std=(m.num_heads * m.dim_attn)**-0.5)
+    elif isinstance(m, T5RelativeEmbedding):
+        nn.init.normal_(
+            m.embedding.weight, std=(2 * m.num_buckets * m.num_heads)**-0.5)
+
+
+class GELU(nn.Module):
+
+    def forward(self, x):
+        return 0.5 * x * (1.0 + torch.tanh(
+            math.sqrt(2.0 / math.pi) * (x + 0.044715 * torch.pow(x, 3.0))))
+
+
+class T5LayerNorm(nn.Module):
+
+    def __init__(self, dim, eps=1e-6):
+        super(T5LayerNorm, self).__init__()
+        self.dim = dim
+        self.eps = eps
+        self.weight = nn.Parameter(torch.ones(dim))
+
+    def forward(self, x):
+        x = x * torch.rsqrt(x.float().pow(2).mean(dim=-1, keepdim=True) +
+                            self.eps)
+        if self.weight.dtype in [torch.float16, torch.bfloat16]:
+            x = x.type_as(self.weight)
+        return self.weight * x
+
+
+class T5Attention(nn.Module):
+
+    def __init__(self, dim, dim_attn, num_heads, dropout=0.1):
+        assert dim_attn % num_heads == 0
+        super(T5Attention, self).__init__()
+        self.dim = dim
+        self.dim_attn = dim_attn
+        self.num_heads = num_heads
+        self.head_dim = dim_attn // num_heads
+
+        # layers
+        self.q = nn.Linear(dim, dim_attn, bias=False)
+        self.k = nn.Linear(dim, dim_attn, bias=False)
+        self.v = nn.Linear(dim, dim_attn, bias=False)
+        self.o = nn.Linear(dim_attn, dim, bias=False)
+        self.dropout = nn.Dropout(dropout)
+
+    def forward(self, x, context=None, mask=None, pos_bias=None):
+        """
+        x:          [B, L1, C].
+        context:    [B, L2, C] or None.
+        mask:       [B, L2] or [B, L1, L2] or None.
+        """
+        # check inputs
+        context = x if context is None else context
+        b, n, c = x.size(0), self.num_heads, self.head_dim
+
+        # compute query, key, value
+        q = self.q(x).view(b, -1, n, c)
+        k = self.k(context).view(b, -1, n, c)
+        v = self.v(context).view(b, -1, n, c)
+
+        # attention bias
+        attn_bias = x.new_zeros(b, n, q.size(1), k.size(1))
+        if pos_bias is not None:
+            attn_bias += pos_bias
+        if mask is not None:
+            assert mask.ndim in [2, 3]
+            mask = mask.view(b, 1, 1,
+                             -1) if mask.ndim == 2 else mask.unsqueeze(1)
+            attn_bias.masked_fill_(mask == 0, torch.finfo(x.dtype).min)
+
+        # compute attention (T5 does not use scaling)
+        attn = torch.einsum('binc,bjnc->bnij', q, k) + attn_bias
+        attn = F.softmax(attn.float(), dim=-1).type_as(attn)
+        x = torch.einsum('bnij,bjnc->binc', attn, v)
+
+        # output
+        x = x.reshape(b, -1, n * c)
+        x = self.o(x)
+        x = self.dropout(x)
+        return x
+
+
+class T5FeedForward(nn.Module):
+
+    def __init__(self, dim, dim_ffn, dropout=0.1):
+        super(T5FeedForward, self).__init__()
+        self.dim = dim
+        self.dim_ffn = dim_ffn
+
+        # layers
+        self.gate = nn.Sequential(nn.Linear(dim, dim_ffn, bias=False), GELU())
+        self.fc1 = nn.Linear(dim, dim_ffn, bias=False)
+        self.fc2 = nn.Linear(dim_ffn, dim, bias=False)
+        self.dropout = nn.Dropout(dropout)
+
+    def forward(self, x):
+        x = self.fc1(x) * self.gate(x)
+        x = self.dropout(x)
+        x = self.fc2(x)
+        x = self.dropout(x)
+        return x
+
+
+class T5SelfAttention(nn.Module):
+
+    def __init__(self,
+                 dim,
+                 dim_attn,
+                 dim_ffn,
+                 num_heads,
+                 num_buckets,
+                 shared_pos=True,
+                 dropout=0.1):
+        super(T5SelfAttention, self).__init__()
+        self.dim = dim
+        self.dim_attn = dim_attn
+        self.dim_ffn = dim_ffn
+        self.num_heads = num_heads
+        self.num_buckets = num_buckets
+        self.shared_pos = shared_pos
+
+        # layers
+        self.norm1 = T5LayerNorm(dim)
+        self.attn = T5Attention(dim, dim_attn, num_heads, dropout)
+        self.norm2 = T5LayerNorm(dim)
+        self.ffn = T5FeedForward(dim, dim_ffn, dropout)
+        self.pos_embedding = None if shared_pos else T5RelativeEmbedding(
+            num_buckets, num_heads, bidirectional=True)
+
+    def forward(self, x, mask=None, pos_bias=None):
+        e = pos_bias if self.shared_pos else self.pos_embedding(
+            x.size(1), x.size(1))
+        x = fp16_clamp(x + self.attn(self.norm1(x), mask=mask, pos_bias=e))
+        x = fp16_clamp(x + self.ffn(self.norm2(x)))
+        return x
+
+
+class T5CrossAttention(nn.Module):
+
+    def __init__(self,
+                 dim,
+                 dim_attn,
+                 dim_ffn,
+                 num_heads,
+                 num_buckets,
+                 shared_pos=True,
+                 dropout=0.1):
+        super(T5CrossAttention, self).__init__()
+        self.dim = dim
+        self.dim_attn = dim_attn
+        self.dim_ffn = dim_ffn
+        self.num_heads = num_heads
+        self.num_buckets = num_buckets
+        self.shared_pos = shared_pos
+
+        # layers
+        self.norm1 = T5LayerNorm(dim)
+        self.self_attn = T5Attention(dim, dim_attn, num_heads, dropout)
+        self.norm2 = T5LayerNorm(dim)
+        self.cross_attn = T5Attention(dim, dim_attn, num_heads, dropout)
+        self.norm3 = T5LayerNorm(dim)
+        self.ffn = T5FeedForward(dim, dim_ffn, dropout)
+        self.pos_embedding = None if shared_pos else T5RelativeEmbedding(
+            num_buckets, num_heads, bidirectional=False)
+
+    def forward(self,
+                x,
+                mask=None,
+                encoder_states=None,
+                encoder_mask=None,
+                pos_bias=None):
+        e = pos_bias if self.shared_pos else self.pos_embedding(
+            x.size(1), x.size(1))
+        x = fp16_clamp(x + self.self_attn(self.norm1(x), mask=mask, pos_bias=e))
+        x = fp16_clamp(x + self.cross_attn(
+            self.norm2(x), context=encoder_states, mask=encoder_mask))
+        x = fp16_clamp(x + self.ffn(self.norm3(x)))
+        return x
+
+
+class T5RelativeEmbedding(nn.Module):
+
+    def __init__(self, num_buckets, num_heads, bidirectional, max_dist=128):
+        super(T5RelativeEmbedding, self).__init__()
+        self.num_buckets = num_buckets
+        self.num_heads = num_heads
+        self.bidirectional = bidirectional
+        self.max_dist = max_dist
+
+        # layers
+        self.embedding = nn.Embedding(num_buckets, num_heads)
+
+    def forward(self, lq, lk):
+        device = self.embedding.weight.device
+        # rel_pos = torch.arange(lk).unsqueeze(0).to(device) - \
+        #     torch.arange(lq).unsqueeze(1).to(device)
+        rel_pos = torch.arange(lk, device=device).unsqueeze(0) - \
+            torch.arange(lq, device=device).unsqueeze(1)
+        rel_pos = self._relative_position_bucket(rel_pos)
+        rel_pos_embeds = self.embedding(rel_pos)
+        rel_pos_embeds = rel_pos_embeds.permute(2, 0, 1).unsqueeze(
+            0)  # [1, N, Lq, Lk]
+        return rel_pos_embeds.contiguous()
+
+    def _relative_position_bucket(self, rel_pos):
+        # preprocess
+        if self.bidirectional:
+            num_buckets = self.num_buckets // 2
+            rel_buckets = (rel_pos > 0).long() * num_buckets
+            rel_pos = torch.abs(rel_pos)
+        else:
+            num_buckets = self.num_buckets
+            rel_buckets = 0
+            rel_pos = -torch.min(rel_pos, torch.zeros_like(rel_pos))
+
+        # embeddings for small and large positions
+        max_exact = num_buckets // 2
+        rel_pos_large = max_exact + (torch.log(rel_pos.float() / max_exact) /
+                                     math.log(self.max_dist / max_exact) *
+                                     (num_buckets - max_exact)).long()
+        rel_pos_large = torch.min(
+            rel_pos_large, torch.full_like(rel_pos_large, num_buckets - 1))
+        rel_buckets += torch.where(rel_pos < max_exact, rel_pos, rel_pos_large)
+        return rel_buckets
+
+
+class T5Encoder(nn.Module):
+
+    def __init__(self,
+                 vocab,
+                 dim,
+                 dim_attn,
+                 dim_ffn,
+                 num_heads,
+                 num_layers,
+                 num_buckets,
+                 shared_pos=True,
+                 dropout=0.1):
+        super(T5Encoder, self).__init__()
+        self.dim = dim
+        self.dim_attn = dim_attn
+        self.dim_ffn = dim_ffn
+        self.num_heads = num_heads
+        self.num_layers = num_layers
+        self.num_buckets = num_buckets
+        self.shared_pos = shared_pos
+
+        # layers
+        self.token_embedding = vocab if isinstance(vocab, nn.Embedding) \
+            else nn.Embedding(vocab, dim)
+        self.pos_embedding = T5RelativeEmbedding(
+            num_buckets, num_heads, bidirectional=True) if shared_pos else None
+        self.dropout = nn.Dropout(dropout)
+        self.blocks = nn.ModuleList([
+            T5SelfAttention(dim, dim_attn, dim_ffn, num_heads, num_buckets,
+                            shared_pos, dropout) for _ in range(num_layers)
+        ])
+        self.norm = T5LayerNorm(dim)
+
+        # initialize weights
+        self.apply(init_weights)
+
+    def forward(self, ids, mask=None):
+        x = self.token_embedding(ids)
+        x = self.dropout(x)
+        e = self.pos_embedding(x.size(1),
+                               x.size(1)) if self.shared_pos else None
+        for block in self.blocks:
+            x = block(x, mask, pos_bias=e)
+        x = self.norm(x)
+        x = self.dropout(x)
+        return x
+
+
+class T5Decoder(nn.Module):
+
+    def __init__(self,
+                 vocab,
+                 dim,
+                 dim_attn,
+                 dim_ffn,
+                 num_heads,
+                 num_layers,
+                 num_buckets,
+                 shared_pos=True,
+                 dropout=0.1):
+        super(T5Decoder, self).__init__()
+        self.dim = dim
+        self.dim_attn = dim_attn
+        self.dim_ffn = dim_ffn
+        self.num_heads = num_heads
+        self.num_layers = num_layers
+        self.num_buckets = num_buckets
+        self.shared_pos = shared_pos
+
+        # layers
+        self.token_embedding = vocab if isinstance(vocab, nn.Embedding) \
+            else nn.Embedding(vocab, dim)
+        self.pos_embedding = T5RelativeEmbedding(
+            num_buckets, num_heads, bidirectional=False) if shared_pos else None
+        self.dropout = nn.Dropout(dropout)
+        self.blocks = nn.ModuleList([
+            T5CrossAttention(dim, dim_attn, dim_ffn, num_heads, num_buckets,
+                             shared_pos, dropout) for _ in range(num_layers)
+        ])
+        self.norm = T5LayerNorm(dim)
+
+        # initialize weights
+        self.apply(init_weights)
+
+    def forward(self, ids, mask=None, encoder_states=None, encoder_mask=None):
+        b, s = ids.size()
+
+        # causal mask
+        if mask is None:
+            mask = torch.tril(torch.ones(1, s, s).to(ids.device))
+        elif mask.ndim == 2:
+            mask = torch.tril(mask.unsqueeze(1).expand(-1, s, -1))
+
+        # layers
+        x = self.token_embedding(ids)
+        x = self.dropout(x)
+        e = self.pos_embedding(x.size(1),
+                               x.size(1)) if self.shared_pos else None
+        for block in self.blocks:
+            x = block(x, mask, encoder_states, encoder_mask, pos_bias=e)
+        x = self.norm(x)
+        x = self.dropout(x)
+        return x
+
+
+class T5Model(nn.Module):
+
+    def __init__(self,
+                 vocab_size,
+                 dim,
+                 dim_attn,
+                 dim_ffn,
+                 num_heads,
+                 encoder_layers,
+                 decoder_layers,
+                 num_buckets,
+                 shared_pos=True,
+                 dropout=0.1):
+        super(T5Model, self).__init__()
+        self.vocab_size = vocab_size
+        self.dim = dim
+        self.dim_attn = dim_attn
+        self.dim_ffn = dim_ffn
+        self.num_heads = num_heads
+        self.encoder_layers = encoder_layers
+        self.decoder_layers = decoder_layers
+        self.num_buckets = num_buckets
+
+        # layers
+        self.token_embedding = nn.Embedding(vocab_size, dim)
+        self.encoder = T5Encoder(self.token_embedding, dim, dim_attn, dim_ffn,
+                                 num_heads, encoder_layers, num_buckets,
+                                 shared_pos, dropout)
+        self.decoder = T5Decoder(self.token_embedding, dim, dim_attn, dim_ffn,
+                                 num_heads, decoder_layers, num_buckets,
+                                 shared_pos, dropout)
+        self.head = nn.Linear(dim, vocab_size, bias=False)
+
+        # initialize weights
+        self.apply(init_weights)
+
+    def forward(self, encoder_ids, encoder_mask, decoder_ids, decoder_mask):
+        x = self.encoder(encoder_ids, encoder_mask)
+        x = self.decoder(decoder_ids, decoder_mask, x, encoder_mask)
+        x = self.head(x)
+        return x
+
+
+def _t5(name,
+        encoder_only=False,
+        decoder_only=False,
+        return_tokenizer=False,
+        tokenizer_kwargs={},
+        dtype=torch.float32,
+        device='cpu',
+        **kwargs):
+    # sanity check
+    assert not (encoder_only and decoder_only)
+
+    # params
+    if encoder_only:
+        model_cls = T5Encoder
+        kwargs['vocab'] = kwargs.pop('vocab_size')
+        kwargs['num_layers'] = kwargs.pop('encoder_layers')
+        _ = kwargs.pop('decoder_layers')
+    elif decoder_only:
+        model_cls = T5Decoder
+        kwargs['vocab'] = kwargs.pop('vocab_size')
+        kwargs['num_layers'] = kwargs.pop('decoder_layers')
+        _ = kwargs.pop('encoder_layers')
+    else:
+        model_cls = T5Model
+
+    # init model
+    with torch.device(device):
+        model = model_cls(**kwargs)
+
+    # set device
+    model = model.to(dtype=dtype, device=device)
+
+    # init tokenizer
+    if return_tokenizer:
+        from .tokenizers import HuggingfaceTokenizer
+        tokenizer = HuggingfaceTokenizer(f'google/{name}', **tokenizer_kwargs)
+        return model, tokenizer
+    else:
+        return model
+
+
+def umt5_xxl(**kwargs):
+    cfg = dict(
+        vocab_size=256384,
+        dim=4096,
+        dim_attn=4096,
+        dim_ffn=10240,
+        num_heads=64,
+        encoder_layers=24,
+        decoder_layers=24,
+        num_buckets=32,
+        shared_pos=False,
+        dropout=0.1)
+    cfg.update(**kwargs)
+    return _t5('umt5-xxl', **cfg)
+
+
+class T5EncoderModel:
+
+    def __init__(
+        self,
+        text_len,
+        dtype=torch.bfloat16,
+        device=torch.cuda.current_device(),
+        checkpoint_path=None,
+        tokenizer_path=None,
+        shard_fn=None,
+    ):
+        self.text_len = text_len
+        self.dtype = dtype
+        self.device = device
+        self.checkpoint_path = checkpoint_path
+        self.tokenizer_path = tokenizer_path
+
+        # init model
+        model = umt5_xxl(
+            encoder_only=True,
+            return_tokenizer=False,
+            dtype=dtype,
+            device=device).eval().requires_grad_(False)
+        logging.info(f'loading {checkpoint_path}')
+        model.load_state_dict(
+            torch.load(checkpoint_path, map_location='cpu', weights_only=True))
+        self.model = model
+        if shard_fn is not None:
+            self.model = shard_fn(self.model, sync_module_states=False)
+        else:
+            self.model.to(self.device)
+        # init tokenizer
+        self.tokenizer = HuggingfaceTokenizer(
+            name=tokenizer_path, seq_len=text_len, clean='whitespace')
+
+    def __call__(self, texts, device):
+        ids, mask = self.tokenizer(
+            texts, return_mask=True, add_special_tokens=True)
+        ids = ids.to(device)
+        mask = mask.to(device)
+        seq_lens = mask.gt(0).sum(dim=1).long()
+        context = self.model(ids, mask)
+        return [u[:v] for u, v in zip(context, seq_lens)]