srt-whiteboard-animation

# SRT Whiteboard Animation Skill A Skill that converts SRT subtitles into whiteboard hand-drawn videos rendered in narrative order. It combines **zoned mask orchestration** with **streaming stroke rendering**: each element appears sequentially along with the subtitles, the pen tip continuously lays down ink within the region, then gradually adds color, and finally exports an MP4. Suitable for turning knowledge explanations, story voiceovers, course subtitles, or short video scripts into hand-drawn animations on a warm beige paper background. ## Example Effect **Scene: Monkey Mountain Banana Grab** — Following the narrative order of the subtitles, it sequentially draws the rockery and the little monkey, the big monkey grabbing the banana, and the watching children.  Original line art: [View PNG](examples/scene-01-monkey-mountain.png). ## Core Capabilities - Parse SRT subtitles and split scenes according to the recommended 25–35 second duration - First output storyboard and illustration strategy, ensuring each scene expresses only one core meaning - Establish a semantic drawing order for elements based on subtitle events rather than screen coordinates - Use `annotation.json` to manage regions, timing, subtitle associations, and overlap protection zones - Each region uses continuous streaming strokes: first `ink` lays down the line art, then `color` adds color - Support browser preview console for adjusting regions, order, timing, and subtitle associations - Support scene-by-scene rendering and multi-scene merging, outputting a complete MP4 ## Working Method The key to this Skill is "subtitle-driven, step-by-step confirmation." Wait for confirmation after each step to avoid wasting rendering costs when the storyboard, line art, or annotations are not yet finalized: 1. Parse SRT and output storyboard and illustration strategy. 2. After confirmation, generate line art in a unified style. 3. After confirming the line art, create annotations based on the subtitles and original image, and load them into the preview console. 4. After confirming the annotations, generate zone and direction check images. 5. Adjust regions, narrative order, timing, and subtitle associations in the preview console and save. 6. After confirming the final annotations, render MP4 scene by scene. 7. For multi-scene projects, merge after confirming each scene's finished video. ## Visual Specifications - Warm beige paper background: recommended `#F5EBD7` - Dark gray sketch lines, with red, orange, and blue used only as small conceptual accents - Minimalist hand-drawn style, clean background, and ample whitespace - No scene text, labels, photographic feel, 3D effects, or complex textures ## Installation and Environment The Skill comes with an independent Python virtual environment preparation script. On first run, execute: ```bash python scripts/prepare_env.py --check python scripts/prepare_env.py ``` After success, the first command will output `ENV_PY=<path>`; use that interpreter for subsequent rendering to ensure dependency isolation. ## Project Asset Structure ```text assets/whiteboard/<project-name>/ ├── scene-01-<name>.png ├── scene-01-<name>.annotation.json ├── scene-01-<name>-whiteboard.mp4 └── scene-01-<name>-preview.mp4 ``` Images and annotations must have the same name, for example `scene-01-demo.png` corresponds to `scene-01-demo.annotation.json`. ## Annotation Format Each element uses integer pixel coordinates from the original image, and associates with subtitle events through `sequence`, `subtitle`, and `narrativeRole`. Regions should be ordered as "scene setup → key characters/objects → action or change → reaction/result." ```json { "sceneId": "scene-01", "canvas": { "width": 1672, "height": 941 }, "storyBasis": "The little monkey sits on the monkey mountain holding a banana, the big monkey grabs the banana, and the children watch nearby.", "sceneDurationMs": 9000, "elements": [ { "id": "rockery", "label": "Monkey mountain scene", "sequence": 1, "narrativeRole": "Scene setup for the story", "subtitle": "The little monkey sits on top of the monkey mountain, holding a banana.", "type": "structure", "region": { "x": 20, "y": 120, "width": 540, "height": 780 }, "reveal": { "direction": "top_to_bottom", "startMs": 300, "durationMs": 2600, "maskPaddingPx": 22, "protectedRegions": [] }, "handPath": { "start": [290, 130], "end": [290, 890], "easing": "easeInOut" } } ] } ``` `direction` and `handPath` are used for rectangular proxies in the preview console; the actual strokes in the final video are automatically generated by the streaming renderer. For objects that occlude each other, mark the regions that need delayed display in the earlier element's `protectedRegions` to prevent subsequent content from appearing prematurely. ## Common Commands Parse subtitles and generate suggested storyboard: ```bash python scripts/parse_srt.py <subtitle.srt> --target-sec 30 --min-sec 25 --max-sec 35 ``` Generate region check image: ```bash python scripts/render_annotation_preview.py <image-path> <annotation-path> <preview-image-output-path> ``` Open `assets/preview.html`, use "Open Folder" to load the scene directory, and you can edit regions, order, timing, and subtitle associations. Render a single scene: ```bash <ENV_PY> scripts/render_stream_whiteboard.py <image-path> <annotation-path> <output.mp4> assets/drawing-hand.png \ --ink-path grid --color-fill contour-wipe ``` Merge multiple scenes: ```bash <ENV_PY> scripts/merge_scenes.py --inputs scene1.mp4 scene2.mp4 scene3.mp4 --output final.mp4 ``` ## Quality Checks - The first frame is a clean warm beige paper background with no lines appearing prematurely - `canvas` matches the original image dimensions, and all regions use integer pixel coordinates within the canvas - `sequence` and `startMs` are consistent with the narrative order of the subtitles - In mid-segment frames, unstarted regions and protected zones do not appear prematurely - The pen tip stays close to the current streaming strokes; when line art is clear, you can choose `--ink-path skeleton` - After each scene ends, hold the complete frame for at least 0.5 seconds; multi-scene merge order matches the subtitle storyboard ## Repository Contents ```text srt-whiteboard-animation/ ├── SKILL.md # Complete workflow and constraints ├── assets/ │ ├── drawing-hand.png # Hand asset │ ├── preview.html # Local editing preview console ├── examples/ # README example assets ├── scripts/ │ ├── parse_srt.py # Subtitle parsing and storyboard suggestions │ ├── render_annotation_preview.py # Annotation check image │ ├── render_stream_whiteboard.py # Streaming stroke MP4 renderer │ ├── merge_scenes.py # Multi-scene merging │ └── prepare_env.py # Dependency environment preparation └── agents/openai.yaml # Codex metadata ``` ## Contributing Issues and Pull Requests are welcome. Any changes involving drawing logic should use real subtitles, annotations, and finished videos to check mask protection, timing, and final frames. ## License This project is open source under the MIT License. See [LICENSE](LICENSE) for details. ## About the Author An old guy who loves keeping fish / AI Builder / Building a one-person company with an AI team. Douyin, Bilibili, WeChat Official Account: 江哥是老登啊