How I Built an Automated Music Video Pipeline with Remotion and Dockerized YouTube CDP

Most AI music workflows hit a brick wall right after the track is generated. You have an MP3 file and maybe a text block of lyrics. If you want to turn that into a finished YouTube video, you typically enter a tedious manual loop: opening a video editor, cutting background footage, adjusting rolling captions line by line, setting keyframes, exporting a multi-gigabyte render, waiting for YouTube Studio to upload, typing descriptions, and pasting subtitle tracks.

Doing that once takes an afternoon. Doing that across dozens of tracks is unsustainable.

I wanted a single automated pipeline: give it an audio file and lyrics, and walk away. A few minutes later, a 1080p music video is rendered with dynamic audio-reactive visuals, synchronized three-line rolling lyrics, smart scene cut alternation, and automatically published to YouTube with 8-language external subtitles.

Here is the exact architecture, the failure points I had to solve, and the code running this in production.

Live Production Proof: The full benchmark artifact rendered and published by this exact pipeline is live on YouTube: 《糖画小仙》 (Sugar Art Fairy) 1080p MV, featuring 8-language CC subtitle tracks (EN, JA, KO, ES, FR, DE, MS, ZH-Hant).

TL;DR — Four decoupled stages: Whisper word-level alignment → Remotion 1080p render → Docker Archive staging → CDP upload to YouTube Studio. Each fails on its own, so each can be debugged on its own; the two that fought back hardest have their own write-ups: staging large files into the browser container and killing render jitter.

1. End-to-End Execution & Timing Breakdown

To be completely honest about performance rather than promising a vague “three minutes”: here is the measured benchmark run for our 4-minute 21-second (261s) 1080p 30fps song:

The actual rendering of a 4-minute video takes 218.3 seconds (under 3.7 minutes) on our workstation, and the upload/subtitle publication pipeline finishes in another minute. Once the storyboard layout is confirmed, the entire sequence runs with zero human intervention.

2. System Architecture

The pipeline consists of four decoupled layers:

[Audio MP3 + Lyrics]
        │
        ▼
1. Acoustic Onset Alignment (Whisper word-level timestamping -> LRC + SRT)
        │
        ▼
2. Dynamic React Video Synthesis (Remotion 1080p 30fps)
   ├─ Dynamic Circular Visualizer (320px base -> 380px bass expansion + 72 radial bars)
   ├─ 3-Line Smooth Scrolling Lyrics (Negative space side-tracking)
   └─ Pure-Opacity Image Slideshow (no per-frame scaling · 1.8s Bézier crossfade)
        │
        ▼
3. Headless Docker Staging (Portainer Docker Archive API -> Container /config/Downloads)
        │
        ▼
4. Autonomous YouTube Publication (Remote Chrome DevTools Protocol / Playwright)
   ├─ Attach video & custom thumbnail
   ├─ Fill SEO metadata, biography, & service contact links
   ├─ Step through upload wizard & set visibility to Public
   └─ Upload & publish 8 multilingual subtitle tracks (EN, JA, KO, ES, FR, DE, MS, ZH-TW)

3. Solving Acoustic Lyric Alignment

AI music generators stretch vowels, add melodic vibrato, and change tempo dynamically. Standard text-to-speech alignment or plain transcription drops entire verses when instruments overpower vocals.

Instead of blind speech recognition, I use forced acoustic onset alignment:

1. Take the ground-truth lyrics written beforehand.
2. Run faster-whisper on the vocal track with word-level timestamps (word_timestamps=True).
3. For each lyric line, find the closest acoustic voice activity onset for the starting character and lock the timestamp.

That yields a .lrc and .srt pair aligned to the rendered frame: at 30 fps one frame is 33.3 ms, so a frame is the resolution that actually matters here. When the singer lands the first consonant of a phrase, the highlight switches on that frame. Word-level timestamps are not microsecond-accurate, and the pipeline does not claim to be — it lines each line up with the acoustic onset and lets the frame grid decide.

4. Negative-Space Side Alternation in Remotion

In music videos with illustrated backgrounds, a central problem is character obstruction. If a slide shows a character standing on the right, centering the lyrics or locking them to the right covers the artwork.

To solve this deterministically, the pipeline extracts a sides.json layout manifest:

["left", "right", "left", "right", "left", "right", "left", "right"]

In Remotion (MusicVideo.tsx), the visualizer ring and lyric stack share a single animated X coordinate:

const getSideX = (idx: number) => {
  const side = sides[idx] || (idx % 2 === 0 ? 'left' : 'right');
  return side === 'left' ? 540 : 1380;
};

During the last 1.8 seconds of a scene, the HUD coordinates glide across the canvas using a smooth cubic Bézier curve (Easing.bezier(0.42, 0, 0.58, 1)). The lyrics drift gracefully into the empty negative space before the new illustration fades in, guaranteeing zero overlap with focal characters.

5. The Remote YouTube Upload Bridge

Rendering happens on a Windows workstation, while my authenticated YouTube Studio session runs inside an isolated Docker container (chrome2) on a Synology NAS.

To upload without browser authentication prompts:

1. Docker Archive Staging: The ~129 MB MP4 video and cover image package is tar-streamed directly into the container filesystem via Portainer’s REST API (PUT /api/endpoints/2/docker/containers/{id}/archive?path=/config/Downloads). It bypasses CDP byte limits and writes in under 2 seconds. The full pattern — code, tar caveats, and what it does not solve — is in Uploading 100MB+ Videos to YouTube via CDP.
2. CDP File Injection: Using Playwright over CDP (ws://192.168.1.1:9242), the upload file input receives the local container path:

cdp.send("DOM.setFileInputFiles", {
    "files": ["/config/Downloads/国风电子 糖画小仙.mp4"],
    "nodeId": video_node["nodeId"],
})

3. Multilingual Subtitle Dispatch: An automated worker splits the calibrated SRT into an array of lines, translates them into 8 languages (English, Japanese, Korean, Spanish, French, German, Malay, Traditional Chinese) via our local OmniRoute LLM gateway, and uploads each .srt file through the Studio Translations interface.

6. What I Would Do Differently

• Decouple rendering from upload: Early revisions tried to pipe ffmpeg buffers directly into a browser tab. Encoding hiccups would freeze the browser socket. Saving the MP4 locally first and staging via Docker Archive completely isolated the two failure domains.
• Strict layout verification: Always generate a 4x2 montage thumbnail of all storyboard scenes before starting full rendering. Visually asserting the negative space of each slide prevents re-rendering four-minute videos due to a single misplaced side flag.

Want this for your business?

I build automated video generation pipelines, custom music production workflows, and self-hosted infrastructure for creators, educators, and brands.

If you want to automate your media workflow or need custom full-stack tools:

• WhatsApp: Chat directly on WhatsApp
• Email: [email protected]
• Website: hoelee.com

Lee Teong Hoe · Mr Hoelee

Full-stack developer & DevOps engineer based in Malaysia. I run my own production infrastructure and build web systems for small businesses — this blog is the living record.

LinkedIn GitHub Gitea [email protected]

Hiring? Profile & stack · Malaysian SME? Website design & development ↗ · All posts