Why I Built a Video Knowledge Base: Turning Saved Short Videos into Searchable Notes
Like many developers and lifelong learners, I found myself accumulating hundreds of bookmarks across platforms like Bilibili, YouTube, and Douyin. A 60-second video explaining a Transformer concept here, a 2-minute architectural teardown there.
The bookmark list kept growing, but when I actually needed that one specific explanation six months later, it was impossible to retrieve.
Saving videos into platform favorites is a dead end. This post explains why I decided to solve this problem by building a dedicated learning portal, how I designed the architecture around local compute, and the lessons learned turning fleeting video content into a permanent, searchable knowledge asset.
1. Why Video Bookmarks Fail
Short-form video is currently the fastest medium for breaking information asymmetry. Creators pack high-density insights into 1 to 3 minutes. However, as a long-term knowledge repository, video suffers from three fatal flaws:
- Unsearchable inside the content: Platform search only indexes video titles and uploader tags. The actual golden nugget spoken at minute 01:15 is completely invisible to search engines.
- High review friction: Reading a 500-word summary takes 30 seconds; watching a video requires 2 to 3 minutes of passive listening, often padded with intro hooks and subscriber calls.
- Fragile shelf life: Links rot, accounts get banned, and videos get taken down without warning.
To turn passive consumption into actionable learning, I set a clear requirement: every video I save must be transcribed, summarized into structured bullet points and mindmaps, and made searchable down to the exact second.
2. Architecture & Design Principles
When designing the system, I established four non-negotiable architectural principles based on the self-hosted infrastructure I manage:
[ Local Input / Inbox ]
│
▼
[ Local GPU Pipeline ] ────► Silero-VAD + Faster-Whisper (Zero cloud API fees)
│
├──────────────────► LLM 6-D Synthesis (TL;DR, Mindmap, Timestamps)
│
▼
[ Decoupled Data Layer ]
├── src/content/videos/{slug}.md (Frontmatter + Structured Insights)
└── src/data/transcripts/{slug}.json (Sub-Second Timestamp Cues [start, end, text])
│
▼
[ Astro 5 Static Portal ] ──► Pagefind Client-Side WASM Search (<20ms latency)
│
▼
[ Dual-Track Video Sync ] ──► Official iFrame Embeds or NVENC H.264 on Cloudflare R2Decouple Summaries from Transcripts
The biggest architectural trap in early prototypes was concatenating timestamped transcripts directly inside the Markdown content body. This caused Markdown files to balloon to thousands of lines, cluttered formatting, and made re-running LLM summaries destructive.
The fix: keep src/content/videos/{slug}.md strictly for high-level synthesis and Frontmatter, while storing sub-second timestamp cues in src/data/transcripts/{slug}.json.
Zero Cloud STT Fees
Rather than paying recurring per-minute charges to cloud transcription APIs, the pipeline runs locally on an NVIDIA RTX 3060 (12GB VRAM). Using faster-whisper and Silero-VAD, speech is automatically transcribed into clean 3-to-8 second semantic sentences with sub-second timestamps.
Never Stream Media Directly from Origin Storage Nodes
Serving video streaming directly from an on-premise storage node to public web visitors saturates origin upload bandwidth and risks ISP restrictions. Public playback uses a dual-track strategy: official iframe embeds (Bilibili/YouTube) whenever possible, or lightweight NVENC 720p H.264 streams hosted on Cloudflare R2.
Sub-20ms Search with Pagefind
Instead of maintaining an always-on search database like Meilisearch or Elasticsearch, the front portal compiles to a pure static Astro site indexed by Pagefind. Search queries run in-browser via WebAssembly, delivering instantaneous Chinese keyword lookups with zero server memory footprint.
3. What Changed in My Daily Learning Loop
Replacing passive “saving for later” with an automated ingestion pipeline fundamentally changed how I retain information:
- From Hoarding to Searching: Instead of scrolling through endless favorited lists, I hit
Cmd+Kin the portal, type a keyword likeRAGorAgent, and instantly see matching quotes highlighted in the transcript. - Click-to-Seek Precision: Clicking on any key point jumps the video player directly to that second, bypassing creator filler and replaying only the core explanation.
- Permanent Ownership: The knowledge lives in plain text and JSON in a Git repository, completely independent of third-party platforms.
4. Work With Me
Building automated knowledge ingestion pipelines, self-hosted AI workflows, and high-performance static portals is part of what I do.
If your organization has internal training videos, product recordings, or course archives that you need converted into a fast, searchable internal knowledge hub:
- WhatsApp: Chat directly on WhatsApp
- Email: [email protected]
- Website: hoelee.com
