Clipsmith: AI-Assisted Twitch Clip Pipeline

Problem

Turning long streams or recordings into short-form vertical clips is repetitive, slow, and hard to prioritize manually.

Solution

Built a local media automation pipeline that downloads VODs, transcribes Spanish audio, ranks candidate moments by chat activity, uses an LLM to select highlights, and cuts 9:16 MP4 clips with optional captions and reframing.

Deliverables

• Local CLI
• Twitch VOD ingestion
• Spanish transcription
• Chat replay parsing
• Candidate scoring
• LLM-assisted clip selection
• FFmpeg clip generation
• Optional captions
• Optional webcam/gameplay stacked layout
• FastAPI/dashboard path

Why it matters

• Turns a 4-hour VOD into 5 ready-to-post vertical clips without a video editor
• Chat-replay scoring prioritizes moments the audience already validated — not guesswork
• LLM selection adds context-awareness that pure engagement metrics miss
• Stacked 9:16 layout works directly for TikTok, Reels, and YouTube Shorts