0:00 Imagine your AI generates a flawless three minute tutorial video. It looks incredible. Right, like totally polished. Exactly, but then you realize it's teaching thousands of viewers to click a button that doesn't actually exist. Oh man, that 0:13 is the worst. It really is. Welcome to the deep dive. Today we are looking at a highly practical guide to building an AI agentic workflow. Yeah, specifically using Claude code and Remotion. 0:27 Right. We're going to explore how loop agents solve AI's biggest blind spot, which is catching its own mistakes. The goal is so you spend less time fixing hallucinated video edits. Because a single AI agent is great at creating, but it's 0:41 fundamentally terrible at self -editing. So today we break down a six -step framework to actually fix that. Let's start with that single agent trap. You know, you give one AI everything. Script, screen recording, voiceover, editing. It sounds 0:54 so efficient. It does. Just one prompt to do it all. It does. But it leads to missing steps or captions just drifting totally out of sync. 1:02 And honestly, I'll make a vulnerable admission here. Go for it. I still wrestle with AI just ignoring my instructions when it checks its own work. Like, I tell it to fix something and it just says, yeah. Looks perfect. Yeah, you're definitely not alone there. Yeah. There's this 1:16 fascinating research from Huang and colleagues at ICLR 2024. They found that language models severely struggled to correct their own reasoning without external feedback. So they basically suffer from confirmation bias. Exactly. So the 1:31 solution is loop agents. You split the job into a maker and a checker. It's an evaluator optimizer workflow. Let's define that real quick. One AI creates the work, another critiques it to improve it. Perfect. Yeah, the maker builds it, the checker 1:45 tears it down. But I gotta push back a little here. Why not just start a fresh chat? You know, give a single agent completely fresh context to review its own draft. Well, because a separate checker is naturally skeptical. It didn't burn 1:58 tokens creating that flawed work, so it has zero attachment to it. So does the maker actually know it's being evaluated, or does it just see the failed prompt? It just receives specific errors from the checker to fix and resubmit. 2:11 So it's just getting direct feedback to iterate on, not judging itself. Right. But introducing this loop creates a dangerous new problem. Infinite loops. Yes. If they disagree, they will waste tokens fixing the same mistake forever without 2:25 rules. That sounds incredibly expensive. So we need guardrails. Exactly. Four rules. Rule one is written checklists. You have to separate code -based checks from AI -based checks. Oh, like checking the video length versus, uh... Checking 2:40 if the narration makes sense. Spot on. Code checks are for objective things like audio loudness, and they are way cheaper. AI is just for semantic understanding. Makes sense. What's rule two? 2:52 Stop after three failed rounds. You just force a hard stop and flag it in a review -log .md file for a human. Oh, that's smart. That stops the token bleed. Rule three is you save absolutely everything in files. Don't rely on the chat history 3:07 at all. I like to think of it like stacking Lego blocks of data, you know, each agent just picks up the exact blocks they need. That's a great analogy. And the final rule, rule four, is two mandatory human checkpoints. Okay, where do those 3:21 happen? One right after the outline is generated, and one after the final render. Let me ask you this, why is saving to files better than just keeping the chat window open? It lets you rerun a failed stage without rebuilding the whole video 3:35 from scratch. You save money and context by isolating errors to specific files. Exactly. It's much cleaner. So we have the rules. Now we need the toolkit to actually execute this factory line. 3:47 The guide uses five core tools to make this happen. 3:50 Right. Claude Code is the orchestrator. Then you have Playwright for screen recording. Yep. 3:55 And 11 Labs handles the incredibly natural voiceovers. 3:58 WhisperX or Scribe does the precise timestamps, right? Mm -hmm. And finally, Remotion or FFmpeg for the actual video assembly. That's quite the stack. And the folder setup is pretty complex too, right? It is. You have 11 specialized subagents 4:14 plus one orchestrator. Wow, 11. Yeah, and they all share a very specific folder structure. Brief, outline, script, shots .json, capture, audio, and render. What really stood out to me in the guide was the pass or fail structure of the subagent 4:30 prompts. Oh, it's so rigid. They don't just chat. 4:34 They output a binary pass or fail and log the exact evidence. Are these 11 agents running simultaneously or waiting in line? The orchestrator coordinates them sequentially, passing files from one stage to the next. The orchestrator acts as a project 4:48 manager, triggering each agent in order. Right, so we're back. We've got the tools set up. Let's see the six -step agentic workflow in action. 4:57 Yeah, the guide builds a three -minute Google Forms tutorial. Step one is the outline. So it generates a brief to an outline .md file, and this is where human checkpoint one happens, right? 5:08 Exactly. You have to approve it here to prevent the AI from recording bad footage later. Smart. 5:13 What's step two? Script and Shots. It converts the script to a natural pace of 150 spoken words per minute. And it creates that shots .json file to map sentences to exact UI targets. Right. 5:25 Which feeds into step three, screen capture via Playwright. Playwright automatically clicks around the web browser. Which takes us to step four, the voiceover with 11 laps. Yeah, and the checker subagent verifies the word level timestamps here. 5:38 It demands 98 % word agreement. Which is incredible precision. Then step five is the assembly phase using remotion. This is where it all comes together programmatically. Whoa. Imagine an AI pulling exact timestamps to automatically sync visual 5:52 zoom -ins with spoken words. It's doing all the timeline dragging for you. It feels like magic when it works. Finally, step six is the final review. So a fresh checker looks at pacing, sync, and captions. Yep, and that's your second human 6:05 checkpoint. You watch the final video. Quick question on step three. What happens when a website has a pop -up cookie banner during the screen capture? Playwright might fail the shot, and the checker logs it so a human can manually record 6:18 it. Automated capture is fragile. For sure. Manual recording is still a valid backup. For sure. 6:23 But looking at all of this, we have to ask... 6:26 Is this massive setup actually worth the time and token cost compared to just a single prompt? 6:32 Well, comparing a single agent to loop agents, the loop agents definitely take more time and tokens upfront. Yeah, significantly more. But the real benefit is that it drastically reduces human refute time at the very end. You're not fixing messy, hallucinated edits. Exactly. It 6:46 catches the mistakes early. Also, we should explain the difference between loop agents and skills. 6:51 Oh, right. Skills give specialized instructions, like the video use skill. Yeah, and you must name the skill explicitly in the prompt or Claude might just guess and use standard bash commands. 7:01 Which ruins the workflow. So what are the limitations of this whole system? Playwright breaks pretty easily on logins or two -factor authentication. 7:11 And small UI updates can ruin your old code selectors. 7:15 True. Though with a file -based system, you let the AI just rerun step three to fix it. But yeah, for a quick 60 -second video, this is way too expensive and complex. It's definitely best for repeatable, high -volume tutorials. Can the AI 7:32 checker still approve a bad video, though? Yes, AI can miss awkward pacing or confusing explanations, which is why human checkpoints remain. AI checkers aren't perfect. Always. Human judgment is still the final filter. Always. Let's recap the big 7:47 idea here. Letting one AI do everything creates quiet disasters. It really does. Just confidently makes things up. Splitting AI into makers and checkers governed by strict checklists and file saves completely shifts our role. We go from 8:02 manual pixel pushers to true directors. We manage the system, not the timeline. I want to leave you with a final thought today. If we rely on AI checkers to approve AI makers, what happens when the checkers start developing their own 8:15 blind spots based on the biases in the checklists we write for them? Who checks the checkers? Oof. 8:22 That's a deep one. But for now maybe just try automating the script and outline phase of your next project first before diving into full video assembly Thanks for hanging out with us. See you next time outro music
Community discussion