How to fully automate video editing with Claude Code

What if you could drop a messy talking-head clip into a folder, write a single prompt, and get back a fully edited video with clean cuts, motion graphics, captions, and background music—without ever opening Premiere Pro or touching a timeline?

That’s exactly what you can do by combining Claude Code with a few powerful open-source tools. In this guide, you’ll see how to set up a simple pipeline that lets Claude handle 80% of the editing work, while you stay in control of the creative decisions that matter.

How the Claude video editing pipeline works

The automated workflow is built around one core idea: Claude acts as an AI editor that understands your footage, then orchestrates real professional tools behind the scenes.

At a high level, the pipeline looks like this:

Raw footage in → Claude transcribes and rough-cuts → motion graphics are generated → captions are added → music is laid in → final video is exported.

Instead of manually scrubbing a timeline, you talk to Claude in natural language. It reads your transcript, plans the edits, and calls the right tools to do the heavy lifting.

The key tools powering this setup

This workflow relies on a small stack of tools that Claude controls for you. You don’t have to wire them together manually—Claude Code can install and configure most of it from a single prompt.

VideoUse: the editing brain

VideoUse is the main skill that runs the edit. It’s free, open source, and acts as the brain of the whole operation.

Here’s what VideoUse does:

• Reads the transcript of your recording
• Detects bad takes, long silences, filler words, and repeated sentences
• Plans and applies the actual cuts using FFmpeg
• Hands off motion graphics work to Hyperframes when needed

You never have to interact with VideoUse directly in a terminal. Claude uses it as a skill inside the Code tab and manages the technical details for you.

Hyperframes: motion graphics as code

Hyperframes is the rendering engine that builds your motion graphics. VideoUse calls Hyperframes whenever it needs to create visuals like:

• Text cards and lower thirds
• Animated callouts
• Data or stat visuals
• Logo and brand placements

Hyperframes treats each graphic as code. Claude writes and tweaks that code for you, then renders the final animations. From your perspective, you just describe the visuals you want in plain language.

FFmpeg: stitching and rendering

FFmpeg is the industry-standard command-line tool that actually stitches clips together and renders the final video file.

In this setup, FFmpeg is used to:

• Cut and join the clean segments of your recording
• Mix in background music at the right level
• Export the final video to a standard format

Claude installs FFmpeg for you as part of the setup and calls it automatically during each edit.

11Labs Scribe: precise word-level transcripts

Most AI editors rely on rough transcripts and then guess where to cut. That’s why you often hear clipped words, chopped breaths, or awkward timing.

11Labs Scribe solves this by giving Claude word-level timestamps. For every word you say, Scribe knows exactly when it starts and ends—down to the millisecond.

This lets Claude:

• Remove filler words without cutting into nearby words
• Trim silences and false starts cleanly
• Collapse multiple takes of the same line into one smooth sentence
• Generate perfectly synced captions later on

Setting everything up inside Claude Code

You can set up the entire stack from inside the Claude app using the Code tab. There’s no need to manually clone repos or run long terminal commands if you don’t want to.

1. Install the VideoUse skill

1. Copy the GitHub link for the VideoUse repository.

2. Open the Claude Code tab.

3. Paste the repo link into the chat and tell Claude something like: “Install this repo, set it up as a skill, and wire it up with FFmpeg.”

Claude will then:

• Read the repository files
• Install dependencies
• Set up FFmpeg
• Register VideoUse as a skill it can call during your projects

When it’s done, Claude will confirm that the skill is ready and wait for you to provide footage.

2. Connect your 11Labs Scribe API key

Next, you need to connect 11Labs Scribe so Claude can generate precise transcripts.

• Go to your 11Labs account and copy your API key.
• The first time Claude runs an edit that needs transcription, it will automatically ask you to paste your key.
• Drop it into the prompt when requested—no hunting through config files.

Once that’s done, your setup is essentially complete. Claude now has everything it needs to run fully automated edits.

From raw clip to clean rough cut

With the tools installed, you can start editing with a single natural-language prompt.

1. Drop your footage into the project folder

Start with a simple talking-head recording: one take, a few minutes long, with all the usual real-world mess—ums, likes, dead air, repeated sentences, and moments where you lose your train of thought.

Save that file into your Claude project folder so the Code tab can access it.

2. Ask Claude to create a rough cut

In the Code tab, you might type something like:

“Start a new project for the [filename] clip and do a rough cut.”

Behind the scenes, Claude will:

• Use 11Labs Scribe to transcribe your full recording with word-level timestamps
• Read the transcript and identify silences, filler words, false starts, and repeated lines
• Plan a clean version of the monologue by marking which segments to keep and which to cut

Crucially, Claude doesn’t just run off and edit blindly. It shows you its plan first, waits for your approval, and only then calls FFmpeg to stitch together the clean segments.

The result is a rough cut where several minutes of raw footage can shrink down to under a minute of tight, intentional-sounding delivery—without sounding robotic or chopped up.

3. Fix small issues with natural language

If something slips through—say, a stray “basically” at the start of a sentence—you don’t have to dive into a timeline.

You can simply say:

“You left a ‘basically’ in at the start of that line. Cut it.”

Claude finds that exact word in the transcript, adjusts the edit, and re-renders only the affected section. No scrubbing, no razor tool—just conversation.

Once you’re happy with the rough cut, it’s a good idea to lock it in before adding graphics. Changing the edit after you’ve layered motion graphics on top can get messy, so treat this as your “picture lock” moment.

Adding motion graphics with Hyperframes

With a clean talking-head edit in place, the next step is to make it look produced: text overlays, callouts, and visual support for what you’re saying.

To keep things organized, start a fresh chat with Claude and point it to the same project folder. Then give it a prompt like:

“Plan a motion graphic for each segment and then build them.”

Claude will:

• Read through the transcript again
• Identify the main segments and ideas
• Decide what kind of visual would support each one
• Use Hyperframes to generate one motion graphic per segment

These can include text cards, animated callouts, occasional data visuals, and logo placements—all based on what you’re actually saying in the video.

Refining the visuals to match your brand

The first pass of graphics is usually about 80% there. It looks solid, but still a bit generic. The final 20%—your taste and brand—is where you come in.

You refine the look and feel by talking to Claude, for example:

• “Make the Claude accent color orange, not the default blue.”
• “Drop my logo PNG onto the intro graphic.”
• “That transition is too fast—slow it down.”

Each time you request a change, Claude only re-renders the specific section you adjusted, leaving the rest of the video untouched. That keeps the feedback loop fast and makes iteration painless.

If you want to go deeper into how Claude and Hyperframes work together for end-to-end editing, you can check out this detailed breakdown of Claude + Hyperframes video editing.

Generating perfectly synced captions

Because you already have a word-level transcript from 11Labs Scribe, captions are essentially free.

Claude already knows:

• Every word you said
• Exactly when each word starts and ends

All that’s left is to decide how those captions should look.

You might say:

“Add captions using the existing transcript. Use the ‘Coolvetica’ font, keep it clean and readable, and animate the words in as I say them.”

Claude then uses the transcript to generate captions that land perfectly in sync with your speech, with each word animating on at the right moment instead of sitting as a static block of text.

Adding background music automatically

To add music, you just need to provide a track and tell Claude how loud it should be.

• Place your chosen music file on your PC.
• Point Claude to that file path.
• Specify a target level, for example: “Lay this track under the whole video at -22 dB.”

For talking-head content, a background level around -22 to -23 dB is often a good starting point, but you can adjust it to taste.

Claude then mixes the music under your voice across the entire video—no manual keyframing, no fiddling with fades. It’s just a file path and a number.

Exporting and keeping your project editable

Once you’re happy with the edit, graphics, captions, and music, you can finish with a simple prompt like:

“Export the final video.”

Claude will:

• Use FFmpeg to render the final file
• Drop a copy into your downloads folder (or equivalent output path)
• Keep the entire project structure intact in your project folder

That last part is important: your project doesn’t disappear after export. If you ever want to change a single graphic, tweak a caption style, or swap the music, the full project is still there and ready to update.

Where to go next with Claude automation

With this setup, you’ve turned Claude into a hands-on video editor that can:

• Read and understand your footage
• Cut out silences, filler, and bad takes
• Build motion graphics for each segment
• Generate perfectly synced captions
• Add background music at the right level
• Export a polished final video

You stay in charge of the creative direction and final calls, while Claude handles the repetitive, technical work that usually slows editing down.

If you want to push this even further—automating not just editing but also research, scripting, scheduling, and publishing—you may find it helpful to explore how Claude Code can fully automate social media workflows as well.

Once you’ve set up this pipeline once, turning raw footage into finished videos becomes as simple as dropping in a file and having a conversation with your AI editor.