I Finally Got a Whole Video Dubbed with Open-Source AI

I’d gotten these tools to dub before, but the second I sat down to do the exact same thing again, the errors came right back. This time I pushed through every one of them, and both tools gave me a real, watchable dub. I still spent most of the day stuck, but it worked, and I finally have finished videos to show instead of a story about almost getting there.

The same clip, dubbed two ways

Both tools dubbed the same clip, a short, public-domain NASA piece about the Artemis Moon mission, so you can compare them side by side. They work in completely different ways:

• KrillinAI transcribes, translates, and dubs with a ready-made voice, and burns subtitles on too.
• Voice-Pro clones a voice from a short sample and dubs in that voice.

Here’s the original, before either tool touched it:

Original: NASA clip, English (before dubbing)

KrillinAI: it looked fine, but nothing got translated

KrillinAI threw a bunch of errors before it would dub, mostly from clashing with another dubbing app I had installed. But the one that really got me wasn’t an error at all. My translations were coming out in plain English, untranslated, with no warning and nothing red on the screen. The translation engine just wasn’t running, and instead of telling me, KrillinAI passed the original text straight through as if it were the translation. That’s the scariest kind of bug for someone like me, because the result looks finished, so you’d never know to check.

KrillinAI: dubbed with an Edge-TTS voice

A few honest notes. On the free tier you’re stuck with Microsoft’s voices, and you pick a voice by typing in its code instead of choosing from a menu, a little quirk you get used to fast. But the dubbing quality was pretty solid for an open-source tool, and the translation held up well too. The interface is simpler than Voice-Pro’s, and I liked it better for that. The thing I liked most: it burns clean, properly-timed subtitles on by itself, so I didn’t have to line anything up. Using it free went well enough that I’m tempted to try its paid voice-cloning someday too.

Voice-Pro: I had to fix the errors myself

Voice-Pro’s errors were deeper, down inside the little libraries it’s built from. One kept demanding a part that isn’t even used for dubbing, and it crashed before it could start. Fixing them one at a time was endless. Every time I blocked one, another from the same family blew up. What finally worked was fixing it in one spot they all pass through: skip the missing optional part instead of crashing. One change, and the whole family of errors disappeared at once.

I ran it two ways so the comparison would be fair. First with the same Microsoft voice KrillinAI uses, about as close to head-to-head as I could get:

Voice-Pro: same Edge-TTS voice as KrillinAI

Then with Voice-Pro’s own voice cloning, the feature it’s really built around:

Voice-Pro: CosyVoice voice clone

The cloning is really good, the best thing either tool does. It does take longer to run than the plain Microsoft voice, but the result is worth the wait. And the big difference between them is this: Voice-Pro gives you everything free, cloning included, while KrillinAI keeps cloning behind a paid API. For someone set on staying completely free, like me, that matters a lot.

Side by side, the two felt really different. KrillinAI’s big win is its subtitles. They come out clean and well-split on their own, no editing needed. The downside is the pacing. To make the dubbed voice fit the original timing, it speeds up and slows down, and all that stretching and squeezing ended up sounding pretty unnatural. Getting the timing right would probably take some extra work. The other thing I missed was that the finished video came out with the background audio stripped away.

Voice-Pro went the other way. You do have to fix its subtitles yourself, but everything else about it made me happy. The voice length came out matched automatically, so the pace stayed steady and natural, and like I said, the cloning sounded really good. Best of all, it kept the background audio in the finished video, so the result was ready to post as-is. For me, that last part made the biggest difference.

How long each one took

On an RTX 3080, dubbing the whole clip from scratch:

On the same Edge-TTS voice, KrillinAI was about twice as fast, mostly because Voice-Pro does more heavy lifting, like splitting out the background audio. Cloning a voice with CosyVoice is the slowest, since it runs a model on the GPU one line at a time. None of it is really slow, though. You’re talking a couple of minutes either way.

What the day taught me

I figured the hard part would be disk space or waiting on downloads. It wasn’t. Everything had downloaded fine. The hardest part was that it had all worked before, and when I sat down to do the exact same thing again, it just wouldn’t run. That’s what messes with your head. Not some brand-new problem, but something that worked perfectly last time quietly refusing to work now. The other thing worth remembering: watch out for the quiet failures. KrillinAI handing back untranslated English without a peep is the perfect trap, because the output looks done.

I’m still a non-coder fumbling around in the dark on most of this. But this time the fumbling ended somewhere real: two finished dubs of the same clip, made entirely with free, open-source tools.