 |
| It's Arten, the Pancake Delivery Frog! |
What's green, hoppy, and delivers delicious golden circles? Why, The Pancake Delivery Frog, of course! Never heard of him (Google search results here)? Well, you should check him out at his channel or, even better, go look at the playlist which includes The Pancake Delivery Theme Song as well as Episode 1 (Pancake Delivery Frog in: The Mayor) and Episode 2 (Pancake Delivery Frog in: The Contraption). Hopefully you'll enjoy the delicious videos (at least the theme song one). But as it turns out, the "making of" story behind these videos is just as much of an experiment as the episodes themselves... so read on for a not‑quite‑excruciating recap of occasionally‑excruciating AI techno‑wizardry.
The Beginning
What's it about? Well, one of my sons created "The Pancake Delivery Frog Theme Song" on Suno.AI. I fell in love with the song and thought, "This should be a cartoon!" And voila, the Pancake Delivery Frog idea was born. But there was a problem... I'm not an artist, and I'm not an animator, and I'm not a cartoonist. And I don’t exactly excel at the patience required to learn those things.
The AI Engineering
So off we went into Veo‑land. Veo is Google's AI video generator, and I was using it in the "Flow (formerly Test Kitchen)" of Google Labs. First, it was The Pancake Delivery Frog Theme Song - generating various random clips of Arten in action (Arten is his name), and stitching them all together in DaVinci Resolve (if you're unfamiliar with Resolve, it's a phenomenal video editing application - very powerful, and also free, although they do have a paid version with more features; kind of like Adobe Premiere but without the high cost).
It was certainly a fun, creative endeavor, an experiment in "what can I do with AI Video" that, I think, turned out pretty well. But there was more to be done. It wasn't enough just to have that theme song video, as awesome as it is; I needed to craft the first episode. And thus came "The Pancake Delivery Frog in: The Mayor."
The Result
Not much of a script, just an on-the-fly generation and rendering of video clips. Now, if you're new to Veo, let me warn you: it's great for individual things. It's tough to get a "full video" out of it (it generates 6-8 second clips that you have to stitch together, and it's somewhat difficult to get the characters consistent between video clips or to do exactly what you want without "breaking" the scenes). One thing I found was to use Flow's scenebuilder to make longer clips, extracting a still from the end of a clip to begin the next clip. But it's difficult to make an entire story that way, as you can't do scene cuts very easily.
In "The Mayor," you may note that Arten walks out of his house directly into the street without going through a door - just a missing external wall. And when he gets to the Mayor's house, it took several tries to get the camera to "pan in" through the window to the inside of the house. I also found towards the end of that episode that my "Google AI credits" were running out, which is why the last few "scenes" are maybe a little rough: I didn't have enough credits to continue running new iterations of the prompts to generate video clips, so I had to use what was on hand (remember my patience factor?).
But I got it done, assembled it all, and tossed it out on the internet (via YouTube), creating a new channel for Arten and his adventures.
The French Connection
After Episode 1 was completed, I turned to AI for some "episode ideas" and eventually settled on "The Contraption" for Episode 2. I had AI generate a script, and then generate a series of "scene prompts" for feeding to Veo to generate the video clips. One thing I learned: AI video generators are pretty terrible at voice consistency. Just watch Episode 2 and you'll see what I mean. And it took, literally, months to get enough video clips generated to make the short video sequence for Episode 2. I kept running out of Veo credits. And at one point, a Veo "update" made things very bad; it would totally ignore my prompts, or go way off the deep end with them; it was overly sensitive to prompt wording, or felt it knew better than what I provided.
See the "deleted scenes" at the end of Ep 2 (after the final music); the first one, about "the zenith" and "ruining the batch of pancakes" - that was totally 100% NOT what I fed Veo for the dialogue prompt! The second, where they're speaking in French, I had included "not an English accent" in a prompt at some point (because Veo kept generating voices with English or Australian accents rather than "typical American English" accents), and I think perhaps I forgot "accent" in one of the prompts, so it interpreted "not English" literally and made them speak French (at least, I think it's French). That whole month was almost completely "wasted credits" - I had maybe 6 usable clips out of the fifty that were generated that month. (One of the scenes in the final episode is video from one clip and audio from a different clip.)
Voice‑over may be in the future of The Pancake Delivery Frog.
The Future
But I'm learning. AI prompt tuning (maybe AI prompt engineering) is an interesting science. Sometimes it helps to have AI (like Gemini or Copilot on the web) generate a prompt to feed to another AI (like Veo for video generation or Github Copilot for code generation and modification). I still have more to learn, but it's a fun endeavor.
I'm also looking into using ComfyUI and Stable Diffusion locally on my EVGA Nvidia 2060 Super 8GB graphics card to generate AI video without being tied to Google Flow/Veo AI credits. It’s early days, and I’ll probably need a GPU upgrade (like an RTX 50 Series) before it becomes practical, but it’s rewarding to learn without worrying about monthly AI credits (which I can do with my current GPU).
The End (for now)
But it's been a fun adventure, and learning a lot along the way. Hopefully you'll enjoy the fruits of my labors, and maybe share with your kids. And if you have ideas for an episode of The Pancake Delivery Frog, be sure to share it with me!
No comments:
Post a Comment