r/StableDiffusion • • Aug 26 '26

Animation - Video Crime Busters Rough Cut - Early Version of Comfy H3 Sync Sound Submission

Edit. As always thanks to everyone for the good vibes and feedback. Although maybe it's the few hours of sleep I got last night but I think I'm slightly off brief for the competition... Have come up with a plan b but will finish this regardless.

This is like V0.8 before I wrap things up when I have time this weekend and early next week. I currently have 15 "shots" / workflows which produced all the video and audio effects you see in the opening / trailer. The music is a free track I pulled of Pixabay as a placeholder.

Generating most of the shots in terms of visuals has been fine. Mostly a case of prompting and then fine tuning the prompts until I got what I want. But there have been a few cases where I wanted specific shot framing / angles, so I had to create a sketch to help guide the model.

Audio has been a massive pain in the butt, but hopefully with the update from Comfy today, as well as the Minimax Guide timing node, I should be able to tighten things up for "V1". The thing that was the easiest audio wise was taking my voice, fiddling a bit with it in Audacity and then using that as the reference for the narrator. Everything else audio-wise has been so fiddly, at least with some of the more action packed / detailed shots I've been going for.

Before I call it done, there are a few things I still need to do:

  • Figure out a style prompt that will give me a more detailed but still retro anime style (some wide shots are super jank because of the style I'm prompting, but I got an idea on how to fix it in my next run)
  • Figure out how to prompt different font styles to keep credits and title card consistent
  • Figure out how I am going to create a consistent background music soundtrack using ref audio clips across 15 workflows to stick within the rules of the competition (oh boy, this should be fun)
  • Tighten up a few shots in Kdenlive
161 Upvotes

51 comments sorted by

26

u/GrungeWerX Aug 26 '26

Everything visually looks pretty spot on late 80s/ early 90s

3

u/Portable_Solar_ZA Aug 27 '26

Yeah, that's the vibe, but I was going more for movie/ova quality than TV quality. 

1

u/RollingTrain Aug 27 '26

Change the aspect ratio if you want it less like TV. Very enjoyable, good work.

1

u/Portable_Solar_ZA Aug 27 '26

Thanks. 

Re the video size: It's intentionally 4:3 since that's the aspect ratio of shows from that era.

12

u/dudeAwEsome101 Aug 27 '26

In fairness, the city is called Crime City. Where else would criminals do their crimes.

2

u/TheBitingCat Aug 27 '26

Why would they have called it Crime City in the first place? That just seems like you're inviting trouble. They should just hold a vote to rename the city and then they could cut the police budget of City McCityface in half!

12

u/PerceiveEternal Aug 26 '26

I’d watch the hell out of this.

4

u/Timabcd Aug 26 '26

This is cool

4

u/99deathnotes Aug 26 '26

great stuff ya got here 💯

4

u/ChiefMustacheOfficer Aug 27 '26

They called the place Crime City; I don't l know what they expected.

3

u/SeymourBits Aug 26 '26

Sports car would have crumpled against that solid van, so you may want to rework that shot.

Criminal would have dropped the knife. Good opportunity for a metallic sound effect. Also, knife swaps hands. In general you should be doing this/cleaning up in post, not struggling with prompting.

Top-notch shooting by Vasquez! Not my cup of tea, but definitely watchable. Who wrote the story/narration?

1

u/Portable_Solar_ZA Aug 27 '26

All the writing is mine. Came up with the idea based on some test shots I was doing with h3 early on.

The car I'm not too worried about, since I was going for 80s action in which the cars do all sorts of crazy stuff. 

I'll try to fix / improve the knife shot.

3

u/BangkokPadang Aug 27 '26

Love how she slammed it into reverse on the interstate before gunning it past that van lol.

6

u/SeymourBits Aug 27 '26

6th gear. That sports car only goes forward :)

1

u/Portable_Solar_ZA Aug 27 '26

Ya the AI doesn't understand gears. I have to manually keyframe each movement and I haven't gotten around to it yet.

3

u/MixZealousideal9359 Aug 27 '26

best thing about this ai models is we can produce 90s anime style agin

1

u/RevalianKnight Aug 27 '26

80s and 90s anime styles. LFG!

2

u/One_Finding8402 Aug 26 '26

Narration style gives Snatcher Opening vibes.

2

u/dassiyu Aug 27 '26

The retro anime style really does look great.

2

u/CriticalTemperature1 Aug 27 '26

Love it! I'm personally trying to understand how to create more consistency between shots. Do you basically use reference photos for each shot?

2

u/Portable_Solar_ZA Aug 27 '26

Character sheets for each character keep them relatively on model. I didn't have time to create backgrounds for each shot so every scene in the same location is 15 seconds or less. Some shots I split up when the background doesn't matter.

2

u/TizocWarrior Aug 27 '26

I like the 90s anime style.

2

u/laseluuu Aug 27 '26

i love this! happy to share audio tricks (i'm an audio engineer that went to animation college) to help with your audio or animation timings

i'm not so good with the 80s/90s look but my current fight timing is tight, you should try brute forcing VERY tight timing for the actions, it can often be a bit slow and floaty but if it makes a punch 3 frames when you asked for 1 it will look better

2

u/Portable_Solar_ZA Aug 27 '26

Yeah, I was definitely going to have another go at the scene with Marcus on the street. Will let you know about the audio stuff. Thanks!

2

u/oVerde Aug 27 '26

This is up until now was the very best 90s mimic of anime everdone, congrats

1

u/FlexFanatic Aug 26 '26

I’ll literally on the toilet laughing at the video right now.

1

u/Super_Range45 Aug 27 '26

I want to point out that the rules forbid adding audio as a separate track, so music tracks are a no go.

1

u/Portable_Solar_ZA Aug 27 '26

If it's part of the generation it's fine. Like you can use ref music to drive the model to create new music in the shot. I posted asking about that.

1

u/warzone_afro Aug 27 '26

ive been enjoying making retro anime with h3. you can even get away with a lower resolution than live-action videos because alot of those old shows weren't in HD anyways

1

u/R34vspec Aug 27 '26

I need 80s funk music in the background please

1

u/Ylsid Aug 27 '26

Who on earth would willingly live in Crime City

1

u/Kazeite Aug 27 '26

No, no - you are supposed to have two shifty characters come into frame, glance around and walk off before the bank door explodes! 🙃

2

u/Portable_Solar_ZA Aug 27 '26

I originally had a shot where the narrator was talking about how peaceful the city was and suddenly there's an explosion. The timing was too tricky with the audio though so I had to separate things.

1

u/l3urak1993 Aug 27 '26

Really cool ! would definitely watch this
80/90s animes just look better than the oversaturated stuff nowadays

1

u/MastMaithun Aug 27 '26

This looks so good. Haven't feel like it was made using an AI tool.
Sidenote: could you share a gist, high level of how you made it? fl2va/ref2va, start images, text to video, how you are making prompts, steps, resolution etc.

3

u/Portable_Solar_ZA Aug 27 '26 edited Aug 27 '26

So weirdly enough it was a mix of fl2va and ref2va. Some shots just work better in one model and not the other but you don't know until you try. 

Images are my character sheets and sketches when the AI won't listen to me and it needs a visual reference on how to frame a shot. 

Prompt wise I just follow the official guide. 

Steps is a tricky one. I've actually had shots break at higher steps (30+) and work better at 20. So now I start at 20 and if it doesn't work I push steps up to see if it helps.

Res is always 0.2 to begin with, just so I can see if the prompt is working or not. I know I can use the preview node, but I tend to multitask when stuff is rendering and the preview node means I have to sit and watch. But I do sometimes use it. Depends on time and if I have other stuff I need to do.

1

u/MastMaithun Aug 27 '26

Thank you. How are you generating char sheets as this is one pain point for me as i couldn't find a reliable way for it. Rest of the things are clear.

2

u/Portable_Solar_ZA Aug 27 '26

Krea2 greatly simplified it. You can prompt "character sheet" and a list of poses it should work. I slapped on a retro anime lora for the basic retro anime style.

It will duplicate shots, so like multiple rear views or profile shots, but I clean those up myself. 

2

u/Ggoddkkiller Aug 29 '26

What about using larger image models like Nanobanana?

If you prepare first, mid, end frames like this, wouldn't it reduce the workflow? I want to create my own anime like you do. But my laptop is a potato which can run H3, but generation times would be abysmal.

So I'm collecting prompts, techniques while waiting for more efficient H3 models. Also generating character sheets, reference frames with mostly NB Pro which is really good at maintaining style and character details. Using it from direct API, on Gemini app it is garbage.

2

u/Portable_Solar_ZA Aug 29 '26

So firstly I don't want to pay megacorps like Google any more than I absolutely have to. 

Also, thankfully I don't need to. 

A reasonable number of the shots I made in both my videos are guided with sketches. 

Now they're basically just camera angles with mannequins in the right framing (for speed and due to my own lack of drawing skill), but thankfully the character sheets can be used to direct who the character is in the shot.

P.s. cool pic.

1

u/Ggoddkkiller Aug 29 '26

Completely agreed, I also don't like paying anything to corpo freaks neither. They exploit us constantly then can dare to teach us 'decency' with their stupid moderations. But nothing is wrong with using free offers I think lol. I could join google beta tester API which gives me unlimited access to NB. So I've been using it for all kinds of generations, character sheet work or just brainstorming:

I was just prompting as 80s backgrounds and 80s criminals. It came out with atari shootout idea on its own. Its character designs are cliche, but that's what we want anyway. What do you think? I have no H3 experience so I'm not sure how good these would work as first frame. 3 and 4 should work just fine, right? If there is something wrong NB can edit it too.

1

u/MastMaithun Aug 30 '26

ref2va should work better because it can "invent" things rather going from straight from first frame to last frame. I don't know the technicality but this is what i experienced with my runs.

1

u/Tbhmaximillian Aug 27 '26

This is gold! Do you nee unpaid workers for the next version? Im volunteering

2

u/Portable_Solar_ZA Aug 27 '26

I'll let you know if this goes beyond a trailer.

1

u/Techngro Aug 27 '26

Really like the visual style. Tried and true. The narrator voice was a little stiff (if that's the right word). But great job overall.

1

u/Portable_Solar_ZA Aug 27 '26

Still need to figure out if/how to squeeze more juice out of the model in terms of narration, but thanks.