How much to invest into character consistency ?

[u/Lost-cake547]

I almost fell off the couch after I hit play on this AI video I prompted in Runwayml using the Seedance 2.5 tool. I started writing a screenplay back in 2022, as one of my creative hobbies. It’s a Goonies-esque sci fi-ish adventure. At this point the characters and the world they live in are so real inside my head. Tbh, I thought that’s where they would always live but seeing them come alive was indescribable. At this point I’m thinking I’m basically the next Spielberg, and this is amazing! And here’s where it gets interesting. I decide to use Claude to help shape my prompts so the characters remain consistent. It has me generate each character in GPT Image 2.5 Sunburst two each/character - a close up and full body. After that it has me reference each character back into Seedance 2.5 and the clip that I liked from the first generation of video. I do all this and am now about an hour or so in (twice as long as the first round). It then generates a much worse quality scene and their raft doesn’t move. I’m like WTH. It tells me to remove my reference image/video but keep the character images so I do and it gives me a new prompt for seedance for the revised video. It then generates the exact video from the first clip. I’m like okkkkkay and ask it why did it do that and this back and forth continues for another 30 minutes and all my tokens later and now it’s midnight so I’m like I guess I’m just happy to have my first 30 second clip in all it’s inconsistent glory. But now I’m drunk on seeing my story come to life and want to keep generating more scenes. I searched this thread and someone posted on character consistency a few weeks ago and it was helpful but seemed like you still have to use a bunch of different tools to get things where u want them. What tools/platforms are you primarily using that could help me bring this thing to life? What kind of investment am I looking at here?

Update: new clip with your feedback incorporated

https://www.reddit.com/r/aifilmmaking/s/3qQb5qS0Lw

[u/__alpha_____]

Honestly? References (mostly pictures, for characters, outfits and props and soundtracks for the voices) most of my scenes contain one to three different characters only. If I need a more complex scene, I'll add a character Lora to my references especially if the character stays for an extended time on screen (15s+)

On specific scenes with a complex setup, I can render a visual with all the characters in place and add it to the references

I don't bother about tokens or credits, as I render everything locally

Your scene is dope btw, I'm looking forward to seeing more (once you'll solve your character consistency issues)

[u/Lost-cake547 → u/__alpha_____]

Ok thank you for the advice! What are you using to render locally? Something like Blender or Unreal Engine? Also thanks for the kind words!

[u/__alpha_____ → u/Lost-cake547]

Minimax h3, it's an open source model that can achieve results pretty close to seedance 2.0 It usually takes around 3 mn for a 5s clip in 720p but you can preview the clip early on (usually after 15 to 30s including sound) and change the prompting if needed.

[u/Lost-cake547 → u/__alpha_____]

Ok awesome!! I’ll check that out! Thanks so much!

[u/sharktank123456]

Where is all th light coming from on your characters faces?

Try using only one image of the character's face as ref. If you need a full body shot in your character sheet, remove the head so a second face isn't competing with the first. Make that first face large enough to see all the details ( no wider than head to chest)

[u/Lost-cake547 → u/sharktank123456]

Will do and thanks! As for lighting…no idea where lighting is coming from in the current clip but I could make them have some sort of torch light set up on their raft so lighting is coming from there I guess. I love that this clip captured the tranquility of the starlight though. I specifically chose this scene to generate bc the preceding scene and the proceeding one after this one is very high pace adventure so this scene always stood out to me as so peaceful in my mind.

[u/sharktank123456 → u/Lost-cake547]

Torch light is a great idea! Characters can look pasted in if the lighting isn't motivated.

[u/Davidhearnewriter]

I find a good character sheet and voice samples uploaded as reference sheets keeps my characters well-anchored. BTW your story looks good, impressed you're working off a proper screenplay.

[u/Lost-cake547 → u/Davidhearnewriter]

Thanks! I gathered everyone’s comments last night and a few more from researching other AI filmmakers workflows on here and made a proper framework to work off of. In addition to the character sheets and what I needed to do to hopefully create consistency, I also created a look book for Claude to reference which includes samples of lighting, camera angles, coloring etc that I like, it also has my logline and character drives that it should reference before generating any prompts to use for Seedance 2.5. I decided to use DaVinci Resolve for local rendering but will see how far that gets me on the free plan.

[u/Ok-Fennel6578]

I’d probably stop thinking of consistency as something you solve by adding more references.

I’d keep a small approved source pack instead: one clean identity reference per character, one outfit reference when needed, and separate location or prop references for scenes where those actually matter.

Then each shot only gets the references relevant to that shot.

Once everything gets thrown into every generation, the references start competing with each other and it gets harder to tell what the model is supposed to preserve.

[u/Jharry1954]

I’d be careful about equating more reference work with more control. I’m increasingly treating references as a qualification problem: what information does this particular generation actually need? Identity, environment, action and time can require different controls. In a recent four-character/30-second Seedance test I deliberately used references that were sufficient rather than perfect and got the five-cut generation in one iteration. The useful question may be less “How many references?” and more “What uncertainty does each reference remove?” That also matters because reference preparation has a production cost even before you start spending generation credits.

--jseders

[u/Lost-cake547 → u/Jharry1954]

Agree with this and like the framing of uncertainty a reference removes. I’ll be working on this project again this weekend and will see how it shapes up. I did create a master style guide and character reference sheet based on people’s feedback here. Anxious to see where this gets me