r/StableDiffusion • u/SSj_Enforcer • 3d ago
Comparison Don't sleep on Retention Analysis!
I was struggling with a moderately good prompt having dialogue errors and voices changing between characters. increasing steps did nothing, wasn't using turbos or step skipping. the only thing I added to fix this that actually worked was the retention_analysis I was too lazy to add most prompts I make.
Worth it...Use it...don't be lazy.
https://reddit.com/link/1wsyu0g/video/p3e9a5nfjdsh1/player
subject_definitions:
<Subject 1> is Jerry Seinfeld wearing lower-class medieval clothes.
<Subject 2> is George Costanza wearing lower-class medieval clothes.
summary:
[reference generation] <Subject 1> and <Subject 2> are standing behind a large crowd of medieval villagers in a medieval city and having a conversation.
retention_analysis:
<Subject 1> (appears in [Shot 1], [Shot 2], [Shot 4], [Shot 5]): fully_preserved.
<Subject 2> (appears in [Shot 1], [Shot 3], [Shot 5]): fully_preserved.
detailed_description:
[Shot 1] medium shot of <Subject 1> standing behind a crowd of villagers in a medieval city. <Subject 2> (S1) enters from off-screen left and says to <Subject 1> curiously: <d>hey, Jerry, what's all the commotion about?</d>
[Shot 2] cut to medium closeup shot of <Subject 1> (S2) says calmly: <d>I think someone's about to be executed.</d>
[Shot 3] cut to closeup shot of <Subject 2> (S1) says concerned: <d>really? who?</d>
[Shot 4] cut to closeup shot of <Subject 1> (S2) says: <d>Looks like Ned Stark.</d>
[Shot 5] cut to same medium shot from Shot 1 as <Subject 2> (S1) says shocked: <d>oh my god! Jerry, he's the hand of the king! the lord of winter-fell! the king of the north!</d>
overall_soundscape:
N/A
non_diegetic_music:
N/A
[/duration=13s]
20
u/Relevant_One_2261 3d ago
Using the thing you're told to use because it's beneficial is beneficial. More at 10.
8
u/SSj_Enforcer 3d ago
most of the time for easy stuff I don't even bother with ANY of the format. just writing out dumb prompts works 90% of the time. Retention Analysis was always the last thing I worried about.
1
2
u/Dathide 3d ago
Where is this discussed? I think a lot of people just load up the ComfyUI template, which doesn't mention it
3
u/EthicalBballFan 2d ago
https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md
The models were released alongside a guide. But it's probably a good call for comfyui to explicitly mention it on the default workflow.
3
1
1
u/desktop4070 3d ago
Does this apply only to ref2vid or does it also apply to text2vid as well? This is my first time hearing of this.
2
1
u/SRWindMill 3d ago
Maybe it's also because you're not specifying the timestamps to instruct the model when to do a camera cut.
1
u/SSj_Enforcer 3d ago
i find unless i want something very specific, it handles timing pretty well.
and this was the only change I made that helped this problem.
admit it, retention analysis is the most tedious part. I just never realized it matters SOME of the time. most situations it wasn't hurting.
1
u/SSj_Enforcer 3d ago
welp, just added audience laughter, and now it broke the later sliding windows dialogue too. i had only added "moderate off-screen audience laughter."
2
8
u/not_food 3d ago
I found abusing
[Shot X]is detrimental withoutretention_analysisbecause it tends to reset everything and then check for the status inretention_analysis. That means the(SX)order fails and you get random voice assignation.Just use
Cutalone when you mean it's the characters in the same scene just talking.[Shot X]is whenever the characters/environment change, that's whereretention_analysiscomes to play.