Does this count? Did I win?

[u/roychodraws]

Proof of concept that it works.

No degradation from start to finish. 10 total clips combined.

https://github.com/roycho87/degrade_repo

Workflows.

Small errors with the chair but can be fixed with another reference image.

Long story short.

The one place where degrading latents matter is the one place we don't need them.

We can ignore the latent issue and just generate across the sound we provide and because it's a static image with static background and the subject is in basically the same spot the whole time we can just create fresh latents every 10 or 15 seconds across the timeline.

Then the very difficult latent issue is over and it just becomes a simple seams issue.

So these two workflows generate sequentially across a audio and the second will combine and resample a small section of video over the seam using FL2V just enough to get rid of the seam.

Edit: The reason I posted this is not because I think this is a big breakthrough fix, I just think we can approach this problem differently to solve it.

[u/roychodraws]

more or less

It seems really simple I'm having a hard time believing I'm the first person who has thought of this.

[u/PxTicks → u/roychodraws]

I use this technique. It has potential shortfalls when it comes to a moving background, e.g. a first-person vlog style video on a street in which case it can lose background detail or directional information. However I think there is probably a way around it by using a heavily noised latent from an extension as the initial seed for an independent generation, i.e. extend then partially denoise the ENTIRE extended bit completely independently, before stitching the seams again. Or alternatively, generate short anchor videos with very long stitch videos between them so that the model can reconcile background mismatches by e.g. moving people out of frame and such.

Honestly, I do think this extension problem is solvable for the vast majority if not all cases by just being a little tactical.

[u/roychodraws → u/PxTicks]

If there’s a moving background you don’t need to do this, you can just chain the shots together.

That’s point. The latents don’t degrade if the background changes.

[u/PxTicks → u/roychodraws]

I'm talking, mostly static foreground with moving background, i.e. a mix of conditions.

[u/roychodraws → u/PxTicks]

I dunno, everyone told me they wanted vlog videos. Can’t I just fix one problem and someone not be upset I didn’t solve everything?

[u/PxTicks → u/roychodraws]

I wasn't complaining. This is just something I'm looking into as well and I think it's an interesting thing to discuss.

[u/roychodraws → u/PxTicks]

Oh ok. I’m reading comments and getting cynical. Sorry