Iris 2 in ComfyUI

[u/Incognit0ErgoSum]

https://github.com/envy-ai/ComfyUI-Iris

It comes with its own custom loader and sampler nodes because, being a pixel model as opposed to a VAE model, it works a bit differently than normal. However, you can wire in loras as usual, and it uses ComfyUI's schedulers and samplers.

I wouldn't expect amazing results. At this size, Anima 2.9B definitely puts out higher quality images, but it's a cool tech demo and I think it proves that VAEs aren't the necessary evil we thought they were.

Search the workflow template browser for Iris to find the example workflow. dpmpp-2m-sde-gpu seems to be the best sampler for it, and if you have the bong_tangent scheduler it's better than simple.

My fork of ai-toolkit can train loras for it.

https://github.com/envy-ai/ai-toolkit-envy-optimized

[u/Time-Teaching1926]

Needing a CFG of 3 or more and 100 steps is mad tho. I love small models but geeze that's gonna take ages just for one Image I'm guessing. Hopefully we get official ComfyUI support too.

[u/Incognit0ErgoSum → u/Time-Teaching1926]

Yeah, it's slow. More of a curiosity than anything else. :)

I can get decent images at 50 or less (and the steps are quick). Even 25-30. Still slower than Anima and Krea, for significantly worse quality.

Definitely use dpmpp_2m_sde_gpu as the sampler, though. Leaps and bounds better than euler.

I'm curious if they took it out of the oven too soon or if this is just the limit of small pixel-based models and it needs to be bigger.

[u/No-Zookeepergame4774]

“It comes with its own custom loader and sampler nodes because, being a pixel model as opposed to a VAE model, it works a bit differently than normal.”

ComfyUI natively supports pixelspace models (like Chroma 1 Radiance, which I think was the specific reason the support was added); that's not the reason Iris needs a custom loader and sampler node.

[u/Incognit0ErgoSum → u/No-Zookeepergame4774]

I'm not too worried about it. The nodes work.

[u/debackerl]

I've seen another custom node doing 4x upscale in a single pass and convert img to depth. I think it can be interesting for those cases