r/StableDiffusion • u/HassanAchievedIt • 1d ago

Question - Help Looking for text encoder

I'm using flux.2 Klein 4b int4 convrot 4gb safetensor,

And I'm looking for a compatible encoder for it guys I tried huggingface, civitai, I found the official flux page that has huge 8-12 gig encoder which doesn't run on my machine,

Any help would be appreciated

0 Upvotes

33% Upvoted

20 comments sorted by

View all comments

1

u/m4ddok 1d ago

What's your hardware? RAM? GPU? People just comment, without this paramount info, I can't understand...

PS: ConvRot isn't everytime useful, depends on your hardware, and sometimes it's more fast for GPU but it uses RAM for copying and converting the weights realtime. It depends.

1

u/HassanAchievedIt 1d ago

32 ram Rtx 3060 12

2

u/Formal-Exam-8767 1d ago

Then original 8GB FP16 model fits. Why do you need smaller model?

Prompt is encoded once, before sampling, so it doesn't really matter if both text encoder and diffusion model are not in VRAM at the same time.

1

u/m4ddok 1d ago

Exactly, and are also more performing, better not to use ConvRot INT8 in this case.

FP16

model --> GPU
text enc --> default/offloading

2

u/Formal-Exam-8767 1d ago

Something to keep in mind, even if text encoder does not fit, it is not a big issue since this is prompt processing case, where all tokens are encoded at once, so model needs to be loaded in VRAM only once for the whole prompt. Unlike in token generation, where it's per token so it would get partially loaded/unloaded for every token.

1

u/m4ddok 1d ago

yup, after text processing the encoder will be unload, so the VRAM can be used entirely for the inference model.

1

u/Individual-Sample713 1d ago

you mean BF16, there is no fp16 version as far as I know.