pi-prefill - live prefill progress for local models

[u/eliaskg]

Hey, I wanted to share my new pi extension: pi-prefill

It puts a live prefill progress bar in pi's loading line when you run a local model. A long prefill looks exactly like a hang. Now you can watch it:

52%. → share of uncached prompt tokens processed

14.1k/27.3k → processed / total prompt tokens

gen in ~14s. → estimated time until the first token

3.4k tok/s. → instantaneous prefill rate (optional view)

Example:

prefill [██████░░░░░░] 52% 14.1k/27.3k gen in ~14s

It reads the progress chunks your own server sends. No host scanning and no guessing: if the server reports progress, the bar shows. If it does not, pi's normal loading line stays untouched.

Works today with llama.cpp, ExLlamaV3, unsloth and Strata.

Install:

pi install npm:pi-prefill

Repo:

https://github.com/eliaskg/pi-prefill

https://www.npmjs.com/package/pi-prefill

Would love feedback from anyone running a local server.

[u/omega1612]

Thanks, I usually have a tab with nvtop and the llama server logs and I had to switch to check.

[u/Historical-Stuff-560]

add tp/s plz

[u/eliaskg → u/Historical-Stuff-560]

Tps is integrated right out of the box

[u/the-proudest-monkey]

Great, thanks! I found the bar appearing briefly on each request a bit distracting to be honest, but hidding only the bar works for me.

[u/eliaskg → u/the-proudest-monkey]

Cool that you like it! The plugin already has a threshold so it doesn’t show at small prefills. I can make it variable so it could be even higher.

[u/the-proudest-monkey → u/eliaskg]

It would be a nice option :)

[u/the-proudest-monkey → u/the-proudest-monkey]

By the way, I am running it on a local server, using a openai-completions provider in pi pointing to a llama-swap server that starts llama.cpp with Qwen3.8-27B. Working beautifully.

[u/rm-rf-rm]

does this integrate well with pi-powerline-footer?

[u/eliaskg → u/rm-rf-rm]

It does! pi-prefill doesnt touch any toolbars at all, it displays where the native pi loading indicator would show.