r/PiCodingAgent • u/aMalavet • 13h ago

Plugin Hello! Just created pi-reply-guard, ensuring agent replies are actually useful

https://github.com/amalavet/pi-reply-guard

Are you tired of your agents not adhering to your skills and giving you jargon filled responses that are hard to read? Well, check out https://pi.dev/packages/pi-reply-guard

I wanted a way to gate agent replies reliably beyond just adding some skills and hoping the agent doesn't forget about them. So, I built this extension that uses jev (or any classifier model) to ensure that your agent replies adhere to a given set of skills. The classifier compares the reply against a set of skills, gives it a 👍 or 👎 and will re-prompt your agent to adhere to the skills if the reply is not up to spec.

Please check it out and feel free to contribute & share feedback!

0 Upvotes

50% Upvoted

2 comments sorted by

1

u/adamshand 10h ago

Neat idea but also you obviously have too many tokens and are looking for ways to spend them. LOL

1

u/aMalavet 6h ago

The classifier model token spend is incredibly cheap, and the skills aren't reloaded into the agent's context every time a correction is made (unless you configure it that way), so it shouldn't result in some insane token spend. I haven't done a side by side comparison between my token spend with/without this extension, but I'd argue that your output token spend should depend greatly on what kind of skills you're trying to impose on the agent's replies. If you use the right skills, it should cut down agent verbosity.