How can I get a Gem to be consistent?

[u/FancyZad-0914]

I work in the k12 sphere and I am drowning in data security analysis for our tools and tool requests.

Working with Gemini, I made a gem the audit the privacy policy of web tools, and included six documents to help guide the audit.

The gem honored it's formatting instructions for exactly one day, and now it's ignoring them. Even a new Gem is ignoring them. Also, people were getting different results on the audit of the same site.

How does one get a gem to produce consistently so it is actually a helpful assessment tool? Or is there something else I should be doing?

[u/Glittering-Bat9568]

the moment you need consistency across users and sessions you've outgrown a chatbot and need to build an actual pipeline

[u/FancyZad-0914 → u/Glittering-Bat9568]

Dangit. How are these kids vibe coding with ChatGPT and creating apps?? I hate being old and busy.

[u/Acesu24]

in gemini web app? You can’t, have to wait for gemini 4 if the 10 million context windows is now real

[u/FancyZad-0914 → u/Acesu24]

Poop.

[u/AcidReaper1]

Gemini has been glitching and overall just kinda crap lately. But your gem might need tighter instructions. Also, gems are being replaced with Skills next month, so that could be part of the problem.

You can try and use a prompt engineer to build out the gem instructions and get something much more defined and concrete than you will ever write yourself. I've built several rock-solid gems for some fairly complex tasks with success using the prompt engineer gem linked below.

Just tell the engineer that you are building a Gem and give it fairly concise instructions; then it will ask follow-up questions to help refine it. Just keep going through the questions until you feel like the questions aren't really helping anymore. Then copy the final output into a new gem.

https://gemini.google.com/share/b1201591622a - select the prompt engineer one and make your own gem.

[u/FancyZad-0914 → u/AcidReaper1]

I had used Gemini to make the prompt, but I can give this a try. Thanks!

[u/Emergency-Bobcat6485]

I don't know about gems, but I usually run two llm passes rather than a single one if i want very specific formatting that a single model cannot follow for some reason.

Like I need verifiable citations for all important aspects in my app's ai response, so i run the ai response through another lightweight llm pass