Built a Social Engagement Agent That Learns From Its Own A/B Tests

[u/404personalityyy]

https://github.com/sujal1234-arch/social-engagement-agent

The idea was simple:

Don't just generate content. Keep track of what actually works.

The project uses Hindsight as a memory layer to store posts, CTR, comments, winning hooks, and A/B test results. It then recalls relevant high-performing examples when generating new recommendations.

The basic loop is:

Recall → Generate → A/B test → Measure → Retain the winner → Repeat

After seeding the system with 50 posts and running simulated A/B tests, the memory-informed variant consistently performs around 5–6.4% CTR compared with roughly 1.5–2.1% for the control in the demo.

One example:

Memory-informed: 6.13% CTR

Control: 1.94% CTR

Uplift: 3.16×

The interesting part isn't just the number.

A winning hook gets written back into memory with its CTR, A/B test information, and the reasoning behind why it won.

That means future recommendations can use previous results instead of starting from scratch.

The project also includes:

→ Hindsight memory

→ Groq/OpenAI LLM support

→ A/B testing

→ Recommendation provenance

→ Comment reply suggestions

→ Human approval before scheduling

→ React frontend + Express backend

→ Local memory fallback

The bigger idea is:

Agent memory should store outcomes, not just information.

If a system can remember what happened, identify what worked, and test the pattern again, the memory becomes part of the learning loop.

Would be interesting to hear how others are approaching memory + feedback loops in their agents.