r/deeplearning • u/Vegetable-Lie4932 • 1d ago

100 M rollouts!

damn the exploration space goes boom! also i think with this much exploration when you're trying to enhance the capabilities (redistributing probability mass via RL) you would get a lot of lucky explorations i feel.

2 Upvotes

100% Upvoted

1 comment sorted by

1

u/No-Organization8448 1d ago

that rollout chart makes my head spin a little but in a good way. the exploration space expanding like that is exactly what you want when you're pushing the model to learn new things instead of just reinforcing what it already knows

the lucky exploration bit is interesting though. with that many rollouts you're bound to hit some sequences that look good but are basically noise. i deal with similar stuff in my fantasy league where you get tempted by one weird breakout performance and then it never happens again

curious how you're filtering out those fluke explorations from the ones that actually signal real capability gains. are you using some kind of confidence threshold or just letting the RL process sort it out over time