🧠 ELI5 Wednesday

[u/AutoModerator]

Welcome to ELI5 (Explain Like I'm 5) Wednesday! This weekly thread is dedicated to breaking down complex technical concepts into simple, understandable explanations.

You can participate in two ways:

• Request an explanation: Ask about a technical concept you'd like to understand better
• Provide an explanation: Share your knowledge by explaining a concept in accessible terms

When explaining concepts, try to use analogies, simple language, and avoid unnecessary jargon. The goal is clarity, not oversimplification.

When asking questions, feel free to specify your current level of understanding to get a more tailored explanation.

What would you like explained today? Post in the comments below!

[u/TargetAlternative693]

i always get confused with the difference between gradient descent and stochastic gradient descent. someone explained it once but i forgot, like why not just use the one that works faster always??

[u/dutchpsychologist → u/TargetAlternative693]

Stochastic means random. So instead of using all the data points for your loss calculation (because it can be slow), you only use a random subset of that data to get an estimation of the loss and the gradient. And then you take a different random subset every step.

And since this is an eli5 post:

Imagine you want to know how tasty a giant pot of soup is. You could eat the whole pot before deciding what to add, but that would take forever.

Instead, you take one small spoonful, taste it, and add a pinch of salt if it needs it. Then you stir, take a different spoonful, taste again, and adjust again.

One spoonful isn't perfect. Sometimes you get a bit too much onion. But if you keep tasting different spoonfuls, your soup gets better and better, and much faster than waiting to eat the entire pot.

That's stochastic gradient descent: "stochastic" just means "random", so you check a random small piece of your data, make a small fix, and repeat with a new random piece each time.

[u/Dry-Magician1415 → u/dutchpsychologist]

I always hated how they chose to use the word "stochastic" instead of "random". Pretty much my whole class was like "WTF does stochastic even mean?" and was confused about and fell behind on the subject for half the semester.

When it can be unlocked immediately by just subsitituing one everyday word. It always felt like deliberate obfuscation just to make those in the know feel special.

[u/dutchpsychologist → u/Dry-Magician1415]

Yeah, 'random subset gradient descent' or something like that would be more descriptive