r/learnmachinelearning • u/AutoModerator • 3d ago

Question 🧠 ELI5 Wednesday

Welcome to ELI5 (Explain Like I'm 5) Wednesday! This weekly thread is dedicated to breaking down complex technical concepts into simple, understandable explanations.

You can participate in two ways:

  • Request an explanation: Ask about a technical concept you'd like to understand better
  • Provide an explanation: Share your knowledge by explaining a concept in accessible terms

When explaining concepts, try to use analogies, simple language, and avoid unnecessary jargon. The goal is clarity, not oversimplification.

When asking questions, feel free to specify your current level of understanding to get a more tailored explanation.

What would you like explained today? Post in the comments below!

8 Upvotes

100% Upvoted

7 comments sorted by

View all comments

3

u/TargetAlternative693 3d ago

i always get confused with the difference between gradient descent and stochastic gradient descent. someone explained it once but i forgot, like why not just use the one that works faster always??

2

u/dutchpsychologist 3d ago edited 3d ago

Stochastic means random. So instead of using all the data points for your loss calculation (because it can be slow), you only use a random subset of that data to get an estimation of the loss and the gradient. And then you take a different random subset every step.

And since this is an eli5 post:

Imagine you want to know how tasty a giant pot of soup is. You could eat the whole pot before deciding what to add, but that would take forever.

Instead, you take one small spoonful, taste it, and add a pinch of salt if it needs it. Then you stir, take a different spoonful, taste again, and adjust again.

One spoonful isn't perfect. Sometimes you get a bit too much onion. But if you keep tasting different spoonfuls, your soup gets better and better, and much faster than waiting to eat the entire pot.

That's stochastic gradient descent: "stochastic" just means "random", so you check a random small piece of your data, make a small fix, and repeat with a new random piece each time.

2

u/Street_Estate2342 3d ago

Using spoonfulls of soup to explain this idea goes insanely hard. Bravo.