How a weight knows which way to go
1 min · 2 chapters
Machine learning. A sample lesson, free to watch.
The question
If a weight can only see the loss at the value it has now, how does it know whether to go up or down?
The lesson in words
The puzzle
People say training walks downhill on the loss. Here is that hill, for one weight. But the weight cannot see this curve. It only sees one number, the loss where it stands. With just that number, which way should it go?
Now put a second weight on the other side of the valley, at exactly the same loss. If the number alone told a weight where to go, both would move the same way. Make a guess.
Reading the slope
They must move in opposite directions, even though the loss is identical. So give the right one a tiny nudge to the right. The loss goes up, so downhill is to the left. Nudge the left one to the right. The loss goes down, so downhill is to the right. That change is the slope, drawn as a straight line through the marker. It tilts up on the right, down on the left, and lies flat at the bottom. Try sliding the weight along the curve, and watch which way the straight line tilts.
The weight reads its slope and steps the opposite way. A steeper slope means a bigger step. From the new spot the slope is gentler, so the next step is smaller. At the bottom the slope is zero, so the weight stops. The weight never needs to see the hill, only the slope where it stands. Going against that slope is what downhill means.
Stuck on something else?
Type it in your own words and Knotstuck makes a lesson like this one for it.