LOSS LANDSCAPE
Learning rate η
0.050
Momentum β
0.90
Adam β₂
0.999
Noise σ
0.020
Landscape
Rosenbrock
Himmelblau
Ackley
Rastrigin
Optimizers:
SGD
Momentum
Adam
Reset
Random Start
SGD Loss:
—
Mom Loss:
—
Adam Loss:
—
Steps:
0
SGD: θ ← θ − η∇L | Momentum: v ← βv + ∇L, θ ← θ − ηv
Adam: m ← β₁m + (1−β₁)g, v ← β₂v + (1−β₂)g², θ ← θ − η·m̂/√v̂
Color map: blue=low loss → red=high loss (log scale)