SGD — 2D Loss Landscape Exploration

θ_{t+1} = θ_t − η·∇L̃(θ_t)  |  L̃ = noisy minibatch estimate  |  Escape rate ∝ e^{−ΔL/η}