Visualizing the gradient descent method

In the gradient descent method of optimization, a hypothesis function, $h_\boldsymbol{\theta}(x)$, is fitted to a data set, $(x^{(i)}, y^{(i)})$ ($i=1,2,\cdots,m$) by minimizing an associated cost function, $J(\boldsymbol{\theta})$ in terms of the parameters $\boldsymbol\theta = \theta_0, \theta_1, \cdots$. The cost function describes how closely the hypothesis fits the data for a given choice of $\boldsymbol \theta$.

Intro to optimization in deep learning: Gradient Descent

Gradient descent.

A Data Scientist's Guide to Gradient Descent and Backpropagation Algorithms

How to visualize Gradient Descent using Contour plot in Python

4. A Beginner's Guide to Gradient Descent in Machine Learning, by Yennhi95zz

Orange Data Mining - Visualizing Gradient Descent

Animations of Gradient Descent and Loss Landscapes of Neural Networks in Python, by Tobias Roeschl

A journey into Optimization algorithms for Deep Neural Networks

Gradient descent visualization - plateau

What is meant by gradient descent in laymen terms? - Quora

Jack McKew's Blog – 3D Gradient Descent in Python

A Visual Explanation of Gradient Descent Methods (Momentum, AdaGrad, RMSProp, Adam), by Lili Jiang