Visualizing the gradient descent method

In the gradient descent method of optimization, a hypothesis function, $h_\boldsymbol{\theta}(x)$, is fitted to a data set, $(x^{(i)}, y^{(i)})$ ($i=1,2,\cdots,m$) by minimizing an associated cost function, $J(\boldsymbol{\theta})$ in terms of the parameters $\boldsymbol\theta = \theta_0, \theta_1, \cdots$. The cost function describes how closely the hypothesis fits the data for a given choice of $\boldsymbol \theta$.

4. A Beginner's Guide to Gradient Descent in Machine Learning, by Yennhi95zz

Subgradient Method and Stochastic Gradient Descent – Optimization in Machine Learning

An overview of gradient descent optimization algorithms

Orange Data Mining - Visualizing Gradient Descent

Simplistic Visualization on How Gradient Descent works

Visualizing the gradient descent in R · Snow of London

Lecture 7: Gradient Descent (and Beyond)

ZO-AdaMM: Derivative-free optimization for black-box problems - MIT-IBM Watson AI Lab

Variance Reduction Methods

How Gradient Descent Algorithm Works - Dataaspirant