For 1D functions, the program will generate a 2D plot of the function and the optimization path. For 2D functions, a 3D plot of the function's surface and the path taken by the gradient descent ...

plotcritpts2d () plots the toy problem function in 3D in the theta-phi plane and marks critical points. grad (t,p) returns the gradient at the point (t,p). Uses the gradient for spherical coordinates.

Gradient variance errors in gradient-based search methods are largely mitigated using momentum, however the bias gradient errors may fail the numerical search methods in reaching the true optimum. We ...