Testing of gradient descent algorithm on Himmelblau's and Rosenbrock's function. Slide a and b, and click on green flag to generate new function and see descent demonstration. Red means high values, dark blue means values close to 0. Purple is descent path. Rays show the amount of possible direction the descent path can take. There are two functions here: Himmelblau's function: (like 4 main dark spots) https://en.wikipedia.org/wiki/Himmelblau%27s_function and the Rosenbrock function: (like a parabola) https://en.wikipedia.org/wiki/Rosenbrock_function Algorithm made by me.
UPDATE 8/31/17: I have added two new functions, Butkin's Function No. 6 and the Six-Hump Camel Function, and gradient descent performs rather poorly o them. I am working on a much more robust optimization method. Gradient descent is an algorithm that always goes forward in a direction when the (partial) derivative on that function is negative (when in that direction the function is downhill) and backward if it is positive (uphill). The algorithm always tries to converge to a "bowl", where it cannot get anywhere else without climbing upward some surface. It hopefully converges to the global minimum of the function.
