A MATLAB package for numerous gradient descent optimization methods, such as Adam and RMSProp. The newest algorithm is the Rectified Adam Optimizer.
To test the software, see the included script for a simple multi-layer perceptron or the MATLAB code for a recurrent neural network (RNN).