Accelerating the computation of parallel trajectories of gradient descent with the Cell-BE multiprocessor environment
Gabriel Wainer and Yuri Boiko
The 2010 Summer Computer Simulation Conference (SCSC 10)
Ottawa, Canada, July 11-14, 2010
Parallelization potential is explored in random scanning of starting points for gradient descent algorithm in Cell multiprocessor environment for (i) 3D nonlinear function approximation and (ii) 2D time series prediction. As 3D nonlinear function the hyperbolic paraboloid has been taken, represented by the equation z=x^2-y^2, which is 2nd order polynomial function with saddle point. As a 2D time series the function x=int[1000*sin^2(t/2)] + int[1000*sin^2(t/20)] + int[1000*sin^2(t/30)] + int[1000*sin^2(t/300)] has been taken, in which the last term represented long term trend for the scale of prediction considered. Multilayer percentron was the neural network of choice to resolve both task (i) and (ii). It is demonstrated, that parallel tracing of gradient descent trajectories of 3D function approximator allows efficiently identifying the suitable starting condition for implementing gradient descent to realize diving trajectory and thus delivering the required accuracy of approximation in shortest time frame. The 2D time series reveal narrow distribution of the gradient descent trajectories, which in itself does not benefit from parallel tracing. The advantage from parallelization here is in splitting n-dimensional time series into n 2D ones because of naturally fast convergence track of its gradient descent training, which allows in shortest time frame to obtain simultaneous predictions for various numbers of steps ahead. Cell multiprocessor offers convenient parallel environment for the above solutions.
Conference Manager (V2.56.8 - Rev. 1182)