Jose Antonio Martin H.
PhD
Madrid, España
|
|
Jose Antonio Martin H.
PhD
Madrid, España
|
|
|
|
| Python numpy ndlinspace, the n-dimensional linspace function: | |
| Python Code of the n-dimensional linspace function nd-linspace (python + numpy) | ndlinspace.py |
| Simple and very useful Multilayer Perceptron Neural Networks with Back Propagation training: | |
| Python Code (pure python) | bpnn.py |
| Python Code (python with numpy - fast for big networks) | Xbpnn.PY |
| Java Code (pure java implementation) | bpnn.java |
| Matlab simple and nice multilayer perceptron (MLP) with back-propagation training (pure Maltab/Octave implementation). | Matlab BackPropagation |
| This implementation is specially designed for neuro-evolution since all the weights are represented in a vector which is then automatically decoded in the evaluate function. | |
| Simple and very useful Kohonen's style Self Organized Maps SOM Neural Networks: | |
| Pure python Self Organized Map SOM | SOM.py |
| Pure python Fuzzy Gaussian Kernel Self Organized Map OSOM (you must install Matplotlib only to see the example) | OSOM.py |
| An Evolutionary Neural Network (Multilayer perceptron) with Differential Evolution Training Algorithm: | DEBPNN.py |
|
Python Code (pure
python), please download both two files
which are needed by the Neural Net. This Network performs better than back propagation. Also it is implemented saving and reading weights to/from file, Enjoy it. |
DESolver.py |
Competition Records for JAMH Team Reinforcement Learning Competition |
||
 |
Results | Event - problem - domain |
| 1º First place | Poly-Athlon 2008 | |
| 2º Second place | Mountain-Car 2008 | |
| 1º First place | Helicopter Hovering 2009 | |
| 2º Second place | Poly-Athlon 2009 | |
|
||||
| It includes as examples a Mountain Car Problem and Cart Pole Control Problem: | ||||
| Finally I have decided to make the graphics output with vpython which I think it is the best option to produce nice and very easy to program real time graphics on python for this kind of things. After the installation, check the examples folder to see the Mountain Car demo and the Cart Pole demo. | ||||
|
| @ | Download the Package FAReinforcement for python: | FAReinforcement |
| It includes as examples: Acrobot, Mountain Car. discrete and continuous Cart Pole and a predator/protector/prey game | You should install Python+Vpython+Numpy+SciPy | |
| note: these are High Quality/Performance Reinforcement Learning implementations! do not think they are simple software just because they are public and free!.I used this same software in the Reinforcement Learning Competitions and I have won!. | ||
Please note that in some versions of Matlab you should delete some empty parenthesis in order avoid some errors.
This should not happen with Matlab release from version 7.
This code is a simple implementation of the SARSA Reinforcement Learning algorithm without eligibility traces, but you can easily extend it and add more features due to the simplicity and modularity of this implementation. Enjoy it!
| Matlab SARSA implementation of the Mountain Car Problem | SARSA Mountain Car.zip |
| Matlab SARSA implementation of the Cart Pole Control Problem | SARSA CartPole.zip |
| Matlab SARSA implementation of the Acrobot Control Problem | SARSA Acrobot.zip |
| Matlab Multi-Agent SARSA implementation of a Three-Link Planar Robot | SARSA Arm.zip |
| Matlab Multi-Agent SARSA implementation of a SCARA 3D Robot | SARSA-scara.zip |
| Matlab Dyna-Q implementation of Dyna Maze World | DynaMaze.rar |
| Some pictures of the Matlab implementations: | |
| Mountain Car: | Cart Pole: |
 |
|
| Three-link Planar Robot: |
Acrobot: |
 |
|
| Online Planning with Dyna-Q Dyna Maze | |
 |
|
@ Eligibility traces:
Matlab SARSA(λ) (sarsa-lambda) eligibility traces implementation for the Mountain Car Problem: SARSA lambda (eligibility traces) Mountain Car
@ The kNN-TD(λ) algorithm: a winner of the Reinforcement Learning Competition 2008, 2009
Matlab kNN-TD(λ) implementation for the Mountain Car Problem:
kNN-TD Mountain Car
Please note that this is a Matlab implementation, not the
competition one (originally in python), and is made for academic
purposes so it is not optimized for performance or software quality
design. It is just an "easy to understand and easy to try"
implementation. comments are welcome!.
@ The Ex<a> Continuous Actions Reinforcement Learning Algorithm. (Exa RL Algorithm)
Matlab Ex<a> implementation for the
continuous Cart-Pole Problem: ExaCartPole.zip
(updated 7/5/2010)
Please note that this is a Matlab implementation that was originally
written in python, and is made for academic
purposes so it is not optimized for performance or software quality
design. It is just an "easy to understand and easy to try"
implementation. comments are welcome!. It can be fastened by
using some knn library such as
ANN.
@ The Dyna-H heuristic planning Reinforcement Learning Algorithm. (Dyna-H RL Algorithm)
Matlab Dyna-H implementation for
path finding in a Maze problem:
Dyna-H.rar
(updated 12/2/2011)
| Dyna-H: Heuristic planning Reinforcement Learning Algorithm |
 |
 |
A collection of experiments to prove the
hypothesis of the reduction of POMDPs to MDP by means of the use of
Internal Clocks.
I am sorry for not having at
this time more theoretical material at hand but you can write me if
you want to talk about or even better, join the rl-list at google.
Matlab SARSA Iclock (sarsa plus internal clock) solution to Altered POMDP Mountain Car
Problem:
SARSA ICLOCK Mountain Car( Altered POMDP Mountain Car Problem. The Agent
cannot see the position of the car but only its speed !!!. )
|
Mountain Car internal clock experiment: |
|
|
|
We will extend the list of solved problems frequently. Any comments or suggestions are welcome.
This is a translation of Martin Butz XCSJava with some pythonic modifications.
Also, I am planning to port some parts tThis is a translation of Martin Butz XCSJava with some pythonic modifications.
Also, I am planning to port some parts to C python extension modules in order to speed up the tool using pyrex.
Please, report any bug.
I am working to free this implementation of any possible bug, despite that its father XCSJava seems to be bug free, this is a translation which may contain unpredictable bugs. When more tests be made I will report that this implementation is bug free.
Cheers.
| Matlab implementation of neuro-evolution learning for robot control: | EvoNeuroControl.zip |
|
|
| Planar Robot | SCARA Robot |
| This software is part of a research paper on neuro-evolutionary methods for multi-link robots, such as the three link planar robot and the SCARA robot. | |
