Download C++ source code pcc-1.0.tar.gz (152 kb) Download html description file pcc-doc.tar.gz (41 kb) PCC. AN APPLICATION TO STUDY RL APPLIED TO CAR RACING PCC is a simulation software of a car competition on a plain track, created to test some reinforcement learning algorithms using a stochastic, discrete and partially observable environment. It's a multiagent system. We have two agents, each trying to control a different car. There are some other cars on the road but they're only obstacles, since "they don't think" and just keep running at an only speed changing their rail randomly. The objective of the simulation is to make it possible that agents find by themselves the necessary knowledge to drive properly, based on their bumps and time wasting through the race. They must learn how to avoid obstacles, learn how to avoid each other and learn how to do a lap in a reasonable time. Statistic results can be observed in 2D graphics generated by the application using gnuplot. It runs with X11 and it was written in C++, using KDevelop and Qt libraries. PCC is far simpler than RARS (an excellent robot-racing simulator that inspired the designer) in many senses. But I though that with PCC I wouldn't have to wait for a long time to see a car learning as it might happen with RARS. With PCC anyone can avoid wasting time looking at such a slow competition since one can hide cars (everything) and see them after many interactions, spending only a few seconds. Also it's possible to save policies in hard disk. Anyway I would like to join RARS competitions when I consider I've worked enough with PCC. The first version includes the SARSA algorithm using a look up table. In file params.cpp we can change parameters for the simulation and in file agente.cpp it's possible to make changes to the learning method. That's what I've been doing since I finished building PCC. I'm also working on the design of a second and less shy version of PCC, which might be used to test a real competitive co-evolution with several RL schemes. I'm thinking of: Including more RL agents, each using independent learning algorithms Change the environment a little: Permit overlapping, increase vision area, increase number of speeds and rails and design a "less discrete environment". Using the standard scheme for programming RL applications, proposed by Rich Sutton and Juan Carlos Santamaria. Distributing the graphic interface separated from the simulation code, so that the simulation can be easily used (included or called) by any other application ( such as a genetic algorithm).

Last modified: 27/07/2002, 13:47 PM. Using Hyper Builder on GNU/ Linux. Thanks!