Reinforcement Learning 2D Humanoid Walkers

"Play that funky music, robot"

Teaching 2d humanoids to walk in your browser using reinforcement learning. You can view the progress at differen't stages of training with the buttons below.


Technical: This uses use reinforcement learning to teach them to walk. This is a branch of machine learning targeted at controlling systems such as system of limbs. Training is done offline in tensorflow.js. The environment is in box2d for javascript. The aglorithm is DDPG with prioritized experince replay.

The dark outlines are when the agent grips the floor, since I found walking was difficult otherwise. The balls are to provide obstacles.


Credits: The walker code is adapted from rednuht.org/genetic_walkers/, the DDPG code was adapted from github.com/thibo73800/metacar.

Load brain:

Be mean: