SISYPHUS (Our Learning Robot) – 2016
SISYPHUS is a robot that learns to crawl using a simple AI algorithm called reinforcement learning. The robot tries random actions at first and learns if it is moving forward or backward. Over time it connects actions that move it forward.
SISYPHUS was created in response to the dramatically increasing role of artificial intelligence algorithms and robots. Our robot is designed to be simple enough for us to understand and build while having the capability to learn and surprise us. Where highly advanced robotic and AI systems may give us a sense of loss of control, SISYPHUS’s modest mission is to provide a hands-on entry point for understanding and collective experimentation.
Please contact me for more information and to be notified about future SISYPHUS workshops and the open source code release.
SISYPHUS uses an artificial intelligence technique called Q-learning (originally developed in 1989 by Chris Watkins), which is a type reinforcement learning. In reinforcement learning, the robot learns by doing. SISYPHUS’s goal is to maximize its forward movement, as sensed by the mouse wheel that it drags along the ground.
Each time the robot moves its arm it receives a reward – positive for traveling forward, negative for traveling backward, or neutral for no movement. Over time the robot learns what rewards to expect for each action and uses this information to build up a kind of map for which sequences of actions will maximize the forward travel.
At first the actions are mostly random, giving a process of exploration. As the learning progresses the best known actions are followed in a process of exploitation. The robot is always learning – if the same actions start to have a different result (for example if the terrain changes) new behaviour can result.
- Arduino Uno
- 2 servo motors (TowerPro SG90 9g servo clones)
- Rechargeable battery pack (4x AAA NiMH flat pack)
- 3D printed parts
- Hacked PS/2 mouse (distance encoder wheel)
- Prototyping shield
- RGB LED
- Jumper wires
- Header pins
- Hard wire (for “claw”)
- Wooden dowel (wheel axle)
“SISYPHUS: Build a robot that learns” (workshop), We Make It #1, SPEKTRUM, Berlin, Germany, 2016
Project lead: Michael Ang
3D Design / Mechatronics: Sean Malikedes
Q-learning code adapted from example by Andrew August