SISYPHUS (Our Learning Robot) – 2016

SISYPHUS is a robot that learns to crawl using a simple AI algorithm called reinforcement learning. The robot tries random actions at first and learns if it is moving forward or backward. Over time it connects actions that move it forward.

SISYPHUS was created in response to the dramatically increasing role of artificial intelligence algorithms and robots. Our robot is designed to be simple enough for us to understand and build while having the capability to learn and surprise us. Where highly advanced robotic and AI systems may give us a sense of loss of control, SISYPHUS’s modest mission is to provide a hands-on entry point for understanding and collective experimentation.

Please contact me for more information and to be notified about future SISYPHUS workshops and the open source code release.

SISYPHUS Robot by mangtronix

Algorithm

SISYPHUS uses an artificial intelligence technique called Q-learning¬†(originally developed in 1989 by Chris Watkins), which is a type reinforcement learning. In reinforcement learning, the robot learns by doing. SISYPHUS’s goal is to maximize its forward movement, as sensed by the mouse wheel that it drags along the ground.

Each time the robot moves its arm it receives a reward – positive for traveling forward, negative for traveling backward, or neutral for no movement. Over time the robot learns what rewards to expect for each action and uses this information to build up a kind of map for which sequences of actions will maximize the forward travel.

At first the actions are mostly random, giving a process of exploration. As the learning progresses the best known actions are followed in a process of exploitation. The robot is always learning – if the same actions start to have a different result (for example if the terrain changes) new behaviour can result.

Design and Electronics

The robot’s code runs on an Arduino Uno. The arm is made of two servos and 3D printed parts. The robot measures the distance traveled by reading a hacked mouse – the mouse is flipped over and the mouse wheel travels along the ground.

The 3D printed parts are designed in FreeCAD and are open source. The FreeCAD files are parametric so they can be adapted relatively easily, for example if you want to make a different shape of arm or use different servos. Please contact me if you’d like to build the robot.

 

Components:

  • Arduino Uno
  • 2 servo motors (TowerPro SG90 9g servo clones)
  • Rechargeable battery pack (4x AAA NiMH flat pack)
  • 3D printed parts
  • Hacked PS/2 mouse (distance encoder wheel)
  • Prototyping shield
  • RGB LED
  • Jumper wires
  • Header pins
  • Hard wire (for “claw”)
  • Wooden dowel (wheel axle)

Presentations

“SISYPHUS: Build a robot that learns” (workshop), We Make It #1, SPEKTRUM, Berlin, Germany, 2016

Credits

Project lead: Michael Ang

3D Design / Mechatronics: Sean Malikedes

Q-learning code adapted from example by Andrew August

References

SISYPHUS Robot by mangtronix