A Balancing Robot Story
My journey into robotics as a career
This is a story I have told probably 100 times but never took the time to write it down. I have many interesting stories like this one, but I figured this would be the best topic for my first Medium article. I never intended to end up doing robots as a career, but it just turned out that way — Maybe the surprises are why I love robotics so much!
Twenty years ago, just after the dot.com meltdown, I was working as a consultant and had lots of free time on my hands. I started spending much more of my time indulging in my hobby of building robots and subsequently met up with other like-minded people at the Silicon Valley Home Brew Robotics club (www.hbrobotics.org). One of the first people I met in the club was Bob Allen, an electronics engineer who had a machine shop in his garage and could make amazing mechanical things. He had built some combat robots in the past and participated with his robot Viper in both the TV shows Robotica, and Robot Wars. We were both crazy excited about working together to do more interesting and complicated robotic projects where we could utilize all three disciplines of robotics: mechanical, electrical, and software engineering.
Back then, my mechanical engineering skills were severely lacking. Robot-suitable mechanical parts were hard to come by, but combat robot parts were more readily available due to their popularity in TV at the time. I had a pile of these parts and was eager to put something together. I remember one afternoon going over to Bob’s house to discuss the possibility of doing a collaborative robot project using this parts pile. His first comment was:
Are you Crazy? You are going to destroy your house if you build it with that stuff.
He then proceeded to show me a video of a robot that could balance on two wheels. I thought, “Wow that’s an amazing thing. I have never seen anything like it before”. It was small and probably wouldn’t destroy my house. We wondered if we could build something similar.
It couldn’t be that hard, right?
Fast forward months later, we finally had a set of electronics together, the right motors figured out, and were ready to take a crack at making it balance and drive around. Then, the hard part came. Building the robot was one thing, but the software problem of making it balance was monumental. I think it took a good 3 months just to get it not to fall over, and many more months to make it drive around. The first robot was named “Bender” after the lovable beer-drinking robot from Futurama. It fell down so much, so it made sense to name it after a robot that was drinking all the time.
Silicon Valley used to have a huge collection of “technology” recycling places where you could go into a warehouse filled with old computer parts, motors, and wiring, and buy these used items for pennies on the dollar. I remember many trips with Bob looking for just the right parts to put Bender together. I have great memories of Bob and others in the robot club coming to a club meeting with some rare, interesting motor they found in a junk bin at one of these places. They are a dying breed of store. One of the more beloved ones, HSC Electronics recently shut down and they will be missed by the hobbyist’s who frequented them every weekend.
A Foray into Control Theory….
I didn’t know much about control theory when I started this project, but I learned fast. A balancing robot is essentially an upside down implementation of the classic inverted pendulum problem:
Every time I tell this story, people want to know the gritty details. “How did you do it? What parts did you use?” I could probably write a thesis on this topic alone, but for a Medium article, a summary is the best I can do here. I learned the hard way that, to make a balancing robot work well, the robot needs the following items:
- High Torque Gear Motors (100 RPM or less preferable)
- Encoder feedback on your gear motor
- IMU (Inertial Measurement Unit)
- A micro-controller sized CPU — Not much needed here.
These days, there are plenty of companies making 6 or 9-axis IMUs for use inside smartphones or other consumer devices. It’s funny to look up the Wikipedia entry for IMU and find all kinds of fun pictures from the Apollo space program. These days an IMU = Gyro+Accelerometer+Compass all fused together on a single chip device, or you roll your own by taking each sensor separately and fusing them together using math.
In 2002 there were very few options. I found some gyro’s at a surplus place that were made by Tokin electronics. They’re a single-axis piezoelectric device originally made for the inside of camcorders and computer mice. They were not widely used, but for some reason the surplus store had a big box of them.
Parallax which is a big educational and hobby robotics vendor had just started selling an accelerometer made by Memsic. It was 3 axis, but pretty noisy, and had linearization issues. What this means is if you tip the accelerometer on it’s edge, and slowly rotate it through 360 degrees, you should be able to draw a perfect circle from the data coming out. The data from this accelerometer looked more like a garden squash than a circle. The big software challenge is you need a control loop where you read the IMU, remove all the drift and bias, and then move the wheels in the correct direction to keep them underneath the robot so it doesn’t fall down. It turns out that the speed of this control loop is not terribly critical. On all my early attempts, about 30hz was enough to keep the robot stable. It still sounds pretty easy as I am describing it, right? Once you dig into the details, it gets tricky.
In 2020 the IMU units you can buy off the shelf, are pretty inexpensive, and not as finicky. In 2005, this was not the case. We had a single-axis gyro, and a two axis accelerometer. Both of these devices had measurement issues. The gyro is the most important measurement device that you use for keeping the robot upright. It gives you the rate of rotational acceleration as the robot starts to fall. The problem is, if you just put a gyro on a non-moving surface, like a desk and start taking readings from it, the zero point will slowly start to move or “drift”. Compensating for this drift is the hard part. How can you measure a sensor when zero is constantly moving? The solution to this problem is to take another sensor that doesn’t move as much when you are sitting still, and reference to its zero instead to remove the drift. There is another problem though. Both sensor measurements, and total sensitivity move around with temperature. Whew!
How can I get stable measurements I can use to balance my robot?
The answer lies in an algorithm from statistics and control called a Kalman Filter. You take these measurements, run them through a Kalman Filter, and you can produce estimates of where you think the true measurement actually is.
My math skills were pretty rusty from college, but when I dug into how to implement one of these, I really had to go back to basics and learn what to do. It took quite a bit of work to get to the point where I could build a simple Kalman filter algorithm that I could use to fuse together my janky gyro and accelerometer. Once I had it working I could see on the bench that when my little IMU board was not actually moving, the output of my filter would stay stable, and when it was moving, the measurements would move in the correct direction.
Next, I needed a control theory method to keep the robot balancing. There are many different ways to go, but I wanted something simple. My brain hurt after figuring out the whole Kalman filter thing. The simplest way to go is a PID controller.
It turns out that a PID controller all by itself is usually just enough to keep the robot standing, but doesn’t necessarily deal with all the problems of driving around, carrying an offset load, or dealing with all the other issues related to bumping into something. I learned I had to add lots of extra terms to my PID equations, as well as software filters, to begin achieving great results. After all my experiences with balancing robots, here are the hard problems to demonstrate solutions to — If you have a robot yourself, try these practical tests and see if you can do it:
- Offset Load — Put a weighted thing on top of your balancing robot. Will it run away, or keep it’s position on the ground?
- Balancing on an Incline — Can you put your robot on a ramp and will it keep it’s place?
- Mass down low — The typical piece of advice balancing robot people will give you, is that it is easier to make the robot balance with its center of mass as high on the robot as possible. It’s sort-of like balancing a broom on your hand. Most people do it with the wicker end up. This is like doing it with the wicker end down.
- Driving at constant velocity — Can it go a constant speed on the ground including up and down ramps?
- Driving over obstacles — Can it drive over a door threshold or an extension cord?
- Running into obstacles while driving — Does it bounce off and recover gracefully, or does it bounce backward, wind up, and blast back into the obstacle?
It was finally time to make a smaller, improved version of Bender that could do more things. One thing I learned from all my work was that bigger balancing robots are usually easier to make balance. The move slower, so the control system doesn’t need to be as exacting in making minor corrections. Making a really tiny balancing robot becomes harder and harder the smaller you go. In the TV show Futurama, Bender has a brother named Flexo. So, it made sense to name the new robot after him.
Since I now had a PID algorithm that could balance the robot with an offset load on top of it, and also be able to drive up and down ramps, I could easily mount a moving, pant-tilt-camera rig to the top of the robot without any fear of the motion of the camera knocking the robot over. We used a color tracking camera, and tuned it to track the color red. So you could put any red object in front of it, and it would slowly try to work it’s way over to you. Once it got close enough it would just stand there and stare at you. We took him to the Robot Club once he was working, and there was a kid there wearing a red sweatshirt the robot really took a liking to!
Once we knew how to do it, making more balancing robots seemed easier. We build an off-road version of Flexo called Big Red, and another version called Fat Albert that could get up on it’s own from laying on the floor.
Fat Albert had a unique mechanism and software implementation that allowed for transitioning from the floor to standing and then back to the floor again. We applied for multiple patents on the whole setup and have kept them active over the years, but still haven’t figured out what to do with these patents. (If you have an idea for them, feel free to reach out to me.)
Bob thought we could maybe find some companies who might be interested in paying for our expertise in developing these kinds of things and possibly the patent too! He bought a booth at the first RoboNexus trade show held in San Jose in 2004. On the first day of the show, we were standing in our booth with Flexo putting on his balancing robot show, and who should come strolling down the aisle riding a Segway, but Dean Kaman himself. He rode up, and stopped for a moment, and said jokingly, “Wow! Look at that! I have never seen anything like that!” 😅 — Although, it wasn’t enough to actually get him to dismount from his scooter.
On the second day, we met a bunch of enthusiastic engineers from Hasbro. They were looking for more interesting robotic toy products they could add to their line as well as some help with more advanced sensors in toy development. We were invited out to Rhode Island to give a presentation to a larger group of the engineering team on our capabilities. Bob asked “Do we need to prepare a more formal presentation?” The response was, “Just show up and bring some robots to show and tell.” When we arrived at the corporate HQ, we were shuttled to an enormous conference room that looked like it could seat 100 people, and there were 20 people sitting in there waiting to hear our “presentation.” OH NOOO! We thought we had completely blown it, but we just turned on the robots and Bob started talking. I am normally a very loud, talkative person, and Bob said to me it was the first time he had seen me at a loss for words. Pretty soon, the crowd warmed up, got out of their seats, and started coming to look at the robots up close, and it all worked out fine.
We signed up with them to help, and a company was born! Bob and I started a robotics consultancy called OLogic, Inc. in 2005. Hasbro was our first customer, and from there we were able to take on more and more interesting robotic projects as well.
Hasbro never lost sight of the thought that this balancing robot idea could be a great product concept, so they made the investment to start turning our prototype robot, Flexo, into something that could be sold as a toy/consumer product. The first step was to get rid of the surplus parts and start to integrate consumer-grade components and motors. The first prototype goal was to get the robot drive train up and working. There were two robots made: one was a full prototype that was 3D printed and seemed fragile as glass, and the other was just the drive train with our existing electronics mounted on it.
We broke both prototypes more times than I can count. Every time it broke, I would call up Bob, and he would come over and we would figure out a clever way to glue it back together. Eventually after tons of trial and error we had something working, so then it was time to make the real thing. 3D printing back then was fairly new and not as easy to access as it is today. (I think it’s maybe a topic for a whole additional article on how the robot got tooled up in a Chinese factory and ready to start manufacturing, so I will leave that story for another day.) After a few trips to China, meeting some fantastic factory people, and a bunch more work, we finally had the real product ready to go.
The product was to be named A.M.P. — Automated Music Personality. Or AMPbot. It was not only a balancing robot, but it had a pretty sweet boom-box inside it, so it was a speaker too. It had software in it where it could vibe to the music being played through the unit.
At this point it was 2008 and it was scheduled for release in early 2009, but that was not to be. The financial crisis of 2008 changed the priorities for all the toy companies, and after only about a dozen pre-production prototypes were made, AMPBot was cancelled. 😢
By 2010 we were wishing there were ways we could revive it. We got permission from Hasbro to take it to market on our own if we could find enough investment to do it. We reinvigorated the stock of limited prototypes we had built, so we had many to show and demo.
We took AMP to the Demo Conference in 2010 and tried to launch him as his own project. We found lots of robot enthusiasts who wanted him in their homes. I mean, who doesn’t want a cool music playing robot?
Here is a full description from IEEE Spectrum on AMP at the Event.
We attended multiple trade shows to attempt to find interest for making AMP. Here I am describing AMP at CES in 2012 to TMC.
To keep OLogic going we spent our time picking up consulting jobs while we tried to figure out how to get AMP out the door. We were able to get a one-off project to build some little balancing Android Robots for Google, and tour them around to various events such as MakerFaire, Google IO, Google Sandbox, and Google Zeitgeist in London. Bob and I had lots of fun, like a travelling circus bringing these robots to all these events. Google is a fantastic customer, and amazing to work with.
We used our balancing robotics expertise to help out many other companies in Silicon Valley get their balancing robots debugged and working well. Many who will need to remain nameless, but we had lots of fun helping them get their project working well.
There is both a happy and sad ending to this story about Balancing Robots. I made many friends along the way and started a cool robotics consultancy that is thriving today, however, it seems AMP was never meant to be. The prototypes still adorn our lobby at OLogic, but we have since moved on to other projects, and for now my balancing robot saga is over. Bob Allen has since retired from OLogic, but he can still be seen now and then at a Home Brew Robotics Club meeting, as his love of robots lives on .Now in the age of COVID the club is meeting virtually, and I highly recommend you check it out (www.hbrobotics.org).
You might catch me or Bob at the Robot Club indulging in our favorite pastime, robotics!
It’s funny, when I look on the internet and do a Google search today for “Balancing Robot,” there are dozens of Arduino kits you can buy that just do it with all the hard stuff figured out for you. Back in 2003 if you searched for such a thing, it yielded very few results. You had to bake it yourself from surplus parts you dug up at an electronics junk store. If you are interested in building a self-balancing robot, I would highly advise you take a look at one of those easy-to-build kits to get you started. Oh my! How the world has changed in 20 years.
Now that you have heard my story, here is a quick shout-out to my friend Ryan Hickman wrote this great sequence of articles about his robotics start-up, and he was a big inspiration to get writing. If you haven’t read his sequence of great articles, read them here:
About the Author
Ted Larson is the CEO of OLogic, Silicon Valley’s premier robotics and consumer electronics consultancy. Ted is a Software/Hardware Geek with 25+ yrs. of experience designing, and building consumer and robotic products. His current interest areas are robotics, artificial intelligence, and embedded systems in consumer electronic devices.
