Bootladder Engineering

Toy Store Prop

Intro

This was one of my favorite client projects. It was a lot of fun, and I got to do something for the kids.

The Client

The client here is a toy store which has an experience room full of fun props for kids to engage with. Most of the props are theater style with lights and maybe some moving parts or switches.

There was one prop which was slightly more technical and was causing several issues. Some of them were "showstoppers".

Prop Description

The system consists of a Unity based game controlled by a Bluetooth BLE custom controller. The controller is an ESP32 platform, with a push button and an accelerometer (IMU) for orientation. A hall effect sensor is used to detect and reset the zero yaw position.

The software is a simple Arduino sketch that sends the IMU data and button press events to the Unity game.

  • Unity
  • BLE
  • ESP32
  • Arduino
  • MPU6050
  • Hall Sensor

Problems

The client had two major problems.

  • The ESP32 would fail regularly, requiring replacement.
  • The IMU was drifting, affecting game play.

Approach

Once I was given the information about these problems, I went up to Boston where the installation was at the time, and investigated the problem. I was able to study the circuitry and determined that one of the pins used on the ESP32 was an incorrect choice.

PICTURE Pin out

Drifting

I was thrown off for a while with the drifting because it was happening on the yaw axis. Because its a 6 axis, the yaw is not absolutely determinable because it is normal to gravity. However, it turned out that this was not the problem, and simply controlling the calibration process was enough.

Brownout

Brownout

I discovered a brownout loop. At first I thought this was due to the long USB cable length without a power supply. This may be related, but the real problem was that the IO pin was burnt out.

ESP32 Pin Choice

ESP32 Pin Choice

It turns out that you have to be very careful selecting IO pins for the ESP32. In fact, if you are unlucky enough to select a really bad choice, you can actually permanently damage the ESP32, which is what happened here.

Zero Yaw Crossing

Zero Yaw Crossing

Absolute pitch and roll are determinable by referencing to the gravity vector. But, absolute yaw is not. With a magnetometer it is possible, but only to align wih the Earth's magnetic field. Inside a big building with lots of interference, this is not reliable.

So, a neat trick is to reset the yaw to zero when it crosses a known point.

In this case it was done with a hall effect sensor and a small magnet.

This is a good idea but there can be issues with implementation.

Wiring

I knew I would be responsible for completing the installation. This is one of those things that always gets lumped into the job of embedded systems engineer. So many little things come up. It's too technical for an electrician, but too "manual" for most engineers.

Site Visit

When I visit a client site, I always come prepared.

hello

Solution

I presented my findings but eventually was asked to rebuild and reinstall these ESP32 modules under a tight deadline. None of this was technically challenging to me, but the project as a whole was. I had to do a complete physical build, including enclosure, and I had to do the installation at a customer site with limited access and strict deadline, and I also had to make sure that my work was going to last over time. There were a couple of hiccups which resulted in three separate trips to the job site.

I had a decision regarding the construction. Do I make PCBs, and what enclosure should I use? I chose to build these by hand because I expected there to be issues at installation time. Doing quick and dirty modifications are easier with protoboard and through hole construction.

PCBs of course have lead time, so I wanted more control of when the constructio would be finished. However, I overestimated my speed at the soldering bench, and it took more than a full day.

There were various other issues that came up. For example, one of the push buttons had been damaged.
I brought a spare one from my inventory that happened to fit, but it was made of plastic and it failed literally the next day after use.

Also, there were issues on the Windows side of setting up the Unity gameplay. There was a conflict with the internal bluetooth hardware and stack, vs. the BLE dongle plugged into USB. The solution is to disable the internal bluetooth but it's not obvious, the menu is hard to get to. Hard to do this when the host ie. laptop is mounted on a drop celiing : ).

BLE Gamepad

The library works out of the box It was tricky to test because of the profile stack. Its not a generic GATT device. This means that even on my arch linux, it is thrown off, and to talk to a BLE Gamepad from the host ca be tricky.

Unity

I didn't touch the Unity game source code but there was something about it that was relevant to my concern. Which is, how it responds to gamepad input. How will it deal with input from the gamepad that is fast, slow, finer resolution, coarser. What if the difference between 2 inputs are large. This affects gameplay. Does it react to the input instantaneously? In this case it was doing a "smoothing".

Bluetooth with noisy environment

The client and I considered trying a USB to ethernet system, which would allow serial control of the ESP32 without bluetooth. Because this system will be installed in many different locations ie. environments, some may be noiser than others.

Calibration

IMU's need to be calibrated. Also the Yaw offset in ths applicatio needed to be set.

Conclusion

At the end, the job was done on time and I am very happy to say that I never heard back from the client except one time to tell me that there were no more problems experienced.

Errosdfdr pretty-printing HTML: not well-formed (invalid token): line 42, column 45 Toy Store Prop - Bootladder

Toy Store Prop

Intro

This was one of my favorite client projects. It was a lot of fun, and I got to do something for the kids.

The Client

The client here is a toy store which has an experience room full of fun props for kids to engage with. Most of the props are theater style with lights and maybe some moving parts or switches.

There was one prop which was slightly more technical and was causing several issues. Some of them were "showstoppers".

Prop Description

The system consists of a Unity based game controlled by a Bluetooth BLE custom controller. The controller is an ESP32 platform, with a push button and an accelerometer (IMU) for orientation. A hall effect sensor is used to detect and reset the zero yaw position.

The software is a simple Arduino sketch that sends the IMU data and button press events to the Unity game.

  • Unity
  • BLE
  • ESP32
  • Arduino
  • MPU6050
  • Hall Sensor

Problems

The client had two major problems.

  • The ESP32 would fail regularly, requiring replacement.
  • The IMU was drifting, affecting game play.

Approach

Once I was given the information about these problems, I went up to Boston where the installation was at the time, and investigated the problem. I was able to study the circuitry and determined that one of the pins used on the ESP32 was an incorrect choice.

PICTURE Pin out

Drifting

I was thrown off for a while with the drifting because it was happening on the yaw axis. Because its a 6 axis, the yaw is not absolutely determinable because it is normal to gravity. However, it turned out that this was not the problem, and simply controlling the calibration process was enough.

Brownout

Brownout

I discovered a brownout loop. At first I thought this was due to the long USB cable length without a power supply. This may be related, but the real problem was that the IO pin was burnt out.

ESP32 Pin Choice

ESP32 Pin Choice

It turns out that you have to be very careful selecting IO pins for the ESP32. In fact, if you are unlucky enough to select a really bad choice, you can actually permanently damage the ESP32, which is what happened here.

Zero Yaw Crossing

Zero Yaw Crossing

Absolute pitch and roll are determinable by referencing to the gravity vector. But, absolute yaw is not. With a magnetometer it is possible, but only to align wih the Earth's magnetic field. Inside a big building with lots of interference, this is not reliable.

So, a neat trick is to reset the yaw to zero when it crosses a known point.

In this case it was done with a hall effect sensor and a small magnet.

This is a good idea but there can be issues with implementation.

Wiring

I knew I would be responsible for completing the installation. This is one of those things that always gets lumped into the job of embedded systems engineer. So many little things come up. It's too technical for an electrician, but too "manual" for most engineers.

Site Visit

When I visit a client site, I always come prepared.

hello

Solution

I presented my findings but eventually was asked to rebuild and reinstall these ESP32 modules under a tight deadline. None of this was technically challenging to me, but the project as a whole was. I had to do a complete physical build, including enclosure, and I had to do the installation at a customer site with limited access and strict deadline, and I also had to make sure that my work was going to last over time. There were a couple of hiccups which resulted in three separate trips to the job site.

I had a decision regarding the construction. Do I make PCBs, and what enclosure should I use? I chose to build these by hand because I expected there to be issues at installation time. Doing quick and dirty modifications are easier with protoboard and through hole construction.

PCBs of course have lead time, so I wanted more control of when the constructio would be finished. However, I overestimated my speed at the soldering bench, and it took more than a full day.

There were various other issues that came up. For example, one of the push buttons had been damaged.
I brought a spare one from my inventory that happened to fit, but it was made of plastic and it failed literally the next day after use.

Also, there were issues on the Windows side of setting up the Unity gameplay. There was a conflict with the internal bluetooth hardware and stack, vs. the BLE dongle plugged into USB. The solution is to disable the internal bluetooth but it's not obvious, the menu is hard to get to. Hard to do this when the host ie. laptop is mounted on a drop celiing : ).

BLE Gamepad

The library works out of the box It was tricky to test because of the profile stack. Its not a generic GATT device. This means that even on my arch linux, it is thrown off, and to talk to a BLE Gamepad from the host ca be tricky.

Unity

I didn't touch the Unity game source code but there was something about it that was relevant to my concern. Which is, how it responds to gamepad input. How will it deal with input from the gamepad that is fast, slow, finer resolution, coarser. What if the difference between 2 inputs are large. This affects gameplay. Does it react to the input instantaneously? In this case it was doing a "smoothing".

Bluetooth with noisy environment

The client and I considered trying a USB to ethernet system, which would allow serial control of the ESP32 without bluetooth. Because this system will be installed in many different locations ie. environments, some may be noiser than others.

Calibration

IMU's need to be calibrated. Also the Yaw offset in ths applicatio needed to be set.

Conclusion

At the end, the job was done on time and I am very happy to say that I never heard back from the client except one time to tell me that there were no more problems experienced.