The XR ball game project assumed the task of creating a mediated reality environment that utilized the unique experience of being in the inflated plastic ball (seen below). This was achieved by creating a novel Head Mounted display that utilized the XIAO ESP32 C3 board’s ESP NOW Communication protocol to transmit data from a 10-axis IMU unit in real time. Using Quaternions, the Roll, Pitch, and Yaw of the IMU are calculated in real time via the gyroscope and acceleration data streaming for the unit. These values are processed by the ESP32 affixed to the IMU and sent to a separate ESP32 unit connected to the ST7789 LCD display unit on the head-mounted display. This allows for the data to be visualized in a user-friendly way via a flight simulator-like user interface that ‘gamifies’ the experience for the end user. Data is also simultaneously collected and stored by the team for further processing. Pictured below are images of the system itself and examples of the graphical representations of the collected data.
