For our final project, we explored the creation of “live” music using sensors that monitor the user’s movement and heart rate. The program reads data from a live body monitor, and queues music samples/overlays depending on the action performed. Additionally, the overall bpm of the music is synced to the user’s heart rate and displayed using a visual indicator.

The monitor consisted of an Arduino nano, 3-axis accelerometer, wireless module, led, and battery. The monitor had an external connection to a heartbeat sensor that would wrap around the user’s finger. The Nano reads in the analog waveform from the heartbeat sensor and uses peak detection to determine the start of each pulse. The start of each pulse is then used to determine the period between pulses in terms of beats per minute. This is averaged to smooth the bpm data. The monitor is able to detect gestures from the accelerometer by taking the discrete-time derivative of the acceleration, this is the jerk of the system. The jerk is then averaged just like the bpm data using a moving average and motion is detected when the value peaks. A debounce is added so that only the desired direction is detected, otherwise, a movement would give two triggers since there is a positive and negative peak in each direction. The bpm data and jerk values are then transmitted wirelessly to another nano with a wireless module. The led is blinked each peak that has been detected for ease of debugging and a cool effect.

Below is the cad of the monitor with all components placed inside a housing.

Above is a picture of the fully assembled monitor with all components soldered together inside a 3D printed housing.

Here is a link to the source code running on the monitor https://pastebin.com/nUEucsSw

To queue the music samples so they correctly synced to the background beat, a single metro was dynamically changed to reflect the user’s bpm. Samples that needed to match the beat were played on the next measure of the song, using a simple counter. Additionally, the samples were sped up and pitch corrected to reflect the current bpm.

 

Here is the final patch.

 

Below is a recording of the monitor and patch in action.