Aidan Sensiba – Computer Vision Gestures

This project is somewhat of a mix between a basic piano and a theremin. I enjoy playing the piano as a hobby, and I wanted to incorporate that into my work through physical movement and audio. In this, there are five tones ascending a musical scale that can be played by moving the orange paper note horizontally across the screen. This can be done in any order or rhythm, and the volume can be controlled by moving the note up and down.

One of the challenges I faced was getting the computer vision to sync up correctly with a certain object. For this, I used the color tracking patch, where I selected the orange color of the note that the program would follow. I found that uniformly colored objects under bright, white lights were fairly easy to track. Another obstacle was getting the tones to change pitch and volume smoothly, which just took some tweaking to fix.



The Max patch

8 Comments Add yours

  1. Adam Moran says:

    I find it a bit odd that the sound gets quieter the closer it is to the top. What was the reasoning behind that decision?

    1. Aidan Sensiba says:

      If I have to hold my arm up high for a while it can get kind of tiring and block my view of the screen, so I found it more convenient to have max volume on the bottom, since that’s what I was playing at most of the time.

  2. Noah Hillman says:

    I really like the use of a colored note as the control object and the fast response of the notes.

  3. Isaiah Fleischer says:

    I really like how the sound gets quieter as you raise the note. It really lets the user play with the patch more than they could if they were restricted to one volume. Did you consider any other sounds, like guitar, choir, or strings? It would be neat to change what synth the note creates, or even using two different notes, each hooked up to different sounds.

  4. Jarod Romankiw says:

    Nice use of the colored note. This a cool way to make the message of the patch tangible. It’s cool because it’s very close to a theremin, but also very unique. If I were to add something to this, I would add different types of instruments with different types of paper, and different colors. Then use the RGB to differentiate between them all.

  5. Joshua Galang says:

    It’s good to use the colored note as a tracker, this is an excellent project. Jarod’s suggestion is excellent, using different colors to trigger different instruments for a new level of interactivity.

  6. Minh-Chau Doan says:

    I really liked this project and wish there were more notes to play but it makes sense that the patch would glitch out from how crowded that would be. I really liked that you could change the volume based on the placement of the note!

  7. Han Liu says:

    The idea that the music scale and volume controlled by the displacement of a paper note is very impressive. I would suggest adding some buttons on the top to choose from different order of music scale or timbre of instruments.

Leave a Reply