Throughout the prototyping and ideation of my final project, I wanted the physical constraint of an Arduino-based input to heavily influence what the function would be. Because of this, I was not entirely sure what exactly the users would do, other than interact with an audiovisualizer-driven environment.

How It’s Made

Construction around a microphone for the audio visualizer was a fun design challenge. My cheap gooseneck mic had a comical useable tactility to it, allowing it to bend and stretch any way I saw fit. To turn it on and off, I thought it would be neat to have a photosensor wiring up along the gooseneck. That way, whenever the user puts their mouth towards the mic, it would detect a change in lighting (darkening). The base was additionally bulky enough to rationalize having the Arduino and breadboard next to it, forming a horn shape when placing them all together. To decorate the shape, I had used one of my dad’s empty yogurt cups to puzzle in the electronics, and a gutted paper towel tube to cover the microphone and photosensor, which allowed for more accurate detections of passerby. I then covered the outside with inverted Oreo lining for a metallic sheen, making it a ‘silver horn’ (and a convenient excuse for a snack). The bottom was draped with black cloth to cover the excess bulk and wiring, akin to a puppet’s bottom black fabric cover. After completing the horn, I immediately knew I wanted its user to fill the role of a bard in the eerie digital environment I had created.

To complement this, the other user controlled all of the monitors in the scene that surrounded the horn player, emulating a possessive pied piper effect. They control this by holding a drawing tablet decorated as a TV and dragging a cursor hand controller (containing a tablet pen to brush across the tablet screen’s surface) horizontally and vertically. This rotated both the collective mass of monitors and their own individual rotations. This way, both the environment and horn-player are in digital interaction with one another, sharing the same screen and subjecting themselves to each other’s reaction, resulting in a shared (somewhat limited) metaverse.

The microphone was sent into Unity and underwent many post-processing effects to further abstract the already abstracted tube distortion. I then connected it to my shader audiovisualization tool and made audio-reactive materials. These materials were then placed on a mix of new and old assets.

To create the horn player, I had found an old free 3D model of a spooky rendered scene, extracting the base body of the posed figure subject. I then painstakingly brought all of its verts into a T-pose and fixed its UVs in order to stitch on an old computer monitor that I had made modeled beforehand onto its neck, making it a monitorhead. After altering Mixamo animations to make a some-what believable horn-playing animation, I had to further expand my audiovisualization tool to actually work on imported complex 3D animation. Making my audiovisualizer tool work on skinned meshes was a great learning achievement, as I had broadened its applicable use and expanded the possibilities of future projects.

The rest of the input handling (rotation) was done with C# in the Unity game engine.

So What?

The abstract environment is meant to further encourage a new language of communication between the users. I wanted the strange and haunting sounds and possessed monitors’ chaotic carousel to be entirely phenomenological, having the users learn how to interact with one another in an entirely new realm with no convenient bias to hide their true intents when communicating and interacting with one another.