A New Direction
Over the weekend, I was reminded of this project that I saw on Harvard’s campus:
I remember feeling the “I should make that” itch, prompting me to quickly forget about the other two projects and feel the need to run after this instead. To scope the project for the rest of the term, I started doing research into how this project was originally built. I believe it uses a camera to detect when a subject is in front of the panel, a machine learning model to detect the person, and a computer to run inference on the model and convert that into what dots need to be flipped to show the person.
Progress
In light of this, I started doing some work producing a version of the project that I could do using a lot of what I already have on hand. I believe I can use my laptop’s webcam (as well as its compute for the ML model), send the information to an ESP32, and then display the corresponding commands to an LED panel. I did some looking into making my own flip dot panel or buying one, and turns out they are super expensive (~250 on eBay) and convoluted to build (multiple youtube channels have spent months making their own, without much real progress). So I think for this project, I want to focus on developing a wireless LED panel that gets real-time depth information from your computer’s webcam.
I got started messing with possible models and landed on DepthAnythingV2 for it’s size to performance ratio. I’m looking into getting it running faster, but I was super happy to get it running in real time:
Shopping List
Like I mentioned before, I think this project can be done with an ESP32 and an LED panel. I’m lucky to have a spare ESP32 in my dorm, but needed to order the linked LED PANEL (which will hopefully be here later this week). The next step is to start messing with the communication protocol and getting the ESP to properly manipulate the LED panel.
(Update) The LED panel came in broken… I love getting sketchy electronics from Amazon:
I like your new direction a lot, I think it is really interesting. Interactive art displays like this are always fun and I can tell you are really excited about this. I think the LED approach discussed in class has a lot of promise, I would suggest trying to get some cheap ones to play around with before you move forward.
While your other two ideas would have been really cool to see, I think this new direction would be a lot more reasonably attainable in your timespan and budget and will likely turn out just as cool. I was wondering, what do you plan on having as your final piece of art to present? Will it be an interactive piece that you will bring to the class to play with, will you install it somewhere that the public can interact with it, or will it be a video showing its use?
I love your new project! I know you said you were planning on doing a binary on/off situation, but if time allows and you use lights that can dim(like those cheap LED strip lights) it could be cool to have a couple of different ranges of brightness for different depths to give a more complex result! Without that, though I still really love this idea, your progress is already very impressive and I can’t wait to see it begin to come to life!