Edit: The video document is the last video near the bottom.
My project’s metaphor can be either a blindspot or tunnel vision. It intends to block the viewer’s focus wherever they look while simultaneously giving them something they want to look at. The largest challenge by far was tracking eye movement. I ended up with two solutions that both use cv.jit.track to track specific points.
The first solution was to simply give the camera something to track that’s more defined than an eye. I took a toilet paper tube to use as a scope, and it works fairly well with a bit of movement scaling. I also recently got another potential scope from some cardboard tubing that was used in packaging. The other solution was to use tracking points on the eyes and scale the movement drastically. The result is quite interesting and sometimes a little out of control, and definitely needs more fine tuning, but it works if it can track the minute eye movements. There was an issue of the focus flying off the screen, completely unretrievable, so I set up a sort of ‘reset’ where hitting the spacebar would center the white dot and make all subsequent movements relative to the reset.
I set up cv.jit.track to take two points, one for each eye. I took the average of their coordinates to find the focus, and I used that to position the large white spot. With the tube, I high recommend doing approximately the same, using the edge of the tube and the hand holding it for the tracking points, It seems to help keep it stable.
I added some flickering circles whose positions are relative to the visual focus. The effect works better with the tube.
The tracking points can get a little off and I couldn’t find a reset message for them, so resetting them requires clicking on the image again. Since my patch essentially has two possible methods of tracking and scaling, I set up a switch with the left and right arrow keys as controls to swap the type of scaling. Left is for eyes, right is for the tube.
I definitely could still clean up my patch. Most things, like the scaling and the objects, could fit nicely in subpatches.
I had a bit of fun figuring out how to track eyes with cv.jit.track. There’s definitely a better way to do it, but I kind of like the method I came up with. There are a few more things I added, like different video sources for some variety in the kaleidoscope effect. That can be changed with the up/down arrow keys.
Click to view
Lastly, I noticed a few issues when others tested my project, particularly with the tracking. I’m adding tabs to the tube for better tracking on that, and a few further instructions are required, such as no sudden movements. It works better for me because I’m the one who set it up, but it’s going to take a lot of extra work to get it to somehow adjust to different people.
Documentation Video:
Here’s the text file and dot png file for anyone interested in seeing if it’ll work for them. It requires the cv.jit package and, of course, a camera. Also, far, far, far to the right of the patch are some basic instructions on how to use the patch since I had once thought to set it up in presentation mode but then changed my mind because things got weird. Lastly, there’s a loadbang that’ll start everything up immediately. Remember that the png has to be in the same folder as the patch, but you can always drag it in and then connect the convenient bang button that’s already in there above where it’s supposed to go. Good luck! Let me know if there are any other issues I forgot about or don’t know about.
