I may have bitten off more than I could chew with this one. I decided to make my Augmented Selfie a reflection of my current state, very fuzzy and uncertain while still blocked in (it comes easy when one tries to overload and lead a new game development team simultaneously) – and to go about it, my first instinct was to make a large cube, with a static- covered view of the user’s face on all sides, follow the user around and rotate with them.<\/p>\n\n\n\n
I first tried to use the standard [cv.jit.faces] patch to do some rudimentary tracking, but quickly realized that it didn’t output the rotation of a face, only the location. I then switched over to a FaceOSC patch I found, which did capture the rotation, location and scale of the user’s face map – but because FaceOSC is an external program, it hogs the webcam’s output stream, and prevents Max from accessing the video feed!<\/p>\n\n\n\n
After trying in vain to extract the webcam data from FaceOSC, I ultimately just switched back to cv.jit.faces as the face detection method. I had a couple challenges with this: a) I had to force [cv.jit.faces.draw] to output the dimensions of the face to manually calculate scale, and b) I had to map the cube’s rotation to how far the face was from the center of the screen. As a result, lots of painful math trickery ensued. But this is the final result:<\/p>\n\n\n\n