This is the paper that accompanied my poster and demo at HAID ’09 (Haptic and Audio Interaction Design). You may see a picture of it in action and actually download the Mac OS X Audio Unit here.
You may view the complete paper here.
Abstract – We present a method for combining computer vision techniques with audio effect parameter control. In particular, we focus on tracking a user’s head position relative to the ca- mera and using this as a control signal for real-time audio effects processing and editing on digital audio workstations. The computer-vision techniques used are robust and use readily available, inexpensive tools, allowing our implementation to be used as an editing tool for audio engineers and composers, a component of larger interactive multimedia installations, and other applications. A test case is presented that utilizes OpenCV and Mac OS X Audio Units to control a resonant lowpass filter in real time.