This Laser Light Plane MT Table was developed as a part of my research with the Music Engineering Technology group at the University of Miami. The system leverages commercial, off-the-shelf technologies to make a robust multi-touch table interface and software development environment. For our inexpensive implementation, we used a technique called Laser Light Plane Illumination (LLP). A laser plane approximately one millimeter thick is created over a clear surface by surrounding the table with infrared lasers diffused by line generators. The infrared nature of the lasers ensures that this plane is not within the human visible spectrum,making it “invisible.” Fortunately, computer cameras do not have this limitation. When a finger breaks the plane over the clear surface, the infrared light is deflected down into a camera. This camera is fitted with a bandpass filter that renders the visible spectrum in such a way that is is sensitive exclusively to the wavelength of the infrared light. This deflected laser light is then interpreted by computer vision software as a touch. This has yielded a platform for which we can write applications for a number of audio research tasks, as well as applications for other disciplines and research problems. Current applications of the platform include interactive/generative music synthesis, a multi-touch pong game, gesture driven music composition, an audio mixing suite, and a musical-tactile rehabilitation program for subjects with Parkinson’s disease.
We present a system that leverages commercial, off-the-shelf technologies to make a robust multi-touch table interface and software development environment. For our inexpensive implementation, we used a technique called Laser Light Plane Illumination (LLP). A laser plane approximately one millimeter thick is created over a clear surface by surrounding the table with infrared lasers diffused by line generators. The infrared nature of the lasers ensures that this plane is not within the human visible spectrum,making it “invisible.” Fortunately, computer cameras do not have this limitation. When a finger breaks the plane over the clear surface, the infrared light is deflected down into a camera. This camera is fitted with a bandpass filter that renders the visible spectrum in such a way that is is sensitive exclusively to the wavelength of the infrared light. This deflected laser light is then interpreted by computer vision software as a touch. This has yielded a platform for which we can write applications for a number of audio research tasks, as well as applications for other disciplines and research problems. Current applications of the platform include interactive/generative music synthesis, a multi-touch pong game, gesture driven music composition, an audio mixing suite, and a musical-tactile rehabilitation program for subjects with Parkinson’s disease.
This four propellor helicopter – or quadcopter – is powered by an Arduino microcontroller. This was a project that I did for fun last year just to see if it was possible. The parts were chosen mostly from the suggestions of German sites (thank you, Google Translate). It features a lithium-polymer battery, an aluminum frame, a Seeeduino (I promise that spelling is correct) microcontroller, a five degree of freedom sensor, a secondary two-axis gyroscope, brushless motors, and an XBee radio.
The whole project was quite a learning process for me. I got to spend some time in the machine shop and even more time on Internet forums. I now have significant experience with interfacing microcontrollers to motors and sensors, XBee radio negotiation and communication, PID control loop theory, and miniature helicopter blade replacement. I also created a program for Mac OS X to receive the data from the XBee to display the quadcopter’s status in an easily readable format. Of course, that involved everyone’s favorite: socket programming.
After some hilarious, and some promising flights, I got the quadcopter to fly in a fairly stable manner. Unfortunately, I became too adventurous in my piloting and it currently has a broken front motor…
This device was commissioned by Arthur Campbell along with a composition by my research advisor, Colby Leider. It consists of a five degree of freedom sensor that is mounted onto the bell of Arthur’s clarinet that interfaces with an Arduino microcontroller. The microcontroller interfaces wirelessly through an XBee module to a SuperCollider program written by Mark Freeman, another member of our group. There is also a foot-switch that can trigger different sound samples during the performance.
As the performer moves his or her instrument, the sensor reports its positioning to the software. This software is directly in the loop of the performer’s microphone, so technically it could control any audio effect you could think to program. For our purposes, it allows the performer to spatially pan the sound.
Arthur Cambell, a virtuoso clarinet soloist, is currently taking the device around the country on a tour.
The name of the piece written by my advisor is “Twin Prime Conjecture.” Click here to listen to the debut performance.
From the program notes:
Colby LeiderTwin Prime Conjecture
Twin Prime Conjecture was composed for Arthur Campbell in 2009. Twin primes are successive pairs of prime numbers (those numbers that are divisible only by one and themselves) whose difference is two. Of the first few prime numbers (2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37…) only several adjacent pairs exhibit this property: (3, 5), (11, 13), (17, 19), (29, 31). The Twin Prime Conjecture is a famous unsolved mathematical proposal that there exist an infinite number of such pairs; in fact, number theorists have struggled to prove this conjecture since Euclid proposed it more than two millennia ago.
Twin Prime Conjecture treats this by successively considering some of these twin-prime pairs. Over the course of seven movements, the numbers comprising each pair are interpreted either harmonically (in the case of chords tuned to frequency intervals that can be expressed as the ratios of two integers, or N-limit just intonation) or rhythmically (in the case of the number of notes the clarinet plays, or N-tuplets). The work requires the clarinetist to mount a small wireless sensor to the bell of the clarinet, along with a microphone. The sensor transmits information about the movement of the instrument over time; specifically, it communicates the three-dimensional acceleration and two-dimensional gyroscopic tilt of the clarinet to a nearby computer via a wireless link. At times throughout the work, this information is used by the computer to process the sound of the clarinet in real time; at other times, the clarinet can serve as a virtual baton, whereby the performer can cue computer-generated sounds as an orchestral accompaniment.
This Audio Unit for Mac OS X combines computer vision head tracking with a low pass filter. For this simple example, the x-position of the head controls the cutoff frequency and the y-position controls filter resonance. I presented this work at HAID ‘09 (Haptic and Audio Interaction Design) in Dresden, Germany.
If you’d like to try this for yourself, click on the link below to download an installer. It works only on Intel macs with a built-in iSight camera. You can activate the plug-in through AU Lab, GarageBand, or Logic:
Forgotten Favorites was created by a company that I started with my two roommates in the summer of 2009. We had lofty ambitions of piggy-backing off of the success of Voice Record and the AppStore in general. We knew three things: we all loved music, we all loved technology, and therefore we all loved music information retrieval. Because we were trying to avoid needing an Internet connection during runtime, we settled on a lightweight form of MIR that just looks at the meta-data of the songs on the device. The idea of Forgotten Favorites is simple – everyone ends up neglecting music they love because they get distracted by new songs/albums. By parsing the meta-data of someone’s library, we were able to present “Forgotten Favorite” songs and albums. Items could be “banned” if someone never wants to hear it again, and a slider in the settings view sets the weighting between how favorite or how forgotten you like your results. It is localized in English, Spanish, French, German, and Japanese.
To quote my roommate’s website, “immediate commercial success did not ensue.” After a month of abysmal sales, we made the app free so that we would at least have more people using it. Regardless, I am very proud of our final product. It was fantastic working on a team with very competent people and I think we made a great app for what we set out to do. Some people love it and some people just don’t get it. The best part about all of this: I started the application with zero knowledge about Core Animation and finished with as much knowledge as you’d find in most books about it.
Voice Record was one of the first 500 applications on the AppStore when it launched in July 2008. It immediately found success and held onto a spot in the Top 25 Paid Applications for over two weeks. David Pogue featured the app in a video on the front page of The New York Times in July 2008. Currently, Voice Record has been downloaded onto over 37,000 devices and has over 1,100 reviews. This app does pretty much what you would think…it records your voice (or any sound for that matter). It also allows playback, renaming, re-ordering, and note association. After some trial and tribulation (a.k.a. socket programming), the app was expanded to include the FTP transfer of files and eventually direct WiFi transfer. Apple has since released a far more powerful recorder that comes free with every device, but Voice Record continues to sell moderately well. Some people write in who appreciate the advanced features such as the ability to specify the sampling rate or to perform FTP transfers. This app isn’t terribly exciting (like Forgotten Favorites), but it was created before there was much documentation or standard practices as far as iPhone apps were concerned.
