Design Student Uses Oculus Rift, VR Gloves to Control Music Creation Software

[[Coi] Mr.CLuBBeR]

Few things can top an Oculus Rift hack that involves hands and music

A Kiwi design student named Bryon Mallett has come up with Pensato, a virtual reality interface for controlling music devices and software that, unlike most other Oculus Rift-based projects out there, makes pretty good use of human hands. In a video he posted to YouTube back in September, Mallet can be seen creating some music — actually the “final compositional output” for his Masters of Design Innovation — using popular music creation software Ableton Live and his Penasto interface, which allows him to interact with various sound controls using a pair of custom VR gloves as if they were physical objects.
His custom VR gloves owe their hand tracking mojo to the modified Razer Hydra motion controllers that Mallett managed to squeeze into them, as well as a pair of Teensy microcontrollers that are equipped with bend sensors for detecting hand position and gestures.
“This performance features a master version of Pensato running in Unity on the computer monitor to my right, which creates the visuals for the Rift and controls Ableton Live. The projector computer runs another copy of Pensato that listens to the position and orientation of the gloves, as well as the hand gesture and displays a copy of my VR interface on 3 projectors. The laptop runs Ableton Live 9 and is connected to each copy of Pensato using Showtime, a framework that I developed for connecting together realtime performance software and hardware,” reads the video’s description on YouTube.

Follow Pulkit on Google+


More...