Augmented Theremin.
Using Unity and C#
This project utilises the Unity engine as well as C# to create a custom augmented reality theremin which can be used through a desktop app or the Vuforia app on phones. The project shows the step by step process taken to develop this concept into a working one.
The brief : Augmented interactive music
Augmented Theremin in action
What is a theremin
A Theremin is an electronic musical instrument controlled without physical contact. The instrument's controlling section usually consists of two metal antennas that sense the relative position of the players hands and control oscillators for frequency with one hand, and volume with the other. The signals are then amplified and are outputted by a loud speaker.
Using targets to control objects
Objects within Unity were assigned to targets which are used to move Unity objects in augmented reality. To create these targets which can be seen below, I used regular paper cut down to a square format and a sharpie marker. To get the best tracking capabilities, the targets need to have as many cross-sections as possible. Each of these targets has a model hand assigned to them. When the target is moved in real life, the screen will display the model hand over the target and will replicate the movement of the target on the screen.
Assigning targets and testing
As stated above both targets had model hand assigned to them in Unity. Both of these hands would represent human hand interacting with a theremin.
Developing UI inputs
An interface was created to be able to control the sound, a simple off-on button was added to allow for the hands to be turned off and on. Alongside this are two sounds that replicate the frequency and the volume of the augmented theremin. However, in this case, because it is an interpretation of a theremin, the sounds are altered and a disco sound and beat are used to replicate a modern version of the theremin, this gives a more funky sound when interacted with.
Distance tracking using C#
A theremin works depending on where the users' hands are and the distance they are away from the device. To replicate this C# is used to track the distance that the target cards are away from the camera. The sound is then altered to a higher or lower pitch depending on the distance calculated, and the speed at which the sound is being played can be sped up or slowed down as well using the same method of calculating distance.
Conclusion
The final outcome shows two hands, both with audio sources assigned to them. When the hands are moved, the pitch and speed of the sound changes. This is an adaptation of a theremin in the augmented form. Once the targets are removed, the hands disappear from the screen until the targets are displayed again.
What I learned
I learned the potential that Unity, C# and Vuforia have when coupled together to create an augmented web-based app. I intend on using Unity and augmented reality as a prototyping tool to test speculative concepts with users in the future.
I learned that trouble shooting, patience and the willingness to fail are key to developing augmented products on Unity. Because of this I can now develop augmented reality apps for IOS and Android using Unity and Vuforia.