HackMe Vectr Open Source 3D Sensing Gesture Controller

Now here's a cool new Kickstarter project;
The Vectr Open Source 3D Sensing Gesture Controller is a nice new way of controlling your modular Eurorack synthesizer.

HackMe Vectr Gesture Controller prototype

It's like a cross between the Korg KaosPad and some kind of (Roland) D-Beam interface (or theremin) that can control sound and effects via hand movements interacting (measuring the distance) with an infrared beam of light.
They hope to fit it all on a 32 HP panel (the prototype size is 36 HP wide).

" The Hackme Vectr is an entirely new and unique way to interact with a modular synthesizer.
Vectr is a three dimensional control interface designed for Eurorack modular system.
It senses the location of a hand over its face and outputs a signal proportional to position for each axis: x (left and right), y (up and down), and z (in and out).
So, you can control three things at one time, like turning three knobs all at once by moving your hand through the air.

HackMe Vectr Gesture Controller prototype

The possibilities are truly infinite. You could control the amplitude of a sound with the x axis, the pitch with the y axis, and the modulation of that sound with the z axis.
By moving your hand slowly over the surface, you can create subtly evolving, changing and dramatic sounds. With quick movements, you can generate bursts of dynamic audio.

To provide feedback, Vectr has LEDs ringing its active sensing area.
These LEDs light up according to the location of your hand.
It lights red for the z axis and blue for the x and y axis.
As you get closer to the surface, the red LEDs get brighter.
As you move closer to a location on the x-y axis, the blue LEDs in that region get brighter and those farther away get dimmer.
This makes Vectr visually stunning, brilliant for live play and, excellent for dark spaces.

HackMe Vectr Gesture Controller pad

Vectr can also recognize a number of simple gestures.
It recognizes swiping gestures as well as circular motions over its surface.
These gestures can be used as trigger outputs and for interacting with the built-in looping feature.
Vectr provides a menu system that uses gestures to change the system behavior.
Vectr can record up to 30 seconds of position and gesture data for looping playback.
Simply press the large illuminated switch to start the loop and release it to end it.
The loop plays back immediately and can be retriggered with swipe gestures or by external signals from a module with trigger outputs.
Vectr records as long as the switch is held down, so long rhythmic sequences can be recorded by moving a hand in an out of the active area.

HackMe Vectr Gesture Controller prototype

The loop switch can also be used as a sample and hold control.
When pressed quickly, it freezes the hand position to lock in the output.
The loop playback can be sped up or slowed by using circular gestures, clockwise for faster, counter-clockwise for slower.
The loop recording can also be triggered externally.
One pulse starts the recording and the next pulse ends it. Using external triggers to start and stop recording as well as to synchronize the playback, you can make perfectly synchronized loops.

Vectr is entirely open source. All aspects of the design will be released under cc-nc-sa 2.0 license. There will be a USB connector on the back of the device. This will allow anyone to program and reprogram Vectr to suit their purposes. Want to use it to fly drones? Make a lighting controller? Make your robot dance? Go for it."

The HackMe Vectr is a Kickstarter project, with the controller available to early supporters for 299 USD

Video: Control audio with movements and gestures in free space


Detailed Features:
Resolution: 16 bit Sensing and 16 bit Digital to Analog Conversion

Outputs:
X Position - 0-8VDC corresponding to hand position
Y Position - 0-8VDC corresponding to hand position
Z Position - 0-8VDC corresponding to hand position
Left-Right Swipe Gesture Trigger - 0-5VDC alternates with each gesture
Up-Down Swipe Gesture Trigger - 0-5VDC alternates with each gesture
Gate - 0V when no hand is present, 5VDC when a hand is present

Inputs:
X Gate - 0V turns off X Position
Y Gate - 0V turns off Y Position
Z Gate - 0V turns off Z Position
Loop Sync - Retriggers the loop during playback, start and end loop during record mode

Switch:
Loop switch - High quality feel illuminated switch for controlling the looping feature

Power Supply:
+/-12V, digital voltages internally regulated

More info at http://www.kickstarter.com/projects/790206393/theremax-3d-gesture-controller

Random Video: Cardio Bass & Respiration Synth by Ukrets

Mark Mennings (@sensien) brought this next video to my attention.
A nice example of biorythmic/posture control of a doepfer modular.

Video: Cardio Bass & Respiration Synth

" Ukrets uses the ephysm.us biomusic performance and authoring platform. Performed by the Holminid with synthesis by Sensien, using a modular analogue synth. Everything is generated from live biosignals and postures, except the obligatory 808 kick that was added for structural support. The track consists of two patches recorded in succession, mixed without any further editing or processing. Jaala, Finland, July 2013. http://ephysm.us"
" ...we convert the signals from the wireless sensor belt to OSC, which is then used to create triggers and control ableton. This is converted to CV."

Read more about it on MuffWiggler,
and don't forget to check out http://ephysm.us/ electronic physiological music

Controlling the Doepfer Modular Synthesizer with a Wiimote

I blogged about the Doepfer Mogli and gesture control earlier on this blog, but in 2007 it was already possible to control your A-100 with a Nintendo Wii Remote (or Wiimote) controller.

Video: Controlling Doepfer modular synth with a Wiimote
http://youtu.be/LHOS2zg-0nI


" This video is about controlling a modular synth with Nintendo's wiimote gamepad, using Doepfer A-100, Doepfer MCV24, Wiimote and computer.
Softwares used are Bluesoleil, Glovepie and Midi-ox.
This video is not an artistic demonstration but a technical explanation of this system (that can be used for making music ofc). "

Uploaded on Sept. 12, 2007 by YouTube user DamagedMeat

The Doepfer Mogli and Gesture Control

In 1993 Doepfer presented the Mogli MIDI Data Glove Controller, an alternative controller device that was based on a Nintendo Power Glove controller.
It had traditional NES controller buttons on the forearm as well as a program button and buttons labeled 0-9.

The complete Mogli set with Power Glove,
receivers,  Mogli interface box and adapters

There were two ultrasonic speakers (transmitters) in the glove and you had to put three ultrasonic microphones (receivers) around your TV monitor.
The ultrasonic speakers took turns transmitting a short burst (a few pulses) of 40 kHz sound and the system measured the time it took for the sound to reach the microphones.
A triangulation calculation was performed to determine the X, Y, Z location of each of the two speakers, which specified the yaw and roll of the hand.
It could also sense the bend of the individual fingers, the only dimension it couldn't calculate was the pitch of the hand, since the hand can pitch without moving the location of the two ultrasonic speakers.

The Power Glove was originally released in 1989 and was in general a critical and commercial failure.
Partly because of the lack of accuracy and the availability of games (for it's original use with the NES)

The Mogli interface box, Power Glove and adapters

In 1993 Doepfer introduced an external box that could receive the glove's data and convert it into MIDI signals.
The name Mogli stands for 'Midi Output GLove Interface'
The original Mogli interface box was equipped with MIDI in and MIDI out, and it had a red 3-digit LED display.
The display was needed to program the unit (i.e.to assign different function to the fingers, to the X/Y/Z coordinates, to the rotating angle and so on).

After a calibrating procedure the glove could be used in different controller modes.
The 'Virtual Play' mode was maybe the most attractive mode, in which could make you play a virtual keyboard in the free space.
In Direct Mode you could assign any movement to any MIDI controller, other modes were Gesture Mode (converting sign-language to MIDI controllers) Vector Mode (position sensing only) and Manipulation Mode (for manipulating incoming MIDI with the glove).

Although i am not the world's greatest electronics-guy i do have a feeling that the original Mogli converter box could be easily transformed into a version that sends Control Voltage and/or Gate signals.

Original Nintendo Power Glove

If you can find one, it will probably just work just fine with one of Doepfer's MIDI-to-CV modules, but you might not get the most out of all the glove's many functions.

The price of the system was 598 DM (i.e. about 300 Euro) for the ready built unit including a Nintendo power glove.
The kit version was 448 DM (i.e. about 225 Euro) incl. Nintendo power glove.
The control box only (without the glove) was 398 DM (i.e, about 200 Euro)
A kit version of the box was also available for just 258 DM (i.e. about 130 Euro)

The unit was available from March 93 until December 1995, but Doepfer had to stop the production because the Nintendo Power Glove was no longer available.
Around 350 units were made in total.
Interesting fact is that the Mogli was also used by Kraftwerk (on 'Pocket Calculator' and 'Music Non Stop') during their concert at the Brucknerhaus in Linz/Austria on the occasion of the ARS Electronica Festival for Art, Technology and Society in 1993.

Okay, i admit that the technology of the Power Glove might be outdated by now, but the idea of controlling sounds/patches with the move of your hand will always stay an interesting subject.
Roland is famous for its D-Beam technology since 1998.
The A-178 Theremin Control Voltage module (or the discontinued A-179 Light Controlled Voltage Source) definitely do not offer all the functionality of a glove like this.

However, i do think we will see some kind of gesture-control-trend coming up this year with the recently announced Kinect motion sensing input device by Microsoft, that was already available for the X-Box platform but that will soon be available for the Windows platform too.
I wonder how soon the first interesting gesture-controlled modular synth-video's will pop up on YouTube...

Find the Mogli's user-manual (in German language only) HERE
Please let me know if you have an English version, or else check the start of my translation project (work in progress).

* Many thanks to Dieter Doepfer for the additional info
* Pictures by Gaiana via http://www.synthforum.nl/

Video by studentsmusic added March 1st, 2017:
Doepfer MOGLI Midi Glove (Rare - Vintage)