UWEE Tech Report Series

The GP-LVM for Vocal Joystick Control


Jon Malkin, Neil Lawrence, Jeff Bilmes

Vocal Joystick, Gaussian processes, dimensionality reduction, latent variable models


The Vocal Joystick (VJ) is an assistive device that uses the rich complexity of the human voice to drive a human-computer interface. The system has previously been shown to work well for control of a computer mouse, yet it does not currently make full use of the continuous nature of the vowel space. This work examines the potential use of the Gaussian process latent variable model (GP-LVM), which provides a non-linear, smooth probabilistic mapping from latent to data space. The results show promise for some speakers but do not currently generalize well across speakers. The GP-LVM provides a well-motivated approach, but additional work is still needed to translate the existing potential into an effective new control scheme for VJ.

Download the PDF version