Controllar : Appropriation of Visual Feedback on Control Surfaces

This video shows the results of the ControllAR project, a collaboration between Florent Berthaut (CRIStAL, Université de Lille) and Alex Jones (University of the West of England).

The software is available at : https://gitlab.cristal.univ-lille.fr/fberthau/controllar
The paper is available at : hal.archives-ouvertes.fr/hal-01356239/file/ControllAR.pdf

Abstract :

Despite the development of touchscreens, many expert systems for working with digital multimedia content, such as in music composition and performance, video editing or visual performance, still rely on control surfaces. This can be due to the accuracy and appropriateness of their sensors, the haptic feedback that they offer, and most importantly the way they can be adapted to the specific subset of gestures and tasks that users need to perform. On the other hand, visual feedback on controllers remains limited and/or fixed, preventing similar personalizing.
In this paper, we propose ControllAR, a novel system that facilitates the appropriation of rich visual feedback on control surfaces through remixing of graphical user interfaces and augmented reality display. We then use our system to study current and potential appropriation of visual feedback in the case of digital musical instruments and derive guidelines for designers and developers.