Direct and expressive interaction on the desktop: increasing granularity, extent, and dimensionality

Jonathan Aceituno 1
1 MJOLNIR - Computing tools to empower users
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : Desktop and laptop personal computers enable knowledge work and creative activities by supporting direct and expressive interaction. But the feeling of directness often disappears when users perform complex actions: expressiveness is ensured at the cost of spatiotemporal separation, cognitive load, or complex operation sequences. This thesis proposes to solve this problem by harnessing the unexploited capabilities of standard input devices and familiar actions performed on them. We investigate how this enables direct increases of the granularity, extent, and dimensionality of user actions. First, we show that the granularity of pointer movements can be increased a hundredfold without impeding normal behavior if pointing transfer functions include device characteristics, user capabilities, and the manipulated data model, thus allowing subpixel interaction. This is limited by the useful resolution, the smallest displacement a user can produce using a pointing device, for which we propose an experimental protocol. Second, we study the design space of a widely used technique, edge-scrolling, that extends dragging actions past a viewport edge by scrolling. We reverse engineer 33 existing implementations, and highlight usability problems through a survey and experiments. We also propose push-edge and slide-edge scrolling, two position control techniques that provide performance comparable to rate control without the shortcomings. Third, we describe three ways of using a standard laptop as a musical instrument, allowing simultaneous multiparametric control of sound synthesis in real time, together with design considerations and examples of successful uses.
Document type :
Complete list of metadatas
Contributor : Jonathan Aceituno <>
Submitted on : Thursday, January 14, 2016 - 10:20:43 AM
Last modification on : Friday, May 17, 2019 - 11:39:42 AM
Long-term archiving on : Friday, November 11, 2016 - 5:33:20 AM




  • HAL Id : tel-01255896, version 1


Jonathan Aceituno. Direct and expressive interaction on the desktop: increasing granularity, extent, and dimensionality. Human-Computer Interaction [cs.HC]. Université Lille1 - Sciences et Technologies, 2015. English. ⟨NNT : 2015LIL10089⟩. ⟨tel-01255896⟩



Record views


Files downloads