in|situ| Lab - Log In

Instrumental Interaction - Exploring a new interaction paradigm

Master-level internship at in|situ|

Advisor: Michel Beaudouin-Lafon

Summary

The goal of this internship is to better understand and to demonstrate the power of interaction instruments, a novel approach to graphical user interfaces and to interaction in general.

Description

Our interaction with the physical world is very often mediated by tools and instruments, such as pens and pencils to write, knives and forks to eat, or bicycles to get places. In fact, the humans are the only animals that create and use tools so extensively.

Instrumental Interaction attempts to apply this notion of tool to the design of interactive systems. While many application already feature toolbars and tool palettes, the tools they hold can be used only within an application. A real tool, however, does not have such limitations. Why should it be the case on a computer? For example, the color selector that I use inside a word processing application ought to be usable in the spreadsheet. Currently, they each have their own color selector which, among other things, makes it very hard to use the same colors across applications. Pushing the idea further, why couldn't I get a different color selector and use it with an existing application? Why couldn't I have a set of tools on my smartphone to interact with content on my laptop or on a large screen?

While Instrumental Interaction seems to have great potential, much works needs to be done to both understand its theoretical underpinnings and create working prototypes. The internship covers both aspects, as follows.

First, we would like to run experiments to validate the power of tools (or instruments), using Gibson's ecological approach as background. In particular, we would like to demonstrate that holding a tool redefines the affordances of the environment.

Second, we would like to build a software architecture for developing instrumental interfaces, including in a distributed environment involving multiple devices.

The internship therefore consists of:

  • familiarizing yourself with Instrumental Interaction;
  • understanding the concept of affordance and its relationship to the notion of instrument;
  • designing and running an experiment to test the hypothesis that tools redefine the affordances of the environment;
  • implementing a prototype for testing instrumental interaction;
  • testing the prototype with a few scenarios, if possible with real users.

The internship can be 4 to 6 months long and hopefully will result in a paper submitted to a conference such as ACM CHI.

Required skills

  • Basic background in Human-Computer Interaction.
  • Programming skills in web applications (mostly Javascript, but also HTML and CSS).
  • A knowledge of node.js is appreciated.
  • A knowledge of Objective-C and programming on MacOS/iOS is a plus.