in|situ| Lab - Log In

all in|situ| jobs

Enhancing paper interaction with small portable assistants

Internship proposal at in|situ|

Advisor: Theophanis Tsandilas (


Our experience at LRI has shown that paper is indispensable for several groups of users. For example, several composers of contemporary music are proficient users of computers but still use paper at different stages of the composition process, from sketching initial ideas to working on final scores [1]. Anoto [2] technology made the connection between paper and computers practical: a pen with a tiny embedded camera captures traces of ink with respect to a unique dot pattern printed on the paper. Yet linking handwritten content with the digital world is still a challenging problem as paper is not interactive and the feedback provided by current digital pens is still extremely limited.

Earlier approaches have succeeded in making paper interactive by using special equipment such as a projector and cameras [3] or a graphics tablet and a PDA [4]. More recently, Song et al. [5] explored paper interaction with pens equipped with miniature projectors. Unfortunately, these approaches have been based on equipment that is cumbersome and hard to employ in real contexts of use. We are currently seeking new forms of feedback that are both rich and lightweight.

The goal of the proposed project is to explore seamless links between paper and digital media and help users effectively switch between computation and freeform interaction. The internship will explore the use of digital pens in conjunction with smartphones. Smartphones are portable, inexpensive and widely available. We hypothesize that they can serve as valuable assistants for users working on paper by increasing the visual space and providing additional channels of input. As a first step, the student will familiarize with research on Human-Computer Interaction (HCI), focusing on augmented paper and mobile interaction. As a second step, the student will work on the design and implementation of bimanual interaction techniques that couple handwriting (dominant hand) and touch-screen interaction (non-dominant hand). Implementation will be based on Anoto digital pens and Android mobile devices. Finally, the student will learn about HCI evaluation methods and, ideally, will conduct an experiment to test the developed techniques with human subjects.

For information about our previous projects on interactive paper, visit our team's page.

Skills of ideal candidate

Solid programming skills (Java), fascinated about user-interface design, interested in pursuing research


  1. Tsandilas, T., Letondal, C. and Mackay, W. E. (2009). Musink: Composing Music through Augmented Drawing. CHI 2009, pp. 819-828.
  2. Anoto,
  3. Wellner, P. (1993). Interacting with Paper on the Digitaldesk. Communications of the ACM, 36(7), 87-96.
  4. Mackay, W. E., Pothier, G., Letondal, C., Bøegh, K. and Sørensen, H. E. (2002). The Missing Link: Augmenting Biology Laboratory Notebooks. UIST 2002, pp. 41-50.
  5. Song, H., Grossman, T., Fitzmaurice, G., Guimbretière, F., Khan, A., Attar, R., et al. (2009). Penlight: Combining a Mobile Projector and a Digital Pen for Dynamic Visual Overlay. ACM CHI, pp. 143-152.