Framework for bimanual inter-device interactions

Ali Roudaki, Jun Kong, Gursimran Walia

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations


A shared interactive display (e.g., a tabletop) provides a large space for collaborative interactions. However, a public display lacks a private space for accessing sensitive information. On the other hand, a mobile device offers a private display and a variety of modalities for personal applications, but it is limited by a small screen. We have developed a framework that supports fluid and seamless interactions among a tabletop and multiple mobile devices. This framework can continuously track each user’s action (e.g., hand movements or gestures) on top of a tabletop and then automatically generate a unique personal interface on an associated mobile device. This type of inter-device interactions integrates a collaborative workspace (i.e., a tabletop) and a private area (i.e., a mobile device) with multimodal feedback. To support this interaction style, an event-driven architecture is applied to implement the framework on the Microsoft PixelSense tabletop. This framework hides the details of user tracking and inter-device communications. Thus, interface designers can focus on the development of domainspecific interactions by mapping user’s actions on a tabletop to a personal interface on his/her mobile device. A user study compared our interaction style with a standard tabletop interface and justified the usability of the proposed interaction.

Original languageEnglish (US)
Title of host publicationProceedings
Subtitle of host publicationDMS 2014 - 20th International Conference on Distributed Multimedia Systems
PublisherKnowledge Systems Institute Graduate School
Number of pages8
ISBN (Electronic)1891706365
StatePublished - 2014
Externally publishedYes
Event20th International Conference on Distributed Multimedia Systems, DMS 2014 - Pittsburgh, United States
Duration: Aug 27 2014Aug 29 2014

Publication series

NameProceedings: DMS 2014 - 20th International Conference on Distributed Multimedia Systems


Conference20th International Conference on Distributed Multimedia Systems, DMS 2014
Country/TerritoryUnited States


  • Bimanual interaction
  • Human computer interaction
  • Multimodal interface
  • Tangible interface

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Software


Dive into the research topics of 'Framework for bimanual inter-device interactions'. Together they form a unique fingerprint.

Cite this