WatchSense: On- and Above-Skin Input Sensing through a Wearable Depth Sensor

ACM Conference on Human Factors in Computing Systems (CHI) 2017, Denver, USA

Download Video: SD (MP4, 720p, 96 MB) | HD (MP4, 1080p, 389 MB)


This paper contributes a novel sensing approach to support on- and above-skin finger input for interaction on the move. WatchSense uses a depth sensor embedded in a wearable device to expand the input space to neighboring areas of skin and the space above it. Our approach addresses challenging camera-based tracking conditions, such as oblique viewing angles and occlusions. It can accurately detect fingertips, their locations, and whether they are touching the skin or hovering above it. It extends previous work that supported either mid-air or multitouch input by simultaneously supporting both. We demonstrate feasibility with a compact, wearable prototype attached to a user's forearm (simulating an integrated depth sensor). Our prototype---which runs in real-time on consumer mobile devices---enables a 3D input space on the back of the hand. We evaluated the accuracy and robustness of the approach in a user study. We also show how WatchSense increases the expressiveness of input by interweaving mid-air and multitouch for several interactive applications.



BibTeX, 1 KB

 author = {Sridhar, Srinath and Markussen, Anders and Oulasvirta, Antti and Theobalt, Christian and Boring, Sebastian},
 title = {WatchSense: On- and Above-Skin Input Sensing through a Wearable Depth Sensor},
 booktitle = {Proceedings of ACM CHI},
 url = {},
 numpages = {12},
 month = May,
 year = {2017}


This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation programme (grant agreements 335545, 648785, and 637991). We would like to thank Sebastian Schuon.


Srinath Sridhar

This page is Zotero translator friendly. Page last updated 09-Jan-2017. Imprint/Impressum.