HoloBots: Augmenting Holographic Telepresence with Mobile Robots for Tangible Remote Collaboration in Mixed Reality

Keiichi Ihara, Mehrad Faridan, Ayumi Ichikawa, Ikkaku Kawaguchi, Ryo Suzuki

Mixed reality telepresence can make a remote person visible in your space, but their hologram can’t touch anything. HoloBots pairs a remote user’s live hologram (HoloLens 2 and Azure Kinect) with small tabletop robots (Sony Toio) that move in sync with their hands, so the remote person can push, grasp and move real objects next to a local collaborator. We explored a design space that includes object actuation, physicalized virtual hands, world-in-miniature exploration, shared tangible interfaces, embodied guidance and haptic communication. In a study with twelve participants, HoloBots significantly increased co-presence and the sense of a shared experience compared with hologram-only and robot-only conditions.

My role

I helped build the link between the remote user’s live hologram (HoloLens 2 and Azure Kinect) and the Sony Toio robots, so the robots move in sync with the remote user’s hands, and helped run the 12-participant user study.

Videos

Cite

@inproceedings{ihara2023holobots,
  title     = {{HoloBots: Augmenting Holographic Telepresence with Mobile Robots for Tangible Remote Collaboration in Mixed Reality}},
  author    = {Ihara, Keiichi and Faridan, Mehrad and Ichikawa, Ayumi and Kawaguchi, Ikkaku and Suzuki, Ryo},
  booktitle = {Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology},
  series    = {UIST '23},
  year      = {2023},
  numpages  = {12},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  doi       = {10.1145/3586183.3606727},
  url       = {https://doi.org/10.1145/3586183.3606727}
}