Research Objective and Plan  

The objective of "BeingThere Centre: International Research Centre for Telepresence" is to make major technological and systems-level advances leading to a credible, 3D experience of telepresence in a fitted room using a mobile platform. The result will be a breakthrough in the quality of interpersonal communication at a distance allowing for eye contact and proper motion parallax among a group of users.

The lack of visual interpersonal communication is a major limitation of the telephone system. Augmenting sound with video has only partially solved the problem. Still, critical information in brainstorming and negotiation situation are missing. If our research is successful, it will mark a major breakthrough and beginning of a new era of tele-communication truly called "tele-presence". Its impact on society can be substantial, like that of the emerging telephone a century ago.

This International Research centre, comprised of groups in UNC, ETH and NTU, will develop four prototypes of the Telepresence system of the 21st century:

  • Prototype 1: Room-based Telepresence System
    A fitted room with a table for the standard group meeting situation. The room provides "windows" into the meeting rooms of the remote groups with 3D representation tailored to each of the participants. The two persons engaged in a conversation experience true eye-contact whereas other participants see them from the side with parallax as in a real meeting room.
  • Prototype 2: Mobile, Autostereoscopic 3D Display and Telepresence Platform
    A roving display which brings a 3D representation of a single distant person to a place, controllable by both users. The display is semi-transparent and will bring the illusion of the other person being present into a room, a laboratory or a hospital.
  • Prototype 3: Mobile Animatronics Telepresence System
    An animatronic robotic mannequin onto which a faithful representation of the remote user is projected. The robot is remote controlled by the remote user and can navigate laboratories or hospital rooms. It is an "avatar", i.e. a virtual representation of the remote user which freely explores the remote environment.
  • Prototype 4: Autonomous Virtual Avatars for TelePresence
    In absence of one of the user, an autonomous Virtual Human (VH) will replace it with a real appearance similar to the user. This VH will have some capabilities of sensing, gesturing, memory, and communication. It adds an "anytime" component to the "anywhere" component of the overall project. The robot will also be capable of awareness.

The application scenarios for the mobile avatar are remote diagnostics by a medical expert, factory floor inspections, teaching or just a person-to-person communication in a less rigorous setting as the conference room of prototype 1.

In order to do this, the centre must develop breakthrough technologies in autostereoscopic display usage, real-time 3D video image capture and animatronics with affordable cost. These key technologies will be developed in Singapore in "3D Display Lab", "3D Capture Lab", "Animatronics Lab", and 3D Behavioural Lab as well as at ETH and UNC in their respective research efforts.

At the end of the four year period, the prototypes are ready for commercial exploitation through start-ups spun of the centre and transfer to established companies. The last year will be largely devoted to making this transfer successful.