VR and Digital Twin Based Teleoperation of Robotic Arm

This research proposes a novel telemanipulation system that enables a human operator to control a dual-arm robot. The operation provides kinesthetic teaching via a digital twin of the robot which the operator cyber-physically guides to perform a task. Its key enabler is the concept of a virtual reality interactive marker, which serves as a simplified end effector of the digital twin robot. In virtual reality, the operator can interact with the marker using bare hands, which are sensed by the Leap Motion on top of a virtual reality headset. Then, the status (e.g. position/orientation) of the marker is transformed to the corresponding joint space command to the remote robot so that its end effector can follow the marker.


On 3rd November 2020, a researcher from The University of Edinburgh successfully teleoperated the robotic arm located at The University of Manchester using a virtual reality headset after less than 20 minutes training using a simulated robot.


For more information contact us on