title: An egocentric vision based assistive co-robot.
abstract: We present the prototype of an egocentric vision based assistive co-robot system. In this co-robot system, the user is wearing a pair of glasses with a forward looking camera, and is actively engaged in the control loop of the robot in navigational tasks. The egocentric vision glasses serve for two purposes. First, it serves as a source of visual input to request the robot to find a certain object in the environment. Second, the motion patterns computed from the egocentric video associated with a specific set of head movements are exploited to guide the robot to find the object. These are especially helpful for quadriplegic individuals who do not have needed hand functionality for interaction and control with other modalities (e.g., joystick). In our co-robot system, when the robot does not fulfill the object finding task in a pre-specified time window, it would actively solicit user controls for guidance. Then the users can use the egocentric vision based gesture interface to orient the robot towards the direction of the object. After that the robot will automatically navigate towards the object until it finds it. Our experiments validated the efficacy of the closed-loop design to engage the human in the loop.
authors.forename: Jingzhe
authors.initials: J
authors.lastname: Zhang
authors.forename: Lishuo
authors.initials: L
authors.lastname: Zhuang
authors.forename: Yang
authors.initials: Y
authors.lastname: Wang
authors.forename: Yameng
authors.initials: Y
authors.lastname: Zhou
authors.forename: Yan
authors.initials: Y
authors.lastname: Meng
authors.forename: Gang
authors.initials: G
authors.lastname: Hua
mesh_terms: D000465:Algorithms; D005139:Eyeglasses; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D008328:Man-Machine Systems; D012371:Robotics / Q000295:instrumentation*; D012656:Self-Help Devices; D014796:Visual Perception