Three-dimensional Adaptive Camera with Object Detection and Foveation
The mission of the TACO project is to enhance the abilities of service robots, such as human-like interaction in the everyday environment, by improving the three dimensional sensing system with real 3D foveation properties.
The area of robotics is an innovative and growing industry. Currently service robots are adopted to execute works which are dull, dangerous, dirty or dumb. Within the further development of service robots, their functionalities are extended and therefore they can fulfill more sophisticated tasks (e.g. in the fields of cleaning, construction, maintenance, security, health care, entertainment and personal assistance).
A novel 3D sensing system (fast, small, lightweight and relatively energy-efficient) will be produced by the TACO project which includes the following three points:
- A novel concept for fast attention level management based on the 3D foveation principle enabled by dedicated sensor hardware.
- A 3D laser scanner sensor based on a miniaturised micro-mirror device combined with time-of-_flight measurement technology, which will enable operation in different modes ranging form coarse broad field-of-view 3D image acquisition to higher resolution narrow field-of-view 3D image acquisition.
- A software framework for fast object detection in everyday scenes based on saliency and visual cues. These cues allow efficient selection of details of interest, controlling the foveation process of the 3D sensing device.
The goal of TACO will be a flexible, compact, robust and low cost 3D image acquisition device providing high resolution, high quality data for robot real-time operations.
The main objective of the TACO project is the development of a 3D sensing system with real 3D foveation properties endowing service robots with a higher level of motion and affordance perception and interaction capabilities with respect to everyday objects and environments. Advantages of the developed project compared to the state-of-the-art systems will be
- at least 10 times better spatial and/or temporal resolution of detected details of interest
- a 10 times smaller and cheaper 3D laser scanner
Project Results and Innovation:
The project covers the whole value chain within 3D sensing systems with partners specialising in MEMS hardware components, time-of-flight sensors, 3D image analysis software algorithms and robotic industry applications.
A 3D sensing system based on miniaturised Silicon MEMSbased technology can be incorporated in small units mountable as payload on both service robots or on the arm of an industrial robot. The development of a 3D sensing system will fasten the research in the robotic fields of 3D attention awareness management and strategies for fast acquisition of images. Furthermore the reliable 3D sensing systems of TACO will pave the way for a much broader deployment of robots for various applications. Thus, there will be worldwide market opportunities open for sensor hardware and software application producers based on the principles of 3D foveation.
Seven European organisations – four research institutes, two companies and one university, which are seated in four different countries (Austria, UK, Germany and Norway) are members of the TACO consortium. The project partners’ expertises cover the fields of robotics and sensors, as well as process design experience.