Robotics and machine perception

Home > Robotics and machine perception

Robotic systems are becoming an increasingly important part of humanity, but today only a tiny fraction of the potential of such systems is being exploited to automate and render monotonous, dangerous, and resource-intensive work, to raise public welfare, and to make people’s lives safer and more convenient. To successfully perform complex tasks in changing conditions, robots require both intelligence for adaptive decision-making and motor control, and the ability to accurately perceive the environment. The mission of EDI is to give robots this power. For EDI scientists, it is a great opportunity to apply digital technology, accumulated knowledge, and years of experience in the fields of image processing, signal processing, and artificial intelligence to address the challenges of robotics and machine perception in several areas:

  • industry 4.0;
  • agriculture;
  • autonomous and cooperative driving.

One of our challenges is the use of innovative digital technologies in the next generation industry, with the aim of increasing the quality and efficiency of European industrial production. To achieve this in a rapidly changing production environment, key prerequisites are robots able to adapt quickly to new production conditions and handle unpredictable situations, minimizing production downtime. EDI solutions in this field include placement of cameras, depth sensors, and other sensors around and on an industrial robot arm, and the processing of signals from these sensors using known and novel techniques.

An example of EDI development is an adaptive robotic system that can recognize and sort various randomly placed objects from a box using various sensors, computer vision, and deep learning. Such intelligence-enabled robotic arm is able to adapt to unexpected situations in today’s dynamic manufacturing environment.

Detecting, picking, and sorting objects in a pile

Another example is a weed controlling system for agriculture, which aims to clean the weeds off the fields tirelessly and efficiently, and without the use of substances harmful to health and the environment. For such a system, EDI developed a computer vision module that can distinguish between crops and weeds and can control the movement of the robot to cover the entire field.

Computer vision model correctly detects the growing crops

EDI is also working on machine perception solutions for autonomous driving, which has the potential both to increase road safety and to efficiently automate the transportation of goods and resources. Developing reliable perception systems is an important challenge, so the EDI self-driving car demonstrator uses multiple sensors (cameras, radars, lidars) and fail-aware/fail-operational artificial intelligence models, which we train to work even when some sensors are defective. A significant focus of our research is on cooperative vehicles and systems, thus embedded intelligence is augmented with communication protocols and cooperative decisions.

In all the above-mentioned areas, EDI develops artificial intelligence techniques based on both classical signal processing approaches and machine learning approaches (deep learning with artificial neural networks, convolutional and recurrent neural networks, generative adversarial networks). The developments in artificial intelligence are key tools in the digitalization process – they offer tremendous opportunities for automation of various processes and economic potential for start-ups and new markets. EDI has the experience and accumulated knowledge over many years in the field of signal processing research and application, providing a fundamental basis for continuing the tradition of signal processing in many applications with artificial intelligence techniques. EDI has extensive experience in computer visioning – image classification, segmentation, and object detection. In addition to these applications, EDI has addressed such tasks as satellite imagery segmentation, vehicle detection, skin lesion classification (normal/suspicious), and human identification by blood vessels of the hand. In cooperation with companies, we have also developed, at their request, a car license plate recognition system, a video game competition analysis software, and a silicon crystal production monitoring system for more accurate crystal growth control.

A precondition for the successful use of supervised machine learning is the availability of labeled training data. One of the most important fundamental areas to be addressed in the field of artificial intelligence is unsupervised and few-shot learning, where neural networks can be trained without or with a limited number of training examples. This would allow the successful use of modern artificial intelligence techniques in a wider range of tasks, not only in data-rich applications. This challenge is also tackled by EDI researchers, whose developments include methods to accelerate data labeling as well as approaches to generate synthetic training data.

In the context of fundamental artificial intelligence research, EDI is interested in:

  • transferring models trained in virtual reality to the real world environments;
  • imitation learning (robots are learning the tasks from a demonstration by humans);
  • trustworthy and explainable AI;
  • bio-inspired navigation systems.

An important line of EDI research is Embedded Intelligence, which processes sensor-generated information as close to the source as possible. Data processing and decision-making are shifted away from data centers and cloud computing to the border between the cloud and the physical world (edge/fog computing). Embedded machine-perception algorithms in specialized chips, such as FPGAs and SoCs, allow for low-latency and energy-efficient solutions. By combining this approach with the development of computationally efficient algorithms, we are able to extend autonomy to power-limited technologies such as small unmanned aerial vehicles.

Projects

Publications

Patents