Advancing autonomous driving and studying with video games Hands-on machine learning challenge for students

The Institute of Communications Engineering at Technische Universität Braunschweig organised the Deep Learning Lab for the seventh time. 30 students had the opportunity to apply machine learning methods and compete with other students in a challenge. This time, the focus was on optimising image recognition methods in urban areas, which are important for the development of autonomous vehicles, for example.

This year’s challenge in a single image: On the left, the reference model with a recognition accuracy of 32%; on the right, the significantly improved winning model with an outstanding recognition accuracy of 75%. Picture credit: Cityscapes dataset, IfN/TU Braunschweig

Human, car, bike or road? Humans can usually tell the difference easily, but computers and software cannot: they have to learn to recognise and differentiate. This is an important skill for autonomous driving. One challenge is that training these programs requires a large amount of imagery, and the labeling/categorisation has to be done by humans, which is time-consuming. An alternative is to train the computer programs with synthetic (artificial) images. However, these images differ from the images that the programme will later “see” in the real world. Unsupervised Domain Adaptation (UDA) is a training method that tries to solve this problem. The idea is to first train the programme on artificial data and then prepare it for the real world with a small set of real, but unlabelled, images.

The Deep Learning Lab brought together 30 students from a variety of courses, including data science, computer science, information systems engineering, electrical engineering, industrial electrical engineering and computational sciences in engineering. Working in teams of three, they were able to try out UDA for themselves. Their task: to train a computer program using synthetic images and labels from an open-world urban video game and a small proportion of labelled cityscape images. The goal: to be able to categorise real images (cars, streets, pedestrians, etc.) as accurately as possible after training.

Four practical phases

The Deep Learning Lab was divided into four different practical phases. In the first phase, students learned the tools of the trade: the Python programming language. In the second phase, they worked on tasks related to selected machine learning methods. These two phases were optional for the students this year. Things really got going in phase three, the first part of the Machine Learning Challenge. Students were given a problem and a training dataset. This allowed them to start applying what they had learned earlier. They were supported by mentors from the institute. At the end of this phase, all groups received an interim result of their implementation. With this feedback in hand, they moved on to the final phase, in which the groups developed their final model. On 5 July, the teams presented their results and three of them were awarded prizes.

The three winning teams

The three winners: Anna Tuma, Yannick Stadler and Soukaina Oujaada. Photo credit: Andreas Gudat, IfN/TU Braunschweig

First place (€600) went to Anna Tuma, Soukaina Oujaada and Yannick Stadler, who improved the baseline or reference model from 32.00% accuracy to an outstanding 75.00% accuracy. In order to win, the three invested a lot of time in the challenge on top of their normal studies, sometimes meeting several times a week. They were motivated to take part by reports from previous participants. “But it’s also just a unique opportunity to try things out in practice and have so much freedom,” says Anna Tuma. The biggest challenge for the group was getting started. “It wasn’t easy for us to get the programme going in the beginning.” Even after completing the third phase, they were still ‘only’ in third place, but eventually fought their way up to first place.

The runners-up also won the Environmental Prize: From left: Kevin Düser, Niklas Schwarz and Matthias Müller. Photo credit: Andreas Gudat, IfN/TU Braunschweig

Second place went to Kevin Düser, Matthias Müller and Niklas Schwarz (€300). They also won the Environmental Prize (€450) for the best performance/processing time ratio, proving that good results can be achieved with low power consumption. The podium was completed by the team of Darius Heerklotz, Berke Katranci and Aravind Meena Krishnan Gopakumar (€150).

Third place: From left: Darius Heerklotz, Berke Katranci and Aravind Meena Krishnan Gopakumar. Photo credit: Andreas Gudat, IfN/TU Braunschweig

“Participating in the Deep Learning Lab is worthwhile not only because of the prize money and the opportunity for hands-on learning. Our closing event gave participants the opportunity to network with three well-known companies, the sponsors. The presence of company representatives at the award ceremony shows the young people the valuable skills they have acquired during their studies,” says Professor Tim Fingscheidt, head of the Deep Learning Lab.