Challenge: Improving the recognition of rare objects Students adapt AI models for imbalanced datasets

The ninth edition of the “Deep Learning Lab” took place in the summer semester of 2026: 30 Bachelor’s and Master’s students from TU Braunschweig formed teams and competed against one another in an AI challenge. The students were tasked with developing methods to enable an AI image processing model to recognise not only objects that occur frequently in the dataset, but also rare objects, some of which appear only once in the training data.

In the live demonstrator, the winning team’s model (right) recognises a wide variety of objects with a high degree of certainty, regardless of whether they occur frequently or rarely in the training data. Photo credit: IfN/TU Braunschweig

Whether it’s autonomous driving, robotics or medical image analysis: wherever AI systems are required to perceive their environment, they must be able to find and identify objects in images. Modern object detectors are now very proficient at this for common classes such as cars, people or chairs. However, reality is not evenly distributed: whilst some object classes appear thousands of times in image datasets, others occur only a few times. If the frequency of all classes is plotted in a sorted diagram, the result is a curve with, initially, a few tall bars for the frequently occurring classes, followed by a large number of flat bars for the rare classes, which together form the so-called “long tail”.

For neural networks, it is precisely this “long tail” that poses a problem. They learn from data, and where only a small amount of data exists, they learn poorly. The result: rare objects are overlooked or confused with common ones. Often, however, it is precisely the rare and unusual objects that are more important than the common ones. Imagine, for example, a luggage scanner at an airport: whilst it can recognise socks, it cannot detect a knife.

“In the data-driven world, frequency and importance are sometimes out of proportion to the real world. In such cases, a balance must be struck using targeted methods,” explains Professor Fingscheidt, founder and head of the Deep Learning Lab.

The challenge: learning effectively from a small number of examples

This was precisely the focus of this year’s task. The students were given images from the LVIS dataset[1], in which some object classes are annotated very frequently, whilst many others are annotated only rarely. The aim was to train an object detection model so that it could reliably locate and identify even the rare classes without losing accuracy for the common classes. A particular challenge this year was that the students were first required to develop their methods on an initial dataset, but then had to apply their final solution to a different dataset containing entirely different object classes, to which they were only granted access two weeks before the final submission deadline. In other words, their methods had to be transferable across different datasets.

The winning team of the Deep Learning Lab 2026, consisting of Paul Höljes and Simon Wöhltjen. They were congratulated by the IfN supervisors, as well as representatives of the donors and industry sponsors (Robert Bosch GmbH, Siemens Mobility GmbH, Volkswagen Group Innovation). Photo credit: Andreas Gudat/TU Braunschweig

There is no single “silver bullet” solution to this problem, and the teams’ approaches were correspondingly diverse. Some gave greater weight to rare objects during training by adjusting the model’s loss function or by showing the model examples of rare objects more frequently. Others relied on data augmentation – the artificial generation of additional training data – for example, by copying a rare object from one image and inserting it into other images.

The winning teams

The winning team led by Paul Höljes and Simon Wöhltjen took the data augmentation approach a step further by also augmenting the images at inference time. The AI model receives several versions of the input image (enlarged, reduced, rotated) and then calculates an average, which both increases robustness and the likelihood that a rare object in the orientation and size shown is already known from the training data.

In recognition of their achievement, the winning team received a prize of 200 euros per person, courtesy of the industry partners. The environmental prize, worth 150 euros per person, for the lowest consumption of GPU computing time whilst still achieving very good performance, went to the two students Mohanad Nachar and Mudassar Iqbal.

Graduation and Outlook

Following the award ceremony, students, research assistants, professors and industry representatives socialised over a barbecue on the roof terrace. Photo credit: Andreas Gudat/TU Braunschweig

The graduation event offered participants the opportunity to present their results and exchange ideas with industry representatives. A live demonstrator allowed visitors to place a selection of common and rare objects in front of a webcam and pit the submitted AI models against one another in an object recognition challenge. Finally, during the barbecue, new contacts were made in a relaxed atmosphere and a wide range of current issues in artificial intelligence were discussed.