Image recognition has become enormously powerful over the last decade. But what happens when it’s taken outside of the lab and placed within the cultural realities of people and places in Berlin and Leipzig?

Context

Strolling through Berlin and Leipzig (Germany), one will be hard-pressed not to see traffic signs, buildings and infrastructure covered in stickers (as well as posters and graffiti). They especially decorate areas such as Kreuzberg, Plagwitz and Connewitz.

As such, these cities have become creative canvases on which residents express their (often sociopolitical) views. While not everyone appreciates this manner of expression – paint covering up graffiti on certain houses is still almost wet – it is part and parcel of their cultural reality.

Problem

However, image recognition systems perform significantly worse on covered surfaces. For example, a small sticker can make traffic signs unreadable for self-driving cars, with potentially disastrous consequences as a result.

One can imagine then that in cities like Berlin and Leipzig, these technologies would have a very hard time. But who should adapt to whom? Should cultural expression be sacrificed for technological progress? And would this even be practically possible?

To explore this issue, I walked around Berlin and Leipzig, took pictures of traffic signs, analyzed these images with Google Vision, and talked to people in the sticker culture about this issue. The results of this follow below.

A reality check on traffic signs in Berlin and Leipzig

Heavily sticker-bombed traffic signs
Let’s start with the juicy stuff: heavily sticker-bombed traffic signs. As seen in the three images below, none of the traffic signs actually got detected. Of course, even humans would have difficulties recognizing some of these signs.

Clean traffic signs
On to the exact opposite: clean traffic signs. In the four images below, all of the signs (8 in total) got detected. In most cases, the confidence of the detection system is quite low. Nevertheless, the system does detect traffic signs.

Mixed traffic signs
Of course, there are also signs which are neither entirely clean nor entirely sticker-bombed, as seen in the four images below. These mixed traffic signs seem to give mixed results. Two of the signs got recognized, while two were not recognized.

Visually edited traffic signs
While the results above are interesting, one might wonder how much the stickers influence the results. To explore this, I’ve visually removed the stickers with image-editing software on some of the images above where no sign was detected.

As can be seen, where previously the system didn’t detect a traffic sign, it now does (though the confidence is still surprisingly low in most cases).

Sticker bombing culture

It’s not unsurprising that the image recognition system seems to perform worse due to stickers. But how does this relate to people from cities like Berlin and Leipzig?

To explore this, I talked to Catherine Tedford, Director of the Brush Art Gallery at St. Lawrence University, New York and curator of the Street Art Graphics Collection and author of stickerkitty.com. Having researched sticker culture in cities all over the world, she was kind enough to give me more insight into the culture.

Stickers as tools of expression
In her work, Catherine found street art and stickers to be the most prolific in cities marked by conflict and struggle. Relatively simple and cheap to produce, stickers present an easy way for people to communicate their feelings and beliefs about topics important to them, Catherine tells me. As such, stickers certainly are not random acts of vandalism, but valuable tools of expression for residents. And where there’s conflict and struggle, people tend to have a great need for expression.

The idea that stickers and street art in general create ways to deal with conflict and struggle, is perhaps most poignant at the East Side Gallery. The East Side gallery – one of the longest standing parts of the Berlin wall – is covered from beginning to end in graffiti/paintings by artists from all over the world. These works often depict imagery of war, peace, and unification, reflecting the city’s desire to remember and learn from its divided past.

Removing stickers for self-driving cars
One might propose to simply remove and ban stickers from traffic signs and other infrastructure to make technologies such as self-driving cars perform better, but would this be practically possible and even desirable?

Berlin – like many cities around the world – is undergoing rapid gentrification. Catherine notes that richer neighborhoods will find ways to fund a cleanup of stickers and other street art, as some cities are already doing. But less affluent neighborhoods, she imagines, will fight back against such top-down, authoritarian measures.

‘Cleaning up’ their neighborhoods for corporate-led technological progress will only entice the sticker community to put up more stickers in revolt. The many anti-police, anti-establishment, and anti-corporate slogans on the stickers I saw while walking through Berlin and Leipzig make me believe Catherine has a very good point here.

Insights

So where does all of the above leave us?

First, companies investing heavily in image recognition would be advised to be critical of where they put their money, as well as where they deploy their systems. From a technical standpoint, the sheer amount of possible ways stickers can be placed (and actually are placed) on traffic signs makes it almost impossible to prepare image recognition systems for the cultural realities in cities like Berlin and Leipzig.

I believe some researchers underestimate the reality of this issue and lean too much on technological solutionism. Alternative solutions (such as digital traffic signs) would perhaps be more strategically wise from a business perspective, though these come with problems of their own.

Second, governments and cultural bodies should think twice on how they welcome image recognition systems. Talking to people in these cultures, removing or banning the stickers (assuming this would be practically possible) to accommodate the technology would also remove a critical form of expression for residents. Seeing as the messages behind the stickers are often sociopolitical in nature – and understanding these messages ripple forth from the deeply divided histories of these cities – the social consequences of removing and banning them would be unpredictable at best.

Finally, companies developing self-driving cars and governments who welcome self-driving cars into their cities should do their best to understand sticker culture. Seeing stickers simply as an obstacle to their technology might backfire. Removing stickers without further consideration of the culture could cause people to revolt in protest with even more stickers. Seeing the safety issues as well as the branding risks this entails for companies, better solutions should be able to be found by going into dialogue with the culture.