It is critical to test self-driving cars (SDCs) in real world conditions. However, it is often unclear which real-world conditions to actually test in. The following article presents a blueprint for testing SDCs in the urban environment of Riga, Latvia.

Context

Self-driving cars (SDCs) have failed to recognize their surroundings in manners that baffle most humans. For instance, these cars failed to properly recognize kangaroos, traffic signs, and people with a dark skin.

SDCs fail to recognize such seemingly simple things because they learn to make sense of the world in a very different manner than humans do.

One of the biggest technical issues for self-driving cars right now, is recognizing all the obscure yet important elements they may encounter in real-world conditions (edge cases).

In this article, I provide a blueprint of simple, yet critical elements that SDCs need to deal with, but simultaneously could have difficulties with in the urban environment of Riga, Latvia. As such, this blueprint helps plan for SDC testing in real-world conditions.

Approach

Through direct observations in Riga and secondary research (i.e. interviews, case analyses, etc.), urban elements of critical importance and of potential relevance to SDCs were identified.

An image recognition (IR) system* was then confronted with a set of original images of the identified elements.

Finally, noteworthy results of the previous step were translated into a concrete and actionable blueprint for testing SDCs based on the intimate knowledge of Riga gained in the first step. Three parts of this blueprint are summarized below.

Insights

Implicit traffic signs and road markings

Traffic signs and road markings help drivers safely navigate roads. Generally speaking, these signs and markings are highly uniform in appearance and placement, making them easily recognizable (by humans as well as SDCs).

However, uncommon signs and markings were found throughout Riga. For instance, as seen in the picture below, flower pots block the passage between two roads, implicitly acting as a stop sign. While human drivers should recognize such implicit signs with relative ease, SDCs might have a much harder time recognizing them.

• 

In fact, out of 216 collected instances of 70 different implicit signs, only 25 instances were correctly detected. All 25 of these instances were either “street lighting” or “flowerpots”. Instances such as chains, poles and concrete pillars were not recognized (see pictures below).

• 

• 

• 

• 

Why it matters: implicit signs and markings perform the critical job of directing traffic. Often times, they cut off traffic from areas with many vulnerable roads users, such as a market square. Missing these implicit signs and markings could result in driving into them or, worse, hitting pedestrians on the opposite side.

While traditional signs and markings sometimes compliment the implicit ones, this certainly isn’t always the case. As a result, SDCs are entirely dependent on detecting the implicit signs and then understanding their implicit meaning. The local varieties in the design of these signs and markings (e.g. a flower pot vs. a chain) make detecting and understanding them even harder.

Recommendation

Test for implicit stop signs and markings throughout Riga’s inner city. These can be found near, for example, Jēkaba iela and Aldaru iela. Implicit stop signs and markings are present all year round in any weather.

The wide variety of terraces

It’s nice to have a drink on one of the many curb-side terraces in Riga when the weather’s sunny. However, due to the narrow side walks and limited open space, these terraces often stand fully or partially on the streets (see pictures below). As they form obstacles on the road, it’s critical for SDCs to detect them, especially people sitting in them.

• 

• 

• 

• 

• 

• 

Unfortunately, out of 67 instances of 33 different terraces, none were properly detected as a whole. Individual elements on these terraces, such as chairs, tables and plants, were sometimes recognized. However, in most cases, these individual elements were undetectable behind the outer structure of the terraces. On a good note, the people sitting on the terraces were recognized most of the time.

• 

Interestingly, some terraces (usually on bigger streets) have signage on the side of them, making them more visible to human drivers. In this analysis, the extra signage did not improve the detection of the terraces. Nevertheless, it could be an example of how to make terraces more uniform for SDCs in the future.

Why it matters: terraces are placed in the middle of the streets throughout Riga (figuratively of course, because usually they’re a little to the side of the street). Therefore, SDCs need to be able to recognize and navigate these obstacle without crashing into them.

As people often sit on these terraces – relaxed and unconcerned about the traffic around them – it’s especially important to properly detect them. Simply recognizing individual elements (e.g. chairs) on the terraces is not enough. SDCs need to understand the context of terraces and act accordingly.

Recommendations

Test for terraces throughout Riga’s inner city during spring and summer time.

Take the time to capture the widely varying looks of terraces. Centra Rajons is a great area to test in, but a custom test map is needed every year as specific terrace locations change.

Quirky tourist transportation

Capital cities like Riga attract many tourists each year. Often times, these cities offer quirky means of transportation to guide tourists around. To capture the attention of these tourists or to practically accommodate sight-seeing, these means of transportation look quite different to the vehicles generally on roads.

• 

• 

• 

Out of 67 instances of 20 different vehicles, only 6 instances were somewhat properly detected. All six of these instances were e-bikes recognized as “motorcycles” (see the picture below). The electric cars were twice recognized as “bus”, once with a high confidence rate of 89%. One instance of a rickshaw was detected as a bicycle. In most of the 67 instances, some “tires” and “wheels” were separately detected.

• 

Why it matters: quirky means of tourist transportation represent almost the definition of edge cases for SDCs. First, to attract the attention of tourists, they look unlike any other means of transportation. This makes it more difficult to detect them. Secondly, they don’t appear on the road often, and only in certain places at particular times. This makes it harder to catch them during road tests.

Nevertheless, tourist transportation forms an integral part of Riga’s road network. SDCs need to be able to adequately detect and respond to them, especially because a lot of people ride on them.

Recommendations

Test for all the different forms of tourist transportation. In Riga, these include the electric tourist cars, rickshaws, party bikes, and e-bikes. A custom map is needed to optimally test for these.

Conclusion

In this article, three insights into testing SDCs in Riga, Latvia were given. These insights advocate for the testing of implicit traffic signs and markings, terraces, and tourist transportation. Where applicable, information on where and when to test for them were given. For others, custom test map needs to be developed depending on the planning constraints of the roads tests.

Though edge cases can never be ruled out by definition, the presented insights make testing more efficient, reliable and, ultimately, reduce the chances of SDC traffic accidents.

For the full report on testing image recognition and SDCs in Riga, Latvia, please send me a message.

*Google Vision