Interim Conclusion #1: Image Recognition Systems Disappoint on images of Food

For the project Recognizing Food from Around the World, I’m testing images of food from around the world on four of the most popular image recognition systems. During this project, I’m trying to figure out how well these systems (1) work on real life, original images and (2) how well these systems interpret nuances between cultures and countries, and (3) where these systems can make improvements.

So far, I’ve tested the systems on food from Belgium, Myanmar, Vietnam and Malaysia (9 meals and 11 images). With these admittedly limited samples, I wanted to write an interim conclusion to make the results of the project easily digestible.

Thus far, the results have been disappointing to very disappointing. Object detection consistently failed across the four systems. The systems labeled the images better, though these labels also often included too general or irrelevant descriptions.

In some cases, these descriptions culturally misrepresented (e.g. mistake the country or culture of the dish) the food, which could be controversial in some contexts. In one instance, “Chicken” was described as “Beef”, which in some contexts could disadvantage people from certain religions. In another instance, “Mock Meat” was labeled as “Meat”, which could similarly disadvantage people from certain religions as well as vegetarians/vegans.

Please stay tuned for more results!