How a neural network learned to recognize Russia

Yandex, Russia’s biggest Internet company, has released an online game that invites players to guess where photos were taken. The images are taken from a database of photos uploaded by users to the Yandex.Maps app (similar to Google Maps). The human players compete against the specially trained neural network Alice, which is already used as a voice assistant in many Yandex products.

To teach Alice to identify cities, the developers took user photos from the Yandex.Maps database and divided them into two groups. Alice was shown the first group of images and told where each shot was taken. After viewing several thousand photos, the neural network formed an idea of ​​each country based on a specific set of features.

To find out which details Alice was using to recognize a particular city or country, the developers cut each photo into small squares and again showed some of these pieces to the network. After that, Alice was shown the other photo segments that "she" hadn’t seen. Despite often missing the mark, the system identified some fragments successfully. The developers were thus able to work out which fragments were helping the neural network to guess the locality.

Yandex

“For example, Alice recognizes the Netherlands by the combination of dark brick walls and white contours of the windows, and Iran by the characteristic arches and ornamented facades,” said the Yandex press service.

The second group of shots is used only in the game. These images the neural network has not seen before and is not programmed to remember during the game.

For now, humans have a slight advantage over the neural network, since Alice has not learned how to recognize car license plates, texts on signs, or national flags. And under the rules of the game, the system is not allowed to search for similar pictures online, which would greatly simplify the task.

To challenge Alice to a game, click here. And try to spot the difference between Samara and Kostroma here.

If using any of Russia Beyond's content, partly or in full, always provide an active hyperlink to the original material.