More accessibility in Maps - Text on Photo Translated Automatically

ErmesT · February 14, 2021, 1:37am

This evening I had an incredible surprise, which I want to share with the whole community
A further step forward for the accessibility program, but also for people who want to travel to another country and want to explore a place in Maps, e.g. to see a restaurant menu.
It can be difficult if the menu is in another language and is “in a photo”
From today this may no longer be a problem, because the translation can be done directly in Google Maps in Android.
Yes, you got it right, the text “in a photo” can be translated directly “from Google Maps”
All this thanks to the integration with Google Lens

And from Google Maps you can directly open Google Translator, and listen the text, if you are visually impaired.

The integration with Google Photo has been going on for a long time, and therefore the arrival of Google Lens was certainly expected by many (I was talking about it yesterday with a friend), but to discover that it is already real was a huge surprise.

Today I visited a beautiful park near Treviso. An important place, the place of the last Italian battle of the First World War (I will write a post soon). I entered through the farthest gate, and at the entrance there was a sign with the rules to follow. Since no one usually adds these useful signs to Maps, I thought I’d do it myself.
When I checked the photos I had added, I couldn’t believe it. Google Maps asked me (my phone is set in English) if I wanted to translate the sign. Of course I said yes, and here is the result

Here below the complete screenshots, so you can see how it works

Have you already seen this function?

If not, and you want to test your device, here is the link to the photo: https://goo.gl/maps/zxstZw2Lh4KtAhrj7

EDIT: for some reason the link doesn’t work on Mobile. However, here is the photo (embedded from Maps)

If yes, please share your screenshot, I am curious to see with how many languages it works

Today for me is a great day, a huge step forward for the integration.

Have a nice day

Ermes

PaulPavlinovich · February 14, 2021, 2:02am

Wow that’s pretty good. I’ll definitely find that handy

In the past I’ve taken screenshots then asked lens to translate but this is great.

Thanks for showing us @ErmesT

LaloPadilla · February 14, 2021, 2:24am

Great @ErmesT , thanks for sharing this useful tip! It’s nice when there is a combination of Google products and it works seamlessly

This will come in handy when traveling abroad and dont make anything illegal haha. I have tried too and I found something really odd, at least in my photos it is only available when there is a large text, not signs (like yours) or menus, but it works!

Caption: Information board in a museum, spanish on the left , translated to english on the right (LG@LaloPadilla )

Kinjalchhaya · February 14, 2021, 3:43am

Wow excellent news and update thanks @ErmesT for sharing this. Such feature with regional language support would definitely be very useful in domestic travel too.

JanVanHaver · February 14, 2021, 6:48am

Great find @ErmesT . I get an error when trying to open the link to your picture, but when looking around a bit on random POIs I soon discovered that this is indeed a new thing for any picture with a decent amount of text.

Here’s a screenshot of another picture from that same POI where the text is translated from Italian to Dutch.

JanVanHaver · February 14, 2021, 6:57am

After some testing with the language settings, I have figured out that: although you can now switch the interface language in which you use Maps can now be switched inside the app, the Google Lens integration still uses the language selected as main language in your phone settings, not the Maps language @ErmesT

MortenCopenhagen · February 14, 2021, 7:46am

Hi @ErmesT

When I tap the link you provided for testing I get the following error message :

Unsupported link

Google Maps can not open this link.

This is the case whether the app language is set to English or Danish.

Cheers

Morten

NareshDarji · February 14, 2021, 7:51am

That’s really great feature! @ErmesT

I’ve just seen this feature on map app after reading your post.

Thanks for sharing with us.

ErmesT · February 14, 2021, 9:23am

I was doing the same @PaulPavlinovich , taking screenshots and asking lens to translate.

I think this is a first phase, @LaloPadilla , and I hope it will be improved in accuracy and resolution soon. But it is indeed interesting to have this function integrate directly in Maps

LuigiZ · February 14, 2021, 9:57am

Hi @ErmesT ,

this is a very nice discovery, and no I wasn’t aware about it. Until now I was translating signs with the camera live but indeed a great integration inside Maps.

Thank you for sharing the finding with us .

ErmesT · February 14, 2021, 10:15am

Thanks @Kinjalchhaya , I agree

I see @MortenCopenhagen , @JanVanHaver , it is weird that a link shared from desktop cannot be opened in the App. I’ve edited the post by embedding the photo

I did some extra text after your reply, @JanVanHaver

For the translation to work, the text must be installed in your device and, by default, the app suggest the language on which your device is set to.

Apparently the translation is made “offline”, and this make sense if you need to translate a text and you don’t have data connection.

So what I did was to install the Spanish language in my device. In this way I was able to change the translation from English to Spanish

Then I selected Spanish instead of English.

Here below the result

Is this function already available also on iOS, @FaridTDF , @KarenVChin ?

Have you tried it already, @Jesi ?

FaridTDF · February 14, 2021, 3:06pm

Hola @ErmesT es una gran opción, la que nos cuentas hoy. No, No verifique si ya esta disponible en America del sur y en particular en español, veré de salir el martes a recorrer Ushuaia y podré verificar esta opción, luego regreso por aquí y te confirmo.

Una cosa que descubro también es que no tenía instalado en el móvil con iOS Google Lens, el que generalmente usaba es Office Lens jejeje , veré ahora como funciona.

Farid.

JanVanHaver · February 14, 2021, 3:40pm

Aha, indeed, @ErmesT clicking the Lens languages-overlay opens a menu where you can select lanugages - really cool. I’m taking notes here for the news section of an upcoming podacast episode

ErmesT · February 14, 2021, 4:18pm

You don’t need to go out, @FaridTDF

Just click on this link and scroll down to find the photo in the place.

Tap on the photo and see if maps is offering you to translate the photo

Jesi · February 14, 2021, 5:45pm

I just tried it @ErmesT ! It works perfectly and it seems to be very useful for trips and restaurants with international cuisine that don’t translate names of dishes. I tried it on another location, just to make sure the feature is global:

This is from an Italian restaurant in Miami

I also checked the “Explore the dishes” option that I had forgotten about hahah. I remember the announcement and trying it but then never using it after that, I’m sure it’s very useful for others though. I guess I just got used to deciding based on photos instead of the menu. Do you use that feature?

ErmesT · February 14, 2021, 8:39pm

I discovered the feature just yesterday, @Jesi , and I was surprised not finding any information, nor any post about that.

So I decided to share my experience in this post

Jesi · February 14, 2021, 9:17pm

I guess I wasn’t clear enough @ErmesT

I was asking about the “Explore dishes” feature I mentioned in my last paragraph, not the translation feature you made the post about.

Well, it’s explorar platos in Spanish as you can see, I’m not sure how it is called in the English version but if you check on Maps a photo of a menu, it appears. I was reminded of it as I checked the menu I shared in my earlier comment, it gave me both the option to translate it and to “Explorar platos”, both use Lens as you can see.

(Explorar platos scans the menu and (tries) to show you photos of the dishes that it finds)

What I was saying is that even though I remember when this one (the explore dishes feature) was announced, I haven’t been using it. Do you regularly use this one?