How I ran the local AI audit

What did I ask?

One plain question, the way a normal customer would ask it: “What are the best [trade] in [city]?”

I picked six trades people look for locally all the time: plumbers, dentists, electricians, car mechanics, hair salons and real estate agents. And four cities in four countries: Austin, Manchester, Toronto and Dubai. Six trades in four cities makes 24 searches.

The wording never changed, so the answers can be compared fairly. On Google Maps I searched the short version, like “plumber in Austin, TX”.

Which AI assistants did I test, and how?

Five of them. Every question went into a brand-new chat, and I saved the full answer with its date.

I wanted to test signed out, because that’s closest to what a stranger sees. ChatGPT and Gemini let me do that. The other three were different.

Google AI Mode showed a robot check on 2 October. So I collected it the next day, from a different browser, signed in to a Google account.

Claude needs an account. I used its incognito chat, which doesn’t use the account’s memory and doesn’t save the chat. The model was Sonnet 5.5, with web search on.

Perplexity needed an account too. I asked the first 11 questions in its Incognito mode. Then the free plan stopped me, so I asked the other 13 on a second, brand-new free account. I switched its search history off, so earlier questions couldn’t shape later answers.

ChatGPT got every question three times, to measure how much its answers change. The other four got one ask each. Signed out, Gemini runs on Google’s lighter model (Flash-Lite), so that’s the version I tested. I didn’t change any other settings.

How did I record Google Maps?

For every search, I recorded the first 20 businesses Google Maps listed, in order, with the ads taken out.

I did this on 2 October 2026, the same day as the ChatGPT and Gemini answers. For each business I kept its position, name, rating, number of reviews, website and Google’s own ID for the place, called a place ID. That’s 480 listings across the 24 searches.

I compare against two groups. The top 7 is roughly what you see first: in my first test, signed-out Maps showed 7 regular results on the first load. The top 20 is the wider net.

One thing I learned: Maps rankings move, even during the day. I loaded a few searches twice. For Austin plumbers and dentists, the top 7 came back exactly the same. For Austin electricians, within about an hour, two businesses swapped places and one dropped out of the top 7 while another came in. Where I had two loads, I used the first.

How did I decide two businesses were the same?

Carefully, because names don’t always line up. A match means the same business, at the same branch, as the listing on Maps.

• Google’s place ID firstWhen Gemini or Google AI Mode showed Google’s own place card, I matched it by its place ID. That’s exact, with no guessing.
• Then name and review countFor the rest, I matched by name, ignoring bits like Ltd or LLC, and used the review count to confirm it.
• Same branch onlyA chain only matches the same branch on Maps. Another branch of the same company doesn’t count.
• An agent isn’t their brokerageA real estate agent only matches if the Maps listing is their own team or office, not just the big firm they work for.
• Only real picks countNames mentioned in passing, and ads marked Sponsored, are kept separate and never counted as picks.

Booking apps that an assistant recommended, like Urban Company for Dubai plumbers, do count as picks. They’re not on Maps, so they count as outside the top 20.

Then I went through the matches by hand. I checked every unclear pair against the Maps listing’s name, address and review count. I also hunted for near misses, like “Smooshine” in an answer and “Smooshine Hair Salon” on Maps. That found 16 pairs the name rule had missed, all in the other assistants’ answers and none in ChatGPT’s.

What went wrong, and what did I do about it?

A few things. When an answer had to be replaced, it was because the capture broke, never because of what it said.

The answers I replaced aren’t used in any number. A note in each answer file says what happened.

What can’t this audit tell you?

Quite a bit, and it’s better to say so up front. This is a careful snapshot, not the final word.

It’s two days. AI answers change from one ask to the next, and Maps rankings move too. Three of the assistants were asked a day after I recorded Maps. Ask on another day and you’ll get a different list. That’s why I report patterns and percentages, not single rankings.

It’s mostly signed out. Signed-in answers can be personalised. Three of the assistants were signed in, as explained above, so keep that in mind when you compare them.

It ignores where you’re standing. The city was in every question, so the results are for that city. But Google also weighs how close you are, and I didn’t test what someone in one part of town would see.

It shows what, not why. I can see which businesses each assistant picked. I can’t see inside the assistants, so I don’t claim to know where their data comes from.

It’s 24 searches. That’s enough to show patterns, and too few to prove rules. Treat a result for one city or one trade as something to watch.

How can you check my numbers?

Download the data. Each article in this series comes with its own dataset, and every number traces back to it.

The dataset has all 1,352 businesses the five AI tools named, and where each one sat on Google Maps. One row per business, with the city, the trade, the AI tool, the ask and its position in the answer. It’s a CSV file, so it opens in Excel or Google Sheets. It goes with the first study, How do ChatGPT and other AI tools choose local businesses?

One detail you’ll spot: for Austin electricians, Maps’ top 7 moved within an hour. The top 7 column uses the earlier load, taken minutes before the AI answers, so U.S. Electric Contractors shows as position 8 but in the top 7.

I’ve kept the full text of every answer with its date, and the Google Maps list for every search. If you spot a mistake, tell me. I’ll fix it and note the change on the page.