April 9th, 2013

Local Search Competitive Research Tool

blogheader_tool

The Local Search Competitive Research Tool

This tool pastes the name, address, phone number, and URL from Google Maps’ search results into a nice and tidy Google Spreadsheet. Type a few keywords, a city, and a state.  That’s it, that’s all, and viola – you have a list of Google Maps results for your keyword terms. From there, you may do what you wish to help with local seo for your business.

These results are NOT based on rank. But why no rankings, you ask? Well, I simply haven’t thought of a way to connect rankings with unique results.  So with out rankings, I’ve used this tool should to speed up my competitive research of local businesses. The way I do my research, having an exact representation of where businesses rank is not as important as having a complete list of the businesses Google’s indexed.  My end goal is not to know where they rank but to cull opportunities from their citations – more on that later.

 Get the local search tool: make a copy from this Google Doc

Google Maps Scraper

Inspiration To Scrape SERPs

I’ve been playing with Google Docs and ImportXML for a few months now to scrape search results – from Google to Bing to Yellowpages.com. The scrapers are intended to eliminate tedious copying and pasting. But an added benefit is getting into the nitty-gritty with formulas. At the expense of being labeled nerdy, I’ll admit I view scraping and spreadsheet formulas as a fun way to excercise my left brain.

I first discovered the power of Google docs following Tom Critchlow’s Google Docs Unleashed presentation on App Sumo. I hardly understood it then, and have a cursory understanding now. Even with basic knowledge, I’ve been able to craft up a few tools to save me time thanks to Distilled’s ImportXML Guide for Google Docs.

The final push of inspiration to publish something about the local search tool came after I read John-Henry Scherck’s recent post about scaling link and citation prospecting.  In one of JHS’s steps, he copies and pastes local business information directly from the search result page. The local search competitive research tool can replace that process.

The Local Search Competitive Research Tool’s Potential Uses:

That was the initial inspiration, but I bet the scraper has potential beyond substituting copy and paste. I’ve come up with a few quick ideas. However, I’m sure there are more to be discovered with the creativity of the masses.

  1. Competitive research to locate link and citation opportunities.  See John-Henry Scherck’s post on scaling link and citation prospecting.
  2. Quick and dirty market research to understand who and what’s out there
  3. Scope competitors’ reviews. Are they actively gaining reviews? What sites is Google pulling their reviews from? Are the reviews positive or negative?
  4. Creative a Google custom search engine of their domains to search for products or services that you both offer
  5. Send them friendly greeting cards (outside of the box idea)
  6. Set up Google Alerts to be on the lookout for any mention of their business. (Note: I’ve only heard people say you can do this. I’ve tried but never had any good results come of it.  Blame Google Alerts or non-competitive verticals, je ne sais pas)

Why Google Maps and Not Google’s Local Results?

Google does some weird thing with the XML of their universal SERPs – making it difficult to find the right Xpath Query  for just the local results.  I imagine unlocking the Xpath Query to separate local results from universal results would be like Skywalker’s swish into the Death Star’s thermal exhaust port.

Anyway, even though Google Maps’ results are not an exact representation of what may show up in a SERP’s local-pack, the Maps’ results typically contain the same business – so it’s not that big of a deal.  My query for “snowboard shops near Philadelphia, PA” demonstrates the existing overlap of businesses:

Maps and Local Search Results Comparision

Being that this tool is not intended to pull rankings, but rather to help with local research, having an exact representation of local-pack rankings is not so important.

How it Works: Search Engine Result Pages’ (SERPs) URL Parameters

See Distilled’s guide for an indepth explanation of using ImportXML in Google Docs. Below is a brief run-down of the various ingredients I baked together for my personal use.

Search engine result pages’ URLs are like snowflakes, they are all unique. The uniqueness comes from the parameters within the URL. These parameters are simple variables we can tweak to change the results on a page. Here are the parameters I tweaked:

&start=

This parameter defines the start page for the results.  I use the first page, or 0.

&q=

This parameter defines search query, which is pulled from the inputs on ‘Le Scraper’ sheet.

&hq=

This parameter appends the extra query terms to the query.

&hnear=

This parameter takes your city and state input to define the location for your search.

How it Works: Scraping Info From SERPs with Xpath Queries & ImportXML

ImportXML is a function in Google Docs that uses Xpath Queries to crawl and scrape XML documents. Xpath has a unique query syntax that did, and still does, confuse me. After a bunch of tries, I found the right query combos to scrape Google Maps’ business information.*

These are the 4 Xpath Queries I used:

//span[@class='pp-place-title']/span”)

This Xpath Query scans the Google Maps results for just the title of the listing.

//span[@class='pp-headline-item pp-headline-address']/span”)

This Xpath Query scans the Google Maps results for the address.

//span[@class='telephone']/nobr”)

This Xpath Query scans the Google Maps results for the phone number.

//span[@class='pp-headline-item pp-headline-authority-page']/span/a/span”)

This Xpath Query scans the Google Maps results for the business’s URL.

*Note: if the page’s HTML structure changes, these queries will be useless and the tool will be useless.

How It Works: Clean Presentation Using G-docs

Each ImportXML function returns the top 10 results for the individual query.

The 4 columns of phone numbers, names, addresses and URLs are combined into one using the concatenate function.

From the concatenated list, the unique function extracts a “duplicate” free list of businesses onto the ‘Le Scraper’ sheet (note: duplicates still arise, be on the lookout).

 

Hopefully all this info will somehow be useful to you. You may even enjoy using the local research tool for your own do-gooding. If you do, please share your comments: did it help, did it hurt, did it create a massive headache? Any thoughts are appreciated.  Thanks for reading and, as always, keep it real.

Post A Comment