Even though Beer Near Me was only ever really built as a weekend project intended to help me learn about using the navigator.geolocation API, it was still pretty important to get some data to display on a map showing where the local beer was. After all, without that what would the point be?

Gathering the Data

To gather the local beer data I make several searches using the Google Local Search API – one that covers pubs, one for cocktail bars, one for off-licences, and so on. The searches I make to Google’s local search look something a little like this:

http://ajax.googleapis.com/ajax/services/search/local
    ?v=1.0
    sll={$sll}
    radius=1
    q={$thing}%20loc:{$sll}
    key={$apiKey}

Lets walk through those query string parameters…

• sll doesn’t actually stand for anything according to the docs, but denotes the centre point of of the search you’d like to perform. It’s a comma separated latitude/longitude value, so you’d use something like 52.2282962,0.1537945 for this parameter.

• radius is the radius around the centre point that you’d like to search within, in miles.

• q is the most important parameter by far though; being the query term for the thing you’re wanting to search for.

  You’ll notice that I’ve added a loc: onto the query. It turns out that without this undocumented extra bit of data, Google really doesn’t return much from its local searches. Before last week’s post I wasn’t using this extra qualifier, and whilst my searches were being centered around the expected point they weren’t returning much data.

  A case in point was around The Bricklayers Arms. Without the loc: qualifier The Bricklayers Arms was nowhere to be found. With the qualifier it suddenly started being returned by the searches to Google, along with a whole bunch of other local pubs.

Interpreting the Data

So, if that’s the search we make, what does the data we get back look like?

{
    "responseData": {
        "results": [
            {
                "GsearchResultClass":"GlocalSearch",
                "lat":"59.334743",
                "lng":"18.063034",
                "accuracy":"8",
                "title":"\u003cb\u003ePub\u003c/b\u003e",
                "titleNoFormatting":"Pub",
                "streetAddress":"Hötorget 13",
                "city":"Stockholm",
                "country":"Sweden",
                …
                "phoneNumbers":[
                    {"type":"","number":"08-782 19 30"},
                    {"type":"","number":"08-789 19 30"}
                ],
                "addressLines":[
                    "Hötorget 13",
                    "111 57 Stockholm, Sweden"
                ]
            },
            {
                // another result
            }
        ]
    }, 
    "responseDetails": null, 
    "responseStatus": 200
}

(The data above is, by necessity, just a snippet of what’s actually returned.)

You’ll notice first of all that each result we get has a GsearchResultClass of GlocalSearch set. This allows easy checking of the data type returned. For our searches, the result class will always be GlocalSearch, but other searches would return values for web searches, images searches and more.

Moving on to the data that actually interests us, each item has the expected lat and lng attributes, along with other useful smidgens of data such as title, streetAddress etc.

Of particular note are the title and titleNoFormatting attributes. You’ll notice that within the title field the word “pub” is encapsulated in encoded bold tags. Two things to note here; first, “pub” was the thing we searched for, so this gets emboldened in the standard display that Google would like to use for results (but we’re still provided titleNoFormatting if we’d like to have access to an unformatted string). Second, any formatting applied by google in its API results will be encoded in this way. In Beer Near Me, I take the easy way out for the data I’m needing, and use titleNoFormatting.

Moving away from the data that comes from a single API request, lets take a quick look at what happens once we move into the world of performing lots of searches at once. Since with Beer Near Me I’m making multiple calls to Google to gather beer related locations, there’s the pretty large possibility that more than one of the searches will return the same place within its results. The deduping I do on Beer Near Me is very simple in this case – I simply keep track of the latitudes and longitudes of all the places that have been returned. That way if any locations turn up in a position that’s already had data assigned it it I can ignore it. It’s not perfect, but it does the job well enough.

Caching the Data

The last thing I want to briefly mention that I’m doing with the data is caching it using PHP’s APC.

This was the first time I’d attempted using APC on one of my own projects, so I had to install it on my Ubuntu based dev box. I did that following these instructions.

The reason for using APC to cache the calls to Google’s Local Search is simple – network calls are expensive and slow. With on average five calls to Local Search occurring on each page load on Beer Near Me you’d hope that they wouldn’t be having to go across network every single time. So, I implemented a very simple cache check:

$url = "http://ajax.google…q=somesearch"
$urlHash = md5($url);

// Try and pull the data from cache
$results = apc_fetch($urlHash);

// If we don't find the data in cache we'll have to make
// an expensive call across the network.
if (!$results) {
    // Get the data however you'd normally get it
    …
    $results = curl_exec($ch);

    // now store the data in APC
    apc_store(
        $urlHash, // Key to identify the cache by
        $results, // Data from our curl call to store 
        $ttl      // Seconds - how long we'll cache for
    );
}

As you can see above, I’m only using two APC methods – apc_fetch and apc_store. Really, this is all you generally need for simple caching; one method for getting the data if it already exists, and one for storing it in the cache if you had to get the data from elsewhere.

With that in place, any further requests to that same location will be a lot faster, since we’ll have stored the results locally. Result. For Beer Near Me I can afford to be fairly aggressive with our caching, since for the most parts new pubs and cocktail bars won’t spring up over the course of a few minutes and become immediately available in Google’s search results.

No more locations

And that’s it. As far as I’m concerned, that’s everything I’ve done so far on Beer Near Me. It’s been a fun little weekend project, and whilst there’s more I’d like to do with it at some point, I think I’m going to leave it alone for a few weeks now. After all, Science HackDay is in just over two weeks time, and no doubt I’ll get wonderfully caught up in something new for that. Hopefully whatever I hack on will even be useful.

So, I went back to scratch and tried installing it on a web-server. Since my normal stack is Apache, MySQL and PHP I didn’t have any of the Java-based web-servers suggested by the W3C. My first attempt at getting the validator working was to use Jigsaw – the same web-server as the W3C uses to run their version on. Unfortunately, it didn’t work for me. Bugger.

So, I tried Tomcat. After following Basil Bourque’s instructions on Apache’s Tomcat wiki pages, I got Tomcat working. I then followed the rest of the CSS Validator installation instructions for Tomcat, and the validator worked in my browser. Excellent.

There was a fly in the ointment though. Whilst the validator worked in my browser, the same could not be said of my attempts to get it working via cURL calls from PHP using POST data. I was using the same code as I’d used to validate against the W3C’s HTML validator, so I knew that I was able to successfully push POST data across to servers. If I tried using GET requests to send the data to my local validator, then that worked. I verified the same thing was happening on the live CSS validator to make sure the problem wasn’t with my installation. I know you’re thinking “well, if GET worked, why not just make requests that way?” Unfortunately, I know that later I’ll be asking to validate much larger CSS rulesets — ones that will be far too large to fit on a URL. So, I needed to use POST.

After a fair bit of poking, I realised that the in-browser version of the CSS Validator was setting an enc attribute value of multipart/form-data on the form when it was uploading files or sending textual data. Conversely, it turns out that PHP was sending application/x-www-form-urlencoded as the encoding type instead, and the W3C CSS Validator just isn’t set up to accept that. After a long time of trying to work out how to send things in multipart/form-data format using PHP’s cURL functions, I happened upon a comment on PHP’s curl_setopt function which told me that if I gave PHP an array instead of a string it would automagically start using multipart/form-data. As if by magic, my problem was solved.

My problems setting up my programmatic connection to the CSS Validator could easily have been solved by clearer documentation. Nowhere that I could find in the Validator’s documentation was there any mention of needing to make POST requests using multipart/form-data. If this had been stated somewhere I would at least I have known the path I needed to be looking down. Likewise, lack of anything other than user-generated documentation for the PHP CURLOPT_POSTFIELDS cURL option made life far more difficult than it should have been.

Still, in the end I have a local version of the W3C’s CSS validator running on my laptop, and I’m able to connect to it programmatically to run tests. Which is nice.

These are the slides I put together for my talk:

If you just want to download the Code, go right ahead. If you want to know more about the project, read the WikiSlurp Project page.