The Code Train

Where Neil Crosby talks about coding on the train…

RSS Entries

Using last.fm tags to make awesome playlists in your local iTunes library

Posted on August 18th, 2010 by Neil Crosby. Filed under Blog Posts.

As I’ve mentioned previously, I have a large music collection that I manage using iTunes. One of the problems with any large collection is the curation and management that goes with it. I carry my iPhone around with me and I listen to music on it, but even the biggest iPhone available would hold only fifth of my collection. So, I need techniques for pulling interesting music from my iTunes library into my iPhone.

Up until now I’ve used a collection of Smart Playlists which take into account my ratings, when I last listened to songs and a whole bunch of other information. At the last count, I had over 70 Smart Playlists feeding into each other to generate the playlists which finally get Synced over to my iPhone.

But there was something missing. Whilst I could generate playlists based on the mid-90s simplicity of the “Genre” tag, that tag is by design only able to identify one piece of information. So I wouldn’t be able to find, for example, all tracks in my collection that are “rock”, “funny” and performed by a “female”. And even if I could, I wouldn’t want to go and tag 40,000 tracks by hand. That way madness lies.

It turns out there’s a fairly simple solution to this – last.fm allows users to tag any and all tracks it knows about, and it keeps track of how many times each track has been tagged with each tag. So, it’s entirely possible to grab the top 15 tags for each track in my collection, add that data to my tracks somehow, and then query that.

The somehow turns out to be pretty simple too – the ID3 “comment” field is there for the taking, and by design is expected to be larger than any other field. Excellent. So that’s the “where to store” sorted, now how about “how to store so that the data is queryable”?

For this I took a trick out of Brian Suda’s book. A couple of years back I took a look at how he’d written his microformats parser using XSLT to be able to check for classes an element may or may not have – the trick being to wrap strings of classes with a space at the beginning and end so that every class always had spaces surrounding it. I used the same trick for storing my tag data, also wrapping it in a ‘lfm’ square bracketed enclosure to separate it from any other data in the comment.

So, the data I saved in the comment looked a little something like this:

[lfm: tag1 tag2 tag23 tag3 ]

You’re probably starting to see why wrapping in spaces is important now. Because iTunes’ Smart Playlist system can only perform simple string matching (Apple doesn’t seem to like regexes – AppleScript can’t do them natively either), you need some way of targetting the beginning and end of tags. Without those spaces, a query of “Comments contains 'tag2'” would return you both tag2 and tag23, which no-one wants. By requesting “Comments contains ' tag2 '” instead (note the extra spaces) you end up just being given tag2.

All this thinking is packaged up into a couple of scripts (though obviously take the latest code if you want to actually use it yourself) I wrote a couple of days ago that takes a given playlist from your iTunes library, asks last.fm for tags for each of the songs in that playlist, and then adds that data to the library using AppleScript.

Problems?

There are, unfortunately, a couple of issues with the script at the moment. First off, because last.fm allows anything and everything to be given as a tag, they can contain characters that need to be normalised out. Right now I’m doing that very simply (spaces to dashes mostly), but I should revisit the code and harden that normalisation – I’ve seen tagging bail out a couple of times because of unexpected data.

Next up, the fact that last.fm allows tags to be incredibly long coupled with the fact that under ID3 v2.x the comments field can only be 256 characters long means that it’s possible to run into issues with tagsets being too long to fit into the comments field. Add that to the fact that some songs in my collection legitimately contain comments already and that I want to add tags non-destructively, and we’re ending up with not much space for tags. So, an improvement that I’m planning on making to the script is to make sure that tags try to take up no more than the space that’s legitimately available to them.

The final issue that I’ve been having lives with last.fm’s track.getTopTags API method. Unfortunately at present the track.getTopTags API method does not perform any auto-correction of track names, which means that if you try to get the tags for something that does get auto-corrected by the system then you end up with the tags for the uncorrected track name. A subtle distinction, but it probably means that we end up with unmaintained tags that haven’t been touched for the year or so since auto-correction was turned on. A possible solution here if last.fm choose not to “fix” this behaviour (it could be argued that this is correct behaviour) would be to make a request out to Musicbrainz to get an ID for the track and then pass that into track.getTopTags instead. But that seems like a whole bunch of extra work for a not massive gain.

The final Smart Playlist

So, what about that Smart Playlist? After all, that’s the thing that all this work was done for.

In iTunes, select File, New Smart Playlist, and then enter the following:

Match all of the following rules:
    Comments contains ' female '
    Comments contains ' rock ' ...
    any of the following rules:
        Comments contains ' comedy '
        Comments contains ' funny '

This cunning playlist uses a Smart Playlist feature I wasn’t aware of until yesterday – rule groups. Up until now, I’d been pulling in other Smart Playlists as sources to perform the same job.

To create a new rule group in a Smart Playlist, simply click an ellipsis button in the Smart Playlist interface, rather than clicking on the plus button.

Essentially what these rule groups give you is the ability to nest rulesets, and generate very complex playlists that don’t have half their rules hidden in external playlists. In the playlist above we’re simply asking for all tracks tagged with “female”, “rock” and either “funny” or “comedy”. So essentially “female comedy rock”. At present this playlist returns me no items from my collection, but not everything’s been tagged yet. I’m hopeful.

Tags: , , , ,

If you enjoyed this post, subscribe to The Code Train and read more when I write more.

One Response to “Using last.fm tags to make awesome playlists in your local iTunes library”

  1. Hey Neil,

    Just found this… I’ve been looking around for something similar for years! Why has the music distribution industry picked up on how users would absolutely love to tag and sort their music similar to how they tag and sort photos? It only makes sense.

    Problem is I’m pretty awful with any kind of code related things… but I find that your approach definitely makes sense (from what I can understand of it.)

    Is there any sort of tutorial for it? I’m really interested in it. This functionality should be built in to modern music organizing applications, IMO.

    So I’m wondering if you may have had any leads on a current solution to this conundrum we’ve been faced with… or if you’re able to point me towards a tutorial or give me some insight on your methods.

    Any response is greatly appreciated!

Comments RSS

Leave a Reply

TheCodeTrain Theme by Neil Crosby, Powered by WordPress