Posted on May 26th, 2009 by Neil Crosby. Filed under Blog Posts.
One of the things that’s annoyed me for quite some time about twitter is the lack of an easy way to follow a question/answer flow. Fairly often someone will ask an interesting question that I’d quite like to see the answers to, but right now there’s no easy way to find the tweets which were written in reply to the original question.
This is unfortunate, since twitter does provide an
in_reply_to_status_id parameter for tweets, which is set when one tweet is specifically written in reply to another. If we were able to query this parameter directly then the Twitter Conversation Yahoo! Pipe that I’ve written would be unnecessary and I wouldn’t have had to spend a couple of hours writing it. But I did have to write it, and that’s okay because it gave me a much better understanding of how to use Pipes.
In order to track twitter conversations (between non-private users), I did the following:
status_idof the tweet that started the conversation.
This can found at the end of the tweet’s URL.
Make a request to the twitter API to find out who wrote that tweet.
This isn’t strictly necessary – the user could provide this too. However, by working this out for them the user only has to provide one piece of information.
Make a request to twitter search for any tweets written since the original tweet that contain “@username”, where ‘username’ is the username of the original tweeter.
For each search result make a request to the twitter API to find out if that tweet has its
in_reply_to_status_idparameter parameter set.
Yup, that’s an API request for each of conceivably a lot of tweets. If the twitter search API returned this data as part of its result set that I could remove this call and be a much happier man.
Discard any search results which do not have
in_reply_to_status_idset to the
status_idof the original tweet.
At this point, I originally wanted to recurse the pipe and run through this process again for all the tweets I’d just found via the search so as to get a richer conversation experience.
Unfortunately, due to the massive number of HTTP requests required, Pipes understandably started timing out.
Finally, add the original tweet to the list of tweets if required and order by time, starting with the original tweet.
There’s a fair bit going on there, and things get a bit complicated in the Pipe due to the way various properties had to be manipulated. It all ends up working pretty well though.
This Pipe becomes more useful once you add a quick little bookmarklet to it. Drag my Twitter Conversation bookmarklet to your browser’s bookmarks folder, and then click on it any time you’re on a tweet page that you think will start an interesting conversation. You’ll immediately be taken to a Pipes page which contains the conversation. Then, you’ll be happy.
Paul Boag: wishes that when somebody asks a question on Twitter you could easily see if anybody else has answered it.
@boagworld You could check out @NeilCrosby’s conversation pipe built for pretty much that purpose. http://bit.ly/bCkvL
Clearly there’s a want out there for people to be able to see these conversation threads.
The fact that I’ve been able to fairly easily hack together the proof of concept in Pipes shows that it wouldn’t be the most difficult thing in the world for twitter to implement themselves. Even if twitter just “flipped a switch” and turned on output of
in_reply_to_status_id in search API results then this hack would become a whole lot more palatable due to the massive reduction in HTTP requests this would mean. It might even mean that Pipes would be able to recurse into the conversations, which would be fantastic.
If you enjoyed this post, subscribe to The Code Train and read more when I write more.