escutcheon

Twitter Statistics with Yahoo Pipes and the Google Chart API

Below is a segment from a scatter plot generated using a specific Twitter user’s “tweet” data, gathered via Yahoo Pipes and rendered using the Google Chart API.

tweet scatter plot

Scatter plot of Tweets - Tweets by Hour and Day of Week

The most interesting aspect of this project is that, other than the static files being served up by xefer.com itself, no other server-side processing is being done here; all the processing is being handled by Yahoo Pipes and Google, and of course Twitter itself which is ultimately hosting the raw data.

Twitter Id:

The initial gathering of data is pretty slow, but subsequent runs are faster because of Yahoo’s caching of the generated output.

The client-side javascript processing the raw data for Google Charts is here.

Update: times should now be displayed in local time, not GMT. Thanks Andy for pointing out the issue.

» Posted: Thursday, May 1, 2008 | Comments (17) | Permanent Link
fleuron

Comments

This is awesome. One thing, the times used seem to be in GMT. (I don’t think I ever tweet at 6 am, much less 27 times).

» Posted by Andy on May 2, 2008 04:54 PM

Argh! You’re right… I’ll see what I can do to fix this up. The whole goal of this project was to do as little processing as possible, but that’s something that should be addressed.

» Posted by winter on May 2, 2008 09:42 PM

Seems like the site can’t find my tweets, even though they’re public. Same goes for other users I’ve tried, such as kevinrose.

» Posted by slonkak on May 6, 2008 10:57 AM

I’ve seen occasional failures from the Twitter side (i.e., 500 Server Errors) which are not currently being bubbled up. FWIW these worked when I tried them just now:

http://xefer.com/twitter/slonkak
http://xefer.com/twitter/kevinrose

» Posted by winter on May 6, 2008 11:11 AM

This is awesome, I like!!

» Posted by joe on May 6, 2008 11:18 AM

That’s cool !! There’s some great potential here…

» Posted by Tim on May 6, 2008 12:25 PM

Interesting site. One comment though, the list of people that you’ve twittered to should only show each person once. But it’s classifying them based on the case that they were entered.

eg. Griffmiester is different to griffmiester is different to gRIFFMIESTER, etc.

Can you fix this?

Thanks,
griffmiester

» Posted by griffmiester on May 6, 2008 12:39 PM

Yes, that’s simple enough to do. I won’t be able to get to it until a bit later on today though. I will try to ping you when this is ready.

» Posted by winter on May 6, 2008 12:45 PM

Why is it case-sensitive? IDisposable fails, but idisposable works.

» Posted by Marc Brooks on May 6, 2008 04:36 PM

It shouldn’t be case-sensitive. I’ve occasionally seen Pipes return empty data structures, but then return the correct ones upon refresh. I’m not sure what to say other than that Pipes is still technically in Beta. At any rate, both these links now return identical charts as they should:

http://xefer.com/twitter/IDisposable
http://xefer.com/twitter/idisposable

» Posted by winter on May 6, 2008 04:57 PM

Very nice ! And very interesting. I am psychologist and i study the interactions which occur in theses worlds. It would be helpfull for me to calculate two or three things (in and outdegree etc). But for that i need to access to the rough data. Does it will cost you a lot a work to add this option. Guess format would be nice. Or CSV :-)

» Posted by Yann Leroux on May 8, 2008 03:33 AM

Showcased your work http://blog.instantcognition.com/visualization/2008/05/23/twitter-charts-an-excellent-use-of-black-and-white/

» Posted by Clint on May 23, 2008 07:39 PM

I use this quite regularly and couldn’t keep sending you a word of appreciation.

Great Work. Thanks.

» Posted by Aravind on June 19, 2008 11:14 AM

Very nice.

To make the display even more information dense and correlated, you might try placing the day-of-the-week histogram (rotated 90 degrees clockwise and flipped) on the right-hand axis of the scatter plot, and the hour-of-the-day histogram at the top of the scatter plot. (Both histograms would need be stretched to align the ticks. When arrranged this way, the histogram bars should probably be less that 100% black, so as not to compete too strongly with the scatter bubbles.)

[I can send you a crude cut ‘n’ paste GIF of what I’m blathering about.]

Cheers,

Tim Black
Software Designer,
Axonwave Software

» Posted by Tim B. on June 25, 2008 12:49 PM

Tim, I had actually toyed with doing just that, but set it aside. I’d like to ultimately produce a single image (even if it’s composed of a number of independent PNG sources) that captures everything the four independent images capture.

Something in the spirit of Minard’s chart made famous by Tufte. :)

» Posted by winter on June 25, 2008 03:06 PM

I put the image here:
http://www.flickr.com/photos/causticsurface/2611526356/sizes/o/


—Tim B.

» Posted by Tim B. on June 25, 2008 04:00 PM

Nice. I’ll see if I can’t make this work.

» Posted by winter on June 25, 2008 04:13 PM
fleuron

Post a comment