Beyond “Hello World!” with MapReduce

If you’ve heard about MapReduce, chances are you’ve already come across the “Hello World!” of the genre : counting words occurrences. While it certainly helps wrapping your head around the main idea behind MapReduce, its usefulness is somehow limited.

Steve Krenzel, wrote a article entitled “Finding friends with MapReduce” which, after going through the basics, explains how you could use MapReduce to compute friendships on a social network, to offer features such as “You & John have XXX friends in common”.

The article does a great job at explaining how the data is processed at each step, and what the output should be. I wrote a quick python script that does exactly what Steve explains, you can find it at http://gist.github.com/184137

I hope it’ll be useful to someone.