It's finally here! About a week ago, I ordered two Zotac MAGs, a $269 mini computer, with the hopes of making my own miniature computing cluster. The two dual-core MAGs, which run Ubuntu 10.10 like champs, allow me to harness 4 extra cores through gridMathematica. One of my favorite features of Mathematica is the LightweightGrid, a tool that allows a Mathematica user to launch Mathematica kernels on computers running gridMathematica on the local network. The selling point of LightweightGrid is that it can turn quiet computer labs into multi-node computing clusters almost instantly.

However, as a Java enthusiast, I am more interested in Hadoop, an open-source cluster computing framework based on the Hadoop Distributed File System (HDFS). Hadoop uses a framework called Map Reduce, where a master node “maps” out a function to slave nodes (i.e. function mapped over {a,b,c} sends function[a], function[b], and function[c] to different slave nodes). Then, a defined “reduce” operation takes the inputs returned from the slave nodes and reduces it to the desired output.

As of right now, my Hadoop cluster, named “spork” after the ingenious utensil, is under construction. If my brain isn't melted from studying for midterm week, I'll have some more to write about in a couple days.