Stem Intiative

Analyzing Map Reduce Frameworks Hadoop and Twister

Student(s):  Joyce Bevins, Elizabeth City State University
Autumn Luke, Elizabeth City State University


The primary focus of this research group was to analyze the attributes of MapReduce frameworks for data intensive computing and to compare two different MapReduce frameworks, Hadoop and Twister. MapReduce is a data processing framework that allows developers to write applications that can process large sets of data in a timely manner with the use of distributed computing resources. One of it's main features is the ability to partition a large computation in to a set of discrete tasks to enable of parallel processing of the computation. Google, the most popular search engine on the internet, uses MapReduce to simplify data processing on it's large clusters. We analyze the performance of Hadoop and Twister using the Word Count application and compare the scalability and efficiency of the two frameworks for this particular application.