ABSTRACT
Processing ever-increasing amounts of information and providing a meaningful analysis of large datasets (Big Data) has become a significant computing challenge in the Enterprise environment. New tools, frameworks, and systems have been proposed for Big Data processing. MapReduce and its open-source implementation Hadoop represent an economically compelling alternative that offer an efficient distributed computing platform for handling large volumes of data and mining petabytes of unstructured information. It is increasingly being used across the enterprise for advanced data analytics and enabling new applications associated with data retention, regulatory compliance, e-discovery, and litigation issues.

Sharing a MapReduce cluster among multiple applications is a common practice in such environments. However, a key challenge in these shared environments is the ability to tailor and control resource allocations to different applications for achieving their performance goals and service level objectives (SLOs). Currently, there is no job scheduler for MapReduce environments that, given a job completion deadline, could allocate the appropriate amount of resources to the job so that it meets the required SLO. Benchmarking Hadoop, optimizing cluster parameters, efficient job scheduling, and workload management are new topics that create an exciting list of challenges and opportunities for modeling in MapReduce environments.

Conference Information