Data Encoding and Parallelization Porting Techniques to Transform Binary Data Formats to Hadoop/MapReduce

The objective of this invention is to transform applications that process big data in arbitrary formats using the MapReduce parallel programming model and executed on the open-source Hadoop platform. Many legacy data-intensive applications do not scale out of a single server. Some applications can load-balance to multicore, but scalability is limited by intensive IO. Hadoop/MapReduce is a massively scalable programming paradigm modeled after Google’s implementation. There are many use cases to show Hadoop/MapReduce can scale linearly to hundreds or even thousands of servers for big data analysis. The problem is how to transparently transform a legacy serial data processing code to take full advantage of Hadoop/MapReduce parallel processing framework.