Skip to content

Big Data

Analytics And More
  • Home
  • Spark
  • Design Patterns
  • streaming
  • Map Reduce
  • Hive
  • Hdfs & Yarn
  • Pig
  • Oozie
  • Hbase

Category: Map Reduce

Chain folding optimization in mapreduce using chainmapper and chainreducer approach

June, 2017 adarsh

Chain folding is an optimization that is applied to MapReduce job chains.Take a look at the map phases in the…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce metapatterns

job chaining in mapreduce with jobcontrol ,controlledJob and driver

June, 2017 adarsh

Job chaining is extremely important to understand and have an operational plan for in your environment. Many people find that…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce metapatterns

mapreduce example to do a composite joins with many very large formatted inputs on mapside

adarsh

Composite joins are particularly useful if you want to join very large data sets together. However, the data sets must…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce join patterns

mapreduce example using replicated join between one large and many small data sets that can be performed on the map side

adarsh

A replicated join is an extremely useful, but has a strict size limit on all but one of the data…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce join patterns

mapreduce example to join large multiple data sets using reduce side join pattern

June, 2017 adarsh 2d Comments

A reduce side join is arguably one of the easiest implementations of a join in MapReduce, and therefore is a…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce join patterns

mapreduce example to shuffle and anonymize data using a random key

June, 2017 adarsh

Shuffling pattern can be used when we want to randomize the data set for repeatable random sampling For example, the…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce data organization patterns

mapreduce example to sort the data using the total order partitioner and input sampler utility

adarsh

Sorting is easy in sequential programming. Sorting in MapReduce, or more generally in parallel, is not easy. This is because…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce data organization patterns

Post navigation

Page 4 of 6
← Previous 1 … 3 4 5 6 Next →

Recent Posts

  • Optimization for Using AWS Lambda to Send Messages to Amazon MSK
  • Rebalancing a Kafka Cluster in AWS MSK using CLI Commands
  • Using StsAssumeRoleCredentialsProvider with Glue Schema Registry Integration in Kafka Producer
  • Home
  • Contact Me
  • About Me
Copyright © 2017 Time Pass Techies