Skip to content

Big Data

Analytics And More
  • Home
  • Spark
  • Design Patterns
  • streaming
  • Map Reduce
  • Hive
  • Hdfs & Yarn
  • Pig
  • Oozie
  • Hbase

Category: Map Reduce

mapreduce pattern for writing data into external source to a system outside of hadoop and hdfs like mysql database

June, 2017 adarsh

External source output pattern writes data to a system outside of Hadoop and HDFS.With this pattern, we are able to…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern

mapreduce partition pruning concept

adarsh

Partition pruning configures the way the framework picks input splits and drops files from being loaded into MapReduce based on…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern

hadoop mapreduce example with custom inputformat,inputsplit,recordreader,outputformat and recordwriter for generating test data

June, 2017 adarsh

Generating data isn’t common. Typically you’ll generate a bunch of the data at once then use it over and over…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern

mapreduce replicate,reduce side join,average pattern with real world example

adarsh 7d Comments

Problem to solve : We wish to know how have the genres ranked by Average Rating, for each profession and…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce real world example

mapreduce reduce side join,average and top n records pattern with real world example

June, 2017 adarsh

Problem to solve : Top twenty rated movies (Condition: The movie should be rated/viewed by at least 40 users) The…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce real world example

mapreduce reduce side join and top n records pattern with real world example

adarsh 17d Comments

The problem mentioned below revolves around movies dataset. The dataset contains 2 files which are follows, File Name    …

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce real world example

job merging optimization to process two unrelated jobs that are loading the same data to share the mapreduce pipeline

June, 2017 adarsh

job merging is an optimization aimed to reduce the amount of I/O through the MapReduce pipeline. Job merging is a…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern, mapreduce metapatterns

Post navigation

Page 3 of 6
← Previous 1 2 3 4 … 6 Next →

Recent Posts

  • Optimization for Using AWS Lambda to Send Messages to Amazon MSK
  • Rebalancing a Kafka Cluster in AWS MSK using CLI Commands
  • Using StsAssumeRoleCredentialsProvider with Glue Schema Registry Integration in Kafka Producer
  • Home
  • Contact Me
  • About Me
Copyright © 2017 Time Pass Techies