Skip to content

Big Data

Analytics And More
  • Home
  • Spark
  • Design Patterns
  • streaming
  • Map Reduce
  • Hive
  • Hdfs & Yarn
  • Pig
  • Oozie
  • Hbase

Category: Map Reduce

mapreduce custom writable example and writablecomparable example

March, 2018 adarsh

In the below example lets see how to create a custom Writable that can be used as a key in…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern

secondary sorting in mapreduce with custom writable as key

November, 2017 adarsh

We can implement secondary sorting in mapreduce using the below steps 1. Make the key a composite of the natural…

Continue Reading →

Posted in: Data Analytics, Map Reduce Filed under: map reduce, map reduce design pattern

oozie workflow example for map reduce action with end to end configuration

August, 2017 adarsh 1 Comment

The following is an ordered sequence of XML elements for map-reduce job and you must specify them in order when…

Continue Reading →

Posted in: Data Analytics, Map Reduce, Oozie Filed under: map reduce, oozie workflow

input formats and output formats in hadoop and mapreduce

July, 2017 adarsh

There are many input and output formats supported in hadoop out of the box and we will explore the same…

Continue Reading →

Posted in: Data Analytics, hadoop input/output, Hdfs, Map Reduce Filed under: hadoop input output, hdfs, map reduce

default mappper, reducer, partitioner, multithreadedmapper and split size configuration in hadoop and mapreduce

adarsh

What will be the mapper,reducer and the partitioner that will be used in mapreduce program if we dont specify any…

Continue Reading →

Posted in: hadoop input/output, Hdfs, Map Reduce Filed under: hadoop input output, hdfs, map reduce

hadoop mapreduce reading the entire file content without splitting the file for example reading an xml file

adarsh 2d Comments

Some applications don’t want files to be split, as this allows a single mapper to process each input file in…

Continue Reading →

Posted in: Hdfs, Map Reduce Filed under: hdfs, hdfs filesystem, map reduce

handling failures in hadoop,mapreduce and yarn

July, 2017 adarsh 1 Comment

In the real world, user code is buggy, processes crash, and machines fail. One of the major benefits of using…

Continue Reading →

Posted in: Data Analytics, Hdfs, Map Reduce, yarn Filed under: hdfs, map reduce, yarn

Post navigation

Page 1 of 6
1 2 … 6 Next →

Recent Posts

  • Optimization for Using AWS Lambda to Send Messages to Amazon MSK
  • Rebalancing a Kafka Cluster in AWS MSK using CLI Commands
  • Using StsAssumeRoleCredentialsProvider with Glue Schema Registry Integration in Kafka Producer
  • Home
  • Contact Me
  • About Me
Copyright © 2017 Time Pass Techies
 

Loading Comments...