In this article I will illustrate how to copy raw files from S3 using spark. Spark out of the box…
There are many input and output formats supported in hadoop out of the box and we will explore the same…
What will be the mapper,reducer and the partitioner that will be used in mapreduce program if we dont specify any…
Sequence files, map files, and Avro datafiles are all row-oriented file formats, which means that the values for each row…
Serialization is the process of turning structured objects into a byte stream for transmission over a network or for writing…
File compression brings two major benefits: it reduces the space needed to store files, and it speeds up data transfer…
HDFS transparently checksums all data written to it and by default verifies checksums when reading data. Datanodes are responsible for…