In this article I will illustrate how to submit a spark job programmatically using SparkLauncher. Let us take a use…
Lets say we want to send a custom object as the kafka value type and we need to push this…
Stateful transformations are operations on DStreams that track data across time that is, some data from previous batches is used…
Stateless transformations like map(), flatMap(), filter(), repartition(), reduceByKey(), groupByKey() are simple RDD transformations being applied on every batch. Keep in…
Spark Streaming provides an abstraction called DStreams, or discretized streams which is build on top of RDD. A DStream is…