Skip to content

Big Data

Analytics And More
  • Home
  • Spark
  • Design Patterns
  • streaming
  • Map Reduce
  • Hive
  • Hdfs & Yarn
  • Pig
  • Oozie
  • Hbase

Category: Spark

spark textFileStream example to process json data

October, 2018 adarsh 2d Comments

Problem To Solve : Calculate the trading volume of the stocks every 10 minutes and decide which stock to purchase.…

Continue Reading →

Posted in: Data Analytics, Spark Filed under: Spark Rdd

spark textFileStream with sliding window and reduceByKeyAndWindow example

adarsh 1 Comment

Problem To Solve : Calculate the maximum profit (average closing price – average opening price) in a 5-minute sliding window…

Continue Reading →

Posted in: Data Analytics, Spark Filed under: Spark Rdd

spark file streaming with sliding window to calculate the simple moving average using reduceByKeyAndWindow

October, 2018 adarsh 1 Comment

Problem To Solve : Calculate the simple moving average closing price of stocks in a 5-minute sliding window for the…

Continue Reading →

Posted in: Data Analytics, Spark Filed under: Spark Rdd

spark example for jaccard similarity for lsh algorithm

adarsh

The Jaccard similarity index or the jaccard similarity coefficient compares two datasets to see which data is shared and which…

Continue Reading →

Posted in: Data Analytics, Spark Filed under: Spark Rdd

spark treeAggregate example and treeReduce example

May, 2018 adarsh 1 Comment

treeAggregate is a specialized implementation of aggregate that iteratively applies the combine function to a subset of partitions. This is…

Continue Reading →

Posted in: Data Analytics, Spark Filed under: Spark Rdd

spark aggregatebykey example in java

adarsh

Both foldByKey() and reduceByKey() require that the return type of our result be the same type as that of the…

Continue Reading →

Posted in: Spark Filed under: Spark Rdd

spark partition level functions by examples

May, 2018 adarsh

Spark has support for partition level functions which operate on per partition data. Working with data on a per partition…

Continue Reading →

Posted in: Spark Filed under: Spark Rdd

Post navigation

Page 6 of 12
← Previous 1 … 5 6 7 … 12 Next →

Recent Posts

  • Optimization for Using AWS Lambda to Send Messages to Amazon MSK
  • Rebalancing a Kafka Cluster in AWS MSK using CLI Commands
  • Using StsAssumeRoleCredentialsProvider with Glue Schema Registry Integration in Kafka Producer
  • Home
  • Contact Me
  • About Me
Copyright © 2017 Time Pass Techies