In this article I will illustrate how to do schema discovery for validation of column name before firing a select…
In this article I will illustrate how to merge two dataframes with different schema. Spark supports below api for the…
In this article I will illustrate how to convert a nested json to csv in apache spark. Spark does not…
In this article i will demonstrate how to read and write avro data in spark from amazon s3. We will…
Avro is a language-neutral data serialization system and its schemas are usually written in JSON, and data is usually encoded…
In this article i will demonstrate how to add a column into a dataframe with a constant or static value…
In this article i will demonstrate how to build a Hive UDTF and execute it in Apache Spark. In hive…