Apache-Spark

Description

This repository has some examples of Hadoop Spark, specially when I explored some courses in order to learn and explore about the benefits about Spark with Big Data.

Python

• friends-by-age.py, this code explores some lambda functions and how used these in to calculate the average of age users in the registers
• ratings-counter.py, this code explores the load of a file and the creation of the KDD
• popular-movies-dataframe.py, and spark-sql.py; these are examples of how use datasets (dateframe) with spark SQL

Datasets

• Datasets used in courses:

Courses

• Taming Big Data with Apache Spark and Python - Hands On!-Udemy