Lending Club Dataset and Credit Score Assessment in Databricks

Overview

This project utilizes the Lending Club dataset to perform credit score assessment. The dataset includes detailed information about loans issued by Lending Club, such as loan amount, interest rate, borrower characteristics, and loan status.

Dataset

• Source: Lending Club
• Content: Loan information, borrower details, repayment status
• Format: Parquet

Objective

The primary goal is to assess the creditworthiness of borrowers using machine learning techniques in Databricks. This involves:

• Data Preprocessing: Cleaning and preparing the data for analysis.
• Exploratory Data Analysis (EDA): Understanding the data distribution and identifying key features.
• Feature Engineering: Creating new features to improve model performance.
• Modeling: Building and evaluating datasets to calculate credit scores.

Tools and Technologies

• Platform: Databricks
• Languages: Python, SQL
• Libraries: PySpark

Steps

1. Data Ingestion: Load the Lending Club dataset into Databricks.
2. Data Cleaning: Handle missing values, outliers, and data inconsistencies.
3. Feature Engineering: Create new features and transform existing ones to enhance dataset performance.
4. Credit score: Calculating credit score

Outcome

The project aims to develop a reliable credit scoring data of loan repayment, aiding in making informed lending decisions.