Loans - Predicting loan default risk of Czech bank customers¶

This notebook demonstrates the application of our relational learning algorithm to predict if a customer of a bank will default on his loan. We train the predictor on customer metadata, transaction history, as well as other successful and unsuccessful loans.

Summary:

• Prediction type: Binary classification
• Domain: Finance
• Prediction target: Loan default
• Source data: 8 tables, 78.8 MB
• Population size: 682

Background¶

This notebook features a textbook example of predictive analytics applied to the financial sector. A loan is the lending of money to companies or individuals. Banks grant loans in exchange for the promise of repayment. Loan default is defined as the failure to meet this legal obligation, for example, when a home buyer fails to make a mortgage payment. A bank needs to estimate the risk it carries when granting loans to potentially non-performing customers.

The analysis is based on the financial dataset from the the CTU Prague Relational Learning Repository (Motl and Schulte, 2015) (Now residing at relational-data.org.) .

Analysis¶

Let's get started with the analysis and set-up your session:

1. Loading data¶

1.1 Download from source¶

Downloading the raw data from the CTU Prague Relational Learning Repository into a prediction ready format takes time. To get to the getML model building as fast as possible, we prepared the data for you and excluded the code from this notebook. It will be made available in a future version.

1.2 Prepare data for getML¶

The getml.datasets.load_loans method took care of the entire data lifting:

• Downloads csv's from our servers in python
• Converts csv's to getML DataFrames
• Sets roles to columns inside getML DataFrames

The only thing left is to set units to columns that the relational learning algorithm is allowed to compare to each other.

Data visualization

To simplify the notebook, original data model (image below) is condensed into 4 tables, by resolving the trivial one-to-one and many-to-one joins:

• A population table population_{train, test}, consiting of loan and account tables
• Three peripheral tables: order, trans, and meta.
• Whereas meta is made up of card, client, disp and district

Population table

• Information on the loan itself (duration, amount, date, ...)
• Geo-information about the branch where the loans was granted (A**)
• Column status contains binary target. Levels [A, C] := loan paid back and [B, D] := loan default; we recoded status to our binary target: default

Peripheral tables

• meta
  • Meta info about the client (card_type, gender, ...)
  • Geo-information about the client
• order
  • Permanent orders related to a loan (amount, balance, ...)
• trans
  • Transactions related to a given loan (amount, ...)

While the contents of meta and order are omitted for brevity, here are contents of trans:

1.3 Define relational model¶

To start with relational learning, we need to specify an abstract data model. Here, we use the high-level star schema API that allows us to define the abstract data model and construct a container with the concrete data at one-go. While a simple StarSchema indeed works in many cases, it is not sufficient for more complex data models like schoflake schemas, where you would have to define the data model and construct the container in separate steps, by utilzing getML's full-fledged data model and container APIs respectively.

2. Predictive modeling¶

We loaded the data, defined the roles, units and the abstract data model. Next, we create a getML pipeline for relational learning.

2.1 getML Pipeline¶

Set-up of feature learners, selectors & predictor

Build the pipeline

2.2 Model training¶

2.3 Model evaluation¶

2.4 Studying features¶

Visualizing the learned features

The feature with the highest importance is:

Feature correlations

We want to analyze how the features are correlated with the target variable.

Feature importances

Feature importances are calculated by analyzing the improvement in predictive accuracy on each node of the trees in the XGBoost predictor. They are then normalized, so that all importances add up to 100%.

Column importances

Because getML uses relational learning, we can apply the principles we used to calculate the feature importances to individual columns as well.

As we can see, a lot of the predictive power stems from the account balance. This is unsurprising: People with less money on their bank accounts are more likely to default on their loans.

The most important feature looks as follows:

2.5 Productionization¶

It is possible to productionize the pipeline by transpiling the features into production-ready SQL code. Please also refer to getML's sqlite3 and spark modules.

3. Conclusion¶

By applying getML to the PKDD'99 Financial dataset, we were able to show the power and relevance of Relational Learning on a real-world data set. Within a training time below 1 minute, we outperformed almost all approaches based on manually generated features. This makes getML the prime choice when dealing with complex relational data schemes. This result holds independent of the problem domain since no expertise in the financial sector was used in this analysis.

The present analysis could be improved in two directions. By performing an extensive hyperparameter optimization, the out of sample AUC could be further improved. On the other hand, the hyperparameters could be tuned to produce less complex features that result in worse performance (in terms of AUC) but are better interpretable by humans.

References¶

Schulte, Oliver, et al. "A hierarchy of independence assumptions for multi-relational Bayes net classifiers." 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM). IEEE, 2013.