Machine Learning Lesson 8: Project, Predict Late Invoice Payments

📎 This article includes 4 downloadable practice files ↓

A collections team cannot call every customer. This project trains on 1,500 past invoices (amount, credit days, how often the customer was late before, customer size, month) and ranks new invoices by risk so the team calls the right ones first.

Steps

1. Load the history and pick the features.
2. Build a pipeline: encode customer size, then a random forest.
3. Test on a held-back quarter (AUC, precision, recall).
4. Retrain on all history and save the model with joblib.
5. Score the 25 new invoices, add an action, and write invoice_risk.xlsx.

# Lesson 8: project - predict late invoice payments and score new invoices into Excel
import pandas as pd, joblib
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import recall_score, precision_score, roc_auc_score

hist = pd.read_csv("invoices.csv")
feats = ["amount", "credit_days", "past_late_count", "customer_size", "invoice_month"]
X, y = hist[feats], hist["paid_late"]
print(f"{len(hist)} past invoices, {y.mean():.1%} paid late")

prep = ColumnTransformer([("cat", OneHotEncoder(handle_unknown="ignore"), ["customer_size"])], remainder="passthrough")
pipe = Pipeline([("prep", prep), ("model", RandomForestClassifier(n_estimators=300, max_depth=6, min_samples_leaf=5, random_state=0))])
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=3, stratify=y)
pipe.fit(X_train, y_train)
prob = pipe.predict_proba(X_test)[:, 1]
print(f"AUC {roc_auc_score(y_test, prob):.3f}, precision {precision_score(y_test, prob >= .5):.1%}, recall {recall_score(y_test, prob >= .5):.1%} at 50%")

pipe.fit(X, y)                                        # final model on all history
joblib.dump(pipe, "late_payment_model.joblib")

new = pd.read_csv("new_invoices.csv")
new["late_risk"] = pipe.predict_proba(new[feats])[:, 1].round(2)
new["action"] = pd.cut(new["late_risk"], [-0.01, 0.3, 0.6, 1], labels=["Normal", "Reminder before due", "Call now"])
new = new.sort_values("late_risk", ascending=False)
new.to_excel("invoice_risk.xlsx", index=False)
print("\nTop 5 risky invoices:")
print(new.head(5)[["invoice_no", "amount", "past_late_count", "late_risk", "action"]].to_string(index=False))
print("\nActions:", new["action"].value_counts().to_dict())
print("Saved invoice_risk.xlsx and late_payment_model.joblib")

1500 past invoices, 36.3% paid late
AUC 0.753, precision 69.4%, recall 50.0% at 50%

Top 5 risky invoices:
invoice_no  amount  past_late_count  late_risk              action
 INV-61531  424019                5       0.66            Call now
 INV-15116  461905                4       0.61            Call now
 INV-87378  236952                5       0.58 Reminder before due
 INV-18548  109063                5       0.56 Reminder before due
 INV-43525  450050                5       0.55 Reminder before due

Actions: {'Reminder before due': 12, 'Normal': 11, 'Call now': 2}
Saved invoice_risk.xlsx and late_payment_model.joblib

AUC 0.75 means that if you pick one late and one on-time invoice at random, the model ranks the late one higher about three times out of four: useful for ordering a call list, not a crystal ball. We opened invoice_risk.xlsx in Excel: 25 invoices sorted by risk, with the action in the last column.

You’ve finished the course

You can prepare data, train regression, classification and clustering models, evaluate them honestly, avoid overfitting and leakage, and turn a model into an Excel output people can act on. Next: deep learning and large language models.

Practice

Download the script and data file below into one folder, open a terminal there and run python lesson-NN.py. Your numbers should match the expected output exactly (the data and random seeds are fixed). Then change one thing (a column, a setting) and see what moves.