Machine Learning Lesson 6: Clustering Customers with K-Means

📎 This article includes 3 downloadable practice files ↓

So far every row had an answer to learn from. Clustering has no answers: you ask the computer to find natural groups. Marketing teams use it to split customers into segments.

Scale first

Spend is in tens of thousands, visits are under 30. Without scaling, K-Means would group almost only by spend. StandardScaler puts both on the same footing.

How many groups?

Try several values of k and look at the inertia (how spread out the groups are). Pick the point where adding a group stops helping much: the “elbow”.

# Lesson 6: clustering - group shoppers without any labels
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans

df = pd.read_csv("shoppers.csv")
X = StandardScaler().fit_transform(df[["yearly_spend", "visits"]])   # put both on the same scale

print("k  inertia (lower = tighter groups)")
for k in range(2, 7):
    print(k, round(KMeans(n_clusters=k, n_init=10, random_state=0).fit(X).inertia_, 1))

km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(X)
df["group"] = km.labels_
summary = df.groupby("group").agg(shoppers=("shopper_id", "count"), avg_spend=("yearly_spend", "mean"), avg_visits=("visits", "mean")).round(0)
print("\n" + summary.sort_values("avg_spend").to_string())
df.to_csv("shoppers_grouped.csv", index=False)
print("\nSaved shoppers_grouped.csv")

k  inertia (lower = tighter groups)
2 225.6
3 67.3
4 44.7
5 36.7
6 29.3

       shoppers  avg_spend  avg_visits
group                                 
1           147     8320.0         6.0
0            88    30261.0        21.0
2            65    70935.0        12.0

Saved shoppers_grouped.csv

Inertia drops sharply from 2 to 3 groups and only slowly after, so 3 it is. The groups describe themselves: occasional low spenders, frequent mid spenders, and a small group that visits less but spends the most. The group numbers (0, 1, 2) are arbitrary labels; name the segments yourself.

Practice

Download the script and data file below into one folder, open a terminal there and run python lesson-NN.py. Your numbers should match the expected output exactly (the data and random seeds are fixed). Then change one thing (a column, a setting) and see what moves.