📎 This article includes 3 downloadable practice files ↓

📘 Python Beginner Course · Lesson 9 of 12

Real exports are never clean: headers with spaces, “north” and “North ”, amounts like “Rs 1,23,000”, dates in two formats, duplicates and blank rows. Cleaning by hand in Excel takes an hour every month. A pandas script does it in a second, every time.

1. Column names

df = pd.read_csv("messy_sales.csv")
df.columns = df.columns.str.strip().str.lower().str.replace(" ", "_")
# " Date", "Region ", "Order ID"  →  "date", "region", "order_id"

2. Text: spaces and case

df["region"] = df["region"].str.strip().str.title()     # " north " → "North"
df["customer"] = df["customer"].str.strip()

The .str accessor applies text methods to every value: strip, upper, lower, title, replace, contains.

3. Numbers stored as text

df["amount"] = pd.to_numeric(
    df["amount"].astype(str).str.replace("Rs", "").str.replace(",", "").str.strip(),
    errors="coerce")

errors="coerce" turns anything that still isn’t a number into NaN (missing) instead of crashing, so you can find and handle it.

4. Mixed date formats

df["date"] = pd.to_datetime(df["date"], format="mixed", dayfirst=True, errors="coerce")

dayfirst=True reads 04/10/2026 as 4 October, the Indian way. Always check a few converted dates.

5. Duplicates and blanks

df = df.drop_duplicates()                    # exact duplicate rows
df = df.drop_duplicates(subset=["order_id"]) # same order id
df = df.dropna(subset=["amount", "date"])    # rows missing key values
df["region"] = df["region"].fillna("Unknown")

6. Check and save

print(df["region"].value_counts())
print(df.isna().sum())          # missing values per column
df.to_csv("clean_sales.csv", index=False)

In the practice file, 32 rows become 30 clean ones (one duplicate and one blank row removed).

Where beginners go wrong

Practice

Download the script (and data file, if listed) below into one folder. Open a terminal in that folder and run python lesson-NN.py. Compare with the expected output, then change something and run it again: that’s how Python is learned.