Run an LLM on Your Own Laptop with Ollama (Private, Free, Offline)

📎 This article includes 1 downloadable practice file ↓

You do not need a cloud account to use a language model. Ollama downloads open models and runs them on your own computer: free, offline, and your data never leaves the machine — ideal for confidential spreadsheets and documents.

Install and run

Download the installer from ollama.com (Windows, macOS or Linux), then in a terminal:

ollama run llama3.2          # downloads ~2 GB the first time, then opens a chat
ollama list                  # models you have
ollama pull qwen2.5:7b       # get another model
ollama rm qwen2.5:7b         # free the disk space

Type a question at the >>> prompt; /bye exits.

What hardware do you need?

Models are “quantized” — their numbers stored in about 4 bits instead of 16 — which shrinks them roughly four times with a small quality loss.

Call it from Python

Ollama runs a local web API on port 11434:

import requests

def ask(prompt, model="llama3.2"):
    r = requests.post("http://localhost:11434/api/generate",
                      json={"model": model, "prompt": prompt, "stream": False},
                      timeout=300)
    return r.json()["response"]

print(ask("Give an Excel formula to extract the domain from an email in A2."))

Process a spreadsheet

import pandas as pd
df = pd.read_excel("feedback.xlsx")               # column: Comment
df["Sentiment"] = [ask(f"Reply with one word, Positive, Negative or Neutral:\n{c}").strip()
                   for c in df["Comment"]]
df.to_excel("feedback_tagged.xlsx", index=False)

Hundreds of rows, no API bill, no data leaving your PC. Check a sample by hand — small models make more mistakes than big cloud ones.

Make your own assistant with a Modelfile

FROM llama3.2
PARAMETER temperature 0.2
SYSTEM You are an Excel expert. Reply with the formula first, then one line explaining it. Assume Excel 365.

ollama create excel-helper -f Modelfile
ollama run excel-helper

Which model to pick?

• Small and fast: llama3.2 (3B), qwen2.5 (3B) — tagging, short summaries.
• Better reasoning: 7–8B models such as llama3.1:8b or qwen2.5:7b.
• Code: coder variants such as qwen2.5-coder.

Next: give the local model your own documents to answer from — build a RAG chatbot.