Mathew K Analytics

Lesson 9 · Python Beginner Course

SQLite with Python for Beginners: SQL Databases in 1 Hour (Beginner Python Course, Part 9)

In this lesson you'll be able to: Explain why a database beats a pile of text files once your data grows Create tables and insert rows safely with ?…

⬇ Download notebook

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Databases for Beginners: SQLite with Python#

In this lesson you'll be able to:

  • Explain why a database beats a pile of text files once your data grows
  • Create tables and insert rows safely with ? placeholders (and see why f-strings are dangerous)
  • Ask questions with SELECT, WHERE, ORDER BY, LIMIT, COUNT, SUM, AVG and GROUP BY
  • Change and remove data with UPDATE and DELETE, and undo mistakes with rollback
  • Connect two tables with a JOIN and get dict-like rows back
  • Build a capstone project: a personal library tracker with books and loans

Warm-Up#

In lesson 3 we saved data in text files, and in lesson 4 we used JSON. Let's remind ourselves how that felt.

import json
from pathlib import Path

folder = Path("lesson_files")
folder.mkdir(exist_ok=True)

orders = [
    {"customer": "Amara", "drink": "Latte", "price": 3.40},
    {"customer": "Jonas", "drink": "Tea", "price": 2.10},
    {"customer": "Amara", "drink": "Mocha", "price": 3.80},
]
(folder / "orders.json").write_text(json.dumps(orders))
print("Saved", len(orders), "orders")
Saved 3 orders
loaded = json.loads((folder / "orders.json").read_text())
amara_total = 0
for order in loaded:
    if order["customer"] == "Amara":
        amara_total += order["price"]
print(f"Amara spent ${amara_total:.2f}")
Amara spent $7.20

1. Why Databases?#

A JSON file is a box of receipts. A database is a filing cabinet with a librarian who fetches exactly what you ask for.

Fine for 3 orders, but for 300,000 every question means loading the whole file and writing a new loop. SQLite is a full database stored in one file, and Python ships with it as the sqlite3 module. No install, no server.

import sqlite3

db_path = folder / "cafe.db"
if db_path.exists():
    db_path.unlink()      # start fresh every time we run the lesson
print("Database file exists yet?", db_path.exists())
Database file exists yet? False
conn = sqlite3.connect(db_path)
print("Connected:", type(conn).__name__)
print("Database file exists now?", db_path.exists())
Connected: Connection
Database file exists now? True
# the connection is the phone line, the cursor runs our SQL commands
cursor = conn.cursor()
cursor.execute("SELECT 2 + 3")
print(cursor.fetchone())
(5,)
cursor.execute("SELECT 'hello', UPPER('sqlite'), 7 * 6, ROUND(10 / 3.0, 2)")
row = cursor.fetchone()
print(row)
print(type(row), "with", len(row), "values")
('hello', 'SQLITE', 42, 3.33)
<class 'tuple'> with 4 values

The connection is the phone line to the database. A cursor is the person on the line who runs your commands and hands back results. We speak to it in SQL (Structured Query Language), a small language just for data.

Pro tip: SQL keywords are not case sensitive, but writing them in CAPITALS makes them stand out from your table and column names.

2. Creating a Table#

A table is like a spreadsheet tab: named columns, and one row per thing. Each column gets a type: INTEGER, REAL (decimals), TEXT.

cursor.execute("""
    CREATE TABLE drinks (
        id      INTEGER PRIMARY KEY,
        name    TEXT NOT NULL,
        size    TEXT,
        price   REAL,
        vegan   INTEGER
    )
""")
print("Table 'drinks' created")
Table 'drinks' created
cursor.execute("SELECT name FROM sqlite_master WHERE type = 'table'")
print("Tables in the database:", cursor.fetchall())
Tables in the database: [('drinks',)]
cursor.execute("PRAGMA table_info(drinks)")
for column in cursor.fetchall():
    print(f"  column {column[1]:<6} type {column[2]}")
  column id     type INTEGER
  column name   type TEXT
  column size   type TEXT
  column price  type REAL
  column vegan  type INTEGER

INTEGER PRIMARY KEY means SQLite numbers each row for you: 1, 2, 3... NOT NULL means the value is required. SQLite has no true/false type, so vegan stores 1 or 0.

Common mistake: creating a table that already exists#

cursor.execute("CREATE TABLE drinks (id INTEGER PRIMARY KEY, name TEXT)")
---------------------------------------------------------------------------
OperationalError                          Traceback (most recent call last)
Cell In[11], line 1
----> 1 cursor.execute("CREATE TABLE drinks (id INTEGER PRIMARY KEY, name TEXT)")

OperationalError: table drinks already exists
cursor.execute("CREATE TABLE IF NOT EXISTS drinks (id INTEGER PRIMARY KEY, name TEXT)")
print("No error: IF NOT EXISTS skips it because the table is already there")
No error: IF NOT EXISTS skips it because the table is already there

3. Inserting Rows#

INSERT puts a new row in the table. commit() is the save button: until you press it, nothing is written to the file.

cursor.execute("INSERT INTO drinks (name, size, price, vegan) VALUES ('Latte', 'regular', 3.40, 0)")
conn.commit()
print("New row id:", cursor.lastrowid)
New row id: 1
new_drink = ("Oat Flat White", "regular", 3.60, 1)
cursor.execute("INSERT INTO drinks (name, size, price, vegan) VALUES (?, ?, ?, ?)", new_drink)
conn.commit()
print("New row id:", cursor.lastrowid)
New row id: 2

Each ? is a placeholder. You hand the values over separately, as a tuple, and sqlite3 slots them in safely. To add many rows at once, use executemany with a list of tuples.

menu = [
    ("Latte", "large", 3.90, 0),
    ("Tea", "regular", 2.10, 1),
    ("Mocha", "regular", 3.80, 0),
    ("Hot Chocolate", "large", 3.50, 0),
    ("Espresso", "small", 2.20, 1),
    ("Iced Latte", "large", 4.10, 0),
    ("Chai Latte", "regular", 3.30, 0),
]
cursor.executemany("INSERT INTO drinks (name, size, price, vegan) VALUES (?, ?, ?, ?)", menu)
conn.commit()
print("Rows added:", cursor.rowcount)
Rows added: 7
cursor.execute("SELECT * FROM drinks")
for row in cursor.fetchall():
    print(row)
(1, 'Latte', 'regular', 3.4, 0)
(2, 'Oat Flat White', 'regular', 3.6, 1)
(3, 'Latte', 'large', 3.9, 0)
(4, 'Tea', 'regular', 2.1, 1)
(5, 'Mocha', 'regular', 3.8, 0)
(6, 'Hot Chocolate', 'large', 3.5, 0)
(7, 'Espresso', 'small', 2.2, 1)
(8, 'Iced Latte', 'large', 4.1, 0)
(9, 'Chai Latte', 'regular', 3.3, 0)

Common mistake: the wrong number of values#

cursor.execute("INSERT INTO drinks (name, size, price, vegan) VALUES (?, ?, ?, ?)", ("Matcha", 3.70))
---------------------------------------------------------------------------
ProgrammingError                          Traceback (most recent call last)
Cell In[17], line 1
----> 1 cursor.execute("INSERT INTO drinks (name, size, price, vegan) VALUES (?, ?, ?, ?)", ("Matcha", 3.70))

ProgrammingError: Incorrect number of bindings supplied. The current statement uses 4, and there are 2 supplied.
cursor.execute("INSERT INTO drinks (name, size, price, vegan) VALUES (?, ?, ?, ?)",
               ("Matcha", "regular", 3.70, 1))
conn.commit()
print("Four placeholders, four values. Matcha is row", cursor.lastrowid)
Four placeholders, four values. Matcha is row 10

Quick check#

We have inserted 10 drinks. If you insert one more, what will cursor.lastrowid be?

cursor.execute("INSERT INTO drinks (name, size, price, vegan) VALUES (?, ?, ?, ?)",
               ("Babyccino", "small", 0.50, 0))
conn.commit()
print("Answer:", cursor.lastrowid)
Answer: 11

4. Why Not f-strings? SQL Injection#

Building SQL with f-strings is like letting a customer write their own order straight onto the kitchen ticket. Most write "latte". One writes "latte, and empty the till".

cursor.execute("CREATE TABLE customers (id INTEGER PRIMARY KEY, name TEXT, city TEXT, card_pin TEXT)")
cursor.executemany("INSERT INTO customers (name, city, card_pin) VALUES (?, ?, ?)", [
    ("Amara", "Leeds", "4821"),
    ("Jonas", "Bristol", "1937"),
    ("Mei", "Leeds", "7702"),
    ("Priya", "York", "5560"),
    ("Leo", "Bristol", "3018"),
    ("Hana", "Leeds", "8844"),
    ("Zoe", "York", "2205"),
])
conn.commit()
print("Customers stored:", cursor.rowcount)
Customers stored: 7
def find_customer_unsafe(name):
    sql = f"SELECT name, city FROM customers WHERE name = '{name}'"
    print("  SQL sent:", sql)
    return cursor.execute(sql).fetchall()

print(find_customer_unsafe("Jonas"))
  SQL sent: SELECT name, city FROM customers WHERE name = 'Jonas'
[('Jonas', 'Bristol')]
sneaky = "nobody' OR '1'='1"
print(find_customer_unsafe(sneaky))
  SQL sent: SELECT name, city FROM customers WHERE name = 'nobody' OR '1'='1'
[('Amara', 'Leeds'), ('Jonas', 'Bristol'), ('Mei', 'Leeds'), ('Priya', 'York'), ('Leo', 'Bristol'), ('Hana', 'Leeds'), ('Zoe', 'York')]

The "name" rewrote our query, and '1'='1' is always true, so every customer came back. Attackers use the same trick to read card PINs or delete whole tables.

Common mistake: a name with an apostrophe#

find_customer_unsafe("O'Brien")
  SQL sent: SELECT name, city FROM customers WHERE name = 'O'Brien'
---------------------------------------------------------------------------
OperationalError                          Traceback (most recent call last)
Cell In[23], line 1
----> 1 find_customer_unsafe("O'Brien")

Cell In[21], line 4, in find_customer_unsafe(name)
      1 def find_customer_unsafe(name):
      2     sql = f"SELECT name, city FROM customers WHERE name = '{name}'"
      3     print("  SQL sent:", sql)
----> 4     return cursor.execute(sql).fetchall()

OperationalError: near "Brien": syntax error
def find_customer(name):
    sql = "SELECT name, city FROM customers WHERE name = ?"
    return cursor.execute(sql, (name,)).fetchall()

print("Jonas   ->", find_customer("Jonas"))
print("O'Brien ->", find_customer("O'Brien"))
print("sneaky  ->", find_customer(sneaky))
Jonas   -> [('Jonas', 'Bristol')]
O'Brien -> []
sneaky  -> []

With ? the value is always treated as data, never as SQL. The sneaky text is just a strange name that matches nobody.

Pro tip: never put user input into SQL with f-strings, + or .format(). Always use ? placeholders.

Note the comma in (name,): a single value still has to be a tuple.

search = input("Search for a customer: ").strip().title()
results = find_customer(search)
if results:
    print(f"Found: {results[0][0]} from {results[0][1]}")
else:
    print("No customer by that name")
Search for a customer: Amara
Found: Amara from Leeds

5. Asking Questions: WHERE, ORDER BY, LIMIT#

SELECT says which columns, FROM says which table, WHERE filters rows, ORDER BY sorts them, LIMIT keeps the top few.

cursor.execute("SELECT name, price FROM drinks WHERE price < 3")
for name, price in cursor.fetchall():
    print(f"{name:<12} ${price:.2f}")
Tea          $2.10
Espresso     $2.20
Babyccino    $0.50
cursor.execute("SELECT name, size FROM drinks WHERE vegan = 1 AND size = 'regular'")
print("Vegan regulars:", cursor.fetchall())

cursor.execute("SELECT name FROM drinks WHERE size = 'small' OR price > 4")
print("Small or pricey:", cursor.fetchall())
Vegan regulars: [('Oat Flat White', 'regular'), ('Tea', 'regular'), ('Matcha', 'regular')]
Small or pricey: [('Espresso',), ('Iced Latte',), ('Babyccino',)]
cursor.execute("SELECT name, size FROM drinks WHERE name LIKE '%latte%'")
for name, size in cursor.fetchall():
    print(f"  {name} ({size})")
  Latte (regular)
  Latte (large)
  Iced Latte (large)
  Chai Latte (regular)

LIKE matches patterns: % means "anything here". In SQLite it ignores upper and lower case for plain letters.

cursor.execute("SELECT name, price FROM drinks ORDER BY price DESC LIMIT 3")
print("Three most expensive:")
for position, (name, price) in enumerate(cursor.fetchall(), start=1):
    print(f"  {position}. {name} ${price:.2f}")
Three most expensive:
  1. Iced Latte $4.10
  2. Latte $3.90
  3. Mocha $3.80
cursor.execute("SELECT name, size, price FROM drinks ORDER BY size, name")
for row in cursor.fetchall()[:6]:
    print(row)
('Hot Chocolate', 'large', 3.5)
('Iced Latte', 'large', 4.1)
('Latte', 'large', 3.9)
('Chai Latte', 'regular', 3.3)
('Latte', 'regular', 3.4)
('Matcha', 'regular', 3.7)
cursor.execute("SELECT name FROM drinks WHERE price BETWEEN 3 AND 3.6")
print("Mid-price:", [row[0] for row in cursor.fetchall()])

cursor.execute("SELECT name FROM drinks WHERE size IN ('small', 'large')")
print("Not regular:", [row[0] for row in cursor.fetchall()])
Mid-price: ['Latte', 'Oat Flat White', 'Hot Chocolate', 'Chai Latte']
Not regular: ['Latte', 'Hot Chocolate', 'Espresso', 'Iced Latte', 'Babyccino']
cursor.execute("SELECT name, price FROM drinks WHERE price > ?", (3.5,))
print("First match:", cursor.fetchone())
print("Next match: ", cursor.fetchone())
print("The rest:   ", cursor.fetchall())
First match: ('Oat Flat White', 3.6)
Next match:  ('Latte', 3.9)
The rest:    [('Mocha', 3.8), ('Iced Latte', 4.1), ('Matcha', 3.7)]

fetchone() gives one row (or None when there are no more). fetchall() gives a list of every remaining row. You can also loop straight over the cursor.

for row in cursor.execute("SELECT name FROM drinks WHERE vegan = 1 ORDER BY name"):
    print("Vegan:", row[0])
Vegan: Espresso
Vegan: Matcha
Vegan: Oat Flat White
Vegan: Tea

Your turn#

  1. Find every drink that costs less than $3.50 and is not large, cheapest first.
  2. Find the single cheapest vegan drink using ORDER BY and LIMIT 1.

6. Counting and Summarising: COUNT, SUM, AVG, GROUP BY#

Instead of fetching every receipt and adding them up yourself, ask the librarian for the total.

import random

cursor.execute("""
    CREATE TABLE orders (
        id          INTEGER PRIMARY KEY,
        customer_id INTEGER,
        drink       TEXT,
        qty         INTEGER,
        price       REAL,
        day         TEXT
    )
""")
print("orders table ready")
orders table ready
random.seed(9)
prices = {"Latte": 3.40, "Tea": 2.10, "Mocha": 3.80, "Espresso": 2.20, "Chai Latte": 3.30}
days = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]

fake_orders = []
for _ in range(40):
    drink = random.choice(list(prices))
    fake_orders.append((random.randint(1, 6), drink, random.randint(1, 3), prices[drink], random.choice(days)))

cursor.executemany("INSERT INTO orders (customer_id, drink, qty, price, day) VALUES (?, ?, ?, ?, ?)", fake_orders)
conn.commit()
print(len(fake_orders), "orders, the first three:", fake_orders[:3])
40 orders, the first three: [(5, 'Espresso', 2, 2.2, 'Wed'), (2, 'Tea', 3, 2.1, 'Mon'), (5, 'Mocha', 2, 3.8, 'Fri')]
count = cursor.execute("SELECT COUNT(*) FROM orders").fetchone()[0]
cups = cursor.execute("SELECT SUM(qty) FROM orders").fetchone()[0]
print(f"{count} orders, {cups} cups served")
40 orders, 76 cups served
cursor.execute("""
    SELECT ROUND(SUM(qty * price), 2), ROUND(AVG(qty * price), 2), MIN(price), MAX(price)
    FROM orders
""")
total, average, cheapest, dearest = cursor.fetchone()
print(f"Takings ${total}, average order ${average}")
print(f"Cheapest drink ${cheapest}, dearest ${dearest}")
Takings $217.3, average order $5.43
Cheapest drink $2.1, dearest $3.8

GROUP BY puts rows into piles (one pile per drink, say), then the summary runs on each pile separately.

cursor.execute("""
    SELECT drink, COUNT(*), SUM(qty)
    FROM orders
    GROUP BY drink
    ORDER BY SUM(qty) DESC
""")
print(f"{'Drink':<11}{'Orders':>7}{'Cups':>6}")
for drink, n_orders, n_cups in cursor.fetchall():
    print(f"{drink:<11}{n_orders:>7}{n_cups:>6}")
Drink       Orders  Cups
Latte           12    27
Espresso        11    19
Tea              9    14
Chai Latte       6    13
Mocha            2     3
cursor.execute("""
    SELECT day, ROUND(SUM(qty * price), 2) AS takings
    FROM orders
    GROUP BY day
    ORDER BY takings DESC
""")
for day, takings in cursor.fetchall():
    print(f"{day}  ${takings:>6.2f}  {'#' * int(takings // 5)}")
Wed  $ 51.00  ##########
Tue  $ 48.20  #########
Fri  $ 45.00  #########
Mon  $ 30.40  ######
Sun  $ 25.40  #####
Sat  $ 10.80  ##
Thu  $  6.50  #

AS takings gives the calculated column a nickname so you can sort by it. To filter piles (not rows), use HAVING after GROUP BY.

cursor.execute("""
    SELECT customer_id, COUNT(*) AS visits
    FROM orders
    GROUP BY customer_id
    HAVING visits >= 7
    ORDER BY visits DESC
""")
print("Regulars (7+ orders):", cursor.fetchall())
Regulars (7+ orders): [(5, 8), (2, 8), (6, 7)]

Quick check#

Which word filters rows before grouping: WHERE or HAVING? Type your answer.

answer = input("WHERE or HAVING? ").strip().upper()
if answer == "WHERE":
    print("Correct! WHERE filters rows first, HAVING filters the groups afterwards.")
else:
    print("Not quite: WHERE filters rows first, HAVING filters the groups afterwards.")
WHERE or HAVING? WHERE
Correct! WHERE filters rows first, HAVING filters the groups afterwards.
cursor.execute("""
    SELECT drink, SUM(qty) FROM orders
    WHERE day IN ('Sat', 'Sun')
    GROUP BY drink ORDER BY 2 DESC
""")
print("Weekend favourites:", cursor.fetchall())
Weekend favourites: [('Espresso', 7), ('Chai Latte', 4), ('Tea', 2), ('Latte', 1)]

7. Changing Data: UPDATE and DELETE#

UPDATE is a correction pen. DELETE is the shredder. Both need a WHERE, or they hit every row.

cursor.execute("SELECT name, price FROM drinks WHERE size = 'large'")
print("Before:", cursor.fetchall())

cursor.execute("UPDATE drinks SET price = price + 0.20 WHERE size = 'large'")
conn.commit()
print("Rows changed:", cursor.rowcount)
Before: [('Latte', 3.9), ('Hot Chocolate', 3.5), ('Iced Latte', 4.1)]
Rows changed: 3
cursor.execute("SELECT name, ROUND(price, 2) FROM drinks WHERE size = 'large'")
print("After: ", cursor.fetchall())
After:  [('Latte', 4.1), ('Hot Chocolate', 3.7), ('Iced Latte', 4.3)]
cursor.execute("UPDATE drinks SET vegan = ?, name = ? WHERE id = ?", (1, "Oat Latte", 1))
conn.commit()
print(cursor.execute("SELECT * FROM drinks WHERE id = 1").fetchone())
(1, 'Oat Latte', 'regular', 3.4, 1)
cursor.execute("DELETE FROM drinks WHERE name = ?", ("Babyccino",))
conn.commit()
print("Deleted:", cursor.rowcount)
print("Drinks left:", cursor.execute("SELECT COUNT(*) FROM drinks").fetchone()[0])
Deleted: 1
Drinks left: 10

Pro tip: before an UPDATE or DELETE, run a SELECT with the same WHERE to see exactly which rows you're about to touch.

Common mistake: a typo in a column name#

cursor.execute("UPDATE drinks SET prise = 2.50 WHERE name = 'Tea'")
---------------------------------------------------------------------------
OperationalError                          Traceback (most recent call last)
Cell In[47], line 1
----> 1 cursor.execute("UPDATE drinks SET prise = 2.50 WHERE name = 'Tea'")

OperationalError: no such column: prise
cursor.execute("UPDATE drinks SET price = 2.50 WHERE name = 'Tea'")
conn.commit()
print("Tea now costs", cursor.execute("SELECT price FROM drinks WHERE name = 'Tea'").fetchone()[0])
Tea now costs 2.5

8. Commit and Rollback#

Changes wait in a basket until you commit, like a shopping trolley before the checkout. rollback() puts everything back on the shelves.

cursor.execute("DELETE FROM drinks")
print("Drinks visible right now:", cursor.execute("SELECT COUNT(*) FROM drinks").fetchone()[0])

conn.rollback()
print("After rollback:", cursor.execute("SELECT COUNT(*) FROM drinks").fetchone()[0])
Drinks visible right now: 0
After rollback: 10

Phew. Rollback matters most when a job has several steps that must all happen, or none: like moving money between two wallets.

cursor.execute("CREATE TABLE wallets (owner TEXT PRIMARY KEY, balance REAL CHECK (balance >= 0))")
cursor.executemany("INSERT INTO wallets VALUES (?, ?)", [("Amara", 20.0), ("Jonas", 5.0)])
conn.commit()

def show_wallets():
    for owner, balance in cursor.execute("SELECT owner, balance FROM wallets"):
        print(f"  {owner:<6} ${balance:.2f}")

show_wallets()
  Amara  $20.00
  Jonas  $5.00
def transfer(sender, receiver, amount):
    try:
        cursor.execute("UPDATE wallets SET balance = balance + ? WHERE owner = ?", (amount, receiver))
        cursor.execute("UPDATE wallets SET balance = balance - ? WHERE owner = ?", (amount, sender))
        conn.commit()
        print(f"Sent ${amount:.2f} from {sender} to {receiver}")
    except sqlite3.IntegrityError as problem:
        conn.rollback()
        print(f"Transfer cancelled ({problem})")

transfer("Amara", "Jonas", 8)
show_wallets()
Sent $8.00 from Amara to Jonas
  Amara  $12.00
  Jonas  $13.00
transfer("Jonas", "Amara", 50)
show_wallets()
Transfer cancelled (CHECK constraint failed: balance >= 0)
  Amara  $12.00
  Jonas  $13.00

Jonas couldn't afford it, so the CHECK rule failed on step two. Rollback also undid step one, so Amara wasn't given free money. A connection can do the commit or rollback for you in a with block:

try:
    with conn:
        conn.execute("UPDATE wallets SET balance = balance + 100 WHERE owner = 'Amara'")
        conn.execute("UPDATE wallets SET balance = balance - 100 WHERE owner = 'Jonas'")
except sqlite3.IntegrityError:
    print("with conn: rolled back automatically")
show_wallets()
with conn: rolled back automatically
  Amara  $12.00
  Jonas  $13.00

Common mistake: leaving out a required value#

cursor.execute("INSERT INTO drinks (size, price, vegan) VALUES (?, ?, ?)", ("small", 2.90, 1))
---------------------------------------------------------------------------
IntegrityError                            Traceback (most recent call last)
Cell In[54], line 1
----> 1 cursor.execute("INSERT INTO drinks (size, price, vegan) VALUES (?, ?, ?)", ("small", 2.90, 1))

IntegrityError: NOT NULL constraint failed: drinks.name
cursor.execute("INSERT INTO drinks (name, size, price, vegan) VALUES (?, ?, ?, ?)",
               ("Cortado", "small", 2.90, 1))
conn.commit()
print("Cortado added as drink", cursor.lastrowid)
Cortado added as drink 11

9. Two Tables and a JOIN#

Our orders only store customer_id. A JOIN is a lookup: "for each order, go and fetch the customer with that id".

print(cursor.execute("SELECT id, customer_id, drink FROM orders LIMIT 3").fetchall())
print(cursor.execute("SELECT id, name FROM customers LIMIT 3").fetchall())
[(1, 5, 'Espresso'), (2, 2, 'Tea'), (3, 5, 'Mocha')]
[(1, 'Amara'), (2, 'Jonas'), (3, 'Mei')]

Why not just store the name in every order? If Amara changes her name, you'd fix 1 row in customers instead of hundreds in orders.

cursor.execute("""
    SELECT orders.id, customers.name, orders.drink, orders.qty
    FROM orders
    JOIN customers ON orders.customer_id = customers.id
    ORDER BY orders.id
    LIMIT 5
""")
for row in cursor.fetchall():
    print(row)
(1, 'Leo', 'Espresso', 2)
(2, 'Jonas', 'Tea', 3)
(3, 'Leo', 'Mocha', 2)
(4, 'Mei', 'Latte', 3)
(5, 'Hana', 'Latte', 2)
cursor.execute("""
    SELECT c.name, c.city, ROUND(SUM(o.qty * o.price), 2) AS spent
    FROM orders AS o
    JOIN customers AS c ON o.customer_id = c.id
    GROUP BY c.name
    ORDER BY spent DESC
""")
for name, city, spent in cursor.fetchall():
    print(f"{name:<6} {city:<8} ${spent:>6.2f}")
Leo    Bristol  $ 44.90
Jonas  Bristol  $ 42.80
Amara  Leeds    $ 39.00
Mei    Leeds    $ 36.90
Priya  York     $ 30.10
Hana   Leeds    $ 23.60

Short nicknames (o, c) save typing. Notice Zoe is missing: a plain JOIN only keeps rows that match on both sides. A LEFT JOIN keeps every customer, even with no orders.

cursor.execute("""
    SELECT c.name, COUNT(o.id) AS n_orders
    FROM customers AS c
    LEFT JOIN orders AS o ON o.customer_id = c.id
    GROUP BY c.name
    ORDER BY n_orders
""")
print(cursor.fetchall())
[('Zoe', 0), ('Priya', 5), ('Amara', 6), ('Mei', 6), ('Hana', 7), ('Jonas', 8), ('Leo', 8)]
cursor.execute("""
    SELECT c.city, COUNT(*) AS orders, SUM(o.qty) AS cups
    FROM orders AS o JOIN customers AS c ON o.customer_id = c.id
    GROUP BY c.city
""")
for city, n, cups in cursor.fetchall():
    print(f"{city}: {n} orders, {cups} cups")
Bristol: 16 orders, 32 cups
Leeds: 19 orders, 34 cups
York: 5 orders, 10 cups

Your turn#

  1. List every order made by customers from Leeds, showing name, drink and day.
  2. Find which customer bought the most cups of Latte.

Mini-Project: The Plant Watering Log#

A tiny database for houseplants: each plant has a watering interval, and a second table logs every watering. We'll use :memory:, a throwaway database that lives only in RAM, which is handy for experiments.

plants_db = sqlite3.connect(":memory:")
plants_db.execute("CREATE TABLE plants (id INTEGER PRIMARY KEY, name TEXT, every_days INTEGER)")
plants_db.execute("CREATE TABLE waterings (plant_id INTEGER, day INTEGER)")
plants_db.executemany("INSERT INTO plants (name, every_days) VALUES (?, ?)",
                      [("Basil", 2), ("Cactus", 14), ("Fern", 4), ("Monstera", 7)])
print("Plants:", plants_db.execute("SELECT name FROM plants").fetchall())
Plants: [('Basil',), ('Cactus',), ('Fern',), ('Monstera',)]
log = [(1, 1), (1, 3), (1, 5), (2, 1), (3, 2), (3, 6), (4, 3)]
plants_db.executemany("INSERT INTO waterings VALUES (?, ?)", log)
print("Waterings logged:", plants_db.execute("SELECT COUNT(*) FROM waterings").fetchone()[0])
Waterings logged: 7
def thirsty_plants(today):
    rows = plants_db.execute("""
        SELECT p.name, p.every_days, MAX(w.day) AS last
        FROM plants AS p JOIN waterings AS w ON w.plant_id = p.id
        GROUP BY p.id
    """).fetchall()
    for name, every, last in rows:
        due = last + every
        status = "WATER NOW" if due <= today else f"ok until day {due}"
        print(f"  {name:<9} last watered day {last:<2} {status}")

print("Day 7:")
thirsty_plants(7)
Day 7:
  Basil     last watered day 5  WATER NOW
  Cactus    last watered day 1  ok until day 15
  Fern      last watered day 6  ok until day 10
  Monstera  last watered day 3  ok until day 10
plants_db.execute("INSERT INTO waterings VALUES (?, ?)", (1, 7))
print("Watered the Basil. Day 8:")
thirsty_plants(8)
plants_db.close()
Watered the Basil. Day 8:
  Basil     last watered day 7  ok until day 9
  Cactus    last watered day 1  ok until day 15
  Fern      last watered day 6  ok until day 10
  Monstera  last watered day 3  ok until day 10

10. Dict-Like Rows with row_factory#

Tuples make you remember that column 2 is the price. Named rows let you just say row["price"].

conn.row_factory = sqlite3.Row
cursor = conn.cursor()

row = cursor.execute("SELECT * FROM drinks WHERE name = 'Mocha'").fetchone()
print(row["name"], "costs", row["price"])
print("Columns:", row.keys())
Mocha costs 3.8
Columns: ['id', 'name', 'size', 'price', 'vegan']
print("Still works by position:", row[1], row[3])
print("As a real dict:", dict(row))
Still works by position: Mocha 3.8
As a real dict: {'id': 5, 'name': 'Mocha', 'size': 'regular', 'price': 3.8, 'vegan': 0}
rows = cursor.execute("SELECT name, price FROM drinks WHERE vegan = 1").fetchall()
vegan_menu = [dict(r) for r in rows]
for item in vegan_menu:
    print(item)
print("Ready for JSON:", json.dumps(vegan_menu[:2]))
{'name': 'Oat Latte', 'price': 3.4}
{'name': 'Oat Flat White', 'price': 3.6}
{'name': 'Tea', 'price': 2.5}
{'name': 'Espresso', 'price': 2.2}
{'name': 'Matcha', 'price': 3.7}
{'name': 'Cortado', 'price': 2.9}
Ready for JSON: [{"name": "Oat Latte", "price": 3.4}, {"name": "Oat Flat White", "price": 3.6}]

Common mistake: a typo in the column key#

print(row["nmae"])
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[68], line 1
----> 1 print(row["nmae"])

IndexError: No item with that key
print(row["name"])
print("Safe check:", "nmae" in row.keys(), "name" in row.keys())
Mocha
Safe check: False True

11. Context Managers: Tidy Connections#

with is a polite guest: it cleans up after itself even when something goes wrong.

Careful: with conn: commits or rolls back, but it does not close the connection. For that, use contextlib.closing.

conn.close()
print("Old connection closed")
Old connection closed

Common mistake: using a connection after closing it#

conn.execute("SELECT COUNT(*) FROM drinks")
---------------------------------------------------------------------------
ProgrammingError                          Traceback (most recent call last)
Cell In[71], line 1
----> 1 conn.execute("SELECT COUNT(*) FROM drinks")

ProgrammingError: Cannot operate on a closed database.
from contextlib import closing

with closing(sqlite3.connect(db_path)) as conn:
    print("Drinks on the menu:", conn.execute("SELECT COUNT(*) FROM drinks").fetchone()[0])
print("Connection closed automatically")
Drinks on the menu: 11
Connection closed automatically
def run_query(sql, params=()):
    """Open the database, run one query, return a list of dicts, close it."""
    with closing(sqlite3.connect(db_path)) as conn:
        conn.row_factory = sqlite3.Row
        return [dict(row) for row in conn.execute(sql, params)]

print(run_query("SELECT name, price FROM drinks WHERE price > ? ORDER BY price", (3.8,)))
[{'name': 'Latte', 'price': 4.1}, {'name': 'Iced Latte', 'price': 4.3}]
def run_change(sql, params=()):
    """Run an INSERT, UPDATE or DELETE and commit it. Returns rows affected."""
    with closing(sqlite3.connect(db_path)) as conn:
        with conn:
            return conn.execute(sql, params).rowcount

print("Updated:", run_change("UPDATE drinks SET price = ? WHERE name = ?", (2.30, "Espresso")))
print(run_query("SELECT name, price FROM drinks WHERE name = ?", ("Espresso",)))
Updated: 1
[{'name': 'Espresso', 'price': 2.3}]

Pro tip: small helper functions like these keep SQL in one place and make the rest of your program read like English.

12. From SQL to pandas#

SQL fetches the right rows. pandas makes them look good and gives you more tools. They're a great team.

A sneak peek: lesson 11 covers pandas properly.

import pandas as pd

conn = sqlite3.connect(db_path)
menu_df = pd.read_sql_query("SELECT name, size, price FROM drinks ORDER BY price DESC", conn)
print(menu_df.head(6))
             name     size  price
0      Iced Latte    large    4.3
1           Latte    large    4.1
2           Mocha  regular    3.8
3   Hot Chocolate    large    3.7
4          Matcha  regular    3.7
5  Oat Flat White  regular    3.6
sales = pd.read_sql_query("""
    SELECT c.name AS customer, o.drink, o.qty, o.price, o.day
    FROM orders AS o JOIN customers AS c ON o.customer_id = c.id
    WHERE o.day = ?
""", conn, params=("Sat",))
print(sales)
  customer     drink  qty  price  day
0     Hana       Tea    1    2.1  Sat
1     Hana       Tea    1    2.1  Sat
2    Jonas  Espresso    3    2.2  Sat
sales["total"] = sales["qty"] * sales["price"]
print("Saturday takings: $", round(sales["total"].sum(), 2))
print(sales.groupby("drink")["qty"].sum())
Saturday takings: $ 10.8
drink
Espresso    3
Tea         2
Name: qty, dtype: int64
specials = pd.DataFrame({"name": ["Pumpkin Latte", "Iced Mocha"], "price": [4.20, 4.30]})
specials.to_sql("specials", conn, if_exists="replace", index=False)
print(conn.execute("SELECT * FROM specials").fetchall())
conn.close()
[('Pumpkin Latte', 4.2), ('Iced Mocha', 4.3)]

Capstone: The Personal Library Tracker#

Everything at once: two tables, placeholders, INSERT, UPDATE, a JOIN, transactions, sqlite3.Row and a menu. Dates are stored as YYYY-MM-DD text, and we use a fixed "today" so the report is repeatable.

from datetime import date, timedelta

library_path = folder / "library.db"
if library_path.exists():
    library_path.unlink()

lib = sqlite3.connect(library_path)
lib.row_factory = sqlite3.Row
TODAY = date(2026, 3, 16)
print("Library database ready, today is", TODAY)
Library database ready, today is 2026-03-16
with lib:
    lib.execute("""
        CREATE TABLE books (
            id     INTEGER PRIMARY KEY,
            title  TEXT NOT NULL UNIQUE,
            author TEXT,
            year   INTEGER
        )""")
    lib.execute("""
        CREATE TABLE loans (
            id        INTEGER PRIMARY KEY,
            book_id   INTEGER NOT NULL,
            friend    TEXT NOT NULL,
            lent_on   TEXT NOT NULL,
            due_on    TEXT NOT NULL,
            returned  INTEGER DEFAULT 0
        )""")
print([r["name"] for r in lib.execute("SELECT name FROM sqlite_master WHERE type = 'table'")])
['books', 'loans']
def add_book(title, author, year):
    try:
        with lib:
            lib.execute("INSERT INTO books (title, author, year) VALUES (?, ?, ?)", (title, author, year))
        print(f"  Added: {title}")
    except sqlite3.IntegrityError:
        print(f"  '{title}' is already on your shelf")

add_book("Dune", "Frank Herbert", 1965)
add_book("Dune", "Frank Herbert", 1965)
  Added: Dune
  'Dune' is already on your shelf
for title, author, year in [
    ("The Hobbit", "J. R. R. Tolkien", 1937),
    ("Matilda", "Roald Dahl", 1988),
    ("Small Island", "Andrea Levy", 2004),
    ("The Martian", "Andy Weir", 2011),
    ("Persuasion", "Jane Austen", 1817),
]:
    add_book(title, author, year)
  Added: The Hobbit
  Added: Matilda
  Added: Small Island
  Added: The Martian
  Added: Persuasion
def find_book(title):
    return lib.execute("SELECT * FROM books WHERE title = ?", (title,)).fetchone()

def is_on_loan(book_id):
    sql = "SELECT COUNT(*) FROM loans WHERE book_id = ? AND returned = 0"
    return lib.execute(sql, (book_id,)).fetchone()[0] > 0

book = find_book("Matilda")
print(dict(book), "| on loan?", is_on_loan(book["id"]))
{'id': 3, 'title': 'Matilda', 'author': 'Roald Dahl', 'year': 1988} | on loan? False
def lend_book(title, friend, lent_on=TODAY, days=14):
    book = find_book(title)
    if book is None:
        print(f"  No book called '{title}'")
        return
    if is_on_loan(book["id"]):
        print(f"  Sorry {friend}, '{title}' is already lent out")
        return
    due = lent_on + timedelta(days=days)
    with lib:
        lib.execute("INSERT INTO loans (book_id, friend, lent_on, due_on) VALUES (?, ?, ?, ?)",
                    (book["id"], friend, lent_on.isoformat(), due.isoformat()))
    print(f"  Lent '{title}' to {friend}, due back {due}")

lend_book("Matilda", "Amara", lent_on=date(2026, 2, 10))
lend_book("Small Island", "Jonas", lent_on=date(2026, 2, 25))
lend_book("Persuasion", "Mei", lent_on=date(2026, 3, 9))
lend_book("Matilda", "Hana")
lend_book("Moby Dick", "Leo")
  Lent 'Matilda' to Amara, due back 2026-02-24
  Lent 'Small Island' to Jonas, due back 2026-03-11
  Lent 'Persuasion' to Mei, due back 2026-03-23
  Sorry Hana, 'Matilda' is already lent out
  No book called 'Moby Dick'
def return_book(title):
    book = find_book(title)
    if book is None:
        print(f"  No book called '{title}'")
        return
    with lib:
        changed = lib.execute("UPDATE loans SET returned = 1 WHERE book_id = ? AND returned = 0",
                              (book["id"],)).rowcount
    print(f"  Thanks for returning '{title}'" if changed else f"  '{title}' wasn't on loan")

return_book("Small Island")
return_book("Dune")
  Thanks for returning 'Small Island'
  'Dune' wasn't on loan
def overdue_report(today=TODAY):
    rows = lib.execute("""
        SELECT b.title, l.friend, l.due_on
        FROM loans AS l JOIN books AS b ON l.book_id = b.id
        WHERE l.returned = 0 AND l.due_on < ?
        ORDER BY l.due_on
    """, (today.isoformat(),)).fetchall()
    print(f"  Overdue on {today}: {len(rows)} book(s)")
    for r in rows:
        late = (today - date.fromisoformat(r["due_on"])).days
        print(f"  - {r['title']:<12} with {r['friend']:<6} {late} days late")

overdue_report()
  Overdue on 2026-03-16: 1 book(s)
  - Matilda      with Amara  20 days late
def shelf_report():
    rows = lib.execute("""
        SELECT b.title, b.author, l.friend
        FROM books AS b
        LEFT JOIN loans AS l ON l.book_id = b.id AND l.returned = 0
        ORDER BY b.title
    """).fetchall()
    for r in rows:
        where = f"with {r['friend']}" if r["friend"] else "on the shelf"
        print(f"  {r['title']:<13} {r['author']:<17} {where}")

shelf_report()
  Dune          Frank Herbert     on the shelf
  Matilda       Roald Dahl        with Amara
  Persuasion    Jane Austen       with Mei
  Small Island  Andrea Levy       on the shelf
  The Hobbit    J. R. R. Tolkien  on the shelf
  The Martian   Andy Weir         on the shelf
def run_library():
    while True:
        choice = input("[l]end [r]eturn [o]verdue [s]helf [q]uit: ").strip().lower()
        if choice == "l":
            lend_book(input("  Title: "), input("  Friend: "))
        elif choice == "r":
            return_book(input("  Title: "))
        elif choice == "o":
            overdue_report()
        elif choice == "s":
            shelf_report()
        elif choice == "q":
            print("  Goodbye, happy reading!")
            break
        else:
            print("  Please choose l, r, o, s or q")
print("Menu ready")
Menu ready
run_library()  # asks for input()
[l]end [r]eturn [o]verdue [s]helf [q]uit: l
  Title: Dune
  Friend: Priya
  Lent 'Dune' to Priya, due back 2026-03-30
[l]end [r]eturn [o]verdue [s]helf [q]uit: l
  Title: Dune
  Friend: Leo
  Sorry Leo, 'Dune' is already lent out
[l]end [r]eturn [o]verdue [s]helf [q]uit: l
  Title: The Hobbit
  Friend: Amara
  Lent 'The Hobbit' to Amara, due back 2026-03-30
[l]end [r]eturn [o]verdue [s]helf [q]uit: r
  Title: Persuasion
  Thanks for returning 'Persuasion'
[l]end [r]eturn [o]verdue [s]helf [q]uit: x
  Please choose l, r, o, s or q
[l]end [r]eturn [o]verdue [s]helf [q]uit: o
  Overdue on 2026-03-16: 1 book(s)
  - Matilda      with Amara  20 days late
[l]end [r]eturn [o]verdue [s]helf [q]uit: q
  Goodbye, happy reading!
history = pd.read_sql_query("""
    SELECT b.title, l.friend, l.lent_on, l.due_on, l.returned
    FROM loans AS l JOIN books AS b ON l.book_id = b.id
    ORDER BY l.lent_on
""", lib)
print(history)
          title friend     lent_on      due_on  returned
0       Matilda  Amara  2026-02-10  2026-02-24         0
1  Small Island  Jonas  2026-02-25  2026-03-11         1
2    Persuasion    Mei  2026-03-09  2026-03-23         1
3          Dune  Priya  2026-03-16  2026-03-30         0
4    The Hobbit  Amara  2026-03-16  2026-03-30         0
busiest = lib.execute("""
    SELECT friend, COUNT(*) AS n FROM loans GROUP BY friend ORDER BY n DESC, friend LIMIT 1
""").fetchone()
print(f"Most loans: {busiest['friend']} ({busiest['n']})")
lib.close()
print("Library saved in", library_path)
Most loans: Amara (2)
Library saved in lesson_files/library.db
with closing(sqlite3.connect(library_path)) as again:
    books, loans = again.execute("SELECT (SELECT COUNT(*) FROM books), (SELECT COUNT(*) FROM loans)").fetchone()
print(f"Reopened the file: {books} books and {loans} loans remembered")
Reopened the file: 6 books and 5 loans remembered

Recap#

  • SQLite is a whole database in one file; sqlite3.connect() opens it and a cursor runs SQL
  • CREATE TABLE with column types, PRIMARY KEY, NOT NULL, UNIQUE and CHECK rules
  • INSERT with ? placeholders and executemany; never build SQL with f-strings (SQL injection)
  • SELECT with WHERE, LIKE, IN, BETWEEN, ORDER BY and LIMIT
  • COUNT, SUM, AVG, MIN, MAX, GROUP BY and HAVING for summaries
  • UPDATE and DELETE always with a WHERE; commit() saves, rollback() undoes
  • JOIN and LEFT JOIN to combine tables through an id
  • sqlite3.Row, with conn:, closing() and pd.read_sql_query for tidy, friendly code

Try it yourself#

  1. Add a rating column to books (ALTER TABLE books ADD COLUMN rating INTEGER) and a rate_book(title, stars) function, then list your top-rated books.
  2. Add a renew_book(title, extra_days) function that pushes the due date back, but refuses if the book is already overdue.
  3. Add a friends table with phone numbers, link loans to it with friend_id, and make the overdue report show who to call.

What's next#

Lesson 10: Python and the Web: HTTP, APIs and Web Scraping. We'll fetch data from APIs, read web pages with BeautifulSoup, and build a price tracker that spots bargains.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.