Python · Lesson 10

The Standard Library

The batteries interviews and real code lean on: Counter, defaultdict, deque, OrderedDict, namedtuple and ChainMap, heapq and bisect, functools and itertools, datetime with time zones, json, re, decimal for money, and statistics, each with when to reach for it.

13 min read

In short

  • Counter, defaultdict, deque, OrderedDict for data
  • heapq for priorities, bisect for sorted lists
  • Aware datetimes in UTC; pass now in
  • json with sort_keys and default
  • Decimal from strings for money

Python ships with a large standard library, and knowing a dozen modules in it well is worth more than knowing a hundred tricks. In an interview, Counter turns a ten-line counting loop into one line, heapq gives you a priority queue without writing one, and deque makes a sliding window O(1) at both ends. In a codebase, datetime, json, re and decimal are where the subtle bugs live: naive datetimes, floats in money, a regex that matches more than you meant.

This topic walks through the modules you'll actually use, with what each one is for. Every python snippet has a ▶ Try button: predict the output, then run it.

Reach forWhen you need
collections.Countercounts of things, the top N, "are these two anagrams?"
collections.defaultdictgrouping: a dict whose missing keys start as [], set() or 0
collections.dequea queue, a sliding window, "the last N items"
collections.OrderedDictan LRU cache: move a key to the end, pop from the front
collections.namedtuplea small immutable record that is still a tuple
collections.ChainMaplayered lookups: CLI flags over env vars over defaults
heapqthe smallest (or largest) item, repeatedly; top K; a priority queue
bisecta sorted list you insert into and query by range
functoolspartial to pre-fill arguments, cache to memoize, reduce to fold
itertoolslazy pipelines: chain, islice, groupby, accumulate, pairwise
datetimedates, durations, and time-zone-aware timestamps
jsonturning dicts and lists into text and back
repattern matching on text you don't control
decimalmoney and anything else that must round like a person would
statisticsmean, median, standard deviation, percentiles

collections.Counter

A Counter is a dict that maps items to how many times they appeared. Missing keys count as 0 instead of raising KeyError.

▶ Try
from collections import Counter

words = "the cat and the hat and the bat".split()
counts = Counter(words)
print(counts["the"], counts["dog"])     # missing keys are 0
print(counts.most_common(2))            # the top 2, as (item, count) pairs
print(Counter("listen") == Counter("silent"))   # anagram check

most_common(n) sorts by count, and items with the same count stay in the order they were first seen. Counters also add and subtract, and update counts more items in place:

▶ Try
from collections import Counter

stock = Counter(apples=3, pears=1)
sold = Counter(apples=1, pears=1)
print(stock - sold)          # subtraction drops zero and negative counts
stock.update(["apples", "kiwis", "kiwis"])
print(stock)
print(stock.total())         # the sum of all counts (3.10+)

collections.defaultdict

defaultdict(factory) calls factory() the first time you read a missing key, stores the result, and returns it. It's the tool for grouping:

▶ Try
from collections import defaultdict

orders = [("ann", "book"), ("bob", "pen"), ("ann", "lamp")]
by_customer = defaultdict(list)
for customer, item in orders:
    by_customer[customer].append(item)     # no "if customer not in ..." needed
print(dict(by_customer))

seen_on = defaultdict(set)
seen_on["ann"].add("mon")
seen_on["ann"].add("mon")
print(seen_on["ann"], seen_on["zoe"])      # reading a missing key creates it
print(sorted(seen_on))

That last line is the trap: reading a missing key inserts it. Use key in d or d.get(key) when you only want to look. dict.setdefault(key, []) does the same job as defaultdict for a one-off.

collections.deque

A deque ("deck") is a double-ended queue: appending and popping at either end is O(1). A list is O(1) only at the end; list.pop(0) shifts every other element.

▶ Try
from collections import deque

q = deque([1, 2, 3])
q.append(4)            # right end
q.appendleft(0)        # left end
print(q.popleft(), q.pop(), list(q))

recent = deque(maxlen=3)       # keeps only the last 3
for event in ["a", "b", "c", "d", "e"]:
    recent.append(event)
print(list(recent))

With maxlen, appending to a full deque silently drops an item from the other end, which is exactly "the last N log lines" or a moving average window. Use a deque for every BFS queue: popleft() is the O(1) dequeue. Indexing into the middle is O(n), so it isn't a list replacement.

collections.OrderedDict

Plain dicts have kept insertion order since Python 3.7, so OrderedDict is now for the two things a dict can't do cheaply: move a key to an end and pop from the front. Those two operations are an LRU cache.

▶ Try
from collections import OrderedDict

cache = OrderedDict(a=1, b=2, c=3)
cache.move_to_end("a")               # "a" was just used: make it newest
print(list(cache))
oldest = cache.popitem(last=False)   # evict from the front
print(oldest, list(cache))

collections.namedtuple

A namedtuple is a tuple with named fields: immutable, unpackable, comparable, and as cheap as a tuple.

▶ Try
from collections import namedtuple

Point = namedtuple("Point", ["x", "y"])
p = Point(3, 4)
print(p.x, p[1], p)
x, y = p                         # still a tuple
print(p == (3, 4))               # equal to a plain tuple with the same values
print(p._replace(y=10))          # a modified copy

In new code, typing.NamedTuple (class syntax with type hints) or a @dataclass(frozen=True) usually reads better. namedtuple shows up wherever a function returns a tuple whose fields deserve names.

collections.ChainMap

A ChainMap looks keys up in several dicts in order and returns the first hit, without copying them. It's the natural shape for configuration layers.

▶ Try
from collections import ChainMap

defaults = {"port": 8080, "debug": False, "host": "localhost"}
env = {"port": 9000}
cli = {"debug": True}
config = ChainMap(cli, env, defaults)
print(config["port"], config["debug"], config["host"])
config["host"] = "0.0.0.0"       # writes go to the first map only
print(cli)

heapq: priority queues

heapq turns a plain list into a binary min-heap: heap[0] is always the smallest item, heappush and heappop are O(log n), and heapify builds a heap from a list in O(n).

▶ Try
import heapq

heap = [5, 1, 8, 3]
heapq.heapify(heap)
heapq.heappush(heap, 2)
print(heap[0])                                   # the smallest, without removing it
print([heapq.heappop(heap) for _ in range(len(heap))])

Python only has a min-heap. For a max-heap, push negated keys and negate them back when you pop. For items with priorities, push tuples: they compare element by element, so (priority, item) orders by priority. Add a counter as a tie-breaker, so equal priorities come out first in, first out and the heap never has to compare the items themselves (which fails for dicts and most objects):

▶ Try
import heapq
from itertools import count

tasks = []
order = count()
for priority, name in [(2, "email"), (1, "deploy"), (2, "backup"), (3, "lint")]:
    heapq.heappush(tasks, (priority, next(order), name))
print([heapq.heappop(tasks)[2] for _ in range(len(tasks))])

scores = [40, 95, 70, 88]
max_heap = [-s for s in scores]
heapq.heapify(max_heap)
print(-heapq.heappop(max_heap))                  # the largest

For a one-off "top K", nlargest and nsmallest are clearer. They take a key, and run in O(n log k):

▶ Try
import heapq

players = {"ann": 120, "bob": 95, "cy": 140, "dee": 110}
print(heapq.nlargest(2, players, key=players.get))
print(heapq.nsmallest(2, [7, 3, 9, 1, 4]))

Rule of thumb: K much smaller than n, use nlargest; K close to n, use sorted(...)[:k]; one item, use max. If you need the top items over and over while adding new ones, keep a heap.

bisect: sorted lists

bisect does binary search on a list that is already sorted. bisect_left(xs, x) is the first index where x could go (before any equal items), bisect_right the last (after them), and insort inserts in place.

▶ Try
from bisect import bisect_left, bisect_right, insort

ages = [18, 21, 21, 30, 45]
print(bisect_left(ages, 21), bisect_right(ages, 21))
print(bisect_right(ages, 30) - bisect_left(ages, 21))   # how many in [21, 30]
insort(ages, 25)
print(ages)

The difference of a bisect_right and a bisect_left counts the items in a range in O(log n). The other classic is mapping a number to a band, like a grade or a pricing tier:

▶ Try
from bisect import bisect_right

def grade(score: int) -> str:
    return "FDCBA"[bisect_right([60, 70, 80, 90], score)]

print([grade(s) for s in [55, 60, 79, 90, 100]])

insort finds the spot in O(log n) but inserting still shifts the list, O(n). That's fine for thousands of items. Both functions take a key= (3.10+) when the list holds records.

functools: partial, cache, reduce

partial pre-fills some arguments of a function and gives you a new one. cache (and lru_cache(maxsize=...)) memoizes a pure function, which recursive and dynamic programming solutions use constantly (the Decorators topic shows how it works inside). reduce folds a sequence into one value.

▶ Try
from functools import cache, partial, reduce
import operator

def power(base: int, exp: int) -> int:
    return base ** exp

square = partial(power, exp=2)
print(square(7))

@cache
def fib(n: int) -> int:
    return n if n < 2 else fib(n - 1) + fib(n - 2)

print(fib(80), fib.cache_info().hits)
print(reduce(operator.mul, [1, 2, 3, 4], 1))     # 1*1*2*3*4

Reach for sum, max, any or math.prod before reduce: they say what they do. reduce earns its place for folds without a name, like merging a list of dicts. Only cache functions whose arguments are hashable and whose result depends on nothing but the arguments.

itertools, briefly

The Iterators and Generators topic covers itertools in depth. These are the ones that come up most:

▶ Try
from itertools import accumulate, chain, groupby, islice, pairwise, product

print(list(chain([1, 2], [3])))                 # concatenate iterables
print(list(islice(range(100), 2, 5)))           # slice anything, lazily
print(list(accumulate([3, 1, 4])))              # running totals
print(list(pairwise("abc")))                    # neighbours
print(list(product("ab", repeat=2)))
logs = ["db", "db", "api", "db"]
print([(k, len(list(g))) for k, g in groupby(logs)])

groupby only groups adjacent equal items, which is why the last "db" is its own group. Sort by the same key first when you want one group per key.

datetime: dates, durations, time zones

datetime has four types you need: date, datetime, timedelta (a duration) and timezone. Subtracting two datetimes gives a timedelta; adding a timedelta moves a datetime.

▶ Try
from datetime import date, datetime, timedelta

start = datetime(2026, 1, 30, 22, 15)
later = start + timedelta(hours=3, minutes=50)
print(later, later.date())
print(later - start, (later - start).total_seconds())
print(date(2026, 3, 1) - date(2026, 2, 1))
print(start.strftime("%Y-%m-%d %H:%M"), start.isoformat())
print(datetime.fromisoformat("2026-07-04T09:30:00").weekday())   # Monday is 0

A datetime is either naive (no time zone: "09:00 somewhere") or aware (has a tzinfo: a real instant). Ordering (<, >) or subtracting a naive and an aware datetime raises TypeError (== just returns False), which is a feature. Store and compute in UTC, convert to local time at the edges:

▶ Try
from datetime import datetime, timedelta, timezone

paid = datetime(2026, 5, 1, 23, 30, tzinfo=timezone.utc)
manila = timezone(timedelta(hours=8))
print(paid.astimezone(manila))                   # the same instant, shown in UTC+8
print(paid.astimezone(manila).date())            # already the next day there
naive = datetime(2026, 5, 1, 23, 30)
print(naive.tzinfo, paid.tzinfo)
try:
    print(paid > naive)
except TypeError as e:
    print("TypeError:", e)

Fixed offsets don't know about daylight saving. For real places, use zoneinfo with an IANA name, which reads the system's time zone database (the playground doesn't ship one, so this block doesn't run here):

# zoneinfo needs a time zone database (not runnable here)
from datetime import datetime
from zoneinfo import ZoneInfo

meeting = datetime(2026, 3, 30, 9, 0, tzinfo=ZoneInfo("Europe/London"))   # BST, UTC+1
print(meeting.astimezone(ZoneInfo("America/New_York")))  # 2026-03-30 04:00:00-04:00

Two habits keep date code testable: pass "now" in as a parameter instead of calling datetime.now() inside the function, and always use datetime.now(timezone.utc), never the naive datetime.now() or the deprecated datetime.utcnow().

json

json.dumps turns Python values into a JSON string and json.loads parses one. Dicts become objects, lists and tuples become arrays, None becomes null, and object keys always become strings.

▶ Try
import json

data = {"b": [1, 2], "a": None, "ok": True}
text = json.dumps(data, sort_keys=True)
print(text)
print(json.loads(text) == data)
print(json.loads(json.dumps({3: (4, 5)})))       # the key comes back as "3", the tuple as a list
print(json.dumps({"name": "Zoë"}), json.dumps({"name": "Zoë"}, ensure_ascii=False))
print(json.dumps({"a": 1, "b": [1, 2]}, indent=2))

sort_keys=True makes the output stable, which matters for caching, signing and snapshot tests. dumps raises TypeError for anything it doesn't know (a datetime, a Decimal, a set). Pass default=, a function it calls for those values:

▶ Try
import json
from datetime import date
from decimal import Decimal

def encode(value):
    if isinstance(value, (date, Decimal)):
        return str(value)
    if isinstance(value, set):
        return sorted(value)
    raise TypeError(f"can't encode {type(value).__name__}")

order = {"day": date(2026, 2, 3), "total": Decimal("19.90"), "tags": {"gift", "eu"}}
print(json.dumps(order, default=encode, sort_keys=True))

separators=(",", ":") drops the spaces for the most compact output.

re: regular expressions

Use re when string methods aren't enough. Write patterns as raw strings (r"\d+") so backslashes reach the regex engine untouched.

FunctionDoes
re.search(p, s)finds the first match anywhere, or returns None
re.match(p, s)matches only at the start
re.fullmatch(p, s)the whole string must match: the one for validation
re.findall(p, s)every match, as strings (or tuples of groups)
re.sub(p, repl, s)replaces every match; repl can be a function
re.compile(p)a pattern object to reuse
▶ Try
import re

line = "2026-04-01 ERROR [db] timeout after 30s"
m = re.search(r"(\d{4})-(\d{2})-(\d{2}) (\w+)", line)
print(m.group(0), "|", m.group(4), m.groups()[:3])

m = re.search(r"\[(?P<component>\w+)\]", line)
print(m["component"])                            # named groups

print(re.findall(r"\d+", "3 apples, 12 pears"))
print(re.findall(r"(\w+)=(\d+)", "a=1 b=22"))    # with groups: tuples of the groups
print(re.match(r"\d+", "abc 123"), re.search(r"\d+", "abc 123").group())

re.sub with a function computes each replacement from the match, and a compiled pattern keeps a regex you use in a loop in one named place:

▶ Try
import re

EMAIL = re.compile(r"([\w.]+)@([\w.]+)")

def mask(m: re.Match) -> str:
    user, domain = m.groups()
    return f"{user[0]}***@{domain}"

print(EMAIL.sub(mask, "contact ann.lee@example.com or bob@test.io"))
print(bool(re.fullmatch(r"[A-Z]{3}-\d{4}", "ABC-1234")), bool(re.fullmatch(r"[A-Z]{3}-\d{4}", "ABC-12345")))
print(re.sub(r"\s+", " ", "  too   many    spaces ").strip())

* and + are greedy: <.+> on <a><b> matches the whole string. Add ? (<.+?>) for the shortest match. re.IGNORECASE and re.MULTILINE go in flags=.

decimal: money

Floats are binary, so 0.1 + 0.2 isn't 0.3. Decimal is decimal arithmetic that rounds the way an accountant expects. Always build it from a string (or an int): Decimal(0.1) faithfully copies the float's binary error.

▶ Try
from decimal import Decimal, ROUND_HALF_UP

print(0.1 + 0.2, Decimal("0.1") + Decimal("0.2"))
print(Decimal(0.1))
price = Decimal("19.99")
total = price * 3 * Decimal("1.0825")            # with sales tax
print(total)
print(total.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP))
print(Decimal("2.675").quantize(Decimal("0.01"), rounding=ROUND_HALF_UP), round(2.675, 2))

quantize(Decimal("0.01")) rounds to cents. The default rounding is ROUND_HALF_EVEN (banker's rounding), so pass ROUND_HALF_UP when a spec says "half up". Arithmetic mixing Decimal and float raises TypeError, which keeps floats out of the sums (comparisons and Decimal(0.1) still accept floats). Store money as Decimal or as integer cents; never as float.

statistics

statistics covers the everyday summary numbers, without NumPy:

▶ Try
import statistics as st

latencies = [12, 15, 11, 90, 14, 13, 15]
print(st.mean(latencies), st.median(latencies), st.mode(latencies))
print(round(st.stdev(latencies), 2))
print(st.quantiles(latencies, n=4))              # the three quartile cut points

The median (14) describes "a typical request" far better than the mean (about 24.3), which one slow outlier pulled up. That's why dashboards show p50 and p95, not averages. stdev is the sample standard deviation; pstdev is the population one.

How the exercises work

Each exercise is a small piece of real work where one of these modules does the heavy lifting: counting words, an inverted index, an LRU cache, a rate limiter, daily active users across time zones, an invoice in JSON, and a priority job queue. The tests pass every date and time in, so nothing depends on the clock.

Practise this lesson

In the app: a short video, 7 exercises checked by real tests and a 12-question concept check. Free with a GitHub account.

Open in the app