Python · Lesson 10
The Standard Library
The batteries interviews and real code lean on: Counter, defaultdict, deque, OrderedDict, namedtuple and ChainMap, heapq and bisect, functools and itertools, datetime with time zones, json, re, decimal for money, and statistics, each with when to reach for it.
In short
Counter,defaultdict,deque,OrderedDictfor dataheapqfor priorities,bisectfor sorted lists- Aware datetimes in UTC; pass now in
jsonwithsort_keysanddefaultDecimalfrom strings for money
Python ships with a large standard library, and knowing a dozen modules in it well is worth more than
knowing a hundred tricks. In an interview, Counter turns a ten-line counting loop into one line, heapq
gives you a priority queue without writing one, and deque makes a sliding window O(1) at both ends. In a
codebase, datetime, json, re and decimal are where the subtle bugs live: naive datetimes, floats in
money, a regex that matches more than you meant.
This topic walks through the modules you'll actually use, with what each one is for. Every python snippet
has a ▶ Try button: predict the output, then run it.
| Reach for | When you need |
|---|---|
collections.Counter | counts of things, the top N, "are these two anagrams?" |
collections.defaultdict | grouping: a dict whose missing keys start as [], set() or 0 |
collections.deque | a queue, a sliding window, "the last N items" |
collections.OrderedDict | an LRU cache: move a key to the end, pop from the front |
collections.namedtuple | a small immutable record that is still a tuple |
collections.ChainMap | layered lookups: CLI flags over env vars over defaults |
heapq | the smallest (or largest) item, repeatedly; top K; a priority queue |
bisect | a sorted list you insert into and query by range |
functools | partial to pre-fill arguments, cache to memoize, reduce to fold |
itertools | lazy pipelines: chain, islice, groupby, accumulate, pairwise |
datetime | dates, durations, and time-zone-aware timestamps |
json | turning dicts and lists into text and back |
re | pattern matching on text you don't control |
decimal | money and anything else that must round like a person would |
statistics | mean, median, standard deviation, percentiles |
collections.Counter
A Counter is a dict that maps items to how many times they appeared. Missing keys count as 0 instead of
raising KeyError.
from collections import Counter
words = "the cat and the hat and the bat".split()
counts = Counter(words)
print(counts["the"], counts["dog"]) # missing keys are 0
print(counts.most_common(2)) # the top 2, as (item, count) pairs
print(Counter("listen") == Counter("silent")) # anagram check
most_common(n) sorts by count, and items with the same count stay in the order they were first seen.
Counters also add and subtract, and update counts more items in place:
from collections import Counter
stock = Counter(apples=3, pears=1)
sold = Counter(apples=1, pears=1)
print(stock - sold) # subtraction drops zero and negative counts
stock.update(["apples", "kiwis", "kiwis"])
print(stock)
print(stock.total()) # the sum of all counts (3.10+)
collections.defaultdict
defaultdict(factory) calls factory() the first time you read a missing key, stores the result, and
returns it. It's the tool for grouping:
from collections import defaultdict
orders = [("ann", "book"), ("bob", "pen"), ("ann", "lamp")]
by_customer = defaultdict(list)
for customer, item in orders:
by_customer[customer].append(item) # no "if customer not in ..." needed
print(dict(by_customer))
seen_on = defaultdict(set)
seen_on["ann"].add("mon")
seen_on["ann"].add("mon")
print(seen_on["ann"], seen_on["zoe"]) # reading a missing key creates it
print(sorted(seen_on))
That last line is the trap: reading a missing key inserts it. Use key in d or d.get(key) when you
only want to look. dict.setdefault(key, []) does the same job as defaultdict for a one-off.
collections.deque
A deque ("deck") is a double-ended queue: appending and popping at either end is O(1). A list is O(1)
only at the end; list.pop(0) shifts every other element.
from collections import deque
q = deque([1, 2, 3])
q.append(4) # right end
q.appendleft(0) # left end
print(q.popleft(), q.pop(), list(q))
recent = deque(maxlen=3) # keeps only the last 3
for event in ["a", "b", "c", "d", "e"]:
recent.append(event)
print(list(recent))
With maxlen, appending to a full deque silently drops an item from the other end, which is exactly "the
last N log lines" or a moving average window. Use a deque for every BFS queue: popleft() is the O(1)
dequeue. Indexing into the middle is O(n), so it isn't a list replacement.
collections.OrderedDict
Plain dicts have kept insertion order since Python 3.7, so OrderedDict is now for the two things a dict
can't do cheaply: move a key to an end and pop from the front. Those two operations are an LRU cache.
from collections import OrderedDict
cache = OrderedDict(a=1, b=2, c=3)
cache.move_to_end("a") # "a" was just used: make it newest
print(list(cache))
oldest = cache.popitem(last=False) # evict from the front
print(oldest, list(cache))
collections.namedtuple
A namedtuple is a tuple with named fields: immutable, unpackable, comparable, and as cheap as a tuple.
from collections import namedtuple
Point = namedtuple("Point", ["x", "y"])
p = Point(3, 4)
print(p.x, p[1], p)
x, y = p # still a tuple
print(p == (3, 4)) # equal to a plain tuple with the same values
print(p._replace(y=10)) # a modified copy
In new code, typing.NamedTuple (class syntax with type hints) or a @dataclass(frozen=True) usually reads
better. namedtuple shows up wherever a function returns a tuple whose fields deserve names.
collections.ChainMap
A ChainMap looks keys up in several dicts in order and returns the first hit, without copying them. It's
the natural shape for configuration layers.
from collections import ChainMap
defaults = {"port": 8080, "debug": False, "host": "localhost"}
env = {"port": 9000}
cli = {"debug": True}
config = ChainMap(cli, env, defaults)
print(config["port"], config["debug"], config["host"])
config["host"] = "0.0.0.0" # writes go to the first map only
print(cli)
heapq: priority queues
heapq turns a plain list into a binary min-heap: heap[0] is always the smallest item, heappush and
heappop are O(log n), and heapify builds a heap from a list in O(n).
import heapq
heap = [5, 1, 8, 3]
heapq.heapify(heap)
heapq.heappush(heap, 2)
print(heap[0]) # the smallest, without removing it
print([heapq.heappop(heap) for _ in range(len(heap))])
Python only has a min-heap. For a max-heap, push negated keys and negate them back when you pop. For
items with priorities, push tuples: they compare element by element, so (priority, item) orders by
priority. Add a counter as a tie-breaker, so equal priorities come out first in, first out and the heap
never has to compare the items themselves (which fails for dicts and most objects):
import heapq
from itertools import count
tasks = []
order = count()
for priority, name in [(2, "email"), (1, "deploy"), (2, "backup"), (3, "lint")]:
heapq.heappush(tasks, (priority, next(order), name))
print([heapq.heappop(tasks)[2] for _ in range(len(tasks))])
scores = [40, 95, 70, 88]
max_heap = [-s for s in scores]
heapq.heapify(max_heap)
print(-heapq.heappop(max_heap)) # the largest
For a one-off "top K", nlargest and nsmallest are clearer. They take a key, and run in O(n log k):
import heapq
players = {"ann": 120, "bob": 95, "cy": 140, "dee": 110}
print(heapq.nlargest(2, players, key=players.get))
print(heapq.nsmallest(2, [7, 3, 9, 1, 4]))
Rule of thumb: K much smaller than n, use nlargest; K close to n, use sorted(...)[:k]; one item, use
max. If you need the top items over and over while adding new ones, keep a heap.
bisect: sorted lists
bisect does binary search on a list that is already sorted. bisect_left(xs, x) is the first index where
x could go (before any equal items), bisect_right the last (after them), and insort inserts in place.
from bisect import bisect_left, bisect_right, insort
ages = [18, 21, 21, 30, 45]
print(bisect_left(ages, 21), bisect_right(ages, 21))
print(bisect_right(ages, 30) - bisect_left(ages, 21)) # how many in [21, 30]
insort(ages, 25)
print(ages)
The difference of a bisect_right and a bisect_left counts the items in a range in O(log n). The other
classic is mapping a number to a band, like a grade or a pricing tier:
from bisect import bisect_right
def grade(score: int) -> str:
return "FDCBA"[bisect_right([60, 70, 80, 90], score)]
print([grade(s) for s in [55, 60, 79, 90, 100]])
insort finds the spot in O(log n) but inserting still shifts the list, O(n). That's fine for thousands of
items. Both functions take a key= (3.10+) when the list holds records.
functools: partial, cache, reduce
partial pre-fills some arguments of a function and gives you a new one. cache (and lru_cache(maxsize=...))
memoizes a pure function, which recursive and dynamic programming solutions use constantly (the Decorators topic shows how it works inside).
reduce folds a sequence into one value.
from functools import cache, partial, reduce
import operator
def power(base: int, exp: int) -> int:
return base ** exp
square = partial(power, exp=2)
print(square(7))
@cache
def fib(n: int) -> int:
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(80), fib.cache_info().hits)
print(reduce(operator.mul, [1, 2, 3, 4], 1)) # 1*1*2*3*4
Reach for sum, max, any or math.prod before reduce: they say what they do. reduce earns its place
for folds without a name, like merging a list of dicts. Only cache functions whose arguments are hashable
and whose result depends on nothing but the arguments.
itertools, briefly
The Iterators and Generators topic covers itertools in depth. These are the ones that come up most:
from itertools import accumulate, chain, groupby, islice, pairwise, product
print(list(chain([1, 2], [3]))) # concatenate iterables
print(list(islice(range(100), 2, 5))) # slice anything, lazily
print(list(accumulate([3, 1, 4]))) # running totals
print(list(pairwise("abc"))) # neighbours
print(list(product("ab", repeat=2)))
logs = ["db", "db", "api", "db"]
print([(k, len(list(g))) for k, g in groupby(logs)])
groupby only groups adjacent equal items, which is why the last "db" is its own group. Sort by the
same key first when you want one group per key.
datetime: dates, durations, time zones
datetime has four types you need: date, datetime, timedelta (a duration) and timezone. Subtracting
two datetimes gives a timedelta; adding a timedelta moves a datetime.
from datetime import date, datetime, timedelta
start = datetime(2026, 1, 30, 22, 15)
later = start + timedelta(hours=3, minutes=50)
print(later, later.date())
print(later - start, (later - start).total_seconds())
print(date(2026, 3, 1) - date(2026, 2, 1))
print(start.strftime("%Y-%m-%d %H:%M"), start.isoformat())
print(datetime.fromisoformat("2026-07-04T09:30:00").weekday()) # Monday is 0
A datetime is either naive (no time zone: "09:00 somewhere") or aware (has a tzinfo: a real
instant). Ordering (<, >) or subtracting a naive and an aware datetime raises TypeError (== just
returns False), which is a feature.
Store and compute in UTC, convert to local time at the edges:
from datetime import datetime, timedelta, timezone
paid = datetime(2026, 5, 1, 23, 30, tzinfo=timezone.utc)
manila = timezone(timedelta(hours=8))
print(paid.astimezone(manila)) # the same instant, shown in UTC+8
print(paid.astimezone(manila).date()) # already the next day there
naive = datetime(2026, 5, 1, 23, 30)
print(naive.tzinfo, paid.tzinfo)
try:
print(paid > naive)
except TypeError as e:
print("TypeError:", e)
Fixed offsets don't know about daylight saving. For real places, use zoneinfo with an IANA name, which
reads the system's time zone database (the playground doesn't ship one, so this block doesn't run here):
# zoneinfo needs a time zone database (not runnable here)
from datetime import datetime
from zoneinfo import ZoneInfo
meeting = datetime(2026, 3, 30, 9, 0, tzinfo=ZoneInfo("Europe/London")) # BST, UTC+1
print(meeting.astimezone(ZoneInfo("America/New_York"))) # 2026-03-30 04:00:00-04:00
Two habits keep date code testable: pass "now" in as a parameter instead of calling datetime.now()
inside the function, and always use datetime.now(timezone.utc), never the naive datetime.now() or the
deprecated datetime.utcnow().
json
json.dumps turns Python values into a JSON string and json.loads parses one. Dicts become objects, lists
and tuples become arrays, None becomes null, and object keys always become strings.
import json
data = {"b": [1, 2], "a": None, "ok": True}
text = json.dumps(data, sort_keys=True)
print(text)
print(json.loads(text) == data)
print(json.loads(json.dumps({3: (4, 5)}))) # the key comes back as "3", the tuple as a list
print(json.dumps({"name": "Zoë"}), json.dumps({"name": "Zoë"}, ensure_ascii=False))
print(json.dumps({"a": 1, "b": [1, 2]}, indent=2))
sort_keys=True makes the output stable, which matters for caching, signing and snapshot tests.
dumps raises TypeError for anything it doesn't know (a datetime, a Decimal, a set). Pass default=, a
function it calls for those values:
import json
from datetime import date
from decimal import Decimal
def encode(value):
if isinstance(value, (date, Decimal)):
return str(value)
if isinstance(value, set):
return sorted(value)
raise TypeError(f"can't encode {type(value).__name__}")
order = {"day": date(2026, 2, 3), "total": Decimal("19.90"), "tags": {"gift", "eu"}}
print(json.dumps(order, default=encode, sort_keys=True))
separators=(",", ":") drops the spaces for the most compact output.
re: regular expressions
Use re when string methods aren't enough. Write patterns as raw strings (r"\d+") so backslashes
reach the regex engine untouched.
| Function | Does |
|---|---|
re.search(p, s) | finds the first match anywhere, or returns None |
re.match(p, s) | matches only at the start |
re.fullmatch(p, s) | the whole string must match: the one for validation |
re.findall(p, s) | every match, as strings (or tuples of groups) |
re.sub(p, repl, s) | replaces every match; repl can be a function |
re.compile(p) | a pattern object to reuse |
import re
line = "2026-04-01 ERROR [db] timeout after 30s"
m = re.search(r"(\d{4})-(\d{2})-(\d{2}) (\w+)", line)
print(m.group(0), "|", m.group(4), m.groups()[:3])
m = re.search(r"\[(?P<component>\w+)\]", line)
print(m["component"]) # named groups
print(re.findall(r"\d+", "3 apples, 12 pears"))
print(re.findall(r"(\w+)=(\d+)", "a=1 b=22")) # with groups: tuples of the groups
print(re.match(r"\d+", "abc 123"), re.search(r"\d+", "abc 123").group())
re.sub with a function computes each replacement from the match, and a compiled pattern keeps a regex
you use in a loop in one named place:
import re
EMAIL = re.compile(r"([\w.]+)@([\w.]+)")
def mask(m: re.Match) -> str:
user, domain = m.groups()
return f"{user[0]}***@{domain}"
print(EMAIL.sub(mask, "contact ann.lee@example.com or bob@test.io"))
print(bool(re.fullmatch(r"[A-Z]{3}-\d{4}", "ABC-1234")), bool(re.fullmatch(r"[A-Z]{3}-\d{4}", "ABC-12345")))
print(re.sub(r"\s+", " ", " too many spaces ").strip())
* and + are greedy: <.+> on <a><b> matches the whole string. Add ? (<.+?>) for the shortest
match. re.IGNORECASE and re.MULTILINE go in flags=.
decimal: money
Floats are binary, so 0.1 + 0.2 isn't 0.3. Decimal is decimal arithmetic that rounds the way an
accountant expects. Always build it from a string (or an int): Decimal(0.1) faithfully copies the
float's binary error.
from decimal import Decimal, ROUND_HALF_UP
print(0.1 + 0.2, Decimal("0.1") + Decimal("0.2"))
print(Decimal(0.1))
price = Decimal("19.99")
total = price * 3 * Decimal("1.0825") # with sales tax
print(total)
print(total.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP))
print(Decimal("2.675").quantize(Decimal("0.01"), rounding=ROUND_HALF_UP), round(2.675, 2))
quantize(Decimal("0.01")) rounds to cents. The default rounding is ROUND_HALF_EVEN (banker's rounding),
so pass ROUND_HALF_UP when a spec says "half up". Arithmetic mixing Decimal and float raises TypeError,
which keeps floats out of the sums (comparisons and Decimal(0.1) still accept floats). Store money as Decimal or as integer cents; never as float.
statistics
statistics covers the everyday summary numbers, without NumPy:
import statistics as st
latencies = [12, 15, 11, 90, 14, 13, 15]
print(st.mean(latencies), st.median(latencies), st.mode(latencies))
print(round(st.stdev(latencies), 2))
print(st.quantiles(latencies, n=4)) # the three quartile cut points
The median (14) describes "a typical request" far better than the mean (about 24.3), which one slow outlier
pulled up. That's why dashboards show p50 and p95, not averages. stdev is the sample standard deviation;
pstdev is the population one.
How the exercises work
Each exercise is a small piece of real work where one of these modules does the heavy lifting: counting words, an inverted index, an LRU cache, a rate limiter, daily active users across time zones, an invoice in JSON, and a priority job queue. The tests pass every date and time in, so nothing depends on the clock.
Practise this lesson
In the app: a short video, 7 exercises checked by real tests and a 12-question concept check. Free with a GitHub account.
Open in the app