Python · Lesson 3
Comprehensions and Iteration
List, dict and set comprehensions, generator expressions, enumerate and zip, sorting with key functions and stable multi-key sorts, any and all, sum, map and filter, and when a plain loop reads better.
In short
- Comprehensions: expression,
for, optionalif - Round brackets are lazy and single-use
enumerate, andzip(..., strict=True)sortedwith a key; tuples for several keys; stableany,all,sum; a loop for side effects
Most code that handles data walks over a collection and builds another one: filter the active users, map
rows to ids, total the invoice lines, sort by date. JavaScript does it with .filter().map() chains, C#
with LINQ. Python's answer is the comprehension, plus a small set of built-ins (enumerate, zip,
sorted, any, all, sum) that take any iterable. Together they replace most hand-written loops, and
interviewers expect you to reach for them.
| You want | JavaScript | C# LINQ | Python |
|---|---|---|---|
| transform each | xs.map(f) | xs.Select(f) | [f(x) for x in xs] |
| keep some | xs.filter(p) | xs.Where(p) | [x for x in xs if p(x)] |
| index and item | xs.forEach((x, i) => ...) | xs.Select((x, i) => ...) | for i, x in enumerate(xs) |
| pair two lists | n/a | a.Zip(b) | zip(a, b) |
| any / every | some / every | Any / All | any(...) / all(...) |
| total | reduce | Sum | sum(...) |
| sort by | sort((a, b) => ...) | OrderBy(k).ThenBy(k2) | sorted(xs, key=...) |
Every snippet has a ▶ Try button. Predict the output, then run it.
List comprehensions
A list comprehension builds a list from an iterable: an expression, a for clause, and an optional if
filter. Read it as "this expression, for each x, if the condition holds":
prices = [12, 7, 30, 4]
print([p * 2 for p in prices])
print([p for p in prices if p >= 10])
print([f"${p}" for p in prices if p % 2 == 0])
The filter if goes at the end and drops items. A conditional expression goes at the front and
picks a value for every item. They're easy to mix up:
nums = [3, -1, 4, -5]
print([n for n in nums if n > 0]) # filter: fewer items
print([n if n > 0 else 0 for n in nums]) # choose: same number of items
A comprehension is an expression, so it goes straight into a return, an argument or a dict value. Its
loop variable stays inside it: after [x for x in xs], there's no x in the surrounding scope.
Dict and set comprehensions
The same shape with curly braces builds a dict (with key: value) or a set:
users = [("u1", "ann"), ("u2", "bo"), ("u3", "ann")]
by_id = {uid: name for uid, name in users}
names = {name for _, name in users}
lengths = {name: len(name) for name in sorted(names)}
print(by_id)
print(sorted(names), lengths)
Inverting a dict is a one-liner, and a classic interview question. When two keys share a value, the last one wins, because a later assignment to the same key replaces the earlier one:
role = {"ann": "admin", "bo": "dev", "cy": "admin"}
print({r: name for name, r in role.items()})
If you need every key per value, that's grouping, and a loop with setdefault is the clearer tool (see the
Collections topic).
Nested comprehensions
Several for clauses nest in the order you'd write the loops, outermost first. That's the rule to
remember, because it reads backwards to many people:
orders = [["pen", "ink"], [], ["mug"]]
items = [item for order in orders for item in order]
print(items)
# The same as:
items = []
for order in orders:
for item in order:
items.append(item)
print(items)
A comprehension inside the expression builds nested lists, like rows of a grid. Transposing a matrix uses one:
matrix = [[1, 2, 3], [4, 5, 6]]
print([[row[c] for row in matrix] for c in range(3)])
print([(r, c) for r in range(2) for c in range(2) if r != c])
Two for clauses are fine. Three, or two with conditions on each, is usually the point where a named
helper function or a plain loop reads better.
Generator expressions: lazy comprehensions
Parentheses instead of brackets make a generator expression. It produces items one at a time, when something asks for them, instead of building a list. Passed straight to a function, it doesn't even need its own parentheses:
nums = range(1, 1_000_001)
print(sum(n * n for n in nums)) # no million-item list is built
print(max(len(w) for w in ["a", "abc", "ab"]))
squares = (n * n for n in [1, 2, 3])
print(squares) # a generator object, not the values
print(list(squares))
print(list(squares)) # already used up
A generator can be consumed once. The second list(squares) is empty, without an error. That's the
same as a C# IEnumerable built with yield return, except that C# reruns the query on each enumeration
and Python's generator is simply exhausted. Pass a list when a function needs to walk the data twice.
next(gen, default) takes the first item and stops there. With a generator expression it's the idiomatic
"find the first match", like JavaScript's find:
users = [{"name": "ann", "admin": False}, {"name": "bo", "admin": True}, {"name": "cy", "admin": True}]
first_admin = next((u["name"] for u in users if u["admin"]), None)
nobody = next((u["name"] for u in users if u["name"] == "zed"), None)
print(first_admin, nobody)
Without the default, next raises StopIteration when nothing matches.
enumerate and zip
enumerate(xs, start=0) gives (index, item) pairs, so you never need range(len(xs)):
steps = ["build", "test", "deploy"]
for n, step in enumerate(steps, start=1):
print(f"{n}. {step}")
zip(a, b, ...) walks several iterables in step and yields tuples. It stops at the shortest input,
silently, which hides bugs when two lists were supposed to match. strict=True (Python 3.10+) raises
ValueError instead:
names = ["ann", "bo", "cy"]
scores = [90, 72]
print(list(zip(names, scores)))
print(dict(zip(["id", "name"], [7, "ann"])))
try:
list(zip(names, scores, strict=True))
except ValueError as e:
print("ValueError:", e)
zip(*rows) "unzips": the star spreads the rows as separate arguments, so zip pairs up their first items,
then their second items. That's a transpose in one line. And zip(xs, xs[1:]) pairs each item with the
next, which is how you compute differences between neighbours:
pairs = [("ann", 90), ("bo", 72), ("cy", 85)]
names, scores = zip(*pairs)
print(names, scores)
times = [0, 4, 5, 12]
print([b - a for a, b in zip(times, times[1:])])
sorted, min and max with a key
sorted(iterable, key=..., reverse=...) returns a new list. list.sort() takes the same arguments and
sorts in place. Both take a key function that maps each item to the value to compare. JavaScript and
C#'s Comparison<T> use a comparator (a, b) => number instead. A key is simpler: you say what to sort by,
not how to compare two items.
words = ["banana", "Cherry", "apple", "fig"]
print(sorted(words)) # uppercase sorts before lowercase
print(sorted(words, key=str.lower))
print(sorted(words, key=len, reverse=True))
min and max take the same key and return the item, not the key. Both raise ValueError on an
empty iterable unless you pass default=:
products = [{"sku": "a", "price": 30}, {"sku": "b", "price": 12}, {"sku": "c", "price": 12}]
print(min(products, key=lambda p: p["price"]))
print(max([], default=None))
When several items tie, min and max return the first of them, which is why that prints product b.
Multi-key sorts and stability
A key that returns a tuple sorts by the first element, then the second for ties, and so on, like
OrderBy(...).ThenBy(...). To sort one numeric part descending, negate it:
people = [("cy", 30), ("ann", 25), ("bo", 30), ("di", 25)]
print(sorted(people, key=lambda p: (p[1], p[0]))) # age, then name
print(sorted(people, key=lambda p: (-p[1], p[0]))) # age descending, then name
You can't negate a string. For mixed directions on strings, use the fact that Python's sort is stable: items that compare equal keep their original relative order. So sort by the least important key first, then by the more important ones:
tickets = [("bug", "bo"), ("feature", "ann"), ("bug", "ann"), ("chore", "cy")]
by_owner = sorted(tickets, key=lambda t: t[1]) # secondary: owner A-Z
print(sorted(by_owner, key=lambda t: t[0], reverse=True)) # primary: kind Z-A
The final sort is by kind, and among the two bugs, the owner order from the first pass survives. reverse=True
keeps stability too: it doesn't reverse the order of equal items. C#'s List.Sort is not stable
(OrderBy is), and JavaScript's sort has been stable since ES2019.
reversed
reversed(seq) returns a lazy iterator that walks a sequence backwards without copying it. seq[::-1]
builds a reversed copy. reversed needs a sequence (a length and indexes: a list, tuple, string or
range) or an object with __reversed__, like a dict, so it fails on a generator or a set:
print(list(reversed([1, 2, 3])), "abc"[::-1], list(reversed(range(3))))
print(list(reversed({"a": 1, "b": 2}))) # dicts reverse their key order
try:
reversed(n for n in [1, 2])
except TypeError as e:
print("TypeError:", e)
any, all and sum
any is true if at least one item is truthy, and all if every item is. Both stop as soon as the answer
is known, so with a generator expression they evaluate only as much as they need. On an empty iterable,
any is False and all is True ("every one of zero items passed"), which matters when the list can be
empty:
ports = [80, 443, 8080]
print(any(p > 1024 for p in ports), all(p > 0 for p in ports))
print(any([]), all([]))
def check(p: int) -> bool:
print("checking", p)
return p == 443
print(any(check(p) for p in ports)) # stops at 443
sum(iterable, start=0) adds numbers. The start value sets the starting point and the type: 0.0 for
floats, Decimal("0") for money. It also concatenates lists, but that copies the growing list at every step,
so it's quadratic. Flatten with a comprehension or itertools.chain instead. sum refuses strings
outright, because "".join(...) is the right tool:
print(sum([1, 2, 3]), sum([1, 2, 3], start=10), sum([1, 2], 0.0))
print(sum([[1], [2, 3]], [])) # works, but O(n^2): avoid for big inputs
print(sum(p["qty"] * p["price"] for p in [{"qty": 2, "price": 5}, {"qty": 1, "price": 3}]))
try:
sum(["a", "b"], "")
except TypeError as e:
print("TypeError:", e)
Counting with sum is a common idiom: True is 1, so sum(x > 0 for x in xs) counts the positives.
map and filter versus comprehensions
map(f, xs) and filter(pred, xs) exist and return lazy iterators, like generator expressions. That
surprises people from JavaScript, where map returns an array:
nums = ["3", "1", "2"]
print(map(int, nums))
print(list(map(int, nums)))
print(list(filter(None, ["a", "", "b", None, 0]))) # None as the predicate drops falsy items
Pythonic code prefers comprehensions in most cases, because they don't need a lambda and can map and
filter in one step:
users = [{"name": "ann", "active": True}, {"name": "bo", "active": False}]
print(list(map(lambda u: u["name"], filter(lambda u: u["active"], users))))
print([u["name"] for u in users if u["active"]])
Both lines do the same; the second is what reviewers expect. map reads fine when you already have a named
function: map(str, ids), map(int, line.split()).
When a plain loop is clearer
Comprehensions are for building a value. Use a plain for loop when:
- The point is a side effect.
[print(x) for x in xs]builds a list ofNones to throw away. Write the loop. - You build several results at once. Splitting into valid and invalid rows in one pass is two
appends in a loop, not two comprehensions that each walk the data. - There's error handling or early exit with logic. A
try/exceptcan't go inside a comprehension. - It needs a third nesting level, or a condition you have to read twice. Name it.
rows = ["7", "x", "12", "", "-3"]
valid, errors = [], []
for line, raw in enumerate(rows, start=1):
try:
valid.append(int(raw))
except ValueError:
errors.append(f"line {line}: {raw!r} is not a number")
print(valid)
print(errors)
itertools, briefly
The itertools module has the building blocks for everything else, and they're all lazy:
| Function | What it does |
|---|---|
chain(a, b) / chain.from_iterable(xss) | one iterable after another; flattens one level |
islice(it, n) | the first n items of any iterable, even an endless one |
groupby(xs, key) | runs of consecutive items with the same key (sort first to group them all) |
pairwise(xs) | (x0, x1), (x1, x2)..., like zip(xs, xs[1:]) for any iterable |
accumulate(xs) | running totals |
from itertools import accumulate, chain, groupby, islice, pairwise
print(list(chain([1, 2], [3])), list(islice("abcdef", 3)))
print([(k, len(list(g))) for k, g in groupby("aaabccaa")])
print(list(pairwise([1, 4, 9])), list(accumulate([5, 10, 20])))
groupby groups only consecutive items, so "aaabccaa" gives two separate runs of a. Sort by the same
key first when you want one group per key.
The Iterators and Generators topic shows how all of this works underneath: the iterator protocol,
yield, and writing your own lazy pipelines.
Practise this lesson
In the app: a short video, 7 exercises checked by real tests and a 12-question concept check. Free with a GitHub account.
Open in the app