Python · Lesson 2
Collections
Lists, tuples, dicts and sets, slicing and unpacking, aliasing and shallow versus deep copies, dict ordering and views, hashability, and why membership in a set is fast while membership in a list is not.
In short
- Pick the collection by the question you ask
- Slices copy and never raise;
*restcollects a list getandsetdefault; dicts keep insertion order- Use a set for
in; keys must be hashable - Assignment shares; copy shallow or deep on purpose
Almost every Python program is lists and dicts passed around between functions. API payloads arrive as dicts of lists, a database query returns a list of tuples, and the "business logic" in the middle is grouping, deduplicating and looking things up. Python gives you four built-in collections, and picking the right one, then knowing whether you are holding a copy or a second name for the same object, prevents most of the bugs.
| Literal | Mutable | Ordered | Lookup by | JavaScript | C# | |
|---|---|---|---|---|---|---|
list | [1, 2, 3] | yes | yes | position | Array | List<T> |
tuple | (1, 2, 3) | no | yes | position | a frozen array | ValueTuple |
dict | {"a": 1} | yes | insertion order | key | Map | Dictionary<K, V> |
set | {1, 2, 3} | yes | no | membership only | Set | HashSet<T> |
Every snippet has a ▶ Try button. Predict the output, then run it.
Lists
A list is a growable array of references to any values. It's the default collection: use it when order matters and you append as you go.
langs = ["python", "go"]
langs.append("rust") # one item at the end
langs.extend(["c#", "ts"]) # every item of another iterable
langs.insert(0, "js") # at a position (O(n): everything shifts)
print(langs)
print(langs.pop(), langs.pop(0), langs) # pop() takes the last, pop(0) the first
print(len(langs), langs.index("go"), "go" in langs)
Indexing past the end raises IndexError. JavaScript returns undefined there, and C# throws
ArgumentOutOfRangeException, so Python behaves like C#:
xs = [10, 20, 30]
print(xs[-1], xs[-3]) # negative indexes count from the end
try:
xs[3]
except IndexError as e:
print("IndexError:", e)
Most methods that change a list return None. append, extend, insert, sort, reverse and
remove all work in place and return None (pop is the exception: it returns the item), so
xs = xs.sort() throws your list away. The built-in sorted(xs) returns a new
list instead.
nums = [3, 1, 2]
result = nums.sort()
print(result, nums)
print(sorted([3, 1, 2]), [3, 1, 2])
+ builds a new list, * repeats one, and == compares contents item by item (JavaScript's === on two
arrays compares identity, so [1] === [1] is false there):
print([1, 2] + [3], [0] * 3, [1, 2] == [1, 2])
Slicing and slice assignment
xs[start:stop:step] works on every sequence and always returns a new list, so xs[:] is a quick
shallow copy. stop is excluded, any part can be left out, and slices clamp out-of-range bounds instead of
raising:
xs = [0, 1, 2, 3, 4, 5, 6]
print(xs[2:5], xs[:2], xs[-2:])
print(xs[::2], xs[::-1])
print(xs[5:100], xs[100:])
A start-stop slice on the left of = replaces that part of the list in place, and the replacement can
have a different length (a slice with a step needs the same length). That's how you insert, delete or splice without index arithmetic:
xs = [0, 1, 2, 3, 4]
xs[1:3] = ["a", "b", "c"] # replace two items with three
print(xs)
xs[1:1] = ["inserted"] # an empty slice inserts
print(xs)
xs[-2:] = [] # assigning nothing deletes
print(xs)
del xs[::2] # del works with slices too
print(xs)
With a step, the lengths must match (xs[::2] = [1, 2] on a five-item list raises ValueError).
xs[:] = new_items replaces the contents of the list while keeping the same object. It matters when
other code holds a reference to that list, as you'll see under aliasing.
Tuples
A tuple is an immutable sequence. Use it for a fixed group of values that belong together: a coordinate, a
row from a database, the several values a function returns. It's the closest thing to C#'s (int, string)
value tuples, but it can't be changed after it's created.
point = (3, 4)
row = ("ann", 31, "admin")
single = ("only",) # the comma makes the tuple, not the parentheses
not_a_tuple = ("only")
print(type(single).__name__, type(not_a_tuple).__name__)
print(point[0], len(row), row.count("ann"))
try:
point[0] = 10
except TypeError as e:
print("TypeError:", e)
The parentheses are optional in most places: return total, count returns a tuple, and x, y = y, x
builds one and unpacks it.
Immutable means the tuple's slots always point at the same objects. If one of those objects is a list, the list can still change:
team = ("core", ["ann", "bo"])
team[1].append("cy")
print(team)
Unpacking
Unpacking assigns the items of any iterable to several names at once. The number of names must match the
number of items, or it raises ValueError. A starred name collects "the rest" into a list:
x, y = (3, 4)
first, *rest = [1, 2, 3, 4]
*init, last = "abcd"
head, *middle, tail = [1, 2]
print(x, y, first, rest)
print(init, last)
print(head, middle, tail)
middle is an empty list there, not an error: a starred name can match zero items. Unpacking also nests,
and works in for loops, which is the clean way to walk a list of pairs:
records = [("ann", (51.5, -0.1)), ("bo", (40.7, -74.0))]
for name, (lat, lon) in records:
print(name, lat, lon)
try:
a, b = [1, 2, 3]
except ValueError as e:
print("ValueError:", e)
By convention _ names a value you don't need: _, ext = name.rsplit(".", 1).
The star also spreads, like JavaScript's .... * spreads an iterable into a list, tuple, set or
call, and ** spreads a dict into a dict or into keyword arguments:
a, b = [1, 2], (3, 4)
print([*a, *b, 5])
print(max(*a, *b))
defaults = {"retries": 3, "timeout": 10}
overrides = {"timeout": 30}
print({**defaults, **overrides})
print(defaults | overrides) # the same merge with the | operator; the right side wins
Dicts
A dict maps hashable keys to values. It's Python's Map or Dictionary<K, V>, and it's also how most
JSON looks once it's parsed.
user = {"name": "ann", "roles": ["admin"]}
user["email"] = "ann@example.com" # add or replace
print(user["name"], len(user), "email" in user)
print(user.get("phone"), user.get("phone", "n/a"))
try:
user["phone"]
except KeyError as e:
print("KeyError:", e)
A missing key raises KeyError, like C#'s KeyNotFoundException. JavaScript would quietly return
undefined. d.get(key, default) is the lookup that doesn't raise; without a default it returns None.
in checks the keys, never the values.
Keys can be any hashable value, not just strings. In JavaScript, obj[1] and obj["1"] are the same
property. In Python they're different keys. And because True == 1 and they hash the same, they are the
same key:
d = {1: "int", "1": "str", (0, 0): "tuple"}
print(d[1], d["1"], d[(0, 0)])
print({1: "a", True: "b", 1.0: "c"})
The first key written is kept and the last value wins, so that prints {1: 'c'}.
get, setdefault, and building dicts up
Two methods remove most if key in d boilerplate. get reads with a fallback, and setdefault reads a
key, inserting a default first if it's missing, and returns the stored value. That makes grouping a
one-liner:
orders = [("ann", "A1"), ("bo", "B7"), ("ann", "A2")]
counts = {}
for customer, _ in orders:
counts[customer] = counts.get(customer, 0) + 1
by_customer = {}
for customer, order_id in orders:
by_customer.setdefault(customer, []).append(order_id)
print(counts)
print(by_customer)
setdefault evaluates its default every time, even when the key exists. For an empty list that costs
nothing. For something expensive, use collections.defaultdict(list), which calls the factory only for
missing keys. collections.Counter does the counting case for you:
from collections import Counter, defaultdict
groups = defaultdict(list)
for word in ["apple", "avocado", "banana"]:
groups[word[0]].append(word)
print(dict(groups))
print(Counter("mississippi").most_common(2))
Removing keys: d.pop(key) returns the value and raises KeyError if it's missing, d.pop(key, None)
doesn't raise, and del d[key] just deletes.
Ordering and views
Since Python 3.7, a dict remembers insertion order. Iterating gives keys in the order they were first
added, and replacing a value doesn't move the key. C#'s Dictionary makes no order promise, so code
that relies on it there is a bug. In Python it's guaranteed and you can rely on it.
stock = {"pens": 10, "ink": 0, "paper": 5}
stock["pens"] = 7 # replacing keeps the position
stock["clips"] = 2 # new keys go at the end
for item, qty in stock.items():
print(item, qty)
print(list(stock), {"a": 1, "b": 2} == {"b": 2, "a": 1})
Order doesn't count for equality: two dicts are equal when they have the same keys and values.
keys(), values() and items() return views: live windows onto the dict, not copies. A view sees
later changes, and keys() and items() support set operations:
before = {"a": 1, "b": 2, "c": 3}
after = {"b": 2, "c": 30, "d": 4}
keys = before.keys()
before["z"] = 26
print(keys) # the view sees the new key
print(sorted(after.keys() - before.keys()), sorted(before.keys() & after.keys()))
Changing a dict's size while you iterate over it raises RuntimeError. Iterate over a snapshot instead,
list(d) or list(d.items()), or build a new dict:
stock = {"pens": 10, "ink": 0, "paper": 5}
try:
for item in stock:
if stock[item] == 0:
del stock[item]
except RuntimeError as e:
print("RuntimeError:", e)
for item in list(stock):
if stock[item] == 0:
del stock[item]
print(stock)
Sets
A set is an unordered collection of unique, hashable values. Use it when you only ask "have I seen this?" or
when you want to compare groups. {} is an empty dict, so an empty set is set().
tags = {"python", "api"}
tags.add("python") # already there: no change
tags.add("async")
tags.discard("missing") # discard ignores missing items; remove raises KeyError
print(len(tags), "api" in tags, type({}).__name__, type(set()).__name__)
print(sorted(set([3, 1, 3, 2, 1])))
The operators read like the maths. Each has a method form (a.union(b), a.issubset(b)...) that also
accepts any iterable:
backend = {"ann", "bo", "cy"}
oncall = {"bo", "di"}
print(sorted(backend | oncall)) # union: in either
print(sorted(backend & oncall)) # intersection: in both
print(sorted(backend - oncall)) # difference: in backend but not on call
print(sorted(backend ^ oncall)) # symmetric difference: in exactly one
print({"bo"} <= oncall, oncall <= backend) # subset
Sets have no order, and the order you see when you print one isn't something to rely on. That's why the
snippets above wrap them in sorted. To deduplicate while keeping the original order, use a dict, whose
keys are unique and ordered:
emails = ["b@x.io", "a@x.io", "b@x.io", "c@x.io", "a@x.io"]
print(list(dict.fromkeys(emails)))
frozenset is the immutable version of set. It's hashable, so it can be a dict key or live inside another
set.
Hashability: why a list can't be a key
A dict or set finds an item by its hash, a number computed from its value, and then checks ==. That
only works if the hash never changes while the item is stored. So mutable built-ins (list, dict, set)
are unhashable, and using one as a key raises TypeError. Immutable ones (int, str, tuple,
frozenset) are hashable, as long as everything inside them is too.
visited = {(0, 0), (0, 1)} # tuples as coordinates: fine
print((0, 1) in visited)
for key in [[0, 1], (0, [1]), {"a": 1}]:
try:
{key: "x"}
except TypeError as e:
print("TypeError:", e)
roles = {frozenset({"read", "write"}): "editor"}
print(roles[frozenset({"write", "read"})]) # order inside a frozenset doesn't matter
The fix is to convert: tuple(xs) for a list, frozenset(s) for a set, tuple(sorted(d.items())) for a
dict. Composite keys are tuples: (customer_id, date) instead of string concatenation like
f"{customer_id}:{date}", which breaks the day an id contains a colon.
Instances of your own classes are hashable by default, by identity: two objects with equal fields are
different keys. A frozen dataclass (@dataclass(frozen=True)) hashes by value, which is usually what you
want for a key. The Dataclasses, Enums, Pattern Matching and Type Hints topic covers it.
Membership cost: list versus set
x in some_list checks items one by one, so it's O(n). x in some_set and key in some_dict hash x and
jump to the right bucket, so they're O(1) on average. This snippet counts the == comparisons each lookup
makes:
class Tracked:
comparisons = 0
def __init__(self, n: int):
self.n = n
def __eq__(self, other: object) -> bool:
Tracked.comparisons += 1
return isinstance(other, Tracked) and self.n == other.n
def __hash__(self) -> int:
return hash(self.n)
items = [Tracked(i) for i in range(10_000)]
lookup = set(items)
target = Tracked(9_999)
print(target in items, Tracked.comparisons)
Tracked.comparisons = 0
print(target in lookup, Tracked.comparisons)
Ten thousand comparisons against one. Inside a loop that becomes O(n²) against O(n), the most common performance problem in interview solutions and in code review:
| Operation | list | set / dict |
|---|---|---|
x in c | O(n) | O(1) average |
| append / add | O(1) amortised | O(1) average |
insert(0, x) / pop(0) | O(n) | n/a |
index xs[i] | O(1) | n/a (dict: d[key] is O(1)) |
| remove a value | O(n) | O(1) average |
If you need a queue that pops from the front, collections.deque does popleft() in O(1).
Mutability and aliasing
Assignment never copies. It binds another name to the same object, and passing an argument to a function does the same. With immutable values that's invisible. With a list or a dict, a change through one name shows through every other:
def add_admin(roles: list[str]) -> list[str]:
roles.append("admin") # changes the caller's list
return roles
mine = ["viewer"]
theirs = add_admin(mine)
print(mine, theirs is mine)
+= on a list is extend, in place. x = x + [...] builds a new list and rebinds only x. The
difference shows when two names share the list:
a = [1]
b = a
a += [2] # in place: b sees it
print(b)
a = a + [3] # new list: b doesn't
print(a, b)
Shallow and deep copies
list(xs), xs[:], xs.copy(), dict(d), d.copy() and copy.copy(x) are shallow copies: a new
outer container whose slots point at the same inner objects. copy.deepcopy(x) copies everything,
all the way down.
import copy
config = {"name": "api", "ports": [80, 443]}
shallow = config.copy()
deep = copy.deepcopy(config)
shallow["name"] = "web" # top-level key: independent
shallow["ports"].append(8080) # shared inner list: both see it
print(config)
print(deep)
Spread syntax in JavaScript ({...config}) is shallow in the same way. Deep copies cost time and memory,
so copy at the boundary where you need independence, not by reflex.
The [[0] * 3] * 3 trap
* on a list repeats references. For numbers that's harmless. For a list of lists, every row is the
same list:
grid = [[0] * 3] * 3
grid[0][0] = 1
print(grid)
print(grid[0] is grid[1])
grid = [[0] * 3 for _ in range(3)] # a new inner list each time
grid[0][0] = 1
print(grid)
The comprehension evaluates [0] * 3 once per row, so each row is its own list. JavaScript has the same trap
with new Array(3).fill([]).
The same mistake hides in default arguments: def f(items=[]) creates the list once, when the function
is defined, and every call shares it. Default to None and create the list inside. The Functions and
Closures topic shows it in detail.
Choosing a collection
- A sequence you build up or keep in order:
list. - A fixed record, a composite key, or several return values:
tuple. - Look something up by a key, or count and group:
dict(andCounter/defaultdict). - Membership tests, deduplication, comparing groups:
set. - Sharing data you don't want changed: a
tupleorfrozenset, or hand out a copy.
The exercises use each of them on jobs you'll see at work: paging results, deduplicating, diffing, grouping, merging configuration and finding duplicate rows.
Practise this lesson
In the app: a short video, 7 exercises checked by real tests and a 12-question concept check. Free with a GitHub account.
Open in the app