Python · Lesson 2

Collections

Lists, tuples, dicts and sets, slicing and unpacking, aliasing and shallow versus deep copies, dict ordering and views, hashability, and why membership in a set is fast while membership in a list is not.

13 min read

In short

  • Pick the collection by the question you ask
  • Slices copy and never raise; *rest collects a list
  • get and setdefault; dicts keep insertion order
  • Use a set for in; keys must be hashable
  • Assignment shares; copy shallow or deep on purpose

Almost every Python program is lists and dicts passed around between functions. API payloads arrive as dicts of lists, a database query returns a list of tuples, and the "business logic" in the middle is grouping, deduplicating and looking things up. Python gives you four built-in collections, and picking the right one, then knowing whether you are holding a copy or a second name for the same object, prevents most of the bugs.

LiteralMutableOrderedLookup byJavaScriptC#
list[1, 2, 3]yesyespositionArrayList<T>
tuple(1, 2, 3)noyespositiona frozen arrayValueTuple
dict{"a": 1}yesinsertion orderkeyMapDictionary<K, V>
set{1, 2, 3}yesnomembership onlySetHashSet<T>

Every snippet has a ▶ Try button. Predict the output, then run it.

Lists

A list is a growable array of references to any values. It's the default collection: use it when order matters and you append as you go.

▶ Try
langs = ["python", "go"]
langs.append("rust")          # one item at the end
langs.extend(["c#", "ts"])    # every item of another iterable
langs.insert(0, "js")         # at a position (O(n): everything shifts)
print(langs)
print(langs.pop(), langs.pop(0), langs)   # pop() takes the last, pop(0) the first
print(len(langs), langs.index("go"), "go" in langs)

Indexing past the end raises IndexError. JavaScript returns undefined there, and C# throws ArgumentOutOfRangeException, so Python behaves like C#:

▶ Try
xs = [10, 20, 30]
print(xs[-1], xs[-3])     # negative indexes count from the end
try:
    xs[3]
except IndexError as e:
    print("IndexError:", e)

Most methods that change a list return None. append, extend, insert, sort, reverse and remove all work in place and return None (pop is the exception: it returns the item), so xs = xs.sort() throws your list away. The built-in sorted(xs) returns a new list instead.

▶ Try
nums = [3, 1, 2]
result = nums.sort()
print(result, nums)
print(sorted([3, 1, 2]), [3, 1, 2])

+ builds a new list, * repeats one, and == compares contents item by item (JavaScript's === on two arrays compares identity, so [1] === [1] is false there):

▶ Try
print([1, 2] + [3], [0] * 3, [1, 2] == [1, 2])

Slicing and slice assignment

xs[start:stop:step] works on every sequence and always returns a new list, so xs[:] is a quick shallow copy. stop is excluded, any part can be left out, and slices clamp out-of-range bounds instead of raising:

▶ Try
xs = [0, 1, 2, 3, 4, 5, 6]
print(xs[2:5], xs[:2], xs[-2:])
print(xs[::2], xs[::-1])
print(xs[5:100], xs[100:])

A start-stop slice on the left of = replaces that part of the list in place, and the replacement can have a different length (a slice with a step needs the same length). That's how you insert, delete or splice without index arithmetic:

▶ Try
xs = [0, 1, 2, 3, 4]
xs[1:3] = ["a", "b", "c"]   # replace two items with three
print(xs)
xs[1:1] = ["inserted"]      # an empty slice inserts
print(xs)
xs[-2:] = []                # assigning nothing deletes
print(xs)
del xs[::2]                 # del works with slices too
print(xs)

With a step, the lengths must match (xs[::2] = [1, 2] on a five-item list raises ValueError).

xs[:] = new_items replaces the contents of the list while keeping the same object. It matters when other code holds a reference to that list, as you'll see under aliasing.

Tuples

A tuple is an immutable sequence. Use it for a fixed group of values that belong together: a coordinate, a row from a database, the several values a function returns. It's the closest thing to C#'s (int, string) value tuples, but it can't be changed after it's created.

▶ Try
point = (3, 4)
row = ("ann", 31, "admin")
single = ("only",)        # the comma makes the tuple, not the parentheses
not_a_tuple = ("only")
print(type(single).__name__, type(not_a_tuple).__name__)
print(point[0], len(row), row.count("ann"))
try:
    point[0] = 10
except TypeError as e:
    print("TypeError:", e)

The parentheses are optional in most places: return total, count returns a tuple, and x, y = y, x builds one and unpacks it.

Immutable means the tuple's slots always point at the same objects. If one of those objects is a list, the list can still change:

▶ Try
team = ("core", ["ann", "bo"])
team[1].append("cy")
print(team)

Unpacking

Unpacking assigns the items of any iterable to several names at once. The number of names must match the number of items, or it raises ValueError. A starred name collects "the rest" into a list:

▶ Try
x, y = (3, 4)
first, *rest = [1, 2, 3, 4]
*init, last = "abcd"
head, *middle, tail = [1, 2]
print(x, y, first, rest)
print(init, last)
print(head, middle, tail)

middle is an empty list there, not an error: a starred name can match zero items. Unpacking also nests, and works in for loops, which is the clean way to walk a list of pairs:

▶ Try
records = [("ann", (51.5, -0.1)), ("bo", (40.7, -74.0))]
for name, (lat, lon) in records:
    print(name, lat, lon)

try:
    a, b = [1, 2, 3]
except ValueError as e:
    print("ValueError:", e)

By convention _ names a value you don't need: _, ext = name.rsplit(".", 1).

The star also spreads, like JavaScript's .... * spreads an iterable into a list, tuple, set or call, and ** spreads a dict into a dict or into keyword arguments:

▶ Try
a, b = [1, 2], (3, 4)
print([*a, *b, 5])
print(max(*a, *b))

defaults = {"retries": 3, "timeout": 10}
overrides = {"timeout": 30}
print({**defaults, **overrides})
print(defaults | overrides)       # the same merge with the | operator; the right side wins

Dicts

A dict maps hashable keys to values. It's Python's Map or Dictionary<K, V>, and it's also how most JSON looks once it's parsed.

▶ Try
user = {"name": "ann", "roles": ["admin"]}
user["email"] = "ann@example.com"      # add or replace
print(user["name"], len(user), "email" in user)
print(user.get("phone"), user.get("phone", "n/a"))
try:
    user["phone"]
except KeyError as e:
    print("KeyError:", e)

A missing key raises KeyError, like C#'s KeyNotFoundException. JavaScript would quietly return undefined. d.get(key, default) is the lookup that doesn't raise; without a default it returns None. in checks the keys, never the values.

Keys can be any hashable value, not just strings. In JavaScript, obj[1] and obj["1"] are the same property. In Python they're different keys. And because True == 1 and they hash the same, they are the same key:

▶ Try
d = {1: "int", "1": "str", (0, 0): "tuple"}
print(d[1], d["1"], d[(0, 0)])
print({1: "a", True: "b", 1.0: "c"})

The first key written is kept and the last value wins, so that prints {1: 'c'}.

get, setdefault, and building dicts up

Two methods remove most if key in d boilerplate. get reads with a fallback, and setdefault reads a key, inserting a default first if it's missing, and returns the stored value. That makes grouping a one-liner:

▶ Try
orders = [("ann", "A1"), ("bo", "B7"), ("ann", "A2")]

counts = {}
for customer, _ in orders:
    counts[customer] = counts.get(customer, 0) + 1

by_customer = {}
for customer, order_id in orders:
    by_customer.setdefault(customer, []).append(order_id)

print(counts)
print(by_customer)

setdefault evaluates its default every time, even when the key exists. For an empty list that costs nothing. For something expensive, use collections.defaultdict(list), which calls the factory only for missing keys. collections.Counter does the counting case for you:

▶ Try
from collections import Counter, defaultdict

groups = defaultdict(list)
for word in ["apple", "avocado", "banana"]:
    groups[word[0]].append(word)
print(dict(groups))
print(Counter("mississippi").most_common(2))

Removing keys: d.pop(key) returns the value and raises KeyError if it's missing, d.pop(key, None) doesn't raise, and del d[key] just deletes.

Ordering and views

Since Python 3.7, a dict remembers insertion order. Iterating gives keys in the order they were first added, and replacing a value doesn't move the key. C#'s Dictionary makes no order promise, so code that relies on it there is a bug. In Python it's guaranteed and you can rely on it.

▶ Try
stock = {"pens": 10, "ink": 0, "paper": 5}
stock["pens"] = 7                 # replacing keeps the position
stock["clips"] = 2                # new keys go at the end
for item, qty in stock.items():
    print(item, qty)
print(list(stock), {"a": 1, "b": 2} == {"b": 2, "a": 1})

Order doesn't count for equality: two dicts are equal when they have the same keys and values.

keys(), values() and items() return views: live windows onto the dict, not copies. A view sees later changes, and keys() and items() support set operations:

▶ Try
before = {"a": 1, "b": 2, "c": 3}
after = {"b": 2, "c": 30, "d": 4}
keys = before.keys()
before["z"] = 26
print(keys)                        # the view sees the new key
print(sorted(after.keys() - before.keys()), sorted(before.keys() & after.keys()))

Changing a dict's size while you iterate over it raises RuntimeError. Iterate over a snapshot instead, list(d) or list(d.items()), or build a new dict:

▶ Try
stock = {"pens": 10, "ink": 0, "paper": 5}
try:
    for item in stock:
        if stock[item] == 0:
            del stock[item]
except RuntimeError as e:
    print("RuntimeError:", e)

for item in list(stock):
    if stock[item] == 0:
        del stock[item]
print(stock)

Sets

A set is an unordered collection of unique, hashable values. Use it when you only ask "have I seen this?" or when you want to compare groups. {} is an empty dict, so an empty set is set().

▶ Try
tags = {"python", "api"}
tags.add("python")            # already there: no change
tags.add("async")
tags.discard("missing")       # discard ignores missing items; remove raises KeyError
print(len(tags), "api" in tags, type({}).__name__, type(set()).__name__)
print(sorted(set([3, 1, 3, 2, 1])))

The operators read like the maths. Each has a method form (a.union(b), a.issubset(b)...) that also accepts any iterable:

▶ Try
backend = {"ann", "bo", "cy"}
oncall = {"bo", "di"}
print(sorted(backend | oncall))   # union: in either
print(sorted(backend & oncall))   # intersection: in both
print(sorted(backend - oncall))   # difference: in backend but not on call
print(sorted(backend ^ oncall))   # symmetric difference: in exactly one
print({"bo"} <= oncall, oncall <= backend)   # subset

Sets have no order, and the order you see when you print one isn't something to rely on. That's why the snippets above wrap them in sorted. To deduplicate while keeping the original order, use a dict, whose keys are unique and ordered:

▶ Try
emails = ["b@x.io", "a@x.io", "b@x.io", "c@x.io", "a@x.io"]
print(list(dict.fromkeys(emails)))

frozenset is the immutable version of set. It's hashable, so it can be a dict key or live inside another set.

Hashability: why a list can't be a key

A dict or set finds an item by its hash, a number computed from its value, and then checks ==. That only works if the hash never changes while the item is stored. So mutable built-ins (list, dict, set) are unhashable, and using one as a key raises TypeError. Immutable ones (int, str, tuple, frozenset) are hashable, as long as everything inside them is too.

▶ Try
visited = {(0, 0), (0, 1)}            # tuples as coordinates: fine
print((0, 1) in visited)

for key in [[0, 1], (0, [1]), {"a": 1}]:
    try:
        {key: "x"}
    except TypeError as e:
        print("TypeError:", e)

roles = {frozenset({"read", "write"}): "editor"}
print(roles[frozenset({"write", "read"})])   # order inside a frozenset doesn't matter

The fix is to convert: tuple(xs) for a list, frozenset(s) for a set, tuple(sorted(d.items())) for a dict. Composite keys are tuples: (customer_id, date) instead of string concatenation like f"{customer_id}:{date}", which breaks the day an id contains a colon.

Instances of your own classes are hashable by default, by identity: two objects with equal fields are different keys. A frozen dataclass (@dataclass(frozen=True)) hashes by value, which is usually what you want for a key. The Dataclasses, Enums, Pattern Matching and Type Hints topic covers it.

Membership cost: list versus set

x in some_list checks items one by one, so it's O(n). x in some_set and key in some_dict hash x and jump to the right bucket, so they're O(1) on average. This snippet counts the == comparisons each lookup makes:

▶ Try
class Tracked:
    comparisons = 0

    def __init__(self, n: int):
        self.n = n

    def __eq__(self, other: object) -> bool:
        Tracked.comparisons += 1
        return isinstance(other, Tracked) and self.n == other.n

    def __hash__(self) -> int:
        return hash(self.n)


items = [Tracked(i) for i in range(10_000)]
lookup = set(items)
target = Tracked(9_999)

print(target in items, Tracked.comparisons)
Tracked.comparisons = 0
print(target in lookup, Tracked.comparisons)

Ten thousand comparisons against one. Inside a loop that becomes O(n²) against O(n), the most common performance problem in interview solutions and in code review:

Operationlistset / dict
x in cO(n)O(1) average
append / addO(1) amortisedO(1) average
insert(0, x) / pop(0)O(n)n/a
index xs[i]O(1)n/a (dict: d[key] is O(1))
remove a valueO(n)O(1) average

If you need a queue that pops from the front, collections.deque does popleft() in O(1).

Mutability and aliasing

Assignment never copies. It binds another name to the same object, and passing an argument to a function does the same. With immutable values that's invisible. With a list or a dict, a change through one name shows through every other:

▶ Try
def add_admin(roles: list[str]) -> list[str]:
    roles.append("admin")        # changes the caller's list
    return roles


mine = ["viewer"]
theirs = add_admin(mine)
print(mine, theirs is mine)

+= on a list is extend, in place. x = x + [...] builds a new list and rebinds only x. The difference shows when two names share the list:

▶ Try
a = [1]
b = a
a += [2]          # in place: b sees it
print(b)
a = a + [3]       # new list: b doesn't
print(a, b)

Shallow and deep copies

list(xs), xs[:], xs.copy(), dict(d), d.copy() and copy.copy(x) are shallow copies: a new outer container whose slots point at the same inner objects. copy.deepcopy(x) copies everything, all the way down.

▶ Try
import copy

config = {"name": "api", "ports": [80, 443]}
shallow = config.copy()
deep = copy.deepcopy(config)

shallow["name"] = "web"          # top-level key: independent
shallow["ports"].append(8080)    # shared inner list: both see it
print(config)
print(deep)

Spread syntax in JavaScript ({...config}) is shallow in the same way. Deep copies cost time and memory, so copy at the boundary where you need independence, not by reflex.

The [[0] * 3] * 3 trap

* on a list repeats references. For numbers that's harmless. For a list of lists, every row is the same list:

▶ Try
grid = [[0] * 3] * 3
grid[0][0] = 1
print(grid)
print(grid[0] is grid[1])

grid = [[0] * 3 for _ in range(3)]   # a new inner list each time
grid[0][0] = 1
print(grid)

The comprehension evaluates [0] * 3 once per row, so each row is its own list. JavaScript has the same trap with new Array(3).fill([]).

The same mistake hides in default arguments: def f(items=[]) creates the list once, when the function is defined, and every call shares it. Default to None and create the list inside. The Functions and Closures topic shows it in detail.

Choosing a collection

  • A sequence you build up or keep in order: list.
  • A fixed record, a composite key, or several return values: tuple.
  • Look something up by a key, or count and group: dict (and Counter / defaultdict).
  • Membership tests, deduplication, comparing groups: set.
  • Sharing data you don't want changed: a tuple or frozenset, or hand out a copy.

The exercises use each of them on jobs you'll see at work: paging results, deduplicating, diffing, grouping, merging configuration and finding duplicate rows.

Practise this lesson

In the app: a short video, 7 exercises checked by real tests and a 12-question concept check. Free with a GitHub account.

Open in the app