Python · Lesson 4
Functions and Closures
Functions as values, every way to pass arguments (defaults, *args, **kwargs, keyword-only and positional-only), the mutable default trap, LEGB scope with global and nonlocal, closures and late binding, higher-order functions and functools.partial.
In short
- Functions are objects: pass, store, return them
- Defaults are evaluated once: use
Nonefor lists and dicts *argsis a tuple,**kwargsa dict;*and/shape the signature- LEGB scope;
nonlocalandglobalto rebind - Closures capture variables: bind loop values with a factory or
partial
In Python a function is an object, like an int or a list. You can store it in a variable, put it in a dict, pass it to another function and return it from one. A function defined inside another function also remembers the variables around it, which makes it a closure. Those two facts give you callbacks, factories, decorators, retry wrappers and caches.
Python's argument passing is richer than JavaScript's or C#'s, and stricter. A call with a missing or an
unexpected argument is a TypeError, not a silent undefined. In exchange you get keyword arguments,
keyword-only and positional-only parameters, and unpacking at the call site, which together let you
design signatures that are hard to call wrongly.
Every python snippet has a ▶ Try button. Predict the output, then run it.
Functions are values
def creates a function object and binds it to a name. The name is an ordinary variable:
def shout(text: str) -> str:
return text.upper() + "!"
yell = shout # another name for the same function object
print(yell("hi"), yell is shout)
print(shout.__name__, type(shout).__name__)
handlers = {"loud": shout, "quiet": str.lower}
for name, handler in handlers.items():
print(name, handler("Hello"))
def is a statement that runs when Python reaches it, so a function doesn't exist before its def line
has executed. There's no hoisting as in JavaScript. Calling a function from inside another one is fine,
because the lookup happens at call time.
A function without return (or with a bare return) returns None. To return several values, return a
tuple and unpack it:
def min_max(xs: list[int]) -> tuple[int, int]:
return min(xs), max(xs)
low, high = min_max([4, 9, 1, 7])
print(low, high)
Positional and keyword arguments
Any parameter can be passed by position or by name (unless you restrict it, see below). Keyword arguments make call sites readable and order-independent:
def transfer(amount: int, source: str, target: str) -> str:
return f"{amount} from {source} to {target}"
print(transfer(50, "checking", "savings"))
print(transfer(50, target="savings", source="checking"))
The rules: positional arguments come first, then keyword arguments, and each parameter gets exactly one
value. Breaking them is a TypeError, raised before the function body runs:
def transfer(amount: int, source: str, target: str) -> str:
return f"{amount} from {source} to {target}"
for call in [
lambda: transfer(50, "checking"),
lambda: transfer(50, "checking", "savings", "extra"),
lambda: transfer(50, "checking", source="savings"),
lambda: transfer(50, "checking", "savings", currency="EUR"),
]:
try:
call()
except TypeError as e:
print("TypeError:", e)
Compare that with JavaScript, where a missing argument is undefined and an extra one is ignored. In
Python both are errors, which catches a whole class of bugs at the first call.
Default values, and the mutable default trap
A parameter with a default can be left out. Parameters with defaults must come after the ones without:
def greet(name: str, greeting: str = "Hello", punctuation: str = "!") -> str:
return f"{greeting}, {name}{punctuation}"
print(greet("Ann"))
print(greet("Ann", punctuation="?"))
Here's the rule that catches everyone once: a default is evaluated once, when the def runs, and the
same object is reused on every call. For an immutable default (a number, a string, None, a tuple) that's
harmless. For a list or a dict it means every call shares, and mutates, one object:
def add_tag(tag: str, tags: list[str] = []) -> list[str]:
tags.append(tag)
return tags
print(add_tag("urgent"))
print(add_tag("billing")) # the same list as the first call
print(add_tag.__defaults__) # the shared default lives on the function
JavaScript evaluates defaults on every call, and C# only allows compile-time constants, so a shared mutable
default can't happen there. Code ported from either hits this bug. The fix is the
None sentinel: default to None and create the fresh object inside the body.
def add_tag(tag: str, tags: list[str] | None = None) -> list[str]:
if tags is None:
tags = []
tags.append(tag)
return tags
print(add_tag("urgent"))
print(add_tag("billing"))
The same applies to any default computed from the outside world, like created=datetime.now(): it's the
time the module was loaded, not the time of the call.
When None is itself a meaningful value (for example "set this field to null"), use a private sentinel
object to mean "not passed":
_MISSING = object()
def update(record: dict, *, email: object = _MISSING) -> dict:
if email is not _MISSING:
record["email"] = email # None is a real value here: clear the email
return record
print(update({"email": "a@x.io"}))
print(update({"email": "a@x.io"}, email=None))
*args and **kwargs
*args collects extra positional arguments into a tuple. **kwargs collects extra keyword
arguments into a dict, in the order they were passed. The names args and kwargs are only a
convention; the stars do the work.
def log(level: str, *parts: object, **fields: object) -> str:
message = " ".join(str(p) for p in parts)
extra = " ".join(f"{k}={v}" for k, v in fields.items())
return f"[{level}] {message} {extra}".rstrip()
print(log("INFO", "user", 42, "logged in", ip="10.0.0.1", mfa=True))
print(log("WARN"))
def show(*args, **kwargs):
print(type(args).__name__, args, type(kwargs).__name__, kwargs)
show(1, 2, x=3)
This is how wrappers forward "whatever they were called with" to another function without knowing its signature, which you'll use for retry and caching below.
Keyword-only and positional-only parameters
A bare * in the parameter list means "everything after this must be passed by name". A / means
"everything before this must be passed by position".
def paginate(items: list, /, *, page: int = 1, per_page: int = 10) -> list:
start = (page - 1) * per_page
return items[start : start + per_page]
data = list(range(25))
print(paginate(data, page=3))
try:
paginate(data, 3) # page can't be passed by position
except TypeError as e:
print("TypeError:", e)
try:
paginate(items=data) # items can't be passed by name
except TypeError as e:
print("TypeError:", e)
Keyword-only parameters are the important ones in everyday code. send_email(to, subject, True, False)
is unreadable; send_email(to, subject, html=True, track=False) isn't, and making html and track
keyword-only forces callers to write it that way. It also lets you add or reorder those options later
without breaking anyone. Positional-only parameters are for when the name means nothing to the caller
(len(obj), abs(x), divmod(a, b)) or when you want to keep the freedom to rename it.
The full order of a signature is:
| Part | Example | Passed as |
|---|---|---|
| Positional-only | a, b, / | position only |
| Normal | c, d=1 | position or keyword |
| Extra positionals | *args | collected into a tuple |
| Keyword-only | *, e, f=2 (or after *args) | keyword only |
| Extra keywords | **kwargs | collected into a dict |
def everything(a, b, /, c, d=1, *args, e, f=2, **kwargs):
print(a, b, c, d, args, e, f, kwargs)
everything(1, 2, 3, 4, 5, 6, e=7, g=8)
A keyword-only parameter without a default (e above) is required. That's how you write "you must say
which one you mean" APIs.
Unpacking into a call
The stars also work at the call site: *iterable spreads items as positional arguments and
**mapping spreads key-value pairs as keyword arguments. It's JavaScript's spread, plus a version for
named arguments.
def connect(host: str, port: int, *, timeout: float = 5.0, tls: bool = False) -> str:
return f"{host}:{port} timeout={timeout} tls={tls}"
address = ("db.internal", 5432)
options = {"timeout": 2.5, "tls": True}
print(connect(*address, **options))
defaults = {"timeout": 5.0, "tls": False}
overrides = {"tls": True}
print({**defaults, **overrides}) # later keys win: the usual way to merge options
Passing the same name twice is still an error: connect(*address, timeout=1, **options) raises
TypeError ("got multiple values for keyword argument 'timeout'"). Merge the dicts first if you mean
"override".
Lambdas
lambda params: expression makes a small anonymous function. The body is one expression, and its value
is returned. No statements, no assignments (except :=), no annotations.
orders = [("ann", 30), ("bob", 120), ("cy", 75)]
print(sorted(orders, key=lambda order: order[1], reverse=True))
print(max(orders, key=lambda order: order[1])[0])
square = lambda n: n * n # works, but PEP 8 says: use def for named functions
print(square(7), square.__name__)
Use lambdas inline, as the key of sorted, min, max and friends, or as a short callback. If you're
naming it, write a def: it gets a real name in tracebacks, a docstring, and type hints. Unlike an arrow
function, a lambda can't contain a block, so multi-step logic always becomes a def.
Scope: LEGB
When Python looks up a name, it searches four scopes in order, and stops at the first hit:
| Scope | What lives there | |
|---|---|---|
| L | Local | names assigned in the current function (including its parameters) |
| E | Enclosing | locals of the functions this one is nested in, inner to outer |
| G | Global | names assigned at the top level of the module |
| B | Built-in | len, print, ValueError... |
rate = 0.2 # global
def outer() -> None:
rate = 0.1 # enclosing, for inner()
def inner() -> None:
print("inner sees", rate)
inner()
outer()
print("global is still", rate)
Two differences from JavaScript and C# matter a lot:
- Only functions (and classes, modules, comprehensions) create scopes.
if,for,while,withandtryblocks don't. A variable assigned inside a loop is still there after it, and so is the loop variable. - Assigning to a name anywhere in a function makes it local to the whole function. Python decides
this when it compiles the function, not line by line. So reading the name before the assignment fails
with
UnboundLocalError, even if a global of the same name exists.
for i in range(3):
last = i * 10
print(i, last) # both survive the loop
count = 0
def increment() -> None:
try:
count += 1 # an assignment: count is local to increment
except UnboundLocalError as e:
print("UnboundLocalError:", e)
increment()
Reading a global from a function is fine. Rebinding it needs a declaration.
global and nonlocal
global name makes assignments in the function rebind the module-level name. nonlocal name makes
them rebind the name in the nearest enclosing function.
calls = 0
def tracked() -> None:
global calls
calls += 1
tracked()
tracked()
print(calls)
def make_counter():
count = 0
def increment() -> int:
nonlocal count
count += 1
return count
return increment
counter = make_counter()
print(counter(), counter(), counter())
You need neither to mutate an object: items.append(x) or cache[key] = value doesn't rebind the
name, so it works without a declaration. You need them only for name = ..., name += ... and friends.
In real code, global is a smell (state that every caller shares and tests have to reset); nonlocal in
a small closure is normal.
Closures
A nested function that uses variables from its enclosing function keeps them alive after the enclosing function returns. Each call of the outer function creates new variables, so each closure gets its own state:
def make_multiplier(factor: int):
def multiply(n: int) -> int:
return n * factor
return multiply
double = make_multiplier(2)
triple = make_multiplier(3)
print(double(10), triple(10))
def make_account(balance: int):
def deposit(amount: int) -> int:
nonlocal balance
balance += amount
return balance
def get_balance() -> int:
return balance
return deposit, get_balance
deposit, get_balance = make_account(100)
deposit(50)
print(get_balance()) # both closures share the same balance variable
The closure captures the variable, not its value at the time. If the variable changes later, the
closure sees the new value. That's how deposit and get_balance stay in sync, and it's also the cause
of the most famous closure bug.
Late binding in loops
callbacks = []
for i in range(3):
callbacks.append(lambda: i)
print([f() for f in callbacks])
All three lambdas print 2. They don't store i's value; they look up the variable i when they're
called, and by then the loop has finished and i is 2. The loop doesn't create a new i per
iteration. JavaScript's for (let i = ...) does, which is why the same code works there (and why the
var version of it is a classic interview question).
Three fixes, all of which bind the current value at the time the function is created:
from functools import partial
# 1. A default argument: evaluated once, when the lambda is created
fixed_default = [lambda i=i: i for i in range(3)]
# 2. A factory function: each call creates a new scope with its own n
def make_getter(n: int):
return lambda: n
fixed_factory = [make_getter(i) for i in range(3)]
# 3. functools.partial: the argument is stored when the partial is made
def identity(n: int) -> int:
return n
fixed_partial = [partial(identity, i) for i in range(3)]
print([f() for f in fixed_default], [f() for f in fixed_factory], [f() for f in fixed_partial])
The default argument trick is the shortest and the one you'll see most, but it adds a parameter a caller
could override by accident. The factory is the clearest. The bug is not limited to lambdas: a def
inside a loop behaves the same, and so do callbacks registered in a loop (button handlers, route
handlers, per-currency formatters).
Higher-order functions
A higher-order function takes a function, returns one, or both. You've used sorted(key=...) already.
map and filter exist, but comprehensions usually read better in Python:
prices = [12.5, 3.0, 99.99, 45.0]
print(list(map(lambda p: round(p * 1.2, 2), prices)))
print([round(p * 1.2, 2) for p in prices]) # the same, more Pythonic
print([p for p in prices if p > 10])
print(sorted(["b", "C", "a"], key=str.lower)) # pass an existing function as the key
Returning a function is how you build behaviour from configuration. A wrapper that takes *args and
**kwargs can forward any call to the function it wraps:
def with_logging(func):
def wrapper(*args, **kwargs):
print(f"calling {func.__name__} with {args} {kwargs}")
result = func(*args, **kwargs)
print(f"{func.__name__} returned {result}")
return result
return wrapper
def add(a: int, b: int) -> int:
return a + b
logged_add = with_logging(add)
logged_add(2, b=3)
That pattern (a function that takes a function and returns a wrapped one) is a decorator, and
@with_logging above a def is shorthand for add = with_logging(add). The Decorators topic covers
them properly, including functools.wraps.
Type hints for callables use Callable[[ArgTypes], ReturnType] from collections.abc (or typing):
from collections.abc import Callable
def apply_twice(func: Callable[[int], int], value: int) -> int:
return func(func(value))
print(apply_twice(lambda n: n + 10, 1))
functools.partial
partial(func, *args, **kwargs) returns a new callable with some arguments filled in. Calling it adds the
rest:
from functools import partial
def format_price(amount: float, currency: str, decimals: int = 2) -> str:
return f"{amount:,.{decimals}f} {currency}"
eur = partial(format_price, currency="EUR")
yen = partial(format_price, currency="JPY", decimals=0)
print(eur(1234.5), "|", yen(1234.5))
print(eur(99, decimals=0)) # keywords can still be overridden
print(eur.func.__name__, eur.keywords)
to_int_base2 = partial(int, base=2)
print(to_int_base2("1011"))
A lambda can do the same job (lambda amount: format_price(amount, "EUR")), with one difference:
partial stores the values now, while the lambda looks its variables up later. That's why partial
is a clean fix for late binding. Partials are also easy to inspect (.func, .args, .keywords), which
helps in logs and tests.
What interviewers ask
| Question | Short answer |
|---|---|
Why is def f(x=[]) a bug? | The default is created once at definition time and shared by every call. Use None and create it inside. |
What do *args and **kwargs hold? | A tuple of extra positional arguments, and a dict of extra keyword arguments. |
What do a bare * and a / in a signature do? | Parameters after * are keyword-only; parameters before / are positional-only. |
| What's LEGB? | The lookup order: local, enclosing, global, built-in. |
Why UnboundLocalError? | An assignment anywhere in the function makes the name local everywhere in it. |
global vs nonlocal? | Rebind a module-level name vs a name in the enclosing function. Neither is needed to mutate. |
Why does [lambda: i for i in range(3)] give 2, 2, 2? | Closures capture variables, not values, and look them up when called. Bind with i=i, a factory, or partial. |
partial vs lambda? | partial freezes argument values now and is inspectable; a lambda looks names up at call time. |
Practise this lesson
In the app: a short video, 7 exercises checked by real tests and a 12-question concept check. Free with a GitHub account.
Open in the app