Python · Lesson 4

Functions and Closures

Functions as values, every way to pass arguments (defaults, *args, **kwargs, keyword-only and positional-only), the mutable default trap, LEGB scope with global and nonlocal, closures and late binding, higher-order functions and functools.partial.

14 min read

In short

  • Functions are objects: pass, store, return them
  • Defaults are evaluated once: use None for lists and dicts
  • *args is a tuple, **kwargs a dict; * and / shape the signature
  • LEGB scope; nonlocal and global to rebind
  • Closures capture variables: bind loop values with a factory or partial

In Python a function is an object, like an int or a list. You can store it in a variable, put it in a dict, pass it to another function and return it from one. A function defined inside another function also remembers the variables around it, which makes it a closure. Those two facts give you callbacks, factories, decorators, retry wrappers and caches.

Python's argument passing is richer than JavaScript's or C#'s, and stricter. A call with a missing or an unexpected argument is a TypeError, not a silent undefined. In exchange you get keyword arguments, keyword-only and positional-only parameters, and unpacking at the call site, which together let you design signatures that are hard to call wrongly.

Every python snippet has a ▶ Try button. Predict the output, then run it.

Functions are values

def creates a function object and binds it to a name. The name is an ordinary variable:

▶ Try
def shout(text: str) -> str:
    return text.upper() + "!"


yell = shout                     # another name for the same function object
print(yell("hi"), yell is shout)
print(shout.__name__, type(shout).__name__)

handlers = {"loud": shout, "quiet": str.lower}
for name, handler in handlers.items():
    print(name, handler("Hello"))

def is a statement that runs when Python reaches it, so a function doesn't exist before its def line has executed. There's no hoisting as in JavaScript. Calling a function from inside another one is fine, because the lookup happens at call time.

A function without return (or with a bare return) returns None. To return several values, return a tuple and unpack it:

▶ Try
def min_max(xs: list[int]) -> tuple[int, int]:
    return min(xs), max(xs)


low, high = min_max([4, 9, 1, 7])
print(low, high)

Positional and keyword arguments

Any parameter can be passed by position or by name (unless you restrict it, see below). Keyword arguments make call sites readable and order-independent:

▶ Try
def transfer(amount: int, source: str, target: str) -> str:
    return f"{amount} from {source} to {target}"


print(transfer(50, "checking", "savings"))
print(transfer(50, target="savings", source="checking"))

The rules: positional arguments come first, then keyword arguments, and each parameter gets exactly one value. Breaking them is a TypeError, raised before the function body runs:

▶ Try
def transfer(amount: int, source: str, target: str) -> str:
    return f"{amount} from {source} to {target}"


for call in [
    lambda: transfer(50, "checking"),
    lambda: transfer(50, "checking", "savings", "extra"),
    lambda: transfer(50, "checking", source="savings"),
    lambda: transfer(50, "checking", "savings", currency="EUR"),
]:
    try:
        call()
    except TypeError as e:
        print("TypeError:", e)

Compare that with JavaScript, where a missing argument is undefined and an extra one is ignored. In Python both are errors, which catches a whole class of bugs at the first call.

Default values, and the mutable default trap

A parameter with a default can be left out. Parameters with defaults must come after the ones without:

▶ Try
def greet(name: str, greeting: str = "Hello", punctuation: str = "!") -> str:
    return f"{greeting}, {name}{punctuation}"


print(greet("Ann"))
print(greet("Ann", punctuation="?"))

Here's the rule that catches everyone once: a default is evaluated once, when the def runs, and the same object is reused on every call. For an immutable default (a number, a string, None, a tuple) that's harmless. For a list or a dict it means every call shares, and mutates, one object:

▶ Try
def add_tag(tag: str, tags: list[str] = []) -> list[str]:
    tags.append(tag)
    return tags


print(add_tag("urgent"))
print(add_tag("billing"))        # the same list as the first call
print(add_tag.__defaults__)      # the shared default lives on the function

JavaScript evaluates defaults on every call, and C# only allows compile-time constants, so a shared mutable default can't happen there. Code ported from either hits this bug. The fix is the None sentinel: default to None and create the fresh object inside the body.

▶ Try
def add_tag(tag: str, tags: list[str] | None = None) -> list[str]:
    if tags is None:
        tags = []
    tags.append(tag)
    return tags


print(add_tag("urgent"))
print(add_tag("billing"))

The same applies to any default computed from the outside world, like created=datetime.now(): it's the time the module was loaded, not the time of the call.

When None is itself a meaningful value (for example "set this field to null"), use a private sentinel object to mean "not passed":

▶ Try
_MISSING = object()


def update(record: dict, *, email: object = _MISSING) -> dict:
    if email is not _MISSING:
        record["email"] = email          # None is a real value here: clear the email
    return record


print(update({"email": "a@x.io"}))
print(update({"email": "a@x.io"}, email=None))

*args and **kwargs

*args collects extra positional arguments into a tuple. **kwargs collects extra keyword arguments into a dict, in the order they were passed. The names args and kwargs are only a convention; the stars do the work.

▶ Try
def log(level: str, *parts: object, **fields: object) -> str:
    message = " ".join(str(p) for p in parts)
    extra = " ".join(f"{k}={v}" for k, v in fields.items())
    return f"[{level}] {message} {extra}".rstrip()


print(log("INFO", "user", 42, "logged in", ip="10.0.0.1", mfa=True))
print(log("WARN"))


def show(*args, **kwargs):
    print(type(args).__name__, args, type(kwargs).__name__, kwargs)


show(1, 2, x=3)

This is how wrappers forward "whatever they were called with" to another function without knowing its signature, which you'll use for retry and caching below.

Keyword-only and positional-only parameters

A bare * in the parameter list means "everything after this must be passed by name". A / means "everything before this must be passed by position".

▶ Try
def paginate(items: list, /, *, page: int = 1, per_page: int = 10) -> list:
    start = (page - 1) * per_page
    return items[start : start + per_page]


data = list(range(25))
print(paginate(data, page=3))
try:
    paginate(data, 3)               # page can't be passed by position
except TypeError as e:
    print("TypeError:", e)
try:
    paginate(items=data)            # items can't be passed by name
except TypeError as e:
    print("TypeError:", e)

Keyword-only parameters are the important ones in everyday code. send_email(to, subject, True, False) is unreadable; send_email(to, subject, html=True, track=False) isn't, and making html and track keyword-only forces callers to write it that way. It also lets you add or reorder those options later without breaking anyone. Positional-only parameters are for when the name means nothing to the caller (len(obj), abs(x), divmod(a, b)) or when you want to keep the freedom to rename it.

The full order of a signature is:

PartExamplePassed as
Positional-onlya, b, /position only
Normalc, d=1position or keyword
Extra positionals*argscollected into a tuple
Keyword-only*, e, f=2 (or after *args)keyword only
Extra keywords**kwargscollected into a dict
▶ Try
def everything(a, b, /, c, d=1, *args, e, f=2, **kwargs):
    print(a, b, c, d, args, e, f, kwargs)


everything(1, 2, 3, 4, 5, 6, e=7, g=8)

A keyword-only parameter without a default (e above) is required. That's how you write "you must say which one you mean" APIs.

Unpacking into a call

The stars also work at the call site: *iterable spreads items as positional arguments and **mapping spreads key-value pairs as keyword arguments. It's JavaScript's spread, plus a version for named arguments.

▶ Try
def connect(host: str, port: int, *, timeout: float = 5.0, tls: bool = False) -> str:
    return f"{host}:{port} timeout={timeout} tls={tls}"


address = ("db.internal", 5432)
options = {"timeout": 2.5, "tls": True}
print(connect(*address, **options))

defaults = {"timeout": 5.0, "tls": False}
overrides = {"tls": True}
print({**defaults, **overrides})       # later keys win: the usual way to merge options

Passing the same name twice is still an error: connect(*address, timeout=1, **options) raises TypeError ("got multiple values for keyword argument 'timeout'"). Merge the dicts first if you mean "override".

Lambdas

lambda params: expression makes a small anonymous function. The body is one expression, and its value is returned. No statements, no assignments (except :=), no annotations.

▶ Try
orders = [("ann", 30), ("bob", 120), ("cy", 75)]
print(sorted(orders, key=lambda order: order[1], reverse=True))
print(max(orders, key=lambda order: order[1])[0])

square = lambda n: n * n               # works, but PEP 8 says: use def for named functions
print(square(7), square.__name__)

Use lambdas inline, as the key of sorted, min, max and friends, or as a short callback. If you're naming it, write a def: it gets a real name in tracebacks, a docstring, and type hints. Unlike an arrow function, a lambda can't contain a block, so multi-step logic always becomes a def.

Scope: LEGB

When Python looks up a name, it searches four scopes in order, and stops at the first hit:

ScopeWhat lives there
LLocalnames assigned in the current function (including its parameters)
EEnclosinglocals of the functions this one is nested in, inner to outer
GGlobalnames assigned at the top level of the module
BBuilt-inlen, print, ValueError...
▶ Try
rate = 0.2                       # global


def outer() -> None:
    rate = 0.1                   # enclosing, for inner()

    def inner() -> None:
        print("inner sees", rate)

    inner()


outer()
print("global is still", rate)

Two differences from JavaScript and C# matter a lot:

  1. Only functions (and classes, modules, comprehensions) create scopes. if, for, while, with and try blocks don't. A variable assigned inside a loop is still there after it, and so is the loop variable.
  2. Assigning to a name anywhere in a function makes it local to the whole function. Python decides this when it compiles the function, not line by line. So reading the name before the assignment fails with UnboundLocalError, even if a global of the same name exists.
▶ Try
for i in range(3):
    last = i * 10
print(i, last)                   # both survive the loop

count = 0


def increment() -> None:
    try:
        count += 1               # an assignment: count is local to increment
    except UnboundLocalError as e:
        print("UnboundLocalError:", e)


increment()

Reading a global from a function is fine. Rebinding it needs a declaration.

global and nonlocal

global name makes assignments in the function rebind the module-level name. nonlocal name makes them rebind the name in the nearest enclosing function.

▶ Try
calls = 0


def tracked() -> None:
    global calls
    calls += 1


tracked()
tracked()
print(calls)


def make_counter():
    count = 0

    def increment() -> int:
        nonlocal count
        count += 1
        return count

    return increment


counter = make_counter()
print(counter(), counter(), counter())

You need neither to mutate an object: items.append(x) or cache[key] = value doesn't rebind the name, so it works without a declaration. You need them only for name = ..., name += ... and friends. In real code, global is a smell (state that every caller shares and tests have to reset); nonlocal in a small closure is normal.

Closures

A nested function that uses variables from its enclosing function keeps them alive after the enclosing function returns. Each call of the outer function creates new variables, so each closure gets its own state:

▶ Try
def make_multiplier(factor: int):
    def multiply(n: int) -> int:
        return n * factor
    return multiply


double = make_multiplier(2)
triple = make_multiplier(3)
print(double(10), triple(10))


def make_account(balance: int):
    def deposit(amount: int) -> int:
        nonlocal balance
        balance += amount
        return balance

    def get_balance() -> int:
        return balance

    return deposit, get_balance


deposit, get_balance = make_account(100)
deposit(50)
print(get_balance())             # both closures share the same balance variable

The closure captures the variable, not its value at the time. If the variable changes later, the closure sees the new value. That's how deposit and get_balance stay in sync, and it's also the cause of the most famous closure bug.

Late binding in loops

▶ Try
callbacks = []
for i in range(3):
    callbacks.append(lambda: i)

print([f() for f in callbacks])

All three lambdas print 2. They don't store i's value; they look up the variable i when they're called, and by then the loop has finished and i is 2. The loop doesn't create a new i per iteration. JavaScript's for (let i = ...) does, which is why the same code works there (and why the var version of it is a classic interview question).

Three fixes, all of which bind the current value at the time the function is created:

▶ Try
from functools import partial

# 1. A default argument: evaluated once, when the lambda is created
fixed_default = [lambda i=i: i for i in range(3)]

# 2. A factory function: each call creates a new scope with its own n
def make_getter(n: int):
    return lambda: n

fixed_factory = [make_getter(i) for i in range(3)]

# 3. functools.partial: the argument is stored when the partial is made
def identity(n: int) -> int:
    return n

fixed_partial = [partial(identity, i) for i in range(3)]

print([f() for f in fixed_default], [f() for f in fixed_factory], [f() for f in fixed_partial])

The default argument trick is the shortest and the one you'll see most, but it adds a parameter a caller could override by accident. The factory is the clearest. The bug is not limited to lambdas: a def inside a loop behaves the same, and so do callbacks registered in a loop (button handlers, route handlers, per-currency formatters).

Higher-order functions

A higher-order function takes a function, returns one, or both. You've used sorted(key=...) already. map and filter exist, but comprehensions usually read better in Python:

▶ Try
prices = [12.5, 3.0, 99.99, 45.0]

print(list(map(lambda p: round(p * 1.2, 2), prices)))
print([round(p * 1.2, 2) for p in prices])                 # the same, more Pythonic
print([p for p in prices if p > 10])
print(sorted(["b", "C", "a"], key=str.lower))              # pass an existing function as the key

Returning a function is how you build behaviour from configuration. A wrapper that takes *args and **kwargs can forward any call to the function it wraps:

▶ Try
def with_logging(func):
    def wrapper(*args, **kwargs):
        print(f"calling {func.__name__} with {args} {kwargs}")
        result = func(*args, **kwargs)
        print(f"{func.__name__} returned {result}")
        return result
    return wrapper


def add(a: int, b: int) -> int:
    return a + b


logged_add = with_logging(add)
logged_add(2, b=3)

That pattern (a function that takes a function and returns a wrapped one) is a decorator, and @with_logging above a def is shorthand for add = with_logging(add). The Decorators topic covers them properly, including functools.wraps.

Type hints for callables use Callable[[ArgTypes], ReturnType] from collections.abc (or typing):

▶ Try
from collections.abc import Callable


def apply_twice(func: Callable[[int], int], value: int) -> int:
    return func(func(value))


print(apply_twice(lambda n: n + 10, 1))

functools.partial

partial(func, *args, **kwargs) returns a new callable with some arguments filled in. Calling it adds the rest:

▶ Try
from functools import partial


def format_price(amount: float, currency: str, decimals: int = 2) -> str:
    return f"{amount:,.{decimals}f} {currency}"


eur = partial(format_price, currency="EUR")
yen = partial(format_price, currency="JPY", decimals=0)
print(eur(1234.5), "|", yen(1234.5))
print(eur(99, decimals=0))                 # keywords can still be overridden
print(eur.func.__name__, eur.keywords)

to_int_base2 = partial(int, base=2)
print(to_int_base2("1011"))

A lambda can do the same job (lambda amount: format_price(amount, "EUR")), with one difference: partial stores the values now, while the lambda looks its variables up later. That's why partial is a clean fix for late binding. Partials are also easy to inspect (.func, .args, .keywords), which helps in logs and tests.

What interviewers ask

QuestionShort answer
Why is def f(x=[]) a bug?The default is created once at definition time and shared by every call. Use None and create it inside.
What do *args and **kwargs hold?A tuple of extra positional arguments, and a dict of extra keyword arguments.
What do a bare * and a / in a signature do?Parameters after * are keyword-only; parameters before / are positional-only.
What's LEGB?The lookup order: local, enclosing, global, built-in.
Why UnboundLocalError?An assignment anywhere in the function makes the name local everywhere in it.
global vs nonlocal?Rebind a module-level name vs a name in the enclosing function. Neither is needed to mutate.
Why does [lambda: i for i in range(3)] give 2, 2, 2?Closures capture variables, not values, and look them up when called. Bind with i=i, a factory, or partial.
partial vs lambda?partial freezes argument values now and is inspectable; a lambda looks names up at call time.

Practise this lesson

In the app: a short video, 7 exercises checked by real tests and a 12-question concept check. Free with a GitHub account.

Open in the app