Python · Lesson 7

Errors and Context Managers

How Python raises and catches exceptions, the built-in hierarchy, try/except/else/finally, domain exception classes, chaining with raise from, ExceptionGroup and except*, EAFP, and the with statement, contextlib and ExitStack that guarantee cleanup.

14 min read

In short

  • Small try, specific except, else and finally
  • Exception is the branch you handle
  • Domain classes with data, raise ... from e
  • ExceptionGroup and except* for many errors
  • with, @contextmanager and ExitStack for cleanup

Every service fails somewhere: a payment gateway times out, a config value is "eighty", a file is missing. Good error handling answers three questions. Does the caller get an error it can act on? Does the log say what failed and why? Is the connection, lock or temp file cleaned up either way? In Python the first two are exceptions, and the third is the with statement.

If you know C#, most of this will look familiar: exceptions are classes, except filters by type, and with is using. Coming from JavaScript, the big differences are that you can (and should) catch by type, that Python code raises exceptions for ordinary situations like a missing dict key, and that cleanup has its own statement instead of a hand-written finally everywhere.

Every python snippet has a ▶ Try button. Predict the output first, then run it.

raise and except

raise stops the current function and unwinds the call stack until a matching except clause catches the exception. Functions in between need no code at all:

▶ Try
def parse_port(text: str) -> int:
    port = int(text)                   # ValueError for "eighty"
    if not 1 <= port <= 65535:
        raise ValueError(f"port out of range: {port}")
    return port


def load(settings: dict[str, str]) -> int:
    return parse_port(settings["port"])   # KeyError if missing


for settings in [{"port": "8080"}, {"port": "eighty"}, {"port": "70000"}, {}]:
    try:
        print("port", load(settings))
    except ValueError as e:
        print("bad port:", e)
    except KeyError as e:
        print("missing setting:", e)

except SomeType as e catches that class and its subclasses, and binds the exception object to e. The first matching clause wins, and the others are skipped. If no clause matches, the exception keeps travelling up. If nothing catches it, the program prints a traceback and exits.

str(e) is the message. For a KeyError it's the repr of the key, which is why the last line shows 'port' in quotes.

The hierarchy

Exceptions are classes, arranged in a tree. You catch a branch by catching its root:

BaseException
 ├── KeyboardInterrupt, SystemExit, GeneratorExit    (not errors: don't catch these by accident)
 └── Exception                                       (everything you normally handle)
      ├── ValueError          a right-typed but bad value: int("abc")
      ├── TypeError           the wrong type: "a" + 1, a missing argument
      ├── LookupError
      │    ├── KeyError       d["missing"]
      │    └── IndexError     [][0]
      ├── AttributeError      None.name
      ├── ArithmeticError
      │    └── ZeroDivisionError
      ├── OSError             files, sockets, the OS
      │    ├── FileNotFoundError, PermissionError
      │    ├── ConnectionError
      │    └── TimeoutError
      ├── RuntimeError
      │    └── NotImplementedError
      └── StopIteration       how iterators say "done"
▶ Try
print(issubclass(KeyError, LookupError), issubclass(FileNotFoundError, OSError))
print(issubclass(KeyboardInterrupt, Exception))
print([cls.__name__ for cls in ZeroDivisionError.__mro__])

try:
    {"a": 1}["b"]
except LookupError as e:          # catches KeyError and IndexError
    print(type(e).__name__, e)

KeyboardInterrupt (Ctrl+C) and SystemExit (sys.exit()) sit outside Exception on purpose: an except Exception handler shouldn't stop the user from interrupting the program.

Raise thisWhen
ValueErrorThe argument has the right type but an unacceptable value
TypeErrorThe argument has the wrong type
KeyError / LookupErrorSomething looked up by key or id doesn't exist
RuntimeErrorThe object is in the wrong state for the call
NotImplementedErrorAn abstract method or a stub (the exercise starters use it)
your own classDomain failures callers handle specifically (see below)

try, except, else, finally

The full statement has four parts, and each one runs at a different moment:

▶ Try
def divide(a: float, b: float) -> None:
    try:
        result = a / b
    except ZeroDivisionError:
        print("  except: can't divide by zero")
    else:
        print("  else: result is", result)
    finally:
        print("  finally: always runs")


print("10 / 2")
divide(10, 2)
print("1 / 0")
divide(1, 0)
  • except runs if the try block raised a matching exception.
  • else runs if the try block didn't raise. Code there isn't protected by the except clauses, which is the point: put only the risky call in try, and the follow-up work in else.
  • finally runs in every case: success, a caught exception, an uncaught one, and even a return, break or continue inside try. It's for cleanup.

One clause can catch several types with a tuple, and order matters, because the first match wins. Always put the more specific class first:

▶ Try
def classify(exc: Exception) -> str:
    try:
        raise exc
    except FileNotFoundError:
        return "missing file"
    except (PermissionError, IsADirectoryError):
        return "can't read it"
    except OSError:
        return "other OS problem"


print(classify(FileNotFoundError("config.toml")))
print(classify(PermissionError("secrets")))
print(classify(TimeoutError("disk")))

If except OSError came first, every one of those would be "other OS problem". Linters warn when a clause can never be reached.

finally and return

A return in finally replaces the function's result and swallows any exception in flight:

▶ Try
def swallowed() -> str:
    try:
        raise RuntimeError("important failure")
    finally:
        return "everything is fine"


print(swallowed())

Python 3.14 warns about this, and C# doesn't compile it at all. Keep finally for cleanup only.

Catch narrowly

The most common error-handling bug is catching too much. Compare:

▶ Try
prices = {"apple": 120, "pear": 95}


def price_bad(item: str) -> int | None:
    try:
        return prices[itme]          # typo: NameError
    except Exception:
        return None                  # the bug is hidden forever


def price_good(item: str) -> int | None:
    try:
        return prices[item]
    except KeyError:                 # only the failure we expect
        return None


print(price_bad("apple"), price_good("apple"), price_good("kiwi"))

price_bad returns None for every item, and nothing in the logs says why. The rules that prevent it:

  1. Catch the specific class you expect, not Exception, and never a bare except: (which also catches KeyboardInterrupt).
  2. Keep the try block small: only the call that can fail. Everything else goes before it or in else.
  3. Handle it or let it go. Catch only if you can do something useful: fall back, retry, translate it, or add context. A top-level handler (the web framework, the job runner) logs the rest with a full traceback.

except Exception does have one right place: the outermost boundary of a request, a job or a thread, where you log the error and keep the process alive.

Exception objects

An exception is an ordinary object. e.args holds the constructor arguments, str(e) is the message, and custom classes can carry any attributes you need. Since 3.11, add_note attaches extra context that shows up in the traceback:

▶ Try
try:
    int("12a")
except ValueError as e:
    print(type(e).__name__, "|", e, "|", e.args)
    e.add_note("while reading line 7 of prices.csv")
    print(e.__notes__)

Custom exception hierarchies

For failures that callers should handle specifically, define your own classes. Give the domain one base class, so callers can catch "anything from payments" or one specific case:

▶ Try
class PaymentError(Exception):
    """Base class for everything the payments module raises."""


class CardDeclined(PaymentError):
    def __init__(self, order_id: str, reason: str) -> None:
        super().__init__(f"card declined for order {order_id}: {reason}")
        self.order_id = order_id
        self.reason = reason


class GatewayTimeout(PaymentError):
    retryable = True


def charge(order_id: str, cents: int) -> str:
    if cents > 100_000:
        raise CardDeclined(order_id, "limit exceeded")
    if cents == 13:
        raise GatewayTimeout("no answer from the gateway after 30s")
    return f"ch_{order_id}"


for cents in [500, 250_000, 13]:
    try:
        print("charged", charge("A1", cents))
    except CardDeclined as e:
        print("declined:", e.reason)
    except PaymentError as e:
        print(type(e).__name__, "retryable:", getattr(e, "retryable", False))

Guidelines that come up in code review:

  • Name them for what went wrong, ending in Error for real errors (PaymentError), or plainly for outcomes (CardDeclined).
  • Subclass Exception (or a built-in that fits: a NotFound that subclasses LookupError is caught by existing except LookupError code).
  • Call super().__init__(message) so str(e) and tracebacks show the message, and put the data in attributes. Code should branch on e.reason, never on the message text.
  • Keep the hierarchy shallow. Two levels cover most domains.

This is the same design as a C# PaymentException : Exception with derived types, minus the boilerplate constructors.

Chaining: raise ... from ...

When you translate a low-level error into a domain error, keep the original as the cause:

▶ Try
import json


class ConfigError(Exception):
    pass


def load_config(text: str) -> dict:
    try:
        return json.loads(text)
    except json.JSONDecodeError as e:
        raise ConfigError("settings.json is not valid JSON") from e


try:
    load_config('{"port": 80,}')
except ConfigError as e:
    print(e)
    print("caused by:", type(e.__cause__).__name__, "-", e.__cause__)

raise X from e sets X.__cause__ = e, and the traceback shows both, joined by "The above exception was the direct cause of the following exception". The caller gets an error from its own vocabulary, and the person debugging still sees which character broke the JSON. It's the Python version of JavaScript's new Error(msg, { cause }) and C#'s innerException.

Raising inside an except block without from chains implicitly: Python stores the original in __context__ and prints "During handling of the above exception, another exception occurred", which reads like a bug in the handler. Choose deliberately:

▶ Try
def find_user(users: dict[int, str], user_id: int) -> str:
    try:
        return users[user_id]
    except KeyError:
        raise LookupError(f"no user {user_id}") from None


try:
    find_user({1: "ann"}, 7)
except LookupError as e:
    print(e, "| cause:", e.__cause__, "| context hidden:", e.__suppress_context__)
You writeTraceback showsUse when
raise New(...) from eboth, "direct cause"translating an error, keeping the details
raise New(...) from Noneonly the new onethe original is an expected detail (a missing key)
raise New(...) in exceptboth, "during handling"almost never on purpose

Re-raising

A bare raise inside except re-raises the exception being handled, with its original traceback. Use it to act on an error without handling it:

▶ Try
attempts = []


def save(record: dict) -> None:
    raise ConnectionError("database unavailable")


def save_with_audit(record: dict) -> None:
    try:
        save(record)
    except ConnectionError as e:
        attempts.append(f"failed: {e}")   # log, count a metric, roll back...
        raise                              # ...and let the caller decide


try:
    save_with_audit({"id": 1})
except ConnectionError as e:
    print("caller saw:", e)
print(attempts)

EAFP versus LBYL

Python style prefers EAFP, "easier to ask forgiveness than permission": try the operation and handle the failure. The alternative, LBYL ("look before you leap"), checks first:

▶ Try
settings = {"timeout": "30", "retries": "three"}

# LBYL: check, then act
if "timeout" in settings and settings["timeout"].isdigit():
    timeout = int(settings["timeout"])

# EAFP: act, handle the failure
try:
    retries = int(settings["retries"])
except (KeyError, ValueError):
    retries = 3

print(timeout, retries)

EAFP wins when the check would duplicate the operation's own rules (isdigit rejects "-5" and " 7", which int accepts), and when the world can change between the check and the act. os.path.exists(path) followed by open(path) is a race: the file can disappear in between. try: open(path) / except FileNotFoundError can't race. LBYL is still right for cheap checks on your own data, like if not items: return [], and when failure would be expensive to undo.

ExceptionGroup and except*

Sometimes several things fail at once: three of five concurrent requests, or every invalid field of a form. Python 3.11 added ExceptionGroup to raise them together, and except* to handle each type within the group:

▶ Try
def validate(form: dict[str, str]) -> None:
    errors: list[Exception] = []
    if "@" not in form.get("email", ""):
        errors.append(ValueError("email: not an address"))
    if "age" not in form:
        errors.append(KeyError("age"))
    if len(form.get("password", "")) < 8:
        errors.append(ValueError("password: too short"))
    if errors:
        raise ExceptionGroup("invalid signup", errors)


try:
    validate({"email": "ann", "password": "123"})
except* ValueError as group:
    print("bad values:", [str(e) for e in group.exceptions])
except* KeyError as group:
    print("missing:", [str(e) for e in group.exceptions])

Unlike except, every matching except* clause runs, each with a group of the exceptions it matched. Anything no clause matched is re-raised as a smaller group. You'll meet groups mostly from asyncio.TaskGroup, which raises one when tasks fail. The Validate a signup form exercise uses them.

Context managers: with

Cleanup that must happen no matter what (closing a file, releasing a lock, rolling back a transaction) belongs in a context manager. The with statement calls __enter__ at the start, binds its return value to the as target, and calls __exit__ at the end, whether the block finished, returned, or raised:

▶ Try
class Traced:
    def __init__(self, name: str) -> None:
        self.name = name

    def __enter__(self) -> "Traced":
        print(f"open {self.name}")
        return self

    def __exit__(self, exc_type, exc, tb) -> bool:
        status = "ok" if exc_type is None else f"error {exc_type.__name__}: {exc}"
        print(f"close {self.name} ({status})")
        return False        # don't suppress: let the exception continue


with Traced("db") as conn:
    print("using", conn.name)

try:
    with Traced("file"):
        raise OSError("disk full")
except OSError:
    print("caller handled it")

__exit__ receives the exception's type, value and traceback, or three Nones on success. Its return value decides what happens next: a truthy value suppresses the exception, and False or None lets it propagate. Suppress only specific, expected exceptions, and return False in every other case.

This is C#'s using with IDisposable, and the using declaration that recent TypeScript versions support with Symbol.dispose. The everyday uses:

▶ Try
from decimal import Decimal, localcontext

with open("notes.txt", "w") as f:   # closed when the block ends, even on error
    f.write("hello\n")
print("closed:", f.closed)

with localcontext() as ctx:         # temporary decimal precision
    ctx.prec = 3
    print(Decimal(1) / Decimal(7))
print(Decimal(1) / Decimal(7))

with open(path) as f: is the one you'll write most. Locks (with lock:), database transactions (with conn: in sqlite3), temporary directories and HTTP sessions all work the same way.

contextlib.contextmanager

Writing a class is verbose for simple cases. @contextmanager turns a generator into a context manager: the code before yield is __enter__, the yielded value is the as target, and the code after is __exit__. The exception from the block is raised at the yield, so put the cleanup in finally:

▶ Try
from contextlib import contextmanager

config = {"debug": False}


@contextmanager
def override(key: str, value: object):
    old = config[key]
    config[key] = value
    try:
        yield config
    finally:
        config[key] = old           # runs even if the block raised


with override("debug", True) as c:
    print("inside:", c["debug"])

try:
    with override("debug", True):
        raise RuntimeError("test failed")
except RuntimeError:
    pass
print("after:", config["debug"])

Without the try/finally, the exception would leave the function at yield, the restore line would never run, and every later test would see debug=True. That's the classic bug with this decorator.

suppress, ExitStack and friends

contextlib has ready-made managers for common jobs:

▶ Try
from contextlib import ExitStack, suppress

cache = {"a": 1}
with suppress(KeyError):            # "ignore this one exception"
    del cache["missing"]
print(cache)


class Conn:
    def __init__(self, name: str) -> None:
        self.name = name

    def __enter__(self) -> "Conn":
        print("open", self.name)
        return self

    def __exit__(self, *exc: object) -> None:
        print("close", self.name)


names = ["primary", "replica-1", "replica-2"]
with ExitStack() as stack:
    conns = [stack.enter_context(Conn(n)) for n in names]
    stack.callback(print, "flush metrics")
    print("using", len(conns), "connections")
  • suppress(*types) replaces a try / except X: pass. Use it only when ignoring is truly correct, like deleting something that may already be gone.
  • ExitStack manages a number of context managers you only know at runtime. Everything it entered is exited in reverse order, and if entering the third one fails, the first two are still closed. stack.callback(fn, *args) registers any cleanup function. The Open many resources safely exercise is built on it.
  • Others worth knowing: closing(obj) calls obj.close(), nullcontext() is a do-nothing manager for "maybe use a lock", and redirect_stdout captures prints.

Several managers in one with statement behave like nested with blocks, and exit in reverse order:

▶ Try
from contextlib import contextmanager


@contextmanager
def step(name: str):
    print("enter", name)
    try:
        yield
    finally:
        print("exit", name)


with step("outer"), step("inner"):
    print("body")

Cleanup guarantees

finally and __exit__ run when the block ends by falling off the end, return, break, continue or any exception, including KeyboardInterrupt. They don't run when the process is killed outright (kill -9, os._exit, a power cut), so anything that must survive a crash belongs in a transaction or a write-ahead log, not in cleanup code.

Two more things follow from the rules:

  • An exception inside finally or __exit__ replaces the original one (which survives only as __context__). Keep cleanup code simple and unlikely to fail.
  • Garbage collection eventually closes a forgotten file, but "eventually" isn't a guarantee, and on PyPy it can be much later. Use with instead of relying on it.

What to reach for

SituationUse
An expected failure you can handle heretry around just that call, except TheSpecificError
A failure you can't handleLet it propagate. Add context with add_note or raise ... from e
Translating a low-level error at a boundaryraise DomainError(...) from e
An expected detail callers shouldn't seeraise ... from None
Logging and re-raisingbare raise
Several failures at onceExceptionGroup, handled with except*
Cleanup that must always runwith and a context manager; finally for one-offs
A simple custom manager@contextmanager with try/finally around yield
A runtime-sized set of resourcesExitStack

The exercises turn these into everyday tasks: reading typed settings, a payment error hierarchy, translating storage errors, a transactional store, temporary overrides for tests, opening many resources safely, and validating a form with exception groups.

Practise this lesson

In the app: a short video, 7 exercises checked by real tests and a 12-question concept check. Free with a GitHub account.

Open in the app