diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a0a6ab363d9..6618b5edf52 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -692,6 +692,8 @@ peps/pep-0814.rst @vstinner @corona10 peps/pep-0815.rst @emmatyping peps/pep-0816.rst @brettcannon # ... +peps/pep-0821.rst @Daraan +# ... peps/pep-2026.rst @hugovk # ... peps/pep-3000.rst @gvanrossum diff --git a/peps/pep-0821.rst b/peps/pep-0821.rst new file mode 100644 index 00000000000..6bcbfe9c81c --- /dev/null +++ b/peps/pep-0821.rst @@ -0,0 +1,358 @@ +PEP: 821 +Title: Support for unpacking TypedDicts in Callable type hints +Author: Daniel Sperber +Sponsor: Pending +Discussions-To: Pending +Status: Draft +Type: Standards Track +Topic: Typing +Created: 30-Dec-2025 +Python-Version: 3.15 +Post-History: `28-Jun-2025 `__ + +Abstract +======== + +This PEP proposes allowing ``Unpack[TypedDict]`` as the parameter list inside +``Callable``, enabling concise and type-safe ways to describe keyword-only +callable signatures. Currently, ``Callable`` assumes positional-only +parameters, and typing keyword-only functions requires verbose callback +protocols. With this proposal, the keyword structure defined by a ``TypedDict`` +can be reused directly in ``Callable``. + + +Motivation +========== + +The typing specification states: + + "Parameters specified using Callable are assumed to be positional-only. + The Callable form provides no way to specify keyword-only parameters, + variadic parameters, or default argument values. For these use cases, + see the section on Callback protocols." + +— https://typing.python.org/en/latest/spec/callables.html#callable + +This limitation makes it cumbersome to declare callables meant to be invoked +with keyword arguments. The existing solution is to define a Protocol:: + + class Signature(TypedDict, closed=True): + a: int + + class KwCallable(Protocol): + def __call__(self, **kwargs: Unpack[Signature]) -> Any: ... + + # or + + class KwCallable(Protocol): + def __call__(self, *, a: int) -> Any: ... + +This works but is verbose. The new syntax allows the equivalent to be written +more succinctly:: + + type KwCallable = Callable[[Unpack[Signature]], Any] + + +Rationale +========= + +This proposal extends the existing Callable semantics by reusing a ``TypedDict``'s +keyed structure for keyword arguments. It avoids verbose Protocol-based +callable definitions while remaining compatible with current typing concepts +(:pep:`692` Unpack for ``kwargs``, and :pep:`728` ``extra_items``). It preserves backward +compatibility by being purely a typing feature. + + +Specification +============= + +New allowed form +---------------- + +It becomes valid to write:: + + Callable[[Unpack[TD]], R] + +where ``TD`` is a ``TypedDict``. A shorter form is also allowed:: + + Callable[Unpack[TD], R] + +Additionally, positional parameters may be combined with an unpacked ``TypedDict``:: + + Callable[[int, str, Unpack[TD]], R] + +Semantics +--------- + +* Each key in the ``TypedDict`` must be accepted as a keyword parameter. +* TypedDict keys cannot be positional-only; they must be valid keyword parameters. +* Positional parameters may appear in ``Callable`` before ``Unpack[TD]`` and follow normal ``Callable`` semantics. +* Required keys must be accepted, but may correspond to parameters with a + default value. +* ``NotRequired`` keys must still be accepted, but may be omitted at call sites. + This respectively applies to ``TypedDict`` with ``total=False``. +* Functions with ``**kwargs`` are compatible if the annotation of ``**kwargs`` + matches or is a supertype of the ``TypedDict`` values. +* ``extra_items`` from PEP 728 is respected: functions accepting additional + ``**kwargs`` are valid if their annotation is compatible with the declared + type. +* If neither ``extra_items`` nor ``closed`` (PEP 728) is specified on the + ``TypedDict``, additional keyword arguments are implicitly permitted with + type ``object`` (i.e., compatible with ``**kwargs: object``). Setting + ``closed=True`` forbids any additional keyword arguments beyond the keys + declared in the ``TypedDict``. Setting ``extra_items`` to a specific type + requires that any additional keyword arguments match that type. +* Only a single ``TypedDict`` may be unpacked inside a ``Callable``. Support + for multiple unpacks may be considered in the future. + +Examples +-------- + +The following examples illustrate how unpacking a ``TypedDict`` into a +``Callable`` enforces acceptance of specific keyword parameters. A function is +compatible if it can be called with the required keywords (even if they are +also accepted positionally); positional-only parameters for those keys are +rejected. + +.. code-block:: python + + from typing import TypedDict, Callable, Unpack, Any, NotRequired + + class Signature(TypedDict): + a: int + + type IntKwCallable = Callable[[Unpack[Signature]], Any] + + def normal(a: int): ... + def kw_only(*, a: int): ... + def pos_only(a: int, /): ... + def different(bar: int): ... + + f1: IntKwCallable = normal # Accepted + f2: IntKwCallable = kw_only # Accepted + f3: IntKwCallable = pos_only # Rejected + f4: IntKwCallable = different # Rejected + +Optional arguments +------------------ + +Keys marked ``NotRequired`` in the ``TypedDict`` correspond to optional +keyword arguments. +Meaning the callable must accept them, but callers may omit them. +Functions that accept the keyword argument must also provide a default value that is compatible; +functions that omit the parameter entirely are rejected. + +.. code-block:: python + + class OptSig(TypedDict): + a: NotRequired[int] + + type OptCallable = Callable[[Unpack[OptSig]], Any] + + def defaulted(a: int = 1): ... + def kw_default(*, a: int = 1): ... + def no_params(): ... + def required(a: int): ... + + g1: OptCallable = defaulted # Accepted + g2: OptCallable = kw_default # Accepted + g3: OptCallable = no_params # Rejected + g4: OptCallable = required # Rejected + +Additional keyword arguments +---------------------------- + +Default Behavior (no ``extra_items`` or ``closed``) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If the ``TypedDict`` does not specify ``extra_items`` or ``closed``, additional +keyword arguments are permitted with type ``object``. This is the default behavior. + +.. code-block:: python + + # implies extra_items=object + class DefaultTD(TypedDict): + a: int + + type DefaultCallable = Callable[[Unpack[DefaultTD]], Any] + + def v_any(**kwargs: object): ... + def v_ints(a: int, b: int=2): ... + + d1: DefaultCallable = v_any # Accepted (implicit object for extras) + d1(a=1, c="more") # Accepted (extras allowed) + d2: DefaultCallable = v_ints # Rejected (b: int is not a supertype of object) + +``closed`` behavior (PEP 728) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If ``closed=True`` is specified on the ``TypedDict``, no additional keyword +arguments beyond those declared are expected. + +.. code-block:: python + + class ClosedTD(TypedDict, closed=True): + a: int + + type ClosedCallable = Callable[[Unpack[ClosedTD]], Any] + + def v_any(**kwargs: object): ... + def v_ints(a: int, b: int=2): ... + + c1: ClosedCallable = v_any # Accepted + c1(a=1, c="more") # Rejected (extra c not allowed) + c2: ClosedCallable = v_ints # Accepted + c2(a=1, b=2) # Rejected (extra b not allowed) + +Interaction with ``extra_items`` (PEP 728) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If a ``TypedDict`` specifies the ``extra_items`` parameter (with the exemption of ``extra_items=Never``), the corresponding ``Callable`` +must accept additional keyword arguments of the specified type. + +For example: + +.. code-block:: python + + class ExtraTD(TypedDict, extra_items=str): + a: int + + type ExtraCallable = Callable[[Unpack[ExtraTD]], Any] + + def accepts_str(**kwargs: str): ... + def accepts_object(**kwargs: object): ... + def accepts_int(**kwargs: int): ... + + e1: ExtraCallable = accepts_str # Accepted (matches extra_items type) + e2: ExtraCallable = accepts_object # Accepted (object is a supertype of str) + e3: ExtraCallable = accepts_int # Rejected (int is not a supertype of str) + + e1(a=1, b="foo") # Accepted + e1(a=1, b=2) # Rejected (b must be str) + + +Interaction with ``ParamSpec`` and ``Concatenate`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A ``ParamSpec`` can be substituted by ``Unpack[TypedDict]`` to define a +parameterized callable alias. Substituting ``Unpack[Signature]`` produces the +same effect as writing the callable with an unpacked ``TypedDict`` directly. +Using a ``TypedDict`` within ``Concatenate`` is not allowed. + +.. code-block:: python + + from typing import ParamSpec + + P = ParamSpec("P") + type CallableP = Callable[P, Any] + + # CallableP[Unpack[Signature]] is equivalent to Callable[[Unpack[Signature]], Any] + h: CallableP[Unpack[Signature]] = normal # Accepted + h2: CallableP[Unpack[Signature]] = kw_only # Accepted + h3: CallableP[Unpack[Signature]] = pos_only # Rejected + +Combined positional parameters and ``Unpack``: + +Positional parameters may precede an unpacked ``TypedDict`` inside ``Callable``. +Functions that accept the required positional arguments and can be called with +the specified keyword(s) are compatible; making the keyword positional-only is +rejected. + +.. code-block:: python + + from typing import TypedDict, Callable, Unpack, Any + + class Signature(TypedDict): + a: int + + type IntKwPosCallable = Callable[[int, str, Unpack[Signature]], Any] + + def mixed_kwonly(x: int, y: str, *, a: int): ... + def mixed_poskw(x: int, y: str, a: int): ... + def mixed_posonly(x: int, y: str, a: int, /): ... + + m1: IntKwPosCallable = mixed_kwonly # Accepted + m2: IntKwPosCallable = mixed_poskw # Accepted + m3: IntKwPosCallable = mixed_posonly # Rejected + +Inline TypedDicts (PEP 764): + +Inline ``TypedDict`` forms are supported like any other ``TypedDict``, allowing compact definitions when the +structure is used only once. + +.. code-block:: python + + Callable[[Unpack[TypedDict({"a": int})]], Any] + + +Backwards Compatibility +======================= + +This feature is additive. Existing code is unaffected. Runtime behavior does +not change; this is a typing-only feature. + +How to Teach This +================= + +This feature is a shorthand for Protocol-based callbacks. Users should be +taught that with + +.. code-block:: python + + class Signature(TypedDict): + a: int + b: NotRequired[str] + +* ``Callable[[Unpack[Signature]], R]`` is equivalent to defining a Protocol with + ``__call__(self, **kwargs: Unpack[Signature]) -> R`` + or ``__call__(self, a: int, b: str = ..., **kwargs: object) -> R``. +* The implicit addition of ``**kwargs: object`` might come surprising to users, + using ``closed=True`` for definitions will create the more intuitive equivalence + of ``__call__(self, a: int, b: str = ...) -> R`` + + +Reference Implementation +======================== + +A prototype exists in mypy: +https://github.com/python/mypy/pull/16083 + + +Rejected Ideas +============== + +- Combining ``Unpack[TD]`` with ``Concatenate``. With such support, one could write + ``Callable[Concatenate[int, Unpack[TD], P], R]`` which in turn would allow a keyword-only parameter between ``*args`` and ``**kwargs``, i.e. + ``def func(*args: Any, a: int, **kwargs: Any) -> R: ...`` which is currently not allowed per :pep:`612`. + To keep the initial implementation simple, this PEP does not propose such + support. + +Open Questions +============== + +* Should multiple ``TypedDict`` unpacks be allowed to form a union, and if so, how to handle + overlapping keys of non-identical types? Which restrictions should apply in such a case? Should the order matter? +* Is there a necessity to differentiate between normal and ``ReadOnly`` keys? +* Is it necessary to specify generic behavior for ``TypedDict`` and the resulting ``Callable`` when the ``TypedDict`` itself is generic? + + +Acknowledgements +================ +TODO + + + +References +========== + +* :pep:`692` - Using ``Unpack`` with ``**kwargs`` +* :pep:`728` - ``extra_items`` in TypedDict +* :pep:`764` - Inline TypedDict +* `mypy PR #16083 - Prototype support `__ +* Revisiting PEP 677 (`discussion thread `__) + + +Copyright +========= + +This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.