docs/source/generics.rst

Generics
========

Defining generic classes
************************

The built-in collection classes are generic classes. Generic types
have one or more type parameters, which can be arbitrary types. For
example, ``Dict[int, str]`` has the type parameters ``int`` and
``str``, and ``List[int]`` has a type parameter ``int``.

Programs can also define new generic classes. Here is a very simple
generic class that represents a stack:

.. code-block:: python

   from typing import TypeVar, Generic

   T = TypeVar('T')

   class Stack(Generic[T]):
       def __init__(self) -> None:
           # Create an empty list with items of type T
           self.items = []  # type: List[T]

       def push(self, item: T) -> None:
           self.items.append(item)

       def pop(self) -> T:
           return self.items.pop()

       def empty(self) -> bool:
           return not self.items

The ``Stack`` class can be used to represent a stack of any type:
``Stack[int]``, ``Stack[Tuple[int, str]]``, etc.

Using ``Stack`` is similar to built-in container types:

.. code-block:: python

   # Construct an empty Stack[int] instance
   stack = Stack[int]()
   stack.push(2)
   stack.pop()
   stack.push('x')        # Type error

Type inference works for user-defined generic types as well:

.. code-block:: python

   def process(stack: Stack[int]) -> None: ...

   process(Stack())   # Argument has inferred type Stack[int]

Construction of instances of generic types is also type checked:

.. code-block:: python

   class Box(Generic[T]):
       def __init__(self, content: T) -> None:
           self.content = content

   Box(1)  # OK, inferred type is Box[int]
   Box[int](1)  # Also OK
   s = 'some string'
   Box[int](s)  # Type error

Generic class internals
***********************

You may wonder what happens at runtime when you index
``Stack``. Actually, indexing ``Stack`` returns essentially a copy
of ``Stack`` that returns instances of the original class on
instantiation:

>>> print(Stack)
__main__.Stack
>>> print(Stack[int])
__main__.Stack[int]
>>> print(Stack[int]().__class__)
__main__.Stack

Note that built-in types ``list``, ``dict`` and so on do not support
indexing in Python. This is why we have the aliases ``List``, ``Dict``
and so on in the ``typing`` module. Indexing these aliases gives
you a class that directly inherits from the target class in Python:

>>> from typing import List
>>> List[int]
typing.List[int]
>>> List[int].__bases__
(<class 'list'>, typing.MutableSequence)

Generic types could be instantiated or subclassed as usual classes,
but the above examples illustrate that type variables are erased at
runtime. Generic ``Stack`` instances are just ordinary
Python objects, and they have no extra runtime overhead or magic due
to being generic, other than a metaclass that overloads the indexing
operator.

.. _generic-functions:

Generic functions
*****************

Generic type variables can also be used to define generic functions:

.. code-block:: python

   from typing import TypeVar, Sequence

   T = TypeVar('T')      # Declare type variable

   def first(seq: Sequence[T]) -> T:   # Generic function
       return seq[0]

As with generic classes, the type variable can be replaced with any
type. That means ``first`` can be used with any sequence type, and the
return type is derived from the sequence item type. For example:

.. code-block:: python

   # Assume first defined as above.

   s = first('foo')      # s has type str.
   n = first([1, 2, 3])  # n has type int.

Note also that a single definition of a type variable (such as ``T``
above) can be used in multiple generic functions or classes. In this
example we use the same type variable in two generic functions:

.. code-block:: python

   from typing import TypeVar, Sequence

   T = TypeVar('T')      # Declare type variable

   def first(seq: Sequence[T]) -> T:
       return seq[0]

   def last(seq: Sequence[T]) -> T:
       return seq[-1]

.. _generic-methods-and-generic-self:

Generic methods and generic self
********************************

You can also define generic methods — just use a type variable in the
method signature that is different from class type variables. In particular,
``self`` may also be generic, allowing a method to return the most precise
type known at the point of access.

.. note::

   This feature is experimental. Checking code with type annotations for self
   arguments is still not fully implemented. Mypy may disallow valid code or
   allow unsafe code.

In this way, for example, you can typecheck chaining of setter methods:

.. code-block:: python

   from typing import TypeVar

   T = TypeVar('T', bound='Shape')

   class Shape:
       def set_scale(self: T, scale: float) -> T:
           self.scale = scale
           return self

   class Circle(Shape):
       def set_radius(self, r: float) -> 'Circle':
           self.radius = r
           return self

   class Square(Shape):
       def set_width(self, w: float) -> 'Square':
           self.width = w
           return self

   circle = Circle().set_scale(0.5).set_radius(2.7)  # type: Circle
   square = Square().set_scale(0.5).set_width(3.2)  # type: Square

Without using generic ``self``, the last two lines could not be type-checked properly.

Other uses are factory methods, such as copy and deserialization.
For class methods, you can also define generic ``cls``, using ``Type[T]``:

.. code-block:: python

   from typing import TypeVar, Tuple, Type

   T = TypeVar('T', bound='Friend')

   class Friend:
       other = None  # type: Friend

       @classmethod
       def make_pair(cls: Type[T]) -> Tuple[T, T]:
           a, b = cls(), cls()
           a.other = b
           b.other = a
           return a, b

   class SuperFriend(Friend):
       pass

   a, b = SuperFriend.make_pair()

Note that when overriding a method with generic ``self``, you must either
return a generic ``self`` too, or return an instance of the current class.
In the latter case, you must implement this method in all future subclasses.

Note also that mypy cannot always verify that the implementation of a copy
or a deserialization method returns the actual type of self. Therefore
you may need to silence mypy inside these methods (but not at the call site),
possibly by making use of the ``Any`` type.

.. _type-variable-value-restriction:

Type variables with value restriction
*************************************

By default, a type variable can be replaced with any type. However, sometimes
it's useful to have a type variable that can only have some specific types
as its value. A typical example is a type variable that can only have values
``str`` and ``bytes``:

.. code-block:: python

   from typing import TypeVar

   AnyStr = TypeVar('AnyStr', str, bytes)

This is actually such a common type variable that ``AnyStr`` is
defined in ``typing`` and we don't need to define it ourselves.

We can use ``AnyStr`` to define a function that can concatenate
two strings or bytes objects, but it can't be called with other
argument types:

.. code-block:: python

   from typing import AnyStr

   def concat(x: AnyStr, y: AnyStr) -> AnyStr:
       return x + y

   concat('a', 'b')    # Okay
   concat(b'a', b'b')  # Okay
   concat(1, 2)        # Error!

Note that this is different from a union type, since combinations
of ``str`` and ``bytes`` are not accepted:

.. code-block:: python

   concat('string', b'bytes')   # Error!

In this case, this is exactly what we want, since it's not possible
to concatenate a string and a bytes object! The type checker
will reject this function:

.. code-block:: python

   def union_concat(x: Union[str, bytes], y: Union[str, bytes]) -> Union[str, bytes]:
       return x + y  # Error: can't concatenate str and bytes

Another interesting special case is calling ``concat()`` with a
subtype of ``str``:

.. code-block:: python

    class S(str): pass

    ss = concat(S('foo'), S('bar')))

You may expect that the type of ``ss`` is ``S``, but the type is
actually ``str``: a subtype gets promoted to one of the valid values
for the type variable, which in this case is ``str``. This is thus
subtly different from *bounded quantification* in languages such as
Java, where the return type would be ``S``. The way mypy implements
this is correct for ``concat``, since ``concat`` actually returns a
``str`` instance in the above example:

.. code-block:: python

    >>> print(type(ss))
    <class 'str'>

You can also use a ``TypeVar`` with a restricted set of possible
values when defining a generic class. For example, mypy uses the type
``typing.Pattern[AnyStr]`` for the return value of ``re.compile``,
since regular expressions can be based on a string or a bytes pattern.

.. _type-variable-upper-bound:

Type variables with upper bounds
********************************

A type variable can also be restricted to having values that are
subtypes of a specific type. This type is called the upper bound of
the type variable, and is specified with the ``bound=...`` keyword
argument to ``TypeVar``.

.. code-block:: python

   from typing import TypeVar, SupportsAbs

   T = TypeVar('T', bound=SupportsAbs[float])

In the definition of a generic function that uses such a type variable
``T``, the type represented by ``T`` is assumed to be a subtype of
its upper bound, so the function can use methods of the upper bound on
values of type ``T``.

.. code-block:: python

   def largest_in_absolute_value(*xs: T) -> T:
       return max(xs, key=abs)  # Okay, because T is a subtype of SupportsAbs[float].

In a call to such a function, the type ``T`` must be replaced by a
type that is a subtype of its upper bound. Continuing the example
above,

.. code-block:: python

   largest_in_absolute_value(-3.5, 2)   # Okay, has type float.
   largest_in_absolute_value(5+6j, 7)   # Okay, has type complex.
   largest_in_absolute_value('a', 'b')  # Error: 'str' is not a subtype of SupportsAbs[float].

Type parameters of generic classes may also have upper bounds, which
restrict the valid values for the type parameter in the same way.

A type variable may not have both a value restriction (see
:ref:`type-variable-value-restriction`) and an upper bound.

.. _declaring-decorators:

Declaring decorators
********************

One common application of type variable upper bounds is in declaring a
decorator that preserves the signature of the function it decorates,
regardless of that signature. Here's a complete example:

.. code-block:: python

   from typing import Any, Callable, TypeVar, Tuple, cast

   FuncType = Callable[..., Any]
   F = TypeVar('F', bound=FuncType)

   # A decorator that preserves the signature.
   def my_decorator(func: F) -> F:
       def wrapper(*args, **kwds):
           print("Calling", func)
           return func(*args, **kwds)
       return cast(F, wrapper)

   # A decorated function.
   @my_decorator
   def foo(a: int) -> str:
       return str(a)

   # Another.
   @my_decorator
   def bar(x: float, y: float) -> Tuple[float, float, bool]:
       return (x, y, x > y)

   a = foo(12)
   reveal_type(a)  # str
   b = bar(3.14, 0)
   reveal_type(b)  # Tuple[float, float, bool]
   foo('x')    # Type check error: incompatible type "str"; expected "int"

From the final block we see that the signatures of the decorated
functions ``foo()`` and ``bar()`` are the same as those of the original
functions (before the decorator is applied).

The bound on ``F`` is used so that calling the decorator on a
non-function (e.g. ``my_decorator(1)``) will be rejected.

Also note that the ``wrapper()`` function is not type-checked. Wrapper
functions are typically small enough that this is not a big
problem. This is also the reason for the ``cast()`` call in the
``return`` statement in ``my_decorator()``. See :ref:`casts`.