blob: b48eefd421e1946d498d47653fc230c942d9dca0 [file] [view] [edit]
---
name: mypyc-optimize
description: Optimize Python source files that are compiled with mypyc (or
will be compiled), or identify performance bottlenecks and optimization
opportunities in them without making changes. Also apply the
performance-related changes from the mypyc-migrate skill, and improve
type annotations so that mypyc can generate faster code.
---
# Optimize Python Code Compiled with Mypyc
This skill builds on the mypyc-migrate skill (`../mypyc-migrate/SKILL.md`).
Read that skill first; it has examples and details for the steps below.
This skill can also be used to only identify bottlenecks and optimization
opportunities, without changing the code. In this case, use the
techniques below (profiling, inspecting generated code, and so on) to find
issues, and report them to the user, ordered by expected impact.
Don't change behavior in significant ways. If an improvement would require
non-trivial refactoring or might change behavior, mention it in your summary
instead.
## Workflow
Optimize incrementally:
1. Find the hot functions using a profile of a realistic workload (see
"Profiling"), unless the user already told you what to optimize. If
no profile is available, make educated guesses about likely hot
functions (e.g. error handling is likely not hot).
2. Make one change, or a small set of related changes, at a time.
3. Run the project's tests after each change, including tests that run
the compiled code, if there are any. If running the tests or
benchmarks is slow, you can make changes in larger batches to make
the process faster.
4. Measure whether the change helps (see "Measure Performance"). Revert
changes that don't clearly help, unless they are simple or also make
the code cleaner (such as adding missing annotations). Keep simple
changes even if their effect can't be measured: many simple changes
are helpful, but the impact of each one in isolation is often too small
to measure. If no benchmark is available, first try to show the impact
using extracted synthetic microbenchmarks that simulate the actual
code. If this isn't practical, assume that changes that follow the
instructions below (including their conditions) improve performance.
In your summary, list the changes you made and the measured effect, if
any. Also list changes that you tried but reverted since they didn't help,
and potential bottlenecks that you found but didn't fix (see "Report Other
Potential Bottlenecks").
## Steps from the Migrate Skill
In the target files, make sure that all the changes described in the migrate
skill have been applied. These are especially important for performance:
* Annotate all constants in module top level and class bodies using `Final`
(import from `typing`). Reading `Final` values is much faster than reading
regular module-level variables. Don't make a value `Final` if any code or
test reassigns it.
* Annotate all class variables using `ClassVar` (import from `typing`),
unless they are already `Final`. Without `ClassVar`, mypyc treats a class
variable as an instance attribute, which takes space in each instance.
* Add `@final` to classes that aren't intended to be subclassed (see the
rules in the migrate skill).
* Move classes nested within functions or other classes to the module top
level.
* Don't cache bound methods or global functions in local variables.
Also go through the compatibility checks in "Code That Won't Compile or Will
Break" in the migrate skill, and fix any issues.
## Improve Type Annotations
Precise type annotations let mypyc use fast, specialized operations. Values
with type `Any` and values in functions without annotations use slow,
generic operations.
* Add type annotations to functions that have missing annotations, including
missing return type annotations (use `-> None` if a function doesn't return
a value). A function without any annotations isn't type checked by mypy
(when using default strictness options), and mypyc compiles it using generic
operations.
* Replace `Any` types with precise types where possible. For example, use
`list[str]` or `dict[str, int]` instead of `Any`, `list[Any]` or
`dict[str, Any]` when the item types are known. Prefer concrete types such
as `list[...]` over abstract types such as `Sequence[...]` when the concrete
type is known.
* Annotate variables that get their value from untyped code (for example,
a library without type annotations), so that the variable doesn't have type
`Any`.
* If you can infer a more precise type for a value from the surrounding
context, but there's no `isinstance` check (or similar) that would narrow
the type, use `cast` (import from `typing`) to give mypyc the precise type.
Assign the result to a new variable, since mypyc uses the declared type of
a variable.
Only use a type if you are confident that it's correct. Compiled code checks
types at runtime and raises `TypeError` if a value has the wrong type (see
"Annotations Are Enforced at Runtime" in the migrate skill). Check callers,
including tests and non-compiled code, to make sure that they never pass
values of other types, such as `None`. Adding annotations to a function that
previously had none also causes mypy to type check its body; fix any new type
errors that this produces.
## Measure Performance
The speedup from the optimizations below varies a lot, depending on the
code, and some changes don't help at all in a particular case. Measure
whether a change helps before keeping it, especially if it makes the code
more verbose or harder to read.
If the target code is self-contained enough, write a throwaway benchmark
script that imports the compiled modules and runs a realistic workload
(for example, the main entry point on representative input). Run it before
and after each change (or set of related changes, and don't run it too
often if the benchmark runs slowly). Make sure each benchmark run isn't
very slow -- since these are synthetic workloads, they may not show very
minor performance changes reliably.
If the code isn't self-contained (for example, it needs a server, a
database, or a large application to run), extract the hot functions into a
separate module and write microbenchmarks that simulate the behavior of
the original functions. Keep the same type annotations and similar data
(types, sizes and shapes of the values), since these determine what code
mypyc generates. Use these microbenchmarks as synthetic proxies for the
performance of the real code, for example to decide which of several ways
of writing a function is fastest.
When benchmarking:
* Compile benchmarks in a temporary directory, since `mypyc` writes build
files to the `build/` directory under the current working directory.
* Check that the benchmarked module was actually compiled
(`mod.__file__` should end in `.so` or `.pyd`), and that mypyc reported
no errors.
* Run the benchmarked code many times, or in a loop, so that each
measurement takes at least several milliseconds, and take the best (or
the median) of several runs using `time.perf_counter()`. Timings can be
noisy, so only trust differences that are clearly larger than the
variation between runs.
* Don't leave throwaway benchmarks in the project, unless asked to.
Benchmarks can also be profiled to find where time is spent (see
"Profiling").
## Profiling
Profile the code (or a benchmark) to find the hot functions before
optimizing, and to check where time is spent after a change. Use a native
(sampling) profiler that shows C-level frames. Good options to try, if
they are available:
* `py-spy record --native -o profile.svg -- python3 bench.py` (native
frames are supported on Linux and Windows)
* `perf` on Linux (`perf record -g python3 bench.py`, then `perf report`)
* On macOS: Instruments (Time Profiler), `samply`, or `sample`
* On Windows: the Visual Studio profiler, or Windows Performance Recorder
and Analyzer
If the profiled workload seems realistic enough, focus most optimization
effort on the functions where the profile shows the most time is spent
(including time spent in functions they call). Optimizing code that
barely shows up in the profile rarely makes a noticeable difference, and
makes the code more verbose for little gain. The simple, mechanical steps
from the migrate skill can still be applied everywhere.
Don't use `cProfile` or `profile`. They don't produce native frames, and
don't see calls within compiled code, so the results are misleading for
compiled modules.
Compiled functions have names with the prefix `CPyDef_` in profiles (e.g.
`CPyDef_mod___parse` for function `parse` in module `mod`). Functions with
the prefix `CPyPy_` are wrappers used when compiled functions are called
from non-compiled code.
Call stacks can be incomplete if CPython or the compiled extensions were
built without frame pointers. A custom CPython build (and compiled modules)
with `-fno-omit-frame-pointer`, or a profiler mode that uses DWARF debug
information for stack unwinding, can give better results. Look up the
current recommended approach for the profiler and Python version used.
## Inspect Generated Code
Looking at the code mypyc generates can reveal bottlenecks that aren't
obvious from the source code, such as unexpected `Any` types that cause
slow, generic operations to be used.
Mypyc writes the generated intermediate representation (IR) to
`build/ops.txt` and the generated C code to `build/__native*.c`. The IR is
easier to read than C. Signs of generic operations include calls to
C API functions such as `PyObject_GetAttr`, `CPyObject_GetAttr`,
`PyObject_GetItem`, `PyObject_Vectorcall`, `PyObject_GetIter`,
`PyIter_Next` or `PyNumber_Add` (and other `PyNumber_` functions) in a
hot function. Specialized operations have names such as `CPyTagged_Add`
(for `int` values) or `CPyList_GetItem`, or use native attribute access.
Common causes of generic operations include values with type `Any`,
missing annotations, non-native classes, and values that come from
non-compiled modules. Fix these using the other techniques in this skill,
such as annotating variables that get their value from untyped code.
### Operations with Fast Implementations
The mypyc documentation lists the operations that have fast,
specialized implementations for each primitive type. Operations that
aren't listed use generic operations. Check these when choosing between
alternative ways of writing a hot function:
* `mypyc/doc/native_operations.rst` (native classes and functions)
* `mypyc/doc/int_operations.rst`, `float_operations.rst` and
`bool_operations.rst`
* `mypyc/doc/str_operations.rst`, `bytes_operations.rst` and
`bytearray_operations.rst`
* `mypyc/doc/list_operations.rst`, `dict_operations.rst`,
`set_operations.rst`, `frozenset_operations.rst` and
`tuple_operations.rst`
These files are in the mypy repository, and they are also available at
https://mypyc.readthedocs.io/. For example, a `str` method that isn't
listed in `str_operations.rst` is called using a generic method call.
## Compile Related Modules Together
All modules compiled in a single mypyc invocation (a single `mypyc`
command, or a single `mypycify(...)` call in `setup.py`) form a
*compilation unit*. Calls to functions, methods and classes within a
compilation unit use fast, direct calls, and `Final` values are inlined.
References to other compilation units use slow, generic operations, and
are about as slow as references to non-compiled modules. Using
`separate=True` with `mypycify` still keeps all the modules in the same
compilation unit.
If modules that frequently call each other are compiled using separate
invocations, suggest compiling them together, or report this to the
user. Similarly, if a hot function frequently calls a module that isn't
compiled, consider compiling that module too (after applying the
migrate skill to it). Don't change the build configuration without
asking the user, since it may affect packaging and build times.
## Keep Classes Native
Native classes are much faster than regular Python classes (non-native
classes). Making an important class native is often the biggest single
improvement. A class is silently compiled as a non-native class if it uses
an unsupported metaclass or class decorator (see "Class Decorators" in the
migrate skill). Where possible, remove the cause.
For example, a class decorator that registers the class can often be
replaced with an explicit call:
```
@register
class JsonHandler: # Non-native, due to the class decorator
...
```
Call `register` after the class definition instead:
```
class JsonHandler: # Native
...
register(JsonHandler)
```
This is only equivalent if the decorator returns the original class
(instead of a wrapper or a different class). Check the implementation of
the decorator.
If a performance-critical class has non-trivial base classes (other than
`object`, `ABC`, `Generic[...]` or native classes in the same project), a
metaclass other than `ABCMeta`, or class decorators, add
`@mypyc_attr(native_class=True)` (import from `mypy_extensions`), so that the
class can't accidentally become non-native later. Don't add it to other
classes, as they are unlikely to become non-native.
### Replace Dataclasses with Regular Classes
Dataclasses are supported, but they aren't as efficient as regular native
classes. Consider replacing performance-critical dataclasses with regular
classes:
```
from dataclasses import dataclass
@dataclass(frozen=True)
class Point:
x: int
y: int
label: str = ""
```
Define `__init__` and `__repr__` explicitly. For a frozen dataclass, declare
the attributes as `Final` in `__init__`, since they can't be assigned to after
initialization:
```
from typing import Final, final
@final
class Point:
def __init__(self, x: int, y: int, label: str = "") -> None:
self.x: Final = x
self.y: Final = y
self.label: Final = label
def __repr__(self) -> str:
return f"Point(x={self.x!r}, y={self.y!r}, label={self.label!r})"
```
Dataclasses also generate other methods, such as `__eq__` (and `__hash__`
for frozen dataclasses, and ordering methods if `order=True`). If any code
compares instances, uses them as dictionary keys or in sets, or sorts them,
define these methods explicitly, as otherwise the behavior changes. Also
check for uses of `dataclasses` functions such as `dataclasses.replace`,
`dataclasses.asdict` or `dataclasses.fields` on the class. Don't replace the
dataclass if these are used.
## Declare Attributes Assigned Only in `__init__` as `Final`
Reading a `Final` attribute of a native class can be faster, since mypyc
knows that the attribute can't be reassigned after construction. For example,
mypyc can often avoid reference count manipulation when reading the attribute,
and reads are cheaper on free-threaded Python builds:
```
from typing import Final, final
@final
class Span:
def __init__(self, start: int, end: int) -> None:
self.start: Final = start
self.end: Final = end
```
A `Final` attribute is read-only at runtime, so assigning it outside
`__init__` fails, including assignments in non-compiled code (such as tests)
that mypy may not check.
Only make an attribute `Final` if it's clearly assigned only in `__init__`
(for example, in an internal class that is only used in a few places), and
you have access to the tests, so that you can verify that they don't assign
the attribute (for example, to set up a test scenario, or using
monkeypatching).
## Avoid Mutable Module-level Variables
Reading and writing a module-level variable that isn't `Final` requires a
dictionary lookup in the module namespace, which is slow. If a mutable
module-level variable is used frequently (for example, a counter or a cache
updated using `global`), store the state in an attribute of a `Final`
instance of a native class:
```
_calls = 0
_errors = 0
def record(ok: bool) -> None:
global _calls, _errors
_calls += 1
if not ok:
_errors += 1
```
Use a `Final` object instead:
```
from typing import Final, final
@final
class _Stats:
def __init__(self) -> None:
self.calls = 0
self.errors = 0
_stats: Final = _Stats()
def record(ok: bool) -> None:
_stats.calls += 1
if not ok:
_stats.errors += 1
```
Only do this for internal (often underscore-prefixed) variables that
aren't accessed from other modules or tests, since the variables will no
longer be available as module attributes.
## Avoid Optional Primitive Types in Performance-critical Code
`int`, `float`, `bool` and fixed-length tuple values are unboxed in
compiled code, but union types such as `int | None` are always boxed
(heap-allocated), and operations on them are slower. If `None` is only used
as a marker for a missing value, and there is an otherwise unused value
that can be used as a marker instead, use it in internal code:
```
class Parser:
def __init__(self) -> None:
self.error_pos: int | None = None # Boxed
def has_error(self) -> bool:
return self.error_pos is not None
```
Use `-1` to represent a missing position:
```
class Parser:
def __init__(self) -> None:
self.error_pos = -1 # -1 if no error; unboxed
def has_error(self) -> bool:
return self.error_pos >= 0
```
Only do this if the attribute or variable isn't part of a public interface
and all uses can be updated, and document the marker value in a comment.
## Use Native Classes Instead of Dictionaries as Records
Dictionaries with a fixed set of string keys are often used as records.
Creating a native class instance and accessing its attributes is much
faster (about 4x faster than `dict[str, Any]` and 2.5x faster than a
`TypedDict` in a benchmark):
```
def make_loc(name: str, line: int) -> dict[str, Any]:
return {"name": name, "line": line}
def next_line(loc: dict[str, Any]) -> int:
return loc["line"] + 1
```
Use a native class instead:
```
from typing import final
@final
class Loc:
def __init__(self, name: str, line: int) -> None:
self.name = name
self.line = line
def make_loc(name: str, line: int) -> Loc:
return Loc(name, line)
def next_line(loc: Loc) -> int:
return loc.line + 1
```
This also gives precise attribute types instead of `Any`. Only do this if
you can find and update all code that uses the dictionaries. Don't do this
if the dictionaries are part of a public interface, if keys are added or
removed dynamically, or if the dictionaries are passed to code that
requires a dictionary (for example, `json.dumps`, `**` unpacking, iteration
over keys, or `.get()` with a default).
## Dictionary Lookups
### Use a Single `dict.get` Instead of `in` Followed by Indexing
Checking for a key using `in` and then indexing performs two dictionary
lookups. A single `get` is faster if the key is usually present:
```
if key in counts:
total += counts[key]
```
Use `get` instead:
```
n = counts.get(key)
if n is not None:
total += n
```
This is only equivalent if the dictionary never contains `None` values.
If the key is usually missing, `in` alone is slightly faster, so only do
this if the key is expected to be present in most cases.
### Replace `defaultdict` with a Plain `dict`
Mypyc has fast, specialized operations for `dict`, but `defaultdict` is
a subclass that doesn't get all of these optimizations. Replacing an
internal `defaultdict` with a plain `dict` is somewhat faster:
```
from collections import defaultdict
counts: defaultdict[str, int] = defaultdict(int)
groups: defaultdict[str, list[int]] = defaultdict(list)
for i, key in enumerate(keys):
counts[key] += 1
groups[key].append(i)
```
Use `get` and `setdefault` with a plain `dict` instead:
```
counts: dict[str, int] = {}
groups: dict[str, list[int]] = {}
for i, key in enumerate(keys):
counts[key] = counts.get(key, 0) + 1
groups.setdefault(key, []).append(i)
```
Mypyc special-cases `setdefault` with an empty `[]`, `{}` or `set()`
default, so that the empty collection is only created if the key is
missing. Other default values are evaluated on each call, even if the key
is present.
Only do this if the dictionary is internal (it's not returned to callers
or stored in a public attribute), since code that reads a `defaultdict`
may rely on missing keys getting a default value. Check all places where
the dictionary is indexed. The benefit is small, so prioritize this only
in very hot code.
## Return Multiple Values Using Fixed-length Tuples
Fixed-length tuple types such as `tuple[int, str]` are unboxed in
compiled code: returning and unpacking them doesn't allocate an object.
Other ways of returning multiple values are much slower. In a benchmark,
returning two `int` values as a `tuple[int, int]` was about 7x faster than
returning a native class instance, 8x faster than `tuple[int, ...]`, and
40x faster than a `NamedTuple`.
Named tuples are particularly inefficient for this, since creating a named
tuple instance is slow in compiled code (about 7x slower than creating a
native class instance in a benchmark). Accessing items by name, by index or
by unpacking is fairly fast (about as fast as with a regular tuple), but
still slower than accessing native class attributes:
```
from typing import NamedTuple
class QR(NamedTuple):
q: int
r: int
def div(a: int, b: int) -> QR:
return QR(a // b, a % b)
q, r = div(x, 7)
```
Use a fixed-length tuple type instead:
```
def div(a: int, b: int) -> tuple[int, int]:
return a // b, a % b
q, r = div(x, 7)
```
Only do this for internal functions where all callers can be updated,
since callers may access items by name (such as `div(x, 7).q`), and
named tuples have a different `repr`. If there are more than a few items,
or the names are important for readability, use a small native class
instead (see below). Use a precise, fixed-length tuple type in annotations
instead of `tuple[int, ...]` when the length is known.
### Replace Named Tuples with Native Classes
A simple named tuple that is used like a record (items are accessed by
name) can be replaced with a native class with `Final` attributes. In a
benchmark, creating instances was about 7x faster and accessing attributes
about 3.5x faster. This helps most if many instances are created:
```
from typing import NamedTuple
class Span(NamedTuple):
start: int
end: int
```
Use a native class instead:
```
from typing import Final, final
@final
class Span:
def __init__(self, start: int, end: int) -> None:
self.start: Final = start
self.end: Final = end
def __repr__(self) -> str:
return f"Span(start={self.start!r}, end={self.end!r})"
```
Only do this if the tuple behavior isn't needed. Check that no code
unpacks the values (`start, end = span`), indexes or iterates over them,
compares them using `==` or `<`, hashes them (uses them as dictionary keys
or in sets), passes them where a tuple is expected, or uses named tuple
methods such as `_replace` or `_asdict`. If equality or hashing is needed,
define `__eq__` and `__hash__` explicitly.
## Initialize Attributes Before `self` Escapes in `__init__`
Mypyc analyzes `__init__` methods to find attributes that are always
initialized. Reading these attributes doesn't need a check for an undefined
value (which would raise `AttributeError`), which makes reads faster. The
analysis stops once `self` may escape: `self` is passed to a function, a
method is called on `self`, or a property of `self` is accessed. Attributes
assigned after this point aren't considered always defined. Calls that
don't involve `self` are fine.
```
class Connection:
def __init__(self, host: str) -> None:
self.host = host
self.register() # 'self' escapes
self.retries = 0 # Not always defined
```
Assign attributes before `self` escapes, when this doesn't change behavior:
```
class Connection:
def __init__(self, host: str) -> None:
self.host = host
self.retries = 0 # Always defined
self.register()
```
A call to `super().__init__()` only makes `self` escape if the base class
`__init__` makes `self` escape (for example, if it calls a method of
`self`). In this case, initialize attributes defined in the subclass before
calling `super().__init__()`, if the values don't depend on `self`:
```
class Base:
def __init__(self, name: str) -> None:
self.name = name
self.setup() # 'self' escapes
class Child(Base):
def __init__(self, name: str) -> None:
# Always defined, since assigned before super().__init__()
self.count = 0
super().__init__(name)
```
Only reorder assignments if the code that runs after `self` escapes (the
called methods, including methods overridden in subclasses, and the base
class `__init__`) doesn't read or assign the attributes being moved, and if
the assigned values don't depend on anything that is initialized later.
The analysis is only performed for native classes where all subclasses are
known at compile time. It's not performed for classes with
`allow_interpreted_subclasses=True` or `serializable=True`, for example.
An attribute defined in a base class is only always defined if it's always
defined in all subclasses.
## Use `is` to Compare Enum Values
Comparing an enum value using `is` can be much faster than `==`, when the
value has an optional type such as `Color | None` (about 6x faster in a
benchmark), or when the enum class is defined in a module that isn't compiled
together with the target files. If both operands have the same enum type and
the enum is compiled, `==` is already fast.
```
def is_red(c: Color | None) -> bool:
return c == Color.RED # Slow
```
Use `is` (and `is not` instead of `!=`):
```
def is_red(c: Color | None) -> bool:
return c is Color.RED # Fast
```
This changes behavior for `IntEnum`, `StrEnum` and other enums that
inherit from a type such as `int` or `str`, since `==` also matches plain
`int` or `str` values, and for enums that define `__eq__`. Only use `is`
with these if you are sure that the other operand is always an enum member.
## Use Integer Code Points Instead of One-character Strings
When processing strings character by character, operating on integer code
points is much faster in compiled code than operating on one-character
strings (often 5x to 15x faster in benchmarks). `ord(s[i])` is a fast
operation when `s` has type `str`, and `ord("x")` of a string literal is a
compile-time constant:
```
def count_separators(s: str) -> int:
n = 0
for c in s:
if c == "," or c == ";":
n += 1
return n
def count_digits(s: str) -> int:
n = 0
for c in s:
if "0" <= c <= "9":
n += 1
return n
```
Use `ord(...)` instead:
```
def count_separators(s: str) -> int:
n = 0
for i in range(len(s)):
c = ord(s[i])
if c == ord(",") or c == ord(";"):
n += 1
return n
def count_digits(s: str) -> int:
n = 0
for i in range(len(s)):
c = ord(s[i])
if ord("0") <= c <= ord("9"):
n += 1
return n
```
For `str` methods such as `c.isdigit()`, `c.isspace()`, `c.isalpha()`,
`c.isalnum()`, `c.upper()` and `c.lower()` on a one-character string, use the
corresponding functions in `librt.strings` that take a code point (see "Use
librt" below), e.g. `isdigit(ord(s[i]))`. Note that `librt.strings.toupper`
and `tolower` leave code points unchanged if the result would have multiple
code points (e.g. `"ß".upper()` is `"SS"`).
To build a string from code points, use `librt.strings.StringWriter`
(`w.append(c)`).
These changes make code slower when it's not compiled, so only do this in
modules that are compiled.
## Move Nested Functions to Module Level or Methods
Each time the enclosing function is called, a nested function (or a
lambda) must be allocated, together with an environment object for the
captured variables. This is relatively slow, and it adds up if the
enclosing function is called often. Module-level functions and methods
don't have this overhead:
```
def total(a: list[int], k: int) -> int:
def weight(x: int) -> int:
return x * k + 1
t = 0
for x in a:
t += weight(x)
return t
```
Move the nested function to the module level, and pass captured variables
as arguments:
```
def _weight(x: int, k: int) -> int:
return x * k + 1
def total(a: list[int], k: int) -> int:
t = 0
for x in a:
t += _weight(x, k)
return t
```
If the nested function is within a method and uses `self`, make it a
method instead (e.g. `self._weight(x)`).
Only do this if the nested function is simple enough (it captures only a
few variables, and it doesn't assign captured variables using
`nonlocal`), or if it's very performance-critical. Otherwise the
refactored code may be harder to read.
If the function is only used as a callback, such as
`sorted(a, key=lambda x: x[0])`, or passed as an argument with type
`Callable[...]`, it's called using a generic operation either way. In
these cases, the only benefit is avoiding the allocation, so move the
function only if the enclosing function is called often.
## Use `vec` Instead of `list` for Packed Item Types
`vec` (from `librt.vecs`) is a growable array type that stores items of
certain types, such as `i64`, `i32`, `u8`, `float` and `bool`, in a packed
binary representation, without boxing. Building, iterating over, and
indexing a `vec` with a packed item type is much faster than with a
`list[int]` or `list[float]` in compiled code, and uses much less memory.
Consider using a `vec` instead of an internal list of integers or floats
in performance-critical code:
```
class Offsets:
def __init__(self) -> None:
self.offsets: list[int] = []
def add(self, x: int) -> None:
self.offsets.append(x)
def total(self) -> int:
t = 0
for x in self.offsets:
t += x
return t
```
Use `vec[i64]` instead:
```
from librt.vecs import append, vec
from mypy_extensions import i64
class Offsets:
def __init__(self) -> None:
self.offsets = vec[i64]()
def add(self, x: i64) -> None:
self.offsets = append(self.offsets, x)
def total(self) -> i64:
t: i64 = 0
for x in self.offsets:
t += x
return t
```
Only do this if the list is internal and changing it doesn't change a
public API. Check all uses of the list, including in tests and
non-compiled code. Be careful to preserve behavior:
* The length of a `vec` value can't be changed. `append`, `extend`,
`remove` and `pop` are functions that return a new value, which must be
assigned back (as in `add` above). If other code holds a reference to
the same list (for example, it was passed to another object that also
appends to it), the references no longer share changes. Don't convert
lists that are shared like this.
* `vec` isn't a full sequence type, and it doesn't support all `list`
operations and methods (such as `sort`). Don't pass it to code that
expects a `list`.
* Integer values must fit in the item type (e.g. 64 bits for `i64`).
Arithmetic on native integer types isn't checked for overflow. Use
native integer types for related local variables too (such as `t`
above), since conversions between `int` and native integers have a
cost.
* In free-threaded Python builds, `vec` gives fewer thread safety
guarantees than `list`.
Avoid union types that include vecs, such as `vec[i64] | vec[float]`,
in performance-critical code. Operations on these use slow, generic
operations, which can make the code much slower than with a `list`.
Optional types such as `vec[i64] | None` are also slower than `vec[i64]`,
even after narrowing using `is not None`. If `None` is only used to mean
"no items", use an empty `vec` instead (an empty `vec` is cheap, since it
doesn't allocate a buffer).
Check the `librt.vecs` documentation (`mypyc/doc/librt_vecs.rst` in the
mypy repository) first, and only use `vec` if the project depends on
`librt` (see "Use librt"). The benefit is much smaller for other item
types, such as `str`, which aren't packed, and there is no benefit in
non-compiled code.
## Generators
Generators are efficient in compiled code. Don't replace a generator with
a function that returns a `list`, even if the result is always fully
consumed, as building the list is usually slower.
An exception is a generator that yields values of a type that `vec` (from
`librt.vecs`) stores in a packed representation, such as `i64` or `float`.
Returning a `vec` can be faster than a generator, especially if the caller
also uses native integer types:
```
from typing import Iterator
def evens(a: list[int]) -> Iterator[int]:
for x in a:
if x % 2 == 0:
yield x
```
Return a `vec` instead:
```
from librt.vecs import append, vec
from mypy_extensions import i64
def evens(a: list[int]) -> vec[i64]:
result = vec[i64]()
for x in a:
if x % 2 == 0:
result = append(result, x)
return result
```
This doesn't help for other item types, such as `vec[str]`. Only do this
for internal functions where all callers can be updated, if the values fit
in the fixed-size type, and if the project depends on `librt` (see "Use
librt"). Check the `librt.vecs` documentation first, since `vec` isn't a
full sequence type. Measure the change using a benchmark (see "Measure
Performance").
## Replace `@contextmanager` with a Class
`contextlib.contextmanager` is implemented in non-compiled Python code,
and entering and exiting the context manager involves several slow
operations, even if the decorated generator function is compiled. This is
significant if the `with` statement is on a hot path and the context
manager does little work:
```
from contextlib import contextmanager
from typing import Iterator
@contextmanager
def nested(s: State) -> Iterator[None]:
s.depth += 1
try:
yield
finally:
s.depth -= 1
```
Use a native class with `__enter__` and `__exit__` methods instead. Use
precise types for the `__exit__` arguments (instead of `*args`), as this
may be faster:
```
from types import TracebackType
from typing import final
@final
class nested:
def __init__(self, s: State) -> None:
self.s = s
def __enter__(self) -> None:
self.s.depth += 1
def __exit__(
self,
typ: type[BaseException] | None,
exc: BaseException | None,
tb: TracebackType | None,
) -> None:
self.s.depth -= 1
```
The `with nested(s):` statements don't need to be changed. Be careful to
preserve behavior:
* Code after `yield` in a `finally:` block runs whether or not there was
an exception, and so does `__exit__`. Code after `yield` that isn't
in a `finally:` block only runs if there was no exception, so in
`__exit__`, only run it if `typ is None`.
* If the generator catches exceptions raised at `yield` (using
`except`), the exception is suppressed unless it's re-raised. Return
`True` from `__exit__` to suppress an exception, and make the return
type `bool`.
* If the generator yields a value (used as `with nested(s) as x:`),
return it from `__enter__`.
## Replace itertools and functools with Plain Code
Mypyc compiles `for` loops over lists, `range(...)` and other primitive
types into fast code, but `itertools` and `functools` functions are called
through the generic Python API. Replacing them with plain loops and calls
is often 2x to 10x faster in performance-critical code (and 20x faster in a
benchmark where a callback created using `functools.partial` was replaced
with a direct call):
```
import functools
import itertools
import operator
def total(a: list[int], b: list[int]) -> int:
t = 0
for x in itertools.chain(a, b):
t += x
return t
def grid_sum(a: list[int], b: list[int]) -> int:
t = 0
for x, y in itertools.product(a, b):
t += x * y
return t
def deltas(a: list[int]) -> int:
t = 0
for x, y in itertools.pairwise(a):
t += y - x
return t
def sum_all(a: list[int]) -> int:
return functools.reduce(operator.add, a, 0)
def triple_all(a: list[int]) -> int:
return apply_all(functools.partial(scale, 3), a)
```
Use plain loops, indexing and lambdas instead:
```
def total(a: list[int], b: list[int]) -> int:
t = 0
for x in a:
t += x
for x in b:
t += x
return t
def grid_sum(a: list[int], b: list[int]) -> int:
t = 0
for x in a:
for y in b:
t += x * y
return t
def deltas(a: list[int]) -> int:
t = 0
for i in range(1, len(a)):
t += a[i] - a[i - 1]
return t
def sum_all(a: list[int]) -> int:
t = 0
for x in a:
t += x
return t
def triple_all(a: list[int]) -> int:
return apply_all(lambda x: scale(3, x), a)
```
If the function that receives the callback is simple and in the same
module, calling `scale` directly in a loop instead is even faster.
Be careful to preserve behavior:
* Replacing `itertools.product` with nested loops is only equivalent if the
inner iterable can be iterated multiple times (`product` makes a copy of
each iterable first).
* Indexing only works for sequences such as lists, not for arbitrary
iterables or iterators.
* `functools.partial` evaluates its arguments once when it's created, but a
lambda evaluates the expressions each time it's called. If an argument is
a variable that may be reassigned later, or an expression with side
effects, assign it to a local variable first.
* A `partial` object has attributes such as `func` and `args`, and a
different `repr`. Check that no code depends on these.
Only do this in modules that are compiled, as the plain code can be slower
when it's not compiled.
The speedup depends on the function, the types involved and how much work
each iteration does. For functions not covered above, or if you aren't sure
which replacement to use (e.g. a lambda, a small native class with
`__call__`, or inlining the call into a loop), write a short
microbenchmark that compares the variants first (see "Measure
Performance"), and only refactor if there is a clear win.
## Use librt
The `librt` package has faster alternatives to some standard library
features, optimized for compiled code:
* `librt.strings.StringWriter`: build a `str` (faster than `io.StringIO`
or `"".join(list_of_parts)`).
* `librt.strings.BytesWriter`: build a `bytes` object (faster than
`io.BytesIO`, `bytearray` or `b"".join(...)`).
* `librt.strings` `read_*` and `write_*` functions: read and write packed
binary integers and floats (much faster than `struct`, `int.from_bytes` and
`int.to_bytes`; see "Reading and Writing Binary Data" below).
* `librt.base64`: `b64encode`, `b64decode` and related functions (faster
than the `base64` module).
* `librt.random`: pseudorandom numbers (faster than the `random` module).
* `librt.time.time()`: faster than `time.time()`.
* `librt.threading.Lock`: faster than `threading.Lock`.
* `librt.vecs.vec`: a growable array type with a packed representation
for item types such as `i64` and `float` (see "Use `vec` Instead of
`list` for Packed Item Types").
Example of using `StringWriter`:
```
def join_items(items: list[str]) -> str:
parts = []
for s in items:
parts.append(s)
parts.append(",")
return "".join(parts)
```
Use `StringWriter` instead:
```
from librt.strings import StringWriter
def join_items(items: list[str]) -> str:
w = StringWriter()
for s in items:
w.write(s)
w.append(ord(",")) # ord(...) of a literal is a compile-time constant
return w.getvalue()
```
### Reading and Writing Binary Data
`librt.strings` has `read_*` and `write_*` functions for signed 16, 32 and
64-bit integers (`i16`, `i32`, `i64`) and 32 and 64-bit floats (`f32`,
`f64`), in little-endian (`_le`) and big-endian (`_be`) byte order. In
compiled code they are much faster than `struct` and `int.from_bytes`,
which use generic operations:
```
import struct
def total(data: bytes) -> int:
t = 0
for i in range(0, len(data), 4):
t += struct.unpack_from("<i", data, i)[0]
# or: t += int.from_bytes(data[i:i + 4], "little", signed=True)
return t
def encode(items: list[int]) -> bytes:
return b"".join(struct.pack("<i", n) for n in items)
```
Use `read_i32_le` and `write_i32_le` instead:
```
from librt.strings import BytesWriter, read_i32_le, write_i32_le
def total(data: bytes) -> int:
t = 0
for i in range(0, len(data), 4):
t += int(read_i32_le(data, i))
return t
def encode(items: list[int]) -> bytes:
b = BytesWriter()
for n in items:
write_i32_le(b, n)
return b.getvalue()
```
Read single bytes using `data[i]`, and write them using `BytesWriter.append`.
Check these differences before replacing anything:
* The integer `read_*` functions return native integers (such as `i32`).
Arithmetic on native integers that overflows has undefined behavior, so
convert the result using `int(...)` before using it in arithmetic that could
exceed the range, such as computing a sum (as above).
* There are no unsigned variants. To read an unsigned 16 or 32-bit value,
mask the result (`int(read_i32_le(data, i)) & 0xFFFFFFFF`). Leave
unsigned 64-bit values and unsigned writes as they are, unless you can
verify that all values fit in the signed range.
* Only replace `struct` formats with an explicit standard byte order (`<`,
`>` or `!`). Native formats (`@`, `=` or no prefix) can use a different
byte order, size or alignment.
* `read_*` functions only accept `bytes` (not `bytearray` or `memoryview`),
and raise `IndexError` instead of `struct.error` if there isn't enough
data. Passing a value that doesn't fit to a `write_*` function raises
`ValueError` instead of `struct.error`. Check whether any code catches
`struct.error`.
These aren't always drop-in replacements. Check the librt documentation
(`mypyc/doc/librt*.rst` in the mypy repository, or
https://mypyc.readthedocs.io/) before replacing anything. Examples of
differences:
* The `librt.base64` decode functions behave differently for malformed data,
and don't support the `altchars` and `validate` arguments.
* `librt.random` produces different values than `random` for the same seed,
and isn't suitable for cryptographic use (neither is `random`).
* Calls to `librt.time.time()` can't be monkey patched in tests.
* `librt.threading.Lock` doesn't support timeouts, subclassing or use with
`threading.Condition`.
Only use `librt` if the project already depends on it (or you add it as a
dependency). It's installed together with mypy, but compiled code
deployed without mypy needs an explicit dependency.
## Tune Garbage Collection
Compiled code often allocates objects faster than non-compiled code, so a
larger fraction of CPU time can be spent in the CPython garbage collector
(GC). Only consider tuning the GC if the GC shows up as expensive in a CPU
profile of a realistic workload (look for CPython functions related to
garbage collection, which usually have names that start with `gc_`).
Options include making collections less frequent (`gc.set_threshold`),
moving long-lived objects out of GC tracking after startup
(`gc.freeze()`), or disabling the GC during an allocation-heavy phase
(`gc.disable()` followed by `gc.enable()`). The effects vary a lot between
Python versions, since the GC implementation changes between releases,
and some of these can make performance worse or increase memory use.
Measure the effect using a realistic workload.
GC settings affect the whole process, so they belong in the application
entry point, not in library code. Suggest the change to the user instead
of making it, unless you were asked to tune the application as a whole.
## Report Other Potential Bottlenecks
Some code patterns are slow in compiled code, but replacing them often
requires non-trivial tradeoffs, such as changing an API, losing
flexibility, or duplicating code. Don't change these. Instead, report them
to the user as potential bottlenecks if they are on a hot path, and
explain what could be done:
* Calls to functions that take `*args` or `**kwargs`, and calls that use
`*args` or `**kwargs` to pass arguments. These use slow, generic calls
instead of fast, direct calls. Explicit parameters would be faster.
* Functions decorated using wrapper decorators (decorators other than
special-cased ones such as `@property`, `@staticmethod` and
`@classmethod`), for example decorators used for logging,
tracing, caching or validation. Calls to decorated functions go through
the wrapper using generic calls, and often also use `*args` and
`**kwargs`, which can make each call much slower than calling an
undecorated function. Possible fixes include calling an undecorated
helper from the hot path, or removing the decorator where it's not
needed. If the decorator runs code before and after each call (such as
for tracing or timing), a `with` statement in the function body that
uses a native context manager class (see "Replace `@contextmanager`
with a Class") can be much faster:
```
@traced
def parse(s: str) -> Node:
...
```
Use a `with` statement instead (`tracing` is a native class with
`__enter__` and `__exit__` methods):
```
def parse(s: str) -> Node:
with tracing("parse"):
...
```
This is only equivalent if the decorator doesn't change the arguments
or the return value, and if nothing depends on the function being
wrapped (for example, by checking for attributes added by the
decorator).