init
This commit is contained in:
commit
38355d2442
9083 changed files with 1225834 additions and 0 deletions
20
.venv/lib/python3.8/site-packages/mypyc/doc/Makefile
Normal file
20
.venv/lib/python3.8/site-packages/mypyc/doc/Makefile
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line, and also
|
||||
# from the environment for the first two.
|
||||
SPHINXOPTS ?=
|
||||
SPHINXBUILD ?= sphinx-build
|
||||
SOURCEDIR = .
|
||||
BUILDDIR = _build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
Binary file not shown.
|
|
@ -0,0 +1,27 @@
|
|||
.. _bool-ops:
|
||||
|
||||
Native boolean operations
|
||||
=========================
|
||||
|
||||
Operations on ``bool`` values that are listed here have fast,
|
||||
optimized implementations.
|
||||
|
||||
Construction
|
||||
------------
|
||||
|
||||
* ``True``
|
||||
* ``False``
|
||||
* ``bool(obj)``
|
||||
|
||||
Operators
|
||||
---------
|
||||
|
||||
* ``b1 and b2``
|
||||
* ``b1 or b2``
|
||||
* ``not b``
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
* ``any(expr for ... in ...)``
|
||||
* ``all(expr for ... in ...)``
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
.. _compilation-units:
|
||||
|
||||
Compilation units
|
||||
=================
|
||||
|
||||
When you run mypyc to compile a set of modules, these modules form a
|
||||
*compilation unit*. Mypyc will use early binding for references within
|
||||
the compilation unit.
|
||||
|
||||
If you run mypyc multiple times to compile multiple sets of modules,
|
||||
each invocation will result in a new compilation unit. References
|
||||
between separate compilation units will fall back to late binding,
|
||||
i.e. looking up names using Python namespace dictionaries. Also, all
|
||||
calls will use the slower Python calling convention, where arguments
|
||||
and the return value will be boxed (and potentially unboxed again in
|
||||
the called function).
|
||||
|
||||
For maximal performance, minimize interactions across compilation
|
||||
units. The simplest way to achieve this is to compile your entire
|
||||
program as a single compilation unit.
|
||||
64
.venv/lib/python3.8/site-packages/mypyc/doc/conf.py
Normal file
64
.venv/lib/python3.8/site-packages/mypyc/doc/conf.py
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file only contains a selection of the most common options. For a full
|
||||
# list see the documentation:
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html
|
||||
|
||||
import sys
|
||||
import os
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
sys.path.insert(0, os.path.abspath('../..'))
|
||||
|
||||
from mypy.version import __version__ as mypy_version
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = 'mypyc'
|
||||
copyright = '2020, mypyc team'
|
||||
author = 'mypyc team'
|
||||
|
||||
# The version info for the project you're documenting, acts as replacement for
|
||||
# |version| and |release|, also used in various other places throughout the
|
||||
# built documents.
|
||||
#
|
||||
# The short X.Y version.
|
||||
version = mypy_version.split('-')[0]
|
||||
# The full version, including alpha/beta/rc tags.
|
||||
release = mypy_version
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [ # type: ignore
|
||||
]
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['_templates']
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path.
|
||||
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
try:
|
||||
import sphinx_rtd_theme # type: ignore
|
||||
except:
|
||||
html_theme = 'default'
|
||||
else:
|
||||
html_theme = 'sphinx_rtd_theme'
|
||||
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ['_static']
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
# Timings of CPython Operations
|
||||
|
||||
Here are some *very rough* approximate timings of CPython interpreter
|
||||
operations:
|
||||
|
||||
* `f(1)` (empty function body): 70-90ns
|
||||
* `f(n=1)` (empty function body): 90-110ns
|
||||
* `o.x`: 30-40ns
|
||||
* `o.f(1)` (empty method body): 80-160ns
|
||||
* `Cls(1)` (initialize attribute in `__init__`): 290-330ns
|
||||
* `x + y` (integers): 20-35ns
|
||||
* `a[i]` (list) : 20-40ns
|
||||
* `[i]` (also dealloc): 35-55ns
|
||||
* `a.append(i)` (list, average over 5 appends): 70ns
|
||||
* `d[s]` (dict, shared str key): 20ns
|
||||
* `d[s] = i` (dict, shared str key): 40ns
|
||||
* `isinstance(x, A)`: 100ns
|
||||
* `(x, y)`: 20-35ns
|
||||
* `x, y = t` (tuple expand): 10ns
|
||||
|
||||
Note that these results are very imprecise due to many factors, but
|
||||
these should give a rough idea of the relative costs of various
|
||||
operations.
|
||||
|
||||
Details: CPython 3.6.2, Macbook Pro 15" (Mid 2015), macOS Sierra
|
||||
548
.venv/lib/python3.8/site-packages/mypyc/doc/dev-intro.md
Normal file
548
.venv/lib/python3.8/site-packages/mypyc/doc/dev-intro.md
Normal file
|
|
@ -0,0 +1,548 @@
|
|||
# Introduction for Mypyc Contributors
|
||||
|
||||
This is a short introduction aimed at anybody who is interested in
|
||||
contributing to mypyc, or anybody who is curious to understand how
|
||||
mypyc works internally.
|
||||
|
||||
## Key Differences from Python
|
||||
|
||||
Code compiled using mypyc is often much faster than CPython since it
|
||||
does these things differently:
|
||||
|
||||
* Mypyc generates C that is compiled to native code, instead of
|
||||
compiling to interpreted byte code, which CPython uses. Interpreted
|
||||
byte code always has some interpreter overhead, which slows things
|
||||
down.
|
||||
|
||||
* Mypyc doesn't let you arbitrarily monkey patch classes and functions
|
||||
in compiled modules. This allows *early binding* -- mypyc
|
||||
statically binds calls to compiled functions, instead of going
|
||||
through a namespace dictionary. Mypyc can also call methods of
|
||||
compiled classes using vtables, which are more efficient than
|
||||
dictionary lookups used by CPython.
|
||||
|
||||
* Mypyc compiles classes to C extension classes, which are generally
|
||||
more efficient than normal Python classes. They use an efficient,
|
||||
fixed memory representation (essentially a C struct). This lets us
|
||||
use direct memory access instead of (typically) two hash table
|
||||
lookups to access an attribute.
|
||||
|
||||
* As a result of early binding, compiled code can use C calls to call
|
||||
compiled functions. Keyword arguments can be translated to
|
||||
positional arguments during compilation. Thus most calls to native
|
||||
functions and methods directly map to simple C calls. CPython calls
|
||||
are quite expensive, since mapping of keyword arguments, `*args`,
|
||||
and so on has to mostly happen at runtime.
|
||||
|
||||
* Compiled code has runtime type checks to ensure that runtimes types
|
||||
match the declared static types. Compiled code can thus make
|
||||
assumptions about the types of expressions, resulting in both faster
|
||||
and smaller code, since many runtime type checks performed by the
|
||||
CPython interpreter can be omitted.
|
||||
|
||||
* Compiled code can often use unboxed (not heap allocated)
|
||||
representations for integers, booleans and tuples.
|
||||
|
||||
## Supported Python Features
|
||||
|
||||
Mypyc supports a large subset of Python. Note that if you try to
|
||||
compile something that is not supported, you may not always get a very
|
||||
good error message.
|
||||
|
||||
Here are some major things that aren't yet supported in compiled code:
|
||||
|
||||
* Many dunder methods (only some work, such as `__init__` and `__eq__`)
|
||||
* Monkey patching compiled functions or classes
|
||||
* General multiple inheritance (a limited form is supported)
|
||||
* Named tuple defined using the class-based syntax
|
||||
* Defining protocols
|
||||
|
||||
We are generally happy to accept contributions that implement new Python
|
||||
features.
|
||||
|
||||
## Development Environment
|
||||
|
||||
First you should set up the mypy development environment as described in
|
||||
the [mypy docs](https://github.com/python/mypy/blob/master/README.md).
|
||||
macOS, Linux and Windows are supported.
|
||||
|
||||
## Compiling and Running Programs
|
||||
|
||||
When working on a mypyc feature or a fix, you'll often need to run
|
||||
compiled code. For example, you may want to do interactive testing or
|
||||
to run benchmarks. This is also handy if you want to inspect the
|
||||
generated C code (see Inspecting Generated C).
|
||||
|
||||
Run `mypyc` to compile a module to a C extension using your
|
||||
development version of mypyc:
|
||||
|
||||
```
|
||||
$ mypyc program.py
|
||||
```
|
||||
|
||||
This will generate a C extension for `program` in the current working
|
||||
directory. For example, on a Linux system the generated file may be
|
||||
called `program.cpython-37m-x86_64-linux-gnu.so`.
|
||||
|
||||
Since C extensions can't be run as programs, use `python3 -c` to run
|
||||
the compiled module as a program:
|
||||
|
||||
```
|
||||
$ python3 -c "import program"
|
||||
```
|
||||
|
||||
Note that `__name__` in `program.py` will now be `program`, not
|
||||
`__main__`!
|
||||
|
||||
You can manually delete the C extension to get back to an interpreted
|
||||
version (this example works on Linux):
|
||||
|
||||
```
|
||||
$ rm program.*.so
|
||||
```
|
||||
|
||||
Another option is to invoke mypyc through tests (see Testing below).
|
||||
|
||||
## High-level Overview of Mypyc
|
||||
|
||||
Mypyc compiles a Python module (or a set of modules) to C, and
|
||||
compiles the generated C to a Python C extension module (or
|
||||
modules). You can compile only a subset of your program to C --
|
||||
compiled and interpreted code can freely and transparently
|
||||
interact. You can also freely use any Python libraries (including C
|
||||
extensions) in compiled code.
|
||||
|
||||
Mypyc will only make compiled code faster. To see a significant
|
||||
speedup, you must make sure that most of the time is spent in compiled
|
||||
code -- and not in libraries, for example.
|
||||
|
||||
Mypyc has these passes:
|
||||
|
||||
* Type check the code using mypy and infer types for variables and
|
||||
expressions. This produces a mypy AST (defined in `mypy.nodes`) and
|
||||
a type map that describes the inferred types (`mypy.types.Type`) of
|
||||
all expressions (as PEP 484 types).
|
||||
|
||||
* Translate the mypy AST into a mypyc-specific intermediate representation (IR).
|
||||
* The IR is defined in `mypyc.ir` (see below for an explanation of the IR).
|
||||
* Various primitive operations used in the IR are defined in `mypyc.primitives`.
|
||||
* The translation to IR happens in `mypyc.irbuild`. The top-level logic is in
|
||||
`mypyc.irbuild.main`.
|
||||
|
||||
* Insert checks for uses of potentially uninitialized variables
|
||||
(`mypyc.transform.uninit`).
|
||||
|
||||
* Insert exception handling (`mypyc.transform.exceptions`).
|
||||
|
||||
* Insert explicit reference count inc/dec opcodes (`mypyc.transform.refcount`).
|
||||
|
||||
* Translate the IR into C (`mypyc.codegen`).
|
||||
|
||||
* Compile the generated C code using a C compiler (`mypyc.build`).
|
||||
|
||||
## Useful Background Information
|
||||
|
||||
Beyond the mypy documentation, here are some things that are helpful to
|
||||
know for mypyc contributors:
|
||||
|
||||
* Experience with C
|
||||
([The C Programming Language](https://en.wikipedia.org/wiki/The_C_Programming_Language)
|
||||
is a classic book about C)
|
||||
* Basic familiarity with the Python C API (see
|
||||
[Python C API documentation](https://docs.python.org/3/c-api/intro.html)). [Extending and Embedding the Python Interpreter](https://docs.python.org/3/extending/index.html) is a good tutorial for beginners.
|
||||
* Basics of compilers (see the
|
||||
[mypy wiki](https://github.com/python/mypy/wiki/Learning-Resources)
|
||||
for some ideas)
|
||||
|
||||
## Mypyc Intermediate Representation (IR)
|
||||
|
||||
The mypyc IR is defined in `mypyc.ir`. It covers several key concepts
|
||||
that are essential to understand by all mypyc contributors:
|
||||
|
||||
* `mypyc.ir.ops.Op` is an Abstract Base Class for all IR
|
||||
operations. These are low-level and generally map to simple
|
||||
fragments of C each. Mypy expressions are translated to
|
||||
linear sequences of these ops.
|
||||
|
||||
* `mypyc.ir.ops.BasicBlock` is a container of a sequence of ops with a
|
||||
branch/goto/return at the end, and no branch/goto/return ops in the
|
||||
middle. Each function is compiled to a bunch of basic blocks.
|
||||
|
||||
* `mypyc.ir.rtypes.RType` and its subclasses are the types used for
|
||||
everything in the IR. These are lower-level and simpler than mypy or
|
||||
PEP 484 types. For example, there are no general-purpose generic
|
||||
types types here. Each `List[X]` type (for any `X`) is represented
|
||||
by a single `list` type, for example.
|
||||
|
||||
* Primitive types are special RTypes of which mypyc has some special
|
||||
understanding, and there are typically some specialized
|
||||
ops. Examples include `int` (referred to as `int_rprimitive` in the
|
||||
code) and `list` (`list_rprimitive`). Python types for which there
|
||||
is no specific RType type will be represented by the catch-all
|
||||
`object_rprimitive` type.
|
||||
|
||||
* Instances of compiled classes are generally represented using the
|
||||
`RInstance` type. Classes are compiled to C extension classes and
|
||||
contain vtables for fast method calls and fast attribute access.
|
||||
|
||||
* IR representations of functions and classes live in
|
||||
`mypyc.ir.func_ir` and `mypyc.ir.class_ir`, respectively.
|
||||
|
||||
Look at the docstrings and comments in `mypyc.ir` for additional
|
||||
information. See the test cases in
|
||||
`mypyc/test-data/irbuild-basic.test` for examples of what the IR looks
|
||||
like in a pretty-printed form.
|
||||
|
||||
## Testing overview
|
||||
|
||||
Most mypyc test cases are defined in the same format (`.test`) as used
|
||||
for test cases for mypy. Look at mypy developer documentation for a
|
||||
general overview of how things work. Test cases live under
|
||||
`mypyc/test-data/`, and you can run all mypyc tests via `pytest
|
||||
-q mypyc`. If you don't make changes to code under `mypy/`, it's not
|
||||
important to regularly run mypy tests during development.
|
||||
|
||||
When you create a PR, we have Continuous Integration jobs set up that
|
||||
compile mypy using mypyc and run the mypy test suite using the
|
||||
compiled mypy. This will sometimes catch additional issues not caught
|
||||
by the mypyc test suite. It's okay to not do this in your local
|
||||
development environment.
|
||||
|
||||
We discuss writing tests in more detail later in this document.
|
||||
|
||||
## Inspecting Generated IR
|
||||
|
||||
It's often useful to look at the generated IR when debugging issues or
|
||||
when trying to understand how mypyc compiles some code. When you
|
||||
compile some module by running `mypyc`, mypyc will write the
|
||||
pretty-printed IR into `build/ops.txt`. This is the final IR that
|
||||
includes the output from exception and reference count handling
|
||||
insertion passes.
|
||||
|
||||
We also have tests that verify the generate IR
|
||||
(`mypyc/test-data/irbuild-*.text`).
|
||||
|
||||
## Type-checking Mypyc
|
||||
|
||||
`./runtests.py self` type checks mypy and mypyc. This is pretty slow,
|
||||
however, since it's using an uncompiled mypy.
|
||||
|
||||
Installing a released version of mypy using `pip` (which is compiled)
|
||||
and using `dmypy` (mypy daemon) is a much, much faster way to type
|
||||
check mypyc during development.
|
||||
|
||||
## Value Representation
|
||||
|
||||
Mypyc uses a tagged pointer representation for values of type `int`
|
||||
(`CPyTagged`), `char` for booleans, and C structs for tuples. For most
|
||||
other objects mypyc uses the CPython `PyObject *`.
|
||||
|
||||
Python integers that fit in 31/63 bits (depending on whether we are on
|
||||
a 32-bit or 64-bit platform) are represented as C integers
|
||||
(`CPyTagged`) shifted left by 1. Integers that don't fit in this
|
||||
representation are represented as pointers to a `PyObject *` (this is
|
||||
always a Python `int` object) with the least significant bit
|
||||
set. Tagged integer operations are defined in `mypyc/lib-rt/int_ops.c`
|
||||
and `mypyc/lib-rt/CPy.h`.
|
||||
|
||||
There are also low-level integer types, such as `int32` (see
|
||||
`mypyc.ir.rtypes`), that don't use the tagged representation. These
|
||||
types are not exposed to users, but they are used in generated code.
|
||||
|
||||
## Overview of Generated C
|
||||
|
||||
Mypyc compiles a function into two functions, a native function and
|
||||
a wrapper function:
|
||||
|
||||
* The native function takes a fixed number of C arguments with the
|
||||
correct C types. It assumes that all argument have correct types.
|
||||
|
||||
* The wrapper function conforms to the Python C API calling convention
|
||||
and takes an arbitrary set of arguments. It processes the arguments,
|
||||
checks their types, unboxes values with special representations and
|
||||
calls the native function. The return value from the native function
|
||||
is translated back to a Python object ("boxing").
|
||||
|
||||
Calls to other compiled functions don't go through the Python module
|
||||
namespace but directly call the target native C function. This makes
|
||||
calls very fast compared to CPython.
|
||||
|
||||
The generated code does runtime checking so that it can assume that
|
||||
values always have the declared types. Whenever accessing CPython
|
||||
values which might have unexpected types we need to insert a runtime
|
||||
type check operation. For example, when getting a list item we need to
|
||||
insert a runtime type check (an unbox or a cast operation), since
|
||||
Python lists can contain arbitrary objects.
|
||||
|
||||
The generated code uses various helpers defined in
|
||||
`mypyc/lib-rt/CPy.h`. The implementations are in various `.c` files
|
||||
under `mypyc/lib-rt`.
|
||||
|
||||
## Inspecting Generated C
|
||||
|
||||
It's often useful to inspect the C code genenerate by mypyc to debug
|
||||
issues. Mypyc stores the generated C code as `build/__native.c`.
|
||||
Compiled native functions have the prefix `CPyDef_`, while wrapper
|
||||
functions used for calling functions from interpreted Python code have
|
||||
the `CPyPy_` prefix.
|
||||
|
||||
## Other Important Limitations
|
||||
|
||||
All of these limitations will likely be fixed in the future:
|
||||
|
||||
* We don't detect stack overflows.
|
||||
|
||||
* We don't handle Ctrl-C in compiled code.
|
||||
|
||||
## Hints for Implementing Typical Mypyc Features
|
||||
|
||||
This section gives an overview of where to look for and
|
||||
what to do to implement specific kinds of mypyc features.
|
||||
|
||||
### Testing
|
||||
|
||||
Our bread-and-butter testing strategy is compiling code with mypyc and
|
||||
running it. There are downsides to this (kind of slow, tests a huge
|
||||
number of components at once, insensitive to the particular details of
|
||||
the IR), but there really is no substitute for running code. You can
|
||||
also write tests that test the generated IR, however.
|
||||
|
||||
### Tests that compile and run code
|
||||
|
||||
Test cases that compile and run code are located in
|
||||
`mypyc/test-data/run*.test` and the test runner is in
|
||||
`mypyc.test.test_run`. The code to compile comes after `[case
|
||||
test<name>]`. The code gets saved into the file `native.py`, and it
|
||||
gets compiled into the module `native`.
|
||||
|
||||
Each test case uses a non-compiled Python driver that imports the
|
||||
`native` module and typically calls some compiled functions. Some
|
||||
tests also perform assertions and print messages in the driver.
|
||||
|
||||
If you don't provide a driver, a default driver is used. The default
|
||||
driver just calls each module-level function that is prefixed with
|
||||
`test_` and reports any uncaught exceptions as failures. (Failure to
|
||||
build or a segfault also count as failures.) `testStringOps` in
|
||||
`mypyc/test-data/run-strings.test` is an example of a test that uses
|
||||
the default driver.
|
||||
|
||||
You should usually use the default driver (don't include
|
||||
`driver.py`). It's the simplest way to write most tests.
|
||||
|
||||
Here's an example test case that uses the default driver:
|
||||
|
||||
```
|
||||
[case testConcatenateLists]
|
||||
def test_concat_lists() -> None:
|
||||
assert [1, 2] + [5, 6] == [1, 2, 5, 6]
|
||||
|
||||
def test_concat_empty_lists() -> None:
|
||||
assert [] + [] == []
|
||||
```
|
||||
|
||||
There is one test case, `testConcatenateLists`. It has two sub-cases,
|
||||
`test_concat_lists` and `test_concat_empty_lists`. Note that you can
|
||||
use the pytest -k argument to only run `testConcetanateLists`, but you
|
||||
can't filter tests at the sub-case level.
|
||||
|
||||
It's recommended to have multiple sub-cases per test case, since each
|
||||
test case has significant fixed overhead. Each test case is run in a
|
||||
fresh Python subprocess.
|
||||
|
||||
Many of the existing test cases provide a custom driver by having
|
||||
`[file driver.py]`, followed by the driver implementation. Here the
|
||||
driver is not compiled, which is useful if you want to test
|
||||
interactions between compiled and non-compiled code. However, many of
|
||||
the tests don't have a good reason to use a custom driver -- when they
|
||||
were written, the default driver wasn't available.
|
||||
|
||||
Test cases can also have a `[out]` section, which specifies the
|
||||
expected contents of stdout the test case should produce. New test
|
||||
cases should prefer assert statements to `[out]` sections.
|
||||
|
||||
### IR tests
|
||||
|
||||
If the specifics of the generated IR of a change is important
|
||||
(because, for example, you want to make sure a particular optimization
|
||||
is triggering), you should add a `mypyc.irbuild` test as well. Test
|
||||
cases are located in `mypyc/test-data/irbuild-*.test` and the test
|
||||
driver is in `mypyc.test.test_irbuild`. IR build tests do a direct
|
||||
comparison of the IR output, so try to make the test as targeted as
|
||||
possible so as to capture only the important details. (Many of our
|
||||
existing IR build tests do not follow this advice, unfortunately!)
|
||||
|
||||
If you pass the `--update-data` flag to pytest, it will automatically
|
||||
update the expected output of any tests to match the actual
|
||||
output. This is very useful for changing or creating IR build tests,
|
||||
but make sure to carefully inspect the diff!
|
||||
|
||||
You may also need to add some definitions to the stubs used for
|
||||
builtins during tests (`mypyc/test-data/fixtures/ir.py`). We don't use
|
||||
full typeshed stubs to run tests since they would seriously slow down
|
||||
tests.
|
||||
|
||||
### Benchmarking
|
||||
|
||||
Many mypyc improvements attempt to make some operations faster. For
|
||||
any such change, you should run some measurements to verify that
|
||||
there actually is a measurable performance impact.
|
||||
|
||||
A typical benchmark would initialize some data to be operated on, and
|
||||
then measure time spent in some function. In particular, you should
|
||||
not measure time needed to run the entire benchmark program, as this
|
||||
would include Python startup overhead and other things that aren't
|
||||
relevant. In general, for microbenchmarks, you want to do as little as
|
||||
possible in the timed portion. So ideally you'll just have some loops
|
||||
and the code under test. Be ready to provide your benchmark in code
|
||||
review so that mypyc developers can check that the benchmark is fine
|
||||
(writing a good benchmark is non-trivial).
|
||||
|
||||
You should run a benchmark at least five times, in both original and
|
||||
changed versions, ignore outliers, and report the average
|
||||
runtime. Actual performance of a typical desktop or laptop computer is
|
||||
quite variable, due to dynamic CPU clock frequency changes, background
|
||||
processes, etc. If you observe a high variance in timings, you'll need
|
||||
to run the benchmark more times. Also try closing most applications,
|
||||
including web browsers.
|
||||
|
||||
Interleave original and changed runs. Don't run 10 runs with variant A
|
||||
followed by 10 runs with variant B, but run an A run, a B run, an A
|
||||
run, etc. Otherwise you risk that the CPU frequency will be different
|
||||
between variants. You can also try adding a delay of 5 to 20s between
|
||||
runs to avoid CPU frequency changes.
|
||||
|
||||
Instead of averaging over many measurements, you can try to adjust
|
||||
your environment to provide more stable measurements. However, this
|
||||
can be hard to do with some hardware, including many laptops. Victor
|
||||
Stinner has written a series of blog posts about making measurements
|
||||
stable:
|
||||
|
||||
* https://vstinner.github.io/journey-to-stable-benchmark-system.html
|
||||
* https://vstinner.github.io/journey-to-stable-benchmark-average.html
|
||||
|
||||
### Adding C Helpers
|
||||
|
||||
If you add an operation that compiles into a lot of C code, you may
|
||||
also want to add a C helper function for the operation to make the
|
||||
generated code smaller. Here is how to do this:
|
||||
|
||||
* Declare the operation in `mypyc/lib-rt/CPy.h`. We avoid macros, and
|
||||
we generally avoid inline functions to make it easier to target
|
||||
additional backends in the future.
|
||||
|
||||
* Consider adding a unit test for your C helper in `mypyc/lib-rt/test_capi.cc`.
|
||||
We use
|
||||
[Google Test](https://github.com/google/googletest) for writing
|
||||
tests in C++. The framework is included in the repository under the
|
||||
directory `googletest/`. The C unit tests are run as part of the
|
||||
pytest test suite (`test_c_unit_test`).
|
||||
|
||||
### Adding a Specialized Primitive Operation
|
||||
|
||||
Mypyc speeds up operations on primitive types such as `list` and `int`
|
||||
by having primitive operations specialized for specific types. These
|
||||
operations are declared in `mypyc.primitives` (and
|
||||
`mypyc/lib-rt/CPy.h`). For example, `mypyc.primitives.list_ops`
|
||||
contains primitives that target list objects.
|
||||
|
||||
The operation definitions are data driven: you specify the kind of
|
||||
operation (such as a call to `builtins.len` or a binary addition) and
|
||||
the operand types (such as `list_primitive`), and what code should be
|
||||
generated for the operation. Mypyc does AST matching to find the most
|
||||
suitable primitive operation automatically.
|
||||
|
||||
Look at the existing primitive definitions and the docstrings in
|
||||
`mypyc.primitives.registry` for examples and more information.
|
||||
|
||||
### Adding a New Primitive Type
|
||||
|
||||
Some types (typically Python Python built-in types), such as `int` and
|
||||
`list`, are special cased in mypyc to generate optimized operations
|
||||
specific to these types. We'll occasionally want to add additional
|
||||
primitive types.
|
||||
|
||||
Here are some hints about how to add support for a new primitive type
|
||||
(this may be incomplete):
|
||||
|
||||
* Decide whether the primitive type has an "unboxed" representation (a
|
||||
representation that is not just `PyObject *`). For most types we'll
|
||||
use a boxed representation, as it's easier to implement and more
|
||||
closely matches Python semantics.
|
||||
|
||||
* Create a new instance of `RPrimitive` to support the primitive type
|
||||
and add it to `mypyc.ir.rtypes`. Make sure all the attributes are
|
||||
set correctly and also define `<foo>_rprimitive` and
|
||||
`is_<foo>_rprimitive`.
|
||||
|
||||
* Update `mypyc.irbuild.mapper.Mapper.type_to_rtype()`.
|
||||
|
||||
* If the type is not unboxed, update `emit_cast` in `mypyc.codegen.emit`.
|
||||
|
||||
If the type is unboxed, there are some additional steps:
|
||||
|
||||
* Update `emit_box` in `mypyc.codegen.emit`.
|
||||
|
||||
* Update `emit_unbox` in `mypyc.codegen.emit`.
|
||||
|
||||
* Update `emit_inc_ref` and `emit_dec_ref` in `mypypc.codegen.emit`.
|
||||
If the unboxed representation does not need reference counting,
|
||||
these can be no-ops.
|
||||
|
||||
* Update `emit_error_check` in `mypyc.codegen.emit`.
|
||||
|
||||
* Update `emit_gc_visit` and `emit_gc_clear` in `mypyc.codegen.emit`
|
||||
if the type has an unboxed representation with pointers.
|
||||
|
||||
The above may be enough to allow you to declare variables with the
|
||||
type, pass values around, perform runtime type checks, and use generic
|
||||
fallback primitive operations to perform method calls, binary
|
||||
operations, and so on. You likely also want to add some faster,
|
||||
specialized primitive operations for the type (see Adding a
|
||||
Specialized Primitive Operation above for how to do this).
|
||||
|
||||
Add a test case to `mypyc/test-data/run*.test` to test compilation and
|
||||
running compiled code. Ideas for things to test:
|
||||
|
||||
* Test using the type as an argument.
|
||||
|
||||
* Test using the type as a return value.
|
||||
|
||||
* Test passing a value of the type to a function both within
|
||||
compiled code and from regular Python code. Also test this
|
||||
for return values.
|
||||
|
||||
* Test using the type as list item type. Test both getting a list item
|
||||
and setting a list item.
|
||||
|
||||
### Supporting More Python Syntax
|
||||
|
||||
Mypyc supports most Python syntax, but there are still some gaps.
|
||||
|
||||
Support for syntactic sugar that doesn't need additional IR operations
|
||||
typically only requires changes to `mypyc.irbuild`.
|
||||
|
||||
Some new syntax also needs new IR primitives to be added to
|
||||
`mypyc.primitives`. See `mypyc.primitives.registry` for documentation
|
||||
about how to do this.
|
||||
|
||||
### Other Hints
|
||||
|
||||
* This developer documentation is not aimed to be very complete. Much
|
||||
of our documentation is in comments and docstring in the code. If
|
||||
something is unclear, study the code.
|
||||
|
||||
* It can be useful to look through some recent PRs to get an idea of
|
||||
what typical code changes, test cases, etc. look like.
|
||||
|
||||
* Feel free to open GitHub issues with questions if you need help when
|
||||
contributing, or ask questions in existing issues. Note that we only
|
||||
support contributors. Mypyc is not (yet) an end-user product. You
|
||||
can also ask questions in our Gitter chat
|
||||
(https://gitter.im/mypyc-dev/community).
|
||||
|
||||
## Undocumented Workflows
|
||||
|
||||
These workflows would be useful for mypyc contributors. We should add
|
||||
them to mypyc developer documentation:
|
||||
|
||||
* How to inspect the generated IR before some transform passes.
|
||||
|
|
@ -0,0 +1,59 @@
|
|||
.. _dict-ops:
|
||||
|
||||
Native dict operations
|
||||
======================
|
||||
|
||||
These ``dict`` operations have fast, optimized implementations. Other
|
||||
dictionary operations use generic implementations that are often slower.
|
||||
|
||||
Construction
|
||||
------------
|
||||
|
||||
Construct dict from keys and values:
|
||||
|
||||
* ``{key: value, ...}``
|
||||
|
||||
Construct empty dict:
|
||||
|
||||
* ``{}``
|
||||
* ``dict()``
|
||||
|
||||
Construct dict from another object:
|
||||
|
||||
* ``dict(d: dict)``
|
||||
* ``dict(x: Iterable)``
|
||||
|
||||
Dict comprehensions:
|
||||
|
||||
* ``{...: ... for ... in ...}``
|
||||
* ``{...: ... for ... in ... if ...}``
|
||||
|
||||
Operators
|
||||
---------
|
||||
|
||||
* ``d[key]``
|
||||
* ``value in d``
|
||||
|
||||
Statements
|
||||
----------
|
||||
|
||||
* ``d[key] = value``
|
||||
* ``for key in d:``
|
||||
|
||||
Methods
|
||||
-------
|
||||
|
||||
* ``d.get(key)``
|
||||
* ``d.get(key, default)``
|
||||
* ``d.keys()``
|
||||
* ``d.values()``
|
||||
* ``d.items()``
|
||||
* ``d.copy()``
|
||||
* ``d.clear()``
|
||||
* ``d1.update(d2: dict)``
|
||||
* ``d.update(x: Iterable)``
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
* ``len(d: dict)``
|
||||
|
|
@ -0,0 +1,281 @@
|
|||
.. _differences-from-python:
|
||||
|
||||
Differences from Python
|
||||
=======================
|
||||
|
||||
Mypyc aims to be sufficiently compatible with Python semantics so that
|
||||
migrating code to mypyc often doesn't require major code
|
||||
changes. There are various differences to enable performance gains
|
||||
that you need to be aware of, however.
|
||||
|
||||
This section documents notable differences from Python. We discuss
|
||||
many of them also elsewhere, but it's convenient to have them here in
|
||||
one place.
|
||||
|
||||
Running compiled modules
|
||||
------------------------
|
||||
|
||||
You can't use ``python3 <module>.py`` or ``python3 -m <module>``
|
||||
to run compiled modules. Use ``python3 -c "import <module>"`` instead,
|
||||
or write a wrapper script that imports your module.
|
||||
|
||||
As a side effect, you can't rely on checking the ``__name__`` attribute in compiled
|
||||
code, like this::
|
||||
|
||||
if __name__ == "__main__": # Can't be used in compiled code
|
||||
main()
|
||||
|
||||
Type errors prevent compilation
|
||||
-------------------------------
|
||||
|
||||
You can't compile code that generates mypy type check errors. You can
|
||||
sometimes ignore these with a ``# type: ignore`` comment, but this can
|
||||
result in bad code being generated, and it's considered dangerous.
|
||||
|
||||
.. note::
|
||||
|
||||
In the future, mypyc may reject ``# type: ignore`` comments that
|
||||
may be unsafe.
|
||||
|
||||
Runtime type checking
|
||||
---------------------
|
||||
|
||||
Non-erased types in annotations will be type checked at runtime. For example,
|
||||
consider this function::
|
||||
|
||||
def twice(x: int) -> int:
|
||||
return x * 2
|
||||
|
||||
If you try to call this function with a ``float`` or ``str`` argument,
|
||||
you'll get a type error on the call site, even if the call site is not
|
||||
being type checked::
|
||||
|
||||
twice(5) # OK
|
||||
twice(2.2) # TypeError
|
||||
twice("blah") # TypeError
|
||||
|
||||
Also, values with *inferred* types will be type checked. For example,
|
||||
consider a call to the stdlib function ``socket.gethostname()`` in
|
||||
compiled code. This function is not compiled (no stdlib modules are
|
||||
compiled with mypyc), but mypyc uses a *library stub file* to infer
|
||||
the return type as ``str``. Compiled code calling ``gethostname()``
|
||||
will fail with ``TypeError`` if ``gethostname()`` would return an
|
||||
incompatible value, such as ``None``::
|
||||
|
||||
import socket
|
||||
|
||||
# Fail if returned value is not a str
|
||||
name = socket.gethostname()
|
||||
|
||||
Note that ``gethostname()`` is defined like this in the stub file for
|
||||
``socket`` (in typeshed)::
|
||||
|
||||
def gethostname() -> str: ...
|
||||
|
||||
Thus mypyc verifies that library stub files and annotations in
|
||||
non-compiled code match runtime values. This adds an extra layer of
|
||||
type safety.
|
||||
|
||||
Casts such as ``cast(str, x)`` will also result in strict type
|
||||
checks. Consider this example::
|
||||
|
||||
from typing import cast
|
||||
...
|
||||
x = cast(str, y)
|
||||
|
||||
The last line is essentially equivalent to this Python code when compiled::
|
||||
|
||||
if not isinstance(y, str):
|
||||
raise TypeError(...)
|
||||
x = y
|
||||
|
||||
In interpreted mode ``cast`` does not perform a runtime type check.
|
||||
|
||||
Native classes
|
||||
--------------
|
||||
|
||||
Native classes behave differently from Python classes. See
|
||||
:ref:`native-classes` for the details.
|
||||
|
||||
Primitive types
|
||||
---------------
|
||||
|
||||
Some primitive types behave differently in compiled code to improve
|
||||
performance.
|
||||
|
||||
``int`` objects use an unboxed (non-heap-allocated) representation for small
|
||||
integer values. A side effect of this is that the exact runtime type of
|
||||
``int`` values is lost. For example, consider this simple function::
|
||||
|
||||
def first_int(x: List[int]) -> int:
|
||||
return x[0]
|
||||
|
||||
print(first_int([True])) # Output is 1, instead of True!
|
||||
|
||||
``bool`` is a subclass of ``int``, so the above code is
|
||||
valid. However, when the list value is converted to ``int``, ``True``
|
||||
is converted to the corresponding ``int`` value, which is ``1``.
|
||||
|
||||
Note that integers still have an arbitrary precision in compiled code,
|
||||
similar to normal Python integers.
|
||||
|
||||
Fixed-length tuples are unboxed, similar to integers. The exact type
|
||||
and identity of fixed-length tuples is not preserved, and you can't
|
||||
reliably use ``is`` checks to compare tuples that are used in compiled
|
||||
code.
|
||||
|
||||
.. _early-binding:
|
||||
|
||||
Early binding
|
||||
-------------
|
||||
|
||||
References to functions, types, most attributes, and methods in the
|
||||
same :ref:`compilation unit <compilation-units>` use *early binding*:
|
||||
the target of the reference is decided at compile time, whenever
|
||||
possible. This contrasts with normal Python behavior of *late
|
||||
binding*, where the target is found by a namespace lookup at
|
||||
runtime. Omitting these namespace lookups improves performance, but
|
||||
some Python idioms don't work without changes.
|
||||
|
||||
Note that non-final module-level variables still use late binding.
|
||||
You may want to avoid these in very performance-critical code.
|
||||
|
||||
Examples of early and late binding::
|
||||
|
||||
from typing import Final
|
||||
|
||||
import lib # "lib" is not compiled
|
||||
|
||||
x = 0
|
||||
y: Final = 1
|
||||
|
||||
def func() -> None:
|
||||
pass
|
||||
|
||||
class Cls:
|
||||
def __init__(self, attr: int) -> None:
|
||||
self.attr = attr
|
||||
|
||||
def method(self) -> None:
|
||||
pass
|
||||
|
||||
def example() -> None:
|
||||
# Early binding:
|
||||
var = y
|
||||
func()
|
||||
o = Cls()
|
||||
o.x
|
||||
o.method()
|
||||
|
||||
# Late binding:
|
||||
var = x # Module-level variable
|
||||
lib.func() # Accessing library that is not compiled
|
||||
|
||||
Monkey patching
|
||||
---------------
|
||||
|
||||
Since mypyc function and class definitions are immutable, you can't
|
||||
perform arbitrary monkey patching, such as replacing functions or
|
||||
methods with mocks in tests.
|
||||
|
||||
.. note::
|
||||
|
||||
Each compiled module has a Python namespace that is initialized to
|
||||
point to compiled functions and type objects. This namespace is a
|
||||
regular ``dict`` object, and it *can* be modified. However,
|
||||
compiled code generally doesn't use this namespace, so any changes
|
||||
will only be visible to non-compiled code.
|
||||
|
||||
Stack overflows
|
||||
---------------
|
||||
|
||||
Compiled code currently doesn't check for stack overflows. Your
|
||||
program may crash in an unrecoverable fashion if you have too many
|
||||
nested function calls, typically due to out-of-control recursion.
|
||||
|
||||
.. note::
|
||||
|
||||
This limitation will be fixed in the future.
|
||||
|
||||
Final values
|
||||
------------
|
||||
|
||||
Compiled code replaces a reference to an attribute declared ``Final`` with
|
||||
the value of the attribute computed at compile time. This is an example of
|
||||
:ref:`early binding <early-binding>`. Example::
|
||||
|
||||
MAX: Final = 100
|
||||
|
||||
def limit_to_max(x: int) -> int:
|
||||
if x > MAX:
|
||||
return MAX
|
||||
return x
|
||||
|
||||
The two references to ``MAX`` don't involve any module namespace lookups,
|
||||
and are equivalent to this code::
|
||||
|
||||
def limit_to_max(x: int) -> int:
|
||||
if x > 100:
|
||||
return 100
|
||||
return x
|
||||
|
||||
When run as interpreted, the first example will execute slower due to
|
||||
the extra namespace lookups. In interpreted code final attributes can
|
||||
also be modified.
|
||||
|
||||
Unsupported features
|
||||
--------------------
|
||||
|
||||
Some Python features are not supported by mypyc (yet). They can't be
|
||||
used in compiled code, or there are some limitations. You can
|
||||
partially work around some of these limitations by running your code
|
||||
in interpreted mode.
|
||||
|
||||
Operator overloading
|
||||
********************
|
||||
|
||||
Native classes can only use these dunder methods to override operators:
|
||||
|
||||
* ``__eq__``
|
||||
* ``__ne__``
|
||||
* ``__getitem__``
|
||||
* ``__setitem__``
|
||||
|
||||
.. note::
|
||||
|
||||
This limitation will be lifted in the future.
|
||||
|
||||
Generator expressions
|
||||
*********************
|
||||
|
||||
Generator expressions are not supported. To make it easier to compile
|
||||
existing code, they are implicitly replaced with list comprehensions.
|
||||
*This does not always produce the same behavior.*
|
||||
|
||||
To work around this limitation, you can usually use a generator
|
||||
function instead. You can sometimes replace the generator expression
|
||||
with an explicit list comprehension.
|
||||
|
||||
Descriptors
|
||||
***********
|
||||
|
||||
Native classes can't contain arbitrary descriptors. Properties, static
|
||||
methods and class methods are supported.
|
||||
|
||||
Stack introspection
|
||||
*******************
|
||||
|
||||
Frames of compiled functions can't be inspected using ``inspect``.
|
||||
|
||||
Profiling hooks and tracing
|
||||
***************************
|
||||
|
||||
Compiled functions don't trigger profiling and tracing hooks, such as
|
||||
when using the ``profile``, ``cProfile``, or ``trace`` modules.
|
||||
|
||||
Debuggers
|
||||
*********
|
||||
|
||||
You can't set breakpoints in compiled functions or step through
|
||||
compiled functions using ``pdb``. Often you can debug your code in
|
||||
interpreted mode instead.
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
.. _float-ops:
|
||||
|
||||
Native float operations
|
||||
========================
|
||||
|
||||
These ``float`` operations have fast, optimized implementations. Other
|
||||
floating point operations use generic implementations that are often
|
||||
slower.
|
||||
|
||||
.. note::
|
||||
|
||||
At the moment, only a few float operations are optimized. This will
|
||||
improve in future mypyc releases.
|
||||
|
||||
Construction
|
||||
------------
|
||||
|
||||
* Float literal
|
||||
* ``float(string)``
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
* ``abs(f)``
|
||||
42
.venv/lib/python3.8/site-packages/mypyc/doc/future.md
Normal file
42
.venv/lib/python3.8/site-packages/mypyc/doc/future.md
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
# Future
|
||||
|
||||
This document introduces some ideas for future improvements.
|
||||
|
||||
## Basic Optimizations
|
||||
|
||||
Implement basic optimizations such as common subexpression elimination and
|
||||
loop invariant code motion.
|
||||
|
||||
Importantly, common subexpression elimination could be used to avoid
|
||||
redundant type checks.
|
||||
|
||||
## Operation-specific Optimizations
|
||||
|
||||
Some operations or combinations of successive operations can be
|
||||
replaced with more efficient operations. Examples:
|
||||
|
||||
* If `s` is a string, `s[i] == 'x'` doesn't need to construct the
|
||||
intermediate single-character string object `s[i]` but just compare
|
||||
the character value to `ord('x')`.
|
||||
|
||||
* `a + ':' + b` (two string concetenations) can be implemented as
|
||||
single three-operand concatenation that doesn't construct an
|
||||
intermediate object.
|
||||
|
||||
* `x in {1, 3}` can be translated into `x == 1 or x == 3` (more
|
||||
generally we need to evaluate all right-hand-side items).
|
||||
|
||||
## Integer Range Analysis
|
||||
|
||||
Implement integer range analysis. This can be used in various ways:
|
||||
|
||||
* Use untagged representations for some registers.
|
||||
* Use faster integer arithmetic operations for operations that
|
||||
only deal with short integers or that can't overflow.
|
||||
* Remove redundant list and string index checks.
|
||||
|
||||
## Always Defined Attributes
|
||||
|
||||
Somehow make it possible to enforce that attributes in a class are always
|
||||
defined. This makes attribute access faster since we don't need to explicitly
|
||||
check if the attribute is defined.
|
||||
240
.venv/lib/python3.8/site-packages/mypyc/doc/getting_started.rst
Normal file
240
.venv/lib/python3.8/site-packages/mypyc/doc/getting_started.rst
Normal file
|
|
@ -0,0 +1,240 @@
|
|||
Getting started
|
||||
===============
|
||||
|
||||
Here you will learn some basic things you need to know to get started with mypyc.
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
|
||||
You need a Python C extension development environment. The way to set this up
|
||||
depends on your operating system.
|
||||
|
||||
macOS
|
||||
*****
|
||||
|
||||
Install Xcode command line tools:
|
||||
|
||||
.. code-block::
|
||||
|
||||
$ xcode-select --install
|
||||
|
||||
Linux
|
||||
*****
|
||||
|
||||
You need a C compiler and CPython headers and libraries. The specifics
|
||||
of how to install these varies by distribution. Here are instructions for
|
||||
Ubuntu 18.04, for example:
|
||||
|
||||
.. code-block::
|
||||
|
||||
$ sudo apt install python3-dev
|
||||
|
||||
Windows
|
||||
*******
|
||||
|
||||
Install `Visual C++ <https://www.visualstudio.com/downloads/#build-tools-for-visual-studio-2017>`_.
|
||||
|
||||
Installation
|
||||
------------
|
||||
|
||||
Mypyc is shipped as part of the mypy distribution. Install mypy like
|
||||
this (you need Python 3.5 or later):
|
||||
|
||||
.. code-block::
|
||||
|
||||
$ python3 -m pip install -U mypy
|
||||
|
||||
On some systems you need to use this instead:
|
||||
|
||||
.. code-block::
|
||||
|
||||
$ python -m pip install -U mypy
|
||||
|
||||
Example program
|
||||
---------------
|
||||
|
||||
Let's start with a classic micro-benchmark, recursive fibonacci. Save
|
||||
this file as ``fib.py``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import time
|
||||
|
||||
def fib(n: int) -> int:
|
||||
if n <= 1:
|
||||
return n
|
||||
else:
|
||||
return fib(n - 2) + fib(n - 1)
|
||||
|
||||
t0 = time.time()
|
||||
fib(32)
|
||||
print(time.time() - t0)
|
||||
|
||||
Note that we gave the ``fib`` function a type annotation. Without it,
|
||||
performance won't be as impressive after compilation.
|
||||
|
||||
.. note::
|
||||
|
||||
`Mypy documentation
|
||||
<https://mypy.readthedocs.io/en/stable/index.html>`_ is a good
|
||||
introduction if you are new to type annotations or mypy. Mypyc uses
|
||||
mypy to perform type checking and type inference, so some familiarity
|
||||
with mypy is very useful.
|
||||
|
||||
Compiling and running
|
||||
---------------------
|
||||
|
||||
We can run ``fib.py`` as a regular, interpreted program using CPython:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ python3 fib.py
|
||||
0.4125328063964844
|
||||
|
||||
It took about 0.41s to run on my computer.
|
||||
|
||||
Run ``mypyc`` to compile the program to a binary C extension:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ mypyc fib.py
|
||||
|
||||
This will generate a C extension for ``fib`` in the current working
|
||||
directory. For example, on a Linux system the generated file may be
|
||||
called ``fib.cpython-37m-x86_64-linux-gnu.so``.
|
||||
|
||||
Since C extensions can't be run as programs, use ``python3 -c`` to run
|
||||
the compiled module as a program:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ python3 -c "import fib"
|
||||
0.04097270965576172
|
||||
|
||||
After compilation, the program is about 10x faster. Nice!
|
||||
|
||||
.. note::
|
||||
|
||||
``__name__`` in ``fib.py`` would now be ``"fib"``, not ``"__main__"``.
|
||||
|
||||
You can also pass most
|
||||
`mypy command line options <https://mypy.readthedocs.io/en/stable/command_line.html>`_
|
||||
to ``mypyc``.
|
||||
|
||||
Deleting compiled binary
|
||||
------------------------
|
||||
|
||||
You can manually delete the C extension to get back to an interpreted
|
||||
version (this example works on Linux):
|
||||
|
||||
.. code-block::
|
||||
|
||||
$ rm fib.*.so
|
||||
|
||||
Using setup.py
|
||||
--------------
|
||||
|
||||
You can also use ``setup.py`` to compile modules using mypyc. Here is an
|
||||
example ``setup.py`` file::
|
||||
|
||||
from setuptools import setup
|
||||
|
||||
from mypyc.build import mypycify
|
||||
|
||||
setup(
|
||||
name='mylib',
|
||||
packages=['mylib'],
|
||||
ext_modules=mypycify([
|
||||
'mylib/__init__.py',
|
||||
'mylib/mod.py',
|
||||
]),
|
||||
)
|
||||
|
||||
We used ``mypycify(...)`` to specify which files to compile using
|
||||
mypyc. Your ``setup.py`` can include additional Python files outside
|
||||
``mypycify(...)`` that won't be compiled.
|
||||
|
||||
Now you can build a wheel (.whl) file for the package::
|
||||
|
||||
python3 setup.py bdist_wheel
|
||||
|
||||
The wheel is created under ``dist/``.
|
||||
|
||||
You can also compile the C extensions in-place, in the current directory (similar
|
||||
to using ``mypyc`` to compile modules)::
|
||||
|
||||
python3 setup.py build_ext --inplace
|
||||
|
||||
You can include most `mypy command line options
|
||||
<https://mypy.readthedocs.io/en/stable/command_line.html>`_ in the
|
||||
list of arguments passed to ``mypycify()``. For example, here we use
|
||||
the ``--disallow-untyped-defs`` flag to require that all functions
|
||||
have type annotations::
|
||||
|
||||
...
|
||||
setup(
|
||||
name='frobnicate',
|
||||
packages=['frobnicate'],
|
||||
ext_modules=mypycify([
|
||||
'--disallow-untyped-defs', # Pass a mypy flag
|
||||
'frobnicate.py',
|
||||
]),
|
||||
)
|
||||
|
||||
.. note:
|
||||
|
||||
You may be tempted to use `--check-untyped-defs
|
||||
<https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-check-untyped-defs>`_
|
||||
to type check functions without type annotations. Note that this
|
||||
may reduce performance, due to many transitions between type-checked and unchecked
|
||||
code.
|
||||
|
||||
Recommended workflow
|
||||
--------------------
|
||||
|
||||
A simple way to use mypyc is to always compile your code after any
|
||||
code changes, but this can get tedious, especially if you have a lot
|
||||
of code. Instead, you can do most development in interpreted mode.
|
||||
This development workflow has worked smoothly for developing mypy and
|
||||
mypyc (often we forget that we aren't working on a vanilla Python
|
||||
project):
|
||||
|
||||
1. During development, use interpreted mode. This gives you a fast
|
||||
edit-run cycle.
|
||||
|
||||
2. Use type annotations liberally and use mypy to type check your code
|
||||
during development. Mypy and tests can find most errors that would
|
||||
break your compiled code, if you have good type annotation
|
||||
coverage. (Running mypy is pretty quick.)
|
||||
|
||||
3. After you've implemented a feature or a fix, compile your project
|
||||
and run tests again, now in compiled mode. Usually nothing will
|
||||
break here, assuming your type annotation coverage is good. This
|
||||
can happen locally or in a Continuous Integration (CI) job. If you
|
||||
have CI, compiling locally may be rarely needed.
|
||||
|
||||
4. Release or deploy a compiled version. Optionally, include a
|
||||
fallback interpreted version for platforms that mypyc doesn't
|
||||
support.
|
||||
|
||||
This mypyc workflow only involves minor tweaks to a typical Python
|
||||
workflow. Most of development, testing and debugging happens in
|
||||
interpreted mode. Incremental mypy runs, especially when using the
|
||||
mypy daemon, are very quick (often a few hundred milliseconds).
|
||||
|
||||
Next steps
|
||||
----------
|
||||
|
||||
You can sometimes get good results by just annotating your code and
|
||||
compiling it. If this isn't providing meaningful performance gains, if
|
||||
you have trouble getting your code to work under mypyc, or if you want
|
||||
to optimize your code for maximum performance, you should read the
|
||||
rest of the documentation in some detail.
|
||||
|
||||
Here are some specific recommendations, or you can just read the
|
||||
documentation in order:
|
||||
|
||||
* :ref:`using-type-annotations`
|
||||
* :ref:`native-classes`
|
||||
* :ref:`differences-from-python`
|
||||
* :ref:`performance-tips`
|
||||
55
.venv/lib/python3.8/site-packages/mypyc/doc/index.rst
Normal file
55
.venv/lib/python3.8/site-packages/mypyc/doc/index.rst
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
.. mypyc documentation master file, created by
|
||||
sphinx-quickstart on Sun Apr 5 14:01:55 2020.
|
||||
You can adapt this file completely to your liking, but it should at least
|
||||
contain the root `toctree` directive.
|
||||
|
||||
Welcome to mypyc documentation!
|
||||
===============================
|
||||
|
||||
Mypyc compiles Python modules to C extensions. It uses standard Python
|
||||
`type hints
|
||||
<https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html>`_ to
|
||||
generate fast code.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: First steps
|
||||
|
||||
introduction
|
||||
getting_started
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Using mypyc
|
||||
|
||||
using_type_annotations
|
||||
native_classes
|
||||
differences_from_python
|
||||
compilation_units
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Native operations reference
|
||||
|
||||
native_operations
|
||||
int_operations
|
||||
bool_operations
|
||||
float_operations
|
||||
str_operations
|
||||
list_operations
|
||||
dict_operations
|
||||
set_operations
|
||||
tuple_operations
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Advanced topics
|
||||
|
||||
performance_tips_and_tricks
|
||||
|
||||
Indices and tables
|
||||
==================
|
||||
|
||||
* :ref:`genindex`
|
||||
* :ref:`modindex`
|
||||
* :ref:`search`
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
.. _int-ops:
|
||||
|
||||
Native integer operations
|
||||
=========================
|
||||
|
||||
Operations on ``int`` values that are listed here have fast, optimized
|
||||
implementations. Other integer operations use generic implementations
|
||||
that are often slower. Some operations involving integers and other
|
||||
types are documented elsewhere, such as list indexing.
|
||||
|
||||
Construction
|
||||
------------
|
||||
|
||||
* Integer literal
|
||||
* ``int(x: float)``
|
||||
* ``int(x: str)``
|
||||
* ``int(x: str, base: int)``
|
||||
|
||||
Operators
|
||||
---------
|
||||
|
||||
* Arithmetic (``+``, ``-``, ``*``, ``//``, ``%``)
|
||||
* Bitwise operations (``&``, ``|``, ``^``, ``<<``, ``>>``, ``~``)
|
||||
* Comparisons (``==``, ``!=``, ``<``, etc.)
|
||||
* Augmented assignment (``x += y``, etc.)
|
||||
|
||||
Statements
|
||||
----------
|
||||
|
||||
For loop over range:
|
||||
|
||||
* ``for x in range(end)``
|
||||
* ``for x in range(start, end)``
|
||||
* ``for x in range(start, end, step)``
|
||||
150
.venv/lib/python3.8/site-packages/mypyc/doc/introduction.rst
Normal file
150
.venv/lib/python3.8/site-packages/mypyc/doc/introduction.rst
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
Introduction
|
||||
============
|
||||
|
||||
Mypyc compiles Python modules to C extensions. It uses standard Python
|
||||
`type hints
|
||||
<https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html>`_ to
|
||||
generate fast code.
|
||||
|
||||
The compiled language is a strict, *gradually typed* Python variant. It
|
||||
restricts the use of some dynamic Python features to gain performance,
|
||||
but it's mostly compatible with standard Python.
|
||||
|
||||
Mypyc uses `mypy <http://www.mypy-lang.org/>`_ to perform type
|
||||
checking and type inference. Most type system features in the stdlib
|
||||
`typing <https://docs.python.org/3/library/typing.html>`_ module are
|
||||
supported.
|
||||
|
||||
Compiled modules can import arbitrary Python modules and third-party
|
||||
libraries. You can compile anything from a single performance-critical
|
||||
module to your entire codebase. You can run the modules you compile
|
||||
also as normal, interpreted Python modules.
|
||||
|
||||
Existing code with type annotations is often **1.5x to 5x** faster
|
||||
when compiled. Code tuned for mypyc can be **5x to 10x** faster.
|
||||
|
||||
Mypyc currently aims to speed up non-numeric code, such as server
|
||||
applications. Mypyc is also used to compile itself (and mypy).
|
||||
|
||||
Why mypyc?
|
||||
----------
|
||||
|
||||
**Easy to get started.** Compiled code has the look and feel of
|
||||
regular Python code. Mypyc supports familiar Python syntax and idioms.
|
||||
|
||||
**Expressive types.** Mypyc fully supports standard Python type hints.
|
||||
Mypyc has local type inference, generics, optional types, tuple types,
|
||||
union types, and more. Type hints act as machine-checked
|
||||
documentation, making code not only faster but also easier to
|
||||
understand and modify.
|
||||
|
||||
**Python ecosystem.** Mypyc runs on top of CPython, the
|
||||
standard Python implementation. You can use any third-party libraries,
|
||||
including C extensions, installed with pip. Mypyc uses only valid Python
|
||||
syntax, so all Python editors and IDEs work perfectly.
|
||||
|
||||
**Fast program startup.** Mypyc uses ahead-of-time compilation, so
|
||||
compilation does not slow down program startup. Slow program startup
|
||||
is a common issue with JIT compilers.
|
||||
|
||||
**Migration path for existing code.** Existing Python code often
|
||||
requires only minor changes to compile using mypyc.
|
||||
|
||||
**Waiting for compilation is optional.** Compiled code also runs as
|
||||
normal Python code. You can use interpreted Python during development,
|
||||
with familiar and fast workflows.
|
||||
|
||||
**Runtime type safety.** Mypyc protects you from segfaults and memory
|
||||
corruption. Any unexpected runtime type safety violation is a bug in
|
||||
mypyc. Runtime values are checked against type annotations. (Without
|
||||
mypyc, type annotations are ignored at runtime.)
|
||||
|
||||
**Find errors statically.** Mypyc uses mypy for static type checking
|
||||
that helps catch many bugs.
|
||||
|
||||
Use cases
|
||||
---------
|
||||
|
||||
**Fix only performance bottlenecks.** Often most time is spent in a few
|
||||
Python modules or functions. Add type annotations and compile these
|
||||
modules for easy performance gains.
|
||||
|
||||
**Compile it all.** During development you can use interpreted mode,
|
||||
for a quick edit-run cycle. In releases all non-test code is compiled.
|
||||
This is how mypy achieved a 4x performance improvement over interpreted
|
||||
Python.
|
||||
|
||||
**Take advantage of existing type hints.** If you already use type
|
||||
annotations in your code, adopting mypyc will be easier. You've already
|
||||
done most of the work needed to use mypyc.
|
||||
|
||||
**Alternative to a lower-level language.** Instead of writing
|
||||
performance-critical code in C, C++, Cython or Rust, you may get good
|
||||
performance while staying in the comfort of Python.
|
||||
|
||||
**Migrate C extensions.** Maintaining C extensions is not always fun
|
||||
for a Python developer. With mypyc you may get performance similar to
|
||||
the original C, with the convenience of Python.
|
||||
|
||||
Differences from Cython
|
||||
-----------------------
|
||||
|
||||
Mypyc targets many similar use cases as Cython. Mypyc does many things
|
||||
differently, however:
|
||||
|
||||
* No need to use non-standard syntax, such as ``cpdef``, or extra
|
||||
decorators to get good performance. Clean, normal-looking
|
||||
type-annotated Python code can be fast without language extensions.
|
||||
This makes it practical to compile entire codebases without a
|
||||
developer productivity hit.
|
||||
|
||||
* Mypyc has first-class support for features in the ``typing`` module,
|
||||
such as tuple types, union types and generics.
|
||||
|
||||
* Mypyc has powerful type inference, provided by mypy. Variable type
|
||||
annotations are not needed for optimal performance.
|
||||
|
||||
* Mypyc fully integrates with mypy for robust and seamless static type
|
||||
checking.
|
||||
|
||||
* Mypyc performs strict enforcement of type annotations at runtime,
|
||||
resulting in better runtime type safety and easier debugging.
|
||||
|
||||
Unlike Cython, mypyc doesn't directly support interfacing with C libraries
|
||||
or speeding up numeric code.
|
||||
|
||||
How does it work
|
||||
----------------
|
||||
|
||||
Mypyc uses several techniques to produce fast code:
|
||||
|
||||
* Mypyc uses *ahead-of-time compilation* to native code. This removes
|
||||
CPython interpreter overhead.
|
||||
|
||||
* Mypyc enforces type annotations (and type comments) at runtime,
|
||||
raising ``TypeError`` if runtime values don't match annotations.
|
||||
Value types only need to be checked in the boundaries between
|
||||
dynamic and static typing.
|
||||
|
||||
* Compiled code uses optimized, type-specific primitives.
|
||||
|
||||
* Mypyc uses *early binding* to resolve called functions and name
|
||||
references at compile time. Mypyc avoids many dynamic namespace
|
||||
lookups.
|
||||
|
||||
* Classes are compiled to *C extension classes*. They use `vtables
|
||||
<https://en.wikipedia.org/wiki/Virtual_method_table>`_ for fast
|
||||
method calls and attribute access.
|
||||
|
||||
* Mypyc treats compiled functions, classes, and attributes declared
|
||||
``Final`` as immutable.
|
||||
|
||||
* Mypyc has memory-efficient, unboxed representions for integers and
|
||||
booleans.
|
||||
|
||||
Development status
|
||||
------------------
|
||||
|
||||
Mypyc is currently alpha software. It's only recommended for
|
||||
production use cases with careful testing, and if you are willing to
|
||||
contribute fixes or to work around issues you will encounter.
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
.. _list-ops:
|
||||
|
||||
Native list operations
|
||||
======================
|
||||
|
||||
These ``list`` operations have fast, optimized implementations. Other
|
||||
list operations use generic implementations that are often slower.
|
||||
|
||||
Construction
|
||||
------------
|
||||
|
||||
Construct list with specific items:
|
||||
|
||||
* ``[item0, ..., itemN]``
|
||||
|
||||
Construct empty list:
|
||||
|
||||
* ``[]``
|
||||
* ``list()``
|
||||
|
||||
Construct list from iterable:
|
||||
|
||||
* ``list(x: Iterable)``
|
||||
|
||||
List comprehensions:
|
||||
|
||||
* ``[... for ... in ...]``
|
||||
* ``[... for ... in ... if ...]``
|
||||
|
||||
Operators
|
||||
---------
|
||||
|
||||
* ``lst[n]`` (get item by integer index)
|
||||
* ``lst[n:m]``, ``lst[n:]``, ``lst[:m]``, ``lst[:]`` (slicing)
|
||||
* ``lst * n``, ``n * lst``
|
||||
* ``obj in lst``
|
||||
|
||||
Statements
|
||||
----------
|
||||
|
||||
Set item by integer index:
|
||||
|
||||
* ``lst[n] = x``
|
||||
|
||||
For loop over a list:
|
||||
|
||||
* ``for item in lst:``
|
||||
|
||||
Methods
|
||||
-------
|
||||
|
||||
* ``lst.append(obj)``
|
||||
* ``lst.extend(x: Iterable)``
|
||||
* ``lst.insert(index, obj)``
|
||||
* ``lst.pop(index=-1)``
|
||||
* ``lst.remove(obj)``
|
||||
* ``lst.count(obj)``
|
||||
* ``lst.index(obj)``
|
||||
* ``lst.reverse()``
|
||||
* ``lst.sort()``
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
* ``len(lst: list)``
|
||||
35
.venv/lib/python3.8/site-packages/mypyc/doc/make.bat
Normal file
35
.venv/lib/python3.8/site-packages/mypyc/doc/make.bat
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
@ECHO OFF
|
||||
|
||||
pushd %~dp0
|
||||
|
||||
REM Command file for Sphinx documentation
|
||||
|
||||
if "%SPHINXBUILD%" == "" (
|
||||
set SPHINXBUILD=sphinx-build
|
||||
)
|
||||
set SOURCEDIR=.
|
||||
set BUILDDIR=_build
|
||||
|
||||
if "%1" == "" goto help
|
||||
|
||||
%SPHINXBUILD% >NUL 2>NUL
|
||||
if errorlevel 9009 (
|
||||
echo.
|
||||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
|
||||
echo.installed, then set the SPHINXBUILD environment variable to point
|
||||
echo.to the full path of the 'sphinx-build' executable. Alternatively you
|
||||
echo.may add the Sphinx directory to PATH.
|
||||
echo.
|
||||
echo.If you don't have Sphinx installed, grab it from
|
||||
echo.http://sphinx-doc.org/
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
goto end
|
||||
|
||||
:help
|
||||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
|
||||
:end
|
||||
popd
|
||||
198
.venv/lib/python3.8/site-packages/mypyc/doc/native_classes.rst
Normal file
198
.venv/lib/python3.8/site-packages/mypyc/doc/native_classes.rst
Normal file
|
|
@ -0,0 +1,198 @@
|
|||
.. _native-classes:
|
||||
|
||||
Native classes
|
||||
==============
|
||||
|
||||
Classes in compiled modules are *native classes* by default (some
|
||||
exceptions are discussed below). Native classes are compiled to C
|
||||
extension classes, which have some important differences from normal
|
||||
Python classes. Native classes are similar in many ways to built-in
|
||||
types, such as ``int``, ``str``, and ``list``.
|
||||
|
||||
Immutable namespaces
|
||||
--------------------
|
||||
|
||||
The type object namespace of native classes is mostly immutable (but
|
||||
class variables can be assigned to)::
|
||||
|
||||
class Cls:
|
||||
def method1(self) -> None:
|
||||
print("method1")
|
||||
|
||||
def method2(self) -> None:
|
||||
print("method2")
|
||||
|
||||
Cls.method1 = Cls.method2 # Error
|
||||
Cls.new_method = Cls.method2 # Error
|
||||
|
||||
Only attributes defined within a class definition (or in a base class)
|
||||
can be assigned to (similar to using ``__slots__``)::
|
||||
|
||||
class Cls:
|
||||
x: int
|
||||
|
||||
def __init__(self, y: int) -> None:
|
||||
self.x = 0
|
||||
self.y = y
|
||||
|
||||
def method(self) -> None:
|
||||
self.z = "x"
|
||||
|
||||
o = Cls(0)
|
||||
print(o.x, o.y) # OK
|
||||
o.z = "y" # OK
|
||||
o.extra = 3 # Error: no attribute "extra"
|
||||
|
||||
.. _inheritance:
|
||||
|
||||
Inheritance
|
||||
-----------
|
||||
|
||||
Only single inheritance is supported (except for :ref:`traits
|
||||
<trait-types>`). Most non-native classes can't be used as base
|
||||
classes.
|
||||
|
||||
These non-native classes can be used as base classes of native
|
||||
classes:
|
||||
|
||||
* ``object``
|
||||
* ``dict`` (and ``Dict[k, v]``)
|
||||
* ``BaseException``
|
||||
* ``Exception``
|
||||
* ``ValueError``
|
||||
* ``IndexError``
|
||||
* ``LookupError``
|
||||
* ``UserWarning``
|
||||
|
||||
By default, a non-native class can't inherit a native class, and you
|
||||
can't inherit from a native class outside the compilation unit that
|
||||
defines the class. You can enable these through
|
||||
``mypy_extensions.mypyc_attr``::
|
||||
|
||||
from mypy_extensions import mypyc_attr
|
||||
|
||||
@mypyc_attr(allow_interpreted_subclasses=True)
|
||||
class Cls:
|
||||
...
|
||||
|
||||
Allowing interpreted subclasses has only minor impact on performance
|
||||
of instances of the native class. Accessing methods and attributes of
|
||||
a *non-native* subclass (or a subclass defined in another compilation
|
||||
unit) will be slower, since it needs to use the normal Python
|
||||
attribute access mechanism.
|
||||
|
||||
You need to install ``mypy-extensions`` to use ``@mypyc_attr``:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
pip install --upgrade mypy-extensions
|
||||
|
||||
Class variables
|
||||
---------------
|
||||
|
||||
Class variables must be explicitly declared using ``attr: ClassVar``
|
||||
or ``attr: ClassVar[<type>]``. You can't assign to a class variable
|
||||
through an instance. Example::
|
||||
|
||||
from typing import ClassVar
|
||||
|
||||
class Cls:
|
||||
cv: ClassVar = 0
|
||||
|
||||
Cls.cv = 2 # OK
|
||||
o = Cls()
|
||||
print(o.cv) # OK (2)
|
||||
o.cv = 3 # Error!
|
||||
|
||||
Generic native classes
|
||||
----------------------
|
||||
|
||||
Native classes can be generic. Type variables are *erased* at runtime,
|
||||
and instances don't keep track of type variable values.
|
||||
|
||||
Compiled code thus can't check the values of type variables when
|
||||
performing runtime type checks. These checks are delayed to when
|
||||
reading a value with a type variable type::
|
||||
|
||||
from typing import TypeVar, Generic, cast
|
||||
|
||||
T = TypeVar('T')
|
||||
|
||||
class Box(Generic[T]):
|
||||
def __init__(self, item: T) -> None:
|
||||
self.item = item
|
||||
|
||||
x = Box(1) # Box[int]
|
||||
y = cast(Box[str], x) # OK (type variable value not checked)
|
||||
y.item # Runtime error: item is "int", but "str" expected
|
||||
|
||||
Metaclasses
|
||||
-----------
|
||||
|
||||
Most metaclasses aren't supported with native classes, since their
|
||||
behavior is too dynamic. You can use these metaclasses, however:
|
||||
|
||||
* ``abc.ABCMeta``
|
||||
* ``typing.GenericMeta`` (used by ``typing.Generic``)
|
||||
|
||||
.. note::
|
||||
|
||||
If a class definition uses an unsupported metaclass, *mypyc
|
||||
compiles the class into a regular Python class*.
|
||||
|
||||
Class decorators
|
||||
----------------
|
||||
|
||||
Similar to metaclasses, most class decorators aren't supported with
|
||||
native classes, as they are usually too dynamic. These class
|
||||
decorators can be used with native classes, however:
|
||||
|
||||
* ``mypy_extensions.trait`` (for defining :ref:`trait types <trait-types>`)
|
||||
* ``mypy_extensions.mypyc_attr`` (see :ref:`above <inheritance>`)
|
||||
* ``dataclasses.dataclass``
|
||||
|
||||
Dataclasses have partial native support, and they aren't as efficient
|
||||
as pure native classes.
|
||||
|
||||
.. note::
|
||||
|
||||
If a class definition uses an unsupported class decorator, *mypyc
|
||||
compiles the class into a regular Python class*.
|
||||
|
||||
Deleting attributes
|
||||
-------------------
|
||||
|
||||
By default, attributes defined in native classes can't be deleted. You
|
||||
can explicitly allow certain attributes to be deleted by using
|
||||
``__deletable__``::
|
||||
|
||||
class Cls:
|
||||
x: int = 0
|
||||
y: int = 0
|
||||
other: int = 0
|
||||
|
||||
__deletable__ = ['x', 'y'] # 'x' and 'y' can be deleted
|
||||
|
||||
o = Cls()
|
||||
del o.x # OK
|
||||
del o.y # OK
|
||||
del o.other # Error
|
||||
|
||||
You must initialize the ``__deletable__`` attribute in the class body,
|
||||
using a list or a tuple expression with only string literal items that
|
||||
refer to attributes. These are not valid::
|
||||
|
||||
a = ['x', 'y']
|
||||
|
||||
class Cls:
|
||||
x: int
|
||||
y: int
|
||||
|
||||
__deletable__ = a # Error: cannot use variable 'a'
|
||||
|
||||
__deletable__ = ('a',) # Error: not in a class body
|
||||
|
||||
Other properties
|
||||
----------------
|
||||
|
||||
Instances of native classes don't usually have a ``__dict__`` attribute.
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
Miscellaneous native operations
|
||||
===============================
|
||||
|
||||
This is a list of various non-type-specific operations that have
|
||||
custom native implementations. If an operation has no native
|
||||
implementation, mypyc will use fallback generic implementations that
|
||||
are often not as fast.
|
||||
|
||||
.. note::
|
||||
|
||||
Operations specific to various primitive types are described
|
||||
in the following sections.
|
||||
|
||||
Operators
|
||||
---------
|
||||
|
||||
* ``x is y`` (this is very fast for all types)
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
* ``isinstance(obj, type: type)``
|
||||
* ``isinstance(obj, type: tuple)``
|
||||
* ``cast(<type>, obj)``
|
||||
* ``type(obj)``
|
||||
* ``len(obj)``
|
||||
* ``id(obj)``
|
||||
* ``iter(obj)``
|
||||
* ``next(iter: Iterator)``
|
||||
* ``hash(obj)``
|
||||
* ``getattr(obj, attr)``
|
||||
* ``getattr(obj, attr, default)``
|
||||
* ``setattr(obj, attr, value)``
|
||||
* ``hasattr(obj, attr)``
|
||||
* ``delattr(obj, name)``
|
||||
* ``slice(start, stop, step)``
|
||||
* ``globals()``
|
||||
|
||||
Method decorators
|
||||
-----------------
|
||||
|
||||
* ``@property``
|
||||
* ``@staticmethod``
|
||||
* ``@classmethod``
|
||||
* ``@abc.abstractmethod``
|
||||
|
||||
Statements
|
||||
----------
|
||||
|
||||
These variants of statements have custom implementations:
|
||||
|
||||
* ``for ... in seq:`` (for loop over a sequence)
|
||||
* ``for ... in enumerate(...):``
|
||||
* ``for ... in zip(...):``
|
||||
|
|
@ -0,0 +1,244 @@
|
|||
.. _performance-tips:
|
||||
|
||||
Performance tips and tricks
|
||||
===========================
|
||||
|
||||
Performance optimization is part art, part science. Just using mypyc
|
||||
in a simple manner will likely make your code faster, but squeezing
|
||||
the most performance out of your code requires the use of some
|
||||
techniques we'll summarize below.
|
||||
|
||||
Profiling
|
||||
---------
|
||||
|
||||
If you are speeding up existing code, understanding where time is
|
||||
spent is important. Mypyc speeds up code that you compile. If most of
|
||||
the time is spent elsewhere, you may come back disappointed. For
|
||||
example, if you spend 40% of time outside compiled code, even if
|
||||
compiled code would go 100x faster, overall performance will only be
|
||||
2.5x faster.
|
||||
|
||||
A simple (but often effective) approach is to record the time in
|
||||
various points of program execution using ``time.time()``, and to
|
||||
print out elapsed time (or to write it to a log file).
|
||||
|
||||
The stdlib modules ``profile`` and ``cProfile`` can provide much more
|
||||
detailed data. (But these only work well with non-compiled code.)
|
||||
|
||||
Avoiding slow libraries
|
||||
-----------------------
|
||||
|
||||
If profiling indicates that a lot of time is spent in the stdlib or
|
||||
third-party libraries, you still have several options.
|
||||
|
||||
First, if most time is spent in a few library features, you can
|
||||
perhaps easily reimplement them in type-annotated Python, or extract
|
||||
the relevant code and annotate it. Now it may be easy to compile this
|
||||
code to speed it up.
|
||||
|
||||
Second, you may be able to avoid the library altogether, or use an
|
||||
alternative, more efficient library to achieve the same purpose.
|
||||
|
||||
Type annotations
|
||||
----------------
|
||||
|
||||
As discussed earlier, type annotations are key to major performance
|
||||
gains. You should at least consider adding annotations to any
|
||||
performance-critical functions and classes. It may also be helpful to
|
||||
annotate code called by this code, even if it's not compiled, since
|
||||
this may help mypy infer better types in the compile code. If you use
|
||||
libraries, ensure they have stub files with decent type annotation
|
||||
coverage. Writing a stub file is often easy, and you only need to
|
||||
annotate features you use a lot.
|
||||
|
||||
If annotating external code or writing stubs feel too burdensome, a
|
||||
simple workaround is to annotate variables explicitly. For example,
|
||||
here we call ``acme.get_items()``, but it has no type annotation. We
|
||||
can use an explicit type annotation for the variable to which we
|
||||
assign the result::
|
||||
|
||||
from typing import List, Tuple
|
||||
import acme
|
||||
|
||||
def work() -> None:
|
||||
# Annotate "items" to help mypyc
|
||||
items: List[Tuple[int, str]] = acme.get_items()
|
||||
for item in items:
|
||||
... # Do some work here
|
||||
|
||||
Without the annotation on ``items``, the type would be ``Any`` (since
|
||||
``acme`` has no type annotation), resulting in slower, generic
|
||||
operations being used later in the function.
|
||||
|
||||
Avoiding slow Python features
|
||||
-----------------------------
|
||||
|
||||
Mypyc can optimize some features more effectively than others. Here
|
||||
the difference is sometimes big -- some things only get marginally
|
||||
faster at best, while others can get 10x faster, or more. Avoiding
|
||||
these slow features in performance-critical parts of your code can
|
||||
help a lot.
|
||||
|
||||
These are some of the most important things to avoid:
|
||||
|
||||
* Using class decorators or metaclasses in compiled code (that aren't
|
||||
properly supported by mypyc)
|
||||
|
||||
* Heavy reliance on interpreted Python libraries (C extensions are
|
||||
usually fine)
|
||||
|
||||
These things also tend to be relatively slow:
|
||||
|
||||
* Using Python classes and instances of Python classes (native classes
|
||||
are much faster)
|
||||
|
||||
* Calling decorated functions (``@property``, ``@staticmethod``, and
|
||||
``@classmethod`` are special cased and thus fast)
|
||||
|
||||
* Calling nested functions
|
||||
|
||||
* Calling functions or methods defined in other compilation units
|
||||
|
||||
* Using ``*args`` or ``**kwargs``
|
||||
|
||||
* Using generator functions
|
||||
|
||||
* Using floating point numbers (they are relatively unoptimized)
|
||||
|
||||
* Using callable values (i.e. not leveraging early binding to call
|
||||
functions or methods)
|
||||
|
||||
Nested functions can often be replaced with module-level functions or
|
||||
methods of native classes.
|
||||
|
||||
Callable values and nested functions can sometimes be replaced with an
|
||||
instance of a native class with a single method only, such as
|
||||
``call(...)``. You can derive the class from an ABC, if there are
|
||||
multiple possible functions.
|
||||
|
||||
.. note::
|
||||
|
||||
Some slow features will likely get efficient implementations in the
|
||||
future. You should check this section every once in a while to see
|
||||
if some additional operations are fast.
|
||||
|
||||
Using fast native features
|
||||
--------------------------
|
||||
|
||||
Some native operations are particularly quick relative to the
|
||||
corresponding interpreted operations. Using them as much as possible
|
||||
may allow you to see 10x or more in performance gains.
|
||||
|
||||
Some things are not much (or any) faster in compiled code, such as set
|
||||
math operations. In contrast, calling a method of a native class is
|
||||
much faster in compiled code.
|
||||
|
||||
If you are used to optimizing for CPython, you might have replaced
|
||||
some class instances with dictionaries, as they can be
|
||||
faster. However, in compiled code, this "optimization" would likely
|
||||
slow down your code.
|
||||
|
||||
Similarly, caching a frequently called method in a local variable can
|
||||
help in CPython, but it can slow things down in compiled code, since
|
||||
the code won't use :ref:`early binding <early-binding>`::
|
||||
|
||||
def squares(n: int) -> List[int]:
|
||||
a = []
|
||||
append = a.append # Not a good idea in compiled code!
|
||||
for i in range(n):
|
||||
append(i * i)
|
||||
return a
|
||||
|
||||
Here are examples of features that are fast, in no particular order
|
||||
(this list is *not* exhaustive):
|
||||
|
||||
* Calling compiled functions directly defined in the same compilation
|
||||
unit (with positional and/or keyword arguments)
|
||||
|
||||
* Calling methods of native classes defined in the same compilation
|
||||
unit (with positional and/or keyword arguments)
|
||||
|
||||
* Many integer operations
|
||||
|
||||
* Booleans
|
||||
|
||||
* :ref:`Native list operations <list-ops>`, such as indexing,
|
||||
``append``, and list comprehensions
|
||||
|
||||
* While loops
|
||||
|
||||
* For loops over ranges and lists, and with ``enumerate`` or ``zip``
|
||||
|
||||
* Reading dictionary items
|
||||
|
||||
* ``isinstance()`` checks against native classes and instances of
|
||||
primitive types (and unions of them)
|
||||
|
||||
* Accessing local variables
|
||||
|
||||
* Accessing attributes of native classes
|
||||
|
||||
* Accessing final module-level attributes
|
||||
|
||||
* Comparing strings for equality
|
||||
|
||||
These features are also fast, but somewhat less so (relative to other
|
||||
related operations):
|
||||
|
||||
* Constructing instances of native classes
|
||||
|
||||
* Constructing dictionaries
|
||||
|
||||
* Setting dictionary items
|
||||
|
||||
* Native :ref:`dict <dict-ops>` and :ref:`set <set-ops>` operations
|
||||
|
||||
* Accessing module-level variables
|
||||
|
||||
Generally anything documented as a native operation is fast, even if
|
||||
it's not explicitly mentioned here
|
||||
|
||||
Adjusting garbage collection
|
||||
----------------------------
|
||||
|
||||
Compilation does not speed up cyclic garbage collection. If everything
|
||||
else gets much faster, it's possible that garbage collection will take
|
||||
a big fraction of time. You can use ``gc.set_threshold()`` to adjust
|
||||
the garbage collector to run less often::
|
||||
|
||||
import gc
|
||||
|
||||
# Spend less time in gc; do this before significant computation
|
||||
gc.set_threshold(150000)
|
||||
|
||||
... # Actual work happens here
|
||||
|
||||
Fast interpreter shutdown
|
||||
-------------------------
|
||||
|
||||
If you allocate many objects, it's possible that your program spends a
|
||||
lot of time cleaning up when the Python runtime shuts down. Mypyc
|
||||
won't speed up the shutdown of a Python process much.
|
||||
|
||||
You can call ``os._exit(code)`` to immediately terminate the Python
|
||||
process, skipping normal cleanup. This can give a nice boost to a
|
||||
batch process or a command-line tool.
|
||||
|
||||
.. note::
|
||||
|
||||
This can be dangerous and can lose data. You need to ensure
|
||||
that all streams are flushed and everything is otherwise cleaned up
|
||||
properly.
|
||||
|
||||
Work smarter
|
||||
------------
|
||||
|
||||
Usually there are many things you can do to improve performance, even
|
||||
if most tweaks will yield only minor gains. The key to being effective
|
||||
is to focus on things that give a large gain with a small effort.
|
||||
|
||||
For example, low-level optimizations, such as avoiding a nested
|
||||
function, can be pointless, if you could instead avoid a metaclass --
|
||||
to allow a key class to be compiled as a native class. The latter
|
||||
optimization could speed up numerous method calls and attribute
|
||||
accesses, just like that.
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
.. _set-ops:
|
||||
|
||||
Native set operations
|
||||
======================
|
||||
|
||||
These ``set`` operations have fast, optimized implementations. Other
|
||||
set operations use generic implementations that are often slower.
|
||||
|
||||
Construction
|
||||
------------
|
||||
|
||||
Construct set with specific items:
|
||||
|
||||
* ``{item0, ..., itemN}``
|
||||
|
||||
Construct empty set:
|
||||
|
||||
* ``set()``
|
||||
|
||||
Construct set from iterable:
|
||||
|
||||
* ``set(x: Iterable)``
|
||||
|
||||
Set comprehensions:
|
||||
|
||||
* ``{... for ... in ...}``
|
||||
* ``{... for ... in ... if ...}``
|
||||
|
||||
Operators
|
||||
---------
|
||||
|
||||
* ``item in s``
|
||||
|
||||
Methods
|
||||
-------
|
||||
|
||||
* ``s.add(item)``
|
||||
* ``s.remove(item)``
|
||||
* ``s.discard(item)``
|
||||
* ``s.update(x: Iterable)``
|
||||
* ``s.clear()``
|
||||
* ``s.pop()``
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
* ``len(s: set)``
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
.. _str-ops:
|
||||
|
||||
Native string operations
|
||||
========================
|
||||
|
||||
These ``str`` operations have fast, optimized implementations. Other
|
||||
string operations use generic implementations that are often slower.
|
||||
|
||||
Construction
|
||||
------------
|
||||
|
||||
* String literal
|
||||
* ``str(x: int)``
|
||||
* ``str(x: object)``
|
||||
|
||||
Operators
|
||||
---------
|
||||
|
||||
* Concatenation (``s1 + s2``)
|
||||
* Indexing (``s[n]``)
|
||||
* Slicing (``s[n:m]``, ``s[n:]``, ``s[:m]``)
|
||||
* Comparisons (``==``, ``!=``)
|
||||
* Augmented assignment (``s1 += s2``)
|
||||
|
||||
Methods
|
||||
-------
|
||||
|
||||
* ``s1.endswith(s2: str)``
|
||||
* ``s.join(x: Iterable)``
|
||||
* ``s.replace(old: str, new: str)``
|
||||
* ``s.replace(old: str, new: str, count: int)``
|
||||
* ``s.split()``
|
||||
* ``s.split(sep: str)``
|
||||
* ``s.split(sep: str, maxsplit: int)``
|
||||
* ``s1.startswith(s2: str)``
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
.. _tuple-ops:
|
||||
|
||||
Native tuple operations
|
||||
=======================
|
||||
|
||||
These ``tuple`` operations have fast, optimized implementations. Other
|
||||
tuple operations use generic implementations that are often slower.
|
||||
|
||||
Unless mentioned otherwise, these operations apply to both fixed-length
|
||||
tuples and variable-length tuples.
|
||||
|
||||
Construction
|
||||
------------
|
||||
|
||||
* ``item0, ..., itemN`` (construct a tuple)
|
||||
* ``tuple(lst: list)`` (construct a variable-length tuple)
|
||||
* ``tuple(lst: Iterable)`` (construct a variable-length tuple)
|
||||
|
||||
Operators
|
||||
---------
|
||||
|
||||
* ``tup[n]`` (integer index)
|
||||
* ``tup[n:m]``, ``tup[n:]``, ``tup[:m]`` (slicing)
|
||||
|
||||
Statements
|
||||
----------
|
||||
|
||||
* ``item0, ..., itemN = tup`` (for fixed-length tuples)
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
* ``len(tup: tuple)``
|
||||
|
|
@ -0,0 +1,314 @@
|
|||
.. _using-type-annotations:
|
||||
|
||||
Using type annotations
|
||||
======================
|
||||
|
||||
You will get the most out of mypyc if you compile code with precise
|
||||
type annotations. Not all type annotations will help performance
|
||||
equally, however. Using types such as :ref:`primitive types
|
||||
<primitive-types>`, :ref:`native classes <native-class-intro>`,
|
||||
:ref:`union types <union-types>`, :ref:`trait types <trait-types>`,
|
||||
and :ref:`tuple types <tuple-types>` as much as possible is a key to
|
||||
major performance gains over CPython.
|
||||
|
||||
In contrast, some other types, including ``Any``, are treated as
|
||||
:ref:`erased types <erased-types>`. Operations on erased types use
|
||||
generic operations that work with arbitrary objects, similar to how
|
||||
the CPython interpreter works. If you only use erased types, the only
|
||||
notable benefits over CPython will be the removal of interpreter
|
||||
overhead (from compilation) and a bit of :ref:`early binding
|
||||
<early-binding>`, which will usually only give minor performance
|
||||
gains.
|
||||
|
||||
.. _primitive-types:
|
||||
|
||||
Primitive types
|
||||
---------------
|
||||
|
||||
The following built-in types are treated as *primitive types* by
|
||||
mypyc, and many operations on these types have efficient
|
||||
implementations:
|
||||
|
||||
* ``int`` (:ref:`native operations <int-ops>`)
|
||||
* ``float`` (:ref:`native operations <float-ops>`)
|
||||
* ``bool`` (:ref:`native operations <bool-ops>`)
|
||||
* ``str`` (:ref:`native operations <str-ops>`)
|
||||
* ``List[T]`` (:ref:`native operations <list-ops>`)
|
||||
* ``Dict[K, V]`` (:ref:`native operations <dict-ops>`)
|
||||
* ``Set[T]`` (:ref:`native operations <set-ops>`)
|
||||
* ``Tuple[T, ...]`` (variable-length tuple; :ref:`native operations <tuple-ops>`)
|
||||
* ``None``
|
||||
|
||||
The link after each type lists all supported native, optimized
|
||||
operations for the type. You can use all operations supported by
|
||||
Python, but *native operations* will have custom, optimized
|
||||
implementations.
|
||||
|
||||
Primitive containers
|
||||
--------------------
|
||||
|
||||
Primitive container objects such as ``list`` and ``dict`` don't
|
||||
maintain knowledge of the item types at runtime -- the item type is
|
||||
*erased*.
|
||||
|
||||
This means that item types are checked when items are accessed, not
|
||||
when a container is passed as an argument or assigned to another
|
||||
variable. For example, here we have a runtime type error on the final
|
||||
line of ``example`` (the ``Any`` type means an arbitrary, unchecked
|
||||
value)::
|
||||
|
||||
from typing import List, Any
|
||||
|
||||
def example(a: List[Any]) -> None:
|
||||
b: List[int] = a # No error -- items are not checked
|
||||
print(b[0]) # Error here -- got str, but expected int
|
||||
|
||||
example(["x"])
|
||||
|
||||
.. _native-class-intro:
|
||||
|
||||
Native classes
|
||||
--------------
|
||||
|
||||
Classes that get compiled to C extensions are called native
|
||||
classes. Most common operations on instances of these classes are
|
||||
optimized, including construction, attribute access and method calls.
|
||||
|
||||
Native class definitions look exactly like normal Python class
|
||||
definitions. A class is usually native if it's in a compiled module
|
||||
(though there are some exceptions).
|
||||
|
||||
Consider this example:
|
||||
|
||||
.. code-block::
|
||||
|
||||
class Point:
|
||||
def __init__(self, x: int, y: int) -> None:
|
||||
self.x = x
|
||||
self.y = y
|
||||
|
||||
def shift(p: Point) -> Point:
|
||||
return Point(p.x + 1, p.y + 1)
|
||||
|
||||
All operations in the above example use native operations, if the file
|
||||
is compiled.
|
||||
|
||||
Native classes have some notable different from Python classes:
|
||||
|
||||
* Only attributes and methods defined in the class body or methods are
|
||||
supported. If you try to assign to an undefined attribute outside
|
||||
the class definition, ``AttributeError`` will be raised. This enables
|
||||
an efficient memory layout and fast method calls for native classes.
|
||||
|
||||
* Native classes usually don't define the ``__dict__`` attribute (they
|
||||
don't have an attribute dictionary). This follows from only having
|
||||
a specific set of attributes.
|
||||
|
||||
* Native classes can't have an arbitrary metaclass or use most class
|
||||
decorators.
|
||||
|
||||
Native classes only support single inheritance. A limited form of
|
||||
multiple inheritance is supported through *trait types*. You generally
|
||||
must inherit from another native class (or ``object``). By default,
|
||||
you can't inherit a Python class from a native class (but there's
|
||||
an :ref:`override <inheritance>` to allow that).
|
||||
|
||||
See :ref:`native-classes` for more details.
|
||||
|
||||
.. _tuple-types:
|
||||
|
||||
Tuple types
|
||||
-----------
|
||||
|
||||
Fixed-length
|
||||
`tuple types <https://mypy.readthedocs.io/en/stable/kinds_of_types.html#tuple-types>`_
|
||||
such as ``Tuple[int, str]`` are represented
|
||||
as :ref:`value types <value-and-heap-types>` when stored in variables,
|
||||
passed as arguments, or returned from functions. Value types are
|
||||
allocated in the low-level machine stack or in CPU registers, as
|
||||
opposed to *heap types*, which are allocated dynamically from the
|
||||
heap.
|
||||
|
||||
Like all value types, tuples will be *boxed*, i.e. converted to
|
||||
corresponding heap types, when stored in Python containers, or passed
|
||||
to non-native code. A boxed tuple value will be a regular Python tuple
|
||||
object.
|
||||
|
||||
.. _union-types:
|
||||
|
||||
Union types
|
||||
-----------
|
||||
|
||||
`Union types <https://mypy.readthedocs.io/en/stable/kinds_of_types.html#union-types>`_
|
||||
and
|
||||
`optional types <https://mypy.readthedocs.io/en/stable/kinds_of_types.html#optional-types-and-the-none-type>`_
|
||||
that contain primitive types, native class types and
|
||||
trait types are also efficient. If a union type has
|
||||
:ref:`erased <erased-types>` items, accessing items with
|
||||
non-erased types is often still quite efficient.
|
||||
|
||||
A value with a union types is always :ref:`boxed <value-and-heap-types>`,
|
||||
even if it contains a value that also has an unboxed representation, such
|
||||
as an integer or a boolean.
|
||||
|
||||
For example, using ``Optional[int]`` is quite efficient, but the value
|
||||
will always be boxed. A plain ``int`` value will usually be faster, since
|
||||
it has an unboxed representation.
|
||||
|
||||
.. _trait-types:
|
||||
|
||||
Trait types
|
||||
-----------
|
||||
|
||||
Trait types enable a form of multiple inheritance for native classes.
|
||||
A native class can inherit any number of traits. Trait types are
|
||||
defined as classes using the ``mypy_extensions.trait`` decorator::
|
||||
|
||||
from mypy_extensions import trait
|
||||
|
||||
@trait
|
||||
class MyTrait:
|
||||
def method(self) -> None:
|
||||
...
|
||||
|
||||
Traits can define methods, properties and attributes. They often
|
||||
define abstract methods. Traits can be generic.
|
||||
|
||||
If a class subclasses both a non-trait class and traits, the traits
|
||||
must be placed at the end of the base class list::
|
||||
|
||||
class Base: ...
|
||||
|
||||
class Derived(Base, MyTrait, FooTrait): # OK
|
||||
...
|
||||
|
||||
class Derived2(MyTrait, FooTrait, Base):
|
||||
# Error: traits should come last
|
||||
...
|
||||
|
||||
Traits have some special properties:
|
||||
|
||||
* You shouldn't create instances of traits (though mypyc does not
|
||||
prevent it yet).
|
||||
|
||||
* Traits can subclass other traits, but they can't subclass non-trait
|
||||
classes (other than ``object``).
|
||||
|
||||
* Accessing methods or attributes through a trait type is somewhat
|
||||
less efficient than through a native class type, but this is much
|
||||
faster than through Python class types or other
|
||||
:ref:`erased types <erased-types>`.
|
||||
|
||||
You need to install ``mypy-extensions`` to use ``@trait``:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
pip install --upgrade mypy-extensions
|
||||
|
||||
.. _erased-types:
|
||||
|
||||
Erased types
|
||||
------------
|
||||
|
||||
Mypyc supports many other kinds of types as well, beyond those
|
||||
described above. However, these types don't have customized
|
||||
operations, and they are implemented using *type erasure*. Type
|
||||
erasure means that all other types are equivalent to untyped values at
|
||||
runtime, i.e. they are the equivalent of the type ``Any``. Erased
|
||||
types include these:
|
||||
|
||||
* Python classes (including ABCs)
|
||||
|
||||
* Non-mypyc extension types and primitive types (including built-in
|
||||
types that are not primitives)
|
||||
|
||||
* `Callable types <https://mypy.readthedocs.io/en/stable/kinds_of_types.html#callable-types-and-lambdas>`_
|
||||
|
||||
* `Type variable types <https://mypy.readthedocs.io/en/stable/generics.html>`_
|
||||
|
||||
* Type `Any <https://mypy.readthedocs.io/en/stable/dynamic_typing.html>`_
|
||||
|
||||
* Protocol types
|
||||
|
||||
Using erased types can still improve performance, since they can
|
||||
enable better types to be inferred for expressions that use these
|
||||
types. For example, a value with type ``Callable[[], int]`` will not
|
||||
allow native calls. However, the return type is a primitive type, and
|
||||
we can use fast operations on the return value::
|
||||
|
||||
from typing import Callable
|
||||
|
||||
def call_and_inc(f: Callable[[], int]) -> int:
|
||||
# Slow call, since f has an erased type
|
||||
n = f()
|
||||
# Fast increment; inferred type of n is int (primitive type)
|
||||
n += 1
|
||||
return n
|
||||
|
||||
If the type of the argument ``f`` was ``Any``, the type of ``n`` would
|
||||
also be ``Any``, resulting in a generic, slower increment operation
|
||||
being used.
|
||||
|
||||
Strict runtime type checking
|
||||
----------------------------
|
||||
|
||||
Compiled code ensures that any variable or expression with a
|
||||
non-erased type only has compatible values at runtime. This is in
|
||||
contrast with using *optional static typing*, such as by using mypy,
|
||||
when type annotations are not enforced at runtime. Mypyc ensures
|
||||
type safety both statically and at runtime.
|
||||
|
||||
``Any`` types and erased types in general can compromise type safety,
|
||||
and this is by design. Inserting strict runtime type checks for all
|
||||
possible values would be too expensive and against the goal of
|
||||
high performance.
|
||||
|
||||
.. _value-and-heap-types:
|
||||
|
||||
Value and heap types
|
||||
--------------------
|
||||
|
||||
In CPython, memory for all objects is dynamically allocated on the
|
||||
heap. All Python types are thus *heap types*. In compiled code, some
|
||||
types are *value types* -- no object is (necessarily) allocated on the
|
||||
heap. ``bool``, ``None`` and fixed-length tuples are value types.
|
||||
|
||||
``int`` is a hybrid. For typical integer values, it is a value
|
||||
type. Large enough integer values, those that require more than 63
|
||||
bits (or 31 bits on 32-bit platforms) to represent, use a heap-based
|
||||
representation (same as CPython).
|
||||
|
||||
Value types have a few differences from heap types:
|
||||
|
||||
* When an instance of a value type is used in a context that expects a
|
||||
heap value, for example as a list item, it will transparently switch
|
||||
to a heap-based representation (boxing) as needed.
|
||||
|
||||
* Similarly, mypyc transparently changes from a heap-based
|
||||
representation to a value representation (unboxing).
|
||||
|
||||
* Object identity of integers and tuples is not preserved. You should
|
||||
use ``==`` instead of ``is`` if you are comparing two integers or
|
||||
fixed-length tuples.
|
||||
|
||||
* When an instance of a subclass of a value type is converted to the
|
||||
base type, it is implicitly converted to an instance of the target
|
||||
type. For example, a ``bool`` value assigned to a variable with an
|
||||
``int`` type will be converted to the corresponding integer.
|
||||
|
||||
The latter conversion is the only implicit type conversion that
|
||||
happens in mypyc programs.
|
||||
|
||||
Example::
|
||||
|
||||
def example() -> None:
|
||||
# A small integer uses the value (unboxed) representation
|
||||
x = 5
|
||||
# A large integer the the heap (boxed) representation
|
||||
x = 2**500
|
||||
# Lists always contain boxed integers
|
||||
a = [55]
|
||||
# When reading from a list, the object is automatically unboxed
|
||||
x = a[0]
|
||||
# True is converted to 1 on assignment
|
||||
x = True
|
||||
Loading…
Add table
Add a link
Reference in a new issue