PYINSTRUMENT(1)

📦
version 5.1.2 · 3 versions
, in ubuntu resolute, ubuntu stonking

💖 PYINSTRUMENT 💖

okay bestie, welcome to the manual for pyinstrument — aka the Python profiler that's about to become your whole personality. this is the PYINSTRUMENT(1) page, version 5.1.2, and we're gonna read it together like it's a group chat 🫶

✨ NAME ✨

pyinstrument — pyinstrument 5.1.2

🌈 PYINSTRUMENT 🌈

🖼️ [image: Screenshot] — https://github.com/joerick/pyinstrument/raw/main/docs/img/screenshot.jpg

so like. Pyinstrument is a Python profiler. and a profiler, bestie, is a tool that helps you optimize your code — aka make it faster 🔥. To get the biggest speed increase you should focus on the slowest part of your program (that's Amdahl's law, look it up: https://en.wikipedia.org/wiki/Amdahl%27s_law). Pyinstrument helps you find it! no cap.

☕️ Not sure where to start? Check out this video tutorial from calmcode.io — https://calmcode.io/pyinstrument/introduction.html — it's giving tutorial, it's giving helpful 💅

🎀 USER GUIDE 🎀

🦋 Installation 🦋

pip install pyinstrument

Pyinstrument supports Python 3.8+. periodt 💅

🌸 Profile a Python script 🌸

Call Pyinstrument directly from the command line. Instead of writing python3 script.py, type pyinstrument script.py. Your script will run as normal, and at the end (or when you press ^C), Pyinstrument will output a colored summary showing where most of the time was spent. literally the moment you've been waiting for ✨

Here are the options you can use:

Usage: pyinstrument [options] scriptfile [arg] ...

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --load=FILENAME       instead of running a script, load a profile session
                        from a pyisession file
  --load-prev=IDENTIFIER
                        instead of running a script, load a previous profile
                        session as specified by an identifier
  -m MODULE             run library module as a script, like 'python -m
                        module'
  -c PROGRAM            program passed in as string, like 'python -c "..."'
  --from-path           (POSIX only) instead of the working directory, look
                        for scriptfile in the PATH environment variable
  -o OUTFILE, --outfile=OUTFILE
                        save to <outfile>
  -r RENDERER, --renderer=RENDERER
                        how the report should be rendered. One of: 'text',
                        'html', 'json', 'speedscope', 'pyisession', 'pstats',
                        or python import path to a renderer class. Defaults to
                        the appropriate format for the extension if OUTFILE is
                        given, otherwise, defaults to 'text'.
  -p RENDER_OPTION, --render-option=RENDER_OPTION
                        options to pass to the renderer, in the format
                        'flag_name' or 'option_name=option_value'. For
                        example, to set the option 'time', pass '-p
                        time=percent_of_total'. To pass multiple options, use
                        the -p option multiple times. You can set processor
                        options using dot-syntax, like '-p
                        processor_options.filter_threshold=0'. option_value is
                        parsed as a JSON value or a string.
  -t, --timeline        render as a timeline - preserve ordering and don't
                        condense repeated calls
  --target-description=TARGET_DESCRIPTION
                        description text to display in the report. The
                        placeholder '{args}' may be used to include the CLI
                        arguments passed to the target script, including the
                        script name. Default: 'Program: {args}'
  --hide=EXPR           glob-style pattern matching the file paths whose
                        frames to hide. Defaults to hiding non-application
                        code
  --hide-regex=REGEX    regex matching the file paths whose frames to hide.
                        Useful if --hide doesn't give enough control.
  --show=EXPR           glob-style pattern matching the file paths whose
                        frames to show, regardless of --hide or --hide-regex.
                        For example, use --show '*/<library>/*' to show frames
                        within a library that would otherwise be hidden.
  --show-regex=REGEX    regex matching the file paths whose frames to always
                        show. Useful if --show doesn't give enough control.
  --show-all            show everything
  --unicode             (text renderer only) force unicode text output
  --no-unicode          (text renderer only) force ascii text output
  --color               (text renderer only) force ansi color text output
  --no-color            (text renderer only) force no color text output
  -i INTERVAL, --interval=INTERVAL
                        Minimum time, in seconds, between each stack sample.
                        Smaller values allow resolving shorter duration
                        function calls but incur a greater runtime and memory
                        consumption overhead. For longer running scripts,
                        setting a larger interval reduces the memory
                        consumption required to store the stack samples.
  --use-timing-thread   Use a separate thread to time the interval between
                        stack samples. This can reduce the overhead of
                        sampling on some systems.
💡 Protip: -r html will give you a interactive profile report as HTML — you can really explore this way! it's giving main character energy fr fr

🌟 Profile a Python CLI command 🌟

For profiling an installed Python script via the "console_script" entry point — https://packaging.python.org/en/latest/specifications/entry-points/#use-for-scripts — call Pyinstrument directly from the command line with the --from-path flag. Instead of writing cli-script, type pyinstrument --from-path cli-script. Your script will run as normal, and at the end (or when you press ^C), Pyinstrument will output a colored summary showing where most of the time was spent. iykyk 💖

🧚 Profile a specific chunk of code 🧚

Pyinstrument also has a Python API. You can use a with-block, like this:

import pyinstrument

with pyinstrument.profile():
    # code you want to profile

Or you can decorate a function/method, like this:

import pyinstrument

@pyinstrument.profile()
def my_function():
    # code you want to profile

There's also a lower-level API called Profiler, that's more flexible:

from pyinstrument import Profiler

profiler = Profiler()
profiler.start()

# code you want to profile

profiler.stop()
profiler.print()

If you get "No samples were recorded." because your code executed in under 1ms, hooray! 🎉 If you still want to instrument the code, set an interval value smaller than the default 0.001 (1 millisecond) like this:

pyinstrument.profile(interval=0.0001)
# or,
profiler = Profiler(interval=0.0001)
...

Experiment with the interval value to see different depths, but keep in mind that smaller intervals could affect the performance overhead of profiling. not me warning you twice, it's just true 😭

💡 Protip: To explore the profile in a web browser, use profiler.open_in_browser(). To save this HTML for later, use profiler.output_html(). obsessed with this for you honestly 🫶

🌈 Profile code in Jupyter/IPython 🌈

Via IPython magics — https://ipython.readthedocs.io/en/stable/interactive/magics.html — you can profile a line or a cell in IPython or Jupyter.

Example:

%load_ext pyinstrument

%%pyinstrument
import time

def a():
    b()
    c()
def b():
    d()
def c():
    d()
def d():
    e()
def e():
    time.sleep(1)
a()

To customize options, see %%pyinstrument??. yes, question mark question mark, it's iconic 💅

👑 Profile a web request in Django 👑

To profile Django web requests, add pyinstrument.middleware.ProfilerMiddleware to MIDDLEWARE in your settings.py.

⭐ Profile specific request ⭐

Once installed, add ?profile to the end of a request URL to activate the profiler. Your request will run as normal, but instead of getting the response, you'll get pyinstrument's analysis of the request in a web page. slay.

⭐ Save all requests to a directory ⭐

If you're writing an API, it's not easy to change the URL when you want to profile something. In this case, add PYINSTRUMENT_PROFILE_DIR = 'profiles' to your settings.py. Pyinstrument will profile every request and save the HTML output to the folder profiles in your working directory.

⭐ Custom file name by string ⭐

You can further customize the filename by adding PYINSTRUMENT_FILENAME to settings.py, default value is "{total_time:.3f}s {path} {timestamp:.0f}.{ext}".

⭐ Custom file name by callback function ⭐

For more control you can provide a callback function by adding PYINSTRUMENT_FILENAME_CALLBACK to settings.py, that returns a filename as a string.

def get_pyinstrument_filename(request, session, renderer):
    path = request.get_full_path().replace("/", "_")[:100]
    ext = renderer.output_file_extension
    filename = f"{request.method}_{session.duration}{path}.{ext}"
    return filename

PYINSTRUMENT_FILENAME_CALLBACK = get_pyinstrument_filename

(This callback takes precedence over PYINSTRUMENT_FILENAME). so like, it wins, okay? 👑

⭐ Control shown profiling page ⭐

If you want to show the profiling page depending on the request you can define PYINSTRUMENT_SHOW_CALLBACK as dotted path to a function used for determining whether the page should show or not. You can provide your own function callback(request) which returns True or False in your settings.py.

def custom_show_pyinstrument(request):
    return request.user.is_superuser

PYINSTRUMENT_SHOW_CALLBACK = "%s.custom_show_pyinstrument" % __name__

You can configure the profile output type using setting's variable PYINSTRUMENT_PROFILE_DIR_RENDERER. Default value is pyinstrument.renderers.HTMLRenderer. The supported renderers are pyinstrument.renderers.JSONRenderer, pyinstrument.renderers.HTMLRenderer, pyinstrument.renderers.SpeedscopeRenderer.

⭐ Set a custom interval ⭐

You can configure the sampling interval using setting's variable PYINSTRUMENT_INTERVAL. Default value is 0.001.

🍓 Profile a web request in Flask 🍓

A simple setup to profile a Flask application is the following:

from flask import Flask, g, make_response, request
from pyinstrument import Profiler

app = Flask(__name__)

@app.before_request
def before_request():
    if "profile" in request.args:
        g.profiler = Profiler()
        g.profiler.start()
@app.after_request
def after_request(response):
    if not hasattr(g, "profiler"):
        return response
    g.profiler.stop()
    output_html = g.profiler.output_html()
    return make_response(output_html)

okay so here's the tea 🫖 — this lil guy checks for the ?profile query param on every single request, and if it finds it, it starts profiling. Then after each request where the profiler was actually running, it makes the html output and returns that instead of the actual response. It's giving ✨surprise makeover✨ but for your HTTP responses.

🌸 Profile a web request in FastAPI 🌸

To profile call stacks in FastAPI, you can write a middleware extension for pyinstrument. Bestie, it's so easy.

Caution: Only async path operation functions are profiled with this approach. Routes that are defined without async def are executed in a separate execution thread, and therefore not profiled by this approach. See issue #257 at https://github.com/joerick/pyinstrument/issues/257 and FastAPI's "Concurrency and async / await" at https://fastapi.tiangolo.com/async/ for more information. 😭 fr fr, don't get caught out.

Create an async function and decorate with app.middleware('http') where app is the name of your FastAPI application instance.

Make sure you configure a setting to only make this available when required. Do not ship this to prod unlocked, i'm begging 🥺

from fastapi import Request
from fastapi.responses import HTMLResponse
from pyinstrument import Profiler

PROFILING = True  # Set this from a settings model

if PROFILING:
    @app.middleware("http")
    async def profile_request(request: Request, call_next):
        profiling = request.query_params.get("profile", False)
        if profiling:
            profiler = Profiler()
            profiler.start()
            await call_next(request)
            profiler.stop()
            return HTMLResponse(profiler.output_html())
        else:
            return await call_next(request)

To invoke, make any request to your application with the GET parameter profile=1 and it will print the HTML result from pyinstrument. Slay. 💅

🦋 Profile a web request in Falcon 🦋

For profile call stacks in Falcon, you can write a middleware extension using pyinstrument.

Create a middleware class and start the profiler at process_request and stop it at process_response. The middleware can be added to the app.

Make sure you configure a setting to only make this available when required. (same energy as before, okay?)

from pyinstrument import Profiler
import falcon

class ProfilerMiddleware:
    def __init__(self, interval=0.01):
        self.profiler = Profiler(interval=interval)

    def process_request(self, req, resp):
        self.profiler.start()

    def process_response(self, req, resp, resource, req_succeeded):
        self.profiler.stop()
        self.profiler.open_in_browser()

PROFILING = True  # Set this from a settings model

app = falcon.App()
if PROFILING:
    app.add_middleware(ProfilerMiddleware())

To invoke, make any request to your application and it launch a new window printing the HTML result from pyinstrument. iconic 🌈

🎀 Profile a web request in Litestar 🎀

Minimal application setup allowing request profiling.

The middleware overrides the response to return a profiling report in HTML format. it's giving main character middleware 👑

from __future__ import annotations

from asyncio import sleep

from litestar import Litestar, get
from litestar.middleware import MiddlewareProtocol
from litestar.types import ASGIApp, Message, Receive, Scope, Send

from pyinstrument import Profiler

class ProfilingMiddleware(MiddlewareProtocol):
    def __init__(self, app: ASGIApp) -> None:
        super().__init__(app)  # type: ignore
        self.app = app

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        profiler = Profiler(interval=0.001, async_mode="enabled")
        profiler.start()
        profile_html: str | None = None

        async def send_wrapper(message: Message) -> None:
            if message["type"] == "http.response.start":
                profiler.stop()
                nonlocal profile_html
                profile_html = profiler.output_html()
                message["headers"] = [
                    (b"content-type", b"text/html; charset=utf-8"),
                    (b"content-length", str(len(profile_html)).encode()),
                ]
            elif message["type"] == "http.response.body":
                assert profile_html is not None
                message["body"] = profile_html.encode()
            await send(message)

        await self.app(scope, receive, send_wrapper)

@get("/")
async def index() -> str:
    await sleep(1)
    return "Hello, world!"

app = Litestar(
    route_handlers=[index],
    middleware=[ProfilingMiddleware],
)

To invoke, make any request to your application and it will return the HTML result from pyinstrument instead of your application's response. 🫶

🍓 Profile a web request in aiohttp.web 🍓

You can use a simple middleware to profile aiohttp web server requests with Pyinstrument:

from aiohttp import web
from pyinstrument import Profiler

@web.middleware
async def profiler_middleware(request, handler):
    with Profiler() as p:
        await handler(request)
    return web.Response(text=p.output_html(), content_type="text/html")

app = web.Application(middlewares=(profiler_middleware,))

Pyinstrument's HTML output will be returned as response, showing the profiling result of each request. no cap, every request gets its own lil report 🧚

Make use of aiohttp.web development CLI feature to isolate configurations and make sure profiling is only enabled when needed:

...

def dev_app(argv):
    app = web.Application(middlewares=(profiler_middleware,))
    app.add_routes(routes)
    return app # for development

if __name__ == '__main__':
    app = web.Application()
    app.add_routes(routes)
    web.run_app(...) # for deployment

python3 -m aiohttp.web app:dev_app # develop with profiling and debug enabled
python3 ./app.py # run app without profiling

look at that separation tho — dev gets the profiler, prod stays clean. the way that this is literally just two commands 😌

💜 Profile Pytest tests 💜

Pyinstrument can be invoked via the command-line to run pytest, giving you a consolidated report for the test suite.

pyinstrument -m pytest [pytest-args...]

Or, to instrument specific tests, create and auto-use fixture in conftest.py in your test folder:

from pathlib import Path
import pytest
from pyinstrument import Profiler

TESTS_ROOT = Path.cwd()

@pytest.fixture(autouse=True)
def auto_profile(request):
    PROFILE_ROOT = (TESTS_ROOT / ".profiles")
    # Turn profiling on
    profiler = Profiler()
    profiler.start()

    yield  # Run test

    profiler.stop()
    PROFILE_ROOT.mkdir(exist_ok=True)
    results_file = PROFILE_ROOT / f"{request.node.name}.html"
    profiler.write_html(results_file)

This will generate a HTML file for each test node in your test suite inside the .profiles directory. one html per test?? obsessed actually 😭✨

🔥 Profile something else? 🔥

I'd love to have more ways to profile using Pyinstrument - e.g. other web frameworks. PRs are encouraged! come thru bestie 🫶💖

💖 HOW IT WORKS 💖

So like, here's the tea on what's actually happening under the hood, bestie. ✨

Pyinstrument interrupts your program every 1ms<sup>[1]</sup> and records the entire stack at that exact point. It pulls this off with a C extension and PyEval_SetProfile, but it only takes readings every 1ms. If you wanna go deeper, there's a whole blog post about it at http://joerick.me/posts/2017/12/15/pyinstrument-20/ 👑

You might be like "wait, that's SO few samples for a whole report??" 🙈 but don't worry, it genuinely does not decrease accuracy. The default interval of 1ms is a lower bound for recording a stackframe — but if a long time gets spent inside a single function call, it gets recorded at the end of that call instead. So basically those samples were 'bunched up' and recorded at the end. It's giving delayed but faithful, fr fr. 💅

🌸 Statistical profiling (not tracing) 🌸

Pyinstrument is a statistical profiler — it does not track every single function call your program makes. Instead, it's just recording the call stack every 1ms. Periodt. ✨

And that gives it some real advantages over other profilers. First off: statistical profilers are much lower-overhead than tracing profilers. Watch this glow-up:

  • Base — Django template render × 4000 — 0.33s
  • pyinstrument — 0.43s — overhead: 30%
  • cProfile — 0.61s — overhead: 84%
  • profile — 6.79s — overhead: 2057%

(yeah, profile really said "let me ruin your entire afternoon" 💀)

But low overhead is also important because it can distort the results. When you're using a tracing profiler, code that makes a lot of Python function calls invokes the profiler a lot, which makes it slower. That skews everything, and it might lead you to optimise the wrong part of your program! 😭 Main character energy in the wrong scene, honestly.

🦋 Full-stack recording 🦋

The standard Python profilers — profile (http://docs.python.org/2/library/profile.html#module-profile) and cProfile (http://docs.python.org/2/library/profile.html#module-cProfile) — show you a big list of functions, ordered by the time spent in each function. That's great and all, but it can be so hard to interpret why those functions are getting called. What you actually want to know is why they're called, and which parts of user code were involved. 🫶

For example, let's say I want to figure out why a web request in Django is slow. If I use cProfile, I might get this:

   151940 function calls (147672 primitive calls) in 1.696 seconds

      Ordered by: cumulative time

      ncalls  tottime  percall  cumtime  percall filename:lineno(function)
           1    0.000    0.000    1.696    1.696 profile:0(<code object <module> at 0x1053d6a30, file "./manage.py", line 2>)
           1    0.001    0.001    1.693    1.693 manage.py:2(<module>)
           1    0.000    0.000    1.586    1.586 __init__.py:394(execute_from_command_line)
           1    0.000    0.000    1.586    1.586 __init__.py:350(execute)
           1    0.000    0.000    1.142    1.142 __init__.py:254(fetch_command)
          43    0.013    0.000    1.124    0.026 __init__.py:1(<module>)
         388    0.008    0.000    1.062    0.003 re.py:226(_compile)
         158    0.005    0.000    1.048    0.007 sre_compile.py:496(compile)
           1    0.001    0.001    1.042    1.042 __init__.py:78(get_commands)
         153    0.001    0.000    1.036    0.007 re.py:188(compile)
     106/102    0.001    0.000    1.030    0.010 __init__.py:52(__getattr__)
           1    0.000    0.000    1.029    1.029 __init__.py:31(_setup)
           1    0.000    0.000    1.021    1.021 __init__.py:57(_configure_logging)
           2    0.002    0.001    1.011    0.505 log.py:1(<module>)

It's often hard to understand how your own code relates to these traces. Lowkey a maze with no exit. 😵‍💫

Pyinstrument records the entire stack, so tracking expensive calls is so much easier. It also hides library frames by default, letting you focus on how your app/module is affecting performance. ✨

     _     ._   __/__   _ _  _  _ _/_   Recorded: 14:53:35  Samples:  131
    /_//_/// /_\ / //_// / //_'/ //    Duration: 3.131     CPU time: 0.195
   /   _/                    v3.0.0b3

   Program: examples/django_example/manage.py runserver --nothreading --noreload

   3.131 <module>  manage.py:2
   └─ 3.118 execute_from_command_line  django/core/management/__init__.py:378
         [473 frames hidden]  django, socketserver, selectors, wsgi...
            2.836 select  selectors.py:365
            0.126 _get_response  django/core/handlers/base.py:96
            └─ 0.126 hello_world  django_example/views.py:4

💜 'Wall-clock' time (not CPU time) 💜

Pyinstrument records duration using 'wall-clock' time. So when you're writing a program that downloads data, reads files, and talks to databases — all that time is included in the tracked time by pyinstrument. Every last millisecond, accounted for. 🎀

That's really important when debugging performance problems, since Python is often used as a 'glue' language between other services. The problem might not even be in your program, but you should still be able to find out why it's slow. 🌈

💅 Async profiling 💅

pyinstrument can profile async programs that use async and await. This async support works by tracking the 'context' of execution, as provided by the built-in contextvars module (https://docs.python.org/3/library/contextvars.html).

When you start a Profiler with the async_mode (#pyinstrument.Profiler.async_mode) enabled or strict (not disabled), that Profiler is attached to the current async context.

When profiling, pyinstrument keeps an eye on the context. When execution exits the context, it captures the await stack that caused the context to exit. Any time spent outside the context is attributed to that halted execution of the await.

Async contexts are inherited, so tasks started when a profiler is active are also profiled.

[image: Async context inheritance] [image]

pyinstrument supports async mode with Asyncio and Trio — other async/await frameworks should work as long as they use contextvars (https://docs.python.org/3/library/contextvars.html).

Greenlet (https://pypi.org/project/greenlet/) doesn't use async and await, and alters the Python stack during execution, so it is not fully supported. However, because greenlet also supports contextvars (https://docs.python.org/3/library/contextvars.html), we can limit profiling to one green thread, using strict mode. In strict mode, whenever your green thread is halted the time will be tracked in an <out-of-context> frame. Alternatively, if you want to see what's happening when your green thread is halted, you can use async_mode='disabled' — just be aware that readouts might be misleading if multiple tasks are running concurrently. 👀

🫶 footnote moment: [1] Or, your configured interval.

⭐ API REFERENCE ⭐

🌸 Command line interface 🌸

pyinstrument works just like python, on the command line, so you can call your scripts like pyinstrument script.py or pyinstrument -m my_module. So easy, so iconic. 💖

When your script ends, or when you kill it with ctrl-c, pyinstrument will print a profile report to the console.

Usage: pyinstrument [options] scriptfile [arg] ...

okay bestie the options list is still going, we're so back 💅

  • --version — shows the program's version number and dips. ✨
  • -h, --help — shows this help message and exits, classic.
  • --load=FILENAME — instead of running a script, load a profile session from a pyisession file. no script, just vibes from last time 💾
  • --load-prev=IDENTIFIER — instead of running a script, load a previous profile session as specified by an identifier.
  • -m MODULE — run library module as a script, like python -m module. literally the same energy.
  • -c PROGRAM — program passed in as string, like python -c "...".
  • --from-path — (POSIX only!) instead of the working directory, look for scriptfile in the PATH environment variable. 🛣️
  • -o OUTFILE, --outfile=OUTFILE — save to <outfile>.
  • -r RENDERER, --renderer=RENDERER — how the report should be rendered. One of: 'text', 'html', 'json', 'speedscope', 'pyisession', 'pstats', or python import path to a renderer class. Defaults to the appropriate format for the extension if OUTFILE is given, otherwise, defaults to 'text'. the way it just knows 💅
  • -p RENDER_OPTION, --render-option=RENDER_OPTION — options to pass to the renderer, in the format 'flag_name' or 'option_name=option_value'. For example, to set the option 'time', pass -p time=percent_of_total. To pass multiple options, use the -p option multiple times. You can set processor options using dot-syntax, like -p processor_options.filter_threshold=0. option_value is parsed as a JSON value or a string. very customizable of you 🎀
  • -t, --timeline — render as a timeline - preserve ordering and don't condense repeated calls. no folding, no merging, just the raw timeline bestie 🕰️
  • --target-description=TARGET_DESCRIPTION — description text to display in the report. The placeholder '{args}' may be used to include the CLI arguments passed to the target script, including the script name. Default: 'Program: {args}'.
  • --hide=EXPR — glob-style pattern matching the file paths whose frames to hide. Defaults to hiding non-application code. bye non-app frames 👋
  • --hide-regex=REGEX — regex matching the file paths whose frames to hide. Useful if --hide doesn't give enough control. for when you need the big guns 🔥
  • --show=EXPR — glob-style pattern matching the file paths whose frames to show, regardless of --hide or --hide-regex. For example, use --show '*/<library>/*' to show frames within a library that would otherwise be hidden.
  • --show-regex=REGEX — regex matching the file paths whose frames to always show. Useful if --show doesn't give enough control.
  • --show-all — show everything. yes. all of it. the whole thing. 👑
  • --unicode — (text renderer only) force unicode text output. pretty lines ✨
  • --no-unicode — (text renderer only) force ascii text output.
  • --color — (text renderer only) force ansi color text output. rainbow mode activated 🌈
  • --no-color — (text renderer only) force no color text output.
  • -i INTERVAL, --interval=INTERVAL — Minimum time, in seconds, between each stack sample. Smaller values allow resolving shorter duration function calls but incur a greater runtime and memory consumption overhead. For longer running scripts, setting a larger interval reduces the memory consumption required to store the stack samples. it's giving tradeoff, iykyk ⚖️
  • --use-timing-thread — Use a separate thread to time the interval between stack samples. This can reduce the overhead of sampling on some systems. lowkey a glow-up for your sampler 🧵

✨ Python API ✨

The Python API is also available, for calling pyinstrument directly from Python and writing integrations with with other tools. yes bestie, you can import her 💖

🌸 The profile function 🌸

For example:

with pyinstrument.profile():
    time.sleep(1)

This will print something like:

pyinstrument ........................................
.
.  Block at testfile.py:2
.
.  1.000 <module>  testfile.py:1
.  └─ 1.000 sleep  <built-in>
.
.....................................................

You can also use it as a function/method decorator, like this:

@pyinstrument.profile()
def my_function():
    time.sleep(1)

pyinstrument.profile(*, interval=0.001, async_mode='disabled', use_timing_thread=None, renderer=None, target_description=None)

Creates a context-manager or function decorator object, which profiles the given code and prints the output to stdout. 🫶

The interval, async_mode and use_timing_thread parameters are passed through to the underlying pyinstrument.Profiler object.

You can pass a renderer to customise the output. By default, it uses a ConsoleRenderer with short_mode set.

👑 The Profiler object 👑

class pyinstrument.Profiler(interval=0.001, async_mode='enabled', use_timing_thread=None)

The profiler - this is the main way to use pyinstrument. main character energy fr fr

Note the profiling will not start until start() is called. she's waiting for her cue 💅

Parameters

  • interval (float) — See interval.
  • async_mode (AsyncMode) — See async_mode.
  • use_timing_thread (bool | None) — If True, the profiler will use a separate thread to keep track of time. This is useful if you're on a system where getting the time has significant overhead.

property interval: float

The minimum time, in seconds, between each stack sample. This translates into the resolution of the sampling.

property async_mode: str

Configures how this Profiler tracks time in a program that uses async/await.

  • enabled — When this profiler sees an await, time is logged in the function that awaited, rather than observing other coroutines or the event loop.
  • disabled — This profiler doesn't attempt to track await. In a program that uses async/await, this will interleave other coroutines and event loop machinery in the profile. Use this option if async support is causing issues in your use case, or if you want to run multiple profilers at once.
  • strict — Instructs the profiler to only profile the current async context (https://docs.python.org/3/library/contextvars.html). Frames that are observed in another context are ignored, tracked instead as <out-of-context>. very picky of her, we love that 🎯

property last_session: Session <#pyinstrument.session.Session> | None

The previous session recorded by the Profiler. her receipts 📸

start(caller_frame=None, target_description=None)

Instructs the profiler to start - to begin observing the program's execution and recording frames.

The normal way to invoke start() is with a new instance, but you can restart a Profiler that was previously running, too. The sessions are combined. no cap 🔥

Parameters

  • caller_frame (FrameType | None) —

Set this to override the default behaviour of treating the caller of start() as the 'start_call_stack' - the instigator of the profile. Most renderers will trim the 'root' from the call stack up to this frame, to present a simpler output.

You might want to set this to inspect.currentframe().f_back if you are writing a library that wraps pyinstrument.

stop()

Stops the profiler observing, and sets last_session to the captured session.

Returns — The captured session.

Return type — Session <#pyinstrument.session.Session>

property is_running

Returns True if this profiler is running - i.e. observing the program execution.

reset()

Resets the Profiler, clearing the last_session.

__enter__()

Context manager support. 🎉

Profilers can be used in with blocks! See this example:

with Profiler() as p:
    # your code here...
    do_some_work()

# profiling has ended. let's print the output.
p.print()

Print the captured profile to the console, as rendered by renderers.ConsoleRenderer

Parameters

  • file (IO[str]) — the IO stream to write to. Could be a file descriptor or sys.stdout, sys.stderr. Defaults to sys.stdout.

See renderers.ConsoleRenderer for the other parameters.

output_text(unicode=False, color=False, show_all=False, timeline=False, time='seconds', flat=False, flat_time='self', short_mode=False, processor_options=None)

Return the profile output as text, as rendered by ConsoleRenderer

See renderers.ConsoleRenderer for parameter description.

output_html(resample_interval=None)

Return the profile output as HTML, as rendered by HTMLRenderer

See renderers.HTMLRenderer for parameter description.

write_html(path, timeline=False, show_all=False, resample_interval=None)

Writes the profile output as HTML to a file, as rendered by HTMLRenderer

open_in_browser(timeline=False, resample_interval=None) — bestie, this one literally just pops the last profile session open in your web browser. that's it. that's the whole job. 💅

output(renderer) — hands you back the last profile session, all rendered up by whatever renderer you passed in.

  • renderer (Renderer) — the renderer to use. that's the one. 🌸

✨ Sessions ✨

class pyinstrument.session.Session — this is the whole vibe container, ok? it represents a profile session and holds all the data you collected during one. 🎀

static load(filename) — loads a session you saved earlier back off of disk. resurrection behavior, lowkey iconic.

  • filename (str | PathLike[str]) — the path you're loading from.
  • Return type: Session <#pyinstrument.session.Session>

save(filename) — writes a Session object out to disk, in a JSON format. yes bestie, JSON, she's portable.

  • filename (str | PathLike[str]) — the path you're saving to. using the .pyisession extension is recommended. 🫶

static combine(session1, session2) — glues two Session objects together into one.

heads up fr fr: sessions joined this way probably shouldn't be interpreted as timelines, because the samples are simply concatenated. but aggregate views (the default) of this data will work just fine. 💖
  • Return type: Session <#pyinstrument.session.Session>

root_frame(trim_stem=True) — parses the internal frame records and hands you back a whole tree of Frame objects. that tree can then be rendered using a Renderer object. it's giving family tree but for your code. 🌳

  • Return type: a Frame object, or None if the session is empty.

shorten_path(path) — shortens a path into a more readable form, relative to sys_path. used by Frame.short_file_path. she's the glow-up editor of file paths. ✨

resample(interval) — returns a brand new Session object with its frame records resampled to the given interval. glow-up round two.

  • interval (float) — the desired sampling interval in seconds.
  • Return type: Session <#pyinstrument.session.Session>

✨ Renderers ✨

Renderers transform a tree of Frame objects into some form of output. no cap.

Rendering has two steps:

  1. First, the renderer will 'preprocess' the Frame tree, applying each processor in the processor property, in turn.
  2. The resulting tree is rendered into the desired format.

Therefore, rendering can be customised by changing the processors property. For example, you can disable time-aggregation (making the profile into a timeline) by removing aggregate_repeated_calls(). customizing era fr. 💅

class pyinstrument.renderers.FrameRenderer(show_all=False, timeline=False, processor_options=None) — an abstract base class for renderers that process Frame objects using processor functions. provides a common interface to manipulate the processors before rendering. the blueprint of it all. 👑

Parameters:

  • show_all (bool) — don't hide or filter frames — show everything that pyinstrument captures. full transparency, no secrets. 🙈
  • timeline (bool) — instead of aggregating time, leave the samples in chronological order. chronological queen.
  • processor_options (dict[str, Any]) — a dictionary of processor options.

processors: List[Callable[[...], Frame | None]] — the processors installed on this renderer. this property is defined on the base class to provide a common way for users to add and manipulate them before calling render(). 🧚

processor_options: dict[str, Any] — dictionary containing processor options, passed to each processor.

default_processors() — return a list of the processors that this renderer uses by default. the starter pack.

render(session) — return a string that contains the rendered form of frame.

class pyinstrument.renderers.ConsoleRenderer(show_all=False, timeline=False, processor_options=None, unicode=False, color=False, flat=False, time='seconds', flat_time='self', short_mode=False) — produces text-based output, suitable for text files or ANSI-compatible consoles. terminal girlie representation. 🖥️💖

Parameters:

  • unicode (bool) — use unicode, like box-drawing characters in the output. pretty lines ✨
  • color (bool) — enable color support, using ANSI color sequences. rainbow time 🌈
  • flat (bool) — display a flat profile instead of a call graph.
  • time (LiteralStr['seconds', 'percent_of_total']) — how to display the duration of each frame — 'seconds' or 'percent_of_total'
  • flat_time (FlatTimeMode) — show 'self' time or 'total' time (including children) in flat profile.
  • short_mode (bool) — display a short version of the output. she's concise, she's busy. 💅
  • show_all (bool) — see FrameRenderer.
  • timeline (bool) — see FrameRenderer.
  • processor_options (dict[str, Any]) — see FrameRenderer.

class pyinstrument.renderers.HTMLRenderer — renders a rich, interactive web page, as a string of HTML. main character output format, honestly. 👑

Parameters:

  • resample_interval (float | None) — controls how the renderer deals with very large sessions. it typically struggles with sessions of more than 100,000 samples. if the session has more samples than this number, it will be automatically resampled to a coarser interval. you can control this interval with this parameter. if None (the default), the interval will be chosen automatically. setting this to 0 disables resampling.

preprocessors: List[Callable[[...], Frame | None]] — preprocessors installed on this renderer. this property is similar to FrameRenderer.processors, but all pyinstrument's processing is done in the webapp, so these are only used to modify the JSON data sent to the webapp. for example, you might want to use preprocessors to remove unneeded frames from the data to reduce the size of the HTML file. 🦋

preprocessor_options: dict[str, Any] — options to pass to the preprocessors, like FrameRenderer.processor_options.

class pyinstrument.renderers.JSONRenderer(**kwargs) — outputs a tree of JSON, containing processed frames. data girlie. 🎀

Parameters:

  • show_all — don't hide or filter frames — show everything that pyinstrument captures.
  • timeline — instead of aggregating time, leave the samples in chronological order.
  • processor_options — a dictionary of processor options.

class pyinstrument.renderers.SpeedscopeRenderer(**kwargs) — outputs a tree of JSON conforming to the speedscope schema documented at

wiki: <https://github.com/jlfwong/speedscope/wiki/Importing-from-custom-sources> schema: <https://www.speedscope.app/file-format-schema.json> spec: <https://github.com/jlfwong/speedscope/blob/main/src/lib/file-format-spec.ts> example: <https://github.com/jlfwong/speedscope/blob/main/sample/profiles/speedscope/0.0.1/simple.speedscope.json>

Parameters:

  • show_all — don't hide or filter frames — show everything that pyinstrument captures.
  • timeline — instead of aggregating time, leave the samples in chronological order.
  • processor_options — a dictionary of processor options.

✨ Processors ✨

Processors are functions that take a Frame object, and mutate the tree to perform some task. little tree surgeons, basically. 🌸

They can mutate the tree in-place, but also can change the root frame, they should always be called like:

frame = processor(frame, options=...)

pyinstrument.processors.remove_importlib(frame, options) — removes <frozen importlib._bootstrap frames that clutter the output. bye clutter bestie 💅

pyinstrument.processors.remove_tracebackhide(frame, options) — removes frames that have set a local __tracebackhide__ (e.g. __tracebackhide__ = True), to hide them from the output. she saw nothing, no cap.

pyinstrument.processors.aggregate_repeated_calls(frame, options) — converts a timeline into a time-aggregate summary.

Adds together calls along the same call stack, so that repeated calls appear as the same frame. Removes time-linearity — frames are sorted according to total time spent. 🔥

Useful for outputs that display a summary of execution (e.g. text and html outputs)

pyinstrument.processors.group_library_frames_processor(frame, options) — groups frames that should be hidden into FrameGroup objects, according to hide_regex and show_regex in the options dict, as applied to the file path of the source code of the frame. if both match, 'show' has precedence. options:

  • hide_regex — regular expression, which if matches the file path, hides the frame in a frame group.
  • show_regex — regular expression, which if matches the file path, ensures the frame is not hidden

Single frames are not grouped, there must be at least two frames in a group. two's company, one's just lonely. 🥺

pyinstrument.processors.merge_consecutive_self_time(frame, options, recursive=True) — combines consecutive 'self time' frames. merging like a friendship group hug. 🫶

pyinstrument.processors.remove_unnecessary_self_time_nodes(frame, options) — when a frame has only one child, and that is a self-time frame, remove that node and move the time to parent, since it's unnecessary — it clutters the output and offers no additional information. she's tidying up, as she should. ✨

pyinstrument.processors.remove_irrelevant_nodes(frame, options, total_time=None) — remove nodes that represent less than e.g. 1% of the output. options:

  • filter_threshold — sets the minimum duration of a frame to be included in the output. Default: 0.01.

pyinstrument.processors.remove_first_pyinstrument_frames_processor(frame, options) — the first few frames when using the command line are the __main__ of pyinstrument, the eval, and the 'runpy' module. I want to remove that from the output. (not me, the docs said that, but honestly? same energy. 🙈)

Internals notes

Okay so lowkey here's the tea on how this whole thing works under the hood 💅

Frames get recorded by the Profiler in a time-linear fashion. Like, in order, no jumping around. While it's profiling, the profiler builds up a list of frame stacks, and each frame has this exact format:

function_name <null> filename <null> function_line_number

periodt that's the vibe — <null> separators and everything.

Once profiling is done, that list gets turned into a tree structure of Frame objects. 🌳 That tree has literally all the info the profiler gathered, all prepped and ready for a flame render. iconic

Frame objects, the call tree, and processors

So the frames get assembled into a call tree by the profiler session. At this stage the time-linearity is still retained — nothing's been shuffled yet.

Before rendering, that call tree gets fed through a whole sequence of 'processors' whose entire job is transforming the tree for output. 🎀

The most interesting one? That would be aggregate_repeated_calls, bestie — it combines different instances of function calls into the same frame. It's giving "summary of where time was spent during execution," which is honestly so intuitive. ✨

The rest of the processors? They just focus on removing or hiding irrelevant Frames from the output. Clean-up crew energy. 🧹

Self time frames vs. frame.self_time

Self time nodes exist to record time spent in a node, but not in its children. Simple enough, right?

But wait — normal frame objects can have self_time too. Why?? 😭

Okay so here's the answer: frame.self_time is used to store the self_time of any nodes that were removed during processing. So even when a node gets yeeted, its self time doesn't just vanish into the void — it gets parked on frame.self_time. the way that works is actually kind of genius ngl.

INDICES AND TABLES

  • Index
  • Search Page

Author

Joe Rickerby 👑

2021, Joe Rickerby

okay bestie that's a wrap on the whole PYINSTRUMENT(1) page 🫶 this one's from pyinstrument's manual, dated Mar 20, 2026 — go profile something and make it slay ✨💖

📦 where this page comes from