PYXS(3)

📦
version 0.4.2~git20190115.97f14313 · 2 versions
, in debian bookworm, ubuntu jammy, ubuntu noble

✨ pyxs(3) ✨ — pyxs Documentation 💖

✨ NAME ✨

pyxs — pyxs Documentation 🦋

          .---. .-..-..-.,-. .--.
          : .; `: :; :`.  .'`._-.'
          : ._.'`._. ;:_,._;`.__.'
          : :    .-. :
          :_;    `._.'       0.4.2-dev

          -- XenStore access the Python way!

Okay bestie, so pyxs is a pure Python XenStore client implementation. It covers all of the libxs features and adds some cute Pythonic sugar on top. Here's the shortlist:

  • pyxs supports both Python 2 and 3 ✨
  • works over a Unix socket or XenBus 🦋
  • has a clean and well-documented API 💅
  • is written in easy to understand Python 🧚
  • can be used with gevent or eventlet 🌈

If you have pip you can do the usual:

pip install --user pyxs

Otherwise, download the source from GitHub and run:

python setup.py install

Fedora users can install the package from the system repository:

dnf install python2-pyxs
dnf install python3-pyxs

RHEL/CentOS users can install the package from the EPEL repository:

yum install python2-pyxs
yum install python34-pyxs

Go peek at our brief tutorial or, if you're feeling brave, dive right into the api; pyxs also has a couple of examples in the examples directory. 💖

🦋 TUTORIAL 🦋

✨ Basics ✨

Using pyxs is easy! The only class you need to import is Client. It gives you a simple straightforward API to XenStore content with a bit of Python's syntactic sugar here and there. 🍓

Generally, if you just need to fetch or update some XenStore items you can do:

>>> from pyxs import Client
>>> with Client() as c:
...     c[b"/local/domain/0/name"] = b"Ziggy"
...     c[b"/local/domain/0/name"]
b'Ziggy'

Using Client without the with statement is possible, albeit, not recommended:

>>> c = Client()
>>> c.connect()
>>> c[b"/local/domain/0/name"] = b"It works!"
>>> c.close()

The reason for preferring a context manager is simple: you don't have to DIY. The context manager will make sure that a started transaction was either rolled back or committed and close the underlying XenStore connection afterwards. 💅

🔌 Connections 🔌

pyxs supports two ways of communicating with XenStore:

  • over a Unix socket with UnixSocketConnection;
  • over XenBus with XenBusConnection.

Connection type is determined from the arguments passed to Client constructor. For example, the following code creates a Client instance, operating over a Unix socket:

>>> Client(unix_socket_path="/var/run/xenstored/socket_ro")
Client(UnixSocketConnection('/var/run/xenstored/socket_ro'))
>>> Client()
Client(UnixSocketConnection('/var/run/xenstored/socket'))

Use xen_bus_path argument to initialize a Client with XenBusConnection:

>>> Client(xen_bus_path="/dev/xen/xenbus")
Client(XenBusConnection('/dev/xen/xenbus'))

💫 Transactions 💫

Transactions let you operate on an isolated copy of XenStore tree and merge your changes back atomically on commit. Keep in mind, however, that changes made within a transaction become available to other XenStore clients only if and when committed. Here's an example:

>>> with Client() as c:
...     c.transaction()
...     c[b"/foo/bar"] = b"baz"
...     c.commit()  # !
...     print(c[b"/foo/bar"])
b'baz'

The line with an exclamation mark is a bit careless, because it ignores the fact that committing a transaction might fail. A more robust way to commit a transaction is by using a loop:

>>> with Client() as c:
...     success = False
...     while not success:
...         c.transaction()
...         c[b"/foo/bar"] = b"baz"
...         success = c.commit()

You can also abort the current transaction by calling rollback(). 🫶

👀 Events 👀

When a new path is created or an existing path is modified, XenStore fires an event, notifying all watching clients that a change has been made. pyxs implements watching via the Monitor class. To watch a path create a monitor monitor() and call watch() with a path you want to watch and a unique token. Right after that the monitor will start to accumulate incoming events. You can iterate over them via wait():

>>> with Client() as c:
...    m = c.monitor()
...    m.watch(b"/foo/bar", b"a unique token")
...    next(m.wait())
Event(b"/foo/bar", b"a unique token")

XenStore has a notion of special paths, which start with @ and are reserved for special occasions:

  • @introduceDomain — Fired when a new domain is introduced to XenStore -- you can also introduce domains yourself with a introduce_domain() call, but in most of the cases, xenstored will do that for you.
  • @releaseDomain — Fired when XenStore is no longer communicating with a domain, see release_domain().

Events for both special and ordinary paths are simple two element tuples, where the first element is always event target -- a path which triggered the event and second is a token passed to watch(). A rather unfortunate consequence of this is that you can't get domid of the domain, which triggered @introduceDomain or @releaseDomain from the received event. 😭

🪄 Compatibility API 🪄

pyxs also provides a compatibility interface, which mimics that of xen.lowlevel.xs --- so you don't have to change anything in the code to switch to pyxs:

>>> from pyxs import xs
>>> handle = xs()
>>> handle.read("0", b"/local/domain/0/name")
b'Domain-0'
>>> handle.close()

👑 API REFERENCE 👑

🌟 Client and Monitor 🌟

class pyxs.client.Client(unix_socket_path=None, xen_bus_path=None, router=None)

XenStore client. 💅

Parameters

  • unix_socket_path (str) -- path to XenStore Unix domain socket.
  • xen_bus_path (str) -- path to XenBus device.

If unix_socket_path is given or Client was created with no arguments, XenStore is accessed via UnixSocketConnection; otherwise, XenBusConnection is used.

Each client has a Router thread running in the background. The goal of the router is to multiplex requests from different transaction through a single XenStore connection. 🧵

Changed in version 0.4.0: The constructor no longer accepts connection argument. If you wan't to force the use of a specific connection class, wrap it in a Router:

from pyxs import Router, Client
from pyxs.connection import XenBusConnection

router = Router(XenBusConnection())
with Client(router=router) as c:
    do_something(c)
WARNING: Always finalize the client either explicitly by calling close() or implicitly via a context manager to prevent data loss. 😱
SEE ALSO: Xenstore protocol specification for a description of the protocol, implemented by Client.

connect()

Connects to the XenStore daemon. 🔌

Raises pyxs.exceptions.ConnectionError -- if the connection could not be opened. This could happen either because XenStore is not running on the machine or due to the lack of permissions.

WARNING: This method is unsafe. Please use client as a context manager to ensure it is properly finalized. 🙈

close()

Finalizes the client. 🎀

WARNING: This method is unsafe. Please use client as a context manager to ensure it is properly finalized. 🙈

read(path, default=None)

Reads data from a given path. 📖

Parameters

  • path (bytes) -- a path to read from.
  • default (bytes) -- default value, to be used if path doesn't exist.

write(path, value)

Writes data to a given path. ✍️

Parameters

  • value (bytes) -- data to write.
  • path (bytes) -- a path to write to.

mkdir(path)

Ensures that a given path exists, by creating it and any missing parents with empty values. If path or any parent already exist, its value is left unchanged. 📁

Parameters path (bytes) -- path to directory to create.

delete(path)

Ensures that a given does not exist, by deleting it and all of its children. It is not an error if path doesn't exist, but it is an error if path's immediate parent does not exist either. 🗑️

Parameters path (bytes) -- path to directory to remove.

list(path)

Returns a list of names of the immediate children of path. 📝

Parameters path (bytes) -- path to list.

exists(path)

Checks if a given path exists. 🔍

Parameters path (bytes) -- path to check.

get_perms(path)

Returns a list of permissions for a given path, see InvalidPermission for details on permission format. 🔐

Parameters path (bytes) -- path to get permissions for.

set_perms(path, perms)

Sets a access permissions for a given path, see InvalidPermission for details on permission format. 🔐

Parameters

  • path (bytes) — the path you wanna set perms on, bestie.
  • perms (list) — the list of permissions you're granting. Yes, a whole list. Go off. 💅

walk(top, topdown=True)

Walks all of XenStore and hands you 3-tuples (path, value, children) for every node in the tree, rooted at the node top. It's giving full house tour energy. 🏠✨

Parameters

  • top (bytes) — the node you start from.
  • topdown (bool) — see os.walk() for the deets, iykyk.

get_domain_path(domid)

Gives you back the domain's base path, the one used for relative requests — like b"/local/domain/<domid>". And no cap: if that domid doesn't exist, the answer is undefined. Literally undefined. Not "an error", not "None", undefined. 🫠

Parameters

  • domid (int) — the domain you want the base path for.

is_domain_introduced(domid)

Returns True if xenstored is in communication with the domain — meaning INTRODUCE for that domain hasn't yet been followed by domain destruction or an explicit RELEASE — and False otherwise. Basically: "are y'all still talking?" 👀

Parameters

  • domid (int) — the domain whose status you're checking.

introduce_domain(domid, mfn, eventchn)

Tells xenstored to communicate with this domain. Making the introduction. 🤝

Parameters

  • domid (int) — a real domain id, and heads up, 0 is forbidden. 🚫
  • mfn (int) — address of xenstore page in domid.
  • eventchn (int) — an unbound event chanel in domid.

release_domain(domid)

Manually asks xenstored to disconnect from the domain.

Parameters

  • domid (int) — the domain to disconnect.
NOTE: xenstored will in any case detect domain destruction and disconnect by itself. So this is you doing it on purpose, with your chest. 💪

resume_domain(domid)

Tells xenstored to clear its shutdown flag for a domain. This makes sure that a later shutdown fires the appropriate watches. Un-pausing the drama. ⏯️

Parameters

  • domid (int) — the domain to resume.

set_target(domid, target)

Tells xenstored that a domain is targeting another one, so it should let it tinker with it. This grants domain domid full access to paths owned by target. Domain domid also inherits all permissions granted to target on all other paths. It's giving "you're with me now" energy. 😌

Parameters

  • domid (int) — the domain to set a target for.
  • target (int) — the target domain (yours truly, Captain). 🫡

transaction()

Starts a brand new transaction. Fresh start, clean slate, new era. ✨

Returns int — the transaction handle.

Raises pyxs.exceptions.PyXSError — with errno.EALREADY if this client is already in a transaction. One at a time, bestie, we're not doing that.

WARNING: Currently xenstored has a bug where, after 2**32 transactions, it will allocate id 0 for an actual transaction. Not us. It's a xenstored thing. 💀

rollback()

Rolls back a transaction that's currently in progress. Undo, undo, undo. 🙈

commit()

Commits a transaction that's currently in progress. Sending it. 🔥

Returns bool — False if the commit failed because of the intervening writes, and True otherwise. In any case, the transaction is invalidated. The caller is responsible for starting a new transaction, repeating all of the operations and re-committing. Yes, that's on you, not on the library. 🫶

monitor()

Returns a fresh Monitor instance, which is currently the only way of doing PUBSUB.

The monitor shares the router with its parent client. So closing the client invalidates the monitor. Closing the monitor, on the other hand, had no effect on the router state. Asymmetric, but that's the vibe.

NOTE: Using monitor() over XenBusConnection is currently unsupported, because XenBus does not obey the XenStore protocol specification. See the xen-devel discussion for details.

🦋 class pyxs.client.Monitor(client) 🦋

Monitor implements minimal PUBSUB functionality on top of XenStore. It's small, it's focused, it does exactly one thing and it ate. 💅

>>> with Client() as c:
...    m = c.monitor():
...    m.watch("foo/bar")
...    print(next(c.wait()))
Event(...)

Parameters

  • client (Client) — a reference to the parent client.
NOTE: When used as a context manager, the monitor will try to unwatch all watched paths. So it cleans up after itself. Responsible queen behaviour. 👑

property watched

A set of paths currently watched by the monitor.

close()

Finalizes the monitor by unwatching all watched paths. Closing time, everyone out. 🚪

watch(wpath, token)

Adds a watch. Any alteration to the watched path generates an event — and that includes path creation, removal, contents change or permission change. An event can also be triggered spuriously, so like... don't be surprised. 👀

Changes made in transactions cause an event only if and when committed. Periodt.

Parameters

  • wpath (bytes) — the path to watch.
  • token (bytes) — the watch token, returned in the watch notification.

unwatch(wpath, token)

Removes a watch you added earlier. Unfollowing. 🫥

Parameters

  • wpath (bytes) — the path to unwatch.
  • token (bytes) — the watch token, the one you passed to watch().

wait(unwatched=False)

Yields events for all of the watched paths. Sit back, they're coming. 🎀

An event is a (path, token) pair, where the first element is the event path, i.e. the actual path that was modified, and the second is a token — the one you passed to watch().

Parameters

  • unwatched (bool) — if True, wait() might yield spurious unwatched packets; otherwise these are dropped. Defaults to False.

pyxs.monitor(*args, **kwargs)

A simple shortcut for creating Monitor instances. All arguments are forwarded to the Client constructor. Lazy in the best way. 😌

🌈 Exceptions 🌈

class pyxs.exceptions.PyXSError

Base class for all pyxs exceptions. The mom of the family tree. 👑

class pyxs.exceptions.InvalidOperation

Exception raised when Packet is passed an operation which isn't listed in Op.

Parameters

  • operation (int) — invalid operation value.

class pyxs.exceptions.InvalidPayload

Exception raised when Packet is initialized with a payload which exceeds the 4096 bytes restriction, or contains a trailing NULL.

Parameters

  • operation (bytes) — invalid payload value.

class pyxs.exceptions.InvalidPath

Exception raised when a path processed by a command doesn't match these constraints:

  • its length should not exceed 3072 or 2048 for absolute and relative path respectively.
  • it should only consist of ASCII alphanumerics and the four punctuation characters -/_@ — hyphen, slash, underscore and atsign.
  • it shouldn't have a trailing /, except for the root path.

Parameters

  • path (bytes) — invalid path value.

class pyxs.exceptions.InvalidPermission

Exception raised for permissions which don't match this format:

 w<domid>        write only
 r<domid>        read only
 b<domid>        both read and write
 n<domid>        no access

Parameters

  • perm (bytes) — invalid permission value.

class pyxs.exceptions.ConnectionError

Exception raised for failures during socket operations. The socket said no. 😭

class pyxs.exceptions.UnexpectedPacket

Exception raised when the received packet header doesn't match the header of the packet that was sent — for example, if the outgoing packet has op = Op.READ, the incoming packet is expected to have op = Op.READ as well. Catfishing detection, basically. 🎭

🧚 Internals 🧚

class pyxs.client.Router(connection)

Router.

The goal of the router is to multiplex the XenStore connection between multiple clients and monitors. One connection, many friends. 🫂

Parameters

  • FileDescriptorConnection (connection) — owned by the router. The connection is open when the router is started and remains open until the router is terminated.
NOTE: Python lacks API for interrupting a thread from another thread. This means that a router cannot be stopped when it is blocked in select.select() or wait(). The following two "hacks" are used to ensure prompt termination. 1. A router is equipped with a socket.socketpair(). The reader-end of the pair is selected in the mainloop alongside the XenStore connection, while the writer-end is used in terminate() to force-stop the mainloop. 2. All operations with threading.Condition variables use a 1 second timeout. This "hack" is only relevant for Python prior to 3.2, which didn't allow one to interrupt lock acquisitions. See issue8844 on the CPython issue tracker for details. On Python 3.2 and later no timeout is used.

property is_connected

Checks if the underlying connection is active. Still alive? 🫀

subscribe(token, monitor)

Subscribes a monitor from events with a given token.

unsubscribe(token, monitor)

Unsubscribes a monitor to events with a given token.

send(packet)

Sends a packet to XenStore.

Returns RVar — a reference to the XenStore response.

start()

Starts the router thread. Does nothing if the router is already started. No double-starting, it's above that. ✋

terminate()

Terminates the router. After termination, the router can no longer send or receive packets. Does nothing if the router was already terminated. It's done, it's over, leave it alone. 💀

class pyxs.connection.XenBusConnection(path=None)

XenStore connection through XenBus.

Parameters ✨

  • path (str) — path to XenBus. If you don't pass one explicitly, a predefined OS-specific constant gets used instead. 🛣️

class pyxs.connection.UnixSocketConnection(path=None) 🦋

XenStore connection through Unix domain socket.

Parameters

  • path (str) — path to XenStore unix domain socket. If you don't provide it explicitly, it's restored from the process environment — similar to what libxs does. 🌿

class pyxs._internal.Packet(op, payload, rq_id, tx_id=None) 📦

A message to or from XenStore.

Parameters

  • op (int) — an item from Op, representing the operation performed by this packet. 🎯
  • payload (bytes) — packet payload. Should be a valid ASCII-string with characters between [0x20; 0x7f]. 🔤
  • rq_id (int) — request id — hopefully a unique identifier for this packet. XenStore simply echoes this value back in response. 🔁
  • tx_id (int) — transaction id. Defaults to 0, which means no transaction is running. 💤
Changed in version 0.4.0: rq_id no longer defaults to 0 and should be provided explicitly. 🚨

pyxs._internal.Op 👑

pyxs._internal.Op = Operations(DEBUG=0, DIRECTORY=1, READ=2, GET_PERMS=3, WATCH=4, UNWATCH=5, TRANSACTION_START=6, TRANSACTION_END=7, INTRODUCE=8, RELEASE=9, GET_DOMAIN_PATH=10, WRITE=11, MKDIR=12, RM=13, SET_PERMS=14, WATCH_EVENT=15, ERROR=16, IS_DOMAIN_INTRODUCED=17, RESUME=18, SET_TARGET=19, RESTRICT=128)

Operations supported by XenStore. 🎛️

CONTRIBUTING 💖

Submitting a bug report 🐞

If you're experiencing issues with pyxs, please don't hesitate to report it to the Bug Tracker on GitHub. We got you! 🫶

Setting up development environment 🛠️

Writing a XenStore client library without access to a running XenStore instance can be troublesome. Luckily, there's a way to setup a development using VirtualBox. 🧚

  1. Create a VM running Ubuntu 14.04 or later. 🐧
  2. Install Xen hypervisor: sudo apt-get install xen-hypervisor-4.4-amd64. 📦
  3. Configure VM for SSH access. 🔑
  4. Done! You can now rsync your changes to the VM and run the tests. 🎉

Running the tests 🧪

Only root is allowed to access XenStore, so the tests require sudo:

$ sudo python setup.py test

pyxs strives to work across a range of Python versions. Use tox to run the tests on all supported versions:

$ cat tox.ini
[tox]
envlist = py26,py27,py34,py35,pypy

[testenv]
commands = python setup.py test
$ sudo tox

Style guide 🎀

pyxs follows Pocoo style guide. Please read it before you start implementing your changes. 📚

PYXS CHANGELOG 📜

Here you can see the full list of changes between each pyxs release. 👑

Version 0.4.2-dev 🌱

  • Allowed values to be empty b"". Thanks to Stephen Czetty. See PR #13 on GitHub. 🙌

Version 0.4.1 🐛

Bugfix release, released on May 11th, 2016

  • Fixed a bug in XenBusConnection.create_transport which failed on attribute lookup. See PR #7 on GitHub. 🔧

Version 0.4.0 🚀

Released on March 6th, 2016

  • Fixed a bug in Client.set_permissions which coerced permission lists (e.g. ["b0"]) to repr-strings prior to validation. 🛡️
  • The API is now based around bytes, which means that all methods which used to accept str (or text) now require bytes. XenStore paths and values are specified to be 7-bit ASCII, thus it makes little sense to allow any Unicode string as input and then validate if it matches the spec. 📏
  • Removed transaction argument from Client constructor. The user is advised to use the corresponding methods explicitly. 🗑️
  • Removed connection argument from Client constructor. The user should now wrap it in a Router. 🎁
  • Renamed some of the Client methods to more human-readable names:
    • ls → list
    • rm → delete
    • get_permissions → get_perms
    • set_permissions → set_perms
    • transaction_start → transaction
    • transaction_end → commit and rollback
  • Removed Client.transaction context manager, because it didn't provide a way to handle possible commit failure. 💔
  • Added Client.exists for one-line path existence checks. See PR #6 on GitHub. Thanks to Sandeep Murthy. 👀
  • Removed implicit reconnection logic from FileDescriptorConnection. The user is now expected to connect manually. 🔌
  • Changed XenBusConnection to prefer /dev/xen/xenbus on Linux due to a possible deadlock in XenBus backend. 🐧
  • Changed UnixSocketConnection to use socket.socket instead of the corresponding file descriptor. 🔄
  • Disallowed calling Client.monitor over XenBusConnection. See http://lists.xen.org/archives/html/xen-devel/2016-02/msg03816 for details. 🚫

Version 0.3.1 🌸

Released on November 29th 2012

  • Added default argument to Client.read(), which acts similar to dict.get(). 📖
  • Fixed a lot of minor quirks so pyxs can be Debianized. 🐧

Version 0.3 🦋

Released on September 12th 2011

  • Moved all PUBSUB functionality into a separate Monitor class, which uses a separate connection. That way, we'll never have to worry about mixing incoming XenStore events and command replies. 📡
  • Fixed a couple of nasty bugs in concurrent use of Client.wait() with other commands (see above). 🐞

Version 0.2 🌈

Released on August 18th 2011

  • Completely refactored validation -- no more @spec magic, everything is checked explicitly inside Client.execute_command(). ✨
  • Added a compatibility interface, which mimics xen.lowlevel.xs behaviour, using pyxs as a backend, see pyxs/_compat.py. 🔄
  • Restricted SET_TARGET, INTRODUCE and RELEASE operations to Dom0 only -- /proc/xen/capabilities is used to check domain role. 👑
  • Fixed a bug in Client.wait() -- queued watch events weren't wrapped in pyxs._internal.Event class, unlike the received ones. 🎁
  • Added Client.walk() method for walking XenStore tree -- similar to os.walk(). 🌳

Version 0.1 ⭐

Initial release, released on July 16th 2011

  • Added a complete implementation of XenStore protocol, including transactions and path watching, see pyxs.Client for details. 🎉
  • Added generic validation helper -- @spec, which forces arguments to match the scheme from the wire protocol specification. 📜
  • Added two connection backends -- XenBusConnection for connecting from DomU through a block device and UnixSocketConnection, communicating with XenStore via a Unix domain socket. 🔌

AUTHOR ✍️

Sergei Lebedev, Fedor Gogolev

2011-2022, Selectel

And that's the whole manual page, bestie! 💅 This was pyxs version 0.4.2, dated Jan 31, 2022, and it's the PYXS(3) page. Stay iconic! 💖

📦 where this page comes from