nvidia-cuda-mps-control(1)

nvidia-cuda-mps-control 💅

okay bestie, welcome to nvidia-cuda-mps-control, section 1 of the manuals — it's the NVIDIA CUDA Multi Process Service management program and honestly? it's giving backstage-pass-to-the-GPU energy. 🎀

---

✨ SYNOPSIS ✨

nvidia-cuda-mps-control [-d | -f]

---

🦋 DESCRIPTION 🦋

MPS is a runtime service designed so that multiple MPI processes using CUDA can run concurrently, in a way that's totally transparent to the MPI program. No cap. A CUDA program runs in MPS mode if the MPS control daemon is running on the system.

So here's the lore, bestie. When CUDA first gets initialized in a program, the CUDA driver tries to connect to the MPS control daemon. If that connection attempt fails, the program just keeps running like it normally would, no MPS, no drama. But if the connection attempt to the control daemon succeeds? The CUDA driver then asks the daemon to start an MPS server on its behalf. And then it branches:

• If there's already an MPS server running, and the user id of that server process matches the user id of the requesting client process, the control daemon just tells the client process about it, and the client goes and connects to the server.
• If there's no MPS server already running on the system, the control daemon launches an MPS server with the same user id (UID) as the requesting client process.
• If there's an MPS server already running but with a different user id than the client process, the control daemon asks the existing server to shut down as soon as all its clients are done. Once that existing server has terminated, the control daemon launches a new server with the same user id as the queued client process.

The MPS server creates the shared GPU context, and it manages its clients. An MPS server can support a finite amount of CUDA contexts, determined by the hardware architecture it's running on. For compute capability SM 3.5 through SM 6.0 the limit is 16 clients per GPU at a time. Compute capability SM 7.0 has a limit of 48. MPS is transparent to CUDA programs — all the complexity of the communication between the client process, the server and the control daemon is hidden away inside the driver binaries. iconic behaviour honestly.

Currently, CUDA MPS is available on 64-bit Linux only, and it requires a device that supports Unified Virtual Address (UVA) and has compute capability SM 3.5 or higher. Applications requiring pre-CUDA 4.0 APIs are not supported under CUDA MPS. Certain capabilities are only available starting with compute capability SM 7.0.

Refer to the MPS documentation on NVIDIA Docs for more details.

---

🌸 OPTIONS 🌸

• -d — Start the MPS control daemon in background mode, assuming the user has enough privilege (e.g. root). The parent process exits once the control daemon has started listening for client connections.
• -f — Start the MPS control daemon in foreground mode, assuming the user has enough privilege (e.g. root). Debug messages get sent to standard output.
• -multiuser-server — Relax the single-UID user requirement, letting any UID connect and share an MPS server that was started as root.
• -p, --protocol [2|3] — Select the protocol before command parsing. Supported values are 2 (legacy MPS, the default) and 3. If CUDA_MPS_PROTOCOL_VERSION is set, the environment value takes precedence. 👑
• -h, --help — Print a help message. bestie behaviour.
• <no arguments> — Start the front-end management user interface to the MPS control daemon, which needs to be started first. The front-end UI keeps reading commands from stdin until EOF. Commands are separated by the newline character. If an invalid command is issued and rejected, an error message gets printed to stdout. The exit status of the front-end UI is zero if communication with the daemon is successful. A non-zero value is returned if the daemon is not found or the connection to the daemon is broken unexpectedly. See the quit command below for more info about the exit status.

🎀 Commands supported by the MPS control daemon

• get_server_list — Print out a list of PIDs of all MPS servers.
• get_server_status PID — Print out the status of the server with the given (PID).
• start_server -uid UID — Start a new MPS server for the specified user (UID).
• shutdown_server PID [-f] — Shut down the MPS server with the given PID. An MPS server only exits after all clients disconnect, and the MPS server may accept new clients while there's a connected client. -f is forced immediate shutdown. If a client launches a faulty kernel that runs forever, a forced shutdown of the MPS server may be required, since the MPS server creates and issues GPU work on behalf of its clients. 💀
• get_client_list PID — Print out a list of PIDs of all clients connected to the MPS server with the given PID.
• quit [-t TIMEOUT] — Shut down the MPS control daemon process and all MPS servers. The MPS control daemon stops accepting new clients while it waits for the current MPS servers and MPS clients to finish. If TIMEOUT is specified (in seconds), the daemon will force the MPS servers to shut down if they're still running after TIMEOUT seconds.

This command is synchronous. The front-end UI waits for the daemon to shut down, then returns the daemon's exit status. The exit status is zero iff all MPS servers have exited gracefully.

💜 Commands available to Volta MPS control daemon

• get_device_client_list PID — List the devices and PIDs of client applications that enumerated this device. It optionally takes the server instance PID.
• set_default_active_thread_percentage percentage — Set the default active thread percentage for MPS servers. If there's already a server spawned, this command will only affect the next server. The set value is lost if a quit command is executed. The default is 100.
• get_default_active_thread_percentage — Query the current default available thread percentage.
• set_active_thread_percentage PID percentage — Set the active thread percentage for the MPS server instance of the given PID. All clients created with that server afterwards will observe the new limit. Existing clients are not affected.
• get_active_thread_percentage PID — Query the current available thread percentage of the MPS server instance of the given PID.
• set_default_device_pinned_mem_limit dev value — Sets the default device pinned memory limit for the MPS servers. If there's already a server spawned, this command will only affect the next server. The value must be in the form of an integer followed by a qualifier, either “G” or “M”, which specifies the value in Gigabytes or Megabytes respectively.
• get_default_device_pinned_mem_limit dev — Query the current default pinned memory limit for the device.
• set_device_pinned_mem_limit PID dev value — Overrides the device pinned memory limit for the MPS server of the given PID. All the clients created with that server afterwards will observe the new limit. Existing clients are not affected.
• get_default_device_pinned_mem_limit PID dev — Query the current device pinned memory limit of the MPS server instance of the given PID for the device dev.
• terminate_client server PID client PID — Terminates all the outstanding GPU work of the MPS client process of the given client PID running on the MPS server denoted by <server PID>. 🫡
• ps [-p PID] — Reports a snapshot of the current client processes.
• set_default_client_priority priority — Set the client priority level that will be used for new clients. Priority values are only considered as hints to the CUDA driver and can be ignored or overridden depending on platform. priority follows a convention where smaller numbers are higher priorities, and the default priority value is 0. The only other supported value for priority is 1, which represents a below-normal priority level.
• get_default_client_priority — Query the current priority value that will be used for new clients.

---

🌈 ENVIRONMENT 🌈

• CUDA_MPS_PIPE_DIRECTORY — Specify the directory that contains the named pipes and UNIX domain sockets used for communication among the MPS control, MPS server, and MPS clients. The value of this environment variable should be consistent in the MPS control daemon and all MPS client processes. Default directory is /tmp/nvidia-mps.
• CUDA_MPS_LOG_DIRECTORY — Specify the directory that contains the MPS log files. This variable is used by the MPS control daemon only. Default directory is /var/log/nvidia-mps.
• CUDA_VISIBLE_DEVICES — Specify which CUDA devices are visible to the control daemon. Takes either an index value or a UUID.
• CUDA_DEVICE_MAX_CONNECTIONS — Specify the preferred number of connections between the host and device.
• CUDA_MPS_ACTIVE_THREAD_PERCENTAGE — Specify the portion of available threads that clients can use on Volta+ hardware. Setting this for the control daemon will set the default active thread percentage for all servers spawned, while setting this for a client or client context will constrain the active thread percentage for that unit and cannot be higher than the active thread percentage value set for the control daemon.
• CUDA_MPS_ENABLE_PER_CTX_DEVICE_MULTIPROCESSOR_PARTITIONING — Specify whether individual client contexts are allowed to have different values for CUDA_MPS_ACTIVE_THREAD_PERCENTAGE.
• CUDA_MPS_PINNED_DEVICE_MEM_LIMIT — Specify the amount of GPU memory available to MPS client processes.
• CUDA_MPS_CLIENT_PRIORITY — Specify the default client priority value at initialization.
• CUDA_MPS_PROTOCOL_VERSION — Selects the MPS protocol version; valid inputs are 3 or 2. When set, this variable takes precedence over the -p, --protocol command-line flag, and is also consulted by clients to route to an MPS control daemon. 👑

---

🍓 FILES 🍓

Log files created by the MPS control daemon in the specified directory.

• control.log — Record startup and shutdown of the MPS control daemon, user commands issued with their results, and status of MPS servers.
• server.log — Record startup and shutdown of MPS servers, and status of MPS clients.

---

fr fr, this is the whole page bestie — every flag, every command, every env var, every log file. no skips, no summaries. periodt. 💖

mwah 💋 — that was nvidia-cuda-mps-control from the NVIDIA crew, dated 2013-02-26, hanging out in section 1 of the manual. go slay some MPI processes, you've got this. 🦋✨