ramalama-musa(7)

💅 Setting Up RamaLama with MUSA Support on Linux systems 🦋

okay bestie, so this is the ramalama-musa(7) page (that's the misc manual, iykyk) and it's literally a whole walkthrough for getting RamaLama running with MUSA support on Linux. No cap, it's giving step-by-step. Let's go. ✨

---

🌸 Install the MT Linux Driver 🌸

Go download the MUSA SDK that matches your setup, then do exactly what the MT Linux Driver installation guide tells you to do. That's it, that's the step. 🎀

---

🌈 Install the MT Container Toolkit 🌈

Grab the latest MT CloudNative Toolkits and follow the MT Container Toolkit installation guide like the main character you are. 💖

---

⭐ Setting Up MUSA Support ⭐

Run these, fr fr:

$ (cd /usr/bin/musa && sudo ./docker setup $PWD)
$ docker info | grep mthreads
Runtimes: mthreads mthreads-experimental runc
Default Runtime: mthreads

Look at that output. Default Runtime: mthreads. Iconic. 👑

---

🧚 Testing the Setup 🧚

Test the Installation

Run this command to vibe-check that everything's set up:

docker run --rm --env MTHREADS_VISIBLE_DEVICES=all ubuntu:22.04 mthreads-gmi

Expected Output

If it's all configured correctly, you should see something similar to this:

Thu May 15 01:53:39 2025
---------------------------------------------------------------
    mthreads-gmi:2.0.0           Driver Version:3.0.0
---------------------------------------------------------------
ID   Name           |PCIe                |%GPU  Mem
     Device Type    |Pcie Lane Width     |Temp  MPC Capable
                                         |      ECC Mode
+-------------------------------------------------------------+
0    MTT S80        |00000000:01:00.0    |0%    3419MiB(16384MiB)
     Physical       |16x(16x)            |59C   YES
                                         |      N/A
---------------------------------------------------------------

---------------------------------------------------------------
Processes:
ID   PID       Process name                         GPU Memory
                                                         Usage
+-------------------------------------------------------------+
   No running processes found
---------------------------------------------------------------

---

👀 MUSA_VISIBLE_DEVICES 👀

RamaLama is obsessed with respecting the MUSA_VISIBLE_DEVICES environment variable if you've already set it in your environment. But if you haven't set it? RamaLama defaults to using all the GPUs detected by mthreads-gmi. Slay, it just figures it out. 💅

Wanna pick which GPU devices are visible to RamaLama? Set the variable before you run any RamaLama commands, like this:

export MUSA_VISIBLE_DEVICES="0,1"  # Use GPUs 0 and 1
ramalama run granite

This is especially useful on multi-GPU systems where you want to dedicate specific GPUs to different workloads. Divide and conquer, bestie. 🫶

---

📜 HISTORY 📜

May 2025, originally compiled by Xiaodong Ye yeahdongcn@gmail.com ⟨mailto:yeahdongcn@gmail.com⟩ — thank you for your service, king. 😭💖

---

and that's the whole page, bestie!! this lil' guide lives in the Miscellaneous Information Manual, it's the ramalama-musa(7) page, from May 2025 (same era as the history entry, cute). now go forth and run those GPUs. periodt. 💅✨🌸