ramalama-s...ox-opencode(1)

📦
version 0.25.0 · 4 versions
, in fedora 43, fedora 44, nixpkgs unstable

💅 ramalama-sandbox-opencode 💅

ok so bestie, we're looking at the ramalama-sandbox-opencode(1) page from the General Commands Manual and it is a lot in the best way possible. grab a snack, we're doing this.

✨ NAME ✨

ramalama-sandbox-opencode — run OpenCode in a sandbox, backed by a local AI Model 🦋

🎀 SYNOPSIS 🎀

ramalama sandbox opencode [options] [model ...]

🌸 DESCRIPTION 🌸

okay so here's the vibe: you run OpenCode inside a container, and it's hooked up to a local model server that is also chillin' in a container. OpenCode uses that model for reasoning and tool calling. two little containers, one big brain moment. 💖

Now, the model situation is giving ✨conditional logic✨, so let's break it down:

  • When a single model is specified, the server loads that model directly. easy. slay.
  • When no model is given and the url is given, the command gets the list of models from the model server and takes the first model — or fails without a model. no model, no vibes. 😭
  • When multiple models are specified, the server starts in router mode using llama.cpp's --models-dir, dynamically loading and unloading models. it's giving multitasking icon behavior.
  • When no models and no url, the default http://localhost:{port} is used to get the first model.
  • If there is no server or no model, then it falls back to start a model server in router mode where all GGUF models in the store are served. queen of backup plans. 👑

Use --prompt to pass instructions for non-interactive processing. Commands may also be passed via stdin. periodt.

And now the container tea ☕ — if the url is not given, then two containers are started: a model server (llama-server) and the agent container. They communicate via container networking. When the agent session exits, the model server container is automatically stopped and removed. no loose ends, we love that for us. Otherwise only the agent container starts, using the given url for the openai compatible model server.

✨ OPTIONS ✨

  • --api-key — OpenAI-compatible API key. Can also be set via the RAMALAMA_API_KEY environment variable. 🔑
  • --authfile=path — Path to the authentication file for OCI registries.
  • --backend=auto | vulkan | rocm | cuda | sycl | openvino | cann | musa — GPU backend to use for inference (default: auto).

Available backends depend on the detected GPU hardware. iykyk.

auto (default): Automatically selects the preferred backend based on your GPU:

  • AMD GPUs: vulkan (Linux/macOS) or rocm (WSL2)
  • NVIDIA GPUs: cuda; vulkan available as explicit option
  • Intel GPUs: vulkan (Linux/macOS) or sycl (WSL2); openvino available as explicit option
  • Ascend NPUs: cann
  • MUSA GPUs: musa
  • No GPU: vulkan (CPU fallback)

Platform-specific behavior:

  • On Linux/macOS, Vulkan provides broad compatibility and is preferred for AMD and Intel GPUs
  • On WSL2, Vulkan is a poor default, so vendor-specific backends (rocm, sycl) are preferred. It remains available, and --backend=vulkan still selects it. This covers both Windows, where containers run in the WSL2-backed machine, and ramalama running inside a WSL2 distro, which otherwise looks like Linux

Explicit backend selection:

  • vulkan: Use Vulkan-based inference (compatible with AMD, NVIDIA, Intel, and CPU)
  • rocm: Use AMD ROCm backend (AMD GPUs only)
  • cuda: Use NVIDIA CUDA backend (NVIDIA GPUs only)
  • sycl: Use Intel SYCL/oneAPI backend (Intel GPUs only)
  • openvino: Use Intel OpenVINO backend (Intel GPUs only); uses quay.io/ramalama/openvino
  • cann: Use Huawei CANN backend (Ascend NPUs only); uses quay.io/ramalama/cann
  • musa: Use Moore Threads MUSA backend (MUSA GPUs only); uses quay.io/ramalama/musa

Available choices: The allowed values for --backend are dynamically determined based on your detected GPU hardware. For example, on a system with an AMD GPU, only auto, vulkan, and rocm are available. it's giving tailored. 💅

Configuration: The default can be overridden in the ramalama.conf file or via the RAMALAMA_BACKEND environment variable.

Examples:

# Use auto-detection (default)
ramalama sandbox opencode granite

# Force Vulkan backend
ramalama sandbox opencode --backend vulkan granite

# Force ROCm backend on AMD GPU
ramalama sandbox opencode --backend rocm granite
  • --cache-reuse=BYTES — Minimum chunk size (in bytes) to attempt reusing from the cache via KV shifting. When omitted, llama-server uses its built-in default. recycling is self-care ♻️
  • --ctx-size, -c — size of the prompt context. This option is also available as --max-model-len. Applies to llama.cpp and vllm regardless of alias (default: 0, 0 = loaded from model). big brain = big context 🧠
  • --device — Add a host device to the container. Optional permissions parameter can be used to specify device permissions by combining r for read, w for write, and m for mknod(2).

Example: --device=/dev/dri/renderD128:/dev/xvdc:rwm

The device specification is passed directly to the underlying container engine. See documentation of the supported container engine for more information.

Pass --device=none to explicitly add no device to the container, e.g., for running a CPU-only performance comparison. benchmarking era. 📊

  • --engine-args="args" — Add args to the podman or docker invocation (before the container image), after RamaLama-generated options and model bind mounts. The option may be specified multiple times; each value is shell-split and all tokens are passed to the engine in order. Use for extra --mount flags (for example multimodal projector files) or other engine-specific options. Shell-quoting rules match --runtime-args.
  • --env= — Set environment variables inside the container.

This option allows arbitrary environment variables that are available for the process to be launched inside the container. If an environment variable is specified without a value, the container engine checks the host environment for a value and sets the variable only if it is set on the host. sneaky but valid. 🙈

  • --help, -h — Show this help message and exit. the classic. 🫶
  • --host="127.0.0.1" — IP address for the model server to listen on. Defaults to "127.0.0.1", so the served model is only reachable from the local machine. To expose it on the network, set this to a wildcard address such as "0.0.0.0" (IPv4) or "::" (dual-stack).
bestie. if you set this to a wildcard address, your model is on the network. just saying. 👀
  • --image=IMAGE — OCI container image to run with specified AI model. RamaLama defaults to using images based on the accelerator it discovers and the selected --backend. For example: quay.io/ramalama/ramalama. See the table below for all default images. The default image tag is based on the minor version of the RamaLama package. Version 0.25.0 of RamaLama pulls an image with a :0.25 tag from the quay.io/ramalama OCI repository. The --image option overrides this default.

The default can be overridden in the ramalama.conf file or via the RAMALAMA_IMAGE environment variable. export RAMALAMA_IMAGE=quay.io/ramalama/aiimage:1.2 tells RamaLama to use the quay.io/ramalama/aiimage:1.2 image.

Note: The --backend option provides a higher-level way to select the appropriate image based on GPU type. Use --backend to select vulkan, rocm, cuda, sycl, or openvino backends, which will automatically choose the correct image. Use --image only when you need to override the image selection entirely.

Accelerated images:

  • CPU, Vulkan — quay.io/ramalama/ramalama
  • ROCm (AMD) — quay.io/ramalama/rocm
  • CUDA (NVIDIA) — quay.io/ramalama/cuda
  • Intel GPU (sycl) — quay.io/ramalama/intel-gpu
  • Intel GPU (openvino) — quay.io/ramalama/openvino
  • Asahi (Apple Silicon) — quay.io/ramalama/asahi
  • CANN (Ascend) — quay.io/ramalama/cann
  • MUSA (Moore Threads) — quay.io/ramalama/musa

Upstream llama.cpp "full" images from ghcr.io/ggml-org/llama.cpp are also supported. RamaLama automatically detects the image type and adjusts the container CLI accordingly.

ramalama sandbox opencode --image ghcr.io/ggml-org/llama.cpp:full-vulkan MODEL
  • --keep-groups — pass --group-add keep-groups to podman (default: False). If GPU device on host system is accessible to user via group access, this option leaks the groups into the container.
  • --logfile=path — Log output to a file. receipts, but for your terminal. 📝
  • --max-tokens=integer — Maximum number of tokens to generate. Set to 0 for unlimited output (default: 0). This parameter is mapped to the appropriate runtime-specific parameter:
    • llama.cpp: -n parameter
    • MLX: --max-tokens parameter
    • vLLM: --max-model-len parameter (mapped via ctx_size)
  • --model-draft — A draft model is a smaller, faster model that helps accelerate the decoding process of larger, more complex models, like Large Language Models (LLMs). It works by generating candidate sequences of tokens that the larger model then verifies and refines. This approach, often referred to as speculative decoding, can significantly improve the speed of inferencing by reducing the number of times the larger model needs to be invoked. tiny model doing cardio for the big model. 🏃♀️

Use --runtime-args to pass the other draft model related parameters. Make sure the sampling parameters like top_k on the web UI are set correctly.

  • --models-max=integer — Maximum number of models to load concurrently in router mode (default: 4). Only used when invoked in router mode (zero or multiple models).
  • --name, -n — Name of the container to run the Model in.
  • --ncmoe — Keep the Mixture of Experts (MoE) weights of the first N layers in the CPU.
  • --network=none — set the network mode for the container. hermit mode activated. 🧘
  • --ngl — Number of layers to store in VRAM: a number, auto, or all. When omitted, llama-server defaults to auto.

ok we're not done yet bestie, there's more coming in part 2 💖

  • --oci-runtime — override the default OCI runtime that launches the container. Bestie, container engines like Podman and Docker each have their own default OCI runtime, and this option makes RamaLama override those defaults. ✨

Heads up on Nvidia-based GPU systems: RamaLama defaults to using the nvidia-container-runtime there. Use this option if you wanna override that pick.

  • --opencode-image=IMAGE — the OpenCode container image. That's it, that's the whole vibe. 🎀
  • --port, -p — the port for the AI Model server to listen on. It must be available. If you don't specify one, a free port in the 8080-8180 range gets picked, starting with 8080. 💅

The default can be overridden in the ramalama.conf file.

  • --privileged — okay so by default RamaLama containers are unprivileged (=false) and cannot, for example, modify parts of the operating system. Why? Because by default a container is only allowed limited access to devices. A "privileged" container gets the same access to devices as the user launching the container, with one exception: virtual consoles (/dev/tty\d+) when running in systemd mode (--systemd=always).
🚨 A privileged container turns off the security features that isolate the container from the host. Dropped capabilities, limited devices, read-only mount points, Apparmor/SELinux separation, and Seccomp filters are all disabled. Because of that, the privileged field should almost never be set — containers can easily break out of confinement. Not a vibe. 💀

Containers running in a user namespace (e.g., rootless containers) cannot have more privileges than the user that launched them.

  • --prompt — instructions for the sandbox agent to process non-interactively. Like, set it and let it cook. 🔥
  • --pull=policy — pull image policy. The default is missing. Here's the whole menu, no skips: 🍓
    • always: Always pull the image and throw an error if the pull fails.
    • missing: Only pull the image when it does not exist in the local containers storage. Throw an error if no image is found and the pull fails.
    • never: Never pull the image but use the one from the local containers storage. Throw an error when no image is found.
    • newer: Pull if the image on the registry is newer than the one in the local containers storage. An image is considered to be newer when the digests are different. Comparing the time stamps is prone to errors. Pull errors are suppressed if a local image was found.
  • --runtime-args="args" — add args to the runtime (llama.cpp or vllm) invocation. Give it exactly what you want passed through. ✨
  • --seed= — specify a seed rather than using a random seed for model interaction. 🌸
  • --selinux=true — enable SELinux container separation (default: true). 👑
  • --spec-draft-n-max=N — maximum number of tokens to draft per speculative decoding step (default: 3).
  • --spec-draft-n-min=N — minimum number of draft tokens to use for speculative decoding (default: 0).
  • --spec-draft-p-min=P — minimum speculative decoding probability, greedy threshold (default: 0.0).
  • --spec-type=TYPES — comma-separated list of speculative decoding types to enable. The available types are: draft-simple, draft-eagle3, draft-mtp, ngram-simple, ngram-map-k, ngram-map-k4v, ngram-mod, ngram-cache. When omitted, speculative decoding is disabled.
  • --temp="0.8" — temperature of the response from the AI Model. llama.cpp explains it like this:
The lower the number is, the more deterministic the response. The higher the number is the more creative the response is, but more likely to hallucinate when set too high. Usage: Lower numbers are good for virtual assistants where we need deterministic responses. Higher numbers are good for roleplay or creative tasks like editing stories.
  • --thinking=BOOL — enable or disable thinking mode in reasoning models. Maps to --reasoning on or --reasoning off in llama-server. When omitted, llama-server defaults to auto. 🧚
  • --threads, -t — maximum number of cpu threads to use. The default is to use half the cores when more than 4 cores are available; otherwise, the default is 4 threads.
  • --tls-verify=true — require HTTPS and verify certificates when contacting OCI registries. 🫶
  • --url=URL — the host to send requests to (default: http://localhost:8080).

When the URL points at localhost (the host's loopback, e.g. a ramalama serve container or any local OpenAI-compatible process), the agent container is wired to reach that loopback server without requiring it to be exposed on a container-reachable interface. On native Linux the agent shares the host network namespace and reaches the server directly as localhost. On VM-backed engines (podman machine / Docker Desktop on macOS and Windows) the localhost host is rewritten to host.containers.internal / host.docker.internal, which the VM's proxy forwards to the host's loopback. 🌈

  • --webui=on | off — enable or disable the web UI for the served model (enabled by default). When set to "on" (the default), the web interface is properly initialized. When set to "off", the --no-webui option is passed to the llama-server command to disable the web interface. 💖
  • --workdir, -w — local directory to mount into the sandbox container at /work. 🎀

✨ EXAMPLES ✨

Run the OpenCode agent with default settings:

ramalama sandbox opencode qwen3:4b

Run the OpenCode agent with a custom image:

ramalama sandbox opencode --opencode-image ghcr.io/anomalyco/opencode:v1.3.0 qwen3:4b

Turn off thinking mode in the model the agent is connecting to (may result in faster responses):

ramalama sandbox opencode --thinking=off qwen3:4b

Start an interactive session with access to a local directory:

ramalama sandbox opencode -w ./src qwen3:4b

Request the agent to perform actions non-interactively:

ramalama sandbox opencode -w ./src qwen3:4b --prompt "Please analyze the source code in the current directory"

Send instructions to the agent via stdin:

echo "What is the speed of light in meters per second?" | ramalama sandbox opencode qwen3:4b

👀 SEE ALSO 👀

ramalama(1), ramalama-run(1), ramalama-serve(1), ramalama-sandbox(1)

🦋 HISTORY 🦋

Mar 2026, Originally compiled by Mike Bonnet mikeb@redhat.com mailto:mikeb@redhat.com

and that's a wrap bestie, you read the whole ramalama-s...ox-opencode(1) page from the General Commands Manual, dated Mar 2026 — go forth and sandbox responsibly, mwah 💅💖

📦 where this page comes from