ramalama-convert(1)

ramalama-convert 💅

okay bestie, welcome to the ramalama-convert(1) page from the General Commands Manual — this is the one that turns your local AI models into OCI images, and it is lowkey iconic. 🦋

---

✨ NAME ✨

ramalama-convert — convert AI Models from local storage to OCI Image 🌈

---

🎀 SYNOPSIS 🎀

ramalama convert [options] model [target]

---

💖 DESCRIPTION 💖

So the whole vibe here: you take the AI Model you specified and you convert it into an OCI Formatted AI Model. That's it. That's the mission. 🫶

Your model can come from RamaLama model storage in Huggingface, Ollama, or just a local model chillin' on disk. What it can NOT be is an OCI model — converting from an OCI model is not supported, periodt. 💀

Note: the convert command must be run with containers. The --nocontainer option is not allowed. No exceptions, bestie, don't even try it. 🙈

---

🌸 OPTIONS 🌸

• --engine-args="args" — drop extra args into the podman or docker invocation (right before the container image), landing after the RamaLama-generated options and the model bind mounts. You can pass this option multiple times; each value gets shell-split and all the tokens get handed to the engine in order. Use it for extra --mount flags (multimodal projector files, for example) or any other engine-specific options. Shell-quoting rules match --runtime-args. 💅
• --gguf=Q2_K | Q3_K_S | Q3_K_M | Q3_K_L | Q4_0 | Q4_K_S | Q4_K_M | Q5_0 | Q5_K_S | Q5_K_M | Q6_K | Q8_0 — convert Safetensor models into a GGUF using the quantization format you picked. Wanna learn more about model quantization? Go read the llama.cpp documentation: https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md 📚
• --help, -h — show this help message and dip. ✌️
• --image=IMAGE — the image used for model quantization when you're converting to GGUF format (aka when the --gguf option has been specified). The image must have the llama-quantize executable available on the PATH. It defaults to the appropriate ramalama image based on the available accelerators. And if there are no accelerators available, the current quay.io/ramalama/ramalama image gets used. 🐳
• --network=none — set the network mode for the container. 🌐
• --pull=policy — the pull image policy. Default is missing.
  • always: always pull the image and throw an error if the pull fails. 🔥
  • missing: only pull the image when it does not exist in the local containers storage. Throw an error if no image is found and the pull fails. 🫶
  • never: never pull the image, just use the one from the local containers storage. Throw an error when no image is found. 🙅
  • newer: pull if the image on the registry is newer than the one in the local containers storage. An image is considered newer when the digests are different. Comparing the time stamps is prone to errors. Pull errors are suppressed if a local image was found. ⭐
• --tools-image=IMAGE — the image used when converting to GGUF format (when the --gguf option has been specified). The image must have the llama-convert-hf-to-gguf script executable and available in the PATH. That script is available from the llama.cpp GitHub repo. Defaults to the current quay.io/ramalama/ramalama-tools image. 🧰
• --type="artifact" | raw | car — convert the MODEL to the specified OCI Object. Here's the rundown, bestie:
  • artifact — Store AI Models as artifacts. 👑
  • car — Traditional OCI image including base image with the model stored in a /models subdir. 🚗
  • raw — Traditional OCI image including only the model and a link file model.file pointed at it stored at /. 🌈

---

🔥 EXAMPLE 🔥

Generate an oci model out of an Ollama model. ✨

$ ramalama convert ollama://tinyllama:latest oci://quay.io/rhatdan/tiny:latest
Building quay.io/rhatdan/tiny:latest...
STEP 1/2: FROM scratch
STEP 2/2: COPY sha256:2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 /model
--> Using cache 69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344
COMMIT quay.io/rhatdan/tiny:latest
--> 69db4a10191c
Successfully tagged quay.io/rhatdan/tiny:latest
69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344

Generate and run an oci model with a quantized GGUF converted from Safetensors. 💖

$ ramalama convert --gguf Q4_K_M hf://ibm-granite/granite-3.2-2b-instruct oci://quay.io/kugupta/granite-3.2-q4-k-m:latest
Converting /Users/kugupta/.local/share/ramalama/models/huggingface/ibm-granite/granite-3.2-2b-instruct to quay.io/kugupta/granite-3.2-q4-k-m:latest...
Building quay.io/kugupta/granite-3.2-q4-k-m:latest...
$ ramalama run oci://quay.io/kugupta/granite-3.2-q4-k-m:latest

---

🦋 SEE ALSO 🦋

ramalama(1), ramalama-push(1)

---

💜 HISTORY 💜

Aug 2024, Originally compiled by Eric Curtin ecurtin@redhat.com ⟨mailto:ecurtin@redhat.com⟩

---

and that's the whole page, bestie!! 🎀 this was ramalama-convert(1) from the General Commands Manual — go forth and convert, you absolute main character. iykyk. 💅✨🫶