DEBGPT(1)

📦
version 0.8.2 · 3 versions
, in ubuntu stonking

💖 NAME 💖

okay bestie, you're reading DebGPT(1) — the manual page for DebGPT, aka the General Purpose Terminal LLM Tool with Some Debian-Specific Design 🌸

“AI” = “Artificial Idiot” 😭💀 honestly iconic, we stan a self-aware queen

✨ SYNOPSIS ✨

debgpt [arguments] [subcommand [subcommand-arguments]]

🦋 DESCRIPTION 🦋

So DebGPT is a lightweight terminal tool built for everyday hanging-out with Large Language Models (LLMs) 💅 and its whole mission is exploring what LLMs can actually do to help with Debian/Linux development. The possible use cases include general language understanding and editing, code generation, writing documentation, code editing, and more as you can imagine 🌈.

To make that happen, DebGPT gathers the relevant info from all kinds of sources — files, directories, and URLs — and compiles it into a prompt for the LLM 🧚✨. And bestie, it supports a whole roster of LLM backends, both self-hosted and third-party, including:

  • any OpenAI-API compatible service (via the openai frontend)
  • Anthropic
  • Google Gemini
  • ZMQ — DebGPT's built-in backend for self-containment 👑

🎀 TABLE OF CONTENTS 🎀

  • NAME
  • SYNOPSIS
  • DESCRIPTION
  • QUICK START
  • FRONTENDS
  • TUTORIAL
    • 1. Chatting with LLM and CLI Behavior
    • 2. Context Readers for Additional Information
    • 3. Inplace Editing of a File
    • 4. MapReduce for Any Length Context
    • 5. Piping through Everywhere
    • 6. DebGPT Subcommands
    • 7. Prompt Engineering
    • 8. Debian Specific Usage Cases
  • TROUBLESHOOTING
  • BACKENDS
    • ZMQ (DebGPT Built-in)
  • REFERENCES
  • LICENSE and ACKNOWLEDGEMENT

🚀 QUICK START 🚀

First things first, bestie — install DebGPT from PyPI or the Git repository:

pip3 install debgpt
pip3 install git+https://salsa.debian.org/deeplearning-team/debgpt.git

The recommended way to get started is the TUI-based configuration wizard 🧚‍♀️, which walks you through selecting a frontend and providing credentials. If you already have an API key handy, you can skip the wizard and set the environment variable directly (e.g., export OPENAI_API_KEY="your-api-key") 💅.

To (re-)configure at any time, run:

debgpt config

Or use debgpt genconfig to generate a configuration template and place it at $HOME/.config/debgpt/config.toml. Both config and genconfig will inherit any existing configurations ✨.

Upon completion, you can start an interactive chat with the LLM:

debgpt

Enjoy the chat! 💖

Hint: A collection of samples generated using DebGPT can be found at this repository.

🌈 FRONTENDS 🌈

The frontend is a client that communicates with an LLM inference backend. It's the one responsible for sending user input to the backend and receiving responses while — here are the available frontend options (specified by --frontend|-F) 👇

  • openai — the OpenAI-API protocol, compatible with OpenAI's own service as well as any other provider or self-hosted deployment that exposes an OpenAI-compatible endpoint; API reference
  • anthropic — Anthropic's Claude models; API reference
  • google — Google's Gemini models; API reference
  • vllm — a dedicated configuration slot for self-hosted or other OpenAI-API compatible services; vLLM docs
  • zmq — DebGPT's built-in, self-contained LLM backend
  • dryrun — debugging mode; prints the assembled prompt without sending it 👀
ngl bestie, when directing DebGPT at a third-party service, review that provider's terms of use and exercise caution about sharing sensitive material 🔒 — fr fr, protect yourself

✨ TUTORIAL ✨

Okay bestie, listen up — these examples are carefully ordered, so don't be chaotic and jump around. Start with the first one and gradually slide into the next. Baby steps, slay steps. 💅

🌸 1. Interaction Mode with LLM and CLI Behavior

When you hand debgpt zero arguments, it yeets you straight into a general terminal chatting client with LLM backends. Want the deets? Run debgpt -h and it'll show you all the detailed options.

debgpt

While you're vibing in interactive chatting mode, just press / and a whole list of escaped commands pops up — these are the ones that won't be treated as an LLM prompt. Not me having secret menu power 😭

  • /save <path.txt>: saves the last LLM response to the file you named.
  • /reset: clears the context, so you can start a fresh conversation without quitting. Iconic.
  • /quit: quits the chatting mode. You can also smash Ctrl+D to quit. Same energy.

You can also sneak your very first user prompt in through the argument --ask|-A|-a:

debgpt -A "Who are you? And what can LLM do?"

And if you specify the --quit|-Q option, the program dips out right after getting the first response from the LLM. For instance, we can let it mimic fortune with temperature 1.0 (--temperature|-T 1.0) so it's extra random:

debgpt -T 1.0 -QA 'Greet with me, and tell me a joke.'

After every session, the chatting history gets saved in ~/.cache/debgpt as a json file with a unique name. And if you forgot what the LLM said to you (relatable, memory of a goldfish 🐠), the command debgpt replay will replay the last session.

The program can also write the last LLM response to a file through -o <file>, and read the question from stdin:

debgpt -Qa 'write a hello world in rakudo for me' -o hello.raku
debgpt -HQ stdin < question.txt | tee result.txt

So once you're comfy with the fundamental usage and its CLI behavior, we can roll straight into the most important feature of this tool — the special prompt reader, aka MapReduce. Buckle up 🎀

🦋 2. Context Readers for Additional Information

A Context Reader is a function that reads plain text contents from whatever resource you point it at, and wraps them up as part of a prompt for the LLM. The context readers can be combined arbitrarily or specified multiple times through the unified argument --file|-f. It can read from a file, a PDF file, a directory, a URL, a Debian Policy section, a Debian Developer Reference section, a Debian BTS page, a Debian build status page (buildd), a Google search result, etc. So many options, it's giving buffet 🍓

You can find the spec syntax of the unified reader --file|-f with the command debgpt -f : or debgpt -f '?'.

Some simple examples are shown below:

# read a plain text file or PDF file and ask a question
debgpt -Hf README.md -a 'very briefly teach me how to use this DebGPT.'
debgpt -Hf debgpt/policy.py -A 'explain the code'
debgpt -Hf my-resume.pdf -a 'Does this person have any foss-related experience?'

# read a directory
debgpt -Hf debian/ -a 'how is this package built? How many binary packages will be produced?'

# read a URL
debgpt -Hf 'https://www.debian.org/vote/2022/vote_003' -A 'Please explain the differences among the above choices.'

# pipe a command line
debgpt -Hf cmd:'git diff --staged' -A 'briefly summarize the changes and provide a git commit message.'

You can specify the reader multiple times to shove multiple information sources into a single context. And the full list of supported reader specs, plus corresponding examples, can be printed with debgpt -f :, or debgpt -f "?", or debgpt -x :. Go look, you'll be obsessed ✨

💖 3. Inplace Editing of a File

The argument --inplace|-i is for in-place editing of a file. It's a read-write reader that does the same thing as --file|-f (read-only) does — but the inplace one will write the LLM response back into the file. We expect you to use this feature for editing a file. Fancy, I know. 💅

If you specify it, the edits (in UNIX diff format) will be printed to the screen. The --inplace|-i will mandate the --quit|-Q behavior, and will turn off markdown rendering.

The following example asks the LLM to edit the pyproject.toml file, adding pygments to its dependencies. This really works correctly. No cap.

debgpt -Hi pyproject.toml -a 'edit this file, adding pygments to its dependencies.'

If you're working in a Git repository, we can make things even more automated: you may further append --inplace-git-add-commit to automatically add and commit the changes to the Git repository. But if you want to review before commit (main character energy, always double check), specify the --inplace-git-p-add-commit|-I argument instead.

debgpt -Hi pyproject.toml -a 'edit this file, adding pygments to its dependencies.' --inplace-git-add-commit

The commit resulted by the above example can be seen at this link. And recent LLMs are strong enough to easily and correctly add type annotations and doc strings in DebGPT's python codebase, see example here. Programs really said "I'll do it myself" 😭

🔥 4. MapReduce for Any Length Context

The "MapReduce" feature is the choice when you want the LLM to read bulk documentations. Like, huge amounts. It ate.

Generally, LLMs have a limited context length. So if you want to ask a question regarding a very long context, you can split the context into multiple parts, and extract the relevant information from each part. Then, you can ask the LLM to answer the question based on the extracted information. Clever, right?

The implementation of this is fairly simple: split the gathered information texts until the pre-defined maximum chunk size is satisfied, ask the LLM to extract relevant information from each chunk, and then repeatedly merge the extracted information through LLM summarization, until there is only one chunk left. As a result, this functionality can be very quota-consuming if you are going to deal with long texts.

⚠️ Please keep an eye on your bill when you try this on a paid API service. Like, genuinely, watch that bank account bestie 💀

This functionality is implemented as the --mapreduce|-x argument. You have to specify the --ask|-A|-a argument to tell the LLM what kind of question we want to ask, so it can extract the right information. It will simply summarize if the --ask|-A|-a argument is missing.

Some usage examples of MapReduce are as follows:

# load a long file (such as buildlog) and ask question
debgpt -Hx buildlog.txt -A 'Why does the build fail? How to fix it?'
debgpt -Hx 'https://www.debian.org/doc/debian-policy/policy.txt' -A 'what is the purpose of the archive?'
debgpt -Hx policy: -A 'what package should enter contrib instead of main or non-free?'

# this will automatically find the buildlog path
debgpt -Hx sbuild: -A 'why does the build fail? do you have any suggestion?'

# I'm really lazy to learn or recall details
debgpt -H -x policy: -x devref: -a 'which document (and which section) talked about Multi-Arch: ?'

# search google, read pages, and answer question (the --ask|-a|-A is the google search keyword unless specified following "google:")
debgpt -Hx google: -a 'how to start python programming?'
debgpt -Hx google:'debian packaging' -a 'how to learn debian packaging?'

# load a directory and ask question
debgpt -Hx . -a 'which file of this debgpt project implemented mapreduce? how does it work?'
debgpt -Hx . -a 'teach me how to use this software. Is there any hidden functionality that is not written in its readme?'
debgpt -Hx ./debian -A 'how is this package built? how many binary packages will be produced?'

# load Debian mailing list threads and answer question
debgpt -Hx ldo:debian-devel/2025/05 -a 'write a news report to summarize what happened in this month. Always cite the source URL.'
debgpt -Hx ldo:debian-vote/2025/04,05 -a 'what is the result of the AI DFSG gr? Always cite the source URL.'

The full list of supported reader specs, as well as corresponding examples, can be printed using debgpt -f :, or debgpt -f "?", or debgpt -x :.

And real quick — the -H argument will skip printing the first prompt generated by debgpt, because it is typically very lengthy, and only useful for debugging and development purpose. If you want to further tweak the mapreduce behavior, you may want to check the --mapreduce_chunksize <int> and --mapreduce_parallelism <int> arguments. 🎀

🌈 5. Piping through Everywhere

Being able to pipe the inputs and outputs among different programs is one of the reasons why I love the UNIX philosophy. It's giving ✨ elegance ✨

The pipe mode is useful when you want to use debgpt in a shell script. Try the following on the Makefile in the debgpt repo. Later we will introduce an in-place editing functionality which is more convenient than this one, so consider this the warm-up.

cat Makefile | debgpt -a 'delete the deprecated targets' pipe | tee tmp ; mv tmp Makefile; git diff

The pipe mode can also be used for editing something in vim in-place:

# In vim debgpt/task.py, use 'V' mode to select the task_backend function, then
:'<,'>!debgpt -a 'add type annotations and comments to this function' pipe

This looks interesting, right? Well, debgpt also has a git wrapper that automatically generates the git commit message for the staged contents and commits that message. Just try debgpt git commit --amend to see how it works. This will also be mentioned in the subcommands section, iykyk.

👑 6. DebGPT Subcommands

Git subcommand.

Let the LLM automatically generate the git commit message, and call git to commit it:

debgpt git commit --amend

If you don't even want to git commit --amend the committed message, just remove --amend from it. Simple as that 🫶

Sbuild subcommand.

A shortcut to review and fix issues with building a Debian package via sbuild in the current directory. This implies the -H option (hide the first prompt) and automatically asks the LLM to build the package and fix potential problems:

debgpt sbuild

💜 7. Prompt Engineering

As you may have seen, the biggest variation in LLM usage happens in the context — including how you provide the context readers, and how you ask the question through --ask|-A|-a. By adjusting the way you provide that information and ask the question, you can get significantly different results. Lowkey highkey the whole game. To properly make the LLM work for you, you may need to go through some basic prompt engineering methods.

The following are some references on this topic:

  1. OpenAI's Guide https://platform.openai.com/docs/guides/prompt-engineering

Advanced usage of LLM such as Chain-of-Thought (CoT) will not be covered in this document. Please refer to external resources for more information. 🙈

The usage of LLM is limited by our imaginations. I am glad to hear from you if you have more good ideas on how we can make LLMs useful for Debian development: https://salsa.debian.org/deeplearning-team/debgpt/-/issues

🎀 8. Debian Specific Usage Cases

  • Analysis of the sbuild buildlog
  • Analysis of the ratt buildlog directory

🛠️ TROUBLESHOOTING 🛠️

okay bestie, when things get weird, come here first 💖

  • Context overlength — if whatever your context readers fed in is too long (think: you handed --file a genuinely gigantic text file), you've got two moves. Switch to the --mapreduce|-x special reader, or hop over to a model / service provider that eats longer context. That's it, that's the fix 🫶
  • Chunk size (--mapreduce_chunksize) — so here's the tea. Inside DebGPT, text length is measured in bytes. Straight up bytes. Why? Because every LLM out there has its own little tokenizer situation, and DebGPT wants to play nice with as many LLM inference backends as possible. To keep the software development simple, the author just uses bytes. Which means: accurate calculation and detection of context overlength is just not possible 😭 Here's the rough vibe though — figure about 4 characters per token in English. So if your LLM context size is 32k (tokens), you should be able to use roughly 128k (bytes) of chunk size with DebGPT. Math-ish, but make it fashion ✨

🔌 BACKENDS 🔌

So the backend is the server side — the thing that actually runs the LLM inference and talks to the client frontend. It can be self-hosted, or it can be handed to you by a third party. Down here we're only talking about the self-hosted ones, meaning vLLM (or honestly any OpenAI-API compatible service) and the DebGPT built-in ZMQ. 👑

⚡ ZMQ (DebGPT Built-in) ⚡

This tool ships exactly one backend implementation: zmq.

  • zmq — only needed when you pick the ZMQ front end for your self-hosted LLM inference server.

If you're planning to use the openai or dryrun frontends, there's no specific hardware requirement, you're free 🦋 But if you want to self-host the LLM inference backend (the ZMQ backend, that is), bestie... powerful hardware is required. Like, required required.

📚 LLM Selections 📚

The actual hardware requirement depends entirely on which LLM you want. There's a whole buffet of open-access LLMs here — https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

Generally speaking, if you're just doing prompt engineering, the "instruction-tuned" LLMs and the "RL-tuned" LLMs are what we recommend (RL = reinforcement learning, iykyk).

The pretrained (raw) LLMs are not quite useful in this case — they haven't gone through instruction tuning, and they haven't been through the reinforcement learning tuning procedure either. These pretrained LLMs will more likely generate garbage and not follow your instructions at all, or just... repeat your instruction back at you 💀 We'll only revisit the pretrained LLMs when we plan to start collecting data and fine-tune (e.g., LoRA) a model in the far future.

You can specify any Hugging Face model ID directly using the --model_id option. By default, we use:

  • Qwen/Qwen3.5-0.8B (default) — this model is extremely lightweight and requires very little disk space/VRAM 🌸

Different LLMs will pose different hardware requirements. Pleeeease see the "Hardware Requirements" subsection below. ⬇️

💻 Hardware Requirements 💻

By default, we recommend doing LLM inference in fp16 precision. If your VRAM (like CUDA memory) is limited, you may also switch to even lower precisions such as 8bit and 4bit. And for pure CPU inference, we only support fp32 precision right now, no cap.

Also note: Multi-GPU inference is supported by the underlying transformers library. If you have multiple GPUs, this memory requirement is roughly divided by your number of GPUs. Slay ✨

Hardware requirements for the default model (Qwen/Qwen3.5-0.8B):

  • Qwen/Qwen3.5-0.8B + fp16 (cuda/cpu) — extremely lightweight, can run on almost any modern GPU or CPU with 4GB+ RAM/VRAM. She's low-maintenance, we stan 🎀

🚀 Usage of the ZMQ Backend 🚀

If you want to run the default LLM with different precisions:

debgpt backend --max_new_tokens=1024 --device cuda --precision fp16
debgpt backend --max_new_tokens=1024 --device cuda --precision bf16
debgpt backend --max_new_tokens=1024 --device cuda --precision 8bit
debgpt backend --max_new_tokens=1024 --device cuda --precision 4bit

The only supported precision on CPU is fp32 (for now). If you want to fall back to CPU computation (which is very slow, fair warning 😭):

debgpt backend --max_new_tokens=1024 --device cpu --precision fp32

The argument --max_new_tokens does not matter much and you can adjust it — it's the maximum length of each llm reply. Adjust it as you wish, go off 💅

📖 REFERENCES 📖

  1. [1] Access large language models from the command-line — https://github.com/simonw/llm
  2. [2] Turn your task descriptions into precise shell commands — https://github.com/sderev/shellgenius
  3. [3] LangChain: Build context-aware reasoning applications — https://python.langchain.com/docs/introduction/
  4. [4] Moonshot - A simple and modular tool to evaluate and red-team any LLM application. — https://github.com/aiverify-foundation/moonshot?tab=readme-ov-file
  5. [5] LLMxMapReduce (Concurrent work. Their method is more advanced than mine) — https://arxiv.org/abs/2410.09342

⚖️ LICENSE and ACKNOWLEDGEMENT ⚖️

DebGPT development is helped with various open-access and commercial LLMs on code suggestion, code writing, code editing, document writing — with human reviews and modifications. Team effort fr fr 🫶

Copyright (C) 2024-2025 Mo Zhou <lumin@debian.org>

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

👑 AUTHORS 👑

Copyright (C) 2024-2025 Mo Zhou lumin@debian.org

and that's the whole page, bestie 🌈 go forth and prompt responsibly. this one signs off down at the bottom as De...T(1) — you know, the section 1 footer situation with the source/version and the date chilling right there at the page's edge 💖 periodt.

📦 where this page comes from