NQ(1)

✨ nq — job queue utility ✨

okay bestie, welcome to nq(1) from the General Commands Manual, and no this is not going to be boring I promise 💖

---

🌸 SYNOPSIS 🌸

nq [-c] [-q] command line ...
nq -t job id ...
nq -w job id ...

🦋 DESCRIPTION 🦋

The nq utility is a very lightweight queuing system that needs literally zero setup, zero maintenance, zero supervision, and zero long-running processes. it's giving "self-sufficient queen who doesn't need a babysitter" 👑

Job order is enforced by a timestamp that nq grabs the instant it starts. Synchronization happens on the file-system level, bestie. Timer resolution is milliseconds. No sub-second file system time stamps are required. Polling is not used. Exclusive execution is maintained strictly. no cap 🔥

You enqueue(!) new jobs into the queue by running

nq command line ...

The job id (a file name relative to NQDIR, which defaults to the current directory) gets output — unless you suppressed it with -q — and then nq detaches from the terminal immediately, running the job in the background. Standard output and standard error are redirected into the job id file. And then nqtail(1) can be used to conveniently watch the log files, which is honestly such a vibe 🌈

The options are as follows:

• -c — Clean up job id file when process exited with status 0.
• -q — Suppress output of the job id after spawning new job.
• -t — Enter test mode: exit with status 0 when all of the listed job ids are already done, else with status 1.
• -w — Enter waiting mode: wait in the foreground until all listed job ids are done.

---

💜 ENVIRONMENT 💜

• NQDIR — Directory where lock files/job output resides. Each NQDIR can be considered a separate queue. The current working directory is used when NQDIR is unset. NQDIR is created if needed. it's basically "pick your own queue, icon" 🎀
• NQDONEDIR — When set, specifies a directory (must be on the same file system as NQDIR) where lock files/job output is moved to after successful execution of the job.
  • Ignored when -c is used.
• NQFAILDIR — When set, specifies a directory (must be on the same file system as NQDIR) where lock files/job output is moved to after unsuccessful execution of the job.
• NQJOBID — The job id of the currently running job, exposed to the job itself. a little mirror moment for your job 🪞

---

🍓 FILES 🍓

nq expects to control all files in NQDIR (respectively .) which start with , or .,. These files are created according to the following scheme:

,hexadecimal-time-stamp.pid

fr fr, if you're putting your own little ,-files in there, that's on you, bestie 💀

---

⭐ EXIT STATUS ⭐

The nq utility exits 0 on success, and >0 if an error occurs — unless test mode is used, in which case exit status 1 means there is a job running.

On fatal errors, exit codes 111 and 222 are used.

---

🔥 EXAMPLES 🔥

Build make(1) targets clean, depends, all, without occupying the terminal:

% nq make clean
% nq make depends
% nq make all
% nqtail
... look at output, can interrupt with C-c any time
without stopping the build ...

Simple download queue, accessible from multiple terminals:

% alias qget='NQDIR=/tmp/downloads nq wget'
% alias qwait='NQDIR=/tmp/downloads nqtail -q'
window1% qget http://mymirror/big1.iso
window2% qget http://mymirror/big2.iso
window3% qget http://mymirror/big3.iso
% qwait
... wait for all downloads to finish ...

As nohup(1) replacement (The benchmark will run in background, every run gets a different output file, and the command line you ran is logged too.):

% ssh remote
remote% nq ./run-benchmark
,14f6f3034f8.17035
remote% ^D
% ssh remote
remote% nqtail
... see output, nqtail exits when job finished ...

---

💅 TRICKS 💅

The "file extension" of the log file is actually the PID of the job. nq runs all jobs in a separate process group, so you can kill an entire job process tree at once using kill(1) with a negative PID. Before the job is started, it is the PID of nq, so you can cancel a queued job by killing it as well. so much power in your hands, use it responsibly queen 👑

Thanks to the initial exec line in the log files, you can resubmit a job by executing it as a shell command file, i.e. running

sh job id

Creating nq wrappers setting NQDIR to provide different queues for different purposes is encouraged. go off, make your little queue kingdom 🏰

---

🧚 INTERNALS 🧚

Enforcing job order works like this:

• every job has an output file locked using flock(2) and named according to "FILES".
• every job starts only after all earlier flocked files are unlocked.
• the lock is released by the kernel after the job terminates.

---

🌈 ASSUMPTIONS 🌈

nq will only work correctly when:

• NQDIR (respectively .) is writable.
• flock(2) works correctly in NQDIR (respectively .).
• gettimeofday(2) behaves monotonic (using CLOCK_MONOTONIC would create confusing file names after reboot).
• No other programs put files matching ,* into NQDIR (respectively .).

lowkey these are non-negotiable bestie, break one and the whole vibe check fails 😭

---

🫶 SEE ALSO 🫶

nqtail(1), nqterm(1).

Alternatives to the nq system include batch(1), qsub(1), schedule(1), srun(1), and ts(1). (we're not jealous. we're iconic. there's a difference 💖)

---

👑 AUTHORS 👑

Leah Neukirchen <leah@vuxu.org>

---

💀 CAVEATS 💀

All reliable queue status information is in main memory only, which makes restarting a job queue after a reboot difficult. so like… don't reboot mid-queue unless you're ready for chaos 🙈

---

🎀 LICENSE 🎀

nq is in the public domain.

To the extent possible under law, the creator of this work has waived all copyright and related or neighboring rights to this work.

http://creativecommons.org/publicdomain/zero/1.0/

---

and that's the whole page, bestie — brought to you by the Debian crew, dated July 3, 2024. go queue something, you absolute legend 💜✨