deltadb_query(1)

💖 deltadb_query(1) — the manual, but make it cute 💖

okay so bestie, welcome to deltadb_query(1) — section 1 of your manual pages, the Cooperative Computing Tools corner of the universe. this is where we spill the tea on querying historical data stored by the catalog server. yes. the catalog server has a whole memory. it remembers things. and we are going to interrogate it. 🔥

---

🌸 NAME 🌸

deltadb_query — query historical data stored by the catalog server. periodt.

---

✨ SYNOPSIS ✨

deltadb_query --db [source_directory] --from [starttime] --to [endtime]
[--filter [expr]] [--where [expr]] [--output [expr]]

like, that's the whole vibe. --db and --from are the non-negotiables, everything else is garnish. 🎀

---

🦋 DESCRIPTION 🦋

deltadb_query is the tool that looks at and displays historical data of the catalog server. literally time travel but for server records. (and here's the lore: if you give the catalog server the -H option, it will record historical data in a format known as DeltaDB — hence the name of this tool. iykyk. 🧚)

the results can show up two ways:

• as a stream of time-ordered updates, or
• as summaries of properties across all records over time.

which is genuinely so useful — think reporting the total resources and clients served by a huge collection of servers over the course of a whole year. like, a whole year of receipts. 👑

also, there's a paper entitled DeltaDB that describes the operation of these tools in detail — see reference below. academia said "we wrote it down" and honestly, slay. 📚

---

💅 ARGUMENTS 💅

• --db path — Query this database directory.
• --file path — Query the data stream in this file.
• --from time — (required) the starting date and time of the query, given either as an absolute time like "YYYY-MM-DD HH:MM:SS" or "YYYY-MM-DD", or as a relative time like 5s, 5m, 5h, 5d meaning five seconds, minutes, hours, or days ago respectively. this one is mandatory bestie, no --from, no query, no cap.
• --to time — the ending time of the query, in the exact same format as the --from option. if you omit it, the current time is assumed. so it's like "up to rn". ⏰
• --every interval — the intervals at which output should be produced, like 5s, 5m, 5h, 5d to indicate five seconds, minutes, hours, or days ago, respectively.
• --epoch — causes the output to be expressed in integer Unix epoch time, instead of a formatted time. raw numbers. feral. 💀
• --filter expr — (multiple) if given, only records matching this expression will be processed. use --filter to apply expressions that do not change over time, such as the name or type of a record.
• --where expr — (multiple) if given, only records matching this expression will be displayed. use --where to apply expressions that may change over time, such as load average or storage space consumed.
• --output expr — (multiple) display this expression on the output.

---

🌈 JX QUERY LANGUAGE 🌈

the --filter, --where, and --output options all make use of the JX query language, which is described URL(here,http://cctools.readthedocs.io/jx). go read it, bestie, it's giving documentation.

and also! the --output clauses may contain one of the following reduction functions:

• COUNT — Give the count of items in the set.
• SUM — Give the sum of the values in the set.
• FIRST — Give the first value encountered in the set.
• LAST — Give the last value encountered in the set.
• MIN — Give the minimum value in the set.
• MAX — Give the maximum value in the set.
• AVERAGE — Give the average value of the set.
• UNIQUE — Give a list of unique values in the set.

(the UNIQUE function can be applied to any record type, while the other reduction functions assume numeric values. so UNIQUE is the flexible icon it is. 💅)

by default, reductions are computed at local scope. this means the reduction is computed across the set of records available at each periodic time step. but if the prefix GLOBAL is added to a reduction, then the reduction is computed across every record over time. global. main character energy. 🌍

for example, this:

--output 'UNIQUE(name)' --every 7d

will display all of the names in the database, at seven day intervals. however, it will not display records that are created and deleted between those seven day intervals. sneaky little records, escaping the window. 🙈

in comparison, this:

--output 'GLOBALUNIQUE(name)' --every 7d

will display all of the names encountered over the last seven days, at seven day intervals. see? GLOBAL catches everybody. no record left behind. ✨

---

🎀 EXAMPLES 🎀

to show 1 week worth of history starting on 15 April 2013:

% deltadb_query --db /data/catalog.history --from 2013-04-15 --to 2015-04-22

to show all history after 1 March 2013:

% deltadb_query --db /data/catalog.history --from 2013-03-01

to show the names of fred's chirp servers where load5 exceeds 2.0:

% deltadb_query --db /data/catalog.history --from 2013-03-01 --filter 'owner=="fred" && type=="chirp"' --where 'load5>2.0' --output name --output load5

to show the average load of all servers owned by fred at one hour intervals:

% deltadb_query --db /data/catalog.history --from 2013-03-01 --filter 'owner=="fred" && type=="chirp"' --output 'AVERAGE(load5)' --every 1h

to show the project names of all Work Queue applications running at a given time:

% deltadb_query --db /data/catalog.history --at 2020-12-01 --filter 'type=="wq_master"' --output 'UNIQUE(project)'

to show the number of managers, tasks, and cores in use for all Work Queue applications:

% deltadb_query --db /data/catalog.history --from 2020-01-01 --to 2021-01-01 --filter 'type=="wq_master"' --output 'COUNT(name)' --output 'SUM(tasks_running)' -- output 'SUM(cores_inuse)'

and here's a little trick: the raw event output of a query can be saved to a file, and then queried using the --file option, which can accelerate operations on reduced data. so you do the heavy lifting once and then vibe. 🦋

for example:

% deltadb_query --db /data/catalog.history --from 2020-01-01 --to 2021-01-01 --filter 'type=="wq_master"' > wq.data
% deltadb_query --file wq.data --output 'COUNT(name)' --output 'SUM(tasks_running)' -- output 'SUM(cores_inuse)'

---

👑 COPYRIGHT 👑

the Cooperative Computing Tools are Copyright (C) 2005-2019 The University of Notre Dame. this software is distributed under the GNU General Public License. see the file COPYING for details. legal rights, but make it fashion. 📜

---

💜 SEE ALSO 💜

• Cooperative Computing Tools Documentation
• catalog_server(1) catalog_update(1) catalog_query(1) chirp_status(1) work_queue_status(1) deltadb_query(1)

---

and that's the whole page bestie, you ate that 💅✨ — this was CCTools 8.0.0 DEVELOPMENT, and yes, it really is deltadb_query(1), same as the one at the top. go query something iconic. 🫶🌸