RASQLINSERT(1)

💖 RASQLINSERT — the rasqlinsert(1) page, but make it a group chat 💖

okay bestie, so this is rasqlinsert from the General Commands Manual section, and it is literally the glow-up your flow data deserves. it takes your argus(8) data and yeets it straight into mysql database tables. no cap. 🦋

---

✨ NAME ✨

rasqlinsert — write argus(8) data into mysql database tables. that's it, that's the whole vibe. 💅

---

🌸 SYNOPSIS 🌸

rasqlinsert [raoptions] [-- filter-expression]

short, sweet, iconic. 🎀

---

💜 DESCRIPTION 💜

so rasqlinsert writes argus data into a mysql database. that's the main character job description. 👑

its principal function is to insert and update flow data attributes into a MySQL database table. and the way it does that? using the exact same syntax and strategies as all the other ra* programs, because consistency is queen. rasqlinsert creates databases and database tables based on the print specification — either on the command line or in the .rarc file. she reads her notes. 📝

here's the concept, and it's lowkey genius: where a ra* program would print fields to standard out in ascii, rasqlinsert will insert those fields into the database as attributes. the flow key, as defined by the -m fields option, provides the definition of any keys that would be used in the schema. and a -m none option will remove the use of any DBMS keys for inserted data — that's the method to use when inserting streaming, unprocessed, primitive argus data into a database table. no keys, no rules, just vibes. ✨

the schema is important for database utility and performance. fr fr. you can use MySQL querys against the attributes that you insert into the tables — stuff like searching and sorting on IP addresses, time, packet counts, etc.... while rasqlinsert does not limit you to the number of attributes (columns) per record you provide, the RDBMS performance will guide you as to how many fields are useful. she'll let you go feral, but the database will side-eye you. 😭

Rasqlinsert by default includes the actual binary argus record in the schema, and inserts and updates the binary record when needed. this enables a large number of fucnctions that extend beyond simple RDBMS schema's that are useful. adding the record is expensive, and some will elect to not use this feature. this can be controlled using the option -s -record as a print field option in the standard ra.1 command line. when the record attribute is present, rasql.1 can read the records directly from the database, to provide additional processing on the database table contents. it's giving ✨bonus content✨.

when keys are used, the database will enforce that any insertions meet the relaitional requirements, i.e. that the keys be unique. and that requirement? it demands a sense of caching and key tracking, which rasqlinsert is specifically designed to provide. she came prepared. 💅

⚠️ okay important warning time: Rasqlinsert by default will append data to existing tables, without checking the schema for consistency. if your schema has keys, and you attempt to append new records to an existing table, there is a high likelyhood for error, as rasqlinsert will attempt to insert a record that collides with an existing flow key. use the -M cache option to cause rasqlinsert to reference the table contents prior to aggregation and insertion. protect yourself bestie 🫶

the binary data that rasqlinsert inserts by default is retrieved with rasql(1).

---

🔥 RASQLINSERT SPECIFIC OPTIONS 🔥

Rasqlinsert, like all ra based clients, supports a number of ra options including filtering of input argus records through a terminating filter expression. the Rasqlinsert(1) specific options are, and yes they all start with -M because branding:

• -M cache — this causes rasqlinsert to use the database table as its persistent cache store. this mechanism is used to control memory use when dealing with large amounts of data and flow keys. basically she outsources her memory to MySQL. iconic delegation. 🧠✨
• -M drop — this causes rasqlinsert to drop any pre-existing database table that has the same name as the target table name, on startup. out with the old, periodt. 💀
• -M rewrite — update SQL tables with a freshly processed record field. this allows existing tables to be modified in place and is provided as a way to reduce the stored flow data. for example, when combined with the -M dsrs= mode it can be used to remove certain DSRs from the in-table flow record.

some commandline parameters are ignored in rewrite mode and some others conflict. ngl this part is strict:

• the cache and drop modes cannot be used with rewrite.
• no input files (-r) may be specified since the source of data is always the table to be rewritten.
• lastly, the field specifier (-s) and flow model (-m) options will be ignored since these parameters are derived from the table being rewritten.

see the INVOCATION section below for an example of the rewrite mode. 👇

---

🌈 INVOCATION 🌈

okay this is the part where it all comes together. buckle up. 🎀

this invocation writes aggregated argus(8) data from the file into a database table. the standard 5-tuple fields, saddr daddr proto sport dport, are used as keys for each entry. rasqlinsert will aggregate all the data prior to inserting the data into the database:

rasqlinsert -r file -w mysql://user@localhost/db/table

because aggregation can require a lot of memory, rasqlinsert provides an option -M cache to have rasqlinsert use the database table as the persistent cache store for the aggregation. with this example, the standard 5-tuple fields, saddr daddr proto sport dport, are still used as keys for each entry, but rasqlinsert will aggregate data over short spans of time as it reads the data from the file, and then commit the data to the database. if additional data arrives that matches that unique flow, rasqlinsert will fetch the entry from the database, aggregate, and then update the data entry in the database. it's a whole little loop and she's obsessed with it. 🔁💖

rasqlinsert -M cache -r file -w mysql://user@localhost/db/table

rasqlinsert can provide the same function for streaming data read directly from an argus data source. this allows rasqlinsert to reassemble all status records for an individual flow, such that the resulting table has only a single entry for each communciation relationship seen. one flow, one row. minimalism, but make it networking. 🦋

rasqlinsert -M cache -S argus -w mysql://user@localhost/db/table

this invocation writes argus(8) data from the file into a database table, without aggregation, by specifying no relational key in the data:

rasqlinsert -m none -r file -w mysql://user@localhost/db/table

this invocation writes argus(8) data from the stream into a database table, without modification:

rasqlinsert -m none -S argus -w mysql://user@localhost/db/table

this invocation writes argus(8) data from the stream into a daily database table, without modification. rasqlinsert will generate table names based on time and insert its data relative to the timestamps found in the flow records it processes. in this specific example, -M time 1d specifies daily tables. one table per day, it's giving calendar girl. 📅✨

rasqlinsert -m none -S argus -w mysql://user@localhost/db/table_%Y_%m_%d -M time 1d

update tables by removing the source and destination user buffer DSRs from the binary flow record column for the month of February, 2018:

rasqlinsert -M rewrite time 1d dsrs=-suser,-duser \
            -t 2018/02 \
            -w mysql://user@localhost/db/table_%Y_%m_%d

---

👑 COPYRIGHT 👑

Copyright (c) 2000-2024 QoSient. All rights reserved.

(the lawyers said keep this in, and honestly? respect. 🫡)

---

💫 SEE ALSO 💫

rasql(1), ra(1), rarc(5), argus(8),

go read those too bestie, the whole friend group is iconic. 💕

---

🎀 AUTHORS 🎀

Carter Bullard (carter@qosient.com).

---

and that's the whole page!! this was rasqlinsert version 5.0.3, dated 8 April 2023 — so if you're reading this in the future, hi, you're valid, check for a newer one. mwah, slay, go insert some flows 💅✨🦋