SOS_CLEAN(1)

📦
version 4.12.1 · 6 versions
, in fedora 43, fedora 44

👑 SOS_CLEAN(1) — the General Commands Manual glow-up 💅

okay bestie, so this is the manual page for sos_clean and sos_mask, and it lives in the General Commands Manual section. bucket list entry honestly. let's GO 🌸

✨ NAME ✨

sos_clean, sos_mask — obfuscate sensitive data from one or more sos reports 🫶

🎀 SYNOPSIS 🎀

sos clean TARGET [options]
    [--domains]
    [--disable-parsers]
    [--skip-cleaning-files|--skip-masking-files]
    [--keywords]
    [--keyword-file]
    [--usernames]
    [--treat-certificates]
    [--map-file]
    [-j|--jobs]
    [--no-update]
    [--keep-binary-files]
    [--archive-type]
    [--strict-tokenisation]

💖 DESCRIPTION 💖

okay so sos clean or sos mask is an sos subcommand, and its whole job is obfuscating sensitive info out of sos reports you already generated — the stuff the standard plugin-based post processing that runs during report generation did not cover. think IP addresses. the classic oversharer. 😭

here's the cute part: the obfuscation done by this utility is consistent. meaning — say an IP address of 192.168.1.1 in an unprocessed sos report gets obfuscated to, for example, 100.0.0.1 — then it gets changed to 100.0.0.1 in all occurrences found in the report. no doxxing by inconsistency, periodt. ✨

and also, by default, all of these obfuscations get stored in "maps" that are persistently saved to /etc/sos/cleaner/default_mapping and re-used on subsequent runs. she remembers. 🧠💅

this utility can also be used in-line with sos report and sos collect by specifying the --clean or --mask option. iykyk.

when you call it directly via sos clean, the obfuscated archive gets written as an additional file, meaning the original unprocessed report still remains on the filesystem. when it's called via report or collect, the changes are done in-line and so only an obfuscated archive is written and available. in either case, a mapping file containing the relationships between unprocessed and obfuscated elements will be written in the same location as the resulting archive. 🗺️

keep that mapping file PRIVATE, bestie. it literally holds the before-and-after pairs. this mapping file should be kept private by system administrators. 🔐

🌈 REQUIRED ARGUMENTS 🌈

TARGET

The path to the archive that is to be obfuscated. This may be an archive or an unbuilt sos temporary directory. If it's an archive, it will first be extracted and then, after obfuscation is complete, re-compressed using the same compression method as the original. she's polite like that. 🎀

🔥 OPTIONS 🔥

  • --domains DOMAINS — hand over a comma-delimited list of domain names to obfuscate, in addition to those matching the hostname of the system that created the sos report. Subdomains that match a domain given via this option will also get obfuscated. 👑

For example, if --domains redhat.com is specified, then 'redhat.com' will be obfuscated, as will 'www.redhat.com' and subdomains such as 'foo.redhat.com'. slay, all of them gone. 💀

  • --disable-parsers PARSERS — give a comma-delimited list of parsers to disable when cleaning an archive. By default all parsers are enabled.
Note that using this option is very likely to leave sensitive information in place in the target archive, so only use this option when absolutely necessary or you have complete trust in the party/parties that may handle the generated report. 🚨

Valid values for this option are currently: hostname, ip, ipv6, mac, keyword, and username.

  • --skip-cleaning-files, --skip-masking-files FILES — provide a comma-delimited list of files inside an archive that cleaner should skip in cleaning. 🙈

Globs like asterisk are supported, so sos_commands/host/hostname* will match all three usual filenames in that directory (hostname, hostnamectl_status and hostname_-f).

Use this option with caution, only when being certain the given files do not contain any sensitive information. 👀
  • --keywords KEYWORDS — give a comma-delimited list of keywords to scrub in addition to the default parsers.

Keywords provided by this option will be obfuscated as "obfuscatedwordX" where X is an integer based on the keyword's index in the parser. Note that keywords will be replaced as both standalone words and in substring matches. she does not miss. 🔍

  • --keyword-file FILE — provide a file that contains a list of keywords that should be obfuscated. Each word must be specified on a newline within the file. 📝
  • --usernames USERNAMES — give a comma-delimited list of usernames to obfuscate in addition to those discovered by the username parser. 🫶
  • --treat-certificates {obfuscate|keep|remove} — how to treat certificate files (.csr, .crt, .pem) found in the archive. obfuscate converts the file to text and then obfuscates it, keep leaves it in place unmodified, and remove deletes it. Files identified as private keys are always removed regardless of this setting. 🔑💀

Default: obfuscate

  • --map-file FILE — provide a location to a valid mapping file to use as a reference for existing obfuscation pairs. If one is found, the contents are loaded before parsing is started. This allows consistency between runs of this command for obfuscated pairs. By default, sos will write the generated private map file to /etc/sos/cleaner/default_mapping so that consistency is maintained by default. Users may use this option to reference a map file from a different run (perhaps one that was done on another system). 🗂️

Default: /etc/sos/cleaner/default_mapping

  • -j, --jobs JOBS — the number of concurrent archives to process, if more than one. If this utility is called by sos collect then the value of the jobs option for that utility will be used here. ✨

Default: 4

  • --no-update — do not write the mapping file contents to /etc/sos/cleaner/default_mapping. she's not updating her notes today. 🙅♀️
  • --keep-binary-files — keep unprocessable binary files in the archive, rather than removing them. 🩹
Note that binary files cannot be obfuscated, and thus keeping them in the archive may result in otherwise sensitive information being included in the final archive. Users should review any archive that keeps binary files in place before sending to a third party. 🚨

Default: False (remove encountered binary files)

Important exception alert!! Note that uncompressed tarballs produced by the report --pack-dir option (for example proc.tar or sys.tar) are an exception: they are always kept in place regardless of this option, so that the collected data they hold is not silently dropped from the cleaned archive. Their contents are not obfuscated, so the same review caution applies. 👀
  • --archive-type TYPE — specify the type of archive that TARGET was generated as. When sos inspects a TARGET archive, it tries to identify what type of archive it is. For example, it may be a report generated by sos report, or a collection of those reports generated by sos collect, which require separate approaches.

This option may be useful if a given TARGET archive is known to be of a specific type, but due to unknown reasons or some malformed/missing information in the archive directly, that is not properly identified by sos. 😭

The following are accepted values for this option:

  • auto — Automatically detect the archive type
  • report — An archive generated by sos report
  • collect — An archive generated by sos collect
  • insights — An archive generated by the insights-client package

The following may also be used, however note that these do not attempt to pre-load any information from the archives into the parsers. This means that, among other limitations, items like host and domain names may not be obfuscated unless an obfuscated mapping already exists on the system from a previous execution. 💀

  • data-dir — A plain directory on the filesystem.
  • tarball — A generic tar archive not associated with any known tool
  • --strict-tokenisation — enforce more strict tokenisation during the obfuscation process. By default, tokenisation is performed using non-alphanumeric characters as delimiters. This option extends tokenisation to also split on the following escape sequences:
    • URI percent-encoded characters: %XX where XX are two hexadecimal digits (e.g., %20 for space, %1B for escape) 🌐
    • Decimal escape sequences: #NNN where NNN are three decimal digits (e.g., #009 for TAB, #010 for newline, as used by rsyslog/journald) 🔢
    • ANSI terminal escape sequences: \x1B[N;Nm or corrupted format B;N;Nm where N are numbers (e.g., B;1;32m for green text). 🟢

This may be useful for obfuscating sensitive data that appears in encoded or escaped forms within log files and other system data. Note that this option only affects regex-based parsers (hostname, IP address, MAC address); keyword and username parsers always use standard tokenisation. 📌

Default: False

🦋 SEE ALSO 🦋

sos(1) sos-report(1) sos-collect(1) sos.conf(5)

the whole friend group honestly. go read them too. 🫶

💜 MAINTAINER 💜

Maintained on GitHub at https://github.com/sosreport/sos

⭐ AUTHORS & CONTRIBUTORS ⭐

See AUTHORS file in the package documentation.

okay that's the page!! final vibe check: this one is dated Thu May 21 2020 and comes to you from the General Commands Manual (the SOS_CLEAN(1) of it all 💅). that's it bestie, go obfuscate responsibly, and keep that mapping file to yourself. periodt. ✨🎀🦋💖

📦 where this page comes from