DBROWUNIQ(1)

📦
version 3.9 · 3 versions
, in epel 9

💅 dbrowuniq — the group-chat uniq, but make it fsdb

okay bestie so this is a Perl User Contributed man page for dbrowuniq, and honestly? she's giving uniq but with trust issues. Let's get into it. 🫶

🌸 NAME 🌸

dbrowuniq — eliminate adjacent rows with duplicate fields, maybe counting

like, yes, that's the whole vibe. adjacent dupes get yeeted. optionally we count them. slay. ✨

🦋 SYNOPSIS 🦋

dbrowuniq [-cFLB] [uniquifying fields...]

those are the flags, those are the fields, that's the whole mood. 💜

🎀 DESCRIPTION 🎀

So the ✨mission✨ here is: eliminate adjacent rows with duplicate fields, perhaps counting them. Highly highkey roughly equivalent to the Unix uniq command, but — and this is the main character twist — she can optionally only operate on the fields you specify. iykyk. 🔥

By default, all columns must be unique. Periodt. But if you name specific columns, then only those columns have to be unique, and the first row with those columns is what gets returned. That's the deal. 🍓

⚠️ Real talk: dbrowuniq eliminates only identical rows that are adjacent. If you want to eliminate identical rows across the entire file, you gotta make them adjacent first — maybe by using dbsort on your uniquifying field. No cap, this comes up all the time.

The example the original gives, since it's iconic: input with three lines a/b/a will produce three lines of output with both a's. But if you dbsort it, it becomes a/a/b, and then dbrowuniq will output a/b. See how that works? 🧚

By default, dbrowuniq outputs the first unique row. But if you want, with -L she'll output the last unique row, or with -B she outputs both first and last. (This choice only matters when uniqueness is determined by specific fields — so like, context is everything, bestie.) 💅

dbrowuniq can also count how many unique, adjacent lines she finds with -c, with the count going to a new column (defaulting to count). And if that count column already exists? Incremental counting is possible with -I. With incremental counting, the existing count column is summed. literally just adding them up. ate. 👑

✨ OPTIONS ✨

  • -c or --count — Create a new column (count) which counts the number of times each line occurred. The new column is named by the -N argument, defaulting to count. 💖
  • -N on or --new-name — Specify the name of the count column, if any. Please specify the type with the name, if desired (allowing one to pick sizes smaller than the default quad, if desired). (Default is count:q.) 🎀
  • -I on or --incremental — Incremental counting. If the count column exists, it is assumed to have a partial count and the count accumulates. If the count column doesn't exist, it is created. 🦋
  • -L or --last — Output the last unique row only. By default, it outputs the first unique row. 🌸
  • -F or --first — Output the first unique row only. (This output is the default.) ⭐
  • -B or --both — Output both the first and last unique rows. 🌈

💫 the standard fsdb options (she also supports these, obviously)

  • -d — Enable debugging output. 🔍
  • -i or --input InputSource — Read from InputSource, typically a file name, or - for standard input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects. 📥
  • -o or --output OutputDestination — Write to OutputDestination, typically a file name, or - for standard output, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects. 📤
  • --autorun or --noautorun — By default, programs process automatically, but Fsdb::Filter objects in Perl do not run until you invoke the run() method. The --(no)autorun option controls that behavior within Perl. 🏃‍♀️
  • --header H — Use H as the full Fsdb header, rather than reading a header from then input. (yes, "then input" — that's the original's typo, we're keeping her, she's part of the family now 💀)
  • --help — Show help. 🆘
  • --man — Show full manual. 📖

👑 SAMPLE USAGE 👑

Input:

#fsdb      event
_null_getpage+128
_null_getpage+128
_null_getpage+128
_null_getpage+128
_null_getpage+128
_null_getpage+128
_null_getpage+4
_null_getpage+4
_null_getpage+4
_null_getpage+4
_null_getpage+4
_null_getpage+4
#  | /home/johnh/BIN/DB/dbcol event
#  | /home/johnh/BIN/DB/dbsort event

Command:

cat data.fsdb | dbrowuniq -c

Output:

#fsdb       event   count
_null_getpage+128   6
_null_getpage+4     6
#   2       /home/johnh/BIN/DB/dbcol        event
#  | /home/johnh/BIN/DB/dbrowuniq -c

look at those counts. 6 and 6. immaculate. 🧚

💜 SAMPLE USAGE 2 💜

Retaining the last unique row as an example. (she said last, she meant iconic.)

Input:

    #fsdb event i
    _null_getpage+128 10
    _null_getpage+128 11
    _null_getpage+128 12
    _null_getpage+128 13
    _null_getpage+128 14
    _null_getpage+128 15
    _null_getpage+4 16
    _null_getpage+4 17
    _null_getpage+4 18
    _null_getpage+4 19
    _null_getpage+4 20
    _null_getpage+4 21
    #  | /home/johnh/BIN/DB/dbcol event
    #  | /home/johnh/BIN/DB/dbsort event

Command:

cat data.fsdb | dbrowuniq -c -L event

Output:

    #fsdb event i count
    _null_getpage+128       15      6
    #  | /home/johnh/BIN/DB/dbcol event
    #  | /home/johnh/BIN/DB/dbsort event
    _null_getpage+4 21      6
    #   | dbrowuniq -c

see how she kept 15 and 21 instead of 10 and 16? that's the -L working her magic. 💅

🔥 SAMPLE USAGE 3 🔥

Incremental counting. (the -I moment. the sum girlie era.)

Input:

#fsdb       event   count
_null_getpage+128   6
_null_getpage+128   6
_null_getpage+4     6
_null_getpage+4     6
#  /home/johnh/BIN/DB/dbcol event
#  | /home/johnh/BIN/DB/dbrowuniq -c

Command:

cat data.fsdb | dbrowuniq -I -c event

Output:

    #fsdb event count
    _null_getpage+128   12
    _null_getpage+4     12
    #  /home/johnh/BIN/DB/dbcol     event
    #  | /home/johnh/BIN/DB/dbrowuniq -c
    #   | dbrowuniq -I -c event

6 + 6 = 12. the math is mathing. 😭✨

🫶 SEE ALSO 🫶

Fsdb. (that's it. that's the whole list. go read it bestie.) 🦋

Copyright (C) 1997-2022 by John Heidemann <johnh@isi.edu>

This program is distributed under terms of the GNU general public license, version 2. See the file COPYING with the distribution for details. 📜

okay and finally, the little footer moment you always gotta do — this beauty dropped as perl v5.32.1 on 2024-08-02, and honestly? she's aged like fine wine. go forth and dedupe, bestie. 💖🎀⭐

📦 where this page comes from