HDFSCLI(1)

👑 HDFSCLI(1) — the manual, but make it cute 💅

okay bestie, welcome to HDFSCLI(1), section 1, User Commands 🫶 this is the manual page for hdfscli-avro — an Avro extension for HdfsCLI. yep. that's the whole vibe. let's get into it ✨

---

🌈 NAME 🌈

hdfscli-avro – an Avro extension for HdfsCLI. that's it, that's the name, no notes 💖

---

✨ SYNOPSIS ✨

here's the shape of it, fr fr. copy these exactly, they're load-bearing:

hdfscli-avro schema [-a ALIAS] [-v...]  HDFS_PATH

hdfscli-avro read [-a ALIAS] [-v...]  [-F FREQ | -n NUM] [-p PARTS]
HDFS_PATH

hdfscli write [-fa ALIAS] [-v...]  [-C CODEC] [-S SCHEMA] HDFS_PATH

hdfscli-avro -L | -h

🫶 not me pointing out that the third one literally says hdfscli write — yes, that's what it says, so that's what you get. character for character, bestie, i don't make the rules 💅

---

💜 DESCRIPTION 💜

hdfscli-avro is the Avro glow-up for HdfsCLI. it lets you peek at schemas, read Avro files straight off HDFS and spit out JSON, and write JSON back in as one single Avro file. three commands, main character energy, no cap 👑

---

🎀 OPTIONS 🎀

🌸 COMMANDS 🌸

• schema — pretty print the schema. like, it makes the schema look nice. iconic.
• read — read an Avro file off of HDFS and output the records as JSON to standard out. yes, straight to stdout, streaming behavior 🦋
• write — read JSON records from standard in and serialize them into a single Avro file on HDFS. in goes the JSON, out goes the Avro 🔥

---

🧚 ARGUMENTS 🧚

• HDFS_PATH — the remote path to an Avro file, or a directory that's holding Avro part-files. either works, she's flexible 💫

---

💅 OPTIONS 💅

• -C CODEC, --codec=CODEC — compression codec, babes. available values are among: null, deflate, snappy. and the default? deflate 🌸 [default: deflate]
• -F FREQ, --freq=FREQ — probability of sampling a record. so you get a lil random subset, lowkey iconic for big files ✨
• -L, --log — show the path to the current log file and exit. that's it, that's all she does 🙈
• -S SCHEMA, --schema=SCHEMA — the schema used for serializing records. if you don't pass it, it gets inferred from the first record. no schema? she'll figure it out 💜
• -a ALIAS, --alias=ALIAS — the alias of the namenode you want to connect to 🫶
• -f, --force — overwrite any existing file. any. be brave, be careful ⚠️
• -h, --help — show a usage message and exit. the classic escape hatch 🎀
• -n NUM, --num=NUM — cap the number of records to output. a lil ceiling on your output 👑
• -p PARTS, --parts=PARTS — which part-files to read. give it a number and she'll randomly select that many, or give it a comma-separated list of numbers and she'll read only those. use a number followed by a comma (e.g. 1,) to get a unique part-file. the default is to read all part-files 💖
• -v, --verbose — enable log output. can be specified up to three times, and each time it gets more verbose. stack 'em up: -vvv 💅

---

🔥 EXAMPLES 🔥

these go hard, ngl:

hdfscli-avro schema /data/impressions.avro

hdfscli-avro read -a dev snapshot.avro >snapshot.jsonl

hdfscli-avro read -F 0.1 -p 2,3 clicks.avro

hdfscli-avro write -f positives.avro <positives.jsonl -S "$(cat schema.avsc)"

see how read pipes right into a .jsonl file? and write slurps up stdin and pushes it into Avro with a schema you cat'd in? obsessed. that's the whole workflow, bestie 🦋✨

---

👀 SEE ALSO 👀

• hdfscli(1) — the main one. go read her too 💜

---

and that's the page!! 💖 written for section User Commands, in October 2021, and the version tag on it is HDFSCLI(1) — that's the source and date, folded in all cute like you asked. periodt. go read some Avro files, you got this 🎀👑🌈