SG_XCOPY(8)

🎀 SG_XCOPY(8) · SG3_UTILS 🎀

✨ NAME ✨

sg_xcopy is the tool that copies data to and from files and devices using SCSI EXTENDED COPY (XCOPY) 💖

💅 SYNOPSIS 💅

sg_xcopy [bs=BS] [conv=CONV] [count=COUNT] [ibs=BS] [if=IFILE]
[iflag=FLAGS] [obs=BS] [of=OFILE] [oflag=FLAGS] [seek=SEEK] [skip=SKIP]
[--help] [--version]

[app=0|1] [bpt=BPT] [cat=0|1] [dc=0|1] [fco=0|1]
[id_usage={hold|discard|disable}] [list_id=ID] [prio=PRIO] [time=0|1]
[verbose=VERB] [--on_dst|--on_src] [--verbose]

🦋 DESCRIPTION 🦋

okay so this utility copies data to and from any files 🫶 It's especially made for "files" that are Linux SCSI devices which support the SCSI EXTENDED COPY (XCOPY) command. Iconic behaviour 🔥

It has similar syntax and semantics to dd(1), but there are no "conversions" supported 🙈

The first group in the synopsis is the "standard" Unix dd(1) operands, and the second group is the extra options this utility adds on top. Both groups are explained below in combined, alphabetical order 📚

By default the XCOPY command gets sent to OFILE. You can change that with the --on_src or iflag=xflag options, which send the XCOPY command to IFILE instead. Also check the ENVIRONMENT VARIABLES section, it's giving important 👑

Now the lore 🌸 In the SPC-4 standard, the T10 committee expanded the XCOPY command so it now has two variants: "LID1" (for a List Identifier length of 1 byte) and "LID4" (for a List Identifier length of 4 bytes). This utility supports the older LID1 variant, which is also in SPC-3 and earlier. The LID1 variant in SPC-4 is command level (binary) compatible with XCOPY as defined in SPC-3, but some of the command naming has changed. This utility sticks with the older SPC-3 XCOPY names, no cap 🧚

The ddpt utility supports the same xcopy(LID1) functionality as this utility, with the same options and flags. On top of that, ddpt supports a subset of xcopy(LID4) functionality, which is variously called "xcopy version 2, lite" or ODX. ODX is a market name and stands for Offloaded Data Xfer (i.e. transfer) ✨

🌈 OPTIONS 🌈

• app={0|1} — if you set this to 1, the destination of the copy starts at the end of OFILE. This assumes OFILE is a regular file. The default is 0, which means the destination starts at the beginning of OFILE (possibly offset by SEEK). You can't use this with the seek=SEEK option, they're enemies 💀
• bpt=BPT — each IO transaction gets made using BPT blocks (or fewer if you're near the end of the copy). The default is 128 for logical block sizes less than 2048 bytes, otherwise the default is 32. So for bs=512, each read and write conveys 64 KiB of data by default (less if you're near the end of the transfer or hitting memory restrictions). When cd/dvd drives are accessed, the logical block size is typically 2048 bytes and bpt defaults to 32, which again implies 64 KiB transfers 🍓
• bs=BS — BS must be the logical block size of the physical device (if the input or output files are accessed via SCSI commands). This differs from dd(1), which lets BS be an integral multiple. It defaults to the device logical block size 💜
• cat={0|1} — sets the SCSI EXTENDED COPY command segment descriptor CAT bit to 0 or 1 (default: 0). The CAT bit, together with the PAD bit, controls how residual data is handled. Check the HANDLING OF RESIDUAL DATA section for the full tea ☕
• conv=CONV — all CONV arguments are ignored. Bestie, don't even bother 🙈
• count=COUNT — copies COUNT blocks from IFILE to OFILE. The default is the minimum number of blocks (IFILE if dc=0, or OFILE if dc=1) that SCSI devices report from SCSI READ CAPACITY commands, or that block devices (or their partitions) report. Normal files aren't probed for their size. If you give skip=SKIP or seek=SEEK and the count is derived (meaning not explicitly given), then the derived count gets scaled back so the copy won't overrun the device. If the file name is a block device partition and COUNT isn't given, the size of the partition is used instead of the whole device. If COUNT isn't given (or count=-1) and can't be derived, an error message is issued and no copy happens 😭
• dc={0|1} — sets the SCSI EXTENDED COPY command segment descriptor DC bit to 0 or 1 (default: 0). The DC bit decides whether COUNT refers to the source (dc=0) or the target (dc=1) descriptor 🎀
• fco={0|1} — sets the SCSI EXTENDED COPY command segment descriptor FCO bit to 0 or 1 (default: 0). When the Fast Copy Only (FCO) bit is set, the copy still happens, but with a technique that's faster than SCSI READ and WRITE commands. If the copy can't be done in a faster way, a sense key of "Copy aborted" with an additional sense of "Fast copy not possible" gets returned 🔥
• ibs=BS — if you give this, it must be the same as the BS given to the bs= option 🫶
• id_usage={hold|discard|disable} — sets the SCSI EXTENDED COPY command parameter list field called LIST ID USAGE to 0 if the argument is hold, to 2 if it's discard, or to 3 if it's disable. If the device can hold data (which shows up as a "held data limit" greater than zero), id_usage defaults to hold. Otherwise it defaults to discard 🧚
• if=IFILE — reads from IFILE instead of stdin. If IFILE is -, then stdin gets read. Reading starts at the beginning of IFILE unless SKIP is given 📚
• iflag=FLAGS — FLAGS is a comma separated list of one or more flags (listed in the FLAGS section). These flags are tied to IFILE and get ignored when IFILE is stdin 💅
• list_id=ID — sets the SCSI EXTENDED COPY command parameter list field called LIST IDENTIFIER to ID. ID should be a value between 0 and 255 (inclusive). ID usually defaults to 1, unless id_usage=disable is set, in which case it defaults to 0 ✨
• obs=BS — if you give this, it must be the same as the BS given to the bs= option 🌸
• of=OFILE — writes to OFILE instead of stdout. If OFILE is -, then it writes to stdout. If OFILE is /dev/null, no actual writes happen. If OFILE is . (a period), it's treated the same as /dev/null (shorthand notation, very efficient). If OFILE already exists, it is _not_ truncated. It gets overwritten from the start of OFILE, unless oflag=append or SEEK is given 🦋
• oflag=FLAGS — FLAGS is a comma separated list of one or more flags (listed in the FLAGS section). These flags are tied to OFILE and get ignored when OFILE is /dev/null, . (period), or stdout 🌈
• prio=PRIO — sets the SCSI EXTENDED COPY command parameter list field called PRIORITY to PRIO. The default value is 1 💖
• seek=SEEK — starts writing SEEK bs-sized blocks from the start of OFILE. The default is block 0 (the start of the file) 🍓
• skip=SKIP — starts reading SKIP bs-sized blocks from the start of IFILE. The default is block 0 (the start of the file) 🧚
• time={0|1} — when 1, it times the transfer and does a throughput calculation, then outputs the results (to stderr) when it's done. When 0 (the default), no timing happens 💜
• verbose=VERB — as VERB goes up, so does the amount of debug output sent to stderr. The default is zero, which gives the minimum amount of debug output. A value of 1 reports extra information that isn't repetitive. A value of 2 reports cdbs and responses for SCSI commands that aren't repetitive (so, anything other than READ and WRITE). Error processing isn't considered repetitive. Values of 3 and 4 output everything for all SCSI commands (and Unix read() and write() calls), so there can be a LOT of output 😭
• -h, --help — outputs the usage message and exits 🙈
• --on_dst — sends the XCOPY command to the output file/device (i.e. OFILE). This is the default unless you override it with --on_src or iflag=xflag. Also check the ENVIRONMENT VARIABLES section 💅
• --on_src — sends the XCOPY command to the input file/device (i.e. IFILE) 👑
• -v, --verbose — the same as verbose=1. When you use it twice, it's the same as verbose=2, and so on 🫶
• -V, --version — outputs version number information and exits 🌸

✨ HANDLING OF RESIDUAL DATA ✨

okay so the pad and cat bits are the ones that decide what happens to leftover (residual) data 🫶 Source and target can be described in different logical block sizes, and if those sizes don't match, residual data is lowkey going to show up. If both logical block sizes are identical, these bits do absolutely nothing bc residual data just won't happen 💅

If none of these bits are set, the EXTENDED COPY command gets aborted with additional sense UNEXPECTED INEXACT SEGMENT 💀

If only the cat bit is set, the residual data is kept and made available for the subsequent segment descriptors. It gets discarded on the last segment descriptor 🦋

If the pad bit is set for the source descriptor only, any residual data for both source or destination gets discarded 🙈

If the pad bit is set for the target descriptor only, any residual source data gets handled as if the cat bit were set 🎀 but any residual destination data gets padded so you get a whole block transfer ✨

If the pad bit is set for both source and target, any residual source data gets discarded and any residual destination data gets padded 👑

---

🧚 ENVIRONMENT VARIABLES 🧚

if your command line doesn't explicitly (and unambiguously) say whether the XCOPY SCSI command should go to IFILE (aka the source) or OFILE (aka the destination), then the tool peeks at the XCOPY_TO_SRC and XCOPY_TO_DST environment variables 👀 If exactly one of them exists (not both, bestie!) it tells the XCOPY command where to go. By default, the XCOPY command goes to OFILE 🌸

---

🦖 RETIRED OPTIONS 🦖

here are some retired options that are still hanging around:

• append=0 | 1 — when set, it's the same as oflag=append. When clear, it overwrites the existing file (if it exists), and that's the default. Check out the append flag for more tea 🍓

---

📝 NOTES 📝

copying data behind an Operating System's back can cause problems 😭 If you're on Linux, go read this link: https://linux-mm.org/Drop_Caches

this command sequence may be useful 🫶

sync; echo 3 > /proc/sys/vm/drop_caches

various numeric arguments (e.g. SKIP) can include multiplicative suffixes or be given in hexadecimal. See the "NUMERIC ARGUMENTS" section in the sg3_utils(8) man page 🔥

the COUNT, SKIP and SEEK arguments can take 64 bit values (i.e. very big numbers) 💖 Everything else is limited to what fits in a signed 32 bit number.

all informative, warning and error output goes to stderr so that dd's output file can be stdout and stay unpolluted ✨ If no options are given, the usage message is output and nothing else happens 🙈

if a device supports xcopy operations, it should set the 3PC field (3PC stands for Third Party Copy) in its standard INQUIRY response. This utility will attempt an xcopy operation no matter what the 3PC field says, but if it's zero (cleared), you'd expect the xcopy operation to fail 💀

you can query the status of the SCSI EXTENDED COPY command with sg_copy_results(sg3_utils) 👀

currently only block-to-block transfers are implemented, so IFILE and OFILE must refer to a SCSI block device 🦋

no account is taken of partitions, so for example /dev/sbc2, /dev/sdc, /dev/sg2, and /dev/bsg/3:0:0:1 would all refer to the same thing: the whole logical unit (i.e. the whole disk) starting at LBA 0 🌈 So any partition indication (e.g. /dev/sdc2) gets ignored. The user should set SKIP, SEEK and COUNT with information obtained from a command like fdisk -l -u /dev/sdc to account for partitions 🎀

XCOPY (LID1) capability has been added to the ddpt utility, which is in a package of the same name 💜 The ddpt utility runs on other OSes (e.g. FreeBSD and Windows) while sg_xcopy only runs on Linux. Also ddpt permits the arguments to ibs= and ibs= to be different 🙈

---

🍓 EXAMPLES 🍓

copy 2M of data from the start of one device to another:

# sg_xcopy if=/dev/sdo of=/dev/sdp count=2048 list_id=2 dc=1
sg_xcopy: if=/dev/sdo skip=0 of=/dev/sdp seek=0 count=1024
Start of loop, count=1024, bpt=65535, lba_in=0, lba_out=0
sg_xcopy: 1024 blocks, 1 command

now check the status of the EXTENDED COPY command:

# sg_copy_results --status --list_id=2 /dev/sdp
Receive copy results (copy status):
    Held data discarded: Yes
    Copy manager status: Operation completed without errors
    Segments processed: 1
    Transfer count units: 0
    Transfer count: 0

---

📡 SIGNALS 📡

the signal handling was borrowed from dd 🫶 SIGINT, SIGQUIT and SIGPIPE output the number of remaining blocks to be transferred and the records in + out counts, and then they do their default action. SIGUSR1 causes the same info to be output, but the copy keeps going 🏃‍♀️ All output caused by signals goes to stderr.

---

🎯 EXIT STATUS 🎯

the exit status of sg_xcopy is 0 when it's successful 🥳 Otherwise, see the sg3_utils(8) man page.

there's an additional exit status of 90 if the flock flag is given and some other process is holding the advisory exclusive lock 🔒

---

👩‍💻 AUTHORS 👩‍💻

written by Hannes Reinecke and Douglas Gilbert 👑

---

🐛 REPORTING BUGS 🐛

report bugs to <dgilbert at interlog dot com> 💌

---

©️ COPYRIGHT ©️

Copyright © 2000-2021 Hannes Reinecke and Douglas Gilbert this software is distributed under the GPL version 2. There is NO warranty, not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE 📜

---

🔗 SEE ALSO 🔗

there's a web page discussing sg_dd at https://sg.danny.cz/sg/sg_dd.html 🌐

a POSIX threads version of this utility called sgp_dd is in the sg3_utils package 🧵 Another version from that package is called sgm_dd, and it uses memory mapped IO to speed up transfers from sg devices 🚀

the lmbench package contains lmdd, which is also super interesting 🤓 For moving data to and from tapes, see dt, which is found at https://www.scsifaq.org/RMiller_Tools/index.html 📼

to change mode parameters that affect a SCSI device's caching and error recovery, see sdparm(sdparm) 🛠️

also see dd(1), sg_copy_results(sg3_utils), ddrescue(GNU), ddpt,ddptctl(ddpt) 💖

---

that's a wrap, bestie!! 🌈 this was sg3_utils 1.47 from September 2021, and you are literally the most prepared person I know 😭🫶💅