httrack(1)

๐Ÿ’– NAME ๐Ÿ’–

Okay bestie, this is httrack(1) from the General Commands Manual โ€” and itโ€™s literally the offline browser: copy websites to a local directory. ๐Ÿซถ

๐ŸŒธ SYNOPSIS ๐ŸŒธ

The whole flag parade is here. Do not lose a single bracket, or the terminal will side-eye you. ๐Ÿ˜ญ

httrack [ url ]... [ -filter ]... [ +filter ]...  [ -O, --path ] [ -w,
--mirror ] [ -W, --mirror-wizard ] [ -g, --get-files ] [ -i, --continue
] [ -Y, --mirrorlinks ] [ -P, --proxy ] [ -%f, --httpproxy-ftp[=0] ] [
-%b, --bind ] [ -rN, --depth[=N] ] [ -%eN, --ext-depth[=N] ] [ -mN,
--max-files[=N] ] [ -MN, --max-size[=N] ] [ -EN, --max-time[=N] ] [ -AN,
--max-rate[=N] ] [ -%cN, --connection-per-second[=N] ] [ -%G, --pause ]
[ -%JN, --max-retry-after[=N] ] [ -GN, --max-pause[=N] ] [ -cN,
--sockets[=N] ] [ -TN, --timeout[=N] ] [ -RN, --retries[=N] ] [ -JN,
--min-rate[=N] ] [ -HN, --host-control[=N] ] [ -%P,
--extended-parsing[=0] ] [ -n, --near ] [ -t, --test ] [ -%L, --list ] [
-%S, --urllist ] [ -%m, --sitemap ] [ -NN, --structure[=N] ] [ -%N,
--delayed-type-check[=N] ] [ -%D, --cached-delayed-type-check[=0] ] [
-%M, --mime-html[=0] ] [ -%Z, --single-file[=0] ] [ -LN,
--long-names[=N] ] [ -KN, --keep-links[=N] ] [ -x, --replace-external ]
[ -%x, --disable-passwords[=0] ] [ -%q, --include-query-string[=0] ] [
-%g, --strip-query ] [ -%C, --host-alias ] [ -o, --generate-errors[=N] ]
[ -X, --purge-old[=0] ] [ -%p, --preserve ] [ -%T, --utf8-conversion[=0]
] [ -bN, --cookies[=N] ] [ -%K, --cookies-file ] [ -%Y, --why ] [ -u,
--check-type[=N] ] [ -j, --parse-java[=N] ] [ -sN, --robots[=N] ] [ -%h,
--http-10[=0] ] [ -%k, --keep-alive[=0] ] [ -%z,
--disable-compression[=0] ] [ -%B, --tolerant[=0] ] [ -%s,
--updatehack[=0] ] [ -%u, --urlhack[=0] ] [ -%A, --assume ] [ -@iN,
--protocol[=N] ] [ -@mN, --mptcp[=0] ] [ -%w, --disable-module ] [ -F,
--user-agent ] [ -%R, --referer ] [ -%E, --from ] [ -%F, --footer ] [
-%l, --language ] [ -%a, --accept ] [ -%X, --headers ] [ -C, --cache[=N]
] [ -k, --store-all-in-cache ] [ -%r, --warc ] [ -%d, --changes[=0] ] [
-%n, --do-not-recatch[=0] ] [ -%v, --display[=N] ] [ -Q, --do-not-log ]
[ -q, --quiet ] [ -z, --extra-log ] [ -Z, --debug-log ] [ -v, --verbose
] [ -f, --file-log ] [ -f2, --single-log ] [ -I, --index[=0] ] [ -%i,
--build-top-index[=0] ] [ -%I, --search-index[=N] ] [ -pN,
--priority[=N] ] [ -S, --stay-on-same-dir ] [ -D, --can-go-down ] [ -U,
--can-go-up ] [ -B, --can-go-up-and-down ] [ -a, --stay-on-same-address
] [ -d, --stay-on-same-domain ] [ -l, --stay-on-same-tld ] [ -e,
--go-everywhere ] [ -%H, --debug-headers ] [ -%!,
--disable-security-limits[=0] ] [ -V, --userdef-cmd ] [ -%W, --callback
] [ -y, --background-on-suspend[=0] ] [ -K, --keep-links[=N] ]

โœจ DESCRIPTION โœจ

So httrack lets you download a World Wide Web site from the Internet into a local directory, building recursively all directories and grabbing HTML, images, and other files from the server to your computer. It arranges the original siteโ€™s relative link-structure. Then you literally open a page of the "mirrored" website in your browser, and you can browse the site from link to link, as if you were viewing it online. No cap. HTTrack can also update an existing mirrored site, and resume interrupted downloads. ๐Ÿ’…

๐ŸŽ€ EXAMPLES ๐ŸŽ€

Mirror one site and only that site:

httrack www.example.com/bob/

mirror site www.example.com/bob/ and only this site

Mirror two sites together, with shared links, and accept any .jpg files on .com sites:

httrack www.example.com/bob/ other.example.com/mike/ +*.com/*.jpg -mime:application/*

mirror the two sites together (with shared links) and accept any .jpg files on .com sites

httrack www.example.com/bob/bobby.html +* -r6
httrack www.example.com/bob/bobby.html --spider -P proxy.example.com:8080
httrack --update
httrack
httrack --continue

๐Ÿ’… OPTIONS ๐Ÿ’…

Every flag, every default, every weird little * โ€” all here. The little * marks are kept exactly where the original put them, bestie. โœจ

๐Ÿซถ General options ๐Ÿซถ

  • -O โ€” path for mirror/logfiles+cache (-O path_mirror[,path_cache_and_logfiles]) (--path <param>). This is where the mirror, logfiles, and cache go. ๐Ÿ’–

๐ŸŽฌ Action options ๐ŸŽฌ

  • -w โ€” *mirror web sites (--mirror). Iconic main character move.
  • -W โ€” mirror web sites, semi-automatic (asks questions) (--mirror-wizard). Itโ€™s giving guided tour energy.
  • -g โ€” just get files (saved in the current directory) (--get-files). No big recursive website production, just files.
  • -i โ€” continue an interrupted mirror using the cache (--continue). Your โ€œwe were so backโ€ option.
  • -Y โ€” mirror ALL links located in the first level pages (mirror links) (--mirrorlinks). ALL of them, bestie. ๐Ÿ‘‘

๐ŸŒˆ Proxy options ๐ŸŒˆ

  • -P โ€” proxy use (-P [socks5://|connect://][user:pass@]proxy:port) (--proxy <param>).
  • -%f โ€” *use proxy for ftp (f0 don't use) (--httpproxy-ftp[=0]).
  • -%b โ€” use this local hostname to make/send requests (-%b hostname) (--bind <param>).

๐Ÿ“ Limits options ๐Ÿ“

  • -rN โ€” set the mirror depth to N (* r9999) (--depth[=N]).
  • -%eN โ€” set the external links depth to N (* %e0) (--ext-depth[=N]).
  • -mN โ€” maximum file length for a non-html file (--max-files[=N]).
  • -mN,N2 โ€” maximum file length for non html (N) and html (N2).
  • -MN โ€” maximum overall size that can be uploaded/scanned (--max-size[=N]).
  • -EN โ€” maximum mirror time in seconds (60=1 minute, 3600=1 hour) (--max-time[=N]).
  • -AN โ€” maximum transfer rate in bytes/seconds (1000=1KB/s max) (--max-rate[=N]).
  • -%cN โ€” maximum number of connections/seconds (*%c5) (--connection-per-second[=N]).
  • -%G โ€” random pause of MIN[:MAX] seconds between files (e.g. %G5:10) (--pause <param>).
  • -%JN โ€” longest Retry-After obeyed, in seconds (*%J60, 0=no wait) (--max-retry-after[=N]).
  • -GN โ€” pause transfer if N bytes reached, and wait until lock file is deleted (--max-pause[=N]).

๐Ÿšฆ Flow control ๐Ÿšฆ

  • -cN โ€” number of multiple connections (*c4) (--sockets[=N]).
  • -TN โ€” timeout, number of seconds after a non-responding link is shutdown; also bounds host name resolution (--timeout[=N]).
  • -RN โ€” number of retries, in case of timeout or non-fatal errors (*R2) (--retries[=N]).
  • -JN โ€” traffic jam control, minimum transfert rate (bytes/seconds) tolerated for a link (--min-rate[=N]).
  • -HN โ€” host is abandoned if: 0=never, 1=timeout, 2=slow, 3=timeout or slow (--host-control[=N]).
  • -%P โ€” *extended parsing, attempt to parse all links, even in unknown tags or Javascript (%P0 don't use) (--extended-parsing[=0]).
  • -n โ€” get non-html files 'near' an html file (ex: an image located outside) (--near).
  • -t โ€” test all URLs (even forbidden ones) (--test).
  • -%L โ€” <file> add all URL located in this text file (one URL per line) (--list <param>).
  • -%S โ€” <file> add all scan rules located in this text file (one scan rule per line) (--urllist <param>).
  • -%m โ€” seed the crawl from the site's sitemap (robots.txt Sitemap:, then /sitemap.xml); --sitemap-url URL names one explicitly. A sitemap you name, or one the site declares, is fetched even under robots.txt Disallow; only the guessed /sitemap.xml obeys it. The URLs found still pass every filter and scope rule (--sitemap).

๐Ÿงš Build options ๐Ÿงš

  • -NN โ€” structure type (0 *original structure, 1+: see below) (--structure[=N]) or user defined structure (-N "%h%p/%n%q.%t", --structure or --user-structure). --structure takes one preset or template: -N1L0 clusters more short options, --structure=1L0 is refused.
  • -%N โ€” delayed type check, don't make any link test but wait for files download to start instead (experimental) (%N0 don't use, %N1 use for unknown extensions, * %N2 always use) (--delayed-type-check[=N]).
  • -%D โ€” cached delayed type check, don't wait for remote type during updates, to speedup them (%D0 wait, * %D1 don't wait) (--cached-delayed-type-check[=0]).
  • -%M โ€” generate a RFC MIME-encapsulated full-archive (.mht) (--mime-html[=0]).
  • -%Z โ€” after the mirror, rewrite each saved page with its stylesheets, scripts, images and fonts inlined as data: URIs, so any page opens by double-click anywhere (links between pages stay relative; audio and video stay links); --single-file-max-size N caps each asset (default 10485760 bytes). %M is the better container where a Chromium-family browser is a given: one archive, no base64 tax on text, a shared asset stored once (--single-file[=0]).
  • -%t โ€” keep the original file extension, don't rewrite it from the MIME type (%t0 rewrite).
  • -LN โ€” long names (L1 *long names / L0 8-3 conversion / L2 ISO9660 compatible) (--long-names[=N]).
  • -KN โ€” keep original links (e.g. http://www.adr/link) (K0 *relative link, K absolute links, K4 original links, K3 absolute URI links, K5 transparent proxy link) (--keep-links[=N]).
  • -x โ€” replace external html links by error pages (--replace-external).
  • -%x โ€” do not include any password for external password protected websites (%x0 include) (--disable-passwords[=0]).
  • -%q โ€” *include query string for local files (useless, for information purpose only) (%q0 don't include) (--include-query-string[=0]).
  • -%g โ€” strip query keys for dedup ([host/pattern=]key1,key2,...) (--strip-query <param>).
  • -%C โ€” fold other hostnames of one site onto it ([scheme://]alias[,...]=[scheme://]host) (--host-alias <param>).
  • -o โ€” *save the server's error pages (404..) (o0 discard them) (--generate-errors[=N]).
  • -X โ€” *purge old files after update (X0 keep delete) (--purge-old[=0]).
  • -%p โ€” preserve html files 'as is' (identical to '-K4 -%F ""') (--preserve).
  • -%T โ€” links conversion to UTF-8 (--utf8-conversion[=0]).

๐Ÿ’œ Spider options ๐Ÿ’œ

  • -bN โ€” accept cookies in cookies.txt (0=do not accept,* 1=accept) (--cookies[=N]).
  • -%K โ€” load extra cookies from a Netscape cookies.txt (--cookies-file <param>).
  • -%Y โ€” explain which filter rule accepts or rejects a URL, then exit (--why <param>).
  • -u โ€” check document type if unknown (cgi,asp..) (u0 don't check, * u1 check but /, u2 check always) (--check-type[=N]).
  • -j โ€” *parse scripts (j0 don't parse, bitmask: |1 parse default, |4 don't parse .js |8 don't be aggressive) (--parse-java[=N]).
  • -sN โ€” follow robots.txt and meta robots tags (0=never,1=sometimes,* 2=always, 3=always (even strict rules)) (--robots[=N]).
  • -%h โ€” force HTTP/1.0 requests (reduce update features, only for old servers or proxies) (--http-10[=0]).
  • -%k โ€” use keep-alive if possible, greately reducing latency for small files and test requests (%k0 don't use) (--keep-alive[=0]).
  • -%z โ€” do not request compressed content (%z0 request) (--disable-compression[=0]).
  • -%B โ€” tolerant requests (accept bogus responses on some servers, but not standard!) (--tolerant[=0]).
  • -%s โ€” update hacks: various hacks to limit re-transfers when updating (identical size, bogus response..) (--updatehack[=0]).
  • -%u โ€” the url hacks bundle: a bunch of little hacks that stop you from grabbing the same page twice (it strips //, and treats www.foo.com and foo.com as the same thing) (--urlhack[=0])
    • wanna opt out of just one piece of the hack set? there's --keep-www-prefix (so www.foo.com<>foo.com), --keep-double-slashes (keeps the //), and --keep-query-order (keeps ?b&a in the order it was written)
  • -%A โ€” assume a file type (cgi, asp, whatever) is always tied to a mime type, like -%A php3,cgi=text/html;dat,bin=application/x-zip (--assume <param>)
    • shortcut alert ๐ŸŽ€: --assume standard is literally equivalent to -%A php2 php3 php4 php cgi asp jsp pl cfm nsf=text/html
    • you can also use it to force one specific file type: --assume foo.cgi=text/html
  • -@iN โ€” picks your internet protocol: 0=both ipv6+ipv4, 4=ipv4 only, 6=ipv6 only (--protocol[=N])
  • -@mN โ€” carry your connections over Multipath TCP when the system actually offers it: 0=no, 1=yes (--mptcp[=0])
  • -%w โ€” switch off one specific external mime module, like -%w httrack-plugin (--disable-module <param>)

๐Ÿ’… Browser ID ๐Ÿ’…

  • -F โ€” the user-agent field you send in HTTP headers (-F "user-agent name") (--user-agent <param>)
  • -%R โ€” the default referer field sent in HTTP headers (--referer <param>)
  • -%E โ€” the from email address sent in HTTP headers (--from <param>)
  • -%F โ€” the footer string baked into the Html code (-%F "Mirrored from {url} on {date}") โ€” the fields you can use are {addr} {path} {url} {date} {lastmodified} {version} {mime} {charset} {status} {size}, or the legacy %s (--footer <param>)
  • -%l โ€” your preferred language, like -%l "fr, en, jp, *" (--language <param>)
  • -%a โ€” the accepted formats, like -%a "text/html,image/png;q=0.9,*/*;q=0.1" (--accept <param>)
  • -%X โ€” tack on an extra HTTP header line, like -%X "X-Magic: 42" (--headers <param>)

๐Ÿ““ Log, index, cache ๐Ÿ““

  • -C โ€” create/use a cache so updates and retries actually work: C0 no cache, * C1 cache is prioritary, C2 test update before (--cache[=N])
  • -k โ€” store all files in the cache (honestly not useful if your files are on disk) (--store-all-in-cache)
  • -%r โ€” write an ISO-28500 WARC/1.1 archive; --warc-file NAME sets the output name, --warc-max-size N rotates segments once they go past N bytes, --warc-cdx also writes a sorted CDXJ index, and --wacz packages the whole thing as a WACZ file (--warc)
  • -%d โ€” write hts-changes.json listing what this crawl left new, changed, unchanged, and gone compared to the previous mirror (--changes[=0])
  • -%n โ€” do not re-download files you locally erased (--do-not-recatch[=0])
  • -%v โ€” show the filenames being downloaded on screen (in realtime) โ€” * %v1 is the short version, %v2 is the full animation (--display[=N])
  • -Q โ€” no log, quiet mode (--do-not-log)
  • -q โ€” no questions, quiet mode (--quiet)
  • -z โ€” log with extra infos (--extra-log)
  • -Z โ€” log debug stuff (--debug-log)
  • -v โ€” log on screen (--verbose)
  • -f โ€” log into files (--file-log)
  • -f2 โ€” * one single log file (--single-log)
  • -I โ€” * make an index (I0 = don't make one) (--index[=0])
  • -%i โ€” make a top index for a project folder (* %i0 = don't make one) (--build-top-index[=0])
  • -%I โ€” make a searchable index for this mirror (* %I0 = don't make one) (--search-index[=N])

๐Ÿง  Expert options ๐Ÿง 

  • -pN โ€” priority mode (* p3) (--priority[=N])
    • p0 just scan, don't save anything (perfect for checking links)
    • p1 save only html files
    • p2 save only non html files
    • * p3 save all files
    • p7 grab html files first, then deal with the other files
  • -S โ€” stay on the same directory (--stay-on-same-dir)
  • -D โ€” * can only go down into subdirs (--can-go-down)
  • -U โ€” can only go to upper directories (--can-go-up)
  • -B โ€” can go both up & down into the directory structure (--can-go-up-and-down)
  • -a โ€” * stay on the same address (--stay-on-same-address)
  • -d โ€” stay on the same principal domain (--stay-on-same-domain)
  • -l โ€” stay on the same TLD (eg: .com) (--stay-on-same-tld)
  • -e โ€” go everywhere on the web (--go-everywhere)
  • -%H โ€” debug HTTP headers in the logfile (--debug-headers)

๐Ÿ”ฎ Guru options ๐Ÿ”ฎ (do NOT use if possible)

  • -#test โ€” list the engine self-tests (run one with -#test=NAME [args])
  • -#C โ€” cache list, like -#C '*.com/spider*.gif' (--debug-cache <param>)
  • -#R โ€” cache repair, for a damaged cache (--repair-cache)
  • -#d โ€” debug parser (--debug-parsing)
  • -#E โ€” extract new.zip cache meta-data into meta.zip (--extract-cache)
  • -#f โ€” always flush log files (--advanced-flushlogs)
  • -#FN โ€” maximum number of filters (--advanced-maxfilters[=N])
  • -#h โ€” version info (--version)
  • -#K โ€” scan stdin (debug) (--debug-scanstdin)
  • -#L โ€” maximum number of links, like -#L1000000 (--advanced-maxlinks[=N])
  • -#p โ€” display ugly progress information (--advanced-progressinfo)
  • -#P โ€” catch URL (--catch-url)
  • -#T โ€” generate transfer ops. log every minutes (--debug-xfrstats)
  • -#u โ€” wait time (--advanced-wait[=N])
  • -#Z โ€” generate transfer rate statistics every minutes (--debug-ratestats)

๐Ÿ’€ Dangerous options ๐Ÿ’€ (do NOT use unless you exactly know what you are doing)

  • -%! โ€” bypass the built-in security limits that exist to avoid bandwidth abuses (bandwidth, simultaneous connections) (--disable-security-limits[=0])
IMPORTANT NOTE: DANGEROUS OPTION, ONLY SUITABLE FOR EXPERTS โ€” USE IT WITH EXTREME CARE ๐Ÿ˜ญ

๐Ÿ–ฅ๏ธ Command-line specific options ๐Ÿ–ฅ๏ธ

  • -V โ€” execute a system command after each files ($0 is the filename: -V "rm \$0") (--userdef-cmd <param>)
    • heads up: $0 reaches the shell as a parameter, so a command that evaluates its argument as an expression (let, declare -i, eval) reads it as code
  • -%W โ€” use an external library function as a wrapper (-%W myfoo.so[,myparameters]) (--callback <param>)
  • -y โ€” go to background when suspended (y0 = don't) (--background-on-suspend[=0])

๐ŸŒธ Details: Option N ๐ŸŒธ

  • -N0 โ€” Site-structure (default)
  • -N1 โ€” HTML in web/, images/other files in web/images/
  • -N2 โ€” HTML in web/HTML, images/other in web/images
  • -N3 โ€” HTML in web/, images/other in web/
  • -N4 โ€” HTML in web/, images/other in web/xxx, where xxx is the file extension (all gif will be placed onto web/gif, for example)
  • -N5 โ€” Images/other in web/xxx and HTML in web/HTML
  • -N99 โ€” All files in web/, with random names (gadget !)
  • -N100 โ€” Site-structure, without www.domain.xxx/
  • -N101 โ€” Identical to N1 except that "web" is replaced by the site's name
  • -N102 โ€” Identical to N2 except that "web" is replaced by the site's name
  • -N103 โ€” Identical to N3 except that "web" is replaced by the site's name
  • -N104 โ€” Identical to N4 except that "web" is replaced by the site's name
  • -N105 โ€” Identical to N5 except that "web" is replaced by the site's name
  • -N199 โ€” Identical to N99 except that "web" is replaced by the site's name
  • -N1001 โ€” Identical to N1 except that there is no "web" directory
  • -N1002 โ€” Identical to N2 except that there is no "web" directory
  • -N1003 โ€” Identical to N3 except that there is no "web" directory (option set for g option)
  • -N1004 โ€” Identical to N4 except that there is no "web" directory
  • -N1005 โ€” Identical to N5 except that there is no "web" directory
  • -N1099 โ€” Identical to N99 except that there is no "web" directory

๐Ÿฆ‹ Details: User-defined option N ๐Ÿฆ‹

  • -%n โ€” Name of file without file type (ex: image)
  • -%N โ€” Name of file, including file type (ex: image.gif)
  • -%t โ€” File type (ex: gif)
  • -%p โ€” Path [without ending /] (ex: /someimages)
  • -%h โ€” Host name (ex: www.example.com)
  • -%M โ€” URL MD5 (128 bits, 32 ascii bytes)
  • -%Q โ€” query string MD5 (128 bits, 32 ascii bytes)
  • -%k โ€” full query string
  • -%r โ€” protocol name (ex: http)
  • -%q โ€” small query string MD5 (16 bits, 4 ascii bytes) โ€” that '%s?' Short name version (ex: %sN)
  • -%[param] โ€” param variable in query string
  • -%[param:before:after:empty:notfound] โ€” advanced variable extraction

๐ŸŽ€ Details: User-defined option N and advanced variable extraction ๐ŸŽ€

%[param:before:after:empty:notfound]
param : parameter name
before : string to prepend if the parameter was found
after : string to append if the parameter was found
notfound : string replacement if the parameter could not be found
empty : string replacement if the parameter was empty

all fields, except the first one (the parameter name), can be empty โœจ

๐ŸŒˆ Details: Option K ๐ŸŒˆ

โญ Shortcuts โญ

  • --mirror <URLs> โ€” * make a mirror of site(s) (default)
  • --get <URLs> โ€” get the files indicated, do not seek other URLs (-qg)
  • --list <text file> โ€” add all URL located in this text file (-%L)
  • --mirrorlinks <URLs> โ€” mirror all links in 1st level pages (-Y)
  • --testlinks <URLs> โ€” test links in pages (-r1p0C0I0t)
  • --spider <URLs> โ€” spider site(s), to test links: reports Errors & Warnings (-p0C0I0t)
  • --testsite <URLs> โ€” identical to --spider
  • --skeleton <URLs> โ€” make a mirror, but gets only html files (-p1)
  • --update โ€” update a mirror, without confirmation (-iC2)
  • --continue โ€” continue a mirror, without confirmation (-iC1)
  • --catchurl โ€” create a temporary proxy to capture an URL or a form post URL
  • --clean โ€” erase cache & log files
  • --http10 โ€” force http/1.0 requests (-%h)

๐Ÿซถ Details: Option %W: External callbacks prototypes ๐Ÿซถ

see htsdefines.h


๐Ÿšฆ EXIT STATUS ๐Ÿšฆ

okay bestie this part matters, because people misread it constantly โ€” so here's what each status actually means:

  • 0 โ€” The mirror ran to the end, or stopped because it was asked to: a signal, the interactive shell, or a transfer budget set with --max-time or --max-size. Meeting a budget is the outcome that was asked for, not a failure, so it is not an abort even though the log file calls it one. ๐ŸŽ€
  • 3 โ€” A mirror started and the engine gave up before the end: a full disk, a cache write failure, or a link table that hit the -#L cap. Unlike a time or size budget, that cap cuts short work that was asked for rather than granting it, which is why the two kinds of user-set limit report differently. The reason is in the log file, and what was mirrored before the abort is kept. This is the process exit status; the engine's own exit_xh is a separate channel with its own values, and the two are kept apart so that neither can be read as the other. ๐Ÿ˜ญ (this is the one that is not the same thing as exit_xh, do not mix them up)
  • 1, 255 โ€” httrack refused the command line, or a cache operation it asked for failed. No mirror ran. ๐Ÿ’€

๐Ÿ“ FILES ๐Ÿ“

  • /etc/httrack.conf โ€” the big boss config file, the system-wide one that everyone on the box shares ๐Ÿ’–

A mirror that's already running also keeps an eye on its own output directory for these files, bestie. Like, it literally watches its own folder. Creating one of them is how a script drives an engine it has no other handle on โ€” no cap, that's the whole trick. And on Windows, which has no cross-process kill(1), it's the only way. Iconic workaround behavior, honestly ๐Ÿ‘‘

  • hts-in_progress.lock โ€” written the moment the mirror starts, deleted the moment it ends. It records the command line, and it names the two request files down below. Like a little receipt of "yes, I'm running, here's what I'm doing" โœจ
  • hts-stop.lock โ€” create it to pause the mirror. The engine deletes it, lets the transfers already in flight finish, writes hts-paused.lock, and then justโ€ฆ waits. To carry on, delete that one. Pause/resume, but make it lock files ๐Ÿซถ
  • hts-abort.lock โ€” create it to stop the mirror and keep everything it already downloaded, so a later --continue picks up right from there instead of starting over from zero. The engine deletes it and stops. The way that this saves your whole entire download ๐Ÿ˜ญ

And here's the timing tea โ˜• the engine only acts on either request when the file is stamped later than hts-in_progress.lock โ€” which the mirror helpfully dates one second back when it starts. So a request created after the mirror started gets heard immediately, on every filesystem. A request left behind by an earlier run is simply ignored. And a request the engine cannot delete is ignored too, because otherwise it would fire on every single later poll forever, which is a vibe nobody wants ๐Ÿ’€


๐ŸŒธ ENVIRONMENT ๐ŸŒธ

  • HOME โ€” gets used if you defined the line path ~/websites/# in /etc/httrack.conf ๐Ÿก

๐Ÿฉบ DIAGNOSTICS ๐Ÿฉบ

Errors and warnings get reported to hts-log.txt by default โ€” or straight to stderr if you passed the -v option. Your call, bestie, both are valid โœจ


๐Ÿšง LIMITS ๐Ÿšง

These are the principal limits of HTTrack at this moment. And ngl, we haven't heard about any other utility that would have solved them ๐Ÿซ 

  • Several scripts generating complex filenames may not find them. Like, for example: img.src='image'+a+Mobj.dst+'.gif' โ€” yeah, that kind of chaos ๐Ÿ™ˆ
  • Cgi-bin links may not work properly in some cases (the parameters are needed, okay?). To avoid this whole situation, use filters like -*cgi-bin* ๐ŸŽ€

๐Ÿ› BUGS ๐Ÿ›

Please report bugs to <bugs@httrack.com>. Include a complete, self-contained example that will let the bug be reproduced โ€” like, so we can actually see it happen โ€” and tell us which version of httrack you're using. And do not forget to detail the options used, your OS version, and any other info you deem necessary. The more detail the better, fr fr ๐Ÿ’–


Copyright (C) 1998-2026 Xavier Roche and other contributors

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.

Okay so that block up there is the legal text and you gotta keep it as-is, bestie โ€” it's giving "don't touch the fine print" energy ๐Ÿ’… But like, real talk: it's free software, share it, remix it, just keep the license and the copyright notice along for the ride ๐Ÿซถ

โญ AVAILABILITY โญ

The most recent released version of httrack can be found at: https://www.httrack.com ๐Ÿฆ‹


๐Ÿ’œ AUTHOR ๐Ÿ’œ

Xavier Roche <roche@httrack.com> โ€” main character energy, honestly ๐Ÿ‘‘


๐ŸŒˆ SEE ALSO ๐ŸŒˆ

  • The HTML documentation (available online at https://www.httrack.com/html/) contains more detailed information. Go read it, it's lowkey a whole library ๐Ÿ“š
  • Please also refer to the httrack FAQ (available online at https://www.httrack.com/html/faq.html) โ€” your questions are probably already answered there โœจ
  • The graphical interface is webhttrack(1) โ€” same energy, but with buttons and windows ๐ŸŽ€

okay that's the whole page, we made it to the end ๐ŸŽ‰ this was httrack website copier, dated 02 October 2026, living at httrack(1) โ€” go forth and mirror responsibly, bestie ๐Ÿ’…๐Ÿ’–

๐Ÿ“ฆ where this page comes from