DJVU(1)

💖 DJVU 💖 (that's DJVU(1), bestie)

DjVuLibre-3.5 — ok so this is the big-picture page, the whole lore drop about DjVu and DjVuLibre. No options, no flags, just ✨lore✨. Sit down, get comfy, we're going on a journey.

---

🌸 NAME 🌸

DjVu</puruple> — DjVu and DjVuLibre. That's it, that's the name, we're already iconic.

ok quick note before we start: this page is LONG and it comes in 2 parts. This is part 1. More is coming, so don't go anywhere 🫶

---

✨ INTRODUCTION ✨

So like… the Internet gave us a whole worldwide infrastructure to build the universal library on, right? Slay. But plot twist: a huge amount of the world's knowledge, history, and literature is still trapped on paper in the basements of traditional libraries everywhere. Basements. 😭 The disrespect.

Lots of libraries and content owners are out here digitizing their collections. Some of those efforts are the painstaking, hand-cramping process of converting paper docs into computer-friendly form (SGML-based formats, etc.), and honestly the high cost of those conversions limits how far they can go. Scanning documents and distributing the resulting images electronically? Way cheaper, and lowkey more faithful to the original, because it preserves the visual aspect of the document. The vibes stay intact. 💅

But here's the thing — even with network connections and computers getting faster by the second, the number of scanned document images accessible on the Web today is still pretty small. Why? Several reasons, bestie:

Reason 1: scanning anything other than unbound sheets in black and white is expensive. This problem is slowly going away though, with fast and low-cost color scanners with sheet feeders showing up. We love progress. 🌈

Reason 2: long-established image compression standards and file formats have proved inadequate for distributing scanned documents at high resolution — especially color documents. It's not just that file sizes and download times are impractical; the decoding and rendering times are also prohibitive. Like, imagine a typical magazine page scanned in color at 100 dpi in JPEG — that's typically 100 KB to 200 KB, but the text would be hardly readable: not enough for screen viewing and totally unacceptable for printing. Same page at 300 dpi? Quality's fine for viewing and printing, but the file size would be 300 KB to 1000 KB at best, which is impractical for remote access. And there's another major problem: a fully decoded 300 dpi color image of a letter-size page occupies 24 MB of memory and easily causes disk swapping. 24 MB!! For one page!! 💀

Reason 3: digital documents are more than just a pile of individual page images. Pages in a scanned document have a natural serial order. You gotta make special provision so that flipping pages is instantaneous and effortless, otherwise the user experience is trash. And even more important: most existing document formats force users to download the entire document first before showing you a chosen page. But users often want to jump to individual pages without waiting for the whole thing to download. 🙈

Efficient browsing requires:

• efficient random page access
• fast sequential page flipping
• quick rendering

You get all three with a combination of advanced compression, pre-fetching, pre-decoding, caching, and progressive rendering. DjVu decomposes each page into multiple components (text, backgrounds, images, libraries of common shapes…) that may be shared by several pages and downloaded on demand. All of that calls for a very sophisticated but parsimonious control mechanism to handle on-demand downloading, pre-fetching, decoding, caching, and progressive rendering of the page images.

So what we're looking at here isn't just a document image compression technique. It's a whole platform for document delivery. Periodt.

And to wrap the intro: DjVu is an image compression technique, a document format, and a software platform for delivering documents images over the Internet that fulfills all the above requirements. Main character energy, honestly. 👑

---

🦋 DJVU IMAGE COMPRESSION 🦋

The DjVu image compression is based on three technologies. Three! A trio. The holy trinity. Let's meet them:

🎀 DjVuPhoto

DjVuPhoto — also known as IW44 — is a wavelet-based continuous-tone image compression technique with progressive decoding/rendering. Best used for encoding photographic images in colors or in shades of gray. Images are typically half the size as JPEG for the same distortion. Half! For the same distortion! Ate. ✨

🎀 DjVuBitonal

DjVuBitonal — also known as JB2 — is a bitonal image compression that takes advantage of repetitions of nearly identical shapes on the page (such as characters, bestie, characters) to efficiently compress text images. Best used to compress black and white images representing text and simple drawings. A typical 300 dpi page in DjVuBitonal occupies 5 to 25 KB — that's 3 to 8 times better than TIFF-G4 or PDF. Not me being obsessed. 🍓

🎀 DjVuDocument

DjVuDocument is a compression technique specifically designed for color digital document images containing both pictures and text, like a page of a magazine. DjVuDocument represents images into separately compressed layers:

• The foreground layer is usually compressed with DjVu Bitonal and contains the text and drawings.
• The background layer is usually compressed with DjVuPhoto and contains the background texture and the pictures at lower resolution.

Two layers, one flawless page. 💅

---

💜 DJVU DOCUMENT DELIVERY PLATFORM 💜

The DjVu technology is designed from the ground up to support efficient delivery of digital documents over the Internet. It gives you various ways to deal with multi-page documents, and various ways to enrich the content with hyper-links, meta-data, searchable text, etc. Let's go through them:

🌈 MIME types

The DjVu format has an official MIME type of image/vnd.djvu, and that's the preferred content-type for http servers to give for DjVu files. The unofficial mime types used historically are image/x.djvu and image/x-djvu, which you may still run into out in the wild. Ideally, clients should be configured to handle all three. All three, bestie. Don't play favorites.

🌈 Bundled multi-page documents

A bundled multi-page DjVu document uses a single file to represent the entire document. That one file contains all the pages plus ancillary information (e.g. the page directory, data shared by several pages, thumbnails, etc.). Using a single file format is very convenient for storing documents or for sending email attachments. So tidy. So demure. 🎀

Now here's the drama. When you type the URL of a multi-page document, the DjVu browser plugin starts downloading the whole file, but displays the first page as soon as it's available. You can immediately navigate to other pages using the DjVu toolbar. Cute, right?

But suppose the document is stored on a remote web server. You can easily access the first page and see that this is not the document you wanted. And even though you'll never display the other pages, the browser is transferring data for those pages anyway and is wasting the bandwidth of your server (and the bandwidth of the Internet too, smh). 💀

Or maybe you see the summary of the document on the first page and jump to page 100. But page 100 cannot be displayed until data for pages 1 to 99 has been received. You may have to wait for the transmission of unnecessary page data. 😭

That second problem (the unnecessary wait) can be solved using the byte serving options of the HTTP/1.1 protocol. But — and this is a big but — this option has to be supported by the web server, the proxies, the caches, and the browser. And byte serving does not solve the first problem (the waste of bandwidth). Two problems, one solution, that solution is partial. The math isn't mathing.

🌈 Indirect multi-page documents

Indirect multi-page DjVu documents solve both problems. Periodt. 💅

An indirect multi-page DjVu document is composed of several files. The main file is named the index file. You can browse a document using the URL of the index file, just like you do with a bundled multi-page document. The index file however is very small. It simply contains the document directory and the URLs of secondary files containing the page data.

When you browse an indirect multi-page document, the browser only accesses data for the pages you're viewing. This can be done at a reasonable speed because the browser maintains a cache of pages and sometimes pre-fetches a few pages ahead of the current page. This model uses the web serving bandwidth much more effectively. It also eliminates unnecessary delays when jumping ahead to pages located anywhere in a long document. No waiting, no wasted bytes. We love an efficient queen. 👑

🌈 Annotations

Every DjVu image optionally includes so-called annotation chunks. The annotation chunk is often used to define hyper-links to other document pages or to arbitrary web pages. Annotation chunks can also be used for other purposes, such as:

• setting the initial viewing mode of a page
• defining highlighted zones
• storing arbitrary meta-data about the page or the document

Multi-purpose icon. ✨

🌈 Hidden text

Every DjVu image optionally includes a hidden text layer that associated graphical features with the corresponding text. (Yes, "associated" — that's the word, I'm not fixing it, it's iconic.) The hidden text layer is usually generated by running an Optical Character Recognition software. This textual information provides for:

• indexing DjVu documents
• copying/pasting text from DjVu page images

Searchable and copy-pasteable. The way that it just works. 🫶

🌈 Thumbnails

DjVu documents sometimes contain pre-computed page thumbnails. Little previews, little baby pages, adorable. 🎀

🌈 Outline

DjVu documents sometimes contain a navigation chunk containing an outline — that is, a hierarchical table of contents with pointers to the corresponding document pages. Structure! Organization! We stan. 💜

---

ok that's the end of part 1 bestie, part 2 is right behind this one, don't close the tab 🦋

✨ DJVUZONE AND DJVULIBRE ✨

okay bestie, storytime, because the lore here is actually iconic. 💅

The DjVu technology was initially created by a few researchers in AT&T Labs between 1995 and 1999. Then Lizardtech, Inc. obtained a commercial license from AT&T and continued the development. The current owner of the DjVu commercial rights is Cuminas ( https://www.cuminas.jp/en/about_djvu ), who offers solutions for producing and distributing documents using the DjVu technology, as well as a DjVu viewer packaged as a Chrome extension. ✨

The DjVu.org web site ( http://www.djvu.org ) is managed by the few AT&T Labs researchers who created the DjVu technology in the first place. They promote the DjVu technology by providing an independent source of information about DjVu. 🫶

Understanding how little room there is for a proprietary document format, Lizardtech released the DjVu Reference Library under the GNU Public License in December 2000. This library entirely defines the compression format and the elementary codecs. Six month later, Lizardtech released an updated DjVu Reference Library as well as the source code of the Unix viewer. 🌈

These two releases form the basis of the initial DjVuLibre software. They modified the build system to comply with the expectations of the open source community. Various bugs and portability issues have been fixed. They also tried to make it simpler to use and install, while preserving the essential structure of the Lizardtech releases. no cap, that's the whole origin story. 😌

The DjVuLibre software contains the following components:

• bzz(1) — a general purpose compression command line program. Many internal DjVu data structures are compressed using this technique. 🗜️
• c44(1) — a DjVuPhoto command line encoder. This state-of-the-art wavelet compressor produces DjVuPhoto images from PPM or JPEG images. 🔥
• cjb2(1) — a DjVuBitonal command line encoder. This soft-pattern-matching compressor produces DjVuBitonal images from PBM images. It can encode images without loss, or introduce small changes in order to improve the compression ratio. The lossless encoding mode is competitive with that of the Lizardtech commercial encoders. 👑
• cpaldjvu(1) — a DjVuDocument command line encoder for images with few colors. This encoder is well suited to compressing images with a small number of distinct colors (e.g. screen-shots). The dominant color is encoded by the background layer. The other colors are encoded by the foreground layer. 🎨
• csepdjvu(1) — a DjVuDocument command line encoder for separated images. This encoder takes a file containing pre-segmented foreground and background images and produces a DjVuDocument image. ✂️
• ddjvu(1) — a command line decoder for DjVu images. This program produces a PNM image representing any segment of any page of a DjVu document at any resolution. 🔍
• djview(1) — a stand-alone viewer for DjVu images. This sophisticated viewer displays DjVu documents. It implements document navigation as well as fast zooming and panning. 👀
• nsdejavu(1) — a web browser plugin for viewing DjVu images. This small plugin allows for viewing DjVu documents from web browsers. It internally uses djview to perform the actual work. 🌐
• djvups(1) — a command line tool for converting DjVu documents into PostScript. 🖨️
• djvm(1) — a command line tool for manipulating bundled multi-page DjVu documents. This program is often used to collect individual pages and produce a bundled document. 📚
• djvmcvt(1) — a command line tool for converting bundled documents to indirect documents and conversely. 🔄
• djvused(1) — a powerful command line tool for manipulating multi-page documents, creating or editing annotation chunks, creating or editing hidden text layers, pre-computing thumbnail images, and more... the way this one does literally everything?? 💀
• djvutxt(1) — a command line tool to extract the hidden text from DjVu documents. 📝
• djvudump(1) — a command line tool for inspecting DjVu files and displaying their internal structure. 🧐
• djvuextract(1) — a command line tool for dis-assembling DjVu image files. 🧩
• djvumake(1) — a command line tool for assembling DjVu image files. 🔧
• djvuserve(1) — a CGI program for generating indirect multi-page DjVu documents on the fly. ⚡
• djvutoxml(1), djvuxmlparser(1) — command line tools to edit DjVu metadata as XML files. 🏷️

---

🌸 DJVU ENCODERS AND ANY2DJVU 🌸

DjVuLibre comes with a variety of specialized encoders: c44(1) for photographic images, cjb2(1) for bitonal images, and cpaldjvu(1) for images with few distinct colors. Although these encoders perform well in their specialized domain, they cannot handle complex tasks involving segmentation and multipage encoding. 😔

The Lizardtech commercial products (see http://www.lizardtech.com/solutions/document ) can perform these complex encoding tasks. 💼

Another solution is provided by the compression server at ( http://any2djvu.djvu.org ). This machine uses pre-lizardtech prototype encoders from AT&T Labs and performs almost as well as the commercial Lizardtech encoders. 🚀

bestie pls read this part!! Please note that the Any2DjVu compression server comes with no guarantee, that nothing is done to ensure that your documents will remain confidential, and that there is only one computer working for the whole planet. 🙈 that last bit is not a typo. one (1) computer. for everyone. 🫠

---

💖 CREDITS 💖

Numerous people have contributed to the DjVu source code during the last five years. Please submit a sourceforge bug report to update the following list. 👑

• Yoshua Bengio
• Léon Bottou
• Chakradhar Chandaluri
• Regis M. Chaplin
• Ming Chen
• Parag Deshmukh
• Royce Edwards
• Andrew Erofeev
• Praveen Guduru
• Patrick Haffner
• Paul G. Howard
• Orlando Keise
• Yann Le Cun
• Artem Mikheev
• Florin Nicsa
• Joseph M. Orost
• Steven Pigeon
• Bill Riemers
• Patrice Simard
• Jeffery Triggs
• Luc Vincent
• Pascal Vincent

---

and that's the whole page, bestie!! 🦋 this one's from DjVuLibre-3.5, dated 10/11/2001, and it lives at DJVU(1) in the manual. go forth and compress something pretty. 💜✨