RFC 6386
VP8 Data Format and Decoding Guide
VP8 Data Format and Decoding Guide 🎀
Part 1 of 52 🧚 more parts are coming, so stick around bestie!
okay so this is RFC 6386 🦋 the spec for VP8, a compressed video data format, plus a walkthrough of how to decode it ✨
Here's the title block tea 🫖 It's published in November 2011 as an Independent Submission in the Informational category (ISSN 2070-1721). The authors are J. Bankoski, J. Koleszar, L. Quillio, J. Salonen, P. Wilkins, and Y. Xu, all from Google Inc. 💜 The header doesn't list anything it obsoletes or updates, so there's no drama to untangle there 🙈
Abstract ✨
okay the abstract is short and sweet 💅 It says this document describes the VP8 compressed video data format, and it also talks through the decoding procedure for that format. that's the whole vibe 🌸
Status of This Memo 📜
this one's a little formal but lowkey important 🙈 The document is not on the Internet Standards Track, so it's just published for information purposes. Not a standard, just vibes and facts 🧚
It's a contribution to the RFC Series, made independently of any other RFC stream. The RFC Editor picked to publish it at their discretion, and they make zero claims about how useful it is for implementing or deploying anything 💀 Documents the RFC Editor approves for publication are also not a candidate for any level of Internet Standard, see Section 2 of RFC 5741 for the details 📖
If you want the current status, any errata, or a place to send feedback, check http://www.rfc-editor.org/info/rfc6386 🦋
Copyright Notice ©️
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved 💖
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which you can find at http://trustee.ietf.org/license-info and which are in effect on the date this document was published. Please read those carefully, because they spell out your rights and restrictions for this document 🫶
Table of Contents 🗺️
Here's the full map of the doc, copied exactly so the section numbers still line up for you 🍓
1. Introduction ....................................................4
2. Format Overview .................................................5
3. Compressed Frame Types ..........................................7
4. Overview of Compressed Data Format ..............................8
5. Overview of the Decoding Process ................................9
6. Description of Algorithms ......................................14
7. Boolean Entropy Decoder ........................................16
7.1. Underlying Theory of Coding ...............................17
7.2. Practical Algorithm Description ...........................18
7.3. Actual Implementation .....................................20
8. Compressed Data Components .....................................25
8.1. Tree Coding Implementation ................................27
8.2. Tree Coding Example .......................................28
9. Frame Header ...................................................30
9.1. Uncompressed Data Chunk ...................................30
9.2. Color Space and Pixel Type (Key Frames Only) ..............33
9.3. Segment-Based Adjustments .................................34
9.4. Loop Filter Type and Levels ...............................35
9.5. Token Partition and Partition Data Offsets ................36
9.6. Dequantization Indices ....................................37
9.7. Refresh Golden Frame and Altref Frame .....................38
9.8. Refresh Last Frame Buffer .................................39
9.9. DCT Coefficient Probability Update ........................39
9.10. Remaining Frame Header Data (Non-Key Frame) ..............40
9.11. Remaining Frame Header Data (Key Frame) ..................41
10. Segment-Based Feature Adjustments .............................41
11. Key Frame Macroblock Prediction Records .......................42
11.1. mb_skip_coeff ............................................42
11.2. Luma Modes ...............................................42
11.3. Subblock Mode Contexts ...................................45
11.4. Chroma Modes .............................................46
11.5. Subblock Mode Probability Table ..........................47
12. Intraframe Prediction .........................................50
12.1. mb_skip_coeff ............................................51
12.2. Chroma Prediction ........................................51
12.3. Luma Prediction ..........................................54
13. DCT Coefficient Decoding ......................................60
13.1. Macroblock without Non-Zero Coefficient Values ...........61
13.2. Coding of Individual Coefficient Values ..................61
13.3. Token Probabilities ......................................63
13.4. Token Probability Updates ................................68
13.5. Default Token Probability Table ..........................73
14. DCT and WHT Inversion and Macroblock Reconstruction ...........76
14.1. Dequantization ...........................................76
14.2. Inverse Transforms .......................................78
14.3. Implementation of the WHT Inversion ......................78
14.4. Implementation of the DCT Inversion ......................81
14.5. Summation of Predictor and Residue .......................83
15. Loop Filter ...................................................84
15.1. Filter Geometry and Overall Procedure ....................85
15.2. Simple Filter ............................................87
15.3. Normal Filter ............................................91
15.4. Calculation of Control Parameters ........................95
16. Interframe Macroblock Prediction Records ......................97
16.1. Intra-Predicted Macroblocks ..............................97
16.2. Inter-Predicted Macroblocks ..............................98
16.3. Mode and Motion Vector Contexts ..........................99
16.4. Split Prediction ........................................105
17. Motion Vector Decoding .......................................108
17.1. Coding of Each Component ................................108
17.2. Probability Updates .....................................110
18. Interframe Prediction ........................................113
18.1. Bounds on, and Adjustment of, Motion Vectors ............113
18.2. Prediction Subblocks ....................................115
18.3. Sub-Pixel Interpolation .................................115
18.4. Filter Properties .......................................118
19. Annex A: Bitstream Syntax ....................................120
19.1. Uncompressed Data Chunk .................................121
19.2. Frame Header ............................................122
19.3. Macroblock Data .........................................130
20. Attachment One: Reference Decoder Source Code ................133
20.1. bit_ops.h ...............................................133
20.2. bool_decoder.h ..........................................133
20.3. dequant_data.h ..........................................137
20.4. dixie.c .................................................138
20.5. dixie.h .................................................151
20.6. dixie_loopfilter.c ......................................158
20.7. dixie_loopfilter.h ......................................170
20.8. idct_add.c ..............................................171
20.9. idct_add.h ..............................................174
20.10. mem.h ..................................................175
20.11. modemv.c ...............................................176
20.12. modemv.h ...............................................192
20.13. modemv_data.h ..........................................193
20.14. predict.c ..............................................198
20.15. predict.h ..............................................231
20.16. tokens.c ...............................................232
20.17. tokens.h ...............................................242
20.18. vp8_prob_data.h ........................................243
20.19. vpx_codec_internal.h ...................................252
20.20. vpx_decoder.h ..........................................263
20.21. vpx_decoder_compat.h ...................................271
20.22. vpx_image.c ............................................285
20.23. vpx_image.h ............................................291
20.24. vpx_integer.h ..........................................298
20.25. AUTHORS File ...........................................299
20.26. LICENSE ................................................301
20.27. PATENTS ................................................302
21. Security Considerations ......................................302
22. References ...................................................303
22.1. Normative Reference .....................................303
22.2. Informative References ..................................303Next up is Section 1, Introduction 🌈
1. Introduction ✨
okay so bestie, this doc describes the VP8 compressed video data format 🎬 and also walks through how decoding works. it's meant to be read alongside the reference decoder source code in Attachment One (Section 20) 💻. if the narrative and the reference source code ever disagree, the reference source code wins, no debate 💅. the bitstream is defined by that source code, NOT by this writeup 🙅♀️
ok so like most modern video compression, VP8 chops frames into square subblocks of pixels 🧩. it predicts each subblock from blocks it already built, then fixes up that prediction (and makes up the blocks that couldn't be predicted) using a discrete cosine transform (DCT for short) 🌸. there's one special case where VP8 uses a Walsh-Hadamard transform (WHT for short) instead of a DCT 🦋
the big idea is that video is super repetitive, both over time and across space 🫶. it's way cheaper to say "copy that visually similar chunk from a previous frame" than to spell out every single pixel value 🙈. the DCT and WHT split the signal into frequency bands, which helps exploit spatial coherence and also takes advantage of the fact that human eyes are pretty chill about small losses of fidelity 👀✨
VP8 levels this up with, among other things, really sophisticated use of contextual probabilities 🧠👑. the result? a seriously smaller datarate for the same quality. iconic 🔥
now here's the lowkey flex: unlike older schemes like the MPEG formats, VP8 pins down EXACT values for reconstructed pixels 🎯. the DCT and WHT parts don't allow any "drift" from truncated fractions 🙅♀️. instead everything is done with fixed-precision integer math, which makes it WAY easier to verify that a decoder is correct and avoids weird, hard-to-predict visual differences between implementations 💖
also, in a full playback system, what you see on screen might not be exactly the reconstructed frames 👀. lots of systems run a final filtering step (usually called postprocessing) before viewing 🌈. that postprocessing doesn't affect decoding or reconstruction of later frames, since those are predicted from the fully specified reconstructed frames, so it's out of scope here 🙌. in practice, how much of it happens depends on the user's taste and the playback environment's computing power 🎀
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 📝
2. Format Overview ✨
okay so VP8 only works with an 8-bit YUV 4:2:0 image format 🎨. each 8-bit pixel in the two chroma planes (U and V) lines up with a 2x2 block of 8-bit luma pixels in the Y plane 🧁. the upper left corner of the Y block sits at exactly twice the coordinates of the matching chroma pixel 📐. when we talk about pixels or pixel distances without naming a plane, we mean the Y plane or the whole image, and those two both have the same full resolution 💅
pixels are basically one big array of bytes 🧺, stored row by row from top to bottom, with each row going left to right 🌸. that "left to right, then top to bottom" raster-scan order also shows up in how the compressed data is laid out 🫶
the VP8 bitstream header has a reserved bit set aside for supporting a secondary YUV color format 🌈. so it's future-proofed, bestie 🔮
sometimes, at really low datarates, a compression system decides to shrink the input resolution so it can compress more efficiently 🐣. VP8 supports this with optional upscaling of its internal reconstruction buffer before output 📈. this is totally separate from the optional postprocessing we talked about earlier, which has nothing to do with decoding itself 🙅♀️. the upscaling restores the frames to their original resolution ✨. so the whole compression/decompression system can be thought of as a "black box," where input and output are always at one given resolution 🖤. the compressor might "cheat" and process the signal at a lower resolution, and in that case the decompressor needs to be able to bring it back up to the original resolution 🪄
internally, VP8 breaks each output frame into an array of macroblocks 🧱. a macroblock is a square array of pixels, with Y dimensions of 16x16 and U and V dimensions of 8x8 🎀. macroblock-level data in a compressed frame shows up (and has to be processed) in raster order, kind of like the pixels in the frame 🌷
macroblocks get split even further into 4x4 subblocks 🧩. every macroblock has 16 Y subblocks, 4 U subblocks, and 4 V subblocks 💜. subblock-level data (and processing of it) also goes in raster order, but this time it's raster order inside the containing macroblock 🌀
as we'll see in more detail below, data can be specified at both the macroblock level and the subblock level 💫
pixels are always handled at the subblock level at minimum 🍓. subblocks are kind of the "atoms" of the VP8 algorithm ⚛️. the 2x2 chroma blocks that line up with 4x4 Y subblocks never get treated explicitly, neither in the data format nor in the algorithm spec 🙈
the DCT and WHT always operate at 4x4 resolution 🦋. the DCT is used for the 16Y, 4U, and 4V subblocks 🌈. the WHT is used (with some prediction modes but not all of them) to encode a 4x4 array made up of the average intensities of the 16 Y subblocks in a macroblock 🧁. those average intensities are, up to a constant normalization factor, basically just the 0th DCT coefficients of the Y subblocks 🔢. so this "higher-level" WHT is a stand-in for explicitly specifying those coefficients, the same way a subblock's DCT stands in for specifying the pixel values inside that subblock 🫶. we treat this 4x4 array as a second-order subblock called Y2, and a macroblock ends up with 24 "real" subblocks and sometimes a 25th "virtual" subblock 👑. we dig into this more in Section 13 📖
the frame layout the reference decoder uses can be found in the file vpx_image.h (Section 20.23) 📂
3. Compressed Frame Types ✨
okay so there are only two types of frames in VP8 🎞️
first up are intraframes, also called key frames (and I-frames in MPEG terms) 🔑. these get decoded without looking at any other frame in the sequence. the decompressor rebuilds them starting from its "default" state 🧼. key frames are your random access (or seeking) points in a video stream, so they're super important 🎯
next are interframes, also called prediction frames (P-frames in MPEG terms) 🔮. these are encoded with reference to prior frames, specifically all the prior frames up to and including the most recent key frame 🕰️. basically, decoding an interframe correctly depends on correctly decoding the most recent key frame and every frame after it 🔗. so the decoding algorithm is NOT tolerant of dropped frames 😭. if frames can be dropped or corrupted, you can't decode correctly until a key frame gets received correctly 🚫
unlike MPEG, VP8 has no bidirectional prediction 🙅♀️. no frame is predicted from frames that come after it in time, so there's no equivalent to an MPEG B-frame 💅
VP8 also adds alternate prediction frames on top of that, called golden frames and altref frames (short for alternative reference frames) 🏆. blocks in an interframe can be predicted from the block in the immediately previous frame, but also from the most recent golden frame or altref frame 🌟. every key frame is automatically both golden and altref 👑, and any interframe may optionally replace the most recent golden or altref frame 🔄
golden frames and altref frames can also help with that dropped-frame problem we just talked about 🩹. if a compressor only codes golden frames with reference to the prior golden frame (and key frame), then the "substream" of key and golden frames can be decoded even when other interframes get lost 🧚. basically, on the compressor side, golden frames have to subsume and recode any context updates made by the interframes in between 🧩. a typical use for this is video conferencing, where resending a prior golden frame and/or waiting to play until the next golden frame arrives is better than a bigger retransmit or a longer wait for the next key frame 💖
4. Overview of Compressed Data Format ✨
okay so bestie, the input to a VP8 decoder is a sequence of compressed frames, and they have to stay in the same order they happened in time 🕰️. Stuff like how long each frame lasts, the matching audio, and syncing it all up is usually handled by the playback environment, so the decoding process itself doesn't care about any of it 💅. The one exception is that every key frame has a start code in its header, which helps with fast seeking 🔍.
The decoder just gets handed a stream of compressed frames and spits out a matching stream of decompressed (reconstructed) YUV frames 🎞️. Remember how the Introduction said the exact pixel values in the reconstructed frame are part of VP8's specification? Yep, that's still the vibe 💜. This document lays out exactly how the compressed frames are structured and gives unambiguous algorithms for making the reconstructed frames correctly 🎀.
The first frame the decompressor sees is always a key frame 👑. After that, it can be followed by any number of interframes, and each one can only be rebuilt correctly if you have every earlier frame all the way back to that key frame 🔗. The next key frame starts the whole thing over. The decompressor resets to its default initial condition as soon as it gets a key frame, and decoding a key frame (plus the interframes after it) doesn't depend on any decoding that happened before 🌸.
At the top level, every compressed frame has three or more pieces 🧩. It kicks off with an uncompressed data chunk that's 10 bytes for key frames and 3 bytes for interframes 🦋. After that come two or more blocks of compressed data, called partitions 💖. Every partition starts and ends on a byte boundary, so no sneaky half-bytes 🙈.
The first compressed partition has two subsections:
- Header information that applies to the frame as a whole ✨.
- Per-macroblock information specifying how each macroblock is predicted from the already-reconstructed data that is available to the decompressor 🌈.
As we just said, the macroblock-level information shows up in raster-scan order 📖.
The rest of the partitions hold, for each block, the DCT/WHT coefficients (quantized and logically compressed) of the residue signal that gets added onto the predicted block values 🍓. These coefficients are about 70% of the overall datarate, so they're a huge chunk of the data 😭. VP8 can pack the compressed DCT/WHT coefficient data from macroblock rows into separate partitions 🧚. If there's more than one partition for these coefficients, the sizes of every partition except the last one (in bytes) also appear in the bitstream, right after the first partition ⭐. Each size is a 3-byte data item written in little endian format 💅. These sizes give the decoder direct access to all the DCT/WHT coefficient partitions, which means a decoder can process the coefficients in parallel 🔥.
Splitting the prediction data and the coefficient data up like this also gives decompressor implementers some flexibility 🦋. One option is to decode and store the prediction info for the whole frame first, then decode, transform, and add the residue signal to the entire frame 💜. The other option is to decode both partitions at the same time, working out the prediction info and adding in the residue signal for each block in order 🫶. The length field in the frame tag is what lets decoding of the second partition start before the first partition is fully decoded, and it's required for that second "block-at-a-time" implementation 🎀.
All partitions are decoded with separate instances of the boolean entropy decoder described in Section 7 ✨. Some of the data in the partitions is conceptually "flat" (a bit is just a bit, with no probabilistic expectation one way or the other), but because of how these coders work, a "conceptual bit" never lines up directly with a physical bit in the compressed partitions 🌀. The only place that kind of one-to-one physical correspondence exists is in the 3- or 10-byte uncompressed chunk we talked about earlier 🍬.
Also, seeking within a partition is not supported 🚫. You have to decompress and process the data (or at least store it) in the order it appears in the partition 🐢.
Okay, last thing, and it's a big one 🤓. This document gets the ordering of the partition data exactly right, but the details and semantics are explained in a more logical order so they're easier to follow 🧠. For example, the frame header contains updates to many of the probability tables used when decoding per-macroblock data 💅. So the per-macroblock data often gets described before the layouts of the probabilities and their updates, even though that's the opposite of how they appear in the bitstream 😭💖.
5. Overview of the Decoding Process ✨
okay so a VP8 decoder needs to keep four YUV frame buffers 🖼️, and each one has to be at least as big as the encoded image 💖. They hold the current frame being reconstructed, the immediately previous reconstructed frame, the most recent golden frame, and the most recent altref frame. main character energy for sure 👑
Most implementations are gonna want to "pad" these buffers with "invisible" pixels that stretch a moderate number of pixels past all four edges of the visible image 🎀. Why? Because prediction blocks aren't guaranteed to sit inside the visible area of an earlier frame, so the padding lets all (or most) of them still grab usable image data 🧚.
Regardless of how much padding you pick 💅, the invisible rows above (or below) the image get filled with copies of the top (or bottom) row of the image. The invisible columns to the left (or right) of the image get filled with copies of the leftmost (or rightmost) visible row, and the four invisible corners get copies of the matching visible corner pixels 🦋. The way these prediction buffers get used (plus the suggested sizes for the halo) gets elaborated on in the discussion of motion vectors, interframe prediction, and sub-pixel interpolation later in this document 🔥.
As you'll see in the frame header description, the image dimensions are specified (and can change) with every key frame 🌸. These buffers, and any other data structures whose size depends on the size of the image, should be allocated (or re-allocated) immediately after the dimensions are decoded ✨.
Leaving most of the details for later, here's the outline of the decoding process 🫶:
- First, the frame header (the beginning of the first data partition) is decoded. This changes or adds to the maintained state of the decoder, and it provides the context in which the per-macroblock data can be interpreted 💜.
- The macroblock data occurs (and must be processed) in raster-scan order 🍓. This data comes in two or more parts. The first part, the prediction (or mode) part, comes in the remainder of the first data partition. The other parts make up the data partition(s) for the DCT/WHT coefficients of the residue signal. For each macroblock, the prediction data must be processed before the residue 😭.
- Each macroblock is predicted using one (and only one) of four possible frames. All macroblocks in a key frame, and all intra-coded macroblocks in an interframe, are predicted using the already-decoded macroblocks in the current frame. Macroblocks in an interframe may also be predicted using the previous frame, the golden frame, or the altref frame. Those are called inter-coded macroblocks 🌈.
- The purpose of prediction is to use already-constructed image data to approximate the part of the original image being reconstructed. Any of the prediction modes ends up writing a macroblock-sized prediction buffer full of that approximation 🧚.
- No matter which prediction method was used, the residue DCT signal gets decoded, dequantized, reverse-transformed, and added to the prediction buffer. That gives the (almost final) reconstruction value of the macroblock, which gets stored in the correct position of the current frame buffer 💅.
The residue signal is made up of 24 (sixteen Y, four U, and four V) 4x4 quantized and losslessly compressed DCT transforms. Together they approximate the difference between the original macroblock in the uncompressed source and the prediction buffer 💖. For most prediction modes, the 0th coefficients of the sixteen Y subblocks are expressed via a 25th WHT of the second-order virtual Y2 subblock discussed above 🤓.
Intra-prediction exploits the spatial coherence of frames, which is just a fancy way of saying nearby pixels in the same frame look alike 🦋. The 16x16 luma (Y) and 8x8 chroma (UV) components are predicted independently of each other. Each one uses one of four simple means of pixel propagation, starting from the already-reconstructed row above and the column to the left of the current macroblock (16 pixels long for luma, 8 pixels long for chroma) 🌸. The four methods are:
- Copying the row from above throughout the prediction buffer.
- Copying the column from the left throughout the prediction buffer.
- Copying the average value of the row and column throughout the prediction buffer.
- Extrapolation from the row and column using the (fixed) second difference (horizontal and vertical) from the upper left corner.
The sixteen Y subblocks can also be predicted independently of each other, using one of ten different modes 🍭. Four of those are 4x4 analogs of the ones above, and the other six are "diagonal" prediction methods 🔥. There are two types of predictions, one intra and one prediction (among all the modes), for which the residue signal does not use the Y2 block to encode the DC portion of the sixteen 4x4 Y subblock DCTs. This "independent Y subblock" mode has no effect on the 8x8 chroma prediction 🙈.
Inter-prediction exploits the temporal coherence between nearby frames, meaning frames that are next to each other in time look alike too ✨. Apart from the choice of the prediction frame itself, there's no difference between inter-prediction based on the previous frame and inter-prediction based on the golden frame or altref frame 💜.
Inter-prediction is conceptually very simple 🧁. For efficiency reasons there are several ways to encode the relationship between the current macroblock and the matching sections of the prediction frame. Ultimately, though, each of the sixteen Y subblocks is related to a 4x4 subblock of the prediction frame. That subblock's position differs from the current subblock's position by a (usually small) displacement. These two-dimensional displacements are called motion vectors 🦋.
The motion vectors used by VP8 have quarter-pixel precision 🌈. Predicting a subblock with a motion vector whose components are integers (whole numbers) is super easy 💅. The 4x4 block of pixels from the displaced block in the previous, golden, or altref frame just gets copied into the correct position of the current macroblock's prediction buffer ✨.
Fractional displacements are conceptually and implementationally more complex 😭. They need the inference (or synthesis) of sample values that, strictly speaking, don't exist. This is one of the most basic problems in signal processing, and readers who know that subject well will see that VP8's approach gives a good balance of robustness, accuracy, and efficiency 👑.
We'll get into the details in the implementation discussion below, but the pixel interpolation is calculated by applying a kernel filter (using reasonable-precision integer math) three pixels on either side, both horizontally and vertically, of the pixel being synthesized. The resulting 4x4 block of synthetic pixels then gets copied into position, exactly like it does for integer displacements 🧚.
Each of the eight chroma subblocks gets handled similarly 🎀. Their motion vectors are never specified explicitly. Instead, the motion vector for each chroma subblock is calculated by averaging the vectors of the four Y subblocks that occupy the same area of the frame. Since chroma pixels have twice the diameter (and four times the area) of luma pixels, the calculated chroma motion vectors have 1/8-pixel resolution. The procedure for copying or generating pixels for each subblock is basically identical to the one used in the luma plane 🌸.
Once all the macroblocks have been generated (predicted and corrected with the DCT/WHT residue), a filtering step called the loop filter gets applied to the entire frame 🔥. Its purpose is to reduce blocking artifacts at the boundaries between macroblocks and between the subblocks of the macroblocks 🙈. It's called a "loop filter" because it's part of the "coding loop", which means it affects the reconstructed frame buffers that get used to predict the frames that come after 🔁. That's different from the postprocessing filters discussed earlier, which only affect the viewed video and don't "feed into" subsequent frames 💖.
Next, if the data signals it, the current frame may replace the golden frame prediction buffer and/or the altref frame buffer 🌈.
After that, the halos of the frame buffers get filled as specified above ✨. Finally, at least as far as decoding goes, the (references to the) "current" and "last" frame buffers should be exchanged to get ready for the next frame 🫶.
Various processes may be required (or desired) before the generated frame is viewed 👑. As discussed in the frame dimension information below, truncation and/or upscaling of the frame may be required 🧚. Some playback systems may need a different frame format (RGB, YUY2, etc.) 🍓. Finally, as mentioned in the Introduction, further postprocessing or filtering of the image before viewing may be desired 💅. Since the main purpose of this document is a decoding specification, the postprocessing is not specified here 🤓.
The basic ideas of prediction and correction in VP8 are pretty straightforward, but a lot of the details are super complex 😭. The management of probabilities is especially elaborate 🎀. The various intra-prediction modes and motion vector specifications have their own associated probabilities. Those, together with the coding of DCT coefficients and motion vectors, often base their probabilities on all kinds of contextual information (calculated from what's already been decoded), plus explicit modifications made via the frame header 🦋.
The "top-level" part of decoding and frame reconstruction is implemented in the reference decoder file dixie.c (Section 20.4) 💜.
That wraps up our summary of decoding and reconstruction 🌸. Next we'll dig into the individual aspects in more depth 🔥.
A reasonable "divide and conquer" approach to implementing a decoder is to start by decoding streams made up entirely of key frames 🧁. Once that works reliably, interframe handling is way easier to add than it would be if you tried the complete functionality all at once ✨. With that in mind, we'll first talk about the components needed to decode key frames (most of which are also used for decoding interframes), and then wrap up with topics that only apply to interframes 🫶💖.
6. Description of Algorithms ✨
okay so the whole point of this document, together with the reference decoder source code, is to give us a platform-independent way to decode and reconstruct a VP8 video stream 💖 which means lots of (small) algorithms have to be described exactly. no vibes-based implementing, bestie 💅
the reason it's written in C is honestly the main character move 👑 C is near-universal, it's terse, it's really good at describing calculations at specific precisions, and On2's reference VP8 decoder is written in C too. so these algorithm fragments are in C, with a few simple definitions added below. the standard (and best) reference for C is [Kernighan] 📚
there are going to be a LOT of code fragments in here. some will be almost identical to the matching bits of the reference decoder, and others will look different 🦋 roughly, there are three reasons for that:
- for efficiency reasons, the reference decoder version can be less obvious 🙈
- the reference decoder often uses big data structures to hold context that we don't need to describe or use here 🧚
- the authors felt a different way of writing the same algorithm might make it easier to explain 🌸
no matter which version you're reading, the calculation done by any of these algorithms is identical to the calculation done by the matching part of the reference decoder. so the sparkly version and the messy version are literally the same thing ✨
all VP8 decoding algorithms use integer math 🔢 so to make the arithmetic precision super clear, we define the following types:
---- Begin code block --------------------------------------
typedef signed char int8; /* signed int exactly 8 bits wide */
typedef unsigned char uint8; /* unsigned "" */
typedef short int16; /* signed int exactly 16 bits wide */
typedef unsigned int16 uint16; /* unsigned "" */
/* int32 is a signed integer type at least 32 bits wide */
typedef long int32; /* guaranteed to work on all systems */
typedef int int32; /* will be more efficient on some systems */
typedef unsigned int32 uint32;
/* unsigned integer type, at least 16 bits wide, whose exact size
is most convenient to whatever processor we are using */
typedef unsigned int uint;
/* While pixels themselves are 8-bit unsigned integers,
pixel arithmetic often occurs at 16- or 32-bit precision and
the results need to be "saturated" or clamped to an 8-bit
range. */
typedef uint8 Pixel;
Pixel clamp255(int32 v) { return v < 0? 0 : (v < 255? v : 255);}
/* As is elaborated in the discussion of the bool_decoder below,
VP8 represents probabilities as unsigned 8-bit numbers. */
typedef uint8 Prob;
---- End code block ----------------------------------------pixels are 8-bit unsigned integers 🎀 but the maths on them often happens at 16- or 32-bit precision, and then the results need to get "saturated" (clamped) back into the 8-bit range. that's what clamp255 is for 💜 and Prob gets its own type because, as you'll see in the bool_decoder section, VP8 stores probabilities as unsigned 8-bit numbers 🫶
now some maths-y stuff we sometimes need to talk about. we occasionally need "honest-to-goodness" infinite precision real numbers 🤓 the DCT is first described with the cosine function cos 🌈 the ratio of a circle's circumference to its diameter is pi 🥧 at one point we take a (base 1/2) logarithm, written as log 📐 and pow(x, y) means x raised to the power y. if x = 2 and y is a small non-negative integer, then pow(2, y) can be written in C as 1 << y ✨
last one, and it's a bit of a plot twist 😭 sometimes we need to divide signed integers by powers of two, which means right-shifting signed numbers. the thing is, how a shift handles the sign bit (the propagation of the sign bit) is, perhaps surprisingly, NOT defined by the C language itself. it's left up to each compiler 💀 it's arguable that the language should define it (so the sign bit naturally propagates), and at the very least any reasonable compiler should implement it correctly. so for strict portability, we try to call attention to these shifts whenever they show up 🚩
7. Boolean Entropy Decoder 🧠
as we talked about in the overview above, basically the entire VP8 data stream is encoded with a boolean entropy coder 💖
understanding the bool_decoder is critical if you're implementing a VP8 decompressor, so we're going to go through it in detail 🔍 it's way easier to get it when you look at it alongside the bool_encoder, which is what the compressor uses to write the compressed data partitions 🫶
the bool_encoder encodes (and the bool_decoder decodes) one bool at a time 🎯 (a bool is just a zero-or-one boolean value). its job is to losslessly compress a sequence of bools whose probability of being zero or one can be pretty well estimated when they're written (from constants or from info that's already been coded). then, when they're read, the decoder uses the same matching probabilities 🌸
as you're probably aware, if a bool is way more likely to be zero than one, it can, on average, be encoded in much less than one bit per value 🍓 the bool_encoder totally takes advantage of that 😤
in the 1940s, [Shannon] proved two things 🧚 first, there's a lower bound on the average datarate for a faithful encoding of a sequence of bools (when their probability distributions are known and independent of each other). second, there are encoding algorithms that get as close to that lower bound as you like 🦋
if we encode a sequence of bools where the probability of zero is p (and the probability of one is 1-p), then the lowest possible datarate per value is:
plog(p) + (1-p)log(1-p);taking logs to the base 1/2 gives you the datarate in bits/value 📊
two quick examples so it clicks 💅 at one extreme, if p = 1/2, then log(p) = log(1-p) = 1, and the lowest possible datarate per bool is 1/2 + 1/2 = 1. so we can't do any better than just literally writing out the bits 🙃 at the other extreme, if p is very small, say p = 1/1024, then log(p)=10, log(1-p) is roughly .0014, and the lowest possible datarate is approximately 10/1024 + .0014, which is roughly 1/100 of a bit per bool 🤯
most of the bools in the VP8 datastream have zero-probabilities nowhere near 1/2 😌 so the compression the bool_encoder provides is critical to VP8's performance 👑
the boolean coder VP8 uses is a variant of an arithmetic coder 🎀 if you want an excellent discussion of arithmetic coding (and other lossless compression techniques), check out [Bell] 📚
7.1. Underlying Theory of Coding 🫧
the basic idea behind the boolean coder is to treat the entire data stream (either of the partitions, in our case) as the binary expansion of a single number x, where 0 <= x < 1 🌈 the bits (or bytes) in x are written from high to low order. if b[j] (B[j]) is the j^(th) bit (byte) in the partition, then x is just the sum (starting with j = 1) of pow(2, -j) * b[j], or pow(256, -j) * B[j] ✨
before the first bool gets coded, every value of x is possible 🦋
coding each bool narrows down the possible values of x in proportion to the probability of what's being coded 💜 if p1 is the probability of the first bool being zero, and a zero is coded, then the range of possible x values becomes 0 <= x < p1. if a one is coded, the range becomes p1 <= x < 1 🎀
the coding keeps repeating that same idea 🔁 at every stage there's an interval a <= x < b of possible values of x. if p is the probability of a zero being coded at this stage and a zero is coded, the interval becomes a <= x < a + (p(b-a)). if a one is coded, the possible values of x get restricted to a + (p(b-a)) <= x < b 🌸
assuming only finitely many values are going to be coded, once the encoder has received the last bool, it can write out any value x that lies in the final interval 🫶 VP8 simply writes the left endpoint of the final interval 💖 this means that if encoding stopped at any moment, the output it would've produced either increases or stays the same as each bool gets encoded 📈
decoding is basically encoding in reverse 🪞 the decoder gets the number x, which only has the initial restriction 0 <= x < 1. to decode the first bool, the decoder is given the first probability p1. if x < p1, a zero is decoded; if x >= p1, a one is decoded. either way, the new restriction on x (the interval of possible values of x) gets remembered 🧠
decoding carries on in exactly the same way 🔁 if a <= x < b is the current interval and we want to decode a bool with zero-probability p, we return a zero if a <= x < a + (p(b-a)), and a one if a + (p(b-a)) <= x < b. either way, the new restriction is remembered to prep for decoding the next bool 🦋
the process I just described uses infinite-precision real numbers to express the probabilities and ranges 🤓 and yeah, if you could actually run it, and you coded a big number of bools whose supplied probabilities matched their value distributions, the datarate would approach the theoretical minimum as the number of bools went up 🚀
unfortunately, computers work at finite precision, so we need an approximation of that theoretically perfect process 😭 the approximation does increase the datarate, but at fairly moderate precision and for a wide variety of data sets, that increase is negligible 💅
there are only two conceptual limitations. first, coder probabilities have to be expressed at finite precision. second, the decoder has to be able to detect each individual change to the value interval by looking at a fixed amount of input 🎀 as a practical matter, most of the implementation details come from the fact that the coder can work with just a small "window" to read or write the arbitrarily precise number x incrementally 🫧 so that's the vibe, bestie 🌈
7.2. Practical Algorithm Description 🎀
Okay bestie, VP8's boolean coder works with 8-bit probabilities p 💅 The range of p is 0 <= p <= 255, and the real probability it stands for is p/256. Ngl, the coder is designed so that decoding a bool only needs an 8-bit comparison, and the state of both the encoder and the decoder fits in a small number of unsigned 16-bit integers 🧚
The easiest way to get it is to start with bit-at-a-time input and output 🦋 Besides keeping track of a position in the bitstream and writing or reading bits, the encoder also has to be able to add 1 to the bits it already output. After writing n bits, adding 1 to the existing output is the same thing as adding pow(2, -n) to x ✨
The encoder has to keep two unsigned 8-bit numbers alongside the bit position, called bottom and range 💖 Let w be the n bits already written, and S = pow(2, - n - 8) the scale of the current bit position one byte out. Then every future value v of w (including the final value v = x) has to satisfy this:
w + ( S * bottom ) <= v < w + ( S * ( bottom + range ) )So appending bottom to the bits already written gives the left endpoint of the interval of possible values, and appending bottom + range gives the right endpoint. range itself, scaled to the current output position, is the length of that interval 🌈
Here's the vibe check: we want our probabilistic encodings to be reasonably accurate, so range is not allowed to vary by more than a factor of two. It stays within 128 <= range <= 255 👑
Now the process for encoding a boolean value val. Its probability of being zero is prob / 256, and its probability of being one is ( 256 - prob ) / 256, with 1 <= prob <= 255. Here's how it goes 💅
First, an unsigned 16-bit multiply followed by an unsigned right shift gives us an unsigned 8-bit split value:
split = 1 + (((range - 1) * probability)]] >> 8)split is approximately ( prob / 256 ) * range, and it lies within 1 <= split <= range - 1. Those bounds are what make the decoding procedure below correct 🔥
If the incoming boolean val is false, we leave the left interval endpoint bottom alone and replace range with split. If val is true, we move the left endpoint up to bottom + split, propagating any carry into the already-written value w (this is the spot where we need the ability to add 1 to w), and reduce range to range - split 🦋
No matter which value got encoded, range has been reduced and now sits within 1 <= range <= 254. If range < 128, the encoder doubles it and shifts the high-order bit out of bottom to the output while also doubling bottom. It keeps repeating this one bit at a time until 128 <= range <= 255. Once that's done, the encoder is ready for the next bool, and the constraints from above still hold 🍓
After encoding the last bool, the partition can be completed by appending bottom to the bitstream ✨
The decoder mirrors the encoder's state. Along with an input bit position, it keeps two unsigned 8-bit numbers: a range identical to the encoder's, and a value. Decoding one bool at a time, the decoder effectively tracks the same left interval endpoint as the encoder, and subtracts it from the remaining input. Appending the unread part of the bitstream to the 8-bit value gives the difference between the value that was actually encoded and the known left endpoint 💜
To start, the decoder sets range = 255 and reads the first 16 input bits into value. It maintains range and calculates split in exactly the same way the encoder does 🧚
To decode a bool, the decoder compares value to split. If value < split, the bool is zero and range is replaced with split. If value >= split, the bool is one, range is replaced with range - split, and value is replaced with value - split 😭
Again, range is doubled one bit at a time until it is at least 128. value is doubled alongside it, with a new input bit shifted into the bottom each time 🌸
If we write Value for value together with the unread input bits, and Range for range extended indefinitely on the right by zeros, then the condition Value < Range holds at all times on the decoder side. In particular, the bits shifted out of value as it doubles are always zero. Iconic, honestly 👑
7.3. Actual Implementation ✨
The C code below gives complete implementations of the encoder and decoder described above 💻 They're logically identical to the "bit-at-a-time" versions, but internally they buffer a couple of extra bytes of the bitstream. That lets I/O happen (more practically) one byte at a time, and it drastically reduces the number of carries the encoder has to propagate into the already-written data 🎀
Another logically equivalent implementation can be found in the reference decoder file bool_decoder.h (Section 20.2).
---- Begin code block --------------------------------------
/* Encoder first */
typedef struct {
uint8 *output; /* ptr to next byte to be written */
uint32 range; /* 128 <= range <= 255 */
uint32 bottom; /* minimum value of remaining output */
int bit_count; /* # of shifts before an output byte
is available */
} bool_encoder;
/* Must set initial state of encoder before writing any bools. */
void init_bool_encoder(bool_encoder *e, uint8 *start_partition)
{
e->output = start_partition;
e->range = 255;
e->bottom = 0;
e->bit_count = 24;
}
/* Encoding very rarely produces a carry that must be propagated
to the already-written output. The arithmetic guarantees that
the propagation will never go beyond the beginning of the
output. Put another way, the encoded value x is always less
than one. */
void add_one_to_output(uint8 *q)
{
while (*--q == 255)
*q = 0;
++*q;
}
/* Main function writes a bool_value whose probability of being
zero is (expected to be) prob/256. */
void write_bool(bool_encoder *e, Prob prob, int bool_value)
{
/* split is approximately (range * prob) / 256 and,
crucially, is strictly bigger than zero and strictly
smaller than range */
uint32 split = 1 + (((e->range - 1) * prob) >> 8);
if (bool_value) {
e->bottom += split; /* move up bottom of interval */
e->range -= split; /* with corresponding decrease in range */
} else
e->range = split; /* decrease range, leaving bottom alone */
while (e->range < 128)
{
e->range <<= 1;
if (e->bottom & (1 << 31)) /* detect carry */
add_one_to_output(e->output);
e->bottom <<= 1; /* before shifting bottom */
if (!--e->bit_count) { /* write out high byte of bottom ... */
*e->output++ = (uint8) (e->bottom >> 24);
e->bottom &= (1 << 24) - 1; /* ... keeping low 3 bytes */
e->bit_count = 8; /* 8 shifts until next output */
}
}
}
/* Call this function (exactly once) after encoding the last
bool value for the partition being written */
void flush_bool_encoder(bool_encoder *e)
{
int c = e->bit_count;
uint32 v = e->bottom;
if (v & (1 << (32 - c))) /* propagate (unlikely) carry */
add_one_to_output(e->output);
v <<= c & 7; /* before shifting remaining output */
c >>= 3; /* to top of internal buffer */
while (--c >= 0)
v <<= 8;
c = 4;
while (--c >= 0) { /* write remaining data, possibly padded */
*e->output++ = (uint8) (v >> 24);
v <<= 8;
}
}
/* Decoder state exactly parallels that of the encoder.
"value", together with the remaining input, equals the
complete encoded number x less the left endpoint of the
current coding interval. */
typedef struct {
uint8 *input; /* pointer to next compressed data byte */
uint32 range; /* always identical to encoder's range */
uint32 value; /* contains at least 8 significant bits */
int bit_count; /* # of bits shifted out of
value, at most 7 */
} bool_decoder;
/* Call this function before reading any bools from the
partition. */
void init_bool_decoder(bool_decoder *d, uint8 *start_partition)
{
{
int i = 0;
d->value = 0; /* value = first 2 input bytes */
while (++i <= 2)
d->value = (d->value << 8) | *start_partition++;
}
d->input = start_partition; /* ptr to next byte to be read */
d->range = 255; /* initial range is full */
d->bit_count = 0; /* have not yet shifted out any bits */
}
/* Main function reads a bool encoded at probability prob/256,
which of course must agree with the probability used when the
bool was written. */
int read_bool(bool_decoder *d, Prob prob)
{
/* range and split are identical to the corresponding values
used by the encoder when this bool was written */
uint32 split = 1 + (((d->range - 1) * prob) >> 8);
uint32 SPLIT = split << 8;
int retval; /* will be 0 or 1 */
if (d->value >= SPLIT) { /* encoded a one */
retval = 1;
d->range -= split; /* reduce range */
d->value -= SPLIT; /* subtract off left endpoint of interval */
} else { /* encoded a zero */
retval = 0;
d->range = split; /* reduce range, no change in left endpoint */
}
while (d->range < 128) { /* shift out irrelevant value bits */
d->value <<= 1;
d->range <<= 1;
if (++d->bit_count == 8) { /* shift in new bits 8 at a time */
d->bit_count = 0;
d->value |= *d->input++;
}
}
return retval;
}
/* Convenience function reads a "literal", that is, a "num_bits"-
wide unsigned value whose bits come high- to low-order, with
each bit encoded at probability 128 (i.e., 1/2). */
uint32 read_literal(bool_decoder *d, int num_bits)
{
uint32 v = 0;
while (num_bits--)
v = (v << 1) + read_bool(d, 128);
return v;
}
/* Variant reads a signed number */
int32 read_signed_literal(bool_decoder *d, int num_bits)
{
int32 v = 0;
if (!num_bits)
return 0;
if (read_bool(d, 128))
v = -1;
while (--num_bits)
v = (v << 1) + read_bool(d, 128);
return v;
}---- End code block ----------------------------------------
8. Compressed Data Components ✨
okay so at the very bottom, VP8's compressed data is just a long line of probabilistically encoded bools 🫧 Most of the data is built out of slightly bigger chunks made from those bools, and those chunks are what we're about to meet 💖
we sometimes use these names inside C expressions in the data format specs. in that context, they mean whatever a call to an appropriate bool_decoder d returns, reading (as always) from its current reference point 🎀
+--------------+-------+--------------------------------------------+
| Call | Alt. | Return |
+--------------+-------+--------------------------------------------+
| Bool(p) | B(p) | Bool with probability p/256 of being 0. |
| | | Return value of read_bool(d, p). |
| | | |
| Flag | F | A one-bit flag (same thing as a B(128) or |
| | | an L(1)). Abbreviated F. Return value of |
| | | read_bool(d, 128). |
| | | |
| Lit(n) | L(n) | Unsigned n-bit number encoded as n flags |
| | | (a "literal"). Abbreviated L(n). The |
| | | bits are read from high to low order. |
| | | Return value of read_literal(d, n). |
| | | |
| SignedLit(n) | | Signed n-bit number encoded similarly to |
| | | an L(n). Return value of |
| | | read_signed_literal(d, n). These are |
| | | rare. |
| | | |
| P(8) | | An 8-bit probability. No different from |
| | | an L(8), but we sometimes use this |
| | | notation to emphasize that a probability |
| | | is being coded. |
| | | |
| P(7) | | A 7-bit specification of an 8-bit |
| | | probability. Coded as an L(7) number x; |
| | | the resulting 8-bit probability is x ? x |
| | | << 1 : 1. |
| | | |
| F? X | | A flag that, if true, is followed by a |
| | | piece of data X. |
| | | |
| F? X:Y | | A flag that, if true, is followed by X |
| | | and, if false, is followed by Y. Also |
| | | used to express a value where Y is an |
| | | implicit default (not encoded in the data |
| | | stream), as in F? P(8):255, which |
| | | expresses an optional probability: If the |
| | | flag is true, the probability is specified |
| | | as an 8-bit literal, while if the flag is |
| | | false, the probability defaults to 255. |
| | | |
| B(p)? X | B(p)? | Variants of the above using a boolean |
| | X:Y | indicator whose probability is not |
| | | necessarily 128. |
| | | |
| T | | Tree-encoded value from small alphabet. |
+--------------+-------+--------------------------------------------+ok the last one, T, needs a little explaining 🌸 we often want to encode something that can only be one of a small number of things (the alphabet) 🧚
the trick is to represent that alphabet as the leaves of a small binary tree 🌳 the non-leaf nodes have probabilities p and each one corresponds to a call to read_bool(d, p). a zero means "go left" below the node, and a one means "go right" 👑
so every value (leaf) at tree depth x gets decoded after exactly x calls to read_bool 💅
a tree for an alphabet of n possible values always has n-1 non-leaf nodes, no matter what shape it is. you can see this just by induction on n 🦋
there are lots of ways to build a tree for the same alphabet. the choice doesn't change the datarate much, but it does affect how fast the decoder runs 🔥 VP8's trees are picked so that, on average, they make as few calls to read_bool as possible. basically the more probable values get shallower tree depths than the less probable ones 🍓
if you know Huffman coding, you'll notice that given an alphabet and the probability of each value, the matching Huffman tree minimizes the expected number of calls to read_bool 🤓
and this method never gives a higher datarate than Huffman does. it often gives a much lower one, honestly iconic 😭 Huffman is really just a special case of this where every node probability is fixed at 128 (that's 1/2) 💜
8.1. Tree Coding Implementation
here's a suggested way to implement a tree data structure, followed by a couple of real examples of VP8 using it 🎀
it's easiest to represent the values as small positive integers, usually an enum that counts up from zero. the biggest alphabet that VP8 tree-codes is the one for DCT coefficients (we'll get there in Section 13), and it only has 12 values 🌈 its tree adds 11 interior nodes, so it has 23 positions in total. that means an 8-bit number is enough to hold both a tree position and a return value ✨
so a tree can be stored compactly as an array of pairs of 8-bit integers. each even array index is an interior node, and index 0 is the root 🌳 the entries come in pairs for the left (0) and right (1) branches under that interior node. a positive (even) branch entry is the index of a deeper interior node, and a nonpositive entry v is a leaf whose value is -v 🫶
the node probabilities for a tree-coded value live in their own array, and their indices are half the indices of the matching tree positions. that probability array is one shorter than the alphabet 💖
here's the C code for all that. the perks of this data structure are great 💅 the structure itself is tiny, and the tree-directed reading algorithm is basically a single line of code 🔥
---- Begin code block --------------------------------------
/* A tree specification is simply an array of 8-bit integers. */
typedef int8 tree_index;
typedef const tree_index Tree[];
/* Read and return a tree-coded value at the current decoder
position. */
int treed_read(
bool_decoder * const d, /* bool_decoder always returns a 0 or 1 */
Tree t, /* tree specification */
const Prob p[] /* corresponding interior node probabilities */
) {
register tree_index i = 0; /* begin at root */
/* Descend tree until leaf is reached */
while ((i = t[ i + read_bool(d, p[i>>1])]) > 0) {}
return -i; /* return value is negation of nonpositive index */
}
---- End code block ----------------------------------------tree-based decoding lives in the reference decoder file bool_decoder.h (Section 20.2) 🧚
8.2. Tree Coding Example
okay, this is a multi-part example 🎀 without diving too deep into macroblock decoding (which we'll totally cover later), let's look at the "mode" coding for intra-predicted macroblocks 💜
here's the fun part: because of a difference in statistics, the Y (or luma) mode uses two different trees. one is for key frames and the other is for interframes 🙈 this is the only place in VP8 where the same dataset gets coded by different trees depending on the situation 😭 the UV (or chroma) modes are a proper subset of the Y modes, so they get their own decoding tree 🌸
---- Begin code block --------------------------------------
typedef enum
{
DC_PRED, /* predict DC using row above and column to the left */
V_PRED, /* predict rows using row above */
H_PRED, /* predict columns using column to the left */
TM_PRED, /* propagate second differences a la "True Motion" */
B_PRED, /* each Y subblock is independently predicted */
num_uv_modes = B_PRED, /* first four modes apply to chroma */
num_ymodes /* all modes apply to luma */
}
intra_mbmode;
/* The aforementioned trees together with the implied codings as
comments.
Actual (i.e., positive) indices are always even.
Value (i.e., nonpositive) indices are arbitrary. */
const tree_index ymode_tree [2 * (num_ymodes - 1)] =
{
-DC_PRED, 2, /* root: DC_PRED = "0", "1" subtree */
4, 6, /* "1" subtree has 2 descendant subtrees */
-V_PRED, -H_PRED, /* "10" subtree: V_PRED = "100",
H_PRED = "101" */
-TM_PRED, -B_PRED /* "11" subtree: TM_PRED = "110",
B_PRED = "111" */
};
const tree_index kf_ymode_tree [2 * (num_ymodes - 1)] =
{
-B_PRED, 2, /* root: B_PRED = "0", "1" subtree */
4, 6, /* "1" subtree has 2 descendant subtrees */
-DC_PRED, -V_PRED, /* "10" subtree: DC_PRED = "100",
V_PRED = "101" */
-H_PRED, -TM_PRED /* "11" subtree: H_PRED = "110",
TM_PRED = "111" */
};
const tree_index uv_mode_tree [2 * (num_uv_modes - 1)] =
{
-DC_PRED, 2, /* root: DC_PRED = "0", "1" subtree */
-V_PRED, 4, /* "1" subtree: V_PRED = "10",
"11" subtree */
-H_PRED, -TM_PRED /* "11" subtree: H_PRED = "110",
TM_PRED = "111" */
};
/* Given a bool_decoder d, a Y mode might be decoded as follows. */
const Prob pretend_its_huffman [num_ymodes - 1] =
{ 128, 128, 128, 128};
Ymode = (intra_mbmode) treed_read(d, ymode_tree,
pretend_its_huffman);
---- End code block ----------------------------------------so, since it makes reusing reference code so much easier, and there's honestly no good reason not to, it's strongly suggested that any decoder implementation use exactly the same enumeration values and probability table layouts as this document (and the reference code) does for all tree-coded data in VP8 💅🫶
9. Frame Header 🎀
the uncompressed data chunk at the start of each frame, plus the first bit of the first data partition, holds info about the frame as a whole 🌸 we'll go through the fields in the order they show up 🦋 most of the header decoding happens in the reference decoder file dixie.c (Section 20.4) 🧚
9.1. Uncompressed Data Chunk ✨
okay so the uncompressed data chunk starts with a 3-byte frame tag 🎀 that key frames AND interframes both share, and it packs in four fields:
- A 1-bit frame type (0 for key frames, 1 for interframes). 🌸
- A 3-bit version number (0 - 3 are defined as four different profiles with different decoding complexity; other values may be defined for future variants of the VP8 data format). 🦋
- A 1-bit show_frame flag (0 when current frame is not for display, 1 when current frame is for display). 👀
- A 19-bit field containing the size of the first data partition in bytes. 💜
the version number is giving control panel 🎛️ because it turns certain features in the bitstream on or off:
+---------+-------------------------+-------------+
| Version | Reconstruction Filter | Loop Filter |
+---------+-------------------------+-------------+
| 0 | Bicubic | Normal |
| | | |
| 1 | Bilinear | Simple |
| | | |
| 2 | Bilinear | None |
| | | |
| 3 | None | None |
| | | |
| Other | Reserved for future use | |
+---------+-------------------------+-------------+the reference software also tweaks the loop filter based on the version number, like the table says 💅. version 1 implies a "simple" loop filter, and versions 2 and 3 imply no loop filter. BUT (this is the twist 😭) the "simple" setting has no effect whatsoever on decoding, and the "no loop filter" setting only forces the reference encoder to set filter level equal to 0. neither one affects the decoding process. in decoding, the only loop filter settings that matter are the ones in the frame header. not me getting confused by the version number 🙈
for key frames, the frame tag is followed by 7 more bytes of uncompressed data, laid out like this:
Start code byte 0 0x9d
Start code byte 1 0x01
Start code byte 2 0x2a
16 bits : (2 bits Horizontal Scale << 14) | Width (14 bits)
16 bits : (2 bits Vertical Scale << 14) | Height (14 bits)the following source code segment shows how to validate the start code and read the width, height, and scale factors for a key frame:
unsigned char *c = pbi->source+3;
// vet via sync code
if (c[0]!=0x9d||c[1]!=0x01||c[2]!=0x2a)
return -1;where pbi->source points to the beginning of the frame. 📍
the following code reads the image dimension from the bitstream:
pc->Width = swap2(*(unsigned short*)(c+3))&0x3fff;
pc->horiz_scale = swap2(*(unsigned short*)(c+3))>>14;
pc->Height = swap2(*(unsigned short*)(c+5))&0x3fff;
pc->vert_scale = swap2(*(unsigned short*)(c+5))>>14;the swap2 macro handles the endian thing on a different platform 🧚:
#if defined(__ppc__) || defined(__ppc64__)
# define swap2(d) \
((d&0x000000ff)<<8) | \
((d&0x0000ff00)>>8)
#else
# define swap2(d) d
#endifso every frame is encoded as a raster scan of 16x16 macroblocks, but the frame dimensions aren't necessarily evenly divisible by 16. in that case, write ew = 16 - (width & 15) and eh = 16 - (height & 15) for the excess width and height, respectively. even though they're encoded, the last ew columns and eh rows aren't actually part of the image and should be discarded before final output. BUT these "excess pixels" should be maintained in the internal reconstruction buffer used to predict ensuing frames. the extras are hiding in plain sight 🫣
the scaling specifications for each dimension are encoded like this:
+-------+--------------------------------------+
| Value | Scaling |
+-------+--------------------------------------+
| 0 | No upscaling (the most common case). |
| | |
| 1 | Upscale by 5/4. |
| | |
| 2 | Upscale by 5/3. |
| | |
| 3 | Upscale by 2. |
+-------+--------------------------------------+upscaling doesn't touch the reconstruction buffer, which should be maintained at the encoded resolution. any reasonable upsampling method (including any that video hardware in the playback environment might support) may be used. since scaling has no effect on decoding, we're not going to discuss it any further. bye scaling 👋
as discussed in Section 5, allocation (or re-allocation) of data structures (such as the reconstruction buffer) whose size depends on dimension will be triggered here. 🔥
9.2. Color Space and Pixel Type (Key Frames Only) 🌈
+-------+------------------------------------------+
| Field | Value |
+-------+------------------------------------------+
| L(1) | 1-bit color space type specification |
| | |
| L(1) | 1-bit pixel value clamping specification |
+-------+------------------------------------------+the color space type bit works like this:
- 0 - YUV color space similar to the YCrCb color space defined in [ITU-R_BT.601]
- 1 - Reserved for future use
the pixel value clamping type bit works like this:
- 0 - Decoders are required to clamp the reconstructed pixel values to between 0 and 255 (inclusive).
- 1 - Reconstructed pixel values are guaranteed to be between 0 and 255; no clamping is necessary.
info in this subsection does not show up in interframes, so don't go looking for it there 🙅♀️
9.3. Segment-Based Adjustments 🍓
this subsection holds the probability and value info for segment adaptive adjustments to the default decoder behavior 💅. the data here is used when decoding the per-segment info that comes after, and it applies to the entire frame. when segment adaptive adjustments are enabled, every macroblock gets a segment ID. macroblocks with the same segment ID belong to the same segment and get the same adaptive adjustments over the default baseline values for the frame. the adjustments can be quantizer level or loop filter strength. ✨
the context for decoding this feature at the macroblock level comes from a subsection in the frame header, which has:
- A segmentation_enabled flag that enables the feature for this frame if set to 1, and disables it if set to 0. the following fields only occur if the feature is enabled. 🔘
- L(1) indicates if the segment map is updated for the current frame (update_mb_segmentation_map). 🗺️
- L(1) indicates if the segment feature data items are updated for the current frame (update_segment_feature_data). 🔄
- If Item 3 above (update_segment_feature_data) is 1, the following fields occur:
- a. L(1), the mode of segment feature data (segment_feature_mode), can be absolute-value mode (0) or delta value mode (1).
- b. Segment feature data items are decoded segment by segment for each segment feature. For every data item, a one-bit flag indicates whether the item is 0, or a non-zero value to be decoded. If the value is non-zero, then the value is decoded as a magnitude L(n), followed by a one-bit sign (L(1) -- 0 for positive and 1 for negative). The length n can be looked up from a pre-defined length table for all feature data. 📏
- If the L(1) flag as noted in Item 2 above is set to 1, the probabilities of the decoding tree for the segment map are decoded from the bitstream. Each probability is decoded with a one-bit flag indicating whether the probability is the default value of 255 (flag is set to 0), or an 8-bit value, L(8), from the bitstream. 🌳
the layout and semantics that support this feature at the macroblock level are described in Section 10. 🎀
9.4. Loop Filter Type and Levels 🦋
VP8 supports two types of loop filters with different computational complexity 🧠. the following bits show up in the header to choose the baseline type, strength, and sharpness behavior of the loop filter for the current frame:
+-------+-------------------+
| Index | Description |
+-------+-------------------+
| L(1) | filter_type |
| | |
| L(6) | loop_filter_level |
| | |
| L(3) | sharpness_level |
+-------+-------------------+the meaning of these numbers gets explained more in Section 15. 📖
VP8 also has a feature in the bitstream that lets the loop filter level adjust based on a macroblock's prediction mode and reference frame, which is giving customisation 💖. the per-macroblock adjustment is done through delta values against the default loop filter level for the current frame. this subsection holds the flag and value info for per-macroblock loop filter level adjustment over the default decoder behavior. the data here is used when decoding the per-macroblock info that comes after, and it applies to the entire frame. 🫶
L(1) is a one-bit flag saying whether the macroblock loop filter adjustment is on for the current frame. 0 means the feature isn't supported in the current frame, and 1 means it's enabled for the current frame. ✅
whether the adjustment is based on a reference frame or the encoding mode, it's done via a delta value against a baseline loop filter value. the delta values get updated for the current frame if an L(1) bit, mode_ref_lf_delta_update, is set to 1. there are two groups of delta values: one group is for reference frame-based adjustments, and the other group is for mode-based adjustments. the number of delta values in the two groups is MAX_REF_LF_DELTAS and MAX_MODE_LF_DELTAS, respectively. every value in the two groups gets a one-bit L(1) to show whether that particular value is updated. when one is updated (1), it's sent as a six-bit-magnitude L(6) followed by a one-bit sign flag (L(1) -- 0 for positive and 1 for negative). 🌸✨
9.5. Token Partition and Partition Data Offsets
Okay so VP8 lets you pack the DCT coefficients into multiple partitions, separate from the first partition that holds the header and per-macroblock prediction info. That means the decoder can go parallel and decode efficiently 🧚✨. A two-bit L(2) tells you how many coefficient data partitions are in the compressed frame. Here's the lookup table for those two bits 💅:
+-------+-------+----------------------+
| Bit 1 | Bit 0 | Number of Partitions |
+-------+-------+----------------------+
| 0 | 0 | 1 |
| | | |
| 0 | 1 | 2 |
| | | |
| 1 | 0 | 4 |
| | | |
| 1 | 1 | 8 |
+-------+-------+----------------------+The offsets are embedded right in the bitstream so the decoder can jump straight to each token partition 🦋. If there's more than one data partition, every partition's size gets written in 3 bytes (24 bits), except for the last one. The last partition just gets whatever data is left over after all the others 🎀.
The partitioned data sit back to back in the bitstream, so once you know the sizes you can also work out where each partition starts 🌸. Here's the pseudocode showing how those three bytes turn into a size/offset:
Offset/size = (uint32)(byte0) + ((uint32)(byte1)<<8)
+ ((uint32)(byte2)<<16);9.6. Dequantization Indices
Every residue signal is a quantized 4x4 DCT applied to the Y, U, V, or Y2 subblocks of a macroblock 🔥. Section 14 has the full details, but the gist is this: before the transform gets inverted, each decoded coefficient gets multiplied by one of six dequantization factors. Which factor you use depends on the plane (Y, chroma = U or V, or Y2) and the coefficient position (DC = coefficient 0, AC = coefficients 1-15) 👑. Those six values come from 7-bit indices into six matching fixed tables, and the tables themselves live in Section 14 💖.
The first 7-bit index is the dequantization table index for the Y-plane AC coefficients, called yac_qi. It's always coded, and it's the baseline for the other five quantization indices. Each of those is just a delta from this baseline 🫶. Here's how the indices get read:
yac_qi = L(7); /* Y ac index always specified */
ydc_delta = F? delta(): 0; /* Y dc delta specified if
flag is true */
y2dc_delta = F? delta(): 0; /* Y2 dc delta specified if
flag is true */
y2ac_delta = F? delta(): 0; /* Y2 ac delta specified if
flag is true */
uvdc_delta = F? delta(): 0; /* chroma dc delta specified
if flag is true */
uvac_delta = F? delta(): 0; /* chroma ac delta specified
if flag is true */So delta() is the process of reading 5 bits from the bitstream to get a signed delta value 🍓:
+-------+--------------------------------------------------+
| Index | Description |
+-------+--------------------------------------------------+
| L(4) | Magnitude of delta |
| | |
| L(1) | Sign of delta, 0 for positive and 1 for negative |
+-------+--------------------------------------------------+9.7. Refresh Golden Frame and Altref Frame
Okay this one is giving main character energy 👑. On key frames, the golden frame and the altref frame both get refreshed (replaced) by the current reconstructed frame by default. On non-key frames, VP8 uses two bits to say whether each of those two buffers gets refreshed with the reconstructed current frame 🌈:
+-------+----------------------------------------------------------+
| Index | Description |
+-------+----------------------------------------------------------+
| L(1) | Whether golden frame is refreshed (0 for no, 1 for yes). |
| | |
| L(1) | Whether altref frame is refreshed (0 for no, 1 for yes). |
+-------+----------------------------------------------------------+When the golden frame flag is 0, VP8 reads 2 more bits to figure out whether a buffer gets copied into the golden frame, and which one, or whether nothing gets copied at all 🦋:
+-------+------------------------------------------+
| Index | Description |
+-------+------------------------------------------+
| L(2) | Buffer copy flag for golden frame buffer |
+-------+------------------------------------------+Where:
0means no buffer is copied to the golden frame1means last_frame is copied to the golden frame2means alt_ref_frame is copied to the golden frame
Same energy for altref: when its flag is 0, VP8 uses 2 bits in the bitstream to indicate which buffer gets copied to alt_ref_frame 🎀:
+-------+------------------------------------------+
| Index | Description |
+-------+------------------------------------------+
| L(2) | Buffer copy flag for altref frame buffer |
+-------+------------------------------------------+Where:
0means no buffer is copied to the altref frame1means last_frame is copied to the altref frame2means golden_frame is copied to the altref frame
Then there are two bits for ref_frame_sign_bias, one for golden_frame and one for alt_ref_frame 🧚:
+-------+---------------------------------+
| Index | Description |
+-------+---------------------------------+
| L(1) | Sign bias flag for golden frame |
| | |
| L(1) | Sign bias flag for altref frame |
+-------+---------------------------------+These values control the sign of the motion vectors whenever a golden frame or an altref frame is the reference frame for a macroblock. Iykyk 🙈.
9.8. Refresh Last Frame Buffer
VP8 uses a single bit, L(1), to say whether the last frame reference buffer gets refreshed with the constructed current frame 🌸. On a key frame this bit gets overridden, and the last frame buffer is always refreshed. No exceptions, periodt 💅.
9.9. DCT Coefficient Probability Update
This field carries updates to the probability tables used for decoding DCT coefficients ✨. Every probability in the tables has an L(1) flag saying whether it's updated for the current frame. If that flag is 1, an extra 8-bit value follows with the new probability 🫶. These tables carry over across interframes, but they get reset to their defaults at the start of every key frame 🔥.
The layout and semantics of this field get their own deep dive in Section 13 💜.
9.10. Remaining Frame Header Data (Non-Key Frame)
Last stop on the frame header tour 🌈. Here's the layout for the rest of the non-key frame header data:
+-------+-----------------------------------------------------------+
| Index | Description |
+-------+-----------------------------------------------------------+
| L(1) | mb_no_skip_coeff. This flag indicates at the frame level |
| | if skipping of macroblocks with no non-zero coefficients |
| | is enabled. If it is set to 0, then prob_skip_false is |
| | not read and mb_skip_coeff is forced to 0 for all |
| | macroblocks (see Sections 11.1 and 12.1). |
| | |
| L(8) | prob_skip_false = probability used for decoding a |
| | macroblock-level flag, which indicates if a macroblock |
| | has any non-zero coefficients. Only read if |
| | mb_no_skip_coeff is 1. |
| | |
| L(8) | prob_intra = probability that a macroblock is "intra" |
| | predicted (that is, predicted from the already-encoded |
| | portions of the current frame), as opposed to "inter" |
| | predicted (that is, predicted from the contents of a |
| | prior frame). |
| | |
| L(8) | prob_last = probability that an inter-predicted |
| | macroblock is predicted from the immediately previous |
| | frame, as opposed to the most recent golden frame or |
| | altref frame. |
| | |
| L(8) | prob_gf = probability that an inter-predicted macroblock |
| | is predicted from the most recent golden frame, as |
| | opposed to the altref frame. |
| | |
| F | If true, followed by four L(8)s updating the |
| | probabilities for the different types of intra-prediction |
| | for the Y plane. These probabilities correspond to the |
| | four interior nodes of the decoding tree for intra-Y |
| | modes in an interframe, that is, the even positions in |
| | the ymode_tree array given above. |
| | |
| F | If true, followed by three L(8)s updating the |
| | probabilities for the different types of intra-prediction |
| | for the chroma planes. These probabilities correspond to |
| | the even positions in the uv_mode_tree array given above. |
| | |
| X | Motion vector probability update. Details are given in |
| | Section 17.2, "Probability Updates". |
+-------+-----------------------------------------------------------+And the decoding of this whole chunk is handled in the reference decoder file dixie.c (Section 20.4) 🧚💖
9.11. Remaining Frame Header Data (Key Frame) 💖
okay so this is the last bit of the key frame header 🎀 here's the table, with the two flags that wrap things up 💅
+-------+-----------------------------------------------------------+
| Index | Description |
+-------+-----------------------------------------------------------+
| L(1) | mb_no_skip_coeff. This flag indicates at the frame level |
| | if skipping of macroblocks with no non-zero coefficients |
| | is enabled. If it is set to 0, then prob_skip_false is |
| | not read and mb_skip_coeff is forced to 0 for all |
| | macroblocks (see Sections 11.1 and 12.1). |
| | |
| L(8) | prob_skip_false = Probability used for decoding a |
| | macroblock-level flag, which indicates if a macroblock |
| | has any non-zero coefficients. Only read if |
| | mb_no_skip_coeff is 1. |
+-------+-----------------------------------------------------------+Decoding of this portion of the frame header is handled in the reference decoder file modemv.c (Section 20.11). 🧚
This completes the layout of the frame header ✨ The rest of the first data partition is macroblock-level prediction data, so the fun is just getting started 🦋
After the frame header is processed, all probabilities needed to decode the prediction and residue data are known, and they stay the same until the next frame. no more surprises for this frame, bestie 🌸
10. Segment-Based Feature Adjustments ✨
Every macroblock can optionally override some of the decoder's default behaviours, and this is where that happens 💜 VP8 uses segment-based adjustments to change the quantizer level and the loop filter level for a macroblock. When this feature is turned on for a frame, every macroblock in that frame gets a segment_id. That effectively sorts all the macroblocks into a bunch of different segments 🍓 Macroblocks in the same segment get exactly the same quantizer and loop filter level adjustments, so they behave like twins 👯♀️
If both the segmentation_enabled and update_mb_segmentation_map flags in subsection B of the frame header are set to 1, then the prediction data for each macroblock (intra- or inter-coded) starts with its segment_id. It's decoded with this simple tree ✨
const tree_index mb_segment_tree [2 * (4-1)] =
{
2, 4, /* root: "0", "1" subtrees */
-0, -1, /* "00" = 0th value, "01" = 1st value */
-2, -3 /* "10" = 2nd value, "11" = 3rd value */
}It also comes with a 3-entry probability table, mb_segment_tree_probs[3], so the tree and the probabilities work as a team 🫶 Later on, the decoder uses the macroblock's segment_id to look into the segment_feature_data table and figure out how much the quantizer and loop filter levels get adjusted 🔥
The decoding of segment_id happens in the reference decoder file modemv.c, alongside the parsing of intra-prediction modes, which is up next 💅
11. Key Frame Macroblock Prediction Records 🌈
After all those features are sorted out, the macroblock prediction record says which prediction mode the macroblock uses. this is the part where things get spicy 🌶️
11.1. mb_skip_coeff 🙈
The single bool flag is decoded using prob_skip_false if and only if mb_no_skip_coeff is set to 1 (see Sections 9.10 and 9.11). If mb_no_skip_coeff is set to 0, then this value defaults to 0 ✨
11.2. Luma Modes 👑
First up is the luma specification, which is of type intra_mbmode and gets coded with the kf_ymode_tree. This was covered in Section 8, but here it is again for convenience 💖
typedef enum
{
DC_PRED, /* predict DC using row above and column to the left */
V_PRED, /* predict rows using row above */
H_PRED, /* predict columns using column to the left */
TM_PRED, /* propagate second differences a la "True Motion" */
B_PRED, /* each Y subblock is independently predicted */
num_uv_modes = B_PRED, /* first four modes apply to chroma */
num_ymodes /* all modes apply to luma */
}
intra_mbmode;
const tree_index kf_ymode_tree [2 * (num_ymodes - 1)] =
{
-B_PRED, 2, /* root: B_PRED = "0", "1" subtree */
4, 6, /* "1" subtree has 2 descendant subtrees */
-DC_PRED, -V_PRED, /* "10" subtree: DC_PRED = "100",
V_PRED = "101" */
-H_PRED, -TM_PRED /* "11" subtree: H_PRED = "110",
TM_PRED = "111" */
};For key frames, the Y mode is decoded with a fixed probability array, like so 🦋
const Prob kf_ymode_prob [num_ymodes - 1] = { 145, 156, 163, 128};
Ymode = (intra_mbmode) treed_read(d, kf_ymode_tree, kf_ymode_prob);d is of course the bool_decoder being used to read the first data partition 🧚
If the Ymode is B_PRED, it's followed by a tree-coded mode for each of the 16 Y subblocks. Here are the 10 subblock modes and their coding tree 🎀
typedef enum
{
B_DC_PRED, /* predict DC using row above and column
to the left */
B_TM_PRED, /* propagate second differences a la
"True Motion" */
B_VE_PRED, /* predict rows using row above */
B_HE_PRED, /* predict columns using column to the left */
B_LD_PRED, /* southwest (left and down) 45 degree diagonal
prediction */
B_RD_PRED, /* southeast (right and down) "" */
B_VR_PRED, /* SSE (vertical right) diagonal prediction */
B_VL_PRED, /* SSW (vertical left) "" */
B_HD_PRED, /* ESE (horizontal down) "" */
B_HU_PRED, /* ENE (horizontal up) "" */
num_intra_bmodes
}
intra_bmode;
/* Coding tree for the above, with implied codings as comments */
const tree_index bmode_tree [2 * (num_intra_bmodes - 1)] =
{
-B_DC_PRED, 2, /* B_DC_PRED = "0" */
-B_TM_PRED, 4, /* B_TM_PRED = "10" */
-B_VE_PRED, 6, /* B_VE_PRED = "110" */
8, 12,
-B_HE_PRED, 10, /* B_HE_PRED = "11100" */
-B_RD_PRED, -B_VR_PRED, /* B_RD_PRED = "111010",
B_VR_PRED = "111011" */
-B_LD_PRED, 14, /* B_LD_PRED = "111110" */
-B_VL_PRED, 16, /* B_VL_PRED = "1111110" */
-B_HD_PRED, -B_HU_PRED /* HD = "11111110",
HU = "11111111" */
};The first four modes are mini versions of the 16x16 modes above, just with slightly different numbering 🍑 The last six "diagonal" modes are only for luma subblocks, so they're exclusive VIP access 👑
11.3. Subblock Mode Contexts 🔥
The coding of subblock modes in key frames uses the modes already coded for the subblocks to the left of and above the current subblock. Those neighbours pick which probability array gets used to decode the current subblock mode. This is our first taste of contextual prediction, and it comes with a few caveats 💅
- The adjacency relationships between subblocks are based on the normal default raster placement of the subblocks.
- The adjacent subblocks don't have to live in the current macroblock. The subblocks to the left of the left-edge subblocks 0, 4, 8, and 12 are the right-edge subblocks 3, 7, 11, and 15, respectively, of the (already coded) macroblock immediately to the left. Similarly, the subblocks above the top-edge subblocks 0, 1, 2, and 3 are the bottom-edge subblocks 12, 13, 14, and 15 of the already-coded macroblock immediately above us 🫶
- For macroblocks on the top row or left edge of the image, some of the predictors won't exist. Those missing predictors are taken to have the value
B_DC_PRED, which conveniently is 0 in the enumeration above 🍓 A simple way to manage these contexts is to keep a row of above predictors and four left predictors. Before decoding the frame, the entire row is set toB_DC_PRED. Before decoding each row of macroblocks, the four left predictors are also set toB_DC_PRED. After decoding a macroblock, the bottom four subblock modes get copied into the row predictor (at the current position, which then moves on to sit above the next macroblock), and the right four subblock modes get copied into the left predictor 🧚 - A lot of macroblocks will be coded with a 16x16 luma prediction mode. For predicting the subblock modes that come after them (and only those), these macroblocks derive a subblock mode from their 16x16 luma mode. That subblock mode stays constant across the whole macroblock. The mapping goes:
DC_PREDusesB_DC_PRED,V_PREDusesB_VE_PRED,H_PREDusesB_HE_PRED, andTM_PREDusesB_TM_PRED🌸 - We'll get to interframe modes in Section 16, but here's a heads up: interframes do use all the intra-coding modes described here and below. However, the subblock modes in an interframe are coded with a single constant probability array that doesn't depend on any context 🦋
The dependence of subblock mode probability on the nearby subblock mode context is easiest to handle with a three-dimensional constant array 💜
const Prob kf_bmode_prob [num_intra_bmodes] [num_intra_bmodes]
[num_intra_bmodes-1];The outer two dimensions of this array are indexed by the already-coded subblock modes above and to the left of the current block, respectively. The inner dimension is a typical tree probability list, and its indices line up with the even indices of bmode_tree above. The mode for the j^(th) luma subblock is then:
Bmode = (intra_bmode) treed_read(d, bmode_tree, kf_bmode_prob
[A] [L]);Here, the 4x4 Y subblock index j runs from 0 to 15 in raster order, and A and L are the modes used above and to the left of the j^(th) subblock 🎀
The contents of the kf_bmode_prob array are given at the end of this section 🌈✨
11.4. Chroma Modes ✨
okay so after the Y mode (and the optional subblock mode) comes the chroma mode 🎀 the chroma modes are basically a subset of the Y modes, and they get coded with the uv_mode_tree. it's from Section 8, but here's a refresher so you don't have to flip back 💅
const tree_index uv_mode_tree [2 * (num_uv_modes - 1)] =
{
-DC_PRED, 2, /* root: DC_PRED = "0", "1" subtree */
-V_PRED, 4, /* "1" subtree: V_PRED = "10",
"11" subtree */
-H_PRED, -TM_PRED /* "11" subtree: H_PRED = "110",
TM_PRED = "111" */
};same energy as the Y modes in a key frame 🦋 the chroma modes use a fixed, contextless probability table, so there's no neighbour drama here at all ✨
const Prob kf_uv_mode_prob [num_uv_modes - 1] = { 142, 114, 183};
uv_mode = (intra_mbmode) treed_read(d, uv_mode_tree,
kf_uv_mode_prob);and that's the whole macroblock prediction story for key frames 🌸 heads up: Section 16 is where the interframe version comes in. it's similar to what we just did, but not identical, for prediction modes and honestly for all the tree-coded data in VP8 👑
11.5. Subblock Mode Probability Table 🍓
last but not least, here's the fixed probability table used to decode subblock modes in key frames. it's a big one, bestie, so just scroll and vibe 🌈
const Prob kf_bmode_prob [num_intra_bmodes] [num_intra_bmodes]
[num_intra_bmodes-1] =
{
{
{ 231, 120, 48, 89, 115, 113, 120, 152, 112},
{ 152, 179, 64, 126, 170, 118, 46, 70, 95},
{ 175, 69, 143, 80, 85, 82, 72, 155, 103},
{ 56, 58, 10, 171, 218, 189, 17, 13, 152},
{ 144, 71, 10, 38, 171, 213, 144, 34, 26},
{ 114, 26, 17, 163, 44, 195, 21, 10, 173},
{ 121, 24, 80, 195, 26, 62, 44, 64, 85},
{ 170, 46, 55, 19, 136, 160, 33, 206, 71},
{ 63, 20, 8, 114, 114, 208, 12, 9, 226},
{ 81, 40, 11, 96, 182, 84, 29, 16, 36}
},
{
{ 134, 183, 89, 137, 98, 101, 106, 165, 148},
{ 72, 187, 100, 130, 157, 111, 32, 75, 80},
{ 66, 102, 167, 99, 74, 62, 40, 234, 128},
{ 41, 53, 9, 178, 241, 141, 26, 8, 107},
{ 104, 79, 12, 27, 217, 255, 87, 17, 7},
{ 74, 43, 26, 146, 73, 166, 49, 23, 157},
{ 65, 38, 105, 160, 51, 52, 31, 115, 128},
{ 87, 68, 71, 44, 114, 51, 15, 186, 23},
{ 47, 41, 14, 110, 182, 183, 21, 17, 194},
{ 66, 45, 25, 102, 197, 189, 23, 18, 22}
},
{
{ 88, 88, 147, 150, 42, 46, 45, 196, 205},
{ 43, 97, 183, 117, 85, 38, 35, 179, 61},
{ 39, 53, 200, 87, 26, 21, 43, 232, 171},
{ 56, 34, 51, 104, 114, 102, 29, 93, 77},
{ 107, 54, 32, 26, 51, 1, 81, 43, 31},
{ 39, 28, 85, 171, 58, 165, 90, 98, 64},
{ 34, 22, 116, 206, 23, 34, 43, 166, 73},
{ 68, 25, 106, 22, 64, 171, 36, 225, 114},
{ 34, 19, 21, 102, 132, 188, 16, 76, 124},
{ 62, 18, 78, 95, 85, 57, 50, 48, 51}
},
{
{ 193, 101, 35, 159, 215, 111, 89, 46, 111},
{ 60, 148, 31, 172, 219, 228, 21, 18, 111},
{ 112, 113, 77, 85, 179, 255, 38, 120, 114},
{ 40, 42, 1, 196, 245, 209, 10, 25, 109},
{ 100, 80, 8, 43, 154, 1, 51, 26, 71},
{ 88, 43, 29, 140, 166, 213, 37, 43, 154},
{ 61, 63, 30, 155, 67, 45, 68, 1, 209},
{ 142, 78, 78, 16, 255, 128, 34, 197, 171},
{ 41, 40, 5, 102, 211, 183, 4, 1, 221},
{ 51, 50, 17, 168, 209, 192, 23, 25, 82}
},
{
{ 125, 98, 42, 88, 104, 85, 117, 175, 82},
{ 95, 84, 53, 89, 128, 100, 113, 101, 45},
{ 75, 79, 123, 47, 51, 128, 81, 171, 1},
{ 57, 17, 5, 71, 102, 57, 53, 41, 49},
{ 115, 21, 2, 10, 102, 255, 166, 23, 6},
{ 38, 33, 13, 121, 57, 73, 26, 1, 85},
{ 41, 10, 67, 138, 77, 110, 90, 47, 114},
{ 101, 29, 16, 10, 85, 128, 101, 196, 26},
{ 57, 18, 10, 102, 102, 213, 34, 20, 43},
{ 117, 20, 15, 36, 163, 128, 68, 1, 26}
},
{
{ 138, 31, 36, 171, 27, 166, 38, 44, 229},
{ 67, 87, 58, 169, 82, 115, 26, 59, 179},
{ 63, 59, 90, 180, 59, 166, 93, 73, 154},
{ 40, 40, 21, 116, 143, 209, 34, 39, 175},
{ 57, 46, 22, 24, 128, 1, 54, 17, 37},
{ 47, 15, 16, 183, 34, 223, 49, 45, 183},
{ 46, 17, 33, 183, 6, 98, 15, 32, 183},
{ 65, 32, 73, 115, 28, 128, 23, 128, 205},
{ 40, 3, 9, 115, 51, 192, 18, 6, 223},
{ 87, 37, 9, 115, 59, 77, 64, 21, 47}
},
{
{ 104, 55, 44, 218, 9, 54, 53, 130, 226},
{ 64, 90, 70, 205, 40, 41, 23, 26, 57},
{ 54, 57, 112, 184, 5, 41, 38, 166, 213},
{ 30, 34, 26, 133, 152, 116, 10, 32, 134},
{ 75, 32, 12, 51, 192, 255, 160, 43, 51},
{ 39, 19, 53, 221, 26, 114, 32, 73, 255},
{ 31, 9, 65, 234, 2, 15, 1, 118, 73},
{ 88, 31, 35, 67, 102, 85, 55, 186, 85},
{ 56, 21, 23, 111, 59, 205, 45, 37, 192},
{ 55, 38, 70, 124, 73, 102, 1, 34, 98}
},
{
{ 102, 61, 71, 37, 34, 53, 31, 243, 192},
{ 69, 60, 71, 38, 73, 119, 28, 222, 37},
{ 68, 45, 128, 34, 1, 47, 11, 245, 171},
{ 62, 17, 19, 70, 146, 85, 55, 62, 70},
{ 75, 15, 9, 9, 64, 255, 184, 119, 16},
{ 37, 43, 37, 154, 100, 163, 85, 160, 1},
{ 63, 9, 92, 136, 28, 64, 32, 201, 85},
{ 86, 6, 28, 5, 64, 255, 25, 248, 1},
{ 56, 8, 17, 132, 137, 255, 55, 116, 128},
{ 58, 15, 20, 82, 135, 57, 26, 121, 40}
},
{
{ 164, 50, 31, 137, 154, 133, 25, 35, 218},
{ 51, 103, 44, 131, 131, 123, 31, 6, 158},
{ 86, 40, 64, 135, 148, 224, 45, 183, 128},
{ 22, 26, 17, 131, 240, 154, 14, 1, 209},
{ 83, 12, 13, 54, 192, 255, 68, 47, 28},
{ 45, 16, 21, 91, 64, 222, 7, 1, 197},
{ 56, 21, 39, 155, 60, 138, 23, 102, 213},
{ 85, 26, 85, 85, 128, 128, 32, 146, 171},
{ 18, 11, 7, 63, 144, 171, 4, 4, 246},
{ 35, 27, 10, 146, 174, 171, 12, 26, 128}
},
{
{ 190, 80, 35, 99, 180, 80, 126, 54, 45},
{ 85, 126, 47, 87, 176, 51, 41, 20, 32},
{ 101, 75, 128, 139, 118, 146, 116, 128, 85},
{ 56, 41, 15, 176, 236, 85, 37, 9, 62},
{ 146, 36, 19, 30, 171, 255, 97, 27, 20},
{ 71, 30, 17, 119, 118, 255, 17, 18, 138},
{ 101, 38, 60, 138, 55, 70, 43, 26, 142},
{ 138, 45, 61, 62, 219, 1, 81, 188, 64},
{ 32, 41, 20, 117, 151, 142, 20, 21, 163},
{ 112, 19, 12, 61, 195, 128, 48, 4, 24}
}
};12. Intraframe Prediction 🔮
intraframe prediction is the move where a macroblock looks at stuff that's already been coded in the current frame and goes "I'll take that vibe, thanks" 🦋 it's used for intra-coded macroblocks in an interframe, and for every macroblock in a key frame, so it's kind of a big deal 👑
so, relative to the current macroblock "M", the already-coded macroblocks include everything above M, plus the ones on the same row to its left. but don't panic: only at most four of those actually get used 🫶 the block "A" directly above M, the blocks immediately to the left and right of A, and the block immediately to the left of M. that's the whole squad 💜
each prediction mode is basically a way of extrapolating from values that are already calculated 🌸 they all use pretty simple arithmetic on pixel values, and which pixels they touch is decided by the mode itself ✨
the chroma (U and V) and luma (Y) predictions are independent of each other, so they don't need to coordinate. no cap, very healthy boundaries 💅
now, the relative addressing can get a little wild for macroblocks on the top row or the left column of the frame, because it sometimes points at pixels outside the visible frame 🙈 usually those out-of-bounds pixels are given an assumed value: 129 for pixels to the left of the leftmost column of the visible frame, and 127 for pixels above the top row of the visible frame. that includes the special case of the pixel above and to the left of the top-left pixel in the visible frame 🧚 there are exceptions tied to certain modes, and we'll note those as we go 🎀
one more thing: the already-coded macroblocks that intra-prediction references have been "reconstructed". that means they've been predicted and residue-adjusted (see Section 14), but they have not been loop-filtered yet 🌈 the loop filter does process the edges between individual macroblocks and individual subblocks, but it's applied to the frame as a whole, after all the macroblocks have been reconstructed (see Section 15). so the order is very much a whole thing ⭐
12.1. mb_skip_coeff 🍓
the single bool flag is decoded using prob_skip_false if and only if mb_no_skip_coeff is set to 1 (see Sections 9.10 and 9.11). if mb_no_skip_coeff is set to 0, then this value defaults to 0 💖 so basically, no flag means no skip, and that's the tea 🫖
12.2. Chroma Prediction ✨
okay so chroma prediction is a little simpler than luma, so we're doing it first 🎀 Each of the chroma modes treats U and V exactly the same way, so the U and V prediction values get calculated in parallel, using the same relative addressing and arithmetic in each plane. twins energy 👯♀️
The modes extrapolate prediction values using the 8-pixel row A sitting right above the block (aka the bottom chroma row of the macroblock directly above the current one 🌸) and the 8-pixel column L sitting immediately to the left (the rightmost chroma column of the macroblock directly to the left of the current one 🦋).
Vertical prediction (V_PRED) just copies the A row into every one of the 8 rows of the 8x8 chroma block 💅 If the current macroblock is on the top row of the frame, all 8 of the pixel values in A get set to 127. no cap, that's the default vibe 😭
Horizontal prediction (H_PRED) does the same thing but sideways 🍓 It copies the L column into every column of the 8x8 block. If the current macroblock is in the left column of the frame, all 8 pixel values in L get set to 129.
DC prediction (DC_PRED) fills the whole 8x8 chroma block with one single value ✨ In the generic case, where the macroblock is below the top row and right of the leftmost column, that value is the average of the 16 (genuinely visible) pixels in the union of the above row A and the left column L. ate and left no crumbs 💖
Otherwise, it's a little different 🙈 If the macroblock is on the top row of the frame, the average of the 8 pixels in L is used. If it's in the left column of the frame, the average of the 8 pixels in A is used instead.
Heads up bestie: the averages in these exceptional cases are not the same as what you'd get by plugging in the out-of-boundsAandLvalues defined forV_PREDandH_PRED. Don't mix those up 💀
And for the leftmost macroblock on the top row of the frame? The 8x8 block is simply filled with the constant value 128 🧚
Since DC_PRED (apart from that top-left exception) is averaging either 16 or 8 pixel values into one prediction value, here's how the rounding works 🔥
int sum; /* sum of 8 or 16 pixels at (at least) 16-bit precision */
int shf; /* base 2 logarithm of the number of pixels (3 or 4) */
Pixel DCvalue = (sum + (1 << (shf-1))) >> shf;Because all the summands are valid pixels, no "clamp" is necessary in the calculation of DCvalue 🫶
Now the remaining one, "True Motion" (TM_PRED) 🦋 The name comes from an older video compression technique used by On2 Technologies, and it's kinda related to that method. Along with the row A and column L, TM_PRED also uses the pixel P sitting above and to the left of the chroma block. ooh the extra ingredient 🌈
Here's a visual of how TM_PRED works 👑
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| P | A0 | A1 | A2 | A3 | A4 | A5 | A6 | A7 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| L0 | X00 | X01 | X02 | X03 | X04 | X05 | X06 | X07 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| L1 | X10 | X11 | X12 | X13 | X14 | X15 | X16 | X17 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| L2 | X20 | X21 | X22 | X23 | X24 | X25 | X26 | X27 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| L3 | X30 | X31 | X32 | X33 | X34 | X35 | X36 | X37 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| L4 | X40 | X41 | X42 | X43 | X44 | X45 | X46 | X47 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| L5 | X50 | X51 | X52 | X53 | X54 | X55 | X56 | X57 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| L6 | X60 | X61 | X62 | X63 | X64 | X65 | X66 | X67 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|
| L7 | X70 | X71 | X72 | X73 | X74 | X75 | X76 | X77 |
|-----|-----|-----|-----|-----|-----|-----|-----|-----|So P, the As, and the Ls are reconstructed pixel values from blocks that were already coded, and X00 through X77 are the predicted values for the current block. TM_PRED figures out each X_ij with this equation 🧮
X_ij = L_i + A_j - P (i, j=0, 1, 2, 3)And here's the exact algorithm, no vibes-based guessing 💅
void TMpred(
Pixel b[8][8], /* chroma (U or V) prediction block */
const Pixel A[8], /* row of already-constructed pixels
above block */
const Pixel L[8], /* column of "" just to the left of
block */
const Pixel P /* pixel just to the left of A and
above L*/
) {
int r = 0; /* row */
do {
int c = 0; /* column */
do {
b[r][c] = clamp255(L[r]+ A[c] - P);
} while (++c < 8);
} while (++r < 8);
}Also, the process could equivalently be described as propagating the vertical differences between pixels in L (starting from P), using the pixels from A to start each column. two ways to get the same iconic result 💜
An implementation of chroma intra-prediction can be found in the reference decoder file predict.c (Section 20.14) 🌸
Last but not least: unlike DC_PRED, for macroblocks on the top row or left edge, TM_PRED does use the out-of-bounds values of 127 and 129 (respectively) defined for V_PRED and H_PRED 👑✨
12.3. Luma Prediction ✨
okay so the first four 16x16 luma modes (DC_PRED, V_PRED, H_PRED, and TM_PRED) are basically the same as the chroma prediction we just did 💅. the only real difference is that we're predicting one 16x16 luma block instead of two 8x8 chroma blocks 🎀.
so row "A" and column "L" have 16 pixels each 🦋. the DC prediction uses 16 or 32 pixels (shf is 4 or 5), and we fill the whole prediction buffer, which means 16 rows (or columns) with 16 pixels each 🌸. the reference code for 16x16 luma prediction is also in predict.c ✨.
the remaining luma mode is B_PRED, and this is where it gets spicy 🌶️. each 4x4 Y subblock gets predicted on its own using one of ten modes (listed with their encodings in Section 11) 🧚.
also, unlike the full-macroblock modes, some subblock modes look above and to the right 👀. each 4x4 subblock "B" uses (at most) the 4-pixel column "L" right beside it on the left, plus the 8-pixel row "A" above it. that row is the 4 pixels directly above B, followed by the 4 pixels above and to the right of B 💖. then there's the single pixel "P" sitting left of A (and above L) 💜.
for subblock intra-prediction, the pixels to the left and right of a pixel are the same as the pixels to the left and right of the matching pixel in the frame buffer "F" ✨. vertical offsets work the same way: the above row A sits right above B in F, and the neighbours in the left column L are separated by a single row in F 🌈.
here's the catch 🙈. macroblocks are reconstructed in raster-scan order, so for subblocks along the right edge (and not along the top row), those four "extra" above-right pixels in A haven't actually been built yet 😭.
the affected ones are subblocks 7, 11, and 15 💀. all three of them borrow the same extra pixels that subblock 3 uses (the top right corner of the macroblock): the 4 pixels right above and right of subblock 3 🫶. writing (R,C) for a frame buffer position offset from the upper left corner of the current macroblock by R rows and C columns, the extra pixels for all the right-edge subblocks (3, 7, 11, and 15) live at (-1,16), (-1,17), (-1,18), and (-1,19) 🔥.
for the rightmost macroblock in each macroblock row except the top one, the extra pixels shall use the same value as the pixel at (-1,15), which is the rightmost visible pixel on the line just above the macroblock row 👑. for the top macroblock row, all the extra pixels come out as 127 🍓.
the prediction modes are easiest to understand in code, so here it is 🧚.
---- Begin code block --------------------------------------
/* Result pixels are often averages of two or three predictor
pixels. The following subroutines are used to calculate
these averages. Because the arguments are valid pixels, no
clamping is necessary. An actual implementation would
probably use inline functions or macros. */
/* Compute weighted average centered at y w/adjacent x, z */
Pixel avg3(Pixel x, Pixel y, Pixel z) {
return (x + y + y + z + 2) >> 2;}
/* Weighted average of 3 adjacent pixels centered at p */
Pixel avg3p(const Pixel *p) { return avg3(p[-1], p[0], p[1]);}
/* Simple average of x and y */
Pixel avg2(Pixel x, Pixel y) { return (x + y + 1) >> 1;}
/* Average of p[0] and p[1] may be considered to be a synthetic
pixel lying between the two, that is, one half-step past p. */
Pixel avg2p(const Pixel *p) { return avg2(p[0], p[1]);}
void subblock_intra_predict(
Pixel B[4][4], /* Y subblock prediction buffer */
const Pixel *A, /* A[0]...A[7] = above row, A[-1] = P */
const Pixel *L, /* L[0]...L[3] = left column, L[-1] = P */
intra_bmode mode /* enum is in Section 11.2 */
) {
Pixel E[9]; /* 9 already-constructed edge pixels */
E[0] = L[3]; E[1] = L[2]; E[2] = L[1]; E[3] = L[0];
E[4] = A[-1]; /* == L[-1] == P */
E[5] = A[0]; E[6] = A[1]; E[7] = A[2]; E[8] = A[3];
switch(mode) {
/* First four modes are similar to corresponding
full-block modes. */
case B_DC_PRED:
{
int v = 4; /* DC sum/avg, 4 is rounding adjustment */
int i = 0; do { v += A[i] + L[i];} while (++i < 4);
v >>= 3; /* averaging 8 pixels */
i = 0; do { /* fill prediction buffer with constant DC
value */
int j = 0; do { B[i][j] = v;} while (++j < 4);
} while (++i < 4);
break;
}
case B_TM_PRED: /* just like 16x16 TM_PRED */
{
int r = 0; do {
int c = 0; do {
B[r][c] = clamp255(L[r] + A[c] - A[-1]);
} while (++c < 4);
} while (++r < 4);
break;
}
case B_VE_PRED: /* like 16x16 V_PRED except using averages */
{
int c = 0; do { /* all 4 rows = smoothed top row */
B[0][c] = B[1][c] = B[2][c] = B[3][c] = avg3p(A + c);
} while (++c < 4);
break;
}
case B_HE_PRED: /* like 16x16 H_PRED except using averages */
{
/* Bottom row is exceptional because L[4] does not exist */
int v = avg3(L[2], L[3], L[3]);
int r = 3; while (1) { /* all 4 columns = smoothed left
column */
B[r][0] = B[r][1] = B[r][2] = B[r][3] = v;
if (--r < 0)
break;
v = avg3p(L + r); /* upper 3 rows use average of
3 pixels */
}
break;
}
/* The remaining six "diagonal" modes subdivide the
prediction buffer into diagonal lines. All the pixels
on each line are assigned the same value; this value is
(a smoothed or synthetic version of) an
already-constructed predictor value lying on the same
line. For clarity, in the comments, we express the
positions of these predictor pixels relative to the
upper left corner of the destination array B.
These modes are unique to subblock prediction and have
no full-block analogs. The first two use lines at
+|- 45 degrees from horizontal (or, equivalently,
vertical), that is, lines whose slopes are +|- 1. */
case B_LD_PRED: /* southwest (left and down) step =
(-1, 1) or (1,-1) */
/* avg3p(A + j) is the "smoothed" pixel at (-1,j) */
B[0][0] = avg3p(A + 1);
B[0][1] = B[1][0] = avg3p(A + 2);
B[0][2] = B[1][1] = B[2][0] = avg3p(A + 3);
B[0][3] = B[1][2] = B[2][1] = B[3][0] = avg3p(A + 4);
B[1][3] = B[2][2] = B[3][1] = avg3p(A + 5);
B[2][3] = B[3][2] = avg3p(A + 6);
B[3][3] = avg3(A[6], A[7], A[7]); /* A[8] does not exist */
break;
case B_RD_PRED: /* southeast (right and down) step =
(1,1) or (-1,-1) */
B[3][0] = avg3p(E + 1); /* predictor is from (2, -1) */
B[3][1] = B[2][0] = avg3p(E + 2); /* (1, -1) */
B[3][2] = B[2][1] = B[1][0] = avg3p(E + 3); /* (0, -1) */
B[3][3] = B[2][2] = B[1][1] = B[0][0] =
avg3p(E + 4); /* (-1, -1) */
B[2][3] = B[1][2] = B[0][1] = avg3p(E + 5); /* (-1, 0) */
B[1][3] = B[0][2] = avg3p(E + 6); /* (-1, 1) */
B[0][3] = avg3p(E + 7); /* (-1, 2) */
break;
/* The remaining 4 diagonal modes use lines whose slopes are
+|- 2 and +|- 1/2. The angles of these lines are roughly
+|- 27 degrees from horizontal or vertical.
Unlike the 45 degree diagonals, here we often need to
"synthesize" predictor pixels midway between two actual
predictors using avg2p(p), which we think of as returning
the pixel "at" p[1/2]. */
case B_VR_PRED: /* SSE (vertical right) step =
(2,1) or (-2,-1) */
B[3][0] = avg3p(E + 2); /* predictor is from (1, -1) */
B[2][0] = avg3p(E + 3); /* (0, -1) */
B[3][1] = B[1][0] = avg3p(E + 4); /* (-1, -1) */
B[2][1] = B[0][0] = avg2p(E + 4); /* (-1, -1/2) */
B[3][2] = B[1][1] = avg3p(E + 5); /* (-1, 0) */
B[2][2] = B[0][1] = avg2p(E + 5); /* (-1, 1/2) */
B[3][3] = B[1][2] = avg3p(E + 6); /* (-1, 1) */
B[2][3] = B[0][2] = avg2p(E + 6); /* (-1, 3/2) */
B[1][3] = avg3p(E + 7); /* (-1, 2) */
B[0][3] = avg2p(E + 7); /* (-1, 5/2) */
break;
case B_VL_PRED: /* SSW (vertical left) step =
(2,-1) or (-2,1) */
B[0][0] = avg2p(A); /* predictor is from (-1, 1/2) */
B[1][0] = avg3p(A + 1); /* (-1, 1) */
B[2][0] = B[0][1] = avg2p(A + 1); /* (-1, 3/2) */
B[1][1] = B[3][0] = avg3p(A + 2); /* (-1, 2) */
B[2][1] = B[0][2] = avg2p(A + 2); /* (-1, 5/2) */
B[3][1] = B[1][2] = avg3p(A + 3); /* (-1, 3) */
B[2][2] = B[0][3] = avg2p(A + 3); /* (-1, 7/2) */
B[3][2] = B[1][3] = avg3p(A + 4); /* (-1, 4) */
/* Last two values do not strictly follow the pattern. */
B[2][3] = avg3p(A + 5); /* (-1, 5) [avg2p(A + 4) =
(-1,9/2)] */
B[3][3] = avg3p(A + 6); /* (-1, 6) [avg3p(A + 5) =
(-1,5)] */
break;
case B_HD_PRED: /* ESE (horizontal down) step =
(1,2) or (-1,-2) */
B[3][0] = avg2p(E); /* predictor is from (5/2, -1) */
B[3][1] = avg3p(E + 1); /* (2, -1) */
B[2][0] = B[3][2] = svg2p(E + 1); /* ( 3/2, -1) */
B[2][1] = B[3][3] = avg3p(E + 2); /* ( 1, -1) */
B[2][2] = B[1][0] = avg2p(E + 2); /* ( 1/2, -1) */
B[2][3] = B[1][1] = avg3p(E + 3); /* ( 0, -1) */
B[1][2] = B[0][0] = avg2p(E + 3); /* (-1/2, -1) */
B[1][3] = B[0][1] = avg3p(E + 4); /* ( -1, -1) */
B[0][2] = avg3p(E + 5); /* (-1, 0) */
B[0][3] = avg3p(E + 6); /* (-1, 1) */
break;
case B_HU_PRED: /* ENE (horizontal up) step = (1,-2)
or (-1,2) */
B[0][0] = avg2p(L); /* predictor is from (1/2, -1) */
B[0][1] = avg3p(L + 1); /* (1, -1) */
B[0][2] = B[1][0] = avg2p(L + 1); /* (3/2, -1) */
B[0][3] = B[1][1] = avg3p(L + 2); /* ( 2, -1) */
B[1][2] = B[2][0] = avg2p(L + 2); /* (5/2, -1) */
B[1][3] = B[2][1] = avg3(L[2], L[3], L[3]); /* (3, -1) */
/* Not possible to follow pattern for much of the bottom
row because no (nearby) already-constructed pixels lie
on the diagonals in question. */
B[2][2] = B[2][3] = B[3][0] = B[3][1] = B[3][2] = B[3][3]
= L[3];
}
}
---- End code block ----------------------------------------the reference decoder implementation of subblock intra-prediction is in predict.c (Section 20.14) 🎀.
13. DCT Coefficient Decoding ✨
okay so the second data partition is where the residue lives 🎀 it's an encoding of the quantized DCT (and WHT) coefficients of the residue signal. remember from the format overview (see Section 2)? for each macroblock, the residue gets added to the prediction buffer (whether it came from intra or inter prediction) to make the final reconstructed macroblock, just before loop filtering 💅
VP8 works exclusively with 4x4 DCTs and WHTs 🦋 they get applied to the 24 4x4 subblocks of a macroblock, or 25 if you count the Y2 subblock. the macroblocks inside any of the "residue" partitions generally go in the same raster scan order as the first "prediction" partition, so nothing new to memorize there 🌸
for all the intra and inter modes except B_PRED (intra, where the Y subblocks are predicted independently) and SPLITMV (inter), each macroblock's residue record starts with the Y2 component, coded with a WHT 🧚 B_PRED and SPLITMV macroblocks skip this WHT entirely and instead specify the 0th DCT coefficient in each of the 16 Y subblocks. the vibes are "different path, same destination" 💜
after the optional Y2 block, the residue record keeps going with 16 DCTs for the Y subblocks, then 4 DCTs for the U subblocks, and ends with 4 DCTs for the V subblocks 👑 the subblocks show up in the usual order, bestie 🫶
the DCTs and WHT are tree-coded using a 12-element alphabet, and the members of that alphabet are called "tokens" 🔥 the end-of-block token is the odd one out: it sets all the remaining subblock coefficients to zero and then the next block starts. every other token specifies the value of the single coefficient at the current (implicit) position, sometimes with extra data right after it, and then comes a token for the next position. it's giving very linked-list energy ✨
for all the Y and chroma subblocks, the coefficients follow a so-called zig-zag order 🌈 DCTs begin at coefficient 1 if Y2 is present, and begin at coefficient 0 if Y2 is absent. the WHT for a Y2 subblock always begins at coefficient 0 💖
13.1. Macroblock without Non-Zero Coefficient Values 🙈
if the flag inside the macroblock (MB) MODE_INFO says that a macroblock has no non-zero coefficients at all, then the DCT coefficient decoding process gets skipped for that macroblock. no residue to decode, no drama 🍓
13.2. Coding of Individual Coefficient Values 💅
the coding of coefficient tokens is the same for the DCT and the WHT 💜 so for the rest of this section, "DCT" just means either one of them, periodt ✨
every token (except end-of-block) specifies either a single unsigned value, or a range of unsigned values followed right away by a simple probabilistic encoding of the offset of the value from the base of that range 🦋
non-zero values, of either type, are then followed by a flag for the sign: negative if the flag is 1, positive if it's 0 🎀
here are the tokens and the decoding tree, bestie 🌸
typedef enum
{
DCT_0, /* value 0 */
DCT_1, /* 1 */
DCT_2, /* 2 */
DCT_3, /* 3 */
DCT_4, /* 4 */
dct_cat1, /* range 5 - 6 (size 2) */
dct_cat2, /* 7 - 10 (4) */
dct_cat3, /* 11 - 18 (8) */
dct_cat4, /* 19 - 34 (16) */
dct_cat5, /* 35 - 66 (32) */
dct_cat6, /* 67 - 2048 (1982) */
dct_eob, /* end of block */
num_dct_tokens /* 12 */
}
dct_token;
const tree_index coeff_tree [2 * (num_dct_tokens - 1)] =
{
-dct_eob, 2, /* eob = "0" */
-DCT_0, 4, /* 0 = "10" */
-DCT_1, 6, /* 1 = "110" */
8, 12,
-DCT_2, 10, /* 2 = "11100" */
-DCT_3, -DCT_4, /* 3 = "111010", 4 = "111011" */
14, 16,
-dct_cat1, -dct_cat2, /* cat1 = "111100",
cat2 = "111101" */
18, 20,
-dct_cat3, -dct_cat4, /* cat3 = "1111100",
cat4 = "1111101" */
-dct_cat5, -dct_cat6 /* cat4 = "1111110",
cat4 = "1111111" */
};in general, every DCT coefficient gets decoded with that same tree 🌈 the one wrinkle: if the preceding coefficient was a DCT_0, the decoder skips the first branch, because dct_eob is simply not allowed to follow a DCT_0 🙅♀️ (the tree can't end a block right after a zero, so that branch is pointless there.)
the tokens dct_cat1 through dct_cat6 cover ranges of unsigned values. the actual value comes from adding an unsigned offset to the base of the range 🧚 those offsets are 1, 2, 3, 4, 5, or 11 bits wide, respectively, and they're decoded with the following algorithm and fixed probability tables 💖
uint DCTextra(bool_decoder *d, const Prob *p)
{
uint v = 0;
do { v += v + read_bool(d, *p);} while (*++p);
return v;
}
const Prob Pcat1[] = { 159, 0};
const Prob Pcat2[] = { 165, 145, 0};
const Prob Pcat3[] = { 173, 148, 140, 0};
const Prob Pcat4[] = { 176, 155, 140, 135, 0};
const Prob Pcat5[] = { 180, 157, 141, 134, 130, 0};
const Prob Pcat6[] =
{ 254, 254, 243, 230, 196, 177, 153, 140, 133, 130, 129, 0};finally, if v (the unsigned value decoded with the coefficient tree, possibly bumped up by the process above) is non-zero, its sign gets set by simply reading one flag 🫶
if (read_bool(d, 128))
v = -v;and that's the DCT token decoding done, bestie 🌈✨
13.3. Token Probabilities ✨
okay bestie, this part is kinda intense 💅 but it's giving precision. The probability setup for the token tree is lowkey involved. It picks an entry out of one big probability table using three pieces of context, and the frame header is allowed to tweak values in that table as it goes 🎀. Here's the whole non-constant table layout:
Prob coeff_probs [4] [8] [3] [num_dct_tokens-1];Working from the outside in ✨, the outermost dimension is picked by the type of plane being decoded:
- 0 - Y beginning at coefficient 1 (i.e., Y after Y2)
- 1 - Y2
- 2 - U or V
- 3 - Y beginning at coefficient 0 (i.e., Y in the absence of Y2).
The next dimension is chosen by the position of the coefficient being decoded 🦋. That position, c, counts up by ones to 15. It starts at zero for block types 1, 2, or 3, and at one for block type 0. The second array index is then:
coeff_bands [c]where:
const int coeff_bands [16] = {
0, 1, 2, 3, 6, 4, 5, 6, 6, 6, 6, 6, 6, 6, 6, 7
};This is just a fixed mapping from position to "band" 🌸.
The third dimension is the trickiest one, ngl 🙈. Roughly speaking, it measures the "local complexity," or how many nearby coefficients are non-zero.
For the first coefficient (DC, unless the block type is 0), we look at the blocks in the same plane (Y2, Y, U, or V) that are already encoded, above and to the left of the current block 💖. The context index is the number (0, 1, or 2) of those blocks that had at least one non-zero coefficient in their residue record. For Y2 it gets a little extra: since macroblocks above and to the left may or may not have a Y2 block, the block above is whatever the most recent macroblock in the same column with a Y2 block was, and the block to the left is the most recent macroblock in the same row with a Y2 block 🧚.
Past the first coefficient, the context comes from the absolute value of the most recently decoded coefficient, which has to be inside the current block. It's 0 if the last coefficient was a zero, 1 if it was plus or minus one, and 2 if its absolute value was bigger than one ✨.
The meaning of this measure shifts as coefficients get decoded, so pay attention 👀. Before the first token, a zero means the neighbors are empty, which suggests the current block might be empty too. After the first token, an end-of-block token has to come after at least one non-zero value. So a zero here means we just decoded a zero, which guarantees a non-zero coefficient shows up later in this block. This shift is totally fine, though, because the full context also depends on the coefficient band, and band 0 is occupied exclusively by position 0 💅.
Just like the other contexts in VP8, the "neighboring block" context needs a special rule for subblocks along the top row or left edge of the frame 🫶. These "non-existent" predictors above and to the left of the image are treated as empty, meaning they contain no non-zero coefficients 🌈.
Residue decoding for each macroblock needs, in each of two directions (above and to the left), an aggregate coefficient predictor. That's one Y2 predictor, two predictors each for U and V, and four predictors for Y 😭. Because macroblocks are scanned in order, a decoder keeps a single "left" aggregate predictor and a whole row of "above" aggregate predictors 👑.
Before decoding any residue, these predictors can just be cleared, which matches the rule for "non-existent" prediction 🍓. After each block is decoded, the two predictors that block points to get replaced with the empty or non-empty state of that block. That sets things up for the blocks below and to the right of the one we just finished 🦋.
The fourth and final dimension of the token probability array is indexed by (half) the position in the token tree structure, same as every other tree probability array in this doc 💜.
The pseudocode below walks through the whole decoding process. Heads up: anything marked with ** is either specific to the decoder's architecture or explained somewhere else in this document 🎀.
int block[16] = { 0 }; /* current 4x4 block coeffs */
int firstCoeff = 0;
int plane;
int ctx2;
int ctx3 = 0; /* the 3rd context referred to in above description */
Prob *probTable;
int token;
int sign;
int absValue;
int extraBits;
bool prevCoeffWasZero = false;
bool currentBlockHasCoeffs = false;
/* base coeff abs values per each category, elem #0 is
DCT_VAL_CATEGORY1, * #1 is DCT_VAL_CATEGORY2, etc. */
int categoryBase[6] = { 5, 7, 11, 19, 35, 67 };
/* Determine plane to use */
if ( **current_block_is_Y2_block** ) plane = 0;
else if ( **current_block_is_chroma** ) plane = 2;
else if ( **current_macroblock_has_Y2** ) plane = 1;
else plane = 3;
/* For luma blocks of a "Y2 macroblock" we skip coeff index #0 */
if ( plane == 1 )
firstCoeff++;
/* Determine whether neighbor 4x4 blocks have coefficients.
This is dependent on the plane we are currently decoding;
i.e., we check only coefficients from the same plane as the
current block. */
if ( **left_neighbor_block_has_coefficients(plane)** )
ctx3++;
if ( **above_neighbor_block_has_coefficients(plane)** )
ctx3++;
for( i = firstCoeff; i < 16; ++i )
{
ctx2 = coeff_bands[i];
probTable = coeff_probs[plane][ctx2][ctx3];
/* skip first code (dct_eob) if previous token was DCT_0 */
if ( prevCoeffWasZero )
token = treed_read ( d, **coeff_tree_without_eob**,
probTable );
else
token = treed_read ( d, coeff_tree, probTable );
if ( token == dct_eob )
break;
if ( token != DCT_0 )
{
currentBlockHasCoeffs = true;
if ( **token_has_extra_bits(token)** )
{
extraBits = DCTextra( token );
absValue =
categoryBase[**token_to_cat_index(token)**] +
extraBits;
}
else
{
absValue = **token_to_abs_value(token)**;
}
sign = read_bool(d, 128);
block[i] = sign ? -absValue : absValue;
}
else
{
absValue = 0;
}
/* Set contexts and stuff for next coeff */
if ( absValue == 0 ) ctx3 = 0;
else if ( absValue == 1 ) ctx3 = 1;
else ctx3 = 2;
prevCoeffWasZero = true;
}
/* Store current block status to decoder internals */
**block_has_coefficients[currentMb][currentBlock]** =
currentBlockHasCoeffs;Last but not least 🌟, if the pseudocode still feels a bit slippery, the reference implementation in tokens.c (Section 20.16) is a great place to look. Honestly it's the best way to see how all the pieces click together 💖✨
13.4. Token Probability Updates ✨
okay so the token-decoding probabilities can change from frame to frame, bestie 💅 they're not stuck in place! 🦋 After a key frame gets detected, they get reset to their defaults from Section 13.5. That reset has to happen before the rest of the header gets decoded, because both key frames and interframes are allowed to adjust these probabilities 🔥
The coefficient probability update record (Section I of the frame header) is lowkey simple. For every position in the coeff_probs array, there's a fixed-probability bool that says whether that probability should get updated 🌸 If the bool is true, a P(8) follows and replaces the probability. The main thing to know is that updates are cumulative 💖 A probability updated on one frame stays in effect for all the frames after it, until the next key frame comes along or another frame explicitly updates it again. It's giving long-term memory ✨
Here's the algorithm that does all of that 🧚
int i = 0; do {
int j = 0; do {
int k = 0; do {
int t = 0; do {
if (read_bool(d, coeff_update_probs [i] [j] [k] [t]))
coeff_probs [i] [j] [k] [t] = read_literal(d, 8);
} while (++t < num_dct_tokens - 1);
} while (++k < 3);
} while (++j < 8);
} while (++i < 4);And these are the (constant) update probabilities, which never change 🎀
const Prob coeff_update_probs [4] [8] [3] [num_dct_tokens-1] =
{
{
{
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 176, 246, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 223, 241, 252, 255, 255, 255, 255, 255, 255, 255, 255},
{ 249, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 244, 252, 255, 255, 255, 255, 255, 255, 255, 255},
{ 234, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 253, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 246, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 239, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 254, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 248, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 251, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 251, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 254, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 254, 253, 255, 254, 255, 255, 255, 255, 255, 255},
{ 250, 255, 254, 255, 254, 255, 255, 255, 255, 255, 255},
{ 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
}
},
{
{
{ 217, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 225, 252, 241, 253, 255, 255, 254, 255, 255, 255, 255},
{ 234, 250, 241, 250, 253, 255, 253, 254, 255, 255, 255}
},
{
{ 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 223, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 238, 253, 254, 254, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 248, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 249, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 253, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 247, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 252, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 253, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 254, 253, 255, 255, 255, 255, 255, 255, 255, 255},
{ 250, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
}
},
{
{
{ 186, 251, 250, 255, 255, 255, 255, 255, 255, 255, 255},
{ 234, 251, 244, 254, 255, 255, 255, 255, 255, 255, 255},
{ 251, 251, 243, 253, 254, 255, 254, 255, 255, 255, 255}
},
{
{ 255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 236, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 251, 253, 253, 254, 254, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 254, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
}
},
{
{
{ 248, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 250, 254, 252, 254, 255, 255, 255, 255, 255, 255, 255},
{ 248, 254, 249, 253, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255},
{ 246, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255},
{ 252, 254, 251, 254, 254, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 254, 252, 255, 255, 255, 255, 255, 255, 255, 255},
{ 248, 254, 253, 255, 255, 255, 255, 255, 255, 255, 255},
{ 253, 255, 254, 254, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 251, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 245, 251, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 253, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 251, 253, 255, 255, 255, 255, 255, 255, 255, 255},
{ 252, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 252, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 249, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 255, 253, 255, 255, 255, 255, 255, 255, 255, 255},
{ 250, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
},
{
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255},
{ 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
}
}
};13.5. Default Token Probability Table 💖
okay so here's the default token probability table, bestie 🎀 it's a big one, so get comfy 🧚
const Prob default_coeff_probs [4] [8] [3] [num_dct_tokens - 1] =
{
{
{
{ 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128},
{ 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128},
{ 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128}
},
{
{ 253, 136, 254, 255, 228, 219, 128, 128, 128, 128, 128},
{ 189, 129, 242, 255, 227, 213, 255, 219, 128, 128, 128},
{ 106, 126, 227, 252, 214, 209, 255, 255, 128, 128, 128}
},
{
{ 1, 98, 248, 255, 236, 226, 255, 255, 128, 128, 128},
{ 181, 133, 238, 254, 221, 234, 255, 154, 128, 128, 128},
{ 78, 134, 202, 247, 198, 180, 255, 219, 128, 128, 128}
},
{
{ 1, 185, 249, 255, 243, 255, 128, 128, 128, 128, 128},
{ 184, 150, 247, 255, 236, 224, 128, 128, 128, 128, 128},
{ 77, 110, 216, 255, 236, 230, 128, 128, 128, 128, 128}
},
{
{ 1, 101, 251, 255, 241, 255, 128, 128, 128, 128, 128},
{ 170, 139, 241, 252, 236, 209, 255, 255, 128, 128, 128},
{ 37, 116, 196, 243, 228, 255, 255, 255, 128, 128, 128}
},
{
{ 1, 204, 254, 255, 245, 255, 128, 128, 128, 128, 128},
{ 207, 160, 250, 255, 238, 128, 128, 128, 128, 128, 128},
{ 102, 103, 231, 255, 211, 171, 128, 128, 128, 128, 128}
},
{
{ 1, 152, 252, 255, 240, 255, 128, 128, 128, 128, 128},
{ 177, 135, 243, 255, 234, 225, 128, 128, 128, 128, 128},
{ 80, 129, 211, 255, 194, 224, 128, 128, 128, 128, 128}
},
{
{ 1, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128},
{ 246, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128},
{ 255, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128}
}
},
{
{
{ 198, 35, 237, 223, 193, 187, 162, 160, 145, 155, 62},
{ 131, 45, 198, 221, 172, 176, 220, 157, 252, 221, 1},
{ 68, 47, 146, 208, 149, 167, 221, 162, 255, 223, 128}
},
{
{ 1, 149, 241, 255, 221, 224, 255, 255, 128, 128, 128},
{ 184, 141, 234, 253, 222, 220, 255, 199, 128, 128, 128},
{ 81, 99, 181, 242, 176, 190, 249, 202, 255, 255, 128}
},
{
{ 1, 129, 232, 253, 214, 197, 242, 196, 255, 255, 128},
{ 99, 121, 210, 250, 201, 198, 255, 202, 128, 128, 128},
{ 23, 91, 163, 242, 170, 187, 247, 210, 255, 255, 128}
},
{
{ 1, 200, 246, 255, 234, 255, 128, 128, 128, 128, 128},
{ 109, 178, 241, 255, 231, 245, 255, 255, 128, 128, 128},
{ 44, 130, 201, 253, 205, 192, 255, 255, 128, 128, 128}
},
{
{ 1, 132, 239, 251, 219, 209, 255, 165, 128, 128, 128},
{ 94, 136, 225, 251, 218, 190, 255, 255, 128, 128, 128},
{ 22, 100, 174, 245, 186, 161, 255, 199, 128, 128, 128}
},
{
{ 1, 182, 249, 255, 232, 235, 128, 128, 128, 128, 128},
{ 124, 143, 241, 255, 227, 234, 128, 128, 128, 128, 128},
{ 35, 77, 181, 251, 193, 211, 255, 205, 128, 128, 128}
},
{
{ 1, 157, 247, 255, 236, 231, 255, 255, 128, 128, 128},
{ 121, 141, 235, 255, 225, 227, 255, 255, 128, 128, 128},
{ 45, 99, 188, 251, 195, 217, 255, 224, 128, 128, 128}
},
{
{ 1, 1, 251, 255, 213, 255, 128, 128, 128, 128, 128},
{ 203, 1, 248, 255, 255, 128, 128, 128, 128, 128, 128},
{ 137, 1, 177, 255, 224, 255, 128, 128, 128, 128, 128}
}
},
{
{
{ 253, 9, 248, 251, 207, 208, 255, 192, 128, 128, 128},
{ 175, 13, 224, 243, 193, 185, 249, 198, 255, 255, 128},
{ 73, 17, 171, 221, 161, 179, 236, 167, 255, 234, 128}
},
{
{ 1, 95, 247, 253, 212, 183, 255, 255, 128, 128, 128},
{ 239, 90, 244, 250, 211, 209, 255, 255, 128, 128, 128},
{ 155, 77, 195, 248, 188, 195, 255, 255, 128, 128, 128}
},
{
{ 1, 24, 239, 251, 218, 219, 255, 205, 128, 128, 128},
{ 201, 51, 219, 255, 196, 186, 128, 128, 128, 128, 128},
{ 69, 46, 190, 239, 201, 218, 255, 228, 128, 128, 128}
},
{
{ 1, 191, 251, 255, 255, 128, 128, 128, 128, 128, 128},
{ 223, 165, 249, 255, 213, 255, 128, 128, 128, 128, 128},
{ 141, 124, 248, 255, 255, 128, 128, 128, 128, 128, 128}
},
{
{ 1, 16, 248, 255, 255, 128, 128, 128, 128, 128, 128},
{ 190, 36, 230, 255, 236, 255, 128, 128, 128, 128, 128},
{ 149, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128}
},
{
{ 1, 226, 255, 128, 128, 128, 128, 128, 128, 128, 128},
{ 247, 192, 255, 128, 128, 128, 128, 128, 128, 128, 128},
{ 240, 128, 255, 128, 128, 128, 128, 128, 128, 128, 128}
},
{
{ 1, 134, 252, 255, 255, 128, 128, 128, 128, 128, 128},
{ 213, 62, 250, 255, 255, 128, 128, 128, 128, 128, 128},
{ 55, 93, 255, 128, 128, 128, 128, 128, 128, 128, 128}
},
{
{ 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128},
{ 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128},
{ 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128}
}
},
{
{
{ 202, 24, 213, 235, 186, 191, 220, 160, 240, 175, 255},
{ 126, 38, 182, 232, 169, 184, 228, 174, 255, 187, 128},
{ 61, 46, 138, 219, 151, 178, 240, 170, 255, 216, 128}
},
{
{ 1, 112, 230, 250, 199, 191, 247, 159, 255, 255, 128},
{ 166, 109, 228, 252, 211, 215, 255, 174, 128, 128, 128},
{ 39, 77, 162, 232, 172, 180, 245, 178, 255, 255, 128}
},
{
{ 1, 52, 220, 246, 198, 199, 249, 220, 255, 255, 128},
{ 124, 74, 191, 243, 183, 193, 250, 221, 255, 255, 128},
{ 24, 71, 130, 219, 154, 170, 243, 182, 255, 255, 128}
},
{
{ 1, 182, 225, 249, 219, 240, 255, 224, 128, 128, 128},
{ 149, 150, 226, 252, 216, 205, 255, 171, 128, 128, 128},
{ 28, 108, 170, 242, 183, 194, 254, 223, 255, 255, 128}
},
{
{ 1, 81, 230, 252, 204, 203, 255, 192, 128, 128, 128},
{ 123, 102, 209, 247, 188, 196, 255, 233, 128, 128, 128},
{ 20, 95, 153, 243, 164, 173, 255, 203, 128, 128, 128}
},
{
{ 1, 222, 248, 255, 216, 213, 128, 128, 128, 128, 128},
{ 168, 175, 246, 252, 235, 205, 255, 255, 128, 128, 128},
{ 47, 116, 215, 255, 211, 212, 255, 255, 128, 128, 128}
},
{
{ 1, 121, 236, 253, 212, 214, 255, 255, 128, 128, 128},
{ 141, 84, 213, 252, 201, 202, 255, 219, 128, 128, 128},
{ 42, 80, 160, 240, 162, 185, 255, 205, 128, 128, 128}
},
{
{ 1, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128},
{ 244, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128},
{ 238, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128}
}
}
};14. DCT and WHT Inversion and Macroblock Reconstruction ✨
14.1. Dequantization 🌸
okay so after we've decoded the DCTs/WHTs like we talked about above, every (quantized) coefficient in each subblock gets multiplied by one of six dequantization factors 💅 which one depends on the plane (Y2, Y, or chroma) and the position (DC = coefficient zero, AC = any other coefficient) 🧚 if the current macroblock has overridden the quantizer level (see Section 10), then the six factors get looked up from two dequantization tables, with the appropriate scaling and clamping, using the single index that the override gives us 🎀 otherwise we use the frame-level dequantization factors (see Section 9.6) ✨ either way, the multiplies get computed and stored using 16-bit signed integers 💖
the two dequantization tables, which you can also find in the reference decoder file dequant_data.h (Section 20.3), look like this 🦋
static const int dc_qlookup[QINDEX_RANGE] =
{
4, 5, 6, 7, 8, 9, 10, 10, 11, 12, 13, 14, 15,
16, 17, 17, 18, 19, 20, 20, 21, 21, 22, 22, 23, 23,
24, 25, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
36, 37, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 46,
47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72,
73, 74, 75, 76, 76, 77, 78, 79, 80, 81, 82, 83, 84,
85, 86, 87, 88, 89, 91, 93, 95, 96, 98, 100, 101, 102,
104, 106, 108, 110, 112, 114, 116, 118, 122, 124, 126, 128, 130,
132, 134, 136, 138, 140, 143, 145, 148, 151, 154, 157,
};
static const int ac_qlookup[QINDEX_RANGE] =
{
4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42,
43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,
56, 57, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78,
80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100, 102, 104,
106, 108, 110, 112, 114, 116, 119, 122, 125, 128, 131, 134, 137,
140, 143, 146, 149, 152, 155, 158, 161, 164, 167, 170, 173, 177,
181, 185, 189, 193, 197, 201, 205, 209, 213, 217, 221, 225, 229,
234, 239, 245, 249, 254, 259, 264, 269, 274, 279, 284,
};lookup values from the above two tables get used directly for the DC and AC coefficients in Y1, respectively 💜 for Y2 and chroma, though, the values from the above tables go through either scaling or clamping before the multiplies 🙈 the details of that scaling and clamping live in the related lookup functions in dixie.c (Section 20.4) 🫶
14.2. Inverse Transforms 🦋
okay this part is so satisfying 😭 if the Y2 residue block exists (aka the macroblock luma mode is not SPLITMV or B_PRED), it gets inverted FIRST using the inverse WHT 🔥 then the element of the result at row i, column j gets used as the 0th coefficient of the Y subblock at position (i, j), which is the Y subblock whose index is (i * 4) + j 💖 like we discussed in Section 13, if the luma mode is B_PRED or SPLITMV, the 0th Y coefficients are part of the residue signal for the subblocks themselves ✨
either way, the inverse transforms for the sixteen Y subblocks and the eight chroma subblocks come next 🌈 all 24 of these inversions are independent of each other, so their results can (at least conceptually) be stored in 24 separate 4x4 arrays 🧚
as the reference decoder does, an implementation may wish to represent the prediction and residue buffers as macroblock-sized arrays (that is, a 16x16 Y buffer and two 8x8 chroma buffers) 🍓 regarding the inverse DCT implementation given below, this just needs a simple adjustment to the address calculation for the resulting residue pixels 💅
14.3. Implementation of the WHT Inversion ✨
okay so bestie, remember Sections 2 and 13? 💅 For macroblocks that use prediction modes other than B_PRED and SPLITMV, the DC values that come out of the DCT on the 16 Y blocks get gathered up into a 25th block for the macroblock (16 Y + 4 U + 4 V = the 24 blocks we already have). That 25th block goes through a Walsh-Hadamard transform (WHT) 🧚
The inputs to the inverse WHT (aka the dequantized coefficients), the intermediate "horizontally detransformed" signal, and the fully detransformed residue are all stored as arrays of 16-bit signed integers. Tiny numbers, big vibes 🌸
Following the tradition of specifying the bitstream format through the decoding process, the inverse WHT gets defined in the decoding process using the C-style source code below 🎀
void vp8_short_inv_walsh4x4_c(short *input, short *output)
{
int i;
int a1, b1, c1, d1;
int a2, b2, c2, d2;
short *ip = input;
short *op = output;
int temp1, temp2;
for(i=0;i<4;i++)
{
a1 = ip[0] + ip[12];
b1 = ip[4] + ip[8];
c1 = ip[4] - ip[8];
d1 = ip[0] - ip[12];
op[0] = a1 + b1;
op[4] = c1 + d1;
op[8] = a1 - b1;
op[12]= d1 - c1;
ip++;
op++;
}
ip = output;
op = output;
for(i=0;i<4;i++)
{
a1 = ip[0] + ip[3];
b1 = ip[1] + ip[2];
c1 = ip[1] - ip[2];
d1 = ip[0] - ip[3];
a2 = a1 + b1;
b2 = c1 + d1;
c2 = a1 - b1;
d2 = d1 - c1;
op[0] = (a2+3)>>3;
op[1] = (b2+3)>>3;
op[2] = (c2+3)>>3;
op[3] = (d2+3)>>3;
ip+=4;
op+=4;
}
}In the case that there is only one non-zero DC value in input, the inverse transform can be simplified to the following 🙈
void vp8_short_inv_walsh4x4_1_c(short *input, short *output)
{
int i;
int a1;
short *op=output;
a1 = ((input[0] + 3)>>3);
for(i=0;i<4;i++)
{
op[0] = a1;
op[1] = a1;
op[2] = a1;
op[3] = a1;
op+=4;
}
}⚠️ bestie, listen up: a conforming decoder should implement the inverse transform using exactly the same rounding, so that its output matches the output of the process in the C source above bit-for-bit. no wiggle room, no vibes-based rounding 💀
The reference decoder WHT inversion may be found in the file idct_add.c (Section 20.8) 🦋
14.4. Implementation of the DCT Inversion 🌈
All of the DCT inversions are computed in exactly the same way. In principle, VP8 uses a classical 2-D inverse discrete cosine transform, implemented as two passes of 1-D inverse DCT. The 1-D inverse DCT was calculated using a similar algorithm to what was described in [Loeffler]. However, the paper only provided the 8-point and 16-point versions of the algorithms, which On2 adapted to perform the 4-point 1-D DCT 💜
Accurate calculation of the 1-D DCT with that algorithm requires infinite precision, and VP8 of course can only use a finite-precision approximation 🥺 Also, the inverse DCT used by VP8 takes care of normalization of the standard unitary transform. That means every dequantized coefficient has roughly double the size of the corresponding unitary coefficient. Still, at all but the highest datarates, the gap between transmitted and ideal coefficients comes almost entirely from (lossy) compression, not from finite-precision arithmetic errors ✨
The inputs to the inverse DCT (aka the dequantized coefficients), the intermediate "horizontally detransformed" signal, and the completely detransformed residue signal are all stored as arrays of 16-bit signed integers. The details of the computation are as follows 🎀
It should also be noted that this implementation makes use of the 16-bit fixed-point version of two multiplication constants:
sqrt(2) * cos (pi/8)
sqrt(2) * sin (pi/8)
Because the first constant is bigger than 1, the code keeps the same 16-bit fixed-point precision as the second one by using the fact that
x * a = x + x*(a-1)therefore
x * sqrt(2) * cos (pi/8) = x + x * (sqrt(2) * cos(pi/8)-1)/* IDCT implementation */
static const int cospi8sqrt2minus1=20091;
static const int sinpi8sqrt2 =35468;
void short_idct4x4llm_c(short *input, short *output, int pitch)
{
int i;
int a1, b1, c1, d1;
short *ip=input;
short *op=output;
int temp1, temp2;
int shortpitch = pitch>>1;
for(i=0;i<4;i++)
{
a1 = ip[0]+ip[8];
b1 = ip[0]-ip[8];
temp1 = (ip[4] * sinpi8sqrt2)>>16;
temp2 = ip[12]+((ip[12] * cospi8sqrt2minus1)>>16);
c1 = temp1 - temp2;
temp1 = ip[4] + ((ip[4] * cospi8sqrt2minus1)>>16);
temp2 = (ip[12] * sinpi8sqrt2)>>16;
d1 = temp1 + temp2;
op[shortpitch*0] = a1+d1;
op[shortpitch*3] = a1-d1;
op[shortpitch*1] = b1+c1;
op[shortpitch*2] = b1-c1;
ip++;
op++;
}
ip = output;
op = output;
for(i=0;i<4;i++)
{
a1 = ip[0]+ip[2];
b1 = ip[0]-ip[2];
temp1 = (ip[1] * sinpi8sqrt2)>>16;
temp2 = ip[3]+((ip[3] * cospi8sqrt2minus1)>>16);
c1 = temp1 - temp2;
temp1 = ip[1] + ((ip[1] * cospi8sqrt2minus1)>>16);
temp2 = (ip[3] * sinpi8sqrt2)>>16;
d1 = temp1 + temp2;
op[0] = (a1+d1+4)>>3;
op[3] = (a1-d1+4)>>3;
op[1] = (b1+c1+4)>>3;
op[2] = (b1-c1+4)>>3;
ip+=shortpitch;
op+=shortpitch;
}
}The reference decoder DCT inversion may be found in the file idct_add.c (Section 20.8) 🫶
14.5. Summation of Predictor and Residue 👑
Finally, the prediction and residue signals get summed up to form the reconstructed macroblock. Except for loop filtering (which is up next!), that completes the decoding process 🔥
The summing procedure is pretty straightforward, with only a couple of details. The prediction and residue buffers are both arrays of 16-bit signed integers. Each individual result (Y, U, and V pixel) is first calculated as a 32-bit sum of the prediction and residue. Then it gets saturated to 8-bit unsigned range (using, say, the clamp255 function defined above) before being stored as an 8-bit unsigned pixel value 💖
VP8 also supports a mode where the encoding of a bitstream guarantees that all reconstructed pixel values are between 0 and 255. Compliant bitstreams with that guarantee have the clamp_type bit in the frame header set to 1. In that case, the clamp255 function is no longer required 🦋
The summation process is the same regardless of the (intra or inter) mode of prediction in effect for the macroblock. The reference decoder implementation of reconstruction may be found in the file idct_add.c 🌸
15. Loop Filter 🎀
Loop filtering is the last stage of frame reconstruction and the next-to-last stage of the decoding process. The loop filter gets applied to the entire frame after the summation of predictor and residue signals, as described in Section 14 🧚
The purpose of the loop filter is to eliminate (or at least reduce) visually objectionable artifacts that come from the semi-independence of the coding of macroblocks and their constituent subblocks. It's literally the cleanup crew 🧹✨
As was discussed in Section 5, the loop filter is "integral" to decoding, since the results of loop filtering are used in the prediction of subsequent frames. So a functional decoder implementation must perform loop filtering exactly as described here 🙈 This is distinct from any postprocessing that may be applied only to the image right before display. That kind of postprocessing is entirely at the option of the implementor (and/or user) and has no effect on decoding per se 💅
The baseline frame-level parameters controlling the loop filter are defined in the frame header (Section 9.4), along with a mechanism for adjustment based on a macroblock's prediction mode and/or reference frame. The first is a flag (filter_type) selecting the type of filter (normal or simple). The other two are numbers (loop_filter_level and sharpness_level) that adjust the strength or sensitivity of the filter. As described in Sections 9.3 and 10, loop_filter_level may also be overridden on a per-macroblock basis using segmentation 🌈
Loop filtering is one of the more computationally intensive aspects of VP8 decoding. That's the reason the optional, less-demanding simple filter type exists 💜
Note carefully that loop filtering must be skipped entirely if loop_filter_level at either the frame header level or the macroblock override level is 0. In no case should the loop filter be run with a value of 0; it should instead be skipped. ⚠️🥺
We begin by discussing the aspects of loop filtering that are independent of the controlling parameters and the type of filter chosen 🎀
15.1. Filter Geometry and Overall Procedure 🎀
okay so the Y, U, and V planes all get filtered independently and identically 💅 same rules, no favourites, iconic behaviour 👑
the loop filter works on the edges between adjacent macroblocks and on the edges between adjacent subblocks inside a macroblock 🧚 every one of these edges is either horizontal or vertical. for each pixel position on an edge, a small number of pixels (two or three) on each side of that position get looked at and maybe modified 🦋 the pixels we look at sit at a right angle to the edge. so for a horizontal edge we grab the pixels right above and right below it, and for a vertical edge we grab the ones immediately to the left and right 🌸
we call that whole collection of pixels for one edge position a segment ✨ a segment is 2, 4, 6, or 8 pixels long. the normal filter uses slightly different algorithms for macroblock edges versus subblock edges, and either filter can use different control parameters for each, but otherwise the treatment is super uniform 💖 every segment straddling an edge gets handled the exact same way, and horizontal and vertical edges are treated the same too, whether they're between macroblocks or between subblocks 🫶
this means adjacent subblock edges inside a macroblock can be stitched together and processed as one long edge 🍓 there's a single 8-pixel-long vertical edge running through the middle of each of the U and V blocks (that's the upper and lower 4-pixel chroma edges joined up 🌈). then there are three 16-pixel-long vertical edges in the luma macroblock, sitting at horizontal positions 1/4, 1/2, and 3/4 of the way across 🌟 each of those is four 4-pixel sub-edges between pairs of Y subblocks glued together 😭
macroblocks get processed in the usual raster-scan order 🔥 each macroblock is "responsible for" the inter-macroblock edges directly above it and to its left (but NOT the ones below or to its right, those belong to other macroblocks 🙈), plus the edges between its own subblocks 💜
for each macroblock M, there are four filtering steps, done (almost) in order:
- If M is not on the leftmost column of macroblocks, filter across the left (vertical) inter-macroblock edge of M.
- Filter across the vertical subblock edges within M.
- If M is not on the topmost row of macroblocks, filter across the top (horizontal) inter-macroblock edge of M.
- Filter across the horizontal subblock edges within M.
we write MY, MU, and MV for the planar pieces of M 🎀 that's the 16x16 luma block, the 8x8 U block, and the 8x8 V block that make up the macroblock.
in step 1, for each of MY, MU, and MV, we filter each of the segments straddling the column that separates the block from the block immediately to its left (16 segments for luma, 8 for chroma) 🦋 we use the inter-macroblock filter and the controls tied to loop_filter_level and sharpness_level for this. in step 4, we filter across the vertical subblock edges described above (three for luma, plus one each for U and V) using the inter-subblock filter and its controls 🧚
steps 2 and 4 get skipped for macroblocks that meet BOTH of these conditions 💀:
- The macroblock coding mode is neither
B_PREDnorSPLITMV; and - There is no DCT coefficient coded for the whole macroblock.
for these macroblocks, the loop filtering of the edges between subblocks inside the macroblock is effectively skipped 🌸 this skip strategy significantly reduces VP8 loop-filtering complexity, which is literally such a W 👑
⚠️ bestie, the macroblock edges and the subblock edges get different control parameters (and, for the normal filter, different algorithms too) 😭 apart from pixel addressing, there's no difference between how vertical and horizontal edges get treated 💅 luma edges are always 16 pixels long, chroma edges are always 8 pixels long, and the segments on an edge all get treated identically, which totally helps with vector processing 🌈
here's the big one 🔥: lots of pixels belong to segments that straddle two or more edges, so they get filtered more than once. that means the order I listed above MUST be respected by any implementation 💀 (lowercase must, because it's a plain English word here, not a rule keyword, but it's still a whole vibe). within a single edge, though, the segments are disjoint, so the order you process them in doesn't matter at all ✨
before we get into the filtering algorithms, we need to emphasize something I already mentioned: even though the pixel segments tied to a macroblock are "antecedent" to it (meaning they lie inside the macroblock or in macroblocks that were already built), a macroblock must not be filtered right after its "reconstruction" (see Section 14) 🙈 instead, the loop filter runs after ALL the macroblocks have been "reconstructed" (aka had their predictor summed with their residue) 💖 correct decoding depends on the fact that the already-built parts of the current frame that get used for intra-prediction (see Section 12) are NOT filtered yet 🫶 the filter waits its turn, periodt 💅✨
15.2. Simple Filter 🎀
Okay so we've covered the overall loop filter vibe (which pixels it touches and where) and now we're getting into the individual segments straddling edges 💖 We're starting with the simple filter, which, as you might guess, is lowkey simpler than the normal one ✨
Note that the simple filter only applies to luma edges. Chroma edges are left unfiltered 🙈
Roughly speaking, the whole point of loop filtering is, within limits, to shrink the difference between pixels on either side of an edge 🦋 Differences bigger than a threshold (tied to the loop_filter_level) are treated as "natural" and left alone 👑 Differences smaller than that threshold are treated as artifacts of quantization and the (partially) separate coding of blocks, so those get smoothed out 💅 The loop_filter_level could in principle be anything, but the levels a VP8 compressor picks tend to be correlated with its quantizer levels 🌸
Most of the filtering math uses 8-bit signed operands (range -128 to +127, inclusive) 🔥 with 16-bit temporaries holding the results of multiplies 🧚
Sums and other temporaries need to be "clamped" to a valid signed 8-bit range 💜
int8 c(int v)
{
return (int8) (v < -128 ? -128 : (v < 128 ? v : 127));
}Pixel values themselves are unsigned 8-bit numbers, so we need to convert back and forth between signed and unsigned values 🫶
/* Convert pixel value (0 <= v <= 255) to an 8-bit signed
number. */
int8 u2s(Pixel v) { return (int8) (v - 128);}
/* Clamp, then convert signed number back to pixel value. */
Pixel s2u(int v) { return (Pixel) (c(v) + 128);}Filtering is super often based on absolute-value thresholds 🍓 The function below is the equivalent of the standard library function abs, whose prototype lives in the standard header file stdlib.h 🎀 For us, the argument v is always the difference between two pixels and lies in the range -255 <= v <= +255 ✨
int abs(int v) { return v < 0? -v : v;}An actual implementation would obviously use inline functions or macros for these trivial procedures (they're used by both the normal and simple loop filters) 💖 An optimal implementation would probably express them in machine language, maybe using single instruction, multiple data (SIMD) vector instructions 🦋 On lots of SIMD processors, the saturation done by the clamping function above is often folded right into the arithmetic instructions themselves, so the explicit step we wrote out isn't needed 🔥
To keep relative pixel positions straight, we use the word "before" to mean "immediately above" (for a vertical segment straddling a horizontal edge) or "immediately to the left of" (for a horizontal segment straddling a vertical edge) 🌈 And "after" means "immediately below" or "immediately to the right of" 👑
Given an edge, a segment, and a limit value, the simple loop filter computes a value based on the four pixels straddling the edge (two on either side) 💅 If that value is below a supplied limit, then, very roughly, the two pixel values get brought closer together, "shaving off" something like a quarter of the difference 🫶 The same procedure is used for all segments straddling any type of edge, no matter what kind it is (inter-macroblock, inter-subblock, luma, or chroma) 🌸 The only thing that changes by edge type is the limit value 🧚
Here's the exact procedure for a single segment. The subroutine common_adjust is used by both the simple filter we're looking at now and the normal filters in Section 15.3 💜
int8 common_adjust(
int use_outer_taps, /* filter is 2 or 4 taps wide */
const Pixel *P1, /* pixel before P0 */
Pixel *P0, /* pixel before edge */
Pixel *Q0, /* pixel after edge */
const Pixel *Q1 /* pixel after Q0 */
) {
cint8 p1 = u2s(*P1); /* retrieve and convert all 4 pixels */
cint8 p0 = u2s(*P0);
cint8 q0 = u2s(*Q0);
cint8 q1 = u2s(*Q1);
/* Disregarding clamping, when "use_outer_taps" is false,
"a" is 3*(q0-p0). Since we are about to divide "a" by
8, in this case we end up multiplying the edge
difference by 5/8.
When "use_outer_taps" is true (as for the simple filter),
"a" is p1 - 3*p0 + 3*q0 - q1, which can be thought of as
a refinement of 2*(q0 - p0), and the adjustment is
something like (q0 - p0)/4. */
int8 a = c((use_outer_taps? c(p1 - q1) : 0) + 3*(q0 - p0));
/* b is used to balance the rounding of a/8 in the case where
the "fractional" part "f" of a/8 is exactly 1/2. */
cint8 b = (c(a + 3)) >> 3;
/* Divide a by 8, rounding up when f >= 1/2.
Although not strictly part of the C language,
the right shift is assumed to propagate the sign bit. */
a = c(a + 4) >> 3;
/* Subtract "a" from q0, "bringing it closer" to p0. */
*Q0 = s2u(q0 - a);
/* Add "a" (with adjustment "b") to p0, "bringing it closer"
to q0.
The clamp of "a+b", while present in the reference decoder,
is superfluous; we have -16 <= a <= 15 at this point. */
*P0 = s2u(p0 + b);
return a;
}void simple_segment(
uint8 edge_limit, /* do nothing if edge difference
exceeds limit */
const Pixel *P1, /* pixel before P0 */
Pixel *P0, /* pixel before edge */
Pixel *Q0, /* pixel after edge */
const Pixel *Q1 /* pixel after Q0 */
) {
if ((abs(*P0 - *Q0)*2 + abs(*P1 - *Q1)/2) <= edge_limit))
common_adjust(1, P1, P0, Q0, Q1); /* use outer taps */
}Now for the rounding drama 🙈 When b is zero (so the "fractional part" of a isn't 1/2), we're (except for clamping) adding the same number to p0 that we subtract from q0 🦋 That keeps the average value of p0 and q0 the same, but the resulting difference between p0 and q0 is always even. So the smallest non-zero gradation, +-1, just isn't possible here 💀
When b is one, the value we add to p0 (again except for clamping) is one less than the value we subtract from q0 😭 In this case the resulting difference is always odd, so the small gradation +-1 is possible, but the average value drops by 1/2 🌈 That gives a very slight darkening in the luma plane. (In the very unlikely event of appreciable darkening after a large number of interframes, a compressor would of course eventually compensate for this in its selection of predictor and/or residue 🫶)
The derivation of the edge_limit value used above, which depends on the loop_filter_level and sharpness_level, as well as the type of edge being processed, will be taken up after we describe the normal loop filtering algorithm below. 🧚
15.3. Normal Filter ✨
okay so the normal loop filter is basically the simple filter's glow-up 💅 everything from the general discussion above still applies here, bestie. 🎀 The functions c, u2s, s2u, abs, and common_adjust are shared by both the normal and simple filters, so you already know them! 🫶
As we mentioned above, the normal algorithms for inter-macroblock and inter-subblock edges are different, but they're lowkey twins 👯♀️ They both use similar threshold checks to turn the filter off and to detect high internal edge variance, which decides how the filtering actually happens. 🦋 They also both use the simple filter adjustment procedure from above, at least some of the time. 💜
Here are the shared thresholding checks, no cap they're the foundation of everything 🔥
/* All functions take (among other things) a segment (of length
at most 4 + 4 = 8) symmetrically straddling an edge.
The pixel values (or pointers) are always given in order,
from the "beforemost" to the "aftermost". So, for a
horizontal edge (written "|"), an 8-pixel segment would be
ordered p3 p2 p1 p0 | q0 q1 q2 q3. */
/* Filtering is disabled if the difference between any two
adjacent "interior" pixels in the 8-pixel segment exceeds
the relevant threshold (I). A more complex thresholding
calculation is done for the group of four pixels that
straddle the edge, in line with the calculation in
simple_segment() above. */
int filter_yes(
uint8 I, /* limit on interior differences */
uint8 E, /* limit at the edge */
cint8 p3, cint8 p2, cint8 p1, cint8 p0, /* pixels before
edge */
cint8 q0, cint8 q1, cint8 q2, cint8 q3 /* pixels after
edge */
) {
return (abs(p0 - q0)*2 + abs(p1 - q1)/2) <= E
&& abs(p3 - p2) <= I && abs(p2 - p1) <= I &&
abs(p1 - p0) <= I
&& abs(q3 - q2) <= I && abs(q2 - q1) <= I &&
abs(q1 - q0) <= I;
}/* Filtering is altered if (at least) one of the differences
on either side of the edge exceeds a threshold (we have
"high edge variance"). */
int hev(
uint8 threshold,
cint8 p1, cint8 p0, /* pixels before edge */
cint8 q0, cint8 q1 /* pixels after edge */
) {
return abs(p1 - p0) > threshold || abs(q1 - q0) > threshold;
}The subblock filter is basically a cousin of the simple filter 🌸 If we've got high edge variance, the adjustment is exactly the same as the simple filter's. If we don't, the simple adjustment runs without the outer taps, and the two pixels one step in from the edge pixels get adjusted by roughly half of what the edge pixels get. Since the edge adjustment here is essentially 3/8 of the edge difference, the inner adjustment works out to approximately 3/16 of the edge difference. the math is giving precision ✨
void subblock_filter(
uint8 hev_threshold, /* detect high edge variance */
uint8 interior_limit, /* possibly disable filter */
uint8 edge_limit,
cint8 *P3, cint8 *P2, int8 *P1, int8 *P0, /* pixels before
edge */
int8 *Q0, int8 *Q1, cint8 *Q2, cint8 *Q3 /* pixels after
edge */
) {
cint8 p3 = u2s(*P3), p2 = u2s(*P2), p1 = u2s(*P1),
p0 = u2s(*P0);
cint8 q0 = u2s(*Q0), q1 = u2s(*Q1), q2 = u2s(*Q2),
q3 = u2s(*Q3);
if (filter_yes(interior_limit, edge_limit, q3, q2, q1, q0,
p0, p1, p2, p3))
{
const int hv = hev(hev_threshold, p1, p0, q0, q1);
cint8 a = (common_adjust(hv, P1, P0, Q0, Q1) + 1) >> 1;
if (!hv) {
*Q1 = s2u(q1 - a);
*P1 = s2u(p1 + a);
}
}
}The inter-macroblock filter has the bigger reach, bestie 👑 If the edge variance is high, it does the simple adjustment (using the outer taps, just like the simple filter and the matching case of the normal subblock filter). If the edge variance is low, it starts with the same basic filter calculation and applies multiples of it to pixel pairs that are symmetric about the edge. The strength of the adjustment fades as we move away from the edge, and six of the pixels in the segment get affected 🦋
void MBfilter(
uint8 hev_threshold, /* detect high edge variance */
uint8 interior_limit, /* possibly disable filter */
uint8 edge_limit,
cint8 *P3, int8 *P2, int8 *P1, int8 *P0, /* pixels before
edge */
int8 *Q0, int8 *Q1, int8 *Q2, cint8 *Q3 /* pixels after
edge */
) {
cint8 p3 = u2s(*P3), p2 = u2s(*P2), p1 = u2s(*P1),
p0 = u2s(*P0);
cint8 q0 = u2s(*Q0), q1 = u2s(*Q1), q2 = u2s(*Q2),
q3 = u2s(*Q3);
if (filter_yes(interior_limit, edge_limit, q3, q2, q1, q0,
p0, p1, p2, p3))
{
if (!hev(hev_threshold, p1, p0, q0, q1))
{
/* Same as the initial calculation in "common_adjust",
w is something like twice the edge difference */
const int8 w = c(c(p1 - q1) + 3*(q0 - p0));
/* 9/64 is approximately 9/63 = 1/7, and 1<<7 = 128 =
2*64. So this a, used to adjust the pixels adjacent
to the edge, is something like 3/7 the edge
difference. */
int8 a = c((27*w + 63) >> 7);
*Q0 = s2u(q0 - a); *P0 = s2u(p0 + a);
/* Next two are adjusted by 2/7 the edge difference */
a = c((18*w + 63) >> 7);
*Q1 = s2u(q1 - a); *P1 = s2u(p1 + a);
/* Last two are adjusted by 1/7 the edge difference */
a = c((9*w + 63) >> 7);
*Q2 = s2u(q2 - a); *P2 = s2u(p2 + a);
} else /* if hev, do simple filter */
common_adjust(1, P1, P0, Q0, Q1); /* using outer
taps */
}
}15.4. Calculation of Control Parameters 🎀
okay so this is where we wrap up loop filtering by showing how all those thresholds get derived from two control parameters: sharpness_level (an unsigned 3-bit number, max value 7) and loop_filter_level (an unsigned 6-bit number, max value 63). 🧚
sharpness_level stays the same across the whole frame. loop_filter_level is different though: individual macroblocks can override it with one of four possibilities that get supplied in the frame header (see Section 10 for the deets) 💖
Both the simple and normal filters turn off filtering if a value derived from the four pixels straddling the edge (2 on either side) goes over a threshold / limit value. 🙈
/* Luma and Chroma use the same inter-macroblock edge limit */
uint8 mbedge_limit = ((loop_filter_level + 2) * 2) +
interior_limit;
/* Luma and Chroma use the same inter-subblock edge limit */
uint8 sub_bedge_limit = (loop_filter_level * 2) + interior_limit;The remaining thresholds are only used by the normal filters, bestie 💅 The interior difference limit that turns the filter off is the same for every edge type (luma, chroma, inter-subblock, inter-macroblock), and it's given by this:
uint8 interior_limit = loop_filter_level;
if (sharpness_level)
{
interior_limit >>= sharpness_level > 4 ? 2 : 1;
if (interior_limit > 9 - sharpness_level)
interior_limit = 9 - sharpness_level;
}
if (!interior_limit)
interior_limit = 1;Last but not least, here's the derivation of the high edge-variance threshold, which is also the same for all edge types. it's giving main character energy 👑✨
uint8 hev_threshold = 0;
if (we_are_decoding_akey_frame) /* current frame is a key frame */
{
if (loop_filter_level >= 40)
hev_threshold = 2;
else if (loop_filter_level >= 15)
hev_threshold = 1;
}
else /* current frame is an interframe */
{
if (loop_filter_level >= 40)
hev_threshold = 3;
else if (loop_filter_level >= 20)
hev_threshold = 2;
else if (loop_filter_level >= 15)
hev_threshold = 1;
}16. Interframe Macroblock Prediction Records 🦋
okay now we're getting into the prediction records for macroblocks in an interframe, and I'm so ready 🌈
After the feature specification (which is described in Section 10 and is identical for intraframes and interframes), there comes a Bool(prob_intra). It tells us whether the macroblock uses inter-prediction (prediction from prior frames) when it's true, or intra-prediction (prediction from parts of the current frame that have already been coded) when it's false 🍓 The zero-probability prob_intra gets set by field J of the frame header. 💜
16.1. Intra-Predicted Macroblocks ✨
okay so for intra-predicted macroblocks 💖 the layout of the prediction data is basically the same as key frames 🎀, but the contexts the decoder uses are slightly different 🌸
as we already talked about in Section 8, the "outer" Y mode here uses a different tree than key frames do 💅 so here it is again, for convenience:
const tree_index ymode_tree [2 * (num_ymodes - 1)] =
{
-DC_PRED, 2, /* root: DC_PRED = "0", "1" subtree */
4, 6, /* "1" subtree has 2 descendant subtrees */
-V_PRED, -H_PRED, /* "10" subtree: V_PRED = "100",
H_PRED = "101" */
-TM_PRED, -B_PRED /* "11" subtree: TM_PRED = "110",
B_PRED = "111" */
};the probability table for decoding this tree is variable 🦋 like Section 11 explains, it (along with the similarly treated UV table) can be updated by field J of the frame header 🫶. these updates are cumulative, same as the coefficient-decoding probabilities, so they keep affecting every frame after until the next key frame or an explicit update 😭. the default probabilities for the Y and UV tables are:
Prob ymode_prob [num_ymodes - 1] = { 112, 86, 140, 37};
Prob uv_mode_prob [num_uv_modes - 1] = { 162, 101, 204};these defaults must be restored after detection of a key frame 💀
just like key frames, if the Y mode is B_PRED, the next thing is an encoding of the intra_bpred mode used by each of the sixteen Y subblocks 🍓. these use the same tree as key frames, but instead of the key-frame contexts, they use one single fixed probability table ✨
const Prob bmode_prob [num_intra_bmodes - 1] = {
120, 90, 79, 133, 87, 85, 80, 111, 151
};last up is the chroma mode 👑 it's coded with the same tree as key frames, except this time it uses the dynamic uv_mode_prob table from above 💜
the calculation of the intra-prediction buffer is identical to what we do for key frames in Section 12 🌈
16.2. Inter-Predicted Macroblocks 🦋
otherwise (when the above bool is true), we're in inter-prediction territory 🧚, which of course only happens in interframes, and that's what we're focusing on now 🎀
the next datum is another bool, B(prob_last), which selects the reference frame 💅 if it's 0, the reference frame is the previous frame (the last frame) 🌸. if it's 1, another bool (prob_gf) picks between the golden frame (0) and the altref frame (1) ✨ the probabilities prob_last and prob_gf are set in field J of the frame header 🫶
the whole purpose of inter-mode decoding is to set a motion vector for each of the sixteen Y subblocks of the current macroblock 🔥. those settings then define the calculation of the inter-prediction buffer (detailed in Section 18) 🦋. the net effect is pretty straightforward, but the implementation is kinda complex 😭 ngl the (lossless) compression it gets you totally justifies the complexity 👑
after the reference frame selector comes the mode (or motion vector reference) applied to the macroblock as a whole 💖, coded with this enumeration and tree. setting mv_nearest = num_ymodes is a little trick that lets a single variable unambiguously hold either an inter- or intra-prediction mode 🧚
typedef enum
{
mv_nearest = num_ymodes, /* use "nearest" motion vector
for entire MB */
mv_near, /* use "next nearest" "" */
mv_zero, /* use zero "" */
mv_new, /* use explicit offset from
implicit "" */
mv_split, /* use multiple motion vectors */
num_mv_refs = mv_split + 1 - mv_nearest
}
mv_ref;
const tree_index mv_ref_tree [2 * (num_mv_refs - 1)] =
{
-mv_zero, 2, /* zero = "0" */
-mv_nearest, 4, /* nearest = "10" */
-mv_near, 6, /* near = "110" */
-mv_new, -mv_split /* new = "1110", split = "1111" */
};16.3. Mode and Motion Vector Contexts ✨
okay bestie, so the probability table used to decode the mv_ref 💅, along with three reference motion vectors for the selected mode, gets worked out by surveying the motion vectors that were already decoded in (up to) 3 nearby macroblocks 🦋
The algorithm builds a sorted list of distinct motion vectors next to the search site 🌸. best_mv is the vector with the highest score. mv_nearest is the non-zero vector with the highest score, and mv_near is the non-zero vector with the next highest score. The number of motion vectors coded with SPLITMV gets scored with the same weighting and comes back along with the scores for best, nearest, and near 💜
The three neighbours get checked in order: above, left, and above-left 👀. If a macroblock is intra-coded, we do nothing with it 🙈. Otherwise, its motion vector gets compared to the ones we've already found. If it's a repeat, it adds its weight to that vector. If it's new, it gets its own spot in the list ✨. The above and left vectors count for twice as much as the above-left one 💖
Here's the tea 🫖: macroblocks near the top or left edges can point at blocks that are outside the visible image, which is a very common VP8 vibe. VP8 gives us a border of 1 macroblock filled with 0x0 motion vectors to the left of the left edge, and a border filled with 0,0 motion vectors of 1 macroblocks above the top edge 🧚
This is way easier to show in C than to explain in English 🎀 The reference code lives in modemv.c (Section 20.11). The calculation of the reference vectors, the probability table, and finally the inter-prediction mode itself goes like this 👑
typedef union
{
unsigned int as_int;
MV as_mv;
} int_mv; /* facilitates rapid equality tests */
static void mv_bias(MODE_INFO *x,int refframe, int_mv *mvp,
int * ref_frame_sign_bias)
{
MV xmv;
xmv = x->mbmi.mv.as_mv;
if ( ref_frame_sign_bias[x->mbmi.ref_frame] !=
ref_frame_sign_bias[refframe] )
{
xmv.row*=-1;
xmv.col*=-1;
}
mvp->as_mv = xmv;
}void vp8_clamp_mv(MV *mv, const MACROBLOCKD *xd)
{
if ( mv->col < (xd->mb_to_left_edge - LEFT_TOP_MARGIN) )
mv->col = xd->mb_to_left_edge - LEFT_TOP_MARGIN;
else if ( mv->col > xd->mb_to_right_edge + RIGHT_BOTTOM_MARGIN )
mv->col = xd->mb_to_right_edge + RIGHT_BOTTOM_MARGIN;
if ( mv->row < (xd->mb_to_top_edge - LEFT_TOP_MARGIN) )
mv->row = xd->mb_to_top_edge - LEFT_TOP_MARGIN;
else if ( mv->row > xd->mb_to_bottom_edge + RIGHT_BOTTOM_MARGIN )
mv->row = xd->mb_to_bottom_edge + RIGHT_BOTTOM_MARGIN;
}In vp8_find_near_mvs(), the "nearest" and "near" vectors are what the matching modes use 🫶
The vector best_mv is the base that explicitly coded motion vectors get built on top of 🧱
The first three values in the return value cnt are (in order) weighted census values for "zero", "nearest", and "near" vectors 📊. The last value tells us how much SPLITMV showed up in the neighbouring macroblocks 🔥. The biggest possible "weight" in each case is 5 💀
void vp8_find_near_mvs
(
MACROBLOCKD *xd,
const MODE_INFO *here,
MV *nearest,
MV *near,
MV *best_mv,
int cnt[4],
int refframe,
int * ref_frame_sign_bias
)
{
const MODE_INFO *above = here - xd->mode_info_stride;
const MODE_INFO *left = here - 1;
const MODE_INFO *aboveleft = above - 1;
int_mv near_mvs[4];
int_mv *mv = near_mvs;
int *cntx = cnt;
enum {CNT_ZERO, CNT_NEAREST, CNT_NEAR, CNT_SPLITMV};
/* Zero accumulators */
mv[0].as_int = mv[1].as_int = mv[2].as_int = 0;
cnt[0] = cnt[1] = cnt[2] = cnt[3] = 0;
/* Process above */
if (above->mbmi.ref_frame != INTRA_FRAME) {
if (above->mbmi.mv.as_int) {
(++mv)->as_int = above->mbmi.mv.as_int;
mv_bias(above, refframe, mv, ref_frame_sign_bias);
++cntx;
}
*cntx += 2;
}
/* Process left */
if (left->mbmi.ref_frame != INTRA_FRAME) {
if (left->mbmi.mv.as_int) {
int_mv this_mv;
this_mv.as_int = left->mbmi.mv.as_int;
mv_bias(left, refframe, &this_mv, ref_frame_sign_bias);
if (this_mv.as_int != mv->as_int) {
(++mv)->as_int = this_mv.as_int;
++cntx;
}
*cntx += 2;
} else
cnt[CNT_ZERO] += 2;
}
/* Process above left */
if (aboveleft->mbmi.ref_frame != INTRA_FRAME) {
if (aboveleft->mbmi.mv.as_int) {
int_mv this_mv;
this_mv.as_int = aboveleft->mbmi.mv.as_int;
mv_bias(aboveleft, refframe, &this_mv,
ref_frame_sign_bias);
if (this_mv.as_int != mv->as_int) {
(++mv)->as_int = this_mv.as_int;
++cntx;
}
*cntx += 1;
} else
cnt[CNT_ZERO] += 1;
}
/* If we have three distinct MVs ... */
if (cnt[CNT_SPLITMV]) {
/* See if above-left MV can be merged with NEAREST */
if (mv->as_int == near_mvs[CNT_NEAREST].as_int)
cnt[CNT_NEAREST] += 1;
}
cnt[CNT_SPLITMV] = ((above->mbmi.mode == SPLITMV)
+ (left->mbmi.mode == SPLITMV)) * 2
+ (aboveleft->mbmi.mode == SPLITMV);
/* Swap near and nearest if necessary */
if (cnt[CNT_NEAR] > cnt[CNT_NEAREST]) {
int tmp;
tmp = cnt[CNT_NEAREST];
cnt[CNT_NEAREST] = cnt[CNT_NEAR];
cnt[CNT_NEAR] = tmp;
tmp = near_mvs[CNT_NEAREST].as_int;
near_mvs[CNT_NEAREST].as_int = near_mvs[CNT_NEAR].as_int;
near_mvs[CNT_NEAR].as_int = tmp;
}
/* Use near_mvs[0] to store the "best" MV */
if (cnt[CNT_NEAREST] >= cnt[CNT_ZERO])
near_mvs[CNT_ZERO] = near_mvs[CNT_NEAREST];
/* Set up return values */
*best_mv = near_mvs[0].as_mv;
*nearest = near_mvs[CNT_NEAREST].as_mv;
*near = near_mvs[CNT_NEAR].as_mv;
vp8_clamp_mv(nearest, xd);
vp8_clamp_mv(near, xd);
vp8_clamp_mv(best_mv, xd); //TODO: Move this up before
the copy
}Next up, the mv_ref probability table (mv_ref_p) gets derived from that census 🎰
const int vp8_mode_contexts[6][4] =
{
{ 7, 1, 1, 143, },
{ 14, 18, 14, 107, },
{ 135, 64, 57, 68, },
{ 60, 56, 128, 65, },
{ 159, 134, 128, 34, },
{ 234, 188, 128, 28, },
}vp8_prob *vp8_mv_ref_probs(vp8_prob mv_ref_p[VP8_MVREFS-1],
int cnt[4])
{
mv_ref_p[0] = vp8_mode_contexts [cnt[0]] [0];
mv_ref_p[1] = vp8_mode_contexts [cnt[1]] [1];
mv_ref_p[2] = vp8_mode_contexts [cnt[2]] [2];
mv_ref_p[3] = vp8_mode_contexts [cnt[3]] [3];
return p;
}Once mv_ref_p is set up, the mv_ref gets decoded as usual 🥰
mvr = (mv_ref) treed_read(d, mv_ref_tree, mv_ref_p);For the first four inter-coding modes, the same motion vector is used for all the Y subblocks 🍓 The first three modes use an implicit motion vector ✨
+------------+------------------------------------------------------+
| Mode | Instruction |
+------------+------------------------------------------------------+
| mv_nearest | Use the nearest vector returned by |
| | vp8_find_near_mvs. |
| | |
| mv_near | Use the near vector returned by vp8_find_near_mvs. |
| | |
| mv_zero | Use a zero vector; that is, predict the current |
| | macroblock from the corresponding macroblock in the |
| | prediction frame. |
| | |
| NEWMV | This mode is followed by an explicitly coded motion |
| | vector (the format of which is described in the next |
| | section) that is added (component-wise) to the |
| | best_mv reference vector returned by find_near_mvs |
| | and applied to all 16 subblocks. |
+------------+------------------------------------------------------+16.4. Split Prediction ✨
okay so SPLITMV is the mode that gets extra spicy 🔥 It means multiple vectors get applied to the Y subblocks instead of just one for the whole macroblock. Right after it comes a partition specification that decides how many vectors you'll get and which subblocks each one covers. Here are the possible partitions, with their subdivisions and coding tree 💅
typedef enum
{
mv_top_bottom, /* two pieces {0...7} and {8...15} */
mv_left_right, /* {0,1,4,5,8,9,12,13} and
{2,3,6,7,10,11,14,15} */
mv_quarters, /* {0,1,4,5}, {2,3,6,7}, {8,9,12,13},
{10,11,14,15} */
MV_16, /* every subblock gets its own vector
{0} ... {15} */
mv_num_partitions
}
MVpartition;
const tree_index mvpartition_tree [2 * (mvnum_partition - 1)] =
{
-MV_16, 2, /* MV_16 = "0" */
-mv_quarters, 4, /* mv_quarters = "10" */
-mv_top_bottom, -mv_left_right /* top_bottom = "110",
left_right = "111" */
};The partition gets decoded using a fixed, constant probability table, so there's no context drama here 🧚
const Prob mvpartition_probs [mvnum_partition - 1] =
{ 110, 111, 150};
part = (MVpartition) treed_read(d, mvpartition_tree,
mvpartition_probs);After the partition, you get two (for mv_top_bottom or mv_left_right), four (for mv_quarters), or sixteen (for MV_16) subblock inter-prediction modes 🦋 They show up in the order the partition layouts list them (those are the comments on the MVpartition enum), and they're coded like this. Just like with the macroblock-level modes, the enumeration is offset so one variable can hold either an intra- or inter-subblock mode without any mix-ups 💜
Before decoding each subblock, a decoding tree context gets picked, as shown in the snippet below. It's based on the subblock directly to the left and the one directly above, and on whether those two are equal, zero, or some combo of both 🌸
typedef enum
{
LEFT4x4 = num_intra_bmodes, /* use already-coded MV to
my left */
ABOVE4x4, /* use already-coded MV above me */
ZERO4x4, /* use zero MV */
NEW4x4, /* explicit offset from "best" */
num_sub_mv_ref
};
sub_mv_ref;
const tree_index sub_mv_ref_tree [2 * (num_sub_mv_ref - 1)] =
{
-LEFT4X4, 2, /* LEFT = "0" */
-ABOVE4X4, 4, /* ABOVE = "10" */
-ZERO4X4, -NEW4X4 /* ZERO = "110", NEW = "111" */
};
/* Choose correct decoding tree context
* Function parameters are left subblock neighbor MV and above
* subblock neighbor MV */
int vp8_mvCont(MV *l, MV*a)
{
int lez = (l->row == 0 && l->col == 0); /* left neighbor
is zero */
int aez = (a->row == 0 && a->col == 0); /* above neighbor
is zero */
int lea = (l->row == a->row && l->col == a->col); /* left
neighbor equals above neighbor */
if (lea && lez)
return SUBMVREF_LEFT_ABOVE_ZED; /* =4 */
if (lea)
return SUBMVREF_LEFT_ABOVE_SAME; /* =3 */
if (aez)
return SUBMVREF_ABOVE_ZED; /* =2 */
if (lez)
return SUBMVREF_LEFT_ZED; /* =1*/
return SUBMVREF_NORMAL; /* =0 */
}
/* Constant probabilities and decoding procedure. */
const Prob sub_mv_ref_prob [5][num_sub_mv_ref - 1] = {
{ 147,136,18 },
{ 106,145,1 },
{ 179,121,1 },
{ 223,1 ,34 },
{ 208,1 ,1 }
};
sub_ref = (sub_mv_ref) treed_read(d, sub_mv_ref_tree,
sub_mv_ref_prob[context]);So the first two sub-prediction modes are basically copy-paste moves 📋 They just grab the motion vectors already coded for the blocks above and to the left of the top-left subblock in the current subset (aka the group of subblocks being predicted). Those neighbour blocks don't have to live inside the current macroblock, and if your subset sits on the top or left edge of the frame, they don't have to be inside the frame either 🙈 When that happens, their motion vectors count as zero, and so do subblock motion vectors inside an intra-predicted macroblock. Also, to keep prediction correct inside this macroblock, every subblock in an already-decoded subset of the current macroblock needs its motion vector set 💖
ZERO4x4 uses a zero motion vector and predicts the current subset from the matching subset in the prediction frame. Simple and iconic 🌈
NEW4x4 is basically NEWMV, except it only applies to the current subset ✨ It gets followed by a two-dimensional motion vector offset (we'll get to that in the next section), which is added to the best vector returned by the earlier call to find_near_mvs. The sum becomes the motion vector in effect for that subset.
If you want to see how inter-prediction modes and motion vectors get parsed, check out the reference decoder file modemv.c (Section 20.11) 🎀
17. Motion Vector Decoding 🦋
okay so as we talked about earlier, motion vectors show up in two places in the VP8 datastream. One is applied to whole macroblocks in NEWMV mode, and the other is applied to subsets of macroblocks in NEW4x4 mode. The format of the vectors is identical in both cases, which is honestly so clean 😭
Each vector has two pieces: a vertical component (row), followed by a horizontal component (column). The row and column use separate coding probabilities, but they're represented in exactly the same way otherwise 💜
17.1. Coding of Each Component
Each component is a signed integer V that represents a vertical or horizontal luma displacement of V quarter-pixels (and a chroma displacement of V eighth-pixels) 🌸 If the absolute value of V is non-zero, it gets followed by a boolean sign. V can be any value from -1023 up to +1023, inclusive.
The absolute value A gets coded one of two ways, depending on its size 🍓 For 0 <= A <= 7, A is tree-coded. For 8 <= A <= 1023, the bits in the binary expansion of A are each coded using independent boolean probabilities. The coding of A starts with a bool that tells you which range is in effect.
Decoding a motion vector component needs a 19-position probability table. Its offsets, along with the procedure used to decode components, look like this 👑
typedef enum
{
mvpis_short, /* short (<= 7) vs long (>= 8) */
MVPsign, /* sign for non-zero */
MVPshort, /* 8 short values = 7-position tree */
MVPbits = MVPshort + 7, /* 8 long value bits
w/independent probs */
MVPcount = MVPbits + 10 /* 19 probabilities in total */
}
MVPindices;
typedef Prob MV_CONTEXT [MVPcount]; /* Decoding spec for
a single component */
/* Tree used for small absolute values (has expected
correspondence). */
const tree_index small_mvtree [2 * (8 - 1)] =
{
2, 8, /* "0" subtree, "1" subtree */
4, 6, /* "00" subtree, "01" subtree */
-0, -1, /* 0 = "000", 1 = "001" */
-2, -3, /* 2 = "010", 3 = "011" */
10, 12, /* "10" subtree, "11" subtree */
-4, -5, /* 4 = "100", 5 = "101" */
-6, -7 /* 6 = "110", 7 = "111" */
};
/* Read MV component at current decoder position, using
supplied probs. */
int read_mvcomponent(bool_decoder *d, const MV_CONTEXT *mvc)
{
const Prob * const p = (const Prob *) mvc;
int A = 0;
if (read_bool(d, p [mvpis_short])) /* 8 <= A <= 1023 */
{
/* Read bits 0, 1, 2 */
int i = 0;
do { A += read_bool(d, p [MVPbits + i]) << i;}
while (++i < 3);
/* Read bits 9, 8, 7, 6, 5, 4 */
i = 9;
do { A += read_bool(d, p [MVPbits + i]) << i;}
while (--i > 3);
/* We know that A >= 8 because it is coded long,
so if A <= 15, bit 3 is one and is not
explicitly coded. */
if (!(A & 0xfff0) || read_bool(d, p [MVPbits + 3]))
A += 8;
}
else /* 0 <= A <= 7 */
A = treed_read(d, small_mvtree, p + MVPshort);
return A && read_bool(r, p [MVPsign]) ? -A : A;
}17.2. Probability Updates 💖
okay so the decoder keeps an array of two MV_CONTEXTs 💅, one for the row component and one for the column component. bestie, both of these get reset to their defaults on every key frame 🔑. Each individual probability can be updated on every interframe (that's field J of the frame header 🎀) using a constant table of update probabilities. Every optional update is written as B? P(7), so it's a bool, and if that bool is true you get a 7-bit probability spec after it 🦋.
Just like the other dynamic probabilities in VP8, these updates stick around until the next key frame or until another update replaces them ✨.
here's how the probabilities get managed, step by step 👑:
/* Never-changing table of update probabilities for each
individual probability used in decoding motion vectors. */
const MV_CONTEXT vp8_mv_update_probs[2] =
{
{
237,
246,
253, 253, 254, 254, 254, 254, 254,
254, 254, 254, 254, 254, 250, 250, 252, 254, 254
},
{
231,
243,
245, 253, 254, 254, 254, 254, 254,
254, 254, 254, 254, 254, 251, 251, 254, 254, 254
}
};
/* Default MV decoding probabilities. */
const MV_CONTEXT default_mv_context[2] =
{
{ // row
162, // is short
128, // sign
225, 146, 172, 147, 214, 39, 156, // short tree
128, 129, 132, 75, 145, 178, 206, 239, 254, 254 // long bits
},
{ // same for column
164, // is short
128,
204, 170, 119, 235, 140, 230, 228,
128, 130, 130, 74, 148, 180, 203, 236, 254, 254 // long bits
}
};
/* Current MV decoding probabilities, set to above defaults
every key frame. */
MV_CONTEXT mvc [2]; /* always row, then column */
/* Procedure for decoding a complete motion vector. */
typedef struct { int16 row, col;} MV; /* as in previous section */
MV read_mv(bool_decoder *d)
{
MV v;
v.row = (int16) read_mvcomponent(d, mvc);
v.col = (int16) read_mvcomponent(d, mvc + 1);
return v;
}
/* Procedure for updating MV decoding probabilities, called
every interframe with "d" at the appropriate position in
the frame header. */
void update_mvcontexts(bool_decoder *d)
{
int i = 0;
do { /* component = row, then column */
const Prob *up = mv_update_probs[i]; /* update probs
for component */
Prob *p = mvc[i]; /* start decode tbl "" */
Prob * const pstop = p + MVPcount; /* end decode tbl "" */
do {
if (read_bool(d, *up++)) /* update this position */
{
const Prob x = read_literal(d, 7);
*p = x? x<<1 : 1;
}
} while (++p < pstop); /* next position */
} while (++i < 2); /* next component */
}and that wraps up the motion-vector decoding procedure 🎉. With it, we're also done with the whole procedure for decoding interframe macroblock prediction records 🥳 we ate that, no crumbs 🍓.
18. Interframe Prediction ✨
okay now we're talking 🧚. Given an inter-prediction spec for the current macroblock, which means a reference frame plus a motion vector for each of the sixteen Y subblocks, we work out the prediction buffer for that macroblock. After that, frame reconstruction is finished by the processes we already covered: residue summation (Section 14) and loop filtering (Section 15) 💜.
The stuff about inter-predicted subblocks and sub-pixel interpolation lives in the reference decoder file predict.c (Section 20.14) 🌸.
18.1. Bounds on, and Adjustment of, Motion Vectors 🦋
bestie, this part is kinda wild 😭. Each motion vector is differentially encoded from a neighboring block or macroblock, and the only clamp is there to make sure the referenced motion vector points to a valid spot inside a reference frame buffer. So technically, VP8 lets a block or macroblock have arbitrarily large motion vectors, up to the size of the input image plus the extended border areas. But for practical reasons, VP8 limits motion vectors to a range of -4096 to 4095 full pixels, no matter how big the image is. (VP8 has 14 raw bits for width and height, so 16383x16383 is the largest possible image size 📐.) Bitstream-compliant encoders and decoders shall enforce this limit 🔒.
Because chroma motion vectors have 1/8-pixel resolution, the synthetic pixel calculation from Section 5 (more on it below) uses that same resolution for luma subblocks too ✨. So the stored luma motion vectors all get doubled, and each component of each luma vector becomes an even integer in the range -2046 to +2046, inclusive 🍓.
The vector for each chroma subblock comes from averaging the vectors of the 4 luma subblocks that cover the same visible area in the usual correspondence. So U and V block 0 uses the average of Y subblocks { 0, 1, 4, 5}, chroma block 1 matches Y blocks { 2, 3, 6, 7}, chroma block 2 matches { 8, 9, 12, 13}, and chroma block 3 matches { 10, 11, 14, 15} 🧩.
here's the exact math, where each of the two components of each chroma subblock vector is worked out from the matching luma components 👑:
int avg(int c1, int c2, int c3, int c4)
{
int s = c1 + c2 + c3 + c4;
/* The shift divides by 8 (not 4) because chroma pixels
have twice the diameter of luma pixels. The handling
of negative motion vector components is slightly
cumbersome because, strictly speaking, right shifts
of negative numbers are not well-defined in C. */
return s >= 0 ? (s + 4) >> 3 : -((-s + 4) >> 3);
}Also, if the version number in the frame tag says the stream only has full-pel chroma motion vectors, then the fractional parts of both vector components get truncated to zero 🫶. Here's the pseudocode for that, assuming 3 bits of fraction for both luma and chroma vectors:
x = x & (~7);
y = y & (~7);Earlier in this doc we talked about the vp8_clamp_mv() function, which keeps "nearest" and "near" motion vector predictors inside set margins around the frame boundaries 🧱. NEWMV macroblocks get an extra clamp too 💅: after the "best" predictor and the differential vector from the stream get combined, the final motion vector is clamped again.
⚠️ bestie, SPLITMV macroblocks do NOT get that secondary clamp 😭. That means any subblock's motion vector inside a SPLITMV macroblock can point outside the clamping zone 💀. And these unclamped vectors are also used when figuring out the decoding tree context for later subblocks' modes, in the vp8_mvCont() function 🌈.
18.2. Prediction Subblocks 🎀
okay so for each subblock, the prediction goes like this 💖. First you temporarily ignore the fractional part of the motion vector. You round "up" or "left" by right-shifting each component 3 bits with sign propagation 🦋. Then you add the origin (upper left position) of the current macroblock, which is 16x16 for luma or 8x8 for chroma. That gives you an origin in the Y, U, or V plane of the predictor frame, which is either the golden frame or the previous frame ✨.
Now treat that origin as the upper left corner of a luma or chroma macroblock. Next, you need to pin down the relative positions of the pixels tied to that subblock, meaning any pixels that might get pulled into the sub-pixel interpolation for that subblock 🧚💜.
18.3. Sub-Pixel Interpolation ✨
okay so sub-pixel interpolation is done with two one-dimensional convolutions 💅 You can picture them working on a 2D grid of pixels whose origin is the subblock origin. That's the origin of the prediction macroblock plus the offset to the subblock 🦋 Since motion vectors can point anywhere, these "prediction subblock origins" can land anywhere too, which is kinda wild 😭
The integer part of the motion vector gets folded into the origin of the prediction subblock. The 16 (synthetic) pixels we need to build are each given by an offset from that origin. The integer part of each offset is how far that pixel sits from the subblock origin, counted using the vertical stride 📏 To those integer parts we add one constant fractional part. That fraction is just the gap between the real motion vector and its integer truncation (the one used to find the prediction macroblock and subblock origins). Each part of this fraction is an integer from 0 to 7, which means a forward displacement in eighths of a pixel 🍓
These fractions decide how the filtering goes 🔥 If both are zero (so we've got a "whole pixel" motion vector), the prediction subblock just gets copied straight into the matching spot in the current macroblock's prediction buffer. As we talked about in Section 14, the layout of that buffer can depend on which reconstruction implementation you picked. And the vertical distance between lines of the prediction subblock is always the stride, same as every other vertical distance here 🌸
If at least one fraction isn't zero, we fill in the missing pixels with a horizontal interpolation first, then a vertical one 🧚
Both interpolations are basically the same vibe. Each one uses a filter of at most six taps, and which filter you get depends on that one-dimensional offset. So every calculated pixel looks at at most three pixels before it (above or to the left) and at most three after it (below or to the right) ✨ The horizontal pass has to compute two extra rows above and three extra rows below the 4x4 block, so the vertical pass has enough samples to work with 💜
Which tap set you use, bicubic or bilinear, depends on the reconstruction filter type given in the version number field of the frame tag 🎀
Here's the exact subsampling code, no changes:
---- Begin code block --------------------------------------
/* Filter taps taken to 7-bit precision.
Because DC is always passed, taps always sum to 128. */
const int BilinearFilters[8][6] =
{
{ 0, 0, 128, 0, 0, 0 },
{ 0, 0, 112, 16, 0, 0 },
{ 0, 0, 96, 32, 0, 0 },
{ 0, 0, 80, 48, 0, 0 },
{ 0, 0, 64, 64, 0, 0 },
{ 0, 0, 48, 80, 0, 0 },
{ 0, 0, 32, 96, 0, 0 },
{ 0, 0, 16, 112, 0, 0 }
};
const int filters [8] [6] = { /* indexed by displacement */
{ 0, 0, 128, 0, 0, 0 }, /* degenerate whole-pixel */
{ 0, -6, 123, 12, -1, 0 }, /* 1/8 */
{ 2, -11, 108, 36, -8, 1 }, /* 1/4 */
{ 0, -9, 93, 50, -6, 0 }, /* 3/8 */
{ 3, -16, 77, 77, -16, 3 }, /* 1/2 is symmetric */
{ 0, -6, 50, 93, -9, 0 }, /* 5/8 = reverse of 3/8 */
{ 1, -8, 36, 108, -11, 2 }, /* 3/4 = reverse of 1/4 */
{ 0, -1, 12, 123, -6, 0 } /* 7/8 = reverse of 1/8 */
};
/* One-dimensional synthesis of a single sample.
Filter is determined by fractional displacement */
Pixel interp(
const int fil[6], /* filter to apply */
const Pixel *p, /* origin (rounded "before") in
prediction area */
const int s /* size of one forward step "" */
) {
int32 a = 0;
int i = 0;
p -= s + s; /* move back two positions */
do {
a += *p * fil[i];
p += s;
} while (++i < 6);
return clamp255((a + 64) >> 7); /* round to nearest
8-bit value */
}
/* First do horizontal interpolation, producing intermediate
buffer. */
void Hinterp(
Pixel temp[9][4], /* 9 rows of 4 (intermediate)
destination values */
const Pixel *p, /* subblock origin in prediction
frame */
int s, /* vertical stride to be used in
prediction frame */
uint hfrac, /* 0 <= horizontal displacement <= 7 */
uint bicubic /* 1=bicubic filter, 0=bilinear */
) {
const int * const fil = bicubic ? filters [hfrac] :
BilinearFilters[hfrac];
int r = 0; do /* for each row */
{
int c = 0; do /* for each destination sample */
{
/* Pixel separation = one horizontal step = 1 */
temp[r][c] = interp(fil, p + c, 1);
}
while (++c < 4);
}
while (p += s, ++r < 9); /* advance p to next row */
}
/* Finish with vertical interpolation, producing final results.
Input array "temp" is of course that computed above. */
void Vinterp(
Pixel final[4][4], /* 4 rows of 4 (final) destination values */
const Pixel temp[9][4],
uint vfrac, /* 0 <= vertical displacement <= 7 */
uint bicubic /* 1=bicubic filter, 0=bilinear */
) {
const int * const fil = bicubic ? filters [vfrac] :
BilinearFilters[vfrac];
int r = 0; do /* for each row */
{
int c = 0; do /* for each destination sample */
{
/* Pixel separation = one vertical step = width
of array = 4 */
final[r][c] = interp(fil, temp[r] + c, 4);
}
while (++c < 4);
}
while (++r < 4);
}
---- End code block ----------------------------------------18.4. Filter Properties 🎀
okay this part is the "why these filters" section 🤓 It's pretty cursory (a real deep dive would need a couple of books, no cap), so if signal processing isn't your thing you can totally skip it 🙈
All digital signals get sampled somehow 📊 The really common case is when the gap between samples (time for audio, space for pixels) is uniform everywhere, and that's super easy to analyze. A lot of this stuff is way easier to understand in the frequency domain using Fourier Analysis, especially the parts of a signal that don't change when you shift it around, particularly when the shift isn't a whole number of samples 🌈
Non-integer shifts of a sampled signal are a textbook example of this. For non-integer motion vectors, we want to know what the underlying image "really is" at those in-between points 👀 We don't have values there, but it still makes sense to talk about them. That only holds if the underlying signal is band-limited, meaning it has no energy in spatial frequencies the pixel grid can't faithfully show ✨ For one dimension, that range of "OK" frequencies is the Nyquist band. For our 2D integer-grid samples, we can call it a Nyquist rectangle. The finer the grid, the more we know about the image, and the wider that rectangle gets 💖
For band-limited signals, there actually is an exact formula that gives the right sample value at any point 🤯 The catch is that it needs every single sample in the image, and it needs infinite precision 😵 Also, strictly speaking, band-limited signals stretch out infinitely in space (or time), so everything here is really an approximation 🫶
The theoretically correct subsampling, and any approximation of it, is always a translation-invariant weighted sum (a filter) like the one VP8 uses 💅 The reconstruction error from such a filter can be written as a multiplier in the frequency domain. Basically, the filter multiplies the Fourier transform of whatever signal you feed it by a fixed function tied to that filter. That fixed function is called the frequency response (or transfer function). The ideal subsampling filter has a response of one inside the Nyquist rectangle and zero everywhere else 🦋
Another key thing: the wider the sub-rectangle of spatial frequencies (inside the Nyquist rectangle) you want to "pass" (correctly render), or, to say it more precisely, the closer you want to get to the ideal transfer function, the more original samples the subsampling has to look at, and the more calculation precision you need 🔢
The VP8 filters were picked to do the best job they could within the limits of 4 or 6 taps and 7-bit precision 💜 They handle the low spatial frequencies near the 0th DC frequency really well, and they introduce no resonances (spots where the absolute value of the frequency response goes above one) 🧚
Why care about resonances? Two reasons 😤 First, resonances can make really gross visible artifacts when, as often happens in real compressed video, filters get applied over and over. Second, most of the energy in real-world images sits near DC, not at the high end 🌸
To be a bit more specific: the VP8 filters are the best resonance-free 4- or 6-tap filters possible, where "best" is about the frequency response near the origin. The response at 0 has to be 1, and the response curve at 0 should be as flat as it can be 👑
For a more obvious point of reference, the "best" 2-tap filter is just simple linear interpolation between the surrounding real pixels 💖
Last thing! Because of how motion vectors get calculated, the shorter 4-tap filters (used for odd fractional displacements) only get applied to the chroma plane 🎀 Human color perception is notoriously bad, especially with higher spatial frequencies, so the shorter filters are easier to understand mathematically, and the gap between them and a theoretically slightly better 6-tap filter is negligible for chroma. Slay the efficiency, periodt 💅✨
19. Annex A: Bitstream Syntax ✨
okay bestie, this annex is the cheat sheet 📋💖 Everything we've covered in the earlier sections gets collected here in table form so you can look stuff up fast. Each syntax element gets a quick description after the table, with a pointer back to the paragraph in the main doc where it's explained. We're not repeating what each value means here, so you'll need to go back for that 🫶
The top-level hierarchy of the bitstream lives in Section 4 🧚 The coding types for syntax elements are in Section 8 🌸 Here's what the types in this annex mean:
f(n): an n-bit value read straight from the stream (n successive bits, not boolean encoded) 💅L(n): an n-bit number encoded as n booleans, each with equal probability of being 0 or 1 🎀B(p): a bool with probability p of being 0 🦋T: a tree-encoded value 🌈
19.1. Uncompressed Data Chunk 🍓
Here's the table for the uncompressed data chunk. The frame tag is 3 bytes, and if it's a key frame, the start code and size fields come after it 💜
| Frame Tag | Type |
| ------------------------------------------------- | ----- |
| frame_tag | f(24) |
| if (key_frame) { | |
| start_code | f(24) |
| horizontal_size_code | f(16) |
| vertical_size_code | f(16) |
| } | |The 3-byte frame tag can be parsed like this 👑
---- Begin code block --------------------------------------
unsigned char *c = pbi->source;
unsigned int tmp;
tmp = (c[2] << 16) | (c[1] << 8) | c[0];
key_frame = tmp & 0x1;
version = (tmp >> 1) & 0x7;
show_frame = (tmp >> 4) & 0x1;
first_part_size = (tmp >> 5) & 0x7FFFF;
---- End code block ----------------------------------------Here's what each piece means 🔥
key_frametells you whether the current frame is a key frame or not.versiondetermines the bitstream version.show_frametells you whether the current frame is meant to be displayed or not.first_part_sizedetermines the size of the first partition (control partition), excluding the uncompressed data chunk.
The start_code is a constant 3-byte pattern with value 0x9d012a ✨ The rest of the uncompressed chunk, after the start code, gets parsed like this 🌸
---- Begin code block --------------------------------------
unsigned char *c = pbi->source + 6;
unsigned int tmp;
tmp = (c[1] << 8) | c[0];
width = tmp & 0x3FFF;
horizontal_scale = tmp >> 14;
tmp = (c[3] << 8) | c[2];
height = tmp & 0x3FFF;
vertical_scale = tmp >> 14;
---- End code block ----------------------------------------19.2. Frame Header 🎀
This is the big one, bestie 😭 The frame header table is long, and every field has its own little job. Here it is in full 💅
| Frame Header | Type |
| ------------------------------------------------- | ----- |
| if (key_frame) { | |
| color_space | L(1) |
| clamping_type | L(1) |
| } | |
| segmentation_enabled | L(1) |
| if (segmentation_enabled) | |
| update_segmentation() | |
| filter_type | L(1) |
| loop_filter_level | L(6) |
| sharpness_level | L(3) |
| mb_lf_adjustments() | |
| log2_nbr_of_dct_partitions | L(2) |
| quant_indices() | |
| if (key_frame) | |
| refresh_entropy_probs | L(1) |
| else { | |
| refresh_golden_frame | L(1) |
| refresh_alternate_frame | L(1) |
| if (!refresh_golden_frame) | |
| copy_buffer_to_golden | L(2) |
| if (!refresh_alternate_frame) | |
| copy_buffer_to_alternate | L(2) |
| sign_bias_golden | L(1) |
| sign_bias_alternate | L(1) |
| refresh_entropy_probs | L(1) |
| refresh_last | L(1) |
| } | |
| token_prob_update() | |
| mb_no_skip_coeff | L(1) |
| if (mb_no_skip_coeff) | |
| prob_skip_false | L(8) |
| if (!key_frame) { | |
| prob_intra | L(8) |
| prob_last | L(8) |
| prob_gf | L(8) |
| intra_16x16_prob_update_flag | L(1) |
| if (intra_16x16_prob_update_flag) { | |
| for (i = 0; i < 4; i++) | |
| intra_16x16_prob | L(8) |
| } | |
| intra_chroma prob_update_flag | L(1) |
| if (intra_chroma_prob_update_flag) { | |
| for (i = 0; i < 3; i++) | |
| intra_chroma_prob | L(8) |
| } | |
| mv_prob_update() | |
| } | |Now let's decode what each element does 🦋
color_spacedefines the YUV color space of the sequence (Section 9.2) 🌈clamping_typespecifies if the decoder is required to clamp the reconstructed pixel values (Section 9.2) 💅segmentation_enabledenables the segmentation feature for the current frame (Section 9.3) ✨filter_typedetermines whether the normal or the simple loop filter is used (Sections 9.4, 15) 🎀loop_filter_levelcontrols the deblocking filter (Sections 9.4, 15) 🍓sharpness_levelcontrols the deblocking filter (Sections 9.4, 15) 🌸log2_nbr_of_dct_partitionsdetermines the number of separate partitions containing the DCT coefficients of the macroblocks (Section 9.5) 🧚refresh_entropy_probsdetermines whether updated token probabilities are used only for this frame or until further update 💜refresh_golden_framedetermines if the current decoded frame refreshes the golden frame (Section 9.7) 👑refresh_alternate_framedetermines if the current decoded frame refreshes the alternate reference frame (Section 9.7) 🔥copy_buffer_to_goldendetermines if the golden reference is replaced by another reference (Section 9.7) 🌈copy_buffer_to_alternatedetermines if the alternate reference is replaced by another reference (Section 9.7) 💅sign_bias_goldencontrols the sign of motion vectors when the golden frame is referenced (Section 9.7) ✨sign_bias_alternatecontrols the sign of motion vectors when the alternate frame is referenced (Section 9.7) 🎀refresh_lastdetermines if the current decoded frame refreshes the last frame reference buffer (Section 9.8) 🍓mb_no_skip_coeffenables or disables the skipping of macroblocks containing no non-zero coefficients (Section 9.10) 🌸prob_skip_falseindicates the probability that the macroblock is not skipped (flag indicating skipped macroblock is false) (Section 9.10) 🧚prob_intraindicates the probability of an intra macroblock (Section 9.10) 💜prob_lastindicates the probability that the last reference frame is used for inter-prediction (Section 9.10) 👑prob_gfindicates the probability that the golden reference frame is used for inter-prediction (Section 9.10) 🔥intra_16x16_prob_update_flagindicates if the branch probabilities used in the decoding of the luma intra-prediction mode are updated (Section 9.10) 🌈intra_16x16_probindicates the branch probabilities of the luma intra-prediction mode decoding tree 💅intra_chroma_prob_update_flagindicates if the branch probabilities used in the decoding of the chroma intra-prediction mode are updated (Section 9.10) ✨intra_chroma_probindicates the branch probabilities of the chroma intra-prediction mode decoding tree 🎀
The segmentation table comes next, and it's also a whole vibe 🦋 It's pulled in by update_segmentation() whenever segmentation is turned on:
| update_segmentation() | Type |
| ------------------------------------------------- | ----- |
| update_mb_segmentation_map | L(1) |
| update_segment_feature_data | L(1) |
| if (update_segment_feature_data) { | |
| segment_feature_mode | L(1) |
| for (i = 0; i < 4; i++) { | |
| quantizer_update | L(1) |
| if (quantizer_update) { | |
| quantizer_update_value | L(7) |
| quantizer_update_sign | L(1) |
| } | |
| } | |
| for (i = 0; i < 4; i++) { | |
| loop_filter_update | L(1) |
| if (loop_filter_update) { | |
| lf_update_value | L(6) |
| lf_update_sign | L(1) |
| } | |
| } | |
| } | |
| if (update_mb_segmentation_map) { | |
| for (i = 0; i < 3; i++) { | |
| segment_prob_update | L(1) |
| if (segment_prob_update) | |
| segment_prob | L(8) |
| } | |
| } | |And here's what those segmentation elements do 🌸
update_mb_segmentation_mapdetermines if the MB segmentation map is updated in the current frame (Section 9.3) 💖update_segment_feature_dataindicates if the segment feature data is updated in the current frame (Section 9.3) 🧚segment_feature_modeindicates the feature data update mode, 0 for delta and 1 for the absolute value (Section 9.3) 👑quantizer_updateindicates if the quantizer value is updated for the i^(th) segment (Section 9.3) 💅quantizer_update_valueindicates the update value for the segment quantizer (Section 9.3) 🔥quantizer_update_signindicates the update sign for the segment quantizer (Section 9.3) 🌈loop_filter_updateindicates if the loop filter level value is updated for the i^(th) segment (Section 9.3) ✨lf_update_valueindicates the update value for the loop filter level (Section 9.3) 🎀lf_update_signindicates the update sign for the loop filter level (Section 9.3) 🍓segment_prob_updateindicates whether the branch probabilities used to decode the segment_id in the MB header are decoded from the stream or use the default value of 255 (Section 9.3) 🦋
That's the whole annex, bestie! 🌈💖
segment_probholds the branch probabilities for the segment_id decoding tree 🌸 (Section 9.3)
okay so next up is the loop filter adjustment table, mb_lf_adjustments() 🎀 it's where the per-macroblock loop filter tweaks get switched on and updated 💅
mb_lf_adjustments() | Type |
| ------------------------------------------------- | ----- |
| loop_filter_adj_enable | L(1) |
| if (loop_filter_adj_enable) { | |
| mode_ref_lf_delta_update | L(1) |
| if (mode_ref_lf_delta_update) { | |
| for (i = 0; i < 4; i++) { | |
| ref_frame_delta_update_flag | L(1) |
| if (ref_frame_delta_update_flag) { | |
| delta_magnitude | L(6) |
| delta_sign | L(1) |
| } | |
| } | |
| for (i = 0; i < 4; i++) { | |
| mb_mode_delta_update_flag | L(1) |
| if (mb_mode_delta_update_flag) { | |
| delta_magnitude | L(6) |
| delta_sign | L(1) |
| } | |
| } | |
| } | |
| } | |Wait, I need the table exact, including the top border line. Let me redo the fenced block carefully with the first row included.
loop_filter_adj_enablesays whether the MB-level loop filter adjustment (based on the used reference frame and coding mode) is on for the current frame 🦋 (Section 9.4)mode_ref_lf_delta_updatesays whether the delta values used in an adjustment get updated in the current frame ✨ (Section 9.4)ref_frame_delta_update_flagsays whether the adjustment delta value for a certain used reference frame gets updated 💜 (Section 9.4)delta_magnitudeis the absolute value of the delta value 🍓delta_signis the sign of the delta value 🧚mb_mode_delta_update_flagsays whether the adjustment delta value for a certain MB prediction mode gets updated 🌈 (Section 9.4)
Now quant_indices table, with the bullets.
Then token_prob_update and mv_prob_update.
Final check: no caps key words. "delta" fine. Don't sign off.``` mb_lf_adjustments() | Type |
| loop_filter_adj_enable | L(1) |
|---|---|
| if (loop_filter_adj_enable) { | |
| mode_ref_lf_delta_update | L(1) |
| if (mode_ref_lf_delta_update) { | |
| for (i = 0; i < 4; i++) { | |
| ref_frame_delta_update_flag | L(1) |
| if (ref_frame_delta_update_flag) { | |
| delta_magnitude | L(6) |
| delta_sign | L(1) |
| } | |
| } | |
| for (i = 0; i < 4; i++) { | |
| mb_mode_delta_update_flag | L(1) |
| if (mb_mode_delta_update_flag) { | |
| delta_magnitude | L(6) |
| delta_sign | L(1) |
| } | |
| } | |
| } | |
| } |
- `loop_filter_adj_enable` says whether the MB-level loop filter adjustment (based on the used reference frame and coding mode) is on for the current frame 🦋 (Section 9.4)
- `mode_ref_lf_delta_update` says whether the delta values used in an adjustment get updated in the current frame ✨ (Section 9.4)
- `ref_frame_delta_update_flag` says whether the adjustment delta value for a certain used reference frame gets updated 💜 (Section 9.4)
- `delta_magnitude` is the absolute value of the delta value 🍓
- `delta_sign` is the sign of the delta value 🧚
- `mb_mode_delta_update_flag` says whether the adjustment delta value for a certain MB prediction mode gets updated 🌈 (Section 9.4)
okay now the quantizer table, `quant_indices()`, which is where the dequantization indices get set up 🫶 it's a whole family of "present" flags, each with its own magnitude and sign 💅
quant_indices() | Type |
| y_ac_qi | L(7) |
|---|---|
| y_dc_delta_present | L(1) |
| if (y_dc_delta_present) { | |
| y_dc_delta_magnitude | L(4) |
| y_dc_delta_sign | L(1) |
| } | |
| y2_dc_delta_present | L(1) |
| if (y2_dc_delta_present) { | |
| y2_dc_delta_magnitude | L(4) |
| y2_dc_delta_sign | L(1) |
| } | |
| y2_ac_delta_present | L(1) |
| if (y2_ac_delta_present) { | |
| y2_ac_delta_magnitude | L(4) |
| y2_ac_delta_sign | L(1) |
| } | |
| uv_dc_delta_present | L(1) |
| if (uv_dc_delta_present) { | |
| uv_dc_delta_magnitude | L(4) |
| uv_dc_delta_sign | L(1) |
| } | |
| uv_ac_delta_present | L(1) |
| if (uv_ac_delta_present) { | |
| uv_ac_delta_magnitude | L(4) |
| uv_ac_delta_sign | L(1) |
| } |
- `y_ac_qi` is the dequantization table index used for the luma AC coefficients (and for other coefficient groups if no delta value is present) 🌸 (Section 9.6)
- `y_dc_delta_present` says whether the stream carries a delta value that gets added to the baseline index to get the luma DC coefficient dequantization index 🎀 (Section 9.6)
- `y_dc_delta_magnitude` is the magnitude of that delta value 🦋 (Section 9.6)
- `y_dc_delta_sign` is the sign of that delta value ✨ (Section 9.6)
- `y2_dc_delta_present` says whether the stream carries a delta value that gets added to the baseline index to get the Y2 block DC coefficient dequantization index 💜 (Section 9.6)
- `y2_ac_delta_present` says whether the stream carries a delta value that gets added to the baseline index to get the Y2 block AC coefficient dequantization index 🍓 (Section 9.6)
- `uv_dc_delta_present` says whether the stream carries a delta value that gets added to the baseline index to get the chroma DC coefficient dequantization index 🧚 (Section 9.6)
- `uv_ac_delta_present` says whether the stream carries a delta value that gets added to the baseline index to get the chroma AC coefficient dequantization index 🌈 (Section 9.6)
next is `token_prob_update()` 👑 this one loops over every coefficient branch probability and decides which ones get a fresh value this frame, so it's basically a nested-loop glow-up 🔥
token_prob_update() | Type |
| for (i = 0; i < 4; i++) { | |
|---|---|
| for (j = 0; j < 8; j++) { | |
| for (k = 0; k < 3; k++) { | |
| for (l = 0; l < 11; l++) { | |
| coeff_prob_update_flag | L(1) |
| if (coeff_prob_update_flag) | |
| coeff_prob | L(8) |
| } | |
| } | |
| } | |
| } |
- `coeff_prob_update_flag` says whether the corresponding branch probability is updated in the current frame 💖 (Section 13.4)
- `coeff_prob` is the new branch probability 🌸 (Section 13.4)
and last but not least, `mv_prob_update()` 🦋 the motion vector version of the same vibe, just with its own little loop 🫶
mv_prob_update() | Type |
| for (i = 0; i < 2; i++) { | |
|---|---|
| for (j = 0; j < 19; j++) { | |
| mv_prob_update_flag | L(1) |
| if (mv_prob_update_flag) | |
| prob | L(7) |
| } | |
| } |
- `mv_prob_update_flag` says whether the corresponding MV decoding probability is updated in the current frame ✨ (Section 17.2)
- `prob` is the updated probability 🎀 (Section 17.2)
## 19.3. Macroblock Data ✨
okay so this is where each macroblock's data actually lives 🧚 every macroblock is basically a `macroblock_header()` followed by `residual_data()`, and the header tells you what kind of macroblock you're dealing with before you get to the coefficients 💖
Macroblock Data | Type ------------------------------------------------- | ----- macroblock_header() | residual_data() |
macroblock_header() | Type ------------------------------------------------- | ----- if (update_mb_segmentation_map) | segment_id | T if (mb_no_skip_coeff) | mb_skip_coeff | B(p) if (!key_frame) | is_inter_mb | B(p) if (is_inter_mb) { | mb_ref_frame_sel1 | B(p) if (mb_ref_frame_sel1) | mb_ref_frame_sel2 | B(p) mv_mode | T if (mv_mode == SPLITMV) { | mv_split_mode | T for (i = 0; i < numMvs; i++) { | sub_mv_mode | T if (sub_mv_mode == NEWMV4x4) { | read_mvcomponent() | read_mvcomponent() | } | } | } else if (mv_mode == NEWMV) { | read_mvcomponent() | read_mvcomponent() | } | } else { /* intra mb */ | intra_y_mode | T if (intra_y_mode == B_PRED) { | for (i = 0; i < 16; i++) | intra_b_mode | T } | intra_uv_mode | T } |
here's the vibe for each field 🎀
- <pink>segment_id</pink> tells you which segment the macroblock belongs to (Section 10) 🌸
- <pink>mb_skip_coeff</pink> tells you whether the macroblock has any coded coefficients or not (Section 11.1) 💅
- <pink>is_inter_mb</pink> tells you whether the macroblock is intra- or inter-coded (Section 16) 🦋
- <pink>mb_ref_frame_sel1</pink> picks the reference frame: last frame (0), or golden/alternate (1) (Section 16.2) ✨
- <pink>mb_ref_frame_sel2</pink> picks between golden (0) and alternate reference frame (1) (Section 16.2) 🔥
- <pink>mv_mode</pink> sets the macroblock motion vector mode (Section 16.2) 💜
- <pink>mv_split_mode</pink> gives the partitioning spec for the macroblock and decides how many motion vectors you get (numMvs) (Section 16.2) 👑
- <pink>sub_mv_mode</pink> sets the sub-macroblock motion vector mode for macroblocks using SPLITMV (Section 16.2) 🌈
- <pink>intra_y_mode</pink> picks the luminance intra-prediction mode (Section 16.1) ⭐
- <pink>intra_b_mode</pink> picks the sub-macroblock luminance prediction mode for macroblocks using B_PRED (Section 16.1) 🍓
- <pink>intra_uv_mode</pink> picks the chrominance intra-prediction mode (Section 16.1) 🫶
residual_data() | Type ------------------------------------------------- | ----- if (!mb_skip_coeff) { | if ( (is_inter_mb && mv_mode != SPLITMV) || | (!is_inter_mb && intra_y_mode != B_PRED) ) | residual_block() /* Y2 / | for (i = 0; i < 24; i++) | residual_block() / 16 Y, 4 U, 4 V */ | } |
residual_block() | Type ------------------------------------------------- | ----- for (i = firstCoeff; i < 16; i++) { | token | T if (token == EOB) break; | if (token_has_extra_bits) | extra_bits | L(n) if (coefficient != 0) | sign | L(1) } |
and the residual bits, ngl, are a whole mood 😭
- <hotpink>firstCoeff</hotpink> is 1 for luma blocks of macroblocks containing a Y2 subblock, and 0 otherwise 💖
- <hotpink>token</hotpink> defines the value of the coefficient, the value range of the coefficient, or the end of block (Section 13.2) 🎀
- <hotpink>extra_bits</hotpink> determines the value of the coefficient within the value range the token defines (Section 13.2) ✨
- <hotpink>sign</hotpink> indicates the sign of the coefficient (Section 13.2) 🦋
---
# 20. Attachment One: Reference Decoder Source Code 👑
this part is the actual reference decoder code, so it's all verbatim, no vibes applied 🫶
## 20.1. bit_ops.h 🌸
---- Begin code block --------------------------------------
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #ifndef BIT_OPS_H #define BIT_OPS_H
/* Evaluates to a mask with n bits set */ #define BITS_MASK(n) ((1<<(n))-1)
/* Returns len bits, with the LSB at position bit */ #define BITS_GET(val, bit, len) (((val)>>(bit))&BITS_MASK(len))
#endif
---- End code block ----------------------------------------
## 20.2. bool_decoder.h 💜
this one is the boolean decoder, and it's giving main character energy 🔥
---- Begin code block --------------------------------------
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. / #ifndef BOOL_DECODER_H #define BOOL_DECODER_H #include <stddef.h> struct bool_decoder { const unsigned char *input; / next compressed data byte / size_t input_len; / length of the input buffer / unsigned int range; / identical to encoder's
- range / unsigned int value; / contains at least 8
- significant bits / int bit_count; / # of bits shifted out of
- value, max 7 */ };
static void init_bool_decoder(struct bool_decoder d, const unsigned char *start_partition, size_t sz) { if (sz >= 2) { d->value = (start_partition[0] << 8) / first 2 input
- bytes */
| start_partition[1]; |
d->input = start_partition + 2; /* ptr to next byte */ d->input_len = sz - 2; } else { d->value = 0; d->input = NULL; d->input_len = 0; }
d->range = 255; /* initial range is full / d->bit_count = 0; / have not yet shifted out any bits */ }
static int bool_get(struct bool_decoder d, int probability) { / range and split are identical to the corresponding values used by the encoder when this bool was written */
unsigned int split = 1 + (((d->range - 1) * probability) >> 8); unsigned int SPLIT = split << 8; int retval; /* will be 0 or 1 / if (d->value >= SPLIT) / encoded a one / { retval = 1; d->range -= split; / reduce range / d->value -= SPLIT; / subtract off left endpoint of
- interval / } else / encoded a zero / { retval = 0; d->range = split; / reduce range, no change in left
- endpoint */ }
while (d->range < 128) /* shift out irrelevant value bits */ { d->value <<= 1; d->range <<= 1;
if (++d->bit_count == 8) /* shift in new bits 8 at a time */ { d->bit_count = 0;
if (d->input_len) { d->value |= *d->input++; d->input_len--; } } }
return retval; }
static int bool_get_bit(struct bool_decoder *br) { return bool_get(br, 128); } static int bool_get_uint(struct bool_decoder *br, int bits) { int z = 0; int bit;
for (bit = bits - 1; bit >= 0; bit--) { z |= (bool_get_bit(br) << bit); }
return z; }
static int bool_get_int(struct bool_decoder *br, int bits) { int z = 0; int bit;
for (bit = bits - 1; bit >= 0; bit--) { z |= (bool_get_bit(br) << bit); }
return bool_get_bit(br) ? -z : z; }
static int bool_maybe_get_int(struct bool_decoder *br, int bits) { return bool_get_bit(br) ? bool_get_int(br, bits) : 0; }
static int bool_read_tree(struct bool_decoder *bool, const int *t, const unsigned char *p) { int i = 0;
while ((i = t[ i + bool_get(bool, p[i>>1])]) > 0);
return -i; } #endif
---- End code block ----------------------------------------
## 20.3. dequant_data.h ✨
okay so this one is the lookup tables bestie 🧚 the decoder uses to turn the quantizer index into actual dequantization values. 128 entries for DC, 128 for AC, and the whole thing is giving very organized table energy 💅 The copyright header stays exactly as written, obvi 🫶
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
static const int dc_q_lookup[128] = { 4, 5, 6, 7, 8, 9, 10, 10, 11, 12, 13, 14, 15, 16, 17, 17, 18, 19, 20, 20, 21, 21, 22, 22, 23, 23, 24, 25, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 91, 93, 95, 96, 98, 100, 101, 102, 104, 106, 108, 110, 112, 114, 116, 118, 122, 124, 126, 128, 130, 132, 134, 136, 138, 140, 143, 145, 148, 151, 154, 157 };
static const int ac_q_lookup[128] = { 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100, 102, 104, 106, 108, 110, 112, 114, 116, 119, 122, 125, 128, 131, 134, 137, 140, 143, 146, 149, 152, 155, 158, 161, 164, 167, 170, 173, 177, 181, 185, 189, 193, 197, 201, 205, 209, 213, 217, 221, 225, 229, 234, 239, 245, 249, 254, 259, 264, 269, 274, 279, 284 };
---
## 20.4. dixie.c 💖
okay now the big one 👑 this is the reference decoder's C file, and it's giving "everything is parsed in order" vibes. Each function reads one chunk of the frame header, like entropy, reference, quantizer, partitions, loop filter and segmentation. Then the dequant setup kicks in. The copyright is still at the top, periodt 🎀
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #include "vpx_codec_internal.h" #include "bit_ops.h" #include "dixie.h" #include "vp8_prob_data.h" #include "dequant_data.h" #include "modemv.h" #include "tokens.h" #include "predict.h" #include "dixie_loopfilter.h" #include <string.h> #include <assert.h>
enum { FRAME_HEADER_SZ = 3, KEYFRAME_HEADER_SZ = 7 };
#define ARRAY_COPY(a,b) {\ assert(sizeof(a)==sizeof(b));memcpy(a,b,sizeof(a));} static void decode_entropy_header(struct vp8_decoder_ctx *ctx, struct bool_decoder *bool, struct vp8_entropy_hdr *hdr)
{ int i, j, k, l;
/* Read coefficient probability updates */ for (i = 0; i < BLOCK_TYPES; i++) for (j = 0; j < COEFF_BANDS; j++) for (k = 0; k < PREV_COEFF_CONTEXTS; k++) for (l = 0; l < ENTROPY_NODES; l++) if (bool_get(bool, k_coeff_entropy_update_probs [i][j][k][l])) hdr->coeff_probs[i][j][k][l] = bool_get_uint(bool, 8);
/* Read coefficient skip mode probability */ hdr->coeff_skip_enabled = bool_get_bit(bool);
if (hdr->coeff_skip_enabled) hdr->coeff_skip_prob = bool_get_uint(bool, 8);
/* Parse interframe probability updates */ if (!ctx->frame_hdr.is_keyframe) { hdr->prob_inter = bool_get_uint(bool, 8); hdr->prob_last = bool_get_uint(bool, 8); hdr->prob_gf = bool_get_uint(bool, 8);
if (bool_get_bit(bool)) for (i = 0; i < 4; i++) hdr->y_mode_probs[i] = bool_get_uint(bool, 8);
if (bool_get_bit(bool)) for (i = 0; i < 3; i++) hdr->uv_mode_probs[i] = bool_get_uint(bool, 8);
for (i = 0; i < 2; i++) for (j = 0; j < MV_PROB_CNT; j++) if (bool_get(bool, k_mv_entropy_update_probs[i][j])) { int x = bool_get_uint(bool, 7); hdr->mv_probs[i][j] = x ? x << 1 : 1; } } } static void decode_reference_header(struct vp8_decoder_ctx *ctx, struct bool_decoder *bool, struct vp8_reference_hdr *hdr) { unsigned int key = ctx->frame_hdr.is_keyframe;
hdr->refresh_gf = key ? 1 : bool_get_bit(bool); hdr->refresh_arf = key ? 1 : bool_get_bit(bool); hdr->copy_gf = key ? 0 : !hdr->refresh_gf ? bool_get_uint(bool, 2) : 0; hdr->copy_arf = key ? 0 : !hdr->refresh_arf ? bool_get_uint(bool, 2) : 0; hdr->sign_bias[GOLDEN_FRAME] = key ? 0 : bool_get_bit(bool); hdr->sign_bias[ALTREF_FRAME] = key ? 0 : bool_get_bit(bool); hdr->refresh_entropy = bool_get_bit(bool); hdr->refresh_last = key ? 1 : bool_get_bit(bool); }
static void decode_quantizer_header(struct vp8_decoder_ctx *ctx, struct bool_decoder *bool, struct vp8_quant_hdr *hdr) { int update; int last_q = hdr->q_index;
hdr->q_index = bool_get_uint(bool, 7); update = last_q != hdr->q_index; update |= (hdr->y1_dc_delta_q = bool_maybe_get_int(bool, 4)); update |= (hdr->y2_dc_delta_q = bool_maybe_get_int(bool, 4)); update |= (hdr->y2_ac_delta_q = bool_maybe_get_int(bool, 4)); update |= (hdr->uv_dc_delta_q = bool_maybe_get_int(bool, 4)); update |= (hdr->uv_ac_delta_q = bool_maybe_get_int(bool, 4)); hdr->delta_update = update; }
static void decode_and_init_token_partitions(struct vp8_decoder_ctx *ctx, struct bool_decoder *bool, const unsigned char *data, unsigned int sz, struct vp8_token_hdr *hdr)
{ int i;
hdr->partitions = 1 << bool_get_uint(bool, 2);
if (sz < 3 *(hdr->partitions - 1)) vpx_internal_error(&ctx->error, VPX_CODEC_CORRUPT_FRAME, "Truncated packet found parsing partition" " lengths.");
sz -= 3 * (hdr->partitions - 1);
for (i = 0; i < hdr->partitions; i++) { if (i < hdr->partitions - 1) { hdr->partition_sz[i] = (data[2] << 16)
| (data[1] << 8) | data[0]; |
data += 3; } else hdr->partition_sz[i] = sz;
if (sz < hdr->partition_sz[i]) vpx_internal_error(&ctx->error, VPX_CODEC_CORRUPT_FRAME, "Truncated partition %d", i);
sz -= hdr->partition_sz[i]; }
for (i = 0; i < ctx->token_hdr.partitions; i++) { init_bool_decoder(&ctx->tokens[i].bool, data, ctx->token_hdr.partition_sz[i]); data += ctx->token_hdr.partition_sz[i]; } }
static void decode_loopfilter_header(struct vp8_decoder_ctx *ctx, struct bool_decoder *bool, struct vp8_loopfilter_hdr *hdr)
{ if (ctx->frame_hdr.is_keyframe) memset(hdr, 0, sizeof(*hdr));
hdr->use_simple = bool_get_bit(bool); hdr->level = bool_get_uint(bool, 6); hdr->sharpness = bool_get_uint(bool, 3); hdr->delta_enabled = bool_get_bit(bool);
if (hdr->delta_enabled && bool_get_bit(bool)) { int i;
for (i = 0; i < BLOCK_CONTEXTS; i++) hdr->ref_delta[i] = bool_maybe_get_int(bool, 6);
for (i = 0; i < BLOCK_CONTEXTS; i++) hdr->mode_delta[i] = bool_maybe_get_int(bool, 6); } }
static void decode_segmentation_header(struct vp8_decoder_ctx *ctx, struct bool_decoder *bool, struct vp8_segment_hdr *hdr) { if (ctx->frame_hdr.is_keyframe) memset(hdr, 0, sizeof(*hdr));
hdr->enabled = bool_get_bit(bool);
if (hdr->enabled) { int i;
hdr->update_map = bool_get_bit(bool); hdr->update_data = bool_get_bit(bool); if (hdr->update_data) { hdr->abs = bool_get_bit(bool);
for (i = 0; i < MAX_MB_SEGMENTS; i++) hdr->quant_idx[i] = bool_maybe_get_int(bool, 7);
for (i = 0; i < MAX_MB_SEGMENTS; i++) hdr->lf_level[i] = bool_maybe_get_int(bool, 6); }
if (hdr->update_map) { for (i = 0; i < MB_FEATURE_TREE_PROBS; i++) hdr->tree_probs[i] = bool_get_bit(bool) ? bool_get_uint(bool, 8) : 255; } } else { hdr->update_map = 0; hdr->update_data = 0; } }
static void dequant_global_init(struct dequant_factors dqf[MAX_MB_SEGMENTS]) { int i;
for (i = 0; i < MAX_MB_SEGMENTS; i++) dqf[i].quant_idx = -1; }
static int clamp_q(int q) { if (q < 0) return 0; else if (q > 127) return 127;
return q; } static int dc_q(int q) { return dc_q_lookup[clamp_q(q)]; }
static int ac_q(int q) { return ac_q_lookup[clamp_q(q)]; }
static void dequant_init(struct dequant_factors factors[MAX_MB_SEGMENTS], const struct vp8_segment_hdr *seg, const struct vp8_quant_hdr *quant_hdr) { int i, q; struct dequant_factors *dqf = factors;
for (i = 0; i < (seg->enabled ? MAX_MB_SEGMENTS : 1); i++) { q = quant_hdr->q_index;
if (seg->enabled) q = (!seg->abs) ? q + seg->quant_idx[i] : seg->quant_idx[i];
if (dqf->quant_idx != q || quant_hdr->delta_update) { dqf->factor[TOKEN_BLOCK_Y1][0] = dc_q(q + quant_hdr->y1_dc_delta_q); dqf->factor[TOKEN_BLOCK_Y1][1] = ac_q(q); dqf->factor[TOKEN_BLOCK_UV][0] = dc_q(q + quant_hdr->uv_dc_delta_q); dqf->factor[TOKEN_BLOCK_UV][1] = ac_q(q + quant_hdr->uv_ac_delta_q); dqf->factor[TOKEN_BLOCK_Y2][0] = dc_q(q + quant_hdr->y2_dc_delta_q) * 2; dqf->factor[TOKEN_BLOCK_Y2][1] = ac_q(q + quant_hdr->y2_ac_delta_q) * 155 / 100;
if (dqf->factor[TOKEN_BLOCK_Y2][1] < 8) dqf->factor[TOKEN_BLOCK_Y2][1] = 8;
if (dqf->factor[TOKEN_BLOCK_UV][0] > 132) dqf->factor[TOKEN_BLOCK_UV][0] = 132;
dqf->quant_idx = q; }
dqf++; } }
okay so the factor clamps are giving very strict bouncer energy 💅🎀 nobody's getting past the velvet rope without the right numbers ✨
static void decode_frame(struct vp8_decoder_ctx *ctx, const unsigned char *data, unsigned int sz) { vpx_codec_err_t res; struct bool_decoder bool; int i, row, partition;
ctx->saved_entropy_valid = 0;
if ((res = vp8_parse_frame_header(data, sz, &ctx->frame_hdr))) vpx_internal_error(&ctx->error, res, "Failed to parse frame header");
if (ctx->frame_hdr.is_experimental) vpx_internal_error(&ctx->error, VPX_CODEC_UNSUP_BITSTREAM, "Experimental bitstreams not supported.");
data += FRAME_HEADER_SZ; sz -= FRAME_HEADER_SZ;
if (ctx->frame_hdr.is_keyframe) { data += KEYFRAME_HEADER_SZ; sz -= KEYFRAME_HEADER_SZ; ctx->mb_cols = (ctx->frame_hdr.kf.w + 15) / 16; ctx->mb_rows = (ctx->frame_hdr.kf.h + 15) / 16; }
/* Start the bitreader for the header/entropy partition / init_bool_decoder(&bool, data, ctx->frame_hdr.part0_sz); / Skip the colorspace and clamping bits */ if (ctx->frame_hdr.is_keyframe) if (bool_get_uint(&bool, 2)) vpx_internal_error( &ctx->error, VPX_CODEC_UNSUP_BITSTREAM, "Reserved bits not supported.");
decode_segmentation_header(ctx, &bool, &ctx->segment_hdr); decode_loopfilter_header(ctx, &bool, &ctx->loopfilter_hdr); decode_and_init_token_partitions(ctx, &bool, data + ctx->frame_hdr.part0_sz, sz - ctx->frame_hdr.part0_sz, &ctx->token_hdr); decode_quantizer_header(ctx, &bool, &ctx->quant_hdr); decode_reference_header(ctx, &bool, &ctx->reference_hdr);
/* Set keyframe entropy defaults. These get updated on keyframes
- regardless of the refresh_entropy setting. */ if (ctx->frame_hdr.is_keyframe) { ARRAY_COPY(ctx->entropy_hdr.coeff_probs, k_default_coeff_probs); ARRAY_COPY(ctx->entropy_hdr.mv_probs, k_default_mv_probs); ARRAY_COPY(ctx->entropy_hdr.y_mode_probs, k_default_y_mode_probs); ARRAY_COPY(ctx->entropy_hdr.uv_mode_probs, k_default_uv_mode_probs); }
if (!ctx->reference_hdr.refresh_entropy) { ctx->saved_entropy = ctx->entropy_hdr; ctx->saved_entropy_valid = 1; }
decode_entropy_header(ctx, &bool, &ctx->entropy_hdr);
vp8_dixie_modemv_init(ctx); vp8_dixie_tokens_init(ctx); vp8_dixie_predict_init(ctx); dequant_init(ctx->dequant_factors, &ctx->segment_hdr, &ctx->quant_hdr);
for (row = 0, partition = 0; row < ctx->mb_rows; row++) { vp8_dixie_modemv_process_row( ctx, &bool, row, 0, ctx->mb_cols); vp8_dixie_tokens_process_row(ctx, partition, row, 0, ctx->mb_cols); vp8_dixie_predict_process_row(ctx, row, 0, ctx->mb_cols);
if (ctx->loopfilter_hdr.level && row) vp8_dixie_loopfilter_process_row(ctx, row - 1, 0, ctx->mb_cols);
if (++partition == ctx->token_hdr.partitions) partition = 0; }
if (ctx->loopfilter_hdr.level) vp8_dixie_loopfilter_process_row( ctx, row - 1, 0, ctx->mb_cols);
ctx->frame_cnt++;
if (!ctx->reference_hdr.refresh_entropy) { ctx->entropy_hdr = ctx->saved_entropy; ctx->saved_entropy_valid = 0; }
/* Handle reference frame updates */ if (ctx->reference_hdr.copy_arf == 1) { vp8_dixie_release_ref_frame(ctx->ref_frames[ALTREF_FRAME]); ctx->ref_frames[ALTREF_FRAME] = vp8_dixie_ref_frame(ctx->ref_frames[LAST_FRAME]); } else if (ctx->reference_hdr.copy_arf == 2) { vp8_dixie_release_ref_frame(ctx->ref_frames[ALTREF_FRAME]); ctx->ref_frames[ALTREF_FRAME] = vp8_dixie_ref_frame(ctx->ref_frames[GOLDEN_FRAME]); }
if (ctx->reference_hdr.copy_gf == 1) { vp8_dixie_release_ref_frame(ctx->ref_frames[GOLDEN_FRAME]); ctx->ref_frames[GOLDEN_FRAME] = vp8_dixie_ref_frame(ctx->ref_frames[LAST_FRAME]); } else if (ctx->reference_hdr.copy_gf == 2) { vp8_dixie_release_ref_frame(ctx->ref_frames[GOLDEN_FRAME]); ctx->ref_frames[GOLDEN_FRAME] = vp8_dixie_ref_frame(ctx->ref_frames[ALTREF_FRAME]); }
if (ctx->reference_hdr.refresh_gf) { vp8_dixie_release_ref_frame(ctx->ref_frames[GOLDEN_FRAME]); ctx->ref_frames[GOLDEN_FRAME] = vp8_dixie_ref_frame(ctx->ref_frames[CURRENT_FRAME]); }
if (ctx->reference_hdr.refresh_arf) { vp8_dixie_release_ref_frame(ctx->ref_frames[ALTREF_FRAME]); ctx->ref_frames[ALTREF_FRAME] = vp8_dixie_ref_frame(ctx->ref_frames[CURRENT_FRAME]); }
if (ctx->reference_hdr.refresh_last) { vp8_dixie_release_ref_frame(ctx->ref_frames[LAST_FRAME]); ctx->ref_frames[LAST_FRAME] = vp8_dixie_ref_frame(ctx->ref_frames[CURRENT_FRAME]); }
}
void vp8_dixie_decode_init(struct vp8_decoder_ctx *ctx) { dequant_global_init(ctx->dequant_factors); } #define CHECK_FOR_UPDATE(lval,rval,update_flag) do {\ unsigned int old = lval; \ update_flag |= (old != (lval = rval)); \ } while (0)
ngl this whole `decode_frame` function is the main character of the file 😭🦋 it's basically a group chat where every helper gets its turn to speak, row by row, and the reference frames get swapped around like outfits before a party 👑 the way the golden, altref and last frames keep trading places is literally iconic
the header checks are giving "main character but make it strict" too, bestie 🫶
vpx_codec_err_t vp8_parse_frame_header(const unsigned char *data, unsigned int sz, struct vp8_frame_hdr *hdr) { unsigned long raw;
if (sz < 10) return VPX_CODEC_CORRUPT_FRAME;
/* The frame header is defined as a three-byte little endian
- value */ raw = data[0] | (data[1] << 8) | (data[2] << 16); hdr->is_keyframe = !BITS_GET(raw, 0, 1); hdr->version = BITS_GET(raw, 1, 2); hdr->is_experimental = BITS_GET(raw, 3, 1); hdr->is_shown = BITS_GET(raw, 4, 1); hdr->part0_sz = BITS_GET(raw, 5, 19);
if (sz <= hdr->part0_sz + (hdr->is_keyframe ? 10 : 3)) return VPX_CODEC_CORRUPT_FRAME;
hdr->frame_size_updated = 0;
if (hdr->is_keyframe) { unsigned int update = 0;
/* Keyframe header consists of a three-byte sync code
- followed by the width and height and associated scaling
- factors. */ if (data[3] != 0x9d || data[4] != 0x01 || data[5] != 0x2a) return VPX_CODEC_UNSUP_BITSTREAM;
raw = data[6] | (data[7] << 8)
| (data[8] << 16) | (data[9] << 24); |
CHECK_FOR_UPDATE(hdr->kf.w, BITS_GET(raw, 0, 14), update); CHECK_FOR_UPDATE(hdr->kf.scale_w, BITS_GET(raw, 14, 2), update); CHECK_FOR_UPDATE(hdr->kf.h, BITS_GET(raw, 16, 14), update); CHECK_FOR_UPDATE(hdr->kf.scale_h, BITS_GET(raw, 30, 2), update);
hdr->frame_size_updated = update;
if (!hdr->kf.w || !hdr->kf.h) return VPX_CODEC_UNSUP_BITSTREAM; }
return VPX_CODEC_OK; }
vpx_codec_err_t vp8_dixie_decode_frame(struct vp8_decoder_ctx *ctx, const unsigned char *data, unsigned int sz) { volatile struct vp8_decoder_ctx *ctx_ = ctx;
ctx->error.error_code = VPX_CODEC_OK; ctx->error.has_detail = 0;
if (!setjmp(ctx->error.jmp)) decode_frame(ctx, data, sz);
return ctx_->error.error_code; }
void vp8_dixie_decode_destroy(struct vp8_decoder_ctx *ctx) { vp8_dixie_predict_destroy(ctx); vp8_dixie_tokens_destroy(ctx); vp8_dixie_modemv_destroy(ctx); }
---- End code block ----------------------------------------
<pink>that's a whole wrap, bestie 💖</pink> the frame header parser is out here checking the sync code like a bouncer checking IDs 🙈 and the decoder teardown at the end cleans up after the party 🧚 ✨
## 20.5. dixie.h 🎀
okay so this is the header file, bestie 💅 It's basically the big blueprint for the decoder: the structs, the enums, and the function declarations. Everything the other files need to agree on lives here 🫶
---- Begin code block --------------------------------------
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #ifndef DIXIE_H #define DIXIE_H #include "vpx_codec_internal.h" #include "bool_decoder.h"
struct vp8_frame_hdr { unsigned int is_keyframe; /* Frame is a keyframe / unsigned int is_experimental; / Frame is a keyframe / unsigned int version; / Bitstream version / unsigned int is_shown; / Frame is to be displayed. / unsigned int part0_sz; / Partition 0 length, in bytes */
struct vp8_kf_hdr { unsigned int w; /* Width / unsigned int h; / Height / unsigned int scale_w; / Scaling factor, Width / unsigned int scale_h; / Scaling factor, Height */ } kf;
unsigned int frame_size_updated; /* Flag to indicate a resolution
- update. */ };
enum { MB_FEATURE_TREE_PROBS = 3, MAX_MB_SEGMENTS = 4 }; struct vp8_segment_hdr { unsigned int enabled; unsigned int update_data; unsigned int update_map; unsigned int abs; /* 0=deltas, 1=absolute values */ unsigned int tree_probs[MB_FEATURE_TREE_PROBS]; int lf_level[MAX_MB_SEGMENTS]; int quant_idx[MAX_MB_SEGMENTS];
};
enum { BLOCK_CONTEXTS = 4 };
struct vp8_loopfilter_hdr { unsigned int use_simple; unsigned int level; unsigned int sharpness; unsigned int delta_enabled; int ref_delta[BLOCK_CONTEXTS]; int mode_delta[BLOCK_CONTEXTS]; };
enum { MAX_PARTITIONS = 8 };
struct vp8_token_hdr { unsigned int partitions; unsigned int partition_sz[MAX_PARTITIONS]; }; struct vp8_quant_hdr { unsigned int q_index; int delta_update; int y1_dc_delta_q; int y2_dc_delta_q; int y2_ac_delta_q; int uv_dc_delta_q; int uv_ac_delta_q; };
struct vp8_reference_hdr { unsigned int refresh_last; unsigned int refresh_gf; unsigned int refresh_arf; unsigned int copy_gf; unsigned int copy_arf; unsigned int sign_bias[4]; unsigned int refresh_entropy; };
enum { BLOCK_TYPES = 4, PREV_COEFF_CONTEXTS = 3, COEFF_BANDS = 8, ENTROPY_NODES = 11, }; typedef unsigned char coeff_probs_table_t[BLOCK_TYPES][COEFF_BANDS] [PREV_COEFF_CONTEXTS] [ENTROPY_NODES];
enum { MV_PROB_CNT = 2 + 8 - 1 + 10 /* from entropymv.h */ }; typedef unsigned char mv_component_probs_t[MV_PROB_CNT]; struct vp8_entropy_hdr { coeff_probs_table_t coeff_probs; mv_component_probs_t mv_probs[2]; unsigned int coeff_skip_enabled; unsigned char coeff_skip_prob; unsigned char y_mode_probs[4]; unsigned char uv_mode_probs[3]; unsigned char prob_inter; unsigned char prob_last; unsigned char prob_gf; };
enum reference_frame { CURRENT_FRAME, LAST_FRAME, GOLDEN_FRAME, ALTREF_FRAME, NUM_REF_FRAMES };
enum prediction_mode { /* 16x16 intra modes */ DC_PRED, V_PRED, H_PRED, TM_PRED, B_PRED,
/* 16x16 inter modes */ NEARESTMV, NEARMV, ZEROMV, NEWMV, SPLITMV,
MB_MODE_COUNT,
/* 4x4 intra modes */ B_DC_PRED = 0, B_TM_PRED, B_VE_PRED, B_HE_PRED, B_LD_PRED, B_RD_PRED, B_VR_PRED, B_VL_PRED, B_HD_PRED, B_HU_PRED,
/* 4x4 inter modes */ LEFT4X4, ABOVE4X4, ZERO4X4, NEW4X4,
B_MODE_COUNT }; enum splitmv_partitioning { SPLITMV_16X8, SPLITMV_8X16, SPLITMV_8X8, SPLITMV_4X4 };
typedef short filter_t[6];
typedef union mv { struct { int16_t x, y; } d; uint32_t raw; } mv_t;
struct mb_base_info { unsigned char y_mode : 4; unsigned char uv_mode : 4; unsigned char segment_id : 2; unsigned char ref_frame : 2; unsigned char skip_coeff : 1; unsigned char need_mc_border : 1; enum splitmv_partitioning partitioning : 2; union mv mv; unsigned int eob_mask; };
struct mb_info { struct mb_base_info base; union { union mv mvs[16]; enum prediction_mode modes[16]; } split; }; /* A "token entropy context" has 4 Y values, 2 U, 2 V, and 1 Y2 */ typedef int token_entropy_ctx_t[4 + 2 + 2 + 1];
struct token_decoder { struct bool_decoder bool; token_entropy_ctx_t left_token_entropy_ctx; short *coeffs; };
enum token_block_type { TOKEN_BLOCK_Y1, TOKEN_BLOCK_UV, TOKEN_BLOCK_Y2, TOKEN_BLOCK_TYPES, };
struct dequant_factors { int quant_idx; short factor[TOKEN_BLOCK_TYPES][2]; /* [ Y1, UV, Y2 ]
- [ DC, AC ] */ };
struct ref_cnt_img { vpx_image_t img; unsigned int ref_cnt; };
struct vp8_decoder_ctx { struct vpx_internal_error_info error; unsigned int frame_cnt;
struct vp8_frame_hdr frame_hdr; struct vp8_segment_hdr segment_hdr; struct vp8_loopfilter_hdr loopfilter_hdr; struct vp8_token_hdr token_hdr; struct vp8_quant_hdr quant_hdr; struct vp8_reference_hdr reference_hdr; struct vp8_entropy_hdr entropy_hdr;
struct vp8_entropy_hdr saved_entropy; unsigned int saved_entropy_valid; unsigned int mb_rows; unsigned int mb_cols; struct mb_info *mb_info_storage; struct mb_info **mb_info_rows_storage; struct mb_info **mb_info_rows;
token_entropy_ctx_t *above_token_entropy_ctx; struct token_decoder tokens[MAX_PARTITIONS]; struct dequant_factors dequant_factors[MAX_MB_SEGMENTS];
struct ref_cnt_img frame_strg[NUM_REF_FRAMES]; struct ref_cnt_img *ref_frames[NUM_REF_FRAMES]; ptrdiff_t ref_frame_offsets[4];
const filter_t *subpixel_filters; };
void vp8_dixie_decode_init(struct vp8_decoder_ctx *ctx);
void vp8_dixie_decode_destroy(struct vp8_decoder_ctx *ctx);
vpx_codec_err_t vp8_parse_frame_header(const unsigned char *data, unsigned int sz, struct vp8_frame_hdr *hdr);
vpx_codec_err_t vp8_dixie_decode_frame(struct vp8_decoder_ctx *ctx, const unsigned char *data, unsigned int sz);
#define CLAMP_255(x) ((x)<0?0:((x)>255?255:(x)))
#endif
---- End code block ----------------------------------------
---
## 20.6. dixie_loopfilter.c ✨
okay now we're in the loop filter file, and this one is giving main character energy 👑 It's got the little helper functions that check the edges between blocks and decide how to smooth them 🌸
---- Begin code block --------------------------------------
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #include "dixie.h" #include "dixie_loopfilter.h"
#define ABS(x) ((x) >= 0 ? (x) : -(x))
#define p3 pixels[-4*stride] #define p2 pixels[-3*stride] #define p1 pixels[-2*stride] #define p0 pixels[-1*stride] #define q0 pixels[ 0*stride] #define q1 pixels[ 1*stride] #define q2 pixels[ 2*stride] #define q3 pixels[ 3*stride]
#define static static int
saturate_int8(int x) { if (x < -128) return -128;
if (x > 127) return 127;
return x; } static int saturate_uint8(int x) { if (x < 0) return 0;
if (x > 255) return 255;
return x; }
static int high_edge_variance(unsigned char *pixels, int stride, int hev_threshold) { return ABS(p1 - p0) > hev_threshold || ABS(q1 - q0) > hev_threshold; }
static int simple_threshold(unsigned char *pixels, int stride, int filter_limit) { return (ABS(p0 - q0) * 2 + (ABS(p1 - q1) >> 1)) <= filter_limit; }
static int normal_threshold(unsigned char *pixels, int stride, int edge_limit, int interior_limit) { int E = edge_limit; int I = interior_limit;
return simple_threshold(pixels, stride, 2 * E + I) && ABS(p3 - p2) <= I && ABS(p2 - p1) <= I && ABS(p1 - p0) <= I && ABS(q3 - q2) <= I && ABS(q2 - q1) <= I && ABS(q1 - q0) <= I; } static void filter_common(unsigned char *pixels, int stride, int use_outer_taps) { int a, f1, f2;
a = 3 * (q0 - p0);
if (use_outer_taps) a += saturate_int8(p1 - q1);
a = saturate_int8(a);
f1 = ((a + 4 > 127) ? 127 : a + 4) >> 3; f2 = ((a + 3 > 127) ? 127 : a + 3) >> 3;
p0 = saturate_uint8(p0 + f2); q0 = saturate_uint8(q0 - f1);
if (!use_outer_taps) { /* This handles the case of subblock_filter()
- (from the bitstream guide. */ a = (f1 + 1) >> 1; p1 = saturate_uint8(p1 + a); q1 = saturate_uint8(q1 - a); } } static void filter_mb_edge(unsigned char *pixels, int stride) { int w, a;
w = saturate_int8(saturate_int8(p1 - q1) + 3 * (q0 - p0));
a = (27 * w + 63) >> 7; p0 = saturate_uint8(p0 + a); q0 = saturate_uint8(q0 - a);
a = (18 * w + 63) >> 7; p1 = saturate_uint8(p1 + a); q1 = saturate_uint8(q1 - a);
a = (9 * w + 63) >> 7; p2 = saturate_uint8(p2 + a); q2 = saturate_uint8(q2 - a); }
static void filter_mb_v_edge(unsigned char *src, int stride, int edge_limit, int interior_limit, int hev_threshold, int size) { int i;
and that's where we pause for now, bestie 🦋 the function keeps going from here 💜
<pink>okay bestie, the loop filter is literally giving main character energy</pink> 💅✨ here's the next chunk, straight from the source 🫶
for (i = 0; i < 8 * size; i++) { if (normal_threshold(src, 1, edge_limit, interior_limit)) { if (high_edge_variance(src, 1, hev_threshold)) filter_common(src, 1, 1); else filter_mb_edge(src, 1); }
src += stride; } } static void filter_subblock_v_edge(unsigned char *src, int stride, int edge_limit, int interior_limit, int hev_threshold, int size) { int i;
for (i = 0; i < 8 * size; i++) { if (normal_threshold(src, 1, edge_limit, interior_limit)) filter_common(src, 1, high_edge_variance(src, 1, hev_threshold));
src += stride; } }
static void filter_mb_h_edge(unsigned char *src, int stride, int edge_limit, int interior_limit, int hev_threshold, int size) { int i;
for (i = 0; i < 8 * size; i++) { if (normal_threshold(src, stride, edge_limit, interior_limit)) { if (high_edge_variance(src, stride, hev_threshold)) filter_common(src, stride, 1); else filter_mb_edge(src, stride); }
src += 1; } } static void filter_subblock_h_edge(unsigned char *src, int stride, int edge_limit, int interior_limit, int hev_threshold, int size) { int i;
for (i = 0; i < 8 * size; i++) { if (normal_threshold(src, stride, edge_limit, interior_limit)) filter_common(src, stride, high_edge_variance(src, stride, hev_threshold));
src += 1; } }
static void filter_v_edge_simple(unsigned char *src, int stride, int filter_limit) { int i;
for (i = 0; i < 16; i++) { if (simple_threshold(src, 1, filter_limit)) filter_common(src, 1, 1);
src += stride; } } static void filter_h_edge_simple(unsigned char *src, int stride, int filter_limit) { int i;
for (i = 0; i < 16; i++) { if (simple_threshold(src, stride, filter_limit)) filter_common(src, stride, 1);
src += 1; } }
static void calculate_filter_parameters(struct vp8_decoder_ctx *ctx, struct mb_info *mbi, int *edge_limit_, int *interior_limit_, int *hev_threshold_) { int filter_level, interior_limit, hev_threshold;
/* Reference code/spec seems to conflate filter_level and
- edge_limit */
filter_level = ctx->loopfilter_hdr.level;
if (ctx->segment_hdr.enabled) { if (!ctx->segment_hdr.abs) filter_level += ctx->segment_hdr.lf_level[mbi->base.segment_id]; else filter_level = ctx->segment_hdr.lf_level[mbi->base.segment_id]; }
if (filter_level > 63) filter_level = 63; else if (filter_level < 0) filter_level = 0;
if (ctx->loopfilter_hdr.delta_enabled) { filter_level += ctx->loopfilter_hdr.ref_delta[mbi->base.ref_frame];
if (mbi->base.ref_frame == CURRENT_FRAME) { if (mbi->base.y_mode == B_PRED) filter_level += ctx->loopfilter_hdr.mode_delta[0]; } else if (mbi->base.y_mode == ZEROMV) filter_level += ctx->loopfilter_hdr.mode_delta[1]; else if (mbi->base.y_mode == SPLITMV) filter_level += ctx->loopfilter_hdr.mode_delta[3]; else filter_level += ctx->loopfilter_hdr.mode_delta[2]; }
if (filter_level > 63) filter_level = 63; else if (filter_level < 0) filter_level = 0;
interior_limit = filter_level;
if (ctx->loopfilter_hdr.sharpness) { interior_limit >>= ctx->loopfilter_hdr.sharpness > 4 ? 2 : 1;
if (interior_limit > 9 - ctx->loopfilter_hdr.sharpness) interior_limit = 9 - ctx->loopfilter_hdr.sharpness; }
if (interior_limit < 1) interior_limit = 1;
hev_threshold = (filter_level >= 15);
if (filter_level >= 40) hev_threshold++;
if (filter_level >= 20 && !ctx->frame_hdr.is_keyframe) hev_threshold++;
*edge_limit_ = filter_level; *interior_limit_ = interior_limit; *hev_threshold_ = hev_threshold; } static void filter_row_normal(struct vp8_decoder_ctx *ctx, unsigned int row, unsigned int start_col, unsigned int num_cols) { unsigned char *y, *u, *v; int stride, uv_stride; struct mb_info *mbi; unsigned int col;
/* Adjust pointers based on row, start_col */
stride = ctx->ref_frames[CURRENT_FRAME]->img.stride[PLANE_Y]; uv_stride = ctx->ref_frames[CURRENT_FRAME]->img.stride[PLANE_U]; y = ctx->ref_frames[CURRENT_FRAME]->img.planes[PLANE_Y]; u = ctx->ref_frames[CURRENT_FRAME]->img.planes[PLANE_U]; v = ctx->ref_frames[CURRENT_FRAME]->img.planes[PLANE_V]; y += (stride * row + start_col) * 16; u += (uv_stride * row + start_col) * 8; v += (uv_stride * row + start_col) * 8; mbi = ctx->mb_info_rows[row] + start_col;
for (col = start_col; col < start_col + num_cols; col++) { int edge_limit, interior_limit, hev_threshold;
/* TODO: Only need to recalculate every MB if segmentation is
- enabled. */ calculate_filter_parameters(ctx, mbi, &edge_limit, &interior_limit, &hev_threshold);
if (edge_limit) { if (col) { filter_mb_v_edge(y, stride, edge_limit + 2, interior_limit, hev_threshold, 2); filter_mb_v_edge(u, uv_stride, edge_limit + 2, interior_limit, hev_threshold, 1); filter_mb_v_edge(v, uv_stride, edge_limit + 2, interior_limit, hev_threshold, 1); } /* NOTE: This conditional is actually dependent on the
- number of coefficients decoded, not the skip flag as
- coded in the bitstream. The tokens task is expected
- to set 31 if there is any non-zero data. */ if (mbi->base.eob_mask
| mbi->base.y_mode == SPLITMV | |
|---|---|
| mbi->base.y_mode == B_PRED) |
{ filter_subblock_v_edge(y + 4, stride, edge_limit, interior_limit, hev_threshold, 2); filter_subblock_v_edge(y + 8, stride, edge_limit, interior_limit, hev_threshold, 2); filter_subblock_v_edge(y + 12, stride, edge_limit, interior_limit, hev_threshold, 2); filter_subblock_v_edge(u + 4, uv_stride, edge_limit, interior_limit, hev_threshold, 1); filter_subblock_v_edge(v + 4, uv_stride, edge_limit, interior_limit, hev_threshold, 1); }
if (row) { filter_mb_h_edge(y, stride, edge_limit + 2, interior_limit, hev_threshold, 2); filter_mb_h_edge(u, uv_stride, edge_limit + 2, interior_limit, hev_threshold, 1); filter_mb_h_edge(v, uv_stride, edge_limit + 2, interior_limit, hev_threshold, 1); } if (mbi->base.eob_mask
| mbi->base.y_mode == SPLITMV | |
|---|---|
| mbi->base.y_mode == B_PRED) |
{ filter_subblock_h_edge(y + 4 * stride, stride, edge_limit, interior_limit, hev_threshold, 2); filter_subblock_h_edge(y + 8 * stride, stride, edge_limit, interior_limit, hev_threshold, 2); filter_subblock_h_edge(y + 12 * stride, stride, edge_limit, interior_limit, hev_threshold, 2); filter_subblock_h_edge(u + 4 * uv_stride, uv_stride, edge_limit, interior_limit, hev_threshold, 1); filter_subblock_h_edge(v + 4 * uv_stride, uv_stride, edge_limit, interior_limit, hev_threshold, 1); } }
y += 16; u += 8; v += 8; mbi++; } }
static void filter_row_simple(struct vp8_decoder_ctx *ctx, unsigned int row, unsigned int start_col, unsigned int num_cols) { unsigned char *y; int stride; struct mb_info *mbi; unsigned int col;
/* Adjust pointers based on row, start_col */ stride = ctx->ref_frames[CURRENT_FRAME]->img.stride[PLANE_Y]; y = ctx->ref_frames[CURRENT_FRAME]->img.planes[PLANE_Y]; y += (stride * row + start_col) * 16; mbi = ctx->mb_info_rows[row] + start_col; for (col = start_col; col < start_col + num_cols; col++) { int edge_limit, interior_limit, hev_threshold;
/* TODO: Only need to recalculate every MB if segmentation is
- enabled. */ calculate_filter_parameters(ctx, mbi, &edge_limit, &interior_limit, &hev_threshold);
if (edge_limit) {
/* NOTE: This conditional is actually dependent on the
- number of coefficients decoded, not the skip flag as
- coded in the bitstream. The tokens task is expected
- to set 31 if there is any non-zero data. */ int filter_subblocks = (mbi->base.eob_mask
| mbi->base.y_mode == SPLITMV | |
|---|---|
| mbi->base.y_mode == B_PRED); |
int mb_limit = (edge_limit + 2) * 2 + interior_limit; int b_limit = edge_limit * 2 + interior_limit;
if (col) filter_v_edge_simple(y, stride, mb_limit);
if (filter_subblocks) { filter_v_edge_simple(y + 4, stride, b_limit); filter_v_edge_simple(y + 8, stride, b_limit); filter_v_edge_simple(y + 12, stride, b_limit); }
if (row) filter_h_edge_simple(y, stride, mb_limit);
if (filter_subblocks) { filter_h_edge_simple(y + 4 * stride, stride, b_limit); filter_h_edge_simple(y + 8 * stride, stride, b_limit); filter_h_edge_simple(y + 12 * stride, stride, b_limit); } } y += 16; mbi++; } }
void vp8_dixie_loopfilter_process_row(struct vp8_decoder_ctx *ctx, unsigned int row, unsigned int start_col, unsigned int num_cols) { if (ctx->loopfilter_hdr.use_simple) filter_row_simple(ctx, row, start_col, num_cols); else filter_row_normal(ctx, row, start_col, num_cols); }
## 20.7. dixie_loopfilter.h ✨
okay bestie, this one is the tiny header that goes with the loop filter file, so it's basically just the front door 🚪🎀 It has the same Google copyright and license vibe as the rest of the reference decoder 💜
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #ifndef DIXIE_LOOPFILTER_H #define DIXIE_LOOPFILTER_H
void vp8_dixie_loopfilter_process_row(struct vp8_decoder_ctx *ctx, unsigned int row, unsigned int start_col, unsigned int num_cols);
#endif
## 20.8. idct_add.c 🧚
this is the inverse transform file and it's giving math-girl realness 🔥 There's a Walsh transform, then the column and row passes for the IDCT, and then the results get added onto the prediction and clamped to 255 💅
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #include "dixie.h" #include "idct_add.h" #include <assert.h>
void vp8_dixie_walsh(const short *input, short *output) { int i; int a1, b1, c1, d1; int a2, b2, c2, d2; const short *ip = input; short *op = output;
for (i = 0; i < 4; i++) { a1 = ip[0] + ip[12]; b1 = ip[4] + ip[8]; c1 = ip[4] - ip[8]; d1 = ip[0] - ip[12];
op[0] = a1 + b1; op[4] = c1 + d1; op[8] = a1 - b1; op[12] = d1 - c1; ip++; op++; }
ip = output; op = output; for (i = 0; i < 4; i++) { a1 = ip[0] + ip[3]; b1 = ip[1] + ip[2]; c1 = ip[1] - ip[2]; d1 = ip[0] - ip[3];
a2 = a1 + b1; b2 = c1 + d1; c2 = a1 - b1; d2 = d1 - c1;
op[0] = (a2 + 3) >> 3; op[1] = (b2 + 3) >> 3; op[2] = (c2 + 3) >> 3; op[3] = (d2 + 3) >> 3;
ip += 4; op += 4; } }
#define cospi8sqrt2minus1 20091 #define sinpi8sqrt2 35468 #define rounding 0 static void idct_columns(const short *input, short *output) { int i; int a1, b1, c1, d1;
const short *ip = input; short *op = output; int temp1, temp2; int shortpitch = 4;
for (i = 0; i < 4; i++) { a1 = ip[0] + ip[8]; b1 = ip[0] - ip[8];
temp1 = (ip[4] * sinpi8sqrt2 + rounding) >> 16; temp2 = ip[12] + ((ip[12] * cospi8sqrt2minus1 + rounding) >> 16); c1 = temp1 - temp2; temp1 = ip[4] + ((ip[4] * cospi8sqrt2minus1 + rounding) >> 16); temp2 = (ip[12] * sinpi8sqrt2 + rounding) >> 16; d1 = temp1 + temp2;
op[shortpitch*0] = a1 + d1; op[shortpitch*3] = a1 - d1;
op[shortpitch*1] = b1 + c1; op[shortpitch*2] = b1 - c1;
ip++; op++; } }
void vp8_dixie_idct_add(unsigned char *recon, const unsigned char *predict, int stride, const short *coeffs) { int i; int a1, b1, c1, d1, temp1, temp2; short tmp[16]; idct_columns(coeffs, tmp); coeffs = tmp;
for (i = 0; i < 4; i++) { a1 = coeffs[0] + coeffs[2]; b1 = coeffs[0] - coeffs[2];
temp1 = (coeffs[1] * sinpi8sqrt2 + rounding) >> 16; temp2 = coeffs[3] + ((coeffs[3] * cospi8sqrt2minus1 + rounding) >> 16); c1 = temp1 - temp2;
temp1 = coeffs[1] + ((coeffs[1] * cospi8sqrt2minus1 + rounding) >> 16); temp2 = (coeffs[3] * sinpi8sqrt2 + rounding) >> 16; d1 = temp1 + temp2;
recon[0] = CLAMP_255(predict[0] + ((a1 + d1 + 4) >> 3)); recon[3] = CLAMP_255(predict[3] + ((a1 - d1 + 4) >> 3)); recon[1] = CLAMP_255(predict[1] + ((b1 + c1 + 4) >> 3)); recon[2] = CLAMP_255(predict[2] + ((b1 - c1 + 4) >> 3)); coeffs += 4; recon += stride; predict += stride; } }
## 20.9. idct_add.h 🦋
the matching header for the transform file, so every function it promises shows up here too 🫶
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #ifndef IDCT_ADD_H #define IDCT_ADD_H
void vp8_dixie_idct_add_init(struct vp8_decoder_ctx *ctx);
void vp8_dixie_idct_add(unsigned char *recon, const unsigned char *predict, int stride, const short *coeffs);
void vp8_dixie_walsh(const short *in, short *out);
void vp8_dixie_idct_add_process_row(struct vp8_decoder_ctx *ctx, short *coeffs, unsigned int row, unsigned int start_col, unsigned int num_cols); #endif
## 20.10. mem.h 🌸
this one is the little memory helper file, all about alignment macros and making sure variables are happy with their addresses 💖 It also has a note about silencing gcc's "may be used uninitialized" warnings, which is lowkey iconic 🙈
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #ifndef VPX_PORTS_MEM_H #define VPX_PORTS_MEM_H #include "vpx_config.h" #include "vpx_integer.h"
#if defined(__GNUC__) && __GNUC__ #define DECLARE_ALIGNED(n,typ,val) typ val __attribute__ \ ((aligned (n))) #elif defined(_MSC_VER) #define DECLARE_ALIGNED(n,typ,val) __declspec(align(n)) typ val #else #warning No alignment directives known for this compiler. #define DECLARE_ALIGNED(n,typ,val) typ val #endif #endif
/* Declare an aligned array on the stack, for situations where the
- stack pointer may not have the alignment we expect. Creates an
- array with a modified name, then defines val to be a pointer, and
- aligns that pointer within the array. / #define DECLARE_ALIGNED_ARRAY(a,typ,val,n)\ typ val##_[(n)+(a)/sizeof(typ)+1];\ typ *val = (typ)((((intptr_t)val##_)+(a)-1)&((intptr_t)-(a))) /* Indicates that the usage of the specified variable has been
- audited to assure that it's safe to use uninitialized. Silences
- 'may be used uninitialized' warnings on gcc. */ #if defined(__GNUC__) && __GNUC__ #define UNINITIALIZED_IS_SAFE(x) x=x #else #define UNINITIALIZED_IS_SAFE(x) x #endif
## 20.11. modemv.c 👑
okay this is where the mode and motion vector stuff lives, and it's the most main-character file in the whole decoder 🔥 We start with the motion vector clamping helper, then segment ID reading, then the above-block mode lookup, which is where we're going to pick up next time 💅
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #include "dixie.h" #include "modemv_data.h" #include <stdlib.h> #include <assert.h>
struct mv_clamp_rect { int to_left, to_right, to_top, to_bottom; }; static union mv clamp_mv(union mv raw, const struct mv_clamp_rect *bounds) { union mv newmv;
newmv.d.x = (raw.d.x < bounds->to_left) ? bounds->to_left : raw.d.x; newmv.d.x = (raw.d.x > bounds->to_right) ? bounds->to_right : newmv.d.x; newmv.d.y = (raw.d.y < bounds->to_top) ? bounds->to_top : raw.d.y; newmv.d.y = (raw.d.y > bounds->to_bottom) ? bounds->to_bottom : newmv.d.y; return newmv; }
static int read_segment_id(struct bool_decoder *bool, struct vp8_segment_hdr *seg) { return bool_get(bool, seg->tree_probs[0]) ? 2 + bool_get(bool, seg->tree_probs[2]) : bool_get(bool, seg->tree_probs[1]); }
static enum prediction_mode above_block_mode(const struct mb_info *this, const struct mb_info *above, unsigned int b) { if (b < 4) { switch (above->base.y_mode) { case DC_PRED: return B_DC_PRED; case V_PRED: return B_VE_PRED; case H_PRED: return B_HE_PRED;
okay the motion vector stuff is about to get *so* spicy 🔥💅
case TM_PRED: return B_TM_PRED; case B_PRED: return above->split.modes[b+12]; default: assert(0); } }
return this->split.modes[b-4]; }
static enum prediction_mode left_block_mode(const struct mb_info *this, const struct mb_info *left, unsigned int b) { if (!(b & 3)) { switch (left->base.y_mode) { case DC_PRED: return B_DC_PRED; case V_PRED: return B_VE_PRED; case H_PRED: return B_HE_PRED; case TM_PRED: return B_TM_PRED; case B_PRED: return left->split.modes[b+3]; default: assert(0); } }
return this->split.modes[b-1]; } static void decode_kf_mb_mode(struct mb_info *this, struct mb_info *left, struct mb_info *above, struct bool_decoder *bool) { int y_mode, uv_mode;
y_mode = bool_read_tree(bool, kf_y_mode_tree, kf_y_mode_probs);
if (y_mode == B_PRED) { unsigned int i;
for (i = 0; i < 16; i++) { enum prediction_mode a = above_block_mode(this, above, i); enum prediction_mode l = left_block_mode(this, left, i); enum prediction_mode b;
b = bool_read_tree(bool, b_mode_tree, kf_b_mode_probs[a][l]); this->split.modes[i] = b; } }
uv_mode = bool_read_tree(bool, uv_mode_tree, kf_uv_mode_probs);
this->base.y_mode = y_mode; this->base.uv_mode = uv_mode; this->base.mv.raw = 0; this->base.ref_frame = 0; }
static void decode_intra_mb_mode(struct mb_info this, struct vp8_entropy_hdr *hdr, struct bool_decoder *bool) { / Like decode_kf_mb_mode, but with probabilities transmitted in
- the bitstream and no context on the above/left block mode. */ int y_mode, uv_mode;
y_mode = bool_read_tree(bool, y_mode_tree, hdr->y_mode_probs); if (y_mode == B_PRED) { unsigned int i;
for (i = 0; i < 16; i++) { enum prediction_mode b;
b = bool_read_tree(bool, b_mode_tree, default_b_mode_probs); this->split.modes[i] = b; } }
uv_mode = bool_read_tree(bool, uv_mode_tree, hdr->uv_mode_probs);
this->base.y_mode = y_mode; this->base.uv_mode = uv_mode; this->base.mv.raw = 0; this->base.ref_frame = CURRENT_FRAME; }
static int read_mv_component(struct bool_decoder *bool, const unsigned char mvc[MV_PROB_CNT]) { enum {IS_SHORT, SIGN, SHORT, BITS = SHORT + 8 - 1, LONG_WIDTH = 10}; int x = 0;
if (bool_get(bool, mvc[IS_SHORT])) /* Large */ { int i = 0;
for (i = 0; i < 3; i++) x += bool_get(bool, mvc[BITS + i]) << i;
/* Skip bit 3, which is sometimes implicit */ for (i = LONG_WIDTH - 1; i > 3; i--) x += bool_get(bool, mvc[BITS + i]) << i;
if (!(x & 0xFFF0) || bool_get(bool, mvc[BITS + 3])) x += 8; } else /* small */ x = bool_read_tree(bool, small_mv_tree, mvc + SHORT);
if (x && bool_get(bool, mvc[SIGN])) x = -x;
return x << 1; }
static mv_t above_block_mv(const struct mb_info *this, const struct mb_info *above, unsigned int b) { if (b < 4) { if (above->base.y_mode == SPLITMV) return above->split.mvs[b+12];
return above->base.mv; }
return this->split.mvs[b-4]; }
static mv_t left_block_mv(const struct mb_info *this, const struct mb_info *left, unsigned int b) { if (!(b & 3)) { if (left->base.y_mode == SPLITMV) return left->split.mvs[b+3];
return left->base.mv; }
return this->split.mvs[b-1]; } static enum prediction_mode submv_ref(struct bool_decoder *bool, union mv l, union mv a) { enum subblock_mv_ref { SUBMVREF_NORMAL, SUBMVREF_LEFT_ZED, SUBMVREF_ABOVE_ZED, SUBMVREF_LEFT_ABOVE_SAME, SUBMVREF_LEFT_ABOVE_ZED };
int lez = !(l.raw); int aez = !(a.raw); int lea = l.raw == a.raw; enum subblock_mv_ref ctx = SUBMVREF_NORMAL;
if (lea && lez) ctx = SUBMVREF_LEFT_ABOVE_ZED; else if (lea) ctx = SUBMVREF_LEFT_ABOVE_SAME; else if (aez) ctx = SUBMVREF_ABOVE_ZED; else if (lez) ctx = SUBMVREF_LEFT_ZED;
return bool_read_tree(bool, submv_ref_tree, submv_ref_probs2[ctx]); }
static void read_mv(struct bool_decoder *bool, union mv *mv, mv_component_probs_t mvc[2]) { mv->d.y = read_mv_component(bool, mvc[0]); mv->d.x = read_mv_component(bool, mvc[1]); } static void mv_bias(const struct mb_info *mb, const unsigned int sign_bias[3], enum reference_frame ref_frame, union mv *mv) { if (sign_bias[mb->base.ref_frame] ^ sign_bias[ref_frame]) { mv->d.x *= -1; mv->d.y *= -1; } }
enum near_mv_v { CNT_BEST = 0, CNT_ZEROZERO = 0, CNT_NEAREST, CNT_NEAR, CNT_SPLITMV };
static void find_near_mvs(const struct mb_info *this, const struct mb_info *left, const struct mb_info *above, const unsigned int sign_bias[3], union mv near_mvs[4], int cnt[4]) { const struct mb_info *aboveleft = above - 1; union mv *mv = near_mvs; int *cntx = cnt;
/* Zero accumulators / mv[0].raw = mv[1].raw = mv[2].raw = 0; cnt[0] = cnt[1] = cnt[2] = cnt[3] = 0; / Process above */ if (above->base.ref_frame != CURRENT_FRAME) { if (above->base.mv.raw) { (++mv)->raw = above->base.mv.raw; mv_bias(above, sign_bias, this->base.ref_frame, mv); ++cntx; }
*cntx += 2; }
/* Process left */ if (left->base.ref_frame != CURRENT_FRAME) { if (left->base.mv.raw) { union mv this_mv;
this_mv.raw = left->base.mv.raw; mv_bias(left, sign_bias, this->base.ref_frame, &this_mv);
if (this_mv.raw != mv->raw) { (++mv)->raw = this_mv.raw; ++cntx; }
*cntx += 2; } else cnt[CNT_ZEROZERO] += 2; }
/* Process above left */ if (aboveleft->base.ref_frame != CURRENT_FRAME) { if (aboveleft->base.mv.raw) { union mv this_mv;
this_mv.raw = aboveleft->base.mv.raw; mv_bias(aboveleft, sign_bias, this->base.ref_frame, &this_mv);
if (this_mv.raw != mv->raw) { (++mv)->raw = this_mv.raw; ++cntx; }
*cntx += 1; } else cnt[CNT_ZEROZERO] += 1; }
/* If we have three distinct MVs ... */
if (cnt[CNT_SPLITMV]) { /* See if above-left MV can be merged with NEAREST */ if (mv->raw == near_mvs[CNT_NEAREST].raw) cnt[CNT_NEAREST] += 1; }
cnt[CNT_SPLITMV] = ((above->base.y_mode == SPLITMV)
- (left->base.y_mode == SPLITMV)) * 2
- (aboveleft->base.y_mode == SPLITMV);
/* Swap near and nearest if necessary */ if (cnt[CNT_NEAR] > cnt[CNT_NEAREST]) { int tmp; tmp = cnt[CNT_NEAREST]; cnt[CNT_NEAREST] = cnt[CNT_NEAR]; cnt[CNT_NEAR] = tmp; tmp = near_mvs[CNT_NEAREST].raw; near_mvs[CNT_NEAREST].raw = near_mvs[CNT_NEAR].raw; near_mvs[CNT_NEAR].raw = tmp; }
/* Use near_mvs[CNT_BEST] to store the "best" MV. Note that this
- storage shares the same address as near_mvs[CNT_ZEROZERO]. */ if (cnt[CNT_NEAREST] >= cnt[CNT_BEST]) near_mvs[CNT_BEST] = near_mvs[CNT_NEAREST]; } static void decode_split_mv(struct mb_info *this, const struct mb_info *left, const struct mb_info *above, struct vp8_entropy_hdr *hdr, union mv *best_mv, struct bool_decoder *bool) { const int *partition; int j, k, mask, partition_id;
partition_id = bool_read_tree(bool, split_mv_tree, split_mv_probs); partition = mv_partitions[partition_id]; this->base.partitioning = partition_id;
for (j = 0, mask = 0; mask < 65535; j++) { union mv mv, left_mv, above_mv; enum prediction_mode subblock_mode;
/* Find the first subblock in this partition. */ for (k = 0; j != partition[k]; k++);
/* Decode the next MV */ left_mv = left_block_mv(this, left, k); above_mv = above_block_mv(this, above, k); subblock_mode = submv_ref(bool, left_mv, above_mv);
switch (subblock_mode) { case LEFT4X4: mv = left_mv; break; case ABOVE4X4: mv = above_mv; break; case ZERO4X4: mv.raw = 0; break; case NEW4X4: read_mv(bool, &mv, hdr->mv_probs); mv.d.x += best_mv->d.x; mv.d.y += best_mv->d.y; break; default: assert(0); } /* Fill the MVs for this partition */ for (; k < 16; k++) if (j == partition[k]) { this->split.mvs[k] = mv; mask |= 1 << k; } } }
static int need_mc_border(union mv mv, int l, int t, int b_w, int w, int h) { int b, r;
/* Get distance to edge for top-left pixel */ l += (mv.d.x >> 3); t += (mv.d.y >> 3);
/* Get distance to edge for bottom-right pixel */ r = w - (l + b_w); b = h - (t + b_w);
return (l >> 1 < 2 || r >> 1 < 3 || t >> 1 < 2 || b >> 1 < 3); }
static void decode_mvs(struct vp8_decoder_ctx *ctx, struct mb_info *this, const struct mb_info *left, const struct mb_info *above, const struct mv_clamp_rect *bounds, struct bool_decoder *bool) { struct vp8_entropy_hdr *hdr = &ctx->entropy_hdr; union mv near_mvs[4]; union mv clamped_best_mv; int mv_cnts[4]; unsigned char probs[4]; enum {BEST, NEAREST, NEAR}; int x, y, w, h, b;
this->base.ref_frame = bool_get(bool, hdr->prob_last) ? 2 + bool_get(bool, hdr->prob_gf) : 1;
find_near_mvs(this, this - 1, above, ctx->reference_hdr.sign_bias, near_mvs, mv_cnts); probs[0] = mv_counts_to_probs[mv_cnts[0]][0]; probs[1] = mv_counts_to_probs[mv_cnts[1]][1]; probs[2] = mv_counts_to_probs[mv_cnts[2]][2]; probs[3] = mv_counts_to_probs[mv_cnts[3]][3];
this->base.y_mode = bool_read_tree(bool, mv_ref_tree, probs); this->base.uv_mode = this->base.y_mode;
this->base.need_mc_border = 0; x = (-bounds->to_left - 128) >> 3; y = (-bounds->to_top - 128) >> 3; w = ctx->mb_cols * 16; h = ctx->mb_rows * 16;
switch (this->base.y_mode) { case NEARESTMV: this->base.mv = clamp_mv(near_mvs[NEAREST], bounds); break; case NEARMV: this->base.mv = clamp_mv(near_mvs[NEAR], bounds); break; case ZEROMV: this->base.mv.raw = 0; return; //skip need_mc_border check case NEWMV: clamped_best_mv = clamp_mv(near_mvs[BEST], bounds); read_mv(bool, &this->base.mv, hdr->mv_probs); this->base.mv.d.x += clamped_best_mv.d.x; this->base.mv.d.y += clamped_best_mv.d.y; break; case SPLITMV: { union mv chroma_mv[4] = {{{0}}};
clamped_best_mv = clamp_mv(near_mvs[BEST], bounds); decode_split_mv(this, left, above, hdr, &clamped_best_mv, bool); this->base.mv = this->split.mvs[15]; for (b = 0; b < 16; b++) { chroma_mv[(b>>1&1) + (b>>2&2)].d.x += this->split.mvs[b].d.x; chroma_mv[(b>>1&1) + (b>>2&2)].d.y += this->split.mvs[b].d.y;
if (need_mc_border(this->split.mvs[b], x + (b & 3) * 4, y + (b & ~3), 4, w, h)) { this->base.need_mc_border = 1; break; } }
for (b = 0; b < 4; b++) { chroma_mv[b].d.x += 4 + 8 * (chroma_mv[b].d.x >> 31); chroma_mv[b].d.y += 4 + 8 * (chroma_mv[b].d.y >> 31); chroma_mv[b].d.x /= 4; chroma_mv[b].d.y /= 4;
//note we're passing in non-subsampled coordinates
if (need_mc_border(chroma_mv[b], x + (b & 1) * 8, y + (b >> 1) * 8, 16, w, h)) { this->base.need_mc_border = 1; break; } }
return; //skip need_mc_border check } default: assert(0); }
if (need_mc_border(this->base.mv, x, y, 16, w, h)) this->base.need_mc_border = 1; } void vp8_dixie_modemv_process_row(struct vp8_decoder_ctx *ctx, struct bool_decoder *bool, int row, int start_col, int num_cols) { struct mb_info *above, *this; unsigned int col; struct mv_clamp_rect bounds;
this = ctx->mb_info_rows[row] + start_col; above = ctx->mb_info_rows[row - 1] + start_col;
/* Calculate the eighth-pel MV bounds using a 1 MB border. */ bounds.to_left = -((start_col + 1) << 7); bounds.to_right = (ctx->mb_cols - start_col) << 7; bounds.to_top = -((row + 1) << 7); bounds.to_bottom = (ctx->mb_rows - row) << 7;
for (col = start_col; col < start_col + num_cols; col++) { if (ctx->segment_hdr.update_map) this->base.segment_id = read_segment_id(bool, &ctx->segment_hdr);
if (ctx->entropy_hdr.coeff_skip_enabled) this->base.skip_coeff = bool_get(bool, ctx->entropy_hdr.coeff_skip_prob);
if (ctx->frame_hdr.is_keyframe) { if (!ctx->segment_hdr.update_map) this->base.segment_id = 0;
decode_kf_mb_mode(this, this - 1, above, bool); } else { if (bool_get(bool, ctx->entropy_hdr.prob_inter)) decode_mvs(ctx, this, this - 1, above, &bounds, bool); else decode_intra_mb_mode(this, &ctx->entropy_hdr, bool);
bounds.to_left -= 16 << 3; bounds.to_right -= 16 << 3; } /* Advance to next mb */ this++; above++; } }
void vp8_dixie_modemv_init(struct vp8_decoder_ctx *ctx) { unsigned int mbi_w, mbi_h, i; struct mb_info *mbi;
mbi_w = ctx->mb_cols + 1; /* For left border col / mbi_h = ctx->mb_rows + 1; / For above border row */
if (ctx->frame_hdr.frame_size_updated) { free(ctx->mb_info_storage); ctx->mb_info_storage = NULL; free(ctx->mb_info_rows_storage); ctx->mb_info_rows_storage = NULL; }
if (!ctx->mb_info_storage) ctx->mb_info_storage = calloc(mbi_w * mbi_h, sizeof(*ctx->mb_info_storage));
if (!ctx->mb_info_rows_storage)
ctx->mb_info_rows_storage = calloc(mbi_h, sizeof(*ctx->mb_info_rows_storage));
/* Set up row pointers */ mbi = ctx->mb_info_storage + 1;
for (i = 0; i < mbi_h; i++) { ctx->mb_info_rows_storage[i] = mbi; mbi += mbi_w; }
ctx->mb_info_rows = ctx->mb_info_rows_storage + 1; } void vp8_dixie_modemv_destroy(struct vp8_decoder_ctx *ctx) { free(ctx->mb_info_storage); ctx->mb_info_storage = NULL; free(ctx->mb_info_rows_storage); ctx->mb_info_rows_storage = NULL; }
## 20.12. modemv.h 🎀
okay so this one is the header bestie, and it's giving tidy little declarations ✨ 💅 It's the same copyright and license blurb as the other files, and then it just declares the three functions we just went through: `init`, `destroy`, and `process_row`. Iconic. 🦋
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #ifndef MODEMV_H #define MODEMV_H
void vp8_dixie_modemv_init(struct vp8_decoder_ctx *ctx);
void vp8_dixie_modemv_destroy(struct vp8_decoder_ctx *ctx);
void vp8_dixie_modemv_process_row(struct vp8_decoder_ctx *ctx, struct bool_decoder *bool, int row, int start_col, int num_cols);
#endif
## 20.13. modemv_data.h 🎀
okay bestie, this one is all tables and trees, so buckle up 🧚
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
static const unsigned char kf_y_mode_probs[] = { 145, 156, 163, 128}; static const unsigned char kf_uv_mode_probs[] = { 142, 114, 183}; static const unsigned char kf_b_mode_probs[10][10][9] = { { /* above mode 0 / { / left mode 0 / 231, 120, 48, 89, 115, 113, 120, 152, 112}, { / left mode 1 / 152, 179, 64, 126, 170, 118, 46, 70, 95}, { / left mode 2 / 175, 69, 143, 80, 85, 82, 72, 155, 103}, { / left mode 3 / 56, 58, 10, 171, 218, 189, 17, 13, 152}, { / left mode 4 / 144, 71, 10, 38, 171, 213, 144, 34, 26}, { / left mode 5 / 114, 26, 17, 163, 44, 195, 21, 10, 173}, { / left mode 6 / 121, 24, 80, 195, 26, 62, 44, 64, 85}, { / left mode 7 / 170, 46, 55, 19, 136, 160, 33, 206, 71}, { / left mode 8 / 63, 20, 8, 114, 114, 208, 12, 9, 226}, { / left mode 9 / 81, 40, 11, 96, 182, 84, 29, 16, 36} }, { / above mode 1 / { / left mode 0 / 134, 183, 89, 137, 98, 101, 106, 165, 148}, { / left mode 1 / 72, 187, 100, 130, 157, 111, 32, 75, 80}, { / left mode 2 / 66, 102, 167, 99, 74, 62, 40, 234, 128}, { / left mode 3 / 41, 53, 9, 178, 241, 141, 26, 8, 107}, { / left mode 4 / 104, 79, 12, 27, 217, 255, 87, 17, 7}, { / left mode 5 / 74, 43, 26, 146, 73, 166, 49, 23, 157}, { / left mode 6 / 65, 38, 105, 160, 51, 52, 31, 115, 128}, { / left mode 7 / 87, 68, 71, 44, 114, 51, 15, 186, 23}, { / left mode 8 / 47, 41, 14, 110, 182, 183, 21, 17, 194}, { / left mode 9 / 66, 45, 25, 102, 197, 189, 23, 18, 22} }, { / above mode 2 / { / left mode 0 / 88, 88, 147, 150, 42, 46, 45, 196, 205}, { / left mode 1 / 43, 97, 183, 117, 85, 38, 35, 179, 61}, { / left mode 2 / 39, 53, 200, 87, 26, 21, 43, 232, 171}, { / left mode 3 / 56, 34, 51, 104, 114, 102, 29, 93, 77}, { / left mode 4 / 107, 54, 32, 26, 51, 1, 81, 43, 31}, { / left mode 5 / 39, 28, 85, 171, 58, 165, 90, 98, 64}, { / left mode 6 / 34, 22, 116, 206, 23, 34, 43, 166, 73}, { / left mode 7 / 68, 25, 106, 22, 64, 171, 36, 225, 114}, { / left mode 8 / 34, 19, 21, 102, 132, 188, 16, 76, 124}, { / left mode 9 / 62, 18, 78, 95, 85, 57, 50, 48, 51} }, { / above mode 3 / { / left mode 0 / 193, 101, 35, 159, 215, 111, 89, 46, 111}, { / left mode 1 / 60, 148, 31, 172, 219, 228, 21, 18, 111}, { / left mode 2 / 112, 113, 77, 85, 179, 255, 38, 120, 114}, { / left mode 3 / 40, 42, 1, 196, 245, 209, 10, 25, 109}, { / left mode 4 / 100, 80, 8, 43, 154, 1, 51, 26, 71}, { / left mode 5 / 88, 43, 29, 140, 166, 213, 37, 43, 154}, { / left mode 6 / 61, 63, 30, 155, 67, 45, 68, 1, 209}, { / left mode 7 / 142, 78, 78, 16, 255, 128, 34, 197, 171}, { / left mode 8 / 41, 40, 5, 102, 211, 183, 4, 1, 221}, { / left mode 9 / 51, 50, 17, 168, 209, 192, 23, 25, 82} }, { / above mode 4 / { / left mode 0 / 125, 98, 42, 88, 104, 85, 117, 175, 82}, { / left mode 1 / 95, 84, 53, 89, 128, 100, 113, 101, 45}, { / left mode 2 / 75, 79, 123, 47, 51, 128, 81, 171, 1}, { / left mode 3 / 57, 17, 5, 71, 102, 57, 53, 41, 49}, { / left mode 4 / 115, 21, 2, 10, 102, 255, 166, 23, 6}, { / left mode 5 / 38, 33, 13, 121, 57, 73, 26, 1, 85}, { / left mode 6 / 41, 10, 67, 138, 77, 110, 90, 47, 114}, { / left mode 7 / 101, 29, 16, 10, 85, 128, 101, 196, 26}, { / left mode 8 / 57, 18, 10, 102, 102, 213, 34, 20, 43}, { / left mode 9 / 117, 20, 15, 36, 163, 128, 68, 1, 26} }, { / above mode 5 / { / left mode 0 / 138, 31, 36, 171, 27, 166, 38, 44, 229}, { / left mode 1 / 67, 87, 58, 169, 82, 115, 26, 59, 179}, { / left mode 2 / 63, 59, 90, 180, 59, 166, 93, 73, 154}, { / left mode 3 / 40, 40, 21, 116, 143, 209, 34, 39, 175}, { / left mode 4 / 57, 46, 22, 24, 128, 1, 54, 17, 37}, { / left mode 5 / 47, 15, 16, 183, 34, 223, 49, 45, 183}, { / left mode 6 / 46, 17, 33, 183, 6, 98, 15, 32, 183}, { / left mode 7 / 65, 32, 73, 115, 28, 128, 23, 128, 205}, { / left mode 8 / 40, 3, 9, 115, 51, 192, 18, 6, 223}, { / left mode 9 / 87, 37, 9, 115, 59, 77, 64, 21, 47} }, { / above mode 6 / { / left mode 0 / 104, 55, 44, 218, 9, 54, 53, 130, 226}, { / left mode 1 / 64, 90, 70, 205, 40, 41, 23, 26, 57}, { / left mode 2 / 54, 57, 112, 184, 5, 41, 38, 166, 213}, { / left mode 3 / 30, 34, 26, 133, 152, 116, 10, 32, 134}, { / left mode 4 / 75, 32, 12, 51, 192, 255, 160, 43, 51}, { / left mode 5 / 39, 19, 53, 221, 26, 114, 32, 73, 255}, { / left mode 6 / 31, 9, 65, 234, 2, 15, 1, 118, 73}, { / left mode 7 / 88, 31, 35, 67, 102, 85, 55, 186, 85}, { / left mode 8 / 56, 21, 23, 111, 59, 205, 45, 37, 192}, { / left mode 9 / 55, 38, 70, 124, 73, 102, 1, 34, 98} }, { / above mode 7 / { / left mode 0 / 102, 61, 71, 37, 34, 53, 31, 243, 192}, { / left mode 1 / 69, 60, 71, 38, 73, 119, 28, 222, 37}, { / left mode 2 / 68, 45, 128, 34, 1, 47, 11, 245, 171}, { / left mode 3 / 62, 17, 19, 70, 146, 85, 55, 62, 70}, { / left mode 4 / 75, 15, 9, 9, 64, 255, 184, 119, 16}, { / left mode 5 / 37, 43, 37, 154, 100, 163, 85, 160, 1}, { / left mode 6 / 63, 9, 92, 136, 28, 64, 32, 201, 85}, { / left mode 7 / 86, 6, 28, 5, 64, 255, 25, 248, 1}, { / left mode 8 / 56, 8, 17, 132, 137, 255, 55, 116, 128}, { / left mode 9 / 58, 15, 20, 82, 135, 57, 26, 121, 40} }, { / above mode 8 / { / left mode 0 / 164, 50, 31, 137, 154, 133, 25, 35, 218}, { / left mode 1 / 51, 103, 44, 131, 131, 123, 31, 6, 158}, { / left mode 2 / 86, 40, 64, 135, 148, 224, 45, 183, 128}, { / left mode 3 / 22, 26, 17, 131, 240, 154, 14, 1, 209}, { / left mode 4 / 83, 12, 13, 54, 192, 255, 68, 47, 28}, { / left mode 5 / 45, 16, 21, 91, 64, 222, 7, 1, 197}, { / left mode 6 / 56, 21, 39, 155, 60, 138, 23, 102, 213}, { / left mode 7 / 85, 26, 85, 85, 128, 128, 32, 146, 171}, { / left mode 8 / 18, 11, 7, 63, 144, 171, 4, 4, 246}, { / left mode 9 / 35, 27, 10, 146, 174, 171, 12, 26, 128} }, { / above mode 9 / { / left mode 0 / 190, 80, 35, 99, 180, 80, 126, 54, 45}, { / left mode 1 / 85, 126, 47, 87, 176, 51, 41, 20, 32}, { / left mode 2 / 101, 75, 128, 139, 118, 146, 116, 128, 85}, { / left mode 3 / 56, 41, 15, 176, 236, 85, 37, 9, 62}, { / left mode 4 / 146, 36, 19, 30, 171, 255, 97, 27, 20}, { / left mode 5 / 71, 30, 17, 119, 118, 255, 17, 18, 138}, { / left mode 6 / 101, 38, 60, 138, 55, 70, 43, 26, 142}, { / left mode 7 / 138, 45, 61, 62, 219, 1, 81, 188, 64}, { / left mode 8 / 32, 41, 20, 117, 151, 142, 20, 21, 163}, { / left mode 9 / 112, 19, 12, 61, 195, 128, 48, 4, 24} } }; static const int kf_y_mode_tree[] = { -B_PRED, 2, 4, 6, -DC_PRED, -V_PRED, -H_PRED, -TM_PRED }; static const int y_mode_tree[] = { -DC_PRED, 2, 4, 6, -V_PRED, -H_PRED, -TM_PRED, -B_PRED }; static const int uv_mode_tree[6] = { -DC_PRED, 2, -V_PRED, 4, -H_PRED, -TM_PRED }; static const int b_mode_tree[18] = { -B_DC_PRED, 2, / 0 = DC_NODE / -B_TM_PRED, 4, / 1 = TM_NODE / -B_VE_PRED, 6, / 2 = VE_NODE / 8, 12, / 3 = COM_NODE / -B_HE_PRED, 10, / 4 = HE_NODE / -B_RD_PRED, -B_VR_PRED, / 5 = RD_NODE / -B_LD_PRED, 14, / 6 = LD_NODE / -B_VL_PRED, 16, / 7 = VL_NODE / -B_HD_PRED, -B_HU_PRED / 8 = HD_NODE */ }; static const int small_mv_tree[14] = { 2, 8, 4, 6, -0, -1, -2, -3, 10, 12, -4, -5, -6, -7 }; static const int mv_ref_tree[8] = { -ZEROMV, 2, -NEARESTMV, 4, -NEARMV, 6, -NEWMV, -SPLITMV }; static const int submv_ref_tree[6] = { -LEFT4X4, 2, -ABOVE4X4, 4, -ZERO4X4, -NEW4X4 }; static const int split_mv_tree[6] = { -3, 2, -2, 4, -0, -1 }; static const unsigned char default_b_mode_probs[] = { 120, 90, 79, 133, 87, 85, 80, 111, 151}; static const unsigned char mv_counts_to_probs[6][4] = { { 7, 1, 1, 143 }, { 14, 18, 14, 107 }, { 135, 64, 57, 68 }, { 60, 56, 128, 65 }, { 159, 134, 128, 34 }, { 234, 188, 128, 28 }
}; static const unsigned char split_mv_probs[3] = { 110, 111, 150};
static const unsigned char submv_ref_probs2[5][3] = { { 147, 136, 18 }, { 106, 145, 1 }, { 179, 121, 1 }, { 223, 1, 34 }, { 208, 1, 1 } }; const static int mv_partitions[4][16] = { {0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1 }, {0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1 }, {0, 0, 1, 1, 0, 0, 1, 1, 2, 2, 3, 3, 2, 2, 3, 3 }, {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } };
---
## 20.14. predict.c ✨
this one is where the pixels get predicted, and it starts with the usual license header and includes 💅
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #include "dixie.h" #include "predict.h" #include "idct_add.h" #include "mem.h" #include <assert.h> #include <string.h>
enum { BORDER_PIXELS = 16, };
okay bestie, this is all code, and code is the spec, so it stays EXACTLY the same 💅✨
static const filter_t sixtap_filters[8] = { { 0, 0, 128, 0, 0, 0 }, { 0, -6, 123, 12, -1, 0 }, { 2, -11, 108, 36, -8, 1 }, { 0, -9, 93, 50, -6, 0 }, { 3, -16, 77, 77, -16, 3 }, { 0, -6, 50, 93, -9, 0 }, { 1, -8, 36, 108, -11, 2 }, { 0, -1, 12, 123, -6, 0 } }; static const filter_t bilinear_filters[8] = { { 0, 0, 128, 0, 0, 0 }, { 0, 0, 112, 16, 0, 0 }, { 0, 0, 96, 32, 0, 0 }, { 0, 0, 80, 48, 0, 0 }, { 0, 0, 64, 64, 0, 0 }, { 0, 0, 48, 80, 0, 0 }, { 0, 0, 32, 96, 0, 0 }, { 0, 0, 16, 112, 0, 0 } };
static void predict_h_nxn(unsigned char *predict, int stride, int n) { unsigned char *left = predict - 1; int i, j;
for (i = 0; i < n; i++) for (j = 0; j < n; j++) predict[i *stride + j] = left[i * stride]; }
static void predict_v_nxn(unsigned char *predict, int stride, int n) { unsigned char *above = predict - stride; int i, j;
for (i = 0; i < n; i++) for (j = 0; j < n; j++) predict[i stride + j] = above[j]; } static void predict_tm_nxn(unsigned char *predict, int stride, int n) { / Transposes the left column to the top row for later
- consumption by the idct/recon stage */ unsigned char *left = predict - 1; unsigned char *above = predict - stride; unsigned char p = above[-1]; int i, j;
for (j = 0; j < n; j++) { for (i = 0; i < n; i++) predict[i] = CLAMP_255(*left + above[i] - p);
predict += stride; left += stride; } }
static void predict_dc_nxn(unsigned char *predict, int stride, int n) { unsigned char *left = predict - 1; unsigned char *above = predict - stride; int i, j, dc = 0;
for (i = 0; i < n; i++) { dc += *left + above[i]; left += stride; } switch (n) { case 16: dc = (dc + 16) >> 5; break; case 8: dc = (dc + 8) >> 4; break; case 4: dc = (dc + 4) >> 3; break; }
for (i = 0; i < n; i++) for (j = 0; j < n; j++) predict[i *stride + j] = dc; }
static void predict_ve_4x4(unsigned char *predict, int stride) { unsigned char *above = predict - stride; int i, j;
predict[0] = (above[-1] + 2 * above[0] + above[1] + 2) >> 2; predict[1] = (above[ 0] + 2 * above[1] + above[2] + 2) >> 2; predict[2] = (above[ 1] + 2 * above[2] + above[3] + 2) >> 2; predict[3] = (above[ 2] + 2 * above[3] + above[4] + 2) >> 2;
for (i = 1; i < 4; i++) for (j = 0; j < 4; j++) predict[i *stride + j] = predict[j]; } static void predict_he_4x4(unsigned char *predict, int stride) { unsigned char *left = predict - 1;
predict[0] = predict[1] = predict[2] = predict[3] = (left[-stride] + 2 * left[0] + left[stride] + 2) >> 2; predict += stride; left += stride;
predict[0] = predict[1] = predict[2] = predict[3] = (left[-stride] + 2 * left[0] + left[stride] + 2) >> 2; predict += stride; left += stride;
predict[0] = predict[1] = predict[2] = predict[3] = (left[-stride] + 2 * left[0] + left[stride] + 2) >> 2; predict += stride; left += stride;
predict[0] = predict[1] = predict[2] = predict[3] = (left[-stride] + 2 * left[0] + left[0] + 2) >> 2; } static void predict_ld_4x4(unsigned char *predict, int stride) { unsigned char *above = predict - stride; int pred0, pred1, pred2, pred3, pred4, pred5, pred6;
predict[0] = pred0 = (above[0] + 2 * above[1] + above[2] + 2) >> 2; predict[1] = pred1 = (above[1] + 2 * above[2] + above[3] + 2) >> 2; predict[2] = pred2 = (above[2] + 2 * above[3] + above[4] + 2) >> 2; predict[3] = pred3 = (above[3] + 2 * above[4] + above[5] + 2) >> 2; predict += stride;
predict[0] = pred1; predict[1] = pred2; predict[2] = pred3; predict[3] = pred4 = (above[4] + 2 * above[5] + above[6] + 2) >> 2; predict += stride;
predict[0] = pred2; predict[1] = pred3; predict[2] = pred4; predict[3] = pred5 = (above[5] + 2 * above[6] + above[7] + 2) >> 2; predict += stride;
predict[0] = pred3; predict[1] = pred4; predict[2] = pred5; predict[3] = pred6 = (above[6] + 2 * above[7] + above[7] + 2) >> 2; } static void predict_rd_4x4(unsigned char *predict, int stride) { unsigned char *left = predict - 1; unsigned char *above = predict - stride; int pred0, pred1, pred2, pred3, pred4, pred5, pred6;
predict[0] = pred0 = (left[ 0] + 2 * above[-1] + above[0] + 2) >> 2; predict[1] = pred1 = (above[-1] + 2 * above[ 0] + above[1] + 2) >> 2; predict[2] = pred2 = (above[ 0] + 2 * above[ 1] + above[2] + 2) >> 2; predict[3] = pred3 = (above[ 1] + 2 * above[ 2] + above[3] + 2) >> 2; predict += stride;
predict[0] = pred4 = (left[stride] + 2 * left[0] + above[-1] + 2) >> 2; predict[1] = pred0; predict[2] = pred1; predict[3] = pred2; predict += stride;
predict[0] = pred5 = (left[stride*2] + 2 * left[stride] + left[0] + 2) >> 2; predict[1] = pred4; predict[2] = pred0; predict[3] = pred1; predict += stride;
predict[0] = pred6 = (left[stride*3] + 2 * left[stride*2] + left[stride] + 2) >> 2; predict[1] = pred5; predict[2] = pred4; predict[3] = pred0; } static void predict_vr_4x4(unsigned char *predict, int stride) { unsigned char *left = predict - 1; unsigned char *above = predict - stride; int pred0, pred1, pred2, pred3, pred4, pred5, pred6, pred7, pred8, pred9;
predict[0] = pred0 = (above[-1] + above[0] + 1) >> 1; predict[1] = pred1 = (above[ 0] + above[1] + 1) >> 1; predict[2] = pred2 = (above[ 1] + above[2] + 1) >> 1; predict[3] = pred3 = (above[ 2] + above[3] + 1) >> 1; predict += stride;
predict[0] = pred4 = (left[ 0] + 2 * above[-1] + above[0] + 2) >> 2; predict[1] = pred5 = (above[-1] + 2 * above[ 0] + above[1] + 2) >> 2; predict[2] = pred6 = (above[ 0] + 2 * above[ 1] + above[2] + 2) >> 2; predict[3] = pred7 = (above[ 1] + 2 * above[ 2] + above[3] + 2) >> 2; predict += stride;
predict[0] = pred8 = (left[stride] + 2 * left[0] + above[-1] + 2) >> 2; predict[1] = pred0; predict[2] = pred1; predict[3] = pred2; predict += stride;
predict[0] = pred9 = (left[stride*2] + 2 * left[stride] + left[0] + 2) >> 2; predict[1] = pred4; predict[2] = pred5; predict[3] = pred6; } static void predict_vl_4x4(unsigned char *predict, int stride) { unsigned char *above = predict - stride; int pred0, pred1, pred2, pred3, pred4, pred5, pred6, pred7, pred8, pred9;
predict[0] = pred0 = (above[0] + above[1] + 1) >> 1; predict[1] = pred1 = (above[1] + above[2] + 1) >> 1; predict[2] = pred2 = (above[2] + above[3] + 1) >> 1; predict[3] = pred3 = (above[3] + above[4] + 1) >> 1; predict += stride;
predict[0] = pred4 = (above[0] + 2 * above[1] + above[2] + 2) >> 2; predict[1] = pred5 = (above[1] + 2 * above[2] + above[3] + 2) >> 2; predict[2] = pred6 = (above[2] + 2 * above[3] + above[4] + 2) >> 2; predict[3] = pred7 = (above[3] + 2 * above[4] + above[5] + 2) >> 2; predict += stride;
predict[0] = pred1; predict[1] = pred2; predict[2] = pred3; predict[3] = pred8 = (above[4] + 2 * above[5] + above[6] + 2) >> 2; predict += stride;
predict[0] = pred5; predict[1] = pred6; predict[2] = pred7; predict[3] = pred9 = (above[5] + 2 * above[6] + above[7] + 2) >> 2; } static void predict_hd_4x4(unsigned char *predict, int stride) { unsigned char *left = predict - 1; unsigned char *above = predict - stride; int pred0, pred1, pred2, pred3, pred4, pred5, pred6, pred7, pred8, pred9;
predict[0] = pred0 = (left[ 0] + above[-1] + 1) >> 1; predict[1] = pred1 = (left[ 0] + 2 * above[-1] + above[0] + 2) >> 2; predict[2] = pred2 = (above[-1] + 2 * above[ 0] + above[1] + 2) >> 2; predict[3] = pred3 = (above[ 0] + 2 * above[ 1] + above[2] + 2) >> 2; predict += stride;
predict[0] = pred4 = (left[stride] + left[0] + 1) >> 1; predict[1] = pred5 = (left[stride] + 2 * left[0] + above[-1] + 2) >> 2; predict[2] = pred0; predict[3] = pred1; predict += stride;
predict[0] = pred6 = (left[stride*2] + left[stride] + 1) >> 1; predict[1] = pred7 = (left[stride*2] + 2 * left[stride] + left[0] + 2) >> 2; predict[2] = pred4; predict[3] = pred5; predict += stride;
predict[0] = pred8 = (left[stride*3] + left[stride*2] + 1) >> 1; predict[1] = pred9 = (left[stride*3] + 2 * left[stride*2] + left[stride] + 2) >> 2; predict[2] = pred6; predict[3] = pred7; } static void predict_hu_4x4(unsigned char *predict, int stride) { unsigned char *left = predict - 1; int pred0, pred1, pred2, pred3, pred4, pred5, pred6;
predict[0] = pred0 = (left[stride*0] + left[stride*1] + 1) >> 1; predict[1] = pred1 = (left[stride*0] + 2 * left[stride*1] + left[stride*2] + 2) >> 2; predict[2] = pred2 = (left[stride*1] + left[stride*2] + 1) >> 1; predict[3] = pred3 = (left[stride*1] + 2 * left[stride*2] + left[stride*3] + 2) >> 2; predict += stride;
predict[0] = pred2; predict[1] = pred3; predict[2] = pred4 = (left[stride*2] + left[stride*3] + 1) >> 1; predict[3] = pred5 = (left[stride*2] + 2 * left[stride*3] + left[stride*3] + 2) >> 2; predict += stride;
predict[0] = pred4; predict[1] = pred5; predict[2] = pred6 = left[stride*3]; predict[3] = pred6; predict += stride;
predict[0] = pred6; predict[1] = pred6; predict[2] = pred6; predict[3] = pred6; }
static void predict_h_16x16(unsigned char *predict, int stride) { predict_h_nxn(predict, stride, 16); }
static void predict_v_16x16(unsigned char *predict, int stride) { predict_v_nxn(predict, stride, 16); } static void predict_tm_16x16(unsigned char *predict, int stride) { predict_tm_nxn(predict, stride, 16); }
static void predict_h_8x8(unsigned char *predict, int stride) { predict_h_nxn(predict, stride, 8); }
static void predict_v_8x8(unsigned char *predict, int stride) { predict_v_nxn(predict, stride, 8); }
static void predict_tm_8x8(unsigned char *predict, int stride) { predict_tm_nxn(predict, stride, 8); }
static void predict_tm_4x4(unsigned char *predict, int stride) { predict_tm_nxn(predict, stride, 4); }
static void copy_down(unsigned char recon, int stride) { / Copy the four pixels above-right of subblock 3 to
- above-right of subblocks 7, 11, and 15 */ uint32_t tmp, *copy = (void *)(recon + 16 - stride); stride = stride / sizeof(unsigned int); tmp = *copy; copy += stride * 4; *copy = tmp; copy += stride * 4; *copy = tmp; copy += stride * 4; *copy = tmp; }
static void b_pred(unsigned char *predict, int stride, struct mb_info *mbi, short *coeffs) { int i;
copy_down(predict, stride);
for (i = 0; i < 16; i++) { unsigned char *b_predict = predict + (i & 3) * 4;
switch (mbi->split.modes[i]) { case B_DC_PRED: predict_dc_nxn(b_predict, stride, 4); break; case B_TM_PRED: predict_tm_4x4(b_predict, stride); break; case B_VE_PRED: predict_ve_4x4(b_predict, stride); break; case B_HE_PRED: predict_he_4x4(b_predict, stride); break; case B_LD_PRED: predict_ld_4x4(b_predict, stride); break; case B_RD_PRED: predict_rd_4x4(b_predict, stride); break; case B_VR_PRED: predict_vr_4x4(b_predict, stride); break;
case B_VL_PRED: predict_vl_4x4(b_predict, stride); break; case B_HD_PRED: predict_hd_4x4(b_predict, stride); break; case B_HU_PRED: predict_hu_4x4(b_predict, stride); break; default: assert(0); }
vp8_dixie_idct_add(b_predict, b_predict, stride, coeffs); coeffs += 16;
if ((i & 3) == 3) { predict += stride * 4; } } }
static void
fixup_dc_coeffs(struct mb_info *mbi, short *coeffs) { short y2[16]; int i;
vp8_dixie_walsh(coeffs + 24 * 16, y2);
for (i = 0; i < 16; i++) coeffs[i*16] = y2[i]; }
static void predict_intra_luma(unsigned char *predict, int stride, struct mb_info *mbi, short *coeffs) { if (mbi->base.y_mode == B_PRED) b_pred(predict, stride, mbi, coeffs); else { int i;
switch (mbi->base.y_mode) { case DC_PRED: predict_dc_nxn(predict, stride, 16); break; case V_PRED: predict_v_16x16(predict, stride); break; case H_PRED: predict_h_16x16(predict, stride); break; case TM_PRED: predict_tm_16x16(predict, stride); break; default: assert(0); }
fixup_dc_coeffs(mbi, coeffs);
for (i = 0; i < 16; i++) { vp8_dixie_idct_add(predict, predict, stride, coeffs); coeffs += 16; predict += 4;
if ((i & 3) == 3) predict += stride * 4 - 16; } } }
static void predict_intra_chroma(unsigned char *predict_u, unsigned char *predict_v, int stride, struct mb_info *mbi, short *coeffs) { int i; switch (mbi->base.uv_mode) { case DC_PRED: predict_dc_nxn(predict_u, stride, 8); predict_dc_nxn(predict_v, stride, 8); break; case V_PRED: predict_v_8x8(predict_u, stride); predict_v_8x8(predict_v, stride); break; case H_PRED: predict_h_8x8(predict_u, stride); predict_h_8x8(predict_v, stride); break; case TM_PRED: predict_tm_8x8(predict_u, stride); predict_tm_8x8(predict_v, stride); break; default: assert(0); }
coeffs += 16 * 16;
for (i = 16; i < 20; i++) { vp8_dixie_idct_add(predict_u, predict_u, stride, coeffs); coeffs += 16; predict_u += 4;
if (i & 1) predict_u += stride * 4 - 8; }
for (i = 20; i < 24; i++) { vp8_dixie_idct_add(predict_v, predict_v, stride, coeffs); coeffs += 16; predict_v += 4;
if (i & 1) predict_v += stride * 4 - 8; } } static void sixtap_horiz(unsigned char *output, int output_stride, const unsigned char *reference, int reference_stride, int cols, int rows, const filter_t filter ) { int r, c, temp;
for (r = 0; r < rows; r++) { for (c = 0; c < cols; c++) { temp = (reference[-2] * filter[0]) + (reference[-1] * filter[1]) + (reference[ 0] * filter[2]) + (reference[ 1] * filter[3]) + (reference[ 2] * filter[4]) + (reference[ 3] * filter[5]) + 64; temp >>= 7; output[c] = CLAMP_255(temp); reference++; }
reference += reference_stride - cols; output += output_stride; } }
static void sixtap_vert(unsigned char *output, int output_stride, const unsigned char *reference, int reference_stride, int cols, int rows, const filter_t filter ) { int r, c, temp;
for (r = 0; r < rows; r++) {
for (c = 0; c < cols; c++) { temp = (reference[-2*reference_stride] * filter[0]) + (reference[-1*reference_stride] * filter[1]) + (reference[ 0*reference_stride] * filter[2]) + (reference[ 1*reference_stride] * filter[3]) + (reference[ 2*reference_stride] * filter[4]) + (reference[ 3*reference_stride] * filter[5]) + 64; temp >>= 7; output[c] = CLAMP_255(temp); reference++; }
reference += reference_stride - cols; output += output_stride; } }
static void sixtap_2d(unsigned char output, int output_stride, const unsigned char *reference, int reference_stride, int cols, int rows, int mx, int my, const filter_t filters[8] ) { DECLARE_ALIGNED(16, unsigned char, temp[16(16+5)]);
sixtap_horiz(temp, 16, reference - 2 * reference_stride, reference_stride, cols, rows + 5, filters[mx]); sixtap_vert(output, output_stride, temp + 2 * 16, 16, cols, rows, filters[my]); }
struct img_index { unsigned char *y, *u, *v; int stride, uv_stride; }; static const unsigned char * filter_block(unsigned char *output, const unsigned char *reference, int stride, const union mv *mv, const filter_t filters[8]) { int mx, my;
/* Handle 0,0 as a special case. TODO: Does this make it any
- faster? */ if (!mv->raw) return reference;
mx = mv->d.x & 7; my = mv->d.y & 7; reference += ((mv->d.y >> 3) * stride) + (mv->d.x >> 3);
if (mx | my) { sixtap_2d(output, stride, reference, stride, 4, 4, mx, my, filters); reference = output; }
return reference; }
static void recon_1_block(unsigned char *output, const unsigned char *reference, int stride, const union mv *mv, const filter_t filters[8], short *coeffs, struct mb_info *mbi, int b ) { const unsigned char *predict;
predict = filter_block(output, reference, stride, mv, filters); vp8_dixie_idct_add(output, predict, stride, coeffs + 16 * b); } static mv_t calculate_chroma_splitmv(struct mb_info *mbi, int b, int full_pixel) { int temp; union mv mv;
temp = mbi->split.mvs[b].d.x + mbi->split.mvs[b+1].d.x + mbi->split.mvs[b+4].d.x + mbi->split.mvs[b+5].d.x;
if (temp < 0) temp -= 4; else temp += 4;
mv.d.x = temp / 8;
temp = mbi->split.mvs[b].d.y + mbi->split.mvs[b+1].d.y + mbi->split.mvs[b+4].d.y + mbi->split.mvs[b+5].d.y;
if (temp < 0) temp -= 4; else temp += 4;
mv.d.y = temp / 8;
if (full_pixel) { mv.d.x &= ~7; mv.d.y &= ~7; }
bestie the `sixtap_horiz` and `sixtap_vert` pair is giving symmetry 🦋✨ 💖
<pink>okay so this bit is where the decoder handles the edges of the frame and keeps track of reference frames 🧚💖</pink>
return mv; } /* Note: We rely on the reconstructed border having the same stride
- as the reference buffer because the filter_block can't adjust the
- stride with its return value, only the reference pointer. */ static void build_mc_border(unsigned char *dst, const unsigned char *src, int stride, int x, int y, int b_w, int b_h, int w, int h ) { const unsigned char *ref_row;
/* Get a pointer to the start of the real data for this row */ ref_row = src - x - y * stride;
if (y >= h) ref_row += (h - 1) * stride; else if (y > 0) ref_row += y * stride;
do { int left, right = 0, copy;
left = x < 0 ? -x : 0;
if (left > b_w) left = b_w;
if (x + b_w > w) right = x + b_w - w;
if (right > b_w) right = b_w;
copy = b_w - left - right;
if (left) memset(dst, ref_row[0], left);
if (copy) memcpy(dst + left, ref_row + x + left, copy);
if (right) memset(dst + left + copy, ref_row[w-1], right);
dst += stride; y++;
if (y < h && y > 0) ref_row += stride; } while (--b_h); }
static void recon_1_edge_block(unsigned char *output, unsigned char *emul_block, const unsigned char *reference, int stride, const union mv *mv, const filter_t filters[8], short *coeffs, struct mb_info *mbi, int x, int y, int w, int h, int start_b ) { const unsigned char *predict; int b = start_b; const int b_w = 4; const int b_h = 4;
x += mv->d.x >> 3; y += mv->d.y >> 3; /* Need two pixels left/above, 3 right/below for 6-tap */ if (x < 2 || x + b_w - 1 + 3 >= w || y < 2 || y + b_h - 1 + 3 >= h) { reference += (mv->d.x >> 3) + (mv->d.y >> 3) * stride; build_mc_border(emul_block, reference - 2 - 2 * stride, stride, x - 2, y - 2, b_w + 5, b_h + 5, w, h); reference = emul_block + 2 * stride + 2; reference -= (mv->d.x >> 3) + (mv->d.y >> 3) * stride; }
predict = filter_block(output, reference, stride, mv, filters); vp8_dixie_idct_add(output, predict, stride, coeffs + 16 * b); }
static void predict_inter_emulated_edge(struct vp8_decoder_ctx ctx, struct img_index *img, short *coeffs, struct mb_info *mbi, int mb_col, int mb_row) { / TODO: Move this into its own buffer. This only works because
- we still have a border allocated. */ unsigned char *emul_block = ctx->frame_strg[0].img.img_data; unsigned char *reference; unsigned char *output; ptrdiff_t reference_offset; int w, h, x, y, b; union mv chroma_mv[4]; unsigned char *u = img->u, *v = img->v; int full_pixel = ctx->frame_hdr.version == 3;
x = mb_col * 16; y = mb_row * 16; w = ctx->mb_cols * 16; h = ctx->mb_rows * 16;
output = img->y; reference_offset = ctx->ref_frame_offsets[mbi->base.ref_frame]; reference = output + reference_offset; if (mbi->base.y_mode != SPLITMV) { union mv uvmv;
uvmv = mbi->base.mv; uvmv.d.x = (uvmv.d.x + 1 + (uvmv.d.x >> 31) * 2) / 2; uvmv.d.y = (uvmv.d.y + 1 + (uvmv.d.y >> 31) * 2) / 2;
if (full_pixel) { uvmv.d.x &= ~7; uvmv.d.y &= ~7; }
chroma_mv[0] = uvmv; chroma_mv[1] = uvmv; chroma_mv[2] = uvmv; chroma_mv[3] = uvmv; } else { chroma_mv[0] = calculate_chroma_splitmv(mbi, 0, full_pixel); chroma_mv[1] = calculate_chroma_splitmv(mbi, 2, full_pixel); chroma_mv[2] = calculate_chroma_splitmv(mbi, 8, full_pixel); chroma_mv[3] = calculate_chroma_splitmv(mbi, 10, full_pixel); }
/* Luma */ for (b = 0; b < 16; b++) { union mv *ymv;
if (mbi->base.y_mode != SPLITMV) ymv = &mbi->base.mv; else ymv = mbi->split.mvs + b;
recon_1_edge_block(output, emul_block, reference, img->stride, ymv, ctx->subpixel_filters, coeffs, mbi, x, y, w, h, b);
x += 4; output += 4; reference += 4; if ((b & 3) == 3) { x -= 16; y += 4; output += 4 * img->stride - 16; reference += 4 * img->stride - 16; } }
x = mb_col * 16; y = mb_row * 16;
/* Chroma */ x >>= 1; y >>= 1; w >>= 1; h >>= 1;
for (b = 0; b < 4; b++) { recon_1_edge_block(u, emul_block, u + reference_offset, img->uv_stride, &chroma_mv[b], ctx->subpixel_filters, coeffs, mbi, x, y, w, h, b + 16); recon_1_edge_block(v, emul_block, v + reference_offset, img->uv_stride, &chroma_mv[b], ctx->subpixel_filters, coeffs, mbi, x, y, w, h, b + 20); u += 4; v += 4; x += 4;
if (b & 1) { x -= 8; y += 4; u += 4 * img->uv_stride - 8; v += 4 * img->uv_stride - 8; } }
}
static void predict_inter(struct vp8_decoder_ctx *ctx, struct img_index *img, short *coeffs, struct mb_info *mbi)
{ unsigned char *y = img->y; unsigned char *u = img->u; unsigned char *v = img->v; ptrdiff_t reference_offset; union mv chroma_mv[4]; int full_pixel = ctx->frame_hdr.version == 3; int b;
if (mbi->base.y_mode != SPLITMV) { union mv uvmv;
uvmv = mbi->base.mv; uvmv.d.x = (uvmv.d.x + 1 + (uvmv.d.x >> 31) * 2) / 2; uvmv.d.y = (uvmv.d.y + 1 + (uvmv.d.y >> 31) * 2) / 2;
if (full_pixel) { uvmv.d.x &= ~7; uvmv.d.y &= ~7; }
chroma_mv[0] = chroma_mv[1] = chroma_mv[2] = chroma_mv[3] = uvmv; } else { chroma_mv[0] = calculate_chroma_splitmv(mbi, 0, full_pixel); chroma_mv[1] = calculate_chroma_splitmv(mbi, 2, full_pixel); chroma_mv[2] = calculate_chroma_splitmv(mbi, 8, full_pixel); chroma_mv[3] = calculate_chroma_splitmv(mbi, 10, full_pixel); }
reference_offset = ctx->ref_frame_offsets[mbi->base.ref_frame];
for (b = 0; b < 16; b++) { union mv *ymv;
if (mbi->base.y_mode != SPLITMV) ymv = &mbi->base.mv; else ymv = mbi->split.mvs + b; recon_1_block(y, y + reference_offset, img->stride, ymv, ctx->subpixel_filters, coeffs, mbi, b);
y += 4;
if ((b & 3) == 3) y += 4 * img->stride - 16; }
for (b = 0; b < 4; b++) { recon_1_block(u, u + reference_offset, img->uv_stride, &chroma_mv[b], ctx->subpixel_filters, coeffs, mbi, b + 16); recon_1_block(v, v + reference_offset, img->uv_stride, &chroma_mv[b], ctx->subpixel_filters, coeffs, mbi, b + 20); u += 4; v += 4;
if (b & 1) { u += 4 * img->uv_stride - 8; v += 4 * img->uv_stride - 8; } } }
void vp8_dixie_release_ref_frame(struct ref_cnt_img *rcimg) { if (rcimg) { assert(rcimg->ref_cnt); rcimg->ref_cnt--; } }
struct ref_cnt_img * vp8_dixie_ref_frame(struct ref_cnt_img *rcimg) { rcimg->ref_cnt++; return rcimg; } struct ref_cnt_img * vp8_dixie_find_free_ref_frame(struct ref_cnt_img *frames) { int i;
for (i = 0; i < NUM_REF_FRAMES; i++) if (frames[i].ref_cnt == 0) { frames[i].ref_cnt = 1; return &frames[i]; }
assert(0); return NULL; }
static void fixup_left(unsigned char predict, int width, int stride, unsigned int row, enum prediction_mode mode) { / The left column of out-of-frame pixels is taken to be 129,
- unless we're doing DC_PRED, in which case we duplicate the
- above row, unless this is also row 0, in which case we use
- 129. */ unsigned char *left = predict - 1; int i;
if (mode == DC_PRED && row) { unsigned char *above = predict - stride;
for (i = 0; i < width; i++) { left = above[i]; left += stride; } } else { / Need to re-set the above row, in case the above MB was
- DC_PRED. */ left -= stride; for (i = -1; i < width; i++) { *left = 129; left += stride; } } }
static void fixup_above(unsigned char predict, int width, int stride, unsigned int col, enum prediction_mode mode) { / The above row of out-of-frame pixels is taken to be 127,
- unless we're doing DC_PRED, in which case we duplicate the
- left col, unless this is also col 0, in which case we use
- 127. */ unsigned char *above = predict - stride; int i;
if (mode == DC_PRED && col) { unsigned char *left = predict - 1;
for (i = 0; i < width; i++) { above[i] = left; left += stride; } } else / Need to re-set the left col, in case the last MB was
- DC_PRED. */ memset(above - 1, 127, width + 1);
memset(above + width, 127, 4); // for above-right subblock modes } void vp8_dixie_predict_init(struct vp8_decoder_ctx *ctx) {
int i; unsigned char *this_frame_base;
if (ctx->frame_hdr.frame_size_updated) { for (i = 0; i < NUM_REF_FRAMES; i++) { unsigned int w = ctx->mb_cols * 16 + BORDER_PIXELS * 2; unsigned int h = ctx->mb_rows * 16 + BORDER_PIXELS * 2;
vpx_img_free(&ctx->frame_strg[i].img); ctx->frame_strg[i].ref_cnt = 0; ctx->ref_frames[i] = NULL;
if (!vpx_img_alloc(&ctx->frame_strg[i].img, IMG_FMT_I420, w, h, 16)) vpx_internal_error(&ctx->error, VPX_CODEC_MEM_ERROR, "Failed to allocate %dx%d" " framebuffer", w, h);
vpx_img_set_rect(&ctx->frame_strg[i].img, BORDER_PIXELS, BORDER_PIXELS, ctx->frame_hdr.kf.w, ctx->frame_hdr.kf.h);
}
if (ctx->frame_hdr.version) ctx->subpixel_filters = bilinear_filters; else ctx->subpixel_filters = sixtap_filters; }
/* Find a free framebuffer to predict into */ if (ctx->ref_frames[CURRENT_FRAME]) vp8_dixie_release_ref_frame(ctx->ref_frames[CURRENT_FRAME]);
ctx->ref_frames[CURRENT_FRAME] = vp8_dixie_find_free_ref_frame(ctx->frame_strg); this_frame_base = ctx->ref_frames[CURRENT_FRAME]->img.img_data; /* Calculate offsets to the other reference frames */ for (i = 0; i < NUM_REF_FRAMES; i++) { struct ref_cnt_img *ref = ctx->ref_frames[i];
ctx->ref_frame_offsets[i] = ref ? ref->img.img_data - this_frame_base : 0; }
/* TODO: No need to do this on every frame... */ }
void vp8_dixie_predict_destroy(struct vp8_decoder_ctx *ctx) { int i;
for (i = 0; i < NUM_REF_FRAMES; i++) { vpx_img_free(&ctx->frame_strg[i].img); ctx->frame_strg[i].ref_cnt = 0; ctx->ref_frames[i] = NULL; } }
void vp8_dixie_predict_process_row(struct vp8_decoder_ctx ctx, unsigned int row, unsigned int start_col, unsigned int num_cols) { struct img_index img; struct mb_info *mbi; unsigned int col; short *coeffs; / Adjust pointers based on row, start_col */ img.stride = ctx->ref_frames[CURRENT_FRAME]->img.stride[PLANE_Y]; img.uv_stride = ctx->ref_frames[CURRENT_FRAME]->img.stride[PLANE_U]; img.y = ctx->ref_frames[CURRENT_FRAME]->img.planes[PLANE_Y]; img.u = ctx->ref_frames[CURRENT_FRAME]->img.planes[PLANE_U]; img.v = ctx->ref_frames[CURRENT_FRAME]->img.planes[PLANE_V]; img.y += (img.stride * row + start_col) * 16; img.u += (img.uv_stride * row + start_col) * 8; img.v += (img.uv_stride * row + start_col) * 8; mbi = ctx->mb_info_rows[row] + start_col; coeffs = ctx->tokens[row & (ctx->token_hdr.partitions - 1)].coeffs + 25 * 16 * start_col;
/* Fix up the out-of-frame pixels */
if (start_col == 0) { fixup_left(img.y, 16, img.stride, row, mbi->base.y_mode); fixup_left(img.u, 8, img.uv_stride, row, mbi->base.uv_mode); fixup_left(img.v, 8, img.uv_stride, row, mbi->base.uv_mode);
if (row == 0) *(img.y - img.stride - 1) = 127; }
for (col = start_col; col < start_col + num_cols; col++) { if (row == 0) { fixup_above(img.y, 16, img.stride, col, mbi->base.y_mode); fixup_above(img.u, 8, img.uv_stride, col, mbi->base.uv_mode); fixup_above(img.v, 8, img.uv_stride, col, mbi->base.uv_mode); }
if (mbi->base.y_mode <= B_PRED) { predict_intra_luma(img.y, img.stride, mbi, coeffs); predict_intra_chroma(img.u, img.v, img.uv_stride, mbi, coeffs); } else { if (mbi->base.y_mode != SPLITMV) // && != BPRED fixup_dc_coeffs(mbi, coeffs);
if (mbi->base.need_mc_border) predict_inter_emulated_edge(ctx, &img, coeffs, mbi, col, row); else predict_inter(ctx, &img, coeffs, mbi); }
/* Advance to the next macroblock */ mbi++; img.y += 16; img.u += 8; img.v += 8; coeffs += 25 * 16; }
if (col == ctx->mb_cols) { /* Extend the last row by four pixels for intra-prediction.
- This will be propagated later by copy_down. */ uint32_t *extend = (uint32_t *)(img.y + 15 * img.stride); uint32_t val = 0x01010101 * img.y[-1 + 15 * img.stride]; *extend = val; } }
## 20.15. predict.h
Okay bestie, this is the little header that lines up all the predict functions so everyone else can call them 🎀✨
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #ifndef PREDICT_H #define PREDICT_H
void vp8_dixie_predict_init(struct vp8_decoder_ctx *ctx);
void vp8_dixie_predict_destroy(struct vp8_decoder_ctx *ctx);
void vp8_dixie_predict_process_row(struct vp8_decoder_ctx *ctx, unsigned int row, unsigned int start_col, unsigned int num_cols);
void vp8_dixie_release_ref_frame(struct ref_cnt_img *rcimg);
struct ref_cnt_img * vp8_dixie_ref_frame(struct ref_cnt_img *rcimg);
struct ref_cnt_img * vp8_dixie_find_free_ref_frame(struct ref_cnt_img *frames);
#endif
## 20.16. tokens.c
Next up is tokens.c, and honestly the token stuff is giving main character energy 💅🦋
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #include "vpx_codec_internal.h" #include "dixie.h" #include "tokens.h" #include <stdlib.h> #include <string.h> #include <malloc.h>
enum { EOB_CONTEXT_NODE, ZERO_CONTEXT_NODE, ONE_CONTEXT_NODE, LOW_VAL_CONTEXT_NODE, TWO_CONTEXT_NODE, THREE_CONTEXT_NODE, HIGH_LOW_CONTEXT_NODE, CAT_ONE_CONTEXT_NODE, CAT_THREEFOUR_CONTEXT_NODE, CAT_THREE_CONTEXT_NODE, CAT_FIVE_CONTEXT_NODE }; enum { ZERO_TOKEN, ONE_TOKEN, TWO_TOKEN, THREE_TOKEN, FOUR_TOKEN, DCT_VAL_CATEGORY1, DCT_VAL_CATEGORY2, DCT_VAL_CATEGORY3, DCT_VAL_CATEGORY4, DCT_VAL_CATEGORY5, DCT_VAL_CATEGORY6, DCT_EOB_TOKEN, MAX_ENTROPY_TOKENS }; struct extrabits { short min_val; short length; unsigned char probs[12]; }; static const unsigned int left_context_index[25] = { 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8 }; static const unsigned int above_context_index[25] = { 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 4, 5, 4, 5, 6, 7, 6, 7, 8 }; #define X(n) ((n) * PREV_COEFF_CONTEXTS * ENTROPY_NODES) static const unsigned int bands_x[16] = { X(0), X(1), X(2), X(3), X(6), X(4), X(5), X(6), X(6), X(6), X(6), X(6), X(6), X(6), X(6), X(7) }; #undef X static const struct extrabits extrabits[MAX_ENTROPY_TOKENS] = { { 0, -1, { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, //ZERO_TOKEN { 1, 0, { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, //ONE_TOKEN { 2, 0, { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, //TWO_TOKEN { 3, 0, { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, //THREE_TOKEN { 4, 0, { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, //FOUR_TOKEN { 5, 0, {159, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, //DCT_VAL_CATEGORY1 { 7, 1, {145, 165, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, //DCT_VAL_CATEGORY2 {11, 2, {140, 148, 173, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, //DCT_VAL_CATEGORY3
{19, 3, {135, 140, 155, 176, 0, 0, 0, 0, 0, 0, 0, 0 } }, //DCT_VAL_CATEGORY4 {35, 4, {130, 134, 141, 157, 180, 0, 0, 0, 0, 0, 0, 0 } }, //DCT_VAL_CATEGORY5 {67, 10, {129, 130, 133, 140, 153, 177, 196, 230, 243, 254, 254, 0 } }, //DCT_VAL_CATEGORY6 { 0, -1, { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, // EOB TOKEN };
static const unsigned int zigzag[16] = { 0, 1, 4, 8, 5, 2, 3, 6, 9, 12, 13, 10, 7, 11, 14, 15 };
#define DECODE_AND_APPLYSIGN(value_to_sign) \ v = (bool_get_bit(bool) ? -value_to_sign \ : value_to_sign) * dqf[!!c];
#define DECODE_AND_BRANCH_IF_ZERO(probability,branch) \ if (!bool_get(bool, probability)) goto branch;
#define DECODE_AND_LOOP_IF_ZERO(probability,branch) \ if (!bool_get(bool, probability)) \ { \ prob = type_probs; \ if (c<15) {\ ++c; \ prob += bands_x[c]; \ goto branch; \ }\ else \ goto BLOCK_FINISHED; /* for malformed input */\ }
#define DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(val) \ DECODE_AND_APPLYSIGN(val) \ prob = type_probs + (ENTROPY_NODES*2); \ if (c < 15){\ b_tokens[zigzag[c]] = v; \ ++c; \ goto DO_WHILE; }\ b_tokens[zigzag[15]] = v; \ goto BLOCK_FINISHED;
#define DECODE_EXTRABIT_AND_ADJUST_VAL(t,bits_count)\ val += bool_get(bool, extrabits[t].probs[bits_count]) << \ bits_count;
static int decode_mb_tokens(struct bool_decoder *bool, token_entropy_ctx_t left, token_entropy_ctx_t above, short *tokens, enum prediction_mode mode, coeff_probs_table_t probs, short factor[TOKEN_BLOCK_TYPES][2]) { int i, stop, type; int c, t, v; int val, bits_count; int eob_mask; short *b_tokens; // tokens for this block unsigned char *type_probs; // probabilities for this block type unsigned char *prob; short *dqf;
eob_mask = 0;
if (mode != B_PRED && mode != SPLITMV) { i = 24; stop = 24; type = 1; b_tokens = tokens + 24 * 16; dqf = factor[TOKEN_BLOCK_Y2]; } else { i = 0; stop = 16; type = 3; b_tokens = tokens; dqf = factor[TOKEN_BLOCK_Y1]; }
/* Save a pointer to the coefficient probs for the current type.
- Need to repeat this whenever type changes. */ type_probs = probs[type][0][0];
BLOCK_LOOP: t = left[left_context_index[i]] + above[above_context_index[i]]; c = !type; /* all blocks start at 0 except type 0, which starts
- at 1. */
prob = type_probs; prob += t * ENTROPY_NODES;
DO_WHILE: prob += bands_x[c]; DECODE_AND_BRANCH_IF_ZERO(prob[EOB_CONTEXT_NODE], BLOCK_FINISHED);
CHECK_0_: DECODE_AND_LOOP_IF_ZERO(prob[ZERO_CONTEXT_NODE], CHECK_0_); DECODE_AND_BRANCH_IF_ZERO(prob[ONE_CONTEXT_NODE], ONE_CONTEXT_NODE_0_); DECODE_AND_BRANCH_IF_ZERO(prob[LOW_VAL_CONTEXT_NODE], LOW_VAL_CONTEXT_NODE_0_); DECODE_AND_BRANCH_IF_ZERO(prob[HIGH_LOW_CONTEXT_NODE], HIGH_LOW_CONTEXT_NODE_0_); DECODE_AND_BRANCH_IF_ZERO(prob[CAT_THREEFOUR_CONTEXT_NODE], CAT_THREEFOUR_CONTEXT_NODE_0_); DECODE_AND_BRANCH_IF_ZERO(prob[CAT_FIVE_CONTEXT_NODE], CAT_FIVE_CONTEXT_NODE_0_); val = extrabits[DCT_VAL_CATEGORY6].min_val; bits_count = extrabits[DCT_VAL_CATEGORY6].length;
do { DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY6, bits_count); bits_count --; } while (bits_count >= 0);
DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(val);
CAT_FIVE_CONTEXT_NODE_0_: val = extrabits[DCT_VAL_CATEGORY5].min_val; DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY5, 4); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY5, 3); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY5, 2); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY5, 1); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY5, 0); DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(val);
CAT_THREEFOUR_CONTEXT_NODE_0_: DECODE_AND_BRANCH_IF_ZERO(prob[CAT_THREE_CONTEXT_NODE], CAT_THREE_CONTEXT_NODE_0_); val = extrabits[DCT_VAL_CATEGORY4].min_val; DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY4, 3); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY4, 2); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY4, 1); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY4, 0); DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(val);
CAT_THREE_CONTEXT_NODE_0_: val = extrabits[DCT_VAL_CATEGORY3].min_val; DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY3, 2); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY3, 1); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY3, 0); DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(val);
HIGH_LOW_CONTEXT_NODE_0_: DECODE_AND_BRANCH_IF_ZERO(prob[CAT_ONE_CONTEXT_NODE], CAT_ONE_CONTEXT_NODE_0_);
val = extrabits[DCT_VAL_CATEGORY2].min_val; DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY2, 1); DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY2, 0); DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(val);
CAT_ONE_CONTEXT_NODE_0_: val = extrabits[DCT_VAL_CATEGORY1].min_val; DECODE_EXTRABIT_AND_ADJUST_VAL(DCT_VAL_CATEGORY1, 0); DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(val);
LOW_VAL_CONTEXT_NODE_0_: DECODE_AND_BRANCH_IF_ZERO(prob[TWO_CONTEXT_NODE], TWO_CONTEXT_NODE_0_); DECODE_AND_BRANCH_IF_ZERO(prob[THREE_CONTEXT_NODE], THREE_CONTEXT_NODE_0_); DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(4);
THREE_CONTEXT_NODE_0_: DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(3);
TWO_CONTEXT_NODE_0_: DECODE_SIGN_WRITE_COEFF_AND_CHECK_EXIT(2);
ONE_CONTEXT_NODE_0_: DECODE_AND_APPLYSIGN(1); prob = type_probs + ENTROPY_NODES; if (c < 15) { b_tokens[zigzag[c]] = v; ++c; goto DO_WHILE; }
b_tokens[zigzag[15]] = v; BLOCK_FINISHED: eob_mask |= (c > 1) << i; t = (c != !type); // any non-zero data? eob_mask |= t << 31;
left[left_context_index[i]] = above[above_context_index[i]] = t; b_tokens += 16;
i++;
if (i < stop) goto BLOCK_LOOP;
if (i == 25) { type = 0; i = 0; stop = 16; type_probs = probs[type][0][0]; b_tokens = tokens; dqf = factor[TOKEN_BLOCK_Y1]; goto BLOCK_LOOP; }
if (i == 16) { type = 2; type_probs = probs[type][0][0]; stop = 24; dqf = factor[TOKEN_BLOCK_UV]; goto BLOCK_LOOP; }
return eob_mask; } static void reset_row_context(token_entropy_ctx_t *left) { memset(left, 0, sizeof(*left)); }
static void reset_above_context(token_entropy_ctx_t *above, unsigned int cols) { memset(above, 0, cols * sizeof(*above)); }
static void reset_mb_context(token_entropy_ctx_t left, token_entropy_ctx_t *above, enum prediction_mode mode) { / Reset the macroblock context on the left and right. We have
- to preserve the context of the second order block if this mode
- would not have updated it. */ memset(left, 0, sizeof((*left)[0]) * 8); memset(above, 0, sizeof((*above)[0]) * 8);
if (mode != B_PRED && mode != SPLITMV) { (*left)[8] = 0; (*above)[8] = 0; } }
void vp8_dixie_tokens_process_row(struct vp8_decoder_ctx *ctx, unsigned int partition, unsigned int row, unsigned int start_col, unsigned int num_cols) { struct token_decoder *tokens = &ctx->tokens[partition]; short coeffs = tokens->coeffs + 25 * 16 * start_col; unsigned int col; token_entropy_ctx_t *above = ctx->above_token_entropy_ctx
- start_col; token_entropy_ctx_t *left = &tokens->left_token_entropy_ctx; struct mb_info *mbi = ctx->mb_info_rows[row] + start_col; if (row == 0) reset_above_context(above, num_cols);
if (start_col == 0) reset_row_context(left);
for (col = start_col; col < start_col + num_cols; col++) { memset(coeffs, 0, 25 * 16 * sizeof(short));
if (mbi->base.skip_coeff) { reset_mb_context(left, above, mbi->base.y_mode); mbi->base.eob_mask = 0; } else { struct dequant_factors *dqf;
dqf = ctx->dequant_factors + mbi->base.segment_id; mbi->base.eob_mask = decode_mb_tokens(&tokens->bool, *left, *above, coeffs, mbi->base.y_mode, ctx->entropy_hdr.coeff_probs, dqf->factor); }
above++; mbi++; coeffs += 25 * 16; } }
void vp8_dixie_tokens_init(struct vp8_decoder_ctx *ctx) { unsigned int partitions = ctx->token_hdr.partitions;
if (ctx->frame_hdr.frame_size_updated) { unsigned int i; unsigned int coeff_row_sz = ctx->mb_cols * 25 * 16 * sizeof(short);
for (i = 0; i < partitions; i++) { free(ctx->tokens[i].coeffs); ctx->tokens[i].coeffs = memalign(16, coeff_row_sz);
if (!ctx->tokens[i].coeffs) vpx_internal_error(&ctx->error, VPX_CODEC_MEM_ERROR, NULL); }
free(ctx->above_token_entropy_ctx); ctx->above_token_entropy_ctx = calloc(ctx->mb_cols, sizeof(*ctx->above_token_entropy_ctx));
if (!ctx->above_token_entropy_ctx) vpx_internal_error(&ctx->error, VPX_CODEC_MEM_ERROR, NULL); } }
void vp8_dixie_tokens_destroy(struct vp8_decoder_ctx *ctx) { int i;
for (i = 0; i < MAX_PARTITIONS; i++) free(ctx->tokens[i].coeffs);
free(ctx->above_token_entropy_ctx); }
---
## 20.17. tokens.h ✨
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */ #ifndef TOKENS_H #define TOKENS_H
void vp8_dixie_tokens_init(struct vp8_decoder_ctx *ctx);
void vp8_dixie_tokens_destroy(struct vp8_decoder_ctx *ctx);
void vp8_dixie_tokens_process_row(struct vp8_decoder_ctx *ctx, unsigned int partition, unsigned int row, unsigned int start_col, unsigned int num_cols);
#endif
---
## 20.18. vp8_prob_data.h 🦋
> next up: the probability data header 🌸
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
<pink>okay so we're still deep in the vp8_prob_data.h goodies 🎀</pink> the coefficient entropy update table is literally a giant grid of probabilities, and each one is just a little number that tells the decoder how likely a tweak is. <lavender>no cap, it's giving spreadsheet energy 💅</lavender>
static const unsigned char k_coeff_entropy_update_probs[BLOCK_TYPES][COEFF_BANDS] [PREV_COEFF_CONTEXTS] [ENTROPY_NODES] = { { { {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {176, 246, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {223, 241, 252, 255, 255, 255, 255, 255, 255, 255, 255}, {249, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 244, 252, 255, 255, 255, 255, 255, 255, 255, 255}, {234, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {253, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 246, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {239, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {254, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 248, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {251, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {251, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {254, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 254, 253, 255, 254, 255, 255, 255, 255, 255, 255}, {250, 255, 254, 255, 254, 255, 255, 255, 255, 255, 255}, {254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, }, { { {217, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {225, 252, 241, 253, 255, 255, 254, 255, 255, 255, 255}, {234, 250, 241, 250, 253, 255, 253, 254, 255, 255, 255}, }, { {255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {223, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {238, 253, 254, 254, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 248, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {249, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 253, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {247, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {252, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {253, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 254, 253, 255, 255, 255, 255, 255, 255, 255, 255}, {250, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, }, { { {186, 251, 250, 255, 255, 255, 255, 255, 255, 255, 255}, {234, 251, 244, 254, 255, 255, 255, 255, 255, 255, 255}, {251, 251, 243, 253, 254, 255, 254, 255, 255, 255, 255}, }, { {255, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {236, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {251, 253, 253, 254, 254, 255, 255, 255, 255, 255, 255}, }, { {255, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {254, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, }, { { {248, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {250, 254, 252, 254, 255, 255, 255, 255, 255, 255, 255}, {248, 254, 249, 253, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255}, {246, 253, 253, 255, 255, 255, 255, 255, 255, 255, 255}, {252, 254, 251, 254, 254, 255, 255, 255, 255, 255, 255}, }, { {255, 254, 252, 255, 255, 255, 255, 255, 255, 255, 255}, {248, 254, 253, 255, 255, 255, 255, 255, 255, 255, 255}, {253, 255, 254, 254, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 251, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {245, 251, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {253, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 251, 253, 255, 255, 255, 255, 255, 255, 255, 255}, {252, 253, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 252, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {249, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 254, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 255, 253, 255, 255, 255, 255, 255, 255, 255, 255}, {250, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, { {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, {255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}, }, }, };
static const unsigned char k_default_y_mode_probs [] = { 112, 86, 140, 37};
static const unsigned char k_default_uv_mode_probs [] = { 162, 101, 204};
<mint>those two little default mode arrays at the end are the starting values for the luma and chroma mode probs 🧚</mint> <hotpink>the whole table is a total main character moment, periodt 💖</hotpink>
static const unsigned char k_default_coeff_probs [BLOCK_TYPES][COEFF_BANDS] [PREV_COEFF_CONTEXTS][ENTROPY_NODES] = { { /* block type 0 / { / coeff band 0 / { 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128}, { 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128}, { 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128} }, { / coeff band 1 / { 253, 136, 254, 255, 228, 219, 128, 128, 128, 128, 128}, { 189, 129, 242, 255, 227, 213, 255, 219, 128, 128, 128}, { 106, 126, 227, 252, 214, 209, 255, 255, 128, 128, 128} }, { / coeff band 2 / { 1, 98, 248, 255, 236, 226, 255, 255, 128, 128, 128}, { 181, 133, 238, 254, 221, 234, 255, 154, 128, 128, 128}, { 78, 134, 202, 247, 198, 180, 255, 219, 128, 128, 128} }, { / coeff band 3 / { 1, 185, 249, 255, 243, 255, 128, 128, 128, 128, 128}, { 184, 150, 247, 255, 236, 224, 128, 128, 128, 128, 128}, { 77, 110, 216, 255, 236, 230, 128, 128, 128, 128, 128} }, { / coeff band 4 / { 1, 101, 251, 255, 241, 255, 128, 128, 128, 128, 128}, { 170, 139, 241, 252, 236, 209, 255, 255, 128, 128, 128}, { 37, 116, 196, 243, 228, 255, 255, 255, 128, 128, 128} }, { / coeff band 5 / { 1, 204, 254, 255, 245, 255, 128, 128, 128, 128, 128}, { 207, 160, 250, 255, 238, 128, 128, 128, 128, 128, 128}, { 102, 103, 231, 255, 211, 171, 128, 128, 128, 128, 128} }, { / coeff band 6 / { 1, 152, 252, 255, 240, 255, 128, 128, 128, 128, 128}, { 177, 135, 243, 255, 234, 225, 128, 128, 128, 128, 128}, { 80, 129, 211, 255, 194, 224, 128, 128, 128, 128, 128} }, { / coeff band 7 / { 1, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128}, { 246, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128}, { 255, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128} } }, { / block type 1 / { / coeff band 0 / { 198, 35, 237, 223, 193, 187, 162, 160, 145, 155, 62}, { 131, 45, 198, 221, 172, 176, 220, 157, 252, 221, 1}, { 68, 47, 146, 208, 149, 167, 221, 162, 255, 223, 128} }, { / coeff band 1 / { 1, 149, 241, 255, 221, 224, 255, 255, 128, 128, 128}, { 184, 141, 234, 253, 222, 220, 255, 199, 128, 128, 128}, { 81, 99, 181, 242, 176, 190, 249, 202, 255, 255, 128} }, { / coeff band 2 / { 1, 129, 232, 253, 214, 197, 242, 196, 255, 255, 128}, { 99, 121, 210, 250, 201, 198, 255, 202, 128, 128, 128}, { 23, 91, 163, 242, 170, 187, 247, 210, 255, 255, 128} }, { / coeff band 3 / { 1, 200, 246, 255, 234, 255, 128, 128, 128, 128, 128}, { 109, 178, 241, 255, 231, 245, 255, 255, 128, 128, 128}, { 44, 130, 201, 253, 205, 192, 255, 255, 128, 128, 128} }, { / coeff band 4 / { 1, 132, 239, 251, 219, 209, 255, 165, 128, 128, 128}, { 94, 136, 225, 251, 218, 190, 255, 255, 128, 128, 128}, { 22, 100, 174, 245, 186, 161, 255, 199, 128, 128, 128} }, { / coeff band 5 / { 1, 182, 249, 255, 232, 235, 128, 128, 128, 128, 128}, { 124, 143, 241, 255, 227, 234, 128, 128, 128, 128, 128}, { 35, 77, 181, 251, 193, 211, 255, 205, 128, 128, 128} }, { / coeff band 6 / { 1, 157, 247, 255, 236, 231, 255, 255, 128, 128, 128}, { 121, 141, 235, 255, 225, 227, 255, 255, 128, 128, 128}, { 45, 99, 188, 251, 195, 217, 255, 224, 128, 128, 128} }, { / coeff band 7 / { 1, 1, 251, 255, 213, 255, 128, 128, 128, 128, 128}, { 203, 1, 248, 255, 255, 128, 128, 128, 128, 128, 128}, { 137, 1, 177, 255, 224, 255, 128, 128, 128, 128, 128} } }, { / block type 2 / { / coeff band 0 / { 253, 9, 248, 251, 207, 208, 255, 192, 128, 128, 128}, { 175, 13, 224, 243, 193, 185, 249, 198, 255, 255, 128}, { 73, 17, 171, 221, 161, 179, 236, 167, 255, 234, 128} }, { / coeff band 1 / { 1, 95, 247, 253, 212, 183, 255, 255, 128, 128, 128}, { 239, 90, 244, 250, 211, 209, 255, 255, 128, 128, 128}, { 155, 77, 195, 248, 188, 195, 255, 255, 128, 128, 128} }, { / coeff band 2 / { 1, 24, 239, 251, 218, 219, 255, 205, 128, 128, 128}, { 201, 51, 219, 255, 196, 186, 128, 128, 128, 128, 128}, { 69, 46, 190, 239, 201, 218, 255, 228, 128, 128, 128} }, { / coeff band 3 / { 1, 191, 251, 255, 255, 128, 128, 128, 128, 128, 128}, { 223, 165, 249, 255, 213, 255, 128, 128, 128, 128, 128}, { 141, 124, 248, 255, 255, 128, 128, 128, 128, 128, 128} }, { / coeff band 4 / { 1, 16, 248, 255, 255, 128, 128, 128, 128, 128, 128}, { 190, 36, 230, 255, 236, 255, 128, 128, 128, 128, 128}, { 149, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128} }, { / coeff band 5 / { 1, 226, 255, 128, 128, 128, 128, 128, 128, 128, 128}, { 247, 192, 255, 128, 128, 128, 128, 128, 128, 128, 128}, { 240, 128, 255, 128, 128, 128, 128, 128, 128, 128, 128} }, { / coeff band 6 / { 1, 134, 252, 255, 255, 128, 128, 128, 128, 128, 128}, { 213, 62, 250, 255, 255, 128, 128, 128, 128, 128, 128}, { 55, 93, 255, 128, 128, 128, 128, 128, 128, 128, 128} }, { / coeff band 7 / { 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128}, { 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128}, { 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128} } }, { / block type 3 / { / coeff band 0 / { 202, 24, 213, 235, 186, 191, 220, 160, 240, 175, 255}, { 126, 38, 182, 232, 169, 184, 228, 174, 255, 187, 128}, { 61, 46, 138, 219, 151, 178, 240, 170, 255, 216, 128} }, { / coeff band 1 / { 1, 112, 230, 250, 199, 191, 247, 159, 255, 255, 128}, { 166, 109, 228, 252, 211, 215, 255, 174, 128, 128, 128}, { 39, 77, 162, 232, 172, 180, 245, 178, 255, 255, 128} }, { / coeff band 2 / { 1, 52, 220, 246, 198, 199, 249, 220, 255, 255, 128}, { 124, 74, 191, 243, 183, 193, 250, 221, 255, 255, 128}, { 24, 71, 130, 219, 154, 170, 243, 182, 255, 255, 128} }, { / coeff band 3 / { 1, 182, 225, 249, 219, 240, 255, 224, 128, 128, 128}, { 149, 150, 226, 252, 216, 205, 255, 171, 128, 128, 128}, { 28, 108, 170, 242, 183, 194, 254, 223, 255, 255, 128} }, { / coeff band 4 / { 1, 81, 230, 252, 204, 203, 255, 192, 128, 128, 128}, { 123, 102, 209, 247, 188, 196, 255, 233, 128, 128, 128}, { 20, 95, 153, 243, 164, 173, 255, 203, 128, 128, 128} }, { / coeff band 5 / { 1, 222, 248, 255, 216, 213, 128, 128, 128, 128, 128}, { 168, 175, 246, 252, 235, 205, 255, 255, 128, 128, 128}, { 47, 116, 215, 255, 211, 212, 255, 255, 128, 128, 128} }, { / coeff band 6 / { 1, 121, 236, 253, 212, 214, 255, 255, 128, 128, 128}, { 141, 84, 213, 252, 201, 202, 255, 219, 128, 128, 128}, { 42, 80, 160, 240, 162, 185, 255, 205, 128, 128, 128} }, { / coeff band 7 */ { 1, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128}, { 244, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128}, { 238, 1, 255, 128, 128, 128, 128, 128, 128, 128, 128} } } };
static const unsigned char k_mv_entropy_update_probs[2][MV_PROB_CNT] = { { 237, 246, 253, 253, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 250, 250, 252, 254, 254 }, { 231, 243, 245, 253, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 251, 251, 254, 254, 254 } };
static const unsigned char k_default_mv_probs[2][MV_PROB_CNT] = { { // row 162, // is short 128, // sign 225, 146, 172, 147, 214, 39, 156, // short tree 128, 129, 132, 75, 145, 178, 206, 239, 254, 254 // long bits }, { 164, 128, 204, 170, 119, 235, 140, 230, 228, 128, 130, 130, 74, 148, 180, 203, 236, 254, 254
} };
---
## 20.19. vpx_codec_internal.h ✨
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
/*!\file vpx_codec_internal.h
- \brief Describes the decoder algorithm interface for algorithm
- implementations. *
- This file defines the private structures and data types that are
- only relevant to implementing an algorithm, as opposed to using
- it. *
- To create a decoder algorithm class, an interface structure is put
- into the global namespace:
- <pre>
- my_codec.c:
- vpx_codec_iface_t my_codec = {
- "My Codec v1.0",
- VPX_CODEC_ALG_ABI_VERSION,
- ...
- };
- </pre> *
- An application instantiates a specific decoder instance by using
- vpx_codec_init() and a pointer to the algorithm's interface
- structure:
- <pre>
- my_app.c:
- extern vpx_codec_iface_t my_codec;
- {
- vpx_codec_ctx_t algo;
- res = vpx_codec_init(&algo, &my_codec);
- }
- </pre> *
- Once initialized, the instance is managed using other functions
- from the vpx_codec_* family. */ #ifndef VPX_CODEC_INTERNAL_H #define VPX_CODEC_INTERNAL_H #include "vpx_decoder.h" #include <stdarg.h>
/*!\brief Current ABI version number *
- \internal
- If this file is altered in any way that changes the Application
- Binary Interface (ABI), this value must be bumped. Examples
- include, but are not limited to, changing types, removing or
- reassigning enums, adding/removing/rearranging fields to
- structures. */ #define VPX_CODEC_INTERNAL_ABI_VERSION (3)
<lavender>okay bestie, we're still in the vpx_codec_internal.h vibes 💅</lavender> this is where the decoder and encoder function pointers get their little typedef outfits 🎀 every single one has to match its slot exactly or nothing talks to anything, so pay attention 👀
typedef struct vpx_codec_alg_priv vpx_codec_alg_priv_t;
/*!\brief init function pointer prototype *
- Performs algorithm-specific initialization of the decoder context.
- This function is called by the generic vpx_codec_init() wrapper
- function, so plugins implementing this interface may trust the
- input parameters to be properly initialized. *
- \param[in] ctx Pointer to this instance's context
- \retval #VPX_CODEC_OK
- The input stream was recognized and decoder initialized.
- \retval #VPX_CODEC_MEM_ERROR
- Memory operation failed. / typedef vpx_codec_err_t (vpx_codec_init_fn_t)(vpx_codec_ctx_t ctx); /!\brief destroy function pointer prototype *
- Performs algorithm-specific destruction of the decoder context.
- This function is called by the generic vpx_codec_destroy() wrapper
- function, so plugins implementing this interface may trust the
- input parameters to be properly initialized. *
- \param[in] ctx Pointer to this instance's context
- \retval #VPX_CODEC_OK
- The input stream was recognized and decoder initialized.
- \retval #VPX_CODEC_MEM_ERROR
- Memory operation failed. */ typedef vpx_codec_err_t (*vpx_codec_destroy_fn_t)( vpx_codec_alg_priv_t *ctx);
/*!\brief parse stream info function pointer prototype *
- Performs high level parsing of the bitstream. This function is
- called by the generic vpx_codec_parse_stream() wrapper function,
- so plugins implementing this interface may trust the input
- parameters to be properly initialized. *
- \param[in] data Pointer to a block of data to parse
- \param[in] data_sz Size of the data buffer
- \param[in,out] si Pointer to stream info to update. The
- size member \ref MUST be properly
- initialized, but \ref MAY be clobbered by
- the algorithm. This parameter \ref MAY
- be NULL. *
- \retval #VPX_CODEC_OK
- Bitstream is parsable and stream information updated */ typedef vpx_codec_err_t (*vpx_codec_peek_si_fn_t)( const uint8_t *data, unsigned int data_sz, vpx_codec_stream_info_t *si);
/*!\brief Return information about the current stream. *
- Returns information about the stream that has been parsed during
- decoding. *
- \param[in] ctx Pointer to this instance's context
- \param[in,out] si Pointer to stream info to update. The
- size member \ref MUST be properly
- initialized, but \ref MAY be clobbered by
- the algorithm. This parameter \ref MAY
- be NULL. *
- \retval #VPX_CODEC_OK
- Bitstream is parsable and stream information updated */ typedef vpx_codec_err_t (*vpx_codec_get_si_fn_t)( vpx_codec_alg_priv_t *ctx, vpx_codec_stream_info_t *si);
/*!\brief control function pointer prototype *
- This function is used to exchange algorithm-specific data with the
- decoder instance. This can be used to implement features specific
- to a particular algorithm. *
- This function is called by the generic vpx_codec_control() wrapper
- function, so plugins implementing this interface may trust the
- input parameters to be properly initialized. However, this
- interface does not provide type safety for the exchanged data or
- assign meanings to the control codes. Those details should be
- specified in the algorithm's header file. In particular, the
- ctrl_id parameter is guaranteed to exist in the algorithm's
- control mapping table, and the data parameter may be NULL. * *
- \param[in] ctx Pointer to this instance's context
- \param[in] ctrl_id Algorithm-specific control identifier
- \param[in,out] data Data to exchange with algorithm instance. *
- \retval #VPX_CODEC_OK
- The internal state data was deserialized. */ typedef vpx_codec_err_t (*vpx_codec_control_fn_t)( vpx_codec_alg_priv_t *ctx, int ctrl_id, va_list ap);
/*!\brief control function pointer mapping *
- This structure stores the mapping between control identifiers and
- implementing functions. Each algorithm provides a list of these
- mappings. This list is searched by the vpx_codec_control()
- wrapper function to determine which function to invoke. The
- special value {0, NULL} is used to indicate end-of-list, and must
- be present. The special value {0, <non-null>} can be used as a
- catch-all mapping. This implies that ctrl_id values chosen by the
- algorithm \ref MUST be non-zero. */ typedef const struct { int ctrl_id; vpx_codec_control_fn_t fn; } vpx_codec_ctrl_fn_map_t;
<mint>the control map is basically a little phonebook 📞 ctrl_id goes in, the right function comes out, and {0, NULL} is the "that's all folks" 🦋</mint>
/*!\brief decode data function pointer prototype *
- Processes a buffer of coded data. If the processing results in a
- new decoded frame becoming available, #VPX_CODEC_CB_PUT_SLICE and
- #VPX_CODEC_CB_PUT_FRAME events are generated as appropriate.
- This function is called by the generic vpx_codec_decode() wrapper
- function, so plugins implementing this interface may trust the
- input parameters to be properly initialized. *
- \param[in] ctx Pointer to this instance's context
- \param[in] data Pointer to this block of new coded data.
- If NULL, a #VPX_CODEC_CB_PUT_FRAME event is
- posted for the previously decoded frame.
- \param[in] data_sz Size of the coded data, in bytes. *
- \return Returns #VPX_CODEC_OK if the coded data was processed
- completely and future pictures can be decoded without
- error. Otherwise, see the descriptions of the other error
- codes in ::vpx_codec_err_t for recoverability
- capabilities. */ typedef vpx_codec_err_t (*vpx_codec_decode_fn_t)( vpx_codec_alg_priv_t *ctx, const uint8_t *data, unsigned int data_sz, void *user_priv, long deadline);
/*!\brief Decoded frames iterator *
- Iterates over a list of the frames available for display. The
- iterator storage should be initialized to NULL to start the
- iteration. Iteration is complete when this function returns NULL. *
- The list of available frames becomes valid upon completion of the
- vpx_codec_decode call, and remains valid until the next call to
- vpx_codec_decode. *
- \param[in] ctx Pointer to this instance's context
- \param[in out] iter Iterator storage, initialized to NULL *
- \return Returns a pointer to an image, if one is ready for
- display. Frames produced will always be in PTS
- (presentation time stamp) order. / typedef vpx_image_t(*vpx_codec_get_frame_fn_t)( vpx_codec_alg_priv_t *ctx, vpx_codec_iter_t *iter);
/*\brief External Memory Allocation memory map get iterator *
- Iterates over a list of the memory maps requested by the decoder.
- The iterator storage should be initialized to NULL to start the
- iteration. Iteration is complete when this function returns NULL. *
- \param[in out] iter Iterator storage, initialized to NULL *
- \return Returns a pointer to a memory segment descriptor, or NULL
- to indicate end-of-list. */ typedef vpx_codec_err_t (*vpx_codec_get_mmap_fn_t)( const vpx_codec_ctx_t *ctx, vpx_codec_mmap_t *mmap, vpx_codec_iter_t *iter);
/*\brief External Memory Allocation memory map set iterator *
- Sets a memory descriptor inside the decoder instance. *
- \param[in] ctx Pointer to this instance's context
- \param[in] mmap Memory map to store. *
- \retval #VPX_CODEC_OK
- The memory map was accepted and stored.
- \retval #VPX_CODEC_MEM_ERROR
- The memory map was rejected. */ typedef vpx_codec_err_t (*vpx_codec_set_mmap_fn_t)( vpx_codec_ctx_t *ctx, const vpx_codec_mmap_t *mmap);
typedef vpx_codec_err_t (vpx_codec_encode_fn_t)( vpx_codec_alg_priv_t *ctx, const vpx_image_t *img, vpx_codec_pts_t pts, unsigned long duration, vpx_enc_frame_flags_t flags, unsigned long deadline); typedef const vpx_codec_cx_pkt_t(*vpx_codec_get_cx_data_fn_t)( vpx_codec_alg_priv_t *ctx, vpx_codec_iter_t *iter);
typedef vpx_codec_err_t (*vpx_codec_enc_config_set_fn_t)( vpx_codec_alg_priv_t *ctx, const vpx_codec_enc_cfg_t *cfg); typedef vpx_fixed_buf_t * (*vpx_codec_get_global_headers_fn_t)(vpx_codec_alg_priv_t *ctx);
typedef vpx_image_t * (*vpx_codec_get_preview_frame_fn_t)(vpx_codec_alg_priv_t *ctx);
/*!\brief usage configuration mapping *
- This structure stores the mapping between usage identifiers and
- configuration structures. Each algorithm provides a list of these
- mappings. This list is searched by the
- vpx_codec_enc_config_default() wrapper function to determine which
- config to return. The special value {-1, {0}} is used to indicate
- end-of-list, and must be present. At least one mapping must be
- present, in addition to the end-of-list. * */ typedef const struct { int usage; vpx_codec_enc_cfg_t cfg; } vpx_codec_enc_cfg_map_t;
#define NOT_IMPLEMENTED 0
/*!\brief Decoder algorithm interface *
- All decoders \ref MUST expose a variable of this type. / struct vpx_codec_iface { const char *name; int abi_version; vpx_codec_caps_t caps; vpx_codec_init_fn_t init; vpx_codec_destroy_fn_t destroy; vpx_codec_ctrl_fn_map_t *ctrl_maps; vpx_codec_get_mmap_fn_t get_mmap; vpx_codec_set_mmap_fn_t set_mmap; struct { vpx_codec_peek_si_fn_t peek_si; vpx_codec_get_si_fn_t get_si; vpx_codec_decode_fn_t decode; vpx_codec_get_frame_fn_t get_frame; } dec; struct { vpx_codec_enc_cfg_map_t *cfg_maps; vpx_codec_encode_fn_t encode; vpx_codec_get_cx_data_fn_t get_cx_data; vpx_codec_enc_config_set_fn_t cfg_set; vpx_codec_get_global_headers_fn_t get_glob_hdrs; vpx_codec_get_preview_frame_fn_t get_preview; } enc; }; /!\brief Callback function pointer / user data pair storage */ typedef struct vpx_codec_priv_cb_pair { union { vpx_codec_put_frame_cb_fn_t put_frame; vpx_codec_put_slice_cb_fn_t put_slice; }; void *user_priv; } vpx_codec_priv_cb_pair_t;
<hotpink>and that's the whole vpx_codec_internal.h wardrobe, bestie 🌈✨ the iface struct is literally the main character 👑 every decoder has to wear it, and the callback pair is the little tag-along that carries the user_priv along for the ride 🦋💖</hotpink>
/*!\brief Instance private storage *
- This structure is allocated by the algorithm's init function. It
- can be extended in one of two ways. First, a second, algorithm
- specific structure can be allocated and the priv member pointed to
- it. Alternatively, this structure can be made the first member of
- the algorithm-specific structure, and the pointer casted to the
- proper type. */ struct vpx_codec_priv { unsigned int sz; vpx_codec_iface_t *iface; struct vpx_codec_alg_priv *alg_priv; const char *err_detail; vpx_codec_flags_t init_flags; struct { vpx_codec_priv_cb_pair_t put_frame_cb; vpx_codec_priv_cb_pair_t put_slice_cb; } dec; struct { struct vpx_fixed_buf cx_data_dst_buf; unsigned int cx_data_pad_before; unsigned int cx_data_pad_after; vpx_codec_cx_pkt_t cx_data_pkt; } enc; };
#undef VPX_CTRL_USE_TYPE #define VPX_CTRL_USE_TYPE(id, typ) \ static typ id##__value(va_list args) \ {return va_arg(args, typ);} \ static typ id##__convert(void *x)\ {\
union\ {\ void *x;\ typ d;\ } u;\ u.x = x;\ return u.d;\ }
#undef VPX_CTRL_USE_TYPE_DEPRECATED #define VPX_CTRL_USE_TYPE_DEPRECATED(id, typ) \ static typ id##__value(va_list args) \ {return va_arg(args, typ);} \ static typ id##__convert(void *x)\ {\ union\ {\ void *x;\ typ d;\ } u;\ u.x = x;\ return u.d;\ }
#define CAST(id, arg) id##__value(arg) #define RECAST(id, x) id##__convert(x)
/* Internal Utility Functions *
- The following functions are intended to be used inside algorithms
- as utilities for manipulating vpx_codec_* data structures. */ struct vpx_codec_pkt_list { unsigned int cnt; unsigned int max; struct vpx_codec_cx_pkt pkts[1]; };
#define vpx_codec_pkt_list_decl(n)\ union {struct vpx_codec_pkt_list head;\ struct {struct vpx_codec_pkt_list head;\ struct vpx_codec_cx_pkt pkts[n];} alloc;}
#define vpx_codec_pkt_list_init(m)\ (m)->alloc.head.cnt = 0,\ (m)->alloc.head.max = \ sizeof((m)->alloc.pkts) / sizeof((m)->alloc.pkts[0])
int vpx_codec_pkt_list_add(struct vpx_codec_pkt_list *, const struct vpx_codec_cx_pkt *);
const vpx_codec_cx_pkt_t* vpx_codec_pkt_list_get(struct vpx_codec_pkt_list *list, vpx_codec_iter_t *iter);
#include <stdio.h> #include <setjmp.h> struct vpx_internal_error_info { vpx_codec_err_t error_code; int has_detail; char detail[80]; int setjmp; jmp_buf jmp; };
static void vpx_internal_error(struct vpx_internal_error_info *info, vpx_codec_err_t error, const char *fmt, ...) { va_list ap;
info->error_code = error; info->has_detail = 0;
if (fmt) { size_t sz = sizeof(info->detail);
info->has_detail = 1; va_start(ap, fmt); vsnprintf(info->detail, sz - 1, fmt, ap); va_end(ap); info->detail[sz-1] = '\0'; } if (info->setjmp) longjmp(info->jmp, info->error_code); } #endif
`---- End code block ----------------------------------------`
## 20.20. vpx_decoder.h 🎀
Okay, bestie, this next file is the decoder's public face ✨ It's the bit that applications actually touch, so it's worth reading every line 💅
`---- Begin code block --------------------------------------`
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
/*!\defgroup decoder Decoder Algorithm Interface
- \ingroup codec
- This abstraction allows applications using this decoder to easily
- support multiple video formats with minimal code duplication.
- This section describes the interface common to all decoders.
- @{ */
/*!\file vpx_decoder.h
- \brief Describes the decoder algorithm interface to applications. *
- This file describes the interface between an application and a
- video decoder algorithm. * */ #ifdef __cplusplus extern "C" { #endif #ifndef VPX_DECODER_H #define VPX_DECODER_H #include "vpx_codec.h"
/*!\brief Current ABI version number *
- \internal
- If this file is altered in any way that changes the ABI, this
- value must be bumped. Examples include, but are not limited
- to, changing types, removing or reassigning enums,
- adding/removing/rearranging fields to structures */ #define VPX_DECODER_ABI_VERSION (2 + VPX_CODEC_ABI_VERSION)
/*! \brief Decoder capabilities bitfield *
- Each decoder advertises the capabilities it supports as part
- of its ::vpx_codec_iface_t interface structure. Capabilities
- are extra interfaces or functionality, and are not required
- to be supported by a decoder. *
- The available flags are specified by VPX_CODEC_CAP_* defines. */ #define VPX_CODEC_CAP_PUT_SLICE 0x10000 /< Will issue put_slice callbacks */ #define VPX_CODEC_CAP_PUT_FRAME 0x20000 /< Will issue put_frame callbacks */ #define VPX_CODEC_CAP_POSTPROC 0x40000 /**< Can postprocess decoded frame */
/*! \brief Initialization-time Feature Enabling *
- Certain codec features must be known at initialization time,
- to allow for proper memory allocation. *
- The available flags are specified by VPX_CODEC_USE_* defines. */ #define VPX_CODEC_USE_POSTPROC 0x10000 /**< Postprocess decoded frame */
/*!\brief Stream properties *
- This structure is used to query or set properties of the
- decoded stream. Algorithms may extend this structure with
- data specific to their bitstream by setting the sz member
- appropriately. */
typedef struct vpx_codec_stream_info { unsigned int sz; /< Size of this structure */ unsigned int w; /< Width (or 0 for unknown/default) */ unsigned int h; /< Height (or 0 for unknown/default) */ unsigned int is_kf; /< Current frame is a keyframe */ } vpx_codec_stream_info_t;
/* REQUIRED FUNCTIONS *
- The following functions are required to be implemented for all
- decoders. They represent the base case functionality expected
- of all decoders. */
/*!\brief Initialization Configurations *
- This structure is used to pass init time configuration options
- to the decoder. */ typedef struct vpx_codec_dec_cfg { unsigned int threads; /< Maximum number of threads to use, default 1 */ unsigned int w; /< Width */ unsigned int h; /< Height */ } vpx_codec_dec_cfg_t; /< alias for struct vpx_codec_dec_cfg */
/*!\brief Initialize a decoder instance *
- Initializes a decoder context using the given interface.
- Applications should call the vpx_codec_dec_init convenience
- macro instead of this function directly, to ensure that the
- ABI version number parameter is properly initialized. *
- In XMA mode (activated by setting VPX_CODEC_USE_XMA in the
- flags parameter), the storage pointed to by the cfg parameter
- must be kept readable and stable until all memory maps have
- been set. *
- \param[in] ctx Pointer to this instance's context.
- \param[in] iface Pointer to the algorithm interface to
- use.
- \param[in] cfg Configuration to use, if known. May be
- NULL.
- \param[in] flags Bitfield of VPX_CODEC_USE_* flags
- \param[in] ver ABI version number. Must be set to
- VPX_DECODER_ABI_VERSION
- \retval #VPX_CODEC_OK
- The decoder algorithm initialized.
- \retval #VPX_CODEC_MEM_ERROR
- Memory allocation failed. */ vpx_codec_err_t vpx_codec_dec_init_ver( vpx_codec_ctx_t *ctx, vpx_codec_iface_t *iface, vpx_codec_dec_cfg_t *cfg, vpx_codec_flags_t flags, int ver);
/*!\brief Convenience macro for vpx_codec_dec_init_ver() *
- Ensures the ABI version parameter is properly set. */ #define vpx_codec_dec_init(ctx, iface, cfg, flags) \ vpx_codec_dec_init_ver(ctx, iface, cfg, flags, \ VPX_DECODER_ABI_VERSION)
/*!\brief Parse stream info from a buffer *
- Performs high level parsing of the bitstream. Construction of
- a decoder context is not necessary. Can be used to determine
- if the bitstream is of the proper format, and to extract
- information from the stream. *
- \param[in] iface Pointer to the algorithm interface
- \param[in] data Pointer to a block of data to parse
- \param[in] data_sz Size of the data buffer
- \param[in,out] si Pointer to stream info to update. The
- size member
- \ref MUST be properly initialized, but
- \ref MAY be clobbered by the
- algorithm. This parameter \ref MAY be
- NULL. *
- \retval #VPX_CODEC_OK
- Bitstream is parsable and stream information updated */
vpx_codec_err_t vpx_codec_peek_stream_info( vpx_codec_iface_t *iface, const uint8_t *data, unsigned int data_sz, vpx_codec_stream_info_t *si);
/*!\brief Return information about the current stream. *
- Returns information about the stream that has been parsed
- during decoding. *
- \param[in] ctx Pointer to this instance's context
- \param[in,out] si Pointer to stream info to update. The
- size member \ref MUST be properly
- initialized, but \ref MAY be clobbered
- by the algorithm. This parameter \ref
- MAY be NULL. *
- \retval #VPX_CODEC_OK
- Bitstream is parsable and stream information updated */ vpx_codec_err_t vpx_codec_get_stream_info( vpx_codec_ctx_t *ctx, vpx_codec_stream_info_t *si);
The decode and iterator entry points are next, along with the frame and slice callback registration. There's a lot of stuff going on, bestie 💅
/*!\brief Decode data *
- Processes a buffer of coded data. If the processing results
- in a new decoded frame becoming available, PUT_SLICE and
- PUT_FRAME events may be generated, as appropriate. Encoded
- data \ref MUST be passed in DTS (decode time stamp) order.
- Frames produced will always be in PTS (presentation time
- stamp) order. *
- \param[in] ctx Pointer to this instance's context
- \param[in] data Pointer to this block of new coded
- data. If NULL, a
- VPX_CODEC_CB_PUT_FRAME event is posted
- for the previously decoded frame.
- \param[in] data_sz Size of the coded data, in bytes.
- \param[in] user_priv Application-specific data to associate
- with this frame.
- \param[in] deadline Soft deadline the decoder should
- attempt to meet, in us. Set to zero
- for unlimited. *
- \return Returns #VPX_CODEC_OK if the coded data was processed
- completely and future pictures can be decoded without
- error. Otherwise, see the descriptions of the other
- error codes in ::vpx_codec_err_t for recoverability
- capabilities. */ vpx_codec_err_t vpx_codec_decode(vpx_codec_ctx_t *ctx, const uint8_t *data, unsigned int data_sz, void *user_priv, long deadline);
/*!\brief Decoded frames iterator *
- Iterates over a list of the frames available for display. The
- iterator storage should be initialized to NULL to start the
- iteration. Iteration is complete when this function returns
- NULL. *
- The list of available frames becomes valid upon completion of
- the vpx_codec_decode call, and remains valid until the next
- call to vpx_codec_decode. *
- \param[in] ctx Pointer to this instance's context
- \param[in,out] iter Iterator storage, initialized to NULL *
- \return Returns a pointer to an image, if one is ready for
- display. Frames produced will always be in PTS
- (presentation time stamp) order. */ vpx_image_t *vpx_codec_get_frame(vpx_codec_ctx_t *ctx, vpx_codec_iter_t *iter);
/*!\defgroup cap_put_frame Frame-Based Decoding Functions *
- The following functions are required to be implemented for all
- decoders that advertise the VPX_CODEC_CAP_PUT_FRAME
- capability. Calling these functions for codecs that don't
- advertise this capability will result in an error code being
- returned, usually VPX_CODEC_ERROR
- @{ */
/*!\brief put frame callback prototype *
- This callback is invoked by the decoder to notify the
- application of the availability of decoded image data. */ typedef void (*vpx_codec_put_frame_cb_fn_t)( void *user_priv, const vpx_image_t *img);
/*!\brief Register for notification of frame completion. *
- Registers a given function to be called when a decoded frame
- is available. *
- \param[in] ctx Pointer to this instance's context
- \param[in] cb Pointer to the callback function
- \param[in] user_priv User's private data *
- \retval #VPX_CODEC_OK
- Callback successfully registered.
- \retval #VPX_CODEC_ERROR
- Decoder context not initialized, or algorithm not capable
- of posting slice completion. */ vpx_codec_err_t vpx_codec_register_put_frame_cb( vpx_codec_ctx_t *ctx, vpx_codec_put_frame_cb_fn_t cb, void *user_priv);
/*!@} - end defgroup cap_put_frame */
/*!\defgroup cap_put_slice Slice-Based Decoding Functions *
- The following functions are required to be implemented for all
- decoders that advertise the VPX_CODEC_CAP_PUT_SLICE
- capability. Calling these functions for codecs that don't
- advertise this capability will result in an error code being
- returned, usually VPX_CODEC_ERROR
- @{ */
/*!\brief put slice callback prototype *
- This callback is invoked by the decoder to notify the
- application of the availability of partially decoded image
- data. */
typedef void (*vpx_codec_put_slice_cb_fn_t)( void *user_priv, const vpx_image_t *img, const vpx_image_rect_t *valid, const vpx_image_rect_t *update);
/*!\brief Register for notification of slice completion. *
- Registers a given function to be called when a decoded slice
- is available. *
- \param[in] ctx Pointer to this instance's context
- \param[in] cb Pointer to the callback function
- \param[in] user_priv User's private data *
- \retval #VPX_CODEC_OK
- Callback successfully registered.
- \retval #VPX_CODEC_ERROR
- Decoder context not initialized, or algorithm not capable
- of posting slice completion. */ vpx_codec_err_t vpx_codec_register_put_slice_cb( vpx_codec_ctx_t *ctx, vpx_codec_put_slice_cb_fn_t cb, void *user_priv);
/!@} - end defgroup cap_put_slice/
/!@} - end defgroup decoder/
#endif
#ifdef __cplusplus } #endif
#if !defined(VPX_CODEC_DISABLE_COMPAT) || !VPX_CODEC_DISABLE_COMPAT #include "vpx_decoder_compat.h" #endif
## 20.21. vpx_decoder_compat.h
Okay, now the compat layer 🦋 It's deprecated, so the docs are kinda thin on purpose, which is giving "only touch this if you already have it" energy 💀
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
/*!\defgroup decoder Common Decoder Algorithm Interface
- This abstraction allows applications using this decoder to easily
- support multiple video formats with minimal code duplication.
- This section describes the interface common to all codecs.
- @{ */
/*!\file
- \brief Provides a compatibility layer between version 1 and 2 of
- this API. *
- This interface has been deprecated. Only existing code should
- make use of this interface, and therefore, it is only thinly
- documented. Existing code should be ported to the vpx_codec_*
- API. */ #ifdef __cplusplus extern "C" { #endif
#ifndef VPX_DECODER_COMPAT_H #define VPX_DECODER_COMPAT_H
/!\brief Decoder algorithm return codes */ typedef enum { /!\brief Operation completed without error */ VPX_DEC_OK = VPX_CODEC_OK,
/*!\brief Unspecified error */ VPX_DEC_ERROR = VPX_CODEC_ERROR,
/*!\brief Memory operation failed */ VPX_DEC_MEM_ERROR = VPX_CODEC_MEM_ERROR,
/*!\brief ABI version mismatch */ VPX_DEC_ABI_MISMATCH = VPX_CODEC_ABI_MISMATCH,
/*!\brief The given bitstream is not supported. *
- The bitstream was unable to be parsed at the highest
- level. The decoder is unable to proceed. This error \ref
- SHOULD be treated as fatal to the stream. */ VPX_DEC_UNSUP_BITSTREAM = VPX_CODEC_UNSUP_BITSTREAM,
/*!\brief Encoded bitstream uses an unsupported feature *
- The decoder does not implement a feature required by the
- encoder. This return code should only be used for
- features that prevent future pictures from being properly
- decoded. This error \ref MAY be treated as fatal to the
- stream or \ref MAY be treated as fatal to the current
- Group of Pictures (GOP). */ VPX_DEC_UNSUP_FEATURE = VPX_CODEC_UNSUP_FEATURE,
/*!\brief The coded data for this stream is corrupt or
- incomplete *
- There was a problem decoding the current frame. This
- return code should only be used for failures that prevent
- future pictures from being properly decoded. This error
- \ref MAY be treated as fatal to the stream or \ref MAY be
- treated as fatal to the current GOP. If decoding is
- continued for the current GOP, artifacts may be present. / VPX_DEC_CORRUPT_FRAME = VPX_CODEC_CORRUPT_FRAME, /!\brief An application-supplied parameter is not valid. * */ VPX_DEC_INVALID_PARAM = VPX_CODEC_INVALID_PARAM,
/*!\brief An iterator reached the end of list. * */ VPX_DEC_LIST_END = VPX_CODEC_LIST_END
} vpx_dec_err_t;
/*! \brief Decoder capabilities bitfield *
- Each decoder advertises the capabilities it supports as part
- of its ::vpx_dec_iface_t interface structure. Capabilities
- are extra interfaces or functionality, and are not required
- to be supported by a decoder. *
- The available flags are specified by VPX_DEC_CAP_* defines. */ typedef int vpx_dec_caps_t; #define VPX_DEC_CAP_PUT_SLICE 0x0001 /< Will issue put_slice callbacks */ #define VPX_DEC_CAP_PUT_FRAME 0x0002 /< Will issue put_frame callbacks */
#define VPX_DEC_CAP_XMA 0x0004 /**< Supports External Memory Allocation */
/*!\brief Stream properties *
- This structure is used to query or set properties of the
- decoded stream. Algorithms may extend this structure with
- data specific to their bitstream by setting the sz member
- appropriately. */ #if 1 typedef vpx_codec_stream_info_t vpx_dec_stream_info_t; #else typedef struct { unsigned int sz; /< Size of this structure */ unsigned int w; /< Width (or 0 for unknown/default) */ unsigned int h; /< Height (or 0 for unknown/default) */ unsigned int is_kf; /< Current frame is a keyframe */ } vpx_dec_stream_info_t;
#endif
/*!\brief Decoder interface structure. *
- Contains function pointers and other data private to the
- decoder implementation. This structure is opaque to the
- application. */ typedef const struct vpx_codec_iface vpx_dec_iface_t; typedef struct vpx_codec_priv vpx_dec_priv_t;
/*!\brief Iterator *
- Opaque storage used for iterating over lists. */ typedef vpx_codec_iter_t vpx_dec_iter_t;
<pink>okay bestie, the decoder context is the main character here 💅</pink>
/*!\brief Decoder context structure *
- All decoders \ref MUST support this context structure fully.
- In general, this data should be considered private to the
- decoder algorithm, and not be manipulated or examined by the
- calling application. Applications may reference the 'name'
- member to get a printable description of the algorithm. */ #if 1 typedef vpx_codec_ctx_t vpx_dec_ctx_t; #else typedef struct { const char *name; /< Printable interface name */ vpx_dec_iface_t *iface; /< Interface pointers */ vpx_dec_err_t err; /< Last returned error */ vpx_dec_priv_t *priv; /< Algorithm private storage */ } vpx_dec_ctx_t; #endif
<lavender>ngl the build config and name helpers are giving vintage 🧚</lavender>
/*!\brief Return the build configuration *
- Returns a printable string containing an encoded version of
- the build configuration. This may be useful to vpx support. * / const char *vpx_dec_build_config(void) DEPRECATED; /!\brief Return the name for a given interface *
- Returns a human readable string for name of the given decoder
- interface. *
- \param[in] iface Interface pointer * */ const char *vpx_dec_iface_name( vpx_dec_iface_t *iface) DEPRECATED;
<cyan>okay the error helpers are so iconic, one line and no newlines, periodt 🎀</cyan>
/*!\brief Convert error number to printable string *
- Returns a human readable string for the last error returned
- by the algorithm. The returned error will be one line and
- will not contain any newline characters. * *
- \param[in] err Error number. * */ const char *vpx_dec_err_to_string(vpx_dec_err_t err) DEPRECATED;
/*!\brief Retrieve error synopsis for decoder context *
- Returns a human readable string for the last error returned by
- the algorithm. The returned error will be one line and will
- not contain any newline characters. * *
- \param[in] ctx Pointer to this instance's context. * */ const char *vpx_dec_error(vpx_dec_ctx_t *ctx) DEPRECATED;
/*!\brief Retrieve detailed error information for decoder context *
- Returns a human readable string providing detailed information
- about the last error. *
- \param[in] ctx Pointer to this instance's context. *
- \retval NULL
- No detailed information is available. */ const char *vpx_dec_error_detail(vpx_dec_ctx_t *ctx) DEPRECATED;
<mint>and now the REQUIRED FUNCTIONS 🫶 these are the base case that every decoder has to have</mint>
/* REQUIRED FUNCTIONS *
- The following functions are required to be implemented for all
- decoders. They represent the base case functionality expected
- of all decoders. */
/*!\brief Initialize a decoder instance *
- Initializes a decoder context using the given interface.
- Applications should call the vpx_dec_init convenience macro
- instead of this function directly, to ensure that the ABI
- version number parameter is properly initialized. *
- \param[in] ctx Pointer to this instance's context.
- \param[in] iface Pointer to the algorithm interface to use.
- \param[in] ver ABI version number. Must be set to
- VPX_DECODER_ABI_VERSION
- \retval #VPX_DEC_OK
- The decoder algorithm initialized.
- \retval #VPX_DEC_MEM_ERROR
- Memory allocation failed. */ vpx_dec_err_t vpx_dec_init_ver( vpx_dec_ctx_t *ctx, vpx_dec_iface_t *iface, int ver) DEPRECATED;
#define vpx_dec_init(ctx, iface) \ vpx_dec_init_ver(ctx, iface, VPX_DECODER_ABI_VERSION)
/*!\brief Destroy a decoder instance *
- Destroys a decoder context, freeing any associated memory
- buffers. *
- \param[in] ctx Pointer to this instance's context *
- \retval #VPX_DEC_OK
- The decoder algorithm initialized.
- \retval #VPX_DEC_MEM_ERROR
- Memory allocation failed. */ vpx_dec_err_t vpx_dec_destroy(vpx_dec_ctx_t *ctx) DEPRECATED;
/*!\brief Get the capabilities of an algorithm. *
- Retrieves the capabilities bitfield from the algorithm's
- interface. *
- \param[in] iface Pointer to the algorithm interface * */ vpx_dec_caps_t vpx_dec_get_caps( vpx_dec_iface_t *iface) DEPRECATED;
<yellow>the stream info functions are a whole vibe, peeking without even making a context 👀</yellow>
/*!\brief Parse stream info from a buffer *
- Performs high level parsing of the bitstream. Construction of
- a decoder context is not necessary. Can be used to determine
- if the bitstream is of the proper format, and to extract
- information from the stream. *
- \param[in] iface Pointer to the algorithm interface
- \param[in] data Pointer to a block of data to parse
- \param[in] data_sz Size of the data buffer
- \param[in,out] si Pointer to stream info to update. The
- size member \ref MUST be properly
- initialized, but \ref MAY be
- clobbered by the algorithm. This
- parameter \ref MAY be NULL. *
- \retval #VPX_DEC_OK
- Bitstream is parsable and stream information updated */ vpx_dec_err_t vpx_dec_peek_stream_info( vpx_dec_iface_t *iface, const uint8_t *data, unsigned int data_sz, vpx_dec_stream_info_t *si) DEPRECATED;
/*!\brief Return information about the current stream. *
- Returns information about the stream that has been parsed
- during decoding. *
- \param[in] ctx Pointer to this instance's context
- \param[in,out] si Pointer to stream info to update.
- The size member \ref MUST be properly
- initialized, but \ref MAY be clobbered
- by the algorithm. This parameter \ref
- MAY be NULL. *
- \retval #VPX_DEC_OK
- Bitstream is parsable and stream information updated */ vpx_dec_err_t vpx_dec_get_stream_info( vpx_dec_ctx_t *ctx, vpx_dec_stream_info_t *si) DEPRECATED;
<peach>the control function is basically the group chat of the decoder, passing messages to whichever helper handles them 💬</peach>
/*!\brief Control algorithm *
- This function is used to exchange algorithm-specific data with
- the decoder instance. This can be used to implement features
- specific to a particular algorithm. *
- This wrapper function dispatches the request to the helper
- function associated with the given ctrl_id. It tries to call
- this function transparently, but will return #VPX_DEC_ERROR if
- the request could not be dispatched. *
- \param[in] ctx Pointer to this instance's context
- \param[in] ctrl_id Algorithm-specific control
- identifier
- \param[in,out] data Data to exchange with algorithm
- instance. *
- \retval #VPX_DEC_OK
- The control request was processed.
- \retval #VPX_DEC_ERROR
- The control request was not processed.
- \retval #VPX_DEC_INVALID_PARAM
- The data was not valid. */ vpx_dec_err_t vpx_dec_control(vpx_dec_ctx_t *ctx, int ctrl_id, void *data) DEPRECATED;
<hotpink>decode is where the real magic happens ✨ DTS in, PTS out, that's the whole deal</hotpink>
/*!\brief Decode data *
- Processes a buffer of coded data. If the processing results
- in a new decoded frame becoming available,
- #VPX_DEC_CB_PUT_SLICE and #VPX_DEC_CB_PUT_FRAME events may be
- generated, as appropriate. Encoded data \ref MUST be passed
- in DTS (decode time stamp) order. Frames produced will always
- be in PTS (presentation time stamp) order. *
- \param[in] ctx Pointer to this instance's context
- \param[in] data Pointer to this block of new coded
- data. If NULL, a VPX_DEC_CB_PUT_FRAME
- event is posted for the previously
- decoded frame.
- \param[in] data_sz Size of the coded data, in bytes.
- \param[in] user_priv Application-specific data to associate
- with this frame.
- \param[in] rel_pts PTS relative to the previous frame, in
- us. If unknown or unavailable, set to
- zero. *
- \return Returns #VPX_DEC_OK if the coded data was processed
- completely and future pictures can be decoded without
- error. Otherwise, see the descriptions of the other
- error codes in ::vpx_dec_err_t for recoverability
- capabilities. */ vpx_dec_err_t vpx_dec_decode( vpx_dec_ctx_t *ctx, uint8_t *data, unsigned int data_sz, void *user_priv, int rel_pts) DEPRECATED;
<purple>the frame iterator is so satisfying, you start with NULL and keep going until it hands you NULL back 🦋</purple>
/*!\brief Decoded frames iterator *
- Iterates over a list of the frames available for display. The
- iterator storage should be initialized to NULL to start the
- iteration. Iteration is complete when this function returns
- NULL. *
- The list of available frames becomes valid upon completion of
- the vpx_dec_decode call, and remains valid until the next call
- to vpx_dec_decode. *
- \param[in] ctx Pointer to this instance's context
- \param[in out] iter Iterator storage, initialized to NULL *
- \return Returns a pointer to an image, if one is ready for
- display. Frames produced will always be in PTS
- (presentation time stamp) order. */ vpx_image_t *vpx_dec_get_frame(vpx_dec_ctx_t *ctx, vpx_dec_iter_t *iter) DEPRECATED;
<rainbow>and the frame-based decoding group, bestie, it's the whole era 🌈👑</rainbow>
/*!\defgroup cap_put_frame Frame-Based Decoding Functions *
- The following functions are required to be implemented for all
- decoders that advertise the VPX_DEC_CAP_PUT_FRAME capability.
- Calling these functions for codecs that don't advertise this
- capability will result in an error code being returned,
- usually VPX_DEC_ERROR @{ */
/*!\brief put frame callback prototype *
- This callback is invoked by the decoder to notify the
- application of the availability of decoded image data. */ typedef void (*vpx_dec_put_frame_cb_fn_t)( void *user_priv, const vpx_image_t *img);
<pink>okay bestie, we're still in the decoder header and this part is giving deprecated-but-iconic callback energy 💖</pink>
/*!\brief Register for notification of frame completion. *
- Registers a given function to be called when a decoded frame
- is available. *
- \param[in] ctx Pointer to this instance's context
- \param[in] cb Pointer to the callback function
- \param[in] user_priv User's private data *
- \retval #VPX_DEC_OK
- Callback successfully registered.
- \retval #VPX_DEC_ERROR
- Decoder context not initialized, or algorithm not capable
- of posting slice completion. */ vpx_dec_err_t vpx_dec_register_put_frame_cb( vpx_dec_ctx_t *ctx, vpx_dec_put_frame_cb_fn_t cb, void *user_priv) DEPRECATED;
/*!@} - end defgroup cap_put_frame */
/*!\defgroup cap_put_slice Slice-Based Decoding Functions *
- The following functions are required to be implemented for all
- decoders that advertise the VPX_DEC_CAP_PUT_SLICE capability.
- Calling these functions for codecs that don't advertise this
- capability will result in an error code being returned,
- usually VPX_DEC_ERROR
- @{ */
/*!\brief put slice callback prototype *
- This callback is invoked by the decoder to notify the
- application of the availability of partially decoded image
- data. */ typedef void (*vpx_dec_put_slice_cb_fn_t)(void *user_priv, const vpx_image_t *img, const vpx_image_rect_t *valid, const vpx_image_rect_t *update);
/*!\brief Register for notification of slice completion. *
- Registers a given function to be called when a decoded slice
- is available. *
- \param[in] ctx Pointer to this instance's context
- \param[in] cb Pointer to the callback function
- \param[in] user_priv User's private data *
- \retval #VPX_DEC_OK
- Callback successfully registered.
- \retval #VPX_DEC_ERROR
- Decoder context not initialized, or algorithm not capable
- of posting slice completion. */ vpx_dec_err_t vpx_dec_register_put_slice_cb(vpx_dec_ctx_t *ctx, vpx_dec_put_slice_cb_fn_t cb, void *user_priv) DEPRECATED;
/!@} - end defgroup cap_put_slice/ /*!\defgroup cap_xma External Memory Allocation Functions *
- The following functions are required to be implemented for all
- decoders that advertise the VPX_DEC_CAP_XMA capability.
- Calling these functions for codecs that don't advertise this
- capability will result in an error code being returned,
- usually VPX_DEC_ERROR
- @{ */
/*!\brief Memory Map Entry *
- This structure is used to contain the properties of a memory
- segment. It is populated by the decoder in the request phase,
- and by the calling application once the requested allocation
- has been performed. / #if 1 #define VPX_DEC_MEM_ZERO 0x1 /< Segment must be zeroed by allocation */ #define VPX_DEC_MEM_WRONLY 0x2 /< Segment need not be readable */ #define VPX_DEC_MEM_FAST 0x4 /*< Place in fast memory, if available / typedef struct vpx_codec_mmap vpx_dec_mmap_t; #else typedef struct vpx_dec_mmap { /
- The following members are set by the codec when requesting
- a segment */ unsigned int id; /< identifier for the segment's contents */ unsigned long sz; /< size of the segment, in bytes */ unsigned int align; /< required alignment of the segment, in bytes */ unsigned int flags; /< bitfield containing segment properties */ #define VPX_DEC_MEM_ZERO 0x1 /< Segment must be zeroed by allocation */ #define VPX_DEC_MEM_WRONLY 0x2 /< Segment need not be readable */ #define VPX_DEC_MEM_FAST 0x4 /**< Place in fast memory, if available */
/* The following members are to be filled in by the
- allocation function */ void *base; /< pointer to the allocated segment */ void (*dtor)(struct vpx_dec_mmap *map); /< destructor to call */ void *priv; /**< allocator private storage */ } vpx_dec_mmap_t; #endif
/*!\brief Initialize a decoder instance in external allocation
- mode *
- Initializes a decoder context using the given interface.
- Applications should call the vpx_dec_xma_init convenience
- macro instead of this function directly, to ensure that the
- ABI version number parameter is properly initialized. *
- \param[in] ctx Pointer to this instance's context.
- \param[in] iface Pointer to the algorithm interface to
- use.
- \param[in] ver ABI version number. Must be set to
- VPX_DECODER_ABI_VERSION
- \retval #VPX_DEC_OK
- The decoder algorithm initialized.
- \retval #VPX_DEC_ERROR
- Decoder does not support XMA mode. */ vpx_dec_err_t vpx_dec_xma_init_ver(vpx_dec_ctx_t *ctx, vpx_dec_iface_t *iface, int ver) DEPRECATED; #define vpx_dec_xma_init(ctx, iface) \ vpx_dec_xma_init_ver(ctx, iface, VPX_DECODER_ABI_VERSION)
/*!\brief Iterate over the list of segments to allocate. *
- Iterates over a list of the segments to allocate. The
- iterator storage should be initialized to NULL to start the
- iteration. Iteration is complete when this function returns
- VPX_DEC_LIST_END. The amount of memory needed to allocate is
- dependent upon the size of the encoded stream. This means
- that the stream info structure must be known at allocation
- time. It can be populated with the vpx_dec_peek_stream_info()
- function. In cases where the stream to be decoded is not
- available at allocation time, a fixed size must be requested.
- The decoder will not be able to decode streams larger than the
- size used at allocation time.
*
- \param[in] ctx Pointer to this instance's context.
- \param[out] mmap Pointer to the memory map entry to
- populate.
- \param[in] si Pointer to the stream info.
- \param[in out] iter Iterator storage, initialized to NULL *
- \retval #VPX_DEC_OK
- The memory map entry was populated.
- \retval #VPX_DEC_ERROR
- Decoder does not support XMA mode.
- \retval #VPX_DEC_MEM_ERROR
- Unable to determine segment size from stream info. */ vpx_dec_err_t vpx_dec_get_mem_map( vpx_dec_ctx_t *ctx, vpx_dec_mmap_t *mmap, const vpx_dec_stream_info_t *si, vpx_dec_iter_t *iter) DEPRECATED;
/*!\brief Identify allocated segments to decoder instance *
- Stores a list of allocated segments in the decoder. Segments
- \ref MUST be passed in the order they are read from
- vpx_dec_get_mem_map(), but may be passed in groups of any
- size. Segments \ref MUST be set only once. The allocation
- function \ref MUST ensure that the vpx_dec_mmap_t::base member
- is non-NULL. If the segment requires cleanup handling (e.g.,
- calling free() or close()) then the vpx_dec_mmap_t::dtor
- member \ref MUST be populated. *
- \param[in] ctx Pointer to this instance's context.
- \param[in] mmaps Pointer to the first memory map
- entry in the list.
- \param[in] num_maps Number of entries being set at this
- time *
- \retval #VPX_DEC_OK
- The segment was stored in the decoder context.
- \retval #VPX_DEC_ERROR
- Decoder does not support XMA mode.
- \retval #VPX_DEC_MEM_ERROR
- Segment base address was not set, or segment was already
- stored.
*/
vpx_dec_err_t vpx_dec_set_mem_map( vpx_dec_ctx_t *ctx, vpx_dec_mmap_t *mmaps, unsigned int num_maps) DEPRECATED;
/!@} - end defgroup cap_xma/ /!@} - end defgroup decoder/
#endif #ifdef __cplusplus } #endif
---- End code block ----------------------------------------
<hotpink>ngl the frame and slice callbacks are both marked DEPRECATED, but they still get registered with `vpx_dec_register_put_frame_cb` and `vpx_dec_register_put_slice_cb` 🫶</hotpink>
<lavender>the XMA stuff is the external memory allocation side: `vpx_dec_xma_init` is the convenience macro that wraps `vpx_dec_xma_init_ver`, and `vpx_dec_get_mem_map` plus `vpx_dec_set_mem_map` handle the segments, all marked DEPRECATED too 💜</lavender>
## 20.22. vpx_image.c 🖼️
okay bestie, this one is all about making image buffers 🫶 the code is the spec, so it stays exactly as written, no cap 💅
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
#include <stdlib.h> #include <string.h> #include "vpx/vpx_image.h"
static vpx_image_t *img_alloc_helper(vpx_image_t *img, vpx_img_fmt_t fmt, unsigned int d_w, unsigned int d_h, unsigned int stride_align, unsigned char *img_data) {
unsigned int h, w, s, xcs, ycs, bps; int align;
/* Treat align==0 like align==1 */ if (!stride_align) stride_align = 1;
/* Validate alignment (must be power of 2) */ if (stride_align & (stride_align - 1)) goto fail;
/* Get sample size for this format / switch (fmt) { case VPX_IMG_FMT_RGB32: case VPX_IMG_FMT_RGB32_LE: case VPX_IMG_FMT_ARGB: case VPX_IMG_FMT_ARGB_LE: bps = 32; break; case VPX_IMG_FMT_RGB24: case VPX_IMG_FMT_BGR24: bps = 24; break; case VPX_IMG_FMT_RGB565: case VPX_IMG_FMT_RGB565_LE: case VPX_IMG_FMT_RGB555: case VPX_IMG_FMT_RGB555_LE: case VPX_IMG_FMT_UYVY: case VPX_IMG_FMT_YUY2: case VPX_IMG_FMT_YVYU: bps = 16; break; case VPX_IMG_FMT_I420: case VPX_IMG_FMT_YV12: case VPX_IMG_FMT_VPXI420: case VPX_IMG_FMT_VPXYV12: bps = 12; break; default: bps = 16; break; } / Get chroma shift values for this format */ switch (fmt) { case VPX_IMG_FMT_I420: case VPX_IMG_FMT_YV12: case VPX_IMG_FMT_VPXI420: case VPX_IMG_FMT_VPXYV12: xcs = 1; break; default: xcs = 0; break; }
switch (fmt) { case VPX_IMG_FMT_I420: case VPX_IMG_FMT_YV12: case VPX_IMG_FMT_VPXI420: case VPX_IMG_FMT_VPXYV12: ycs = 1; break; default: ycs = 0; break; }
/* Calculate storage sizes given the chroma subsampling */ align = (1 << xcs) - 1; w = (d_w + align) & ~align; align = (1 << ycs) - 1; h = (d_h + align) & ~align; s = (fmt & VPX_IMG_FMT_PLANAR) ? w : bps * w / 8; s = (s + stride_align - 1) & ~(stride_align - 1);
/* Allocate the new image */ if (!img) { img = (vpx_image_t *)calloc(1, sizeof(vpx_image_t));
if (!img) goto fail;
img->self_allocd = 1; } else { memset(img, 0, sizeof(vpx_image_t)); }
img->img_data = img_data;
if (!img_data) { img->img_data = malloc((fmt & VPX_IMG_FMT_PLANAR) ? h * w * bps / 8 : h * s); img->img_data_owner = 1; }
if (!img->img_data) goto fail;
img->fmt = fmt; img->w = w; img->h = h; img->x_chroma_shift = xcs; img->y_chroma_shift = ycs; img->bps = bps;
/* Calculate strides */ img->stride[VPX_PLANE_Y] = img->stride[VPX_PLANE_ALPHA] = s; img->stride[VPX_PLANE_U] = img->stride[VPX_PLANE_V] = s >> xcs;
/* Default viewport to entire image */ if (!vpx_img_set_rect(img, 0, 0, d_w, d_h)) return img;
fail: vpx_img_free(img); return NULL; }
vpx_image_t *vpx_img_alloc(vpx_image_t *img, vpx_img_fmt_t fmt, unsigned int d_w, unsigned int d_h, unsigned int stride_align) { return img_alloc_helper(img, fmt, d_w, d_h, stride_align, NULL); } vpx_image_t *vpx_img_wrap(vpx_image_t *img, vpx_img_fmt_t fmt, unsigned int d_w, unsigned int d_h, unsigned int stride_align, unsigned char *img_data) { return img_alloc_helper(img, fmt, d_w, d_h, stride_align, img_data); }
int vpx_img_set_rect(vpx_image_t *img, unsigned int x, unsigned int y, unsigned int w, unsigned int h) { unsigned char *data;
if (x + w <= img->w && y + h <= img->h) { img->d_w = w; img->d_h = h;
/* Calculate plane pointers */ if (!(img->fmt & VPX_IMG_FMT_PLANAR)) { img->planes[VPX_PLANE_PACKED] = img->img_data + x * img->bps / 8 + y * img->stride[VPX_PLANE_PACKED]; } else { data = img->img_data;
if (img->fmt & VPX_IMG_FMT_HAS_ALPHA) { img->planes[VPX_PLANE_ALPHA] = data + x + y * img->stride[VPX_PLANE_ALPHA]; data += img->h * img->stride[VPX_PLANE_ALPHA]; }
img->planes[VPX_PLANE_Y] = data + x + y * img->stride[VPX_PLANE_Y]; data += img->h * img->stride[VPX_PLANE_Y]; if (!(img->fmt & VPX_IMG_FMT_UV_FLIP)) { img->planes[VPX_PLANE_U] = data
- (x >> img->x_chroma_shift)
- (y >> img->y_chroma_shift) * img->stride[VPX_PLANE_U]; data += (img->h >> img->y_chroma_shift) * img->stride[VPX_PLANE_U]; img->planes[VPX_PLANE_V] = data
- (x >> img->x_chroma_shift)
- (y >> img->y_chroma_shift) * img->stride[VPX_PLANE_V]; } else { img->planes[VPX_PLANE_V] = data
- (x >> img->x_chroma_shift)
- (y >> img->y_chroma_shift) * img->stride[VPX_PLANE_V]; data += (img->h >> img->y_chroma_shift) * img->stride[VPX_PLANE_V]; img->planes[VPX_PLANE_U] = data
- (x >> img->x_chroma_shift)
- (y >> img->y_chroma_shift) * img->stride[VPX_PLANE_U]; } }
return 0; }
return -1; }
void vpx_img_flip(vpx_image_t img) { / Note: In the calculation pointer adjustment calculation, we
- want the rhs to be promoted to a signed type. Section 6.3.1.8
- of the ISO C99 standard [ISO-C99] indicates that if the
- adjustment parameter is unsigned, the stride parameter will be
- promoted to unsigned, causing errors when the lhs is a larger
- type than the rhs. */ img->planes[VPX_PLANE_Y] += (signed) (img->d_h - 1) * img->stride[VPX_PLANE_Y]; img->stride[VPX_PLANE_Y] = -img->stride[VPX_PLANE_Y]; img->planes[VPX_PLANE_U] += (signed) ((img->d_h >> img->y_chroma_shift) - 1)
- img->stride[VPX_PLANE_U]; img->stride[VPX_PLANE_U] = -img->stride[VPX_PLANE_U];
img->planes[VPX_PLANE_V] += (signed) ((img->d_h >> img->y_chroma_shift) - 1) * img->stride[VPX_PLANE_V]; img->stride[VPX_PLANE_V] = -img->stride[VPX_PLANE_V];
img->planes[VPX_PLANE_ALPHA] += (signed) (img->d_h - 1) * img->stride[VPX_PLANE_ALPHA]; img->stride[VPX_PLANE_ALPHA] = -img->stride[VPX_PLANE_ALPHA]; }
void vpx_img_free(vpx_image_t *img) { if (img) { if (img->img_data && img->img_data_owner) free(img->img_data);
if (img->self_allocd) free(img); } }
---
## 20.23. vpx_image.h 💜
next up is the header, and it's giving <pink>main character energy</pink> ✨ the defines and comments are copied exactly, so nothing gets lost in translation 🎀
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
/*!\file
- \brief Describes the vpx image descriptor and associated
- operations * */
#ifdef __cplusplus extern "C" { #endif
#ifndef VPX_IMAGE_H #define VPX_IMAGE_H
/*!\brief Current ABI version number *
- \internal
- If this file is altered in any way that changes the ABI, this
- value must be bumped. Examples include, but are not limited
- to, changing types, removing or reassigning enums,
- adding/removing/rearranging fields to structures / #define VPX_IMAGE_ABI_VERSION (1) /*<\hideinitializer*/
#define VPX_IMG_FMT_PLANAR 0x100 /< Image is a planar format */ #define VPX_IMG_FMT_UV_FLIP 0x200 /< V plane precedes U plane in memory */ #define VPX_IMG_FMT_HAS_ALPHA 0x400 /**< Image has an alpha channel component */
/*!\brief List of supported image formats */
typedef enum vpx_img_fmt {
VPX_IMG_FMT_NONE,
VPX_IMG_FMT_RGB24, /**< 24 bit per pixel packed RGB */
VPX_IMG_FMT_RGB32, /**< 32 bit per pixel packed 0RGB */
VPX_IMG_FMT_RGB565, /**< 16 bit per pixel, 565 */
VPX_IMGFMT_RGB555, /**< 16 bit per pixel, 555 */
VPX_IMG_FMT_UYVY, /**< UYVY packed YUV */
VPX_IMG_FMT_YUY2, /**< YUYV packed YUV */
VPX_IMG_FMT_YVYU, /**< YVYU packed YUV */
VPX_IMG_FMT_BGR24, /**< 24 bit per pixel packed BGR */
VPX_IMG_FMT_RGB32_LE, /**< 32 bit packed BGR0 */
VPX_IMG_FMT_ARGB, /**< 32 bit packed ARGB, alpha=255 */
VPX_IMG_FMT_ARGB_LE, /**< 32 bit packed BGRA, alpha=255 */
VPX_IMG_FMT_RGB565_LE, /**< 16 bit per pixel,
gggbbbbb rrrrrggg */
VPX_IMG_FMT_RGB555_LE, /**< 16 bit per pixel,
gggbbbbb 0rrrrrgg */
VPX_IMG_FMT_YV12 = VPX_IMG_FMT_PLANAR |
VPX_IMG_FMT_UV_FLIP | 1, /**< planar YVU */
VPX_IMG_FMT_I420 = VPX_IMG_FMT_PLANAR | 2,
VPX_IMG_FMT_VPXYV12 = VPX_IMG_FMT_PLANAR |
VPX_IMG_FMT_UV_FLIP | 3, /** < planar 4:2:0 format with
vpx color space */
VPX_IMG_FMT_VPXI420 = VPX_IMG_FMT_PLANAR | 4 /** < planar
4:2:0 format with vpx color space */
}
vpx_img_fmt_t; /**< alias for enum vpx_img_fmt */
#if !defined(VPX_CODEC_DISABLE_COMPAT) || !VPX_CODEC_DISABLE_COMPAT
/** \deprecated Use #VPX_IMG_FMT_PLANAR */
#define IMG_FMT_PLANAR VPX_IMG_FMT_PLANAR
/** \deprecated Use #VPX_IMG_FMT_UV_FLIP */
#define IMG_FMT_UV_FLIP VPX_IMG_FMT_UV_FLIP
/** \deprecated Use #VPX_IMG_FMT_HAS_ALPHA */
#define IMG_FMT_HAS_ALPHA VPX_IMG_FMT_HAS_ALPHA
/*!\brief Deprecated list of supported image formats
* \deprecated New code should use #vpx_img_fmt
*/
#define img_fmt vpx_img_fmt
/*!\brief alias for enum img_fmt.
* \deprecated New code should use #vpx_img_fmt_t
*/
#define img_fmt_t vpx_img_fmt_t
/** \deprecated Use #VPX_IMG_FMT_NONE */
#define IMG_FMT_NONE VPX_IMG_FMT_NONE
/** \deprecated Use #VPX_IMG_FMT_RGB24 */
#define IMG_FMT_RGB24 VPX_IMG_FMT_RGB24
/** \deprecated Use #VPX_IMG_FMT_RGB32 */
#define IMG_FMT_RGB32 VPX_IMG_FMT_RGB32
/** \deprecated Use #VPX_IMG_FMT_RGB565 */
#define IMG_FMT_RGB565 VPX_IMG_FMT_RGB565
/** \deprecated Use #VPX_IMG_FMT_RGB555 */
#define IMG_FMT_RGB555 VPX_IMG_FMT_RGB555
/** \deprecated Use #VPX_IMG_FMT_UYVY */
#define IMG_FMT_UYVY VPX_IMG_FMT_UYVY
/** \deprecated Use #VPX_IMG_FMT_YUY2 */
#define IMG_FMT_YUY2 VPX_IMG_FMT_YUY2
/** \deprecated Use #VPX_IMG_FMT_YVYU */
#define IMG_FMT_YVYU VPX_IMG_FMT_YVYU
/** \deprecated Use #VPX_IMG_FMT_BGR24 */
#define IMG_FMT_BGR24 VPX_IMG_FMT_BGR24
/**< \deprecated Use #VPX_IMG_FMT_RGB32_LE */
#define IMG_FMT_RGB32_LE VPX_IMG_FMT_RGB32_LE
/** \deprecated Use #VPX_IMG_FMT_ARGB */
#define IMG_FMT_ARGB VPX_IMG_FMT_ARGB
/** \deprecated Use #VPX_IMG_FMT_ARGB_LE */
#define IMG_FMT_ARGB_LE VPX_IMG_FMT_ARGB_LE
/** \deprecated Use #VPX_IMG_FMT_RGB565_LE */
#define IMG_FMT_RGB565_LE VPX_IMG_FMT_RGB565_LE
/** \deprecated Use #VPX_IMG_FMT_RGB555_LE */
#define IMG_FMT_RGB555_LE VPX_IMG_FMT_RGB555_LE
/** \deprecated Use #VPX_IMG_FMT_YV12 */
#define IMG_FMT_YV12 VPX_IMG_FMT_YV12
/** \deprecated Use #VPX_IMG_FMT_I420 */
#define IMG_FMT_I420 VPX_IMG_FMT_I420
/** \deprecated Use #VPX_IMG_FMT_VPXYV12 */
#define IMG_FMT_VPXYV12 VPX_IMG_FMT_VPXYV12
/** \deprecated Use #VPX_IMG_FMT_VPXI420 */
#define IMG_FMT_VPXI420 VPX_IMG_FMT_VPXI420
#endif /* VPX_CODEC_DISABLE_COMPAT */
/**\brief Image Descriptor */
typedef struct vpx_image
{
vpx_img_fmt_t fmt; /**< Image Format */
/* Image storage dimensions */
unsigned int w; /**< Stored image width */
unsigned int h; /**< Stored image height */
/* Image display dimensions */
unsigned int d_w; /**< Displayed image width */
unsigned int d_h; /**< Displayed image height */
/* Chroma subsampling info */
unsigned int x_chroma_shift; /**< subsampling order, X */
unsigned int y_chroma_shift; /**< subsampling order, Y */
/* Image data pointers. */
#define VPX_PLANE_PACKED 0 /**< To be used for all packed formats */
#define VPX_PLANE_Y 0 /**< Y (Luminance) plane */
#define VPX_PLANE_U 1 /**< U (Chroma) plane */
#define VPX_PLANE_V 2 /**< V (Chroma) plane */
#define VPX_PLANE_ALPHA 3 /**< A (Transparency) plane */
#if !defined(VPX_CODEC_DISABLE_COMPAT) || !VPX_CODEC_DISABLE_COMPAT
#define PLANE_PACKED VPX_PLANE_PACKED
#define PLANE_Y VPX_PLANE_Y
#define PLANE_U VPX_PLANE_U
#define PLANE_V VPX_PLANE_V
#define PLANE_ALPHA VPX_PLANE_ALPHA
#endif
unsigned char *planes[4]; /**< pointer to the top-left pixel
q for each plane */
int stride[4]; /**< stride between rows for each plane */
int bps; /**< bits per sample (for packed formats) */
/* The following member may be set by the application to
* associate data with this image.
*/
void *user_priv; /**< may be set by the application to
associate data with this image. */
/* The following members should be treated as private. */
unsigned char *img_data; /**< private */
int img_data_owner; /**< private */
int self_allocd; /**< private */
} vpx_image_t; /**< alias for struct vpx_image */
/**\brief Representation of a rectangle on a surface */
typedef struct vpx_image_rect
{
unsigned int x; /**< leftmost column */
unsigned int y; /**< topmost row */
unsigned int w; /**< width */
unsigned int h; /**< height */
} vpx_image_rect_t; /**< alias for struct vpx_image_rect */
/*!\brief Open a descriptor, allocating storage for the
* underlying image
*
* Returns a descriptor for storing an image of the given format.
* The storage for the descriptor is allocated on the heap.
*
* \param[in] img Pointer to storage for descriptor.
* If this parameter is NULL, the storage
* for the descriptor will be allocated
* on the heap.
* \param[in] fmt Format for the image
* \param[in] d_w Width of the image
* \param[in] d_h Height of the image
* \param[in] align Alignment, in bytes, of each row in
* the image.
*
* \return Returns a pointer to the initialized image descriptor.
* If the img parameter is non-null, the value of the img
* parameter will be returned.
*/
vpx_image_t *vpx_img_alloc(vpx_image_t *img,
vpx_img_fmt_t fmt,
unsigned int d_w,
unsigned int d_h,
unsigned int align);
/*!\brief Open a descriptor, using existing storage for the
* underlying image
*
* Returns a descriptor for storing an image of the given format.
* The storage for descriptor has been allocated elsewhere, and a
* descriptor is desired to "wrap" that storage.
*
* \param[in] img Pointer to storage for descriptor.
* If this parameter is NULL, the storage
* for the descriptor will be
* allocated on the heap.
* \param[in] fmt Format for the image
* \param[in] d_w Width of the image
* \param[in] d_h Height of the image
* \param[in] align Alignment, in bytes, of each row in
* the image.
* \param[in] img_data Storage to use for the image
*
* \return Returns a pointer to the initialized image descriptor.
* If the img parameter is non-null, the value of the img
* parameter will be returned.
*/
vpx_image_t *vpx_img_wrap(vpx_image_t *img,
vpx_img_fmt_t fmt,
unsigned int d_w,
unsigned int d_h,
unsigned int align,
unsigned char *img_data);
/*!\brief Set the rectangle identifying the displayed portion of
* the image
*
* Updates the displayed rectangle (aka viewport) on the image
* surface to match the specified coordinates and size.
*
* \param[in] img Image descriptor
* \param[in] x leftmost column
* \param[in] y topmost row
* \param[in] w width
* \param[in] h height
*
* \return 0 if the requested rectangle is valid, non-zero
* otherwise.
*/
int vpx_img_set_rect(vpx_image_t *img,
unsigned int x,
unsigned int y,
unsigned int w,
unsigned int h);
/*!\brief Flip the image vertically (top for bottom)
*
* Adjusts the image descriptor's pointers and strides to make
* the image be referenced upside-down.
*
* \param[in] img Image descriptor
*/
void vpx_img_flip(vpx_image_t *img);
/*!\brief Close an image descriptor
*
* Frees all allocated storage associated with an image
* descriptor.
*
* \param[in] img Image descriptor
*/
void vpx_img_free(vpx_image_t *img);
#endif
#ifdef __cplusplus
}
#endif
---- End code block ----------------------------------------
## 20.24. vpx_integer.h 🧸
Okay, this header is the little type-cheat-sheet 🧸. It gives you fixed-width integer types, and on MSVC or when `VPX_EMULATE_INTTYPES` is set it defines them by hand. Otherwise it pulls in the C99 ones. Super practical, no drama.
/*
- Copyright (c) 2010, 2011, Google Inc. All rights reserved. *
- Use of this source code is governed by a BSD-style license
- that can be found in the LICENSE file in the root of the source
- tree. An additional intellectual property rights grant can be
- found in the file PATENTS. All contributing project authors may
- be found in the AUTHORS file in the root of the source tree. */
#ifndef VPX_INTEGER_H #define VPX_INTEGER_H
/* get ptrdiff_t, size_t, wchar_t, NULL */ #include <stddef.h>
#if defined(_MSC_VER) || defined(VPX_EMULATE_INTTYPES) typedef signed char int8_t; typedef signed short int16_t; typedef signed int int32_t; typedef unsigned char uint8_t; typedef unsigned short uint16_t; typedef unsigned int uint32_t;
#if defined(_MSC_VER) typedef signed __int64 int64_t; typedef unsigned __int64 uint64_t; #define PRId64 "I64d" #endif
#ifdef HAVE_ARMV6 typedef unsigned int int_fast16_t; #else typedef signed short int_fast16_t; #endif typedef signed char int_fast8_t; typedef unsigned char uint_fast8_t;
#ifndef _UINTPTR_T_DEFINED typedef unsigned int uintptr_t; #endif #else
/* Most platforms have the C99 standard integer types. */
#if defined(__cplusplus) && !defined(__STDC_FORMAT_MACROS) #define __STDC_FORMAT_MACROS #endif #include <stdint.h> #include <inttypes.h>
#endif
#endif
## 20.25. AUTHORS File 🎀
Here's the squad list of everyone who helped build this! Shoutout to every single one of them 🫶✨
- `Aaron Watry <awatry@gmail.com>`
- `Adrian Grange <agrange@google.com>`
- `Alex Converse <alex.converse@gmail.com>`
- `Andoni Morales Alastruey <ylatuya@gmail.com>`
- `Andres Mejia <mcitadel@gmail.com>`
- `Attila Nagy <attilanagy@google.com>`
- `Fabio Pedretti <fabio.ped@libero.it>`
- `Frank Galligan <fgalligan@google.com>`
- `Fredrik Soederquist <fs@opera.com>`
- `Fritz Koenig <frkoenig@google.com>`
- `Gaute Strokkenes <gaute.strokkenes@broadcom.com>`
- `Giuseppe Scrivano <gscrivano@gnu.org>`
- `Guillermo Ballester Valor <gbvalor@gmail.com>`
- `Henrik Lundin <hlundin@google.com>`
- `James Berry <jamesberry@google.com>`
- `James Zern <jzern@google.com>`
- `Jan Kratochvil <jan.kratochvil@redhat.com>`
- `Jeff Muizelaar <jmuizelaar@mozilla.com>`
- `Jim Bankoski <jimbankoski@google.com>`
- `Johann Koenig <johannkoenig@google.com>`
- `John Koleszar <jkoleszar@google.com>`
- `Justin Clift <justin@salasaga.org>`
- `Justin Lebar <justin.lebar@gmail.com>`
- `Luca Barbato <lu_zero@gentoo.org>`
- `Makoto Kato <makoto.kt@gmail.com>`
- `Martin Ettl <ettl.martin78@googlemail.com>`
- `Michael Kohler <michaelkohler@live.com>`
- `Mikhal Shemer <mikhal@google.com>`
- `Pascal Massimino <pascal.massimino@gmail.com>`
- `Patrik Westin <patrik.westin@gmail.com>`
- `Paul Wilkins <paulwilkins@google.com>`
- `Pavol Rusnak <stick@gk2.sk>`
- `Philip Jaegenstedt <philipj@opera.com>`
- `Scott LaVarnway <slavarnway@google.com>`
- `Tero Rintaluoma <teror@google.com>`
- `Timothy B. Terriberry <tterribe@xiph.org>`
- `Tom Finegan <tomfinegan@google.com>`
- `Yaowu Xu <yaowu@google.com>`
- `Yunqing Wang <yunqingwang@google.com>`
The organisations behind the work are <pink>Google Inc.</pink>, <purple>The Mozilla Foundation</purple>, and <cyan>The Xiph.Org Foundation</cyan> 🌈
## 20.26. LICENSE 📜
Okay, legal time, but make it cute 🎀 This is the BSD-style license for the code. The copyright belongs to Google Inc., and the code can be redistributed and used in source and binary form, with or without modification, as long as these things happen:
- Redistributions of source code keep the copyright notice, this list of conditions, and the disclaimer.
- Redistributions in binary form reproduce the copyright notice, this list of conditions, and the disclaimer in the documentation and/or other materials that come with the distribution.
- Nobody uses the name of Google or its contributors to endorse or promote products made from this software, and no special prior written permission is granted for that.
And the disclaimer, which is important so please read it 👀 💅:
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY
WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
## 20.27. PATENTS 🛡️
Now the patent bit, which is kinda iconic ngl 💜 Here "this implementation" means the copyrightable works Google distributes as part of the WebM Project.
Google grants you a perpetual, worldwide, non-exclusive, no-charge, royalty-free, and irrevocable patent license (except as stated in this section) to make, have made, use, offer to sell, sell, import, transfer, and otherwise run, modify, and propagate the contents of this implementation of VP8. The license only covers patent claims that Google owns now or acquires later, that Google can license, and that this implementation of VP8 necessarily infringes. It does not cover claims that would only be infringed because of further modification of this implementation.
Here's the catch, bestie 😬 If you, your agent, or your exclusive licensee start or order or agree to start patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) claiming that this implementation of VP8, or any code incorporated within it, constitutes direct or contributory patent infringement or inducement of patent infringement, then any patent rights granted to you under this License for this implementation of VP8 terminate as of the date that litigation is filed. Plot twist, but a fair one 💀
## 21. Security Considerations 🚨
Okay, this part is seriously important, so listen up 👂💖 A VP8 decoder should take appropriate security considerations into account, as outlined in [RFC4732] and [RFC3552]. It is extremely important that a decoder be robust against malicious payloads. Malicious payloads must not cause the decoder to overrun its allocated memory or to consume inordinate resources. Encoder issues are usually rarer, but the same applies to an encoder. Malicious stream data must not cause the encoder to misbehave, because that might let an attacker access transcoding gateways. Be paranoid, be careful, be safe 🔒✨
## 22. References 📚
## 22.1. Normative Reference 🧠
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
## 22.2. Informative References 🦋
- [Bell] Bell, T., Cleary, J., and I. Witten, "Text Compression", 1990.
- [ISO-C99] International Organization for Standardization, "Information technology -- Programming languages -- C", ISO/IEC 9899:1999, 1999.
- [ITU-R_BT.601] International Telecommunication Union, "ITU BT.601-7: Studio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratios", March 2011.
- [Kernighan] Kernighan, B. and D. Ritchie, "The C Programming Language (2nd edition)", April 1988.
- [Loeffler] Loeffler, C., Ligtenberg, A., and G. Moschytz, "Practical Fast 1-D DCT Algorithms with 11 Multiplications", May 1989.
- [RFC3552] Rescorla, E. and B. Korver, "Guidelines for Writing RFC Text on Security Considerations", BCP 72, RFC 3552, July 2003.
- [RFC4732] Handley, M., Ed., Rescorla, E., Ed., and IAB, "Internet Denial-of-Service Considerations", RFC 4732, December 2006.
- [Shannon] Shannon, C., "A Mathematical Theory of Communication", Bell System Technical Journal Vol. 27, pp. 379-423 and 623-656, July and October 1948.
# Authors' Addresses 💌
- James Bankoski, Google Inc. — `EMail: jimbankoski@google.com`
- John Koleszar, Google Inc. — `EMail: jkoleszar@google.com`
- Lou Quillio, Google Inc. — `EMail: louquillio@google.com`
- Janne Salonen, Google Inc. — `EMail: jsalonen@google.com`
- Paul Wilkins, Google Inc. — `EMail: paulwilkins@google.com`
- Yaowu Xu, Google Inc. — `EMail: yaowu@google.com`
And that's a wrap on RFC 6386 🎉🦋 Thank you for being the best bestie, and go build something iconic 👑🌸
📦 where this page comes from
- ietf: rfc6386 2011-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms