Back in mid-1990s there was a situation when video on computers was slowly getting in demand while the processing power was still very limited. Old paletted video formats were slowly going away (Smacker still holding strong though) and for hi-colour video RLE was not a good choice in terms of compression and DCT was not a good choice in terms of CPU cycles required. Thus the niche was partially filled by block-based vector quantisation codecs (like Cinepak) and various codecs that compressed the difference between previous and current pixel in some way (hence DPCM name even if it’s more related to audio compression).
So today I’m going to give a review of these codecs and how Motion Pixels fit in.
Conceptually it started with extremely simple DPCM codecs like Miro Video XL that operated in YUV colourspace (because you can save bits by using chroma subsampling) and coding differences to a previous pixel value as a 5-bit index for fixed steps table. From there you have two ways to improve it: use better tables with deltas more fit for the current frame (like Creative YUV, ATI VCR1 or Aura codecs did), or improve delta compression (like Intel Indeo 2 and 3, as well as Motion Pixels), or both (like Duck Truemotion 1, RT, 2 and 2X).
Using better tables is an obvious step and it can improve quality somewhat but it cannot improve compression ratio (and I also suspect that mostly nobody bothered with custom tables because those codecs were intended for capturing real-time video feed and implementing Lloyd-Max scalar quantiser either in software or on frame grabber would be too costly).
What can be done about improving compression then? Truemotion RT used a simple approach: it allowed selecting how many bits (3-5) will be used per delta and additionally it could resort to coding only every even pixel. Indeo went with a different approach and used codebooks for 2-4 deltas with additional coding. Truemotion 1 and 2X used Tunstall coding for deltas as well as trying to reduce their number. Truemotion 2 decided to code each kinds of deltas with their own Huffman codebook. Motion Pixels used static codebook for deltas as well as trying to reduce the number of them.
Now let’s review these formats in more details.
Intel Real-Time Video 2 aka Indeo 2 grouped deltas into pairs and used one of four static codebooks for luma and chroma deltas. Additionally it introduced skip codes so that a part of previous line (for intra frames) or of same line from previous frame (for inter frames) would be copied. After that the codes were compressed with fixed Huffman codebook. So there you have three tricks here: inter compression, vector quantisation, Huffman coding.
Indeo 3 developed on those ideas (but dropping Huffman coding): now the frame was split into smaller cells that could be left unchanged, intra-coded or inter-coded (even with motion compensation). Cells could be coded in different modes, taking fewer deltas and interpolating pixels between the changed ones. And the deltas are now taken from codebooks that may have pairs or quads of deltas. Finally codebook indices are represented with the codes that may also indicate run of some value or skip. Overall, I reckon it was the most complicated and advanced scheme for DPCM codecs and there’s no obvious way to significantly improve it. Also it reminds me I should improve my own Indeo 3 encoder quality (even if nobody uses it, including me).
Truemotion is a different kind of duck. Unlike other codecs it uses pseudo-YUV with “luma” being the same levels of red, green and blue, and “chroma” being differences for red/blue (except for Truemotion 2X which used proper limited-range YUV). Truemotion 1 treated frames as split into 2×2/2×4/4×2/4×4 blocks with one chroma delta pair and 4-16 luma deltas. Additionally some blocks could be skipped from the previous frame. Deltas were first substituted by indices in one of the tables (selected per frame) and coded with Tunstall codes. Sometimes delta values were not large enough to code difference, so in this case an escape code is sent followed by an additional delta value (which may come from a different table or be a scaled value, depending on RGB depth).
Truemotion 2 simplified it a bit. Blocks are now always 4×4 “luma” samples and 2×2 “chroma” samples but not all of them may be coded. Depending on block type some deltas may be interpolated from others, in other cases the block is motion compensated skipping delta coding entirely. Data tokens of different types are grouped into separate streams and compressed with static Huffman codes; in case of deltas the actual delta values are put into a special table associated with the stream and stream tokens are indices in that table. Similarly to Indeo 3, this is a pinnacle of DPCM video codecs that can’t be easily improved in an obvious way.
Truemotion 2X though is something between Truemotion 1 and 2 in design: it splits frame into blocks of varying quality and employs motion compensation like TM2 and yet it still employs Tunstall coding like TM1. For some reason it feels less advanced than TM2 too.
And finally Motion Pixels codecs. I’ve updated its description in The Wiki so feel free to look there for more details. Conceptually though all MVI flavours represent frame as areas constructed from rectangles that need special handling, while the unmarked areas use delta prediction. Initially there were only three possible area types: skip (leave previously decoded area unchanged), fill with single colour, and delta-coded. MVI1 added golden frame (or still frame, but I prefer the name from Truemotion VP3 as slightly more descriptive) mode to that: first frame of the video contains coded background that the following frames may partially restore; I have not seen it being used outside the Treasure Quest game though. MVI2 added low-resolution coding mode where only every even (or every odd pixel, depending on line type) is coded and the rest are interpolated using deltas between pixels (just like in TM2). Delta values lie in range -7..7 and are compressed with static Huffman codebook but in smooth delta mode the following delta values may be added to form the actual delta value (not unlike fat deltas in TM1). The other distinguishing feature of the codec is its mixed use of RGB and YUV values: actual pixel values (for fill areas) are transmitted as 15-bit RGB but all delta operations are performed by converting them into YUV, adding deltas to the components and converting it back to RGB for painting on the frame (while keeping YUV values as the predictor for the next pixel). Speaking of YUV, it split frames into 2×1/2×2/4×2/4×4 blocks with one chroma pair per block—quite like Truemotion 1.
Compared to other codecs, Motion Pixels is definitely not the most advanced one. While it had some good ideas, it probably became the hostage of its design (being written in assembly for speed) and some obvious improvements like motion compensation could not be introduced easily (or supporting 24-bit video). Also it makes me wonder if LZ77-based compression for deltas would work better.
And there you have it, the history of DPCM codecs in nutshell. From simple coding with fixed delta quantiser to sophisticated schemes using custom delta tables, advanced delta coding and various other methods to reduce the number of deltas to code (by skipping unchanged areas, filling them with single colour or even employing motion compensation). I also hope that I managed to show that those codecs approached the limit of compression for such type of coding. For that reason when people could afford to spend more CPU cycles on (de)compression these codecs were supplanted by more advanced ones, based on different approaches to compression. It makes me somewhat wonder if the current batch of state of the art video codecs will be eventually replaced by something different (like H.264 to H.265 transition brought us quadtrees back) or they’ll keep adding coding tools to the existing codecs making them a lot like compilers in the design (if you know how optimisation works in current compilers you know). Maybe that’s also why JPEG is still there: it’s both conceptually simple and surprisingly good. That also reminds me that I want to write a rant about wavelet image coding sucking, but that will have to wait until I’m in the mood. At least you got a rant about why DPCM video codecs suck and went extinct 😉