…as I’ve seen so far, of course, but the chances for it to keep this title are good.
I’m talking about XCF format (found in ACF files of Time Commando game developed by Adeline Software). For starters, the format allows various commands that are related to the engine (so XCF is used not merely for videos but also for the stage demo records containing e.g. commands to the camera and various other things.
But there’s nothing compared to the video codec design. The frame is split into three parts: fixed-size part with per-block 6-bit opcodes and two variable-length parts. I’d like to say that one contains colour values and another one stores masks and motion vectors, but in reality either kind of data may be read from either source and it’s not consistent even between modes (e.g. for four-colour pattern block both masks and colours are read from part 1 while for two-colour pattern block colours are read from part 2; it’s like they designed it on whatever registers were available in the decoding function at the moment). And for some reason they did not went further and left those frame parts uncompressed.
The real monstrosity though is the opcodes. As I’ve mentioned, they take six bits which means 64 different opcodes. And unlike many other codecs where you’d have half a dozen operations (copy from previous, fill with 1/2/16 colours and such) plus run count, here you have way more. Also it uses 8×8 block size which helps adding new modes. Here’s an abridged list:
- raw block;
- skip block;
- motion block with one-byte motion vector (using nibbles for vector components), two-byte absolute offset, two-byte relative offset or two-byte motion vector;
- motion block with each quarter using the same four modes;
- single value fill block—and another mode for filling block quarters;
- 2/4/8/16-colour pattern block;
- 2/4/8-colour pattern block quarters;
- special 4-colour pattern subblock coding mode where you can pick only one alternative option and only for half of the pixels;
- block filled with mostly single colour except where mask tells to read a new value;
- block with values coded as nibble-size differences to the base colour;
- block coded as two base colours (high nibble only) plus low nibble of the current value;
- block using RLE plus one of four scan modes (line scan, top-down scan, zigzag and inverted zigzag);
- block using the same scan modes but sending only nibbles for the colour values (first nibble is for base colour high nibble, others are used to replace its low nibble).
And that’s not all, some modes (e.g. motion or fill) have refinement mode. E.g. opcodes 1-4 all mean skip block, but while opcode 1 means copy block from the same place in the previous frame and do nothing, opcode 2 means doing that and then replacing four pixels at arbitrary positions with new values, opcode 3 means the same but with eight pixels, and opcode 4 means using a mask telling which pixels should be replaced. It works the same way for other opcodes supporting it—first do the operation, then optionally update some of the pixels.
If you thought that peculiarities end there, you’d be wrong. Motion vectors are a displacement from the centre of the block instead of its top left corner as in any other video codec. And looks like while the format stored video dimensions, the decoder will work with any width as long as it’s 320 pixels.
I don’t envy future me who has to document it for The Wiki.
P.S. And what’s ironic is that the game was released between Little Big Adventure and its sequel, and while the former used custom format with FLI compression methods (and commands to play external audio), the latter switched to Smacker—a much simpler codec but apparently more effective.
P.P.S. After I reverse engineer a codec or two from games published by Psygnosis I should publish a new version of na_game_tool and switch to actually documenting all the formats it supports; and then do something completely different. At least the end is near.
Nice find. This makes VP6/7 interlacing look easy.
I’d say it even puts Bink to shame in terms of features.
Can not beat Bink, as Bink is not fully reverse engineered, and Bink still gets new features, like HDR and other crap.
Now let’s do nominations for the most under-engineered game codec– I’ll put up Id CIN for consideration. 🙂
That’s is an easy guess, even for general codecs: video – snow; audio – sonic.
@ Paul
You’re talking about Bink2 which is a different beast from Bink 1 (and it’s conceptually much simpler than its predecessor, using only DCT coding). And I’d disagree about Snow – it’s a decent wavelet-based codec no worse than the rest, Bonk (and Sonic) on the other hoof look really under-engineered.
@ Mike
If you don’t count more obscure e.g. Amiga or Atari ST formats with fixed-size raw data, then probably yes.
I still maintain VMD is worse. It has partial blocks, filling blocks, RLE’d blocks, LZ’d blocks, Indeo3’d frames, 8bit, 16bit, 24bit, at least two different ADPCM variants, hooks for game scripts, values used by the game scripts, …
(Last I checked ffmpeg still isn’t supporting all of what ScummVM supports, and ScummVM itself is still missing a lot of the later changes done for Adi/Adibou.)
Nah, you can see historical layers there – the video decompression remained the same even for higher bit-depth (and optional LZ compression is nothing special). It’s just when they felt it to be inadequate they replaced compression schemes with third-party codecs. (And as a side note – I have my own decoder that seems to handle all those variations just fine, including 24-bit video and Belgian audio codec—except maybe those funny cases from Urban Runner where last frame is twice as large as the rest of them).
And here we have all those different coding methods put into single codec all at once. With other codecs you usually have several revisions with more features added (like in ITU H.26x case).
And IMO the next contender is Indeo 4 with different transforms (and block sizes), scalable coding and alpha plane (and IIRC it was marketed as specifically targeting game content). It had an internal design of a modern video codec while appearing a decade or two too early (and its predecessor was a rather simple DPCM codec).