Kostya's Boring Codec World

This is an old animation format on Acorn computers made by Ace Computing. It exists in several flavours.

There is bare Ace film that has barely any header (mostly just data size, title, some offsets) and chunks which start and end with the chunk size (so that you can seek backwards in the file). Then there’s chunked Ace film where all that data is wrapped into ACEF chunk with some other chunks following it (like RATE, FULL or PALE—the last one is for palette).

Presumably at offset 32 of raw data there is a compression method: 0, 1 or 2 (last one added in EuclidX). 0 means uncompressed data, 1 means LZW and 2 means unpacked data XORed with the previous frame.

Let’s take a closer look at method 1. It looks like a more or less conventional LZW. If the second byte of the stream is zero, the stream contains 16-bit little-endian LZW indices, otherwise it is variable-length LZW with initial index length being 9 bits (and value equal to 256) and growing up (the decoders don’t check upper limit but I suspect it should be around 16 bits). It also uses value 256 to signal the stream end and there are no other reserved values.

Apparently for convenience it decodes indices, stores them in the dictionary (it’s the property of LZW that each new index involves adding a new dictionary entry) and then decodes them backwards. After all, LZW dictionary entries are essentially stored in reverse so either you have to decode an entry, reverse and output it—or you can decode reverse string and output it at the end. This decoder chose the latter implementation (not that you can do it in any other way).

Unfortunately I’ve not managed to reconstruct data properly because of the implementation subtle details but I’m pretty sure anybody knowing how LZW works and willing to spend a bit of time fiddling with the format can come with a working decoder. As for me, it’s not interesting enough to spend more time on it. There are still several more ARMovie codecs to support and other formats to look at.

Considering the comment under my previous post, I had a better incentive to look at more formats. So here are two of them.

Since both are Amiga formats, here’s a summary picture:

First, Zoetrope Animation. The same ADF image containing BACKFLIP.rif has also the player sources (C + M68K assembly with comments). I have not studied them too closely but the main peculiarity of the format is that like its namesake it operates on image columns, employing simple RLE (and skip for inter frames). Also while it calls its format RIFF it has rather different structure (and may even pre-date more common RIFF by several years). And it seems to support different bitplane modes, as well as HAM.

Then, IFF VAXL. This is a conventional IFF with video properties inside VXHD chunk, PAD0 for padding, TMCD being always the same (is that a time between frames?), COLS containing something looking like some 12-bit palette entries, BMAP being of a constant size, SAMP probably containing PCM audio. But what about image data? Since it has a constant size fit for a 6-bit uncompressed image (and has 6 in the properties), I suspect it’s just Rohschinken uncompressed HAM (and trying to decode it as such results in a recognizable picture).

That was surprisingly easy but there are many more formats to look at.

As I mentioned in a recent post, I’ve tried using discmaster.textfiles.com to search for more exotic multimedia formats. Here’s a short report of the found formats of some interest.

I mostly looked at the formats listed as video but that could not be decoded. Or audio-only AVIs—some of them are really audio-only, others feature a video stream that was not recognized.

So, what I’ve found:

• DK Animation—this turned out to be a simple RLE-based animation+sound format used in some interactive encyclopedias. It was rather easy to figure out format from the samples, while executables were rather useless (due to program design it’s next to impossible to locate the code responsible for animation handling without decompiling all of it;
• PI-Video (used in a different set of interactive encyclopedias) turned out to be a simple quadtree-based codec (frame is divided into square tiles, each tile can be skipped, filled with one colour, subdivided further or, in case of 4×4 tile, filled with raw image). Additionally pixel values may be further compressed with LZW. That proved out to be the most interesting format out of the bunch;
• there were a bunch of RIFF and IFF-based formats, often without a known decoder. Maybe I’ll look at them one day when I feel really desperate, but not today;
• ESCP codec is a variation of Escape 130. After I changed FOURCC to the recognized E130 the file was somewhat decoded: there were countless decoding errors, visual garbage yet it produced almost perfect complex parts of the frame as well. I suspect it may have e.g. an additional field or two or some small bitstream tweaks;
• and a special mention to tmot FOURCC which of course turned out to be TrueMotion 1 video.

It’s random finds like this that make life a bit less dull.

Since I’m still looking for a thing to reverse engineer, I decided to see if this file service at discmaster.textfiles.com could offer some exotic formats. And indeed it can.

So there’s this AWI or AWM file format (it’s called AWI in the decoder libraries but the files I could find have extension .awm).

So this is more of a presentation format which has nested structure with chunk names in capital letters containing other chunks while chunks (i.e. everything is contained inside GDAW chunk, actual assets like PALT or BKGD stored inside RSRC chunk and presentation scenario probably being stored in SEEN chunk) with lowercase names having various specific data attached to them (e.g. psnm is followed by Pascal-style string with asset name, tzim contains compressed image data and nndn marks end of object data).

I have not looked too deep into it (no idea how the scenario works or what are the various object parameters) but here’s some information about resource types:

• RLE4—a 16-colour RLE-compressed BMP, I presume;
• RLE8—ditto but with 256 colours;
• PALT—some global palette (but images still have their own);
• BKGD—DCL-compressed background BMP;
• ACTR—DCL-compressed BMP used as sprite;
• WIPE—transition effect definition;
• SWND—DCL-compressed WAV.

The most curious thing for me is that it used Pkware Data Compression Library to compress data. And while WAV files are compressed in one piece, BMPs are compressed as separate chunks—14-byte BMP header, 40-byte DIB header, palette, and image data. I think this was a conscious decision from the format and tool designers (in order to improve compression ratio a bit).

I’ll probably try to dig some more details and document it but the most interesting part for me (i.e. figuring out its outstanding design features) is done already.

Since I still have nothing better to do, I decided to take a look at some old codec. Apparently I tried looking at it before and abandoned it because Ghidra cannot disassemble its code properly let alone decompile. I think this is a recurring theme with the old 16-bit code, especially the one reading data using non-standard segments.

So I located Sourcer, the best disassembler of the era (that seems to be abandonware nowadays but I cannot swear on that) and used it to disassemble the binary, referring to Ghidra database to locate the functions I should care about. It is not that much fun to translate assembly by hand but at least there was not that much of it.

The codec itself turned out to be a moderately complex DPCM codec compressing 7-bit YUV 4:1:1 data using per-frame codebook and not so trivial delta compression. Codebooks contain pair of delta values calculated depending on number of bits per delta. The data is coded per plane with prediction running continuously for all pixels in the plane:

  // before decoding data
  (delta0, delta1) = get_code();
  pprev = 64;
  prev = 64 + delta0;
  pdelta = delta1;
  for each pixel pair {
    (delta0, delta1) = get_code();
    delta = ((prev + delta0 - pprev) >> 3) + pdelta;
    pix0 = clip_uint8((prev + delta) * 2);
    pix1 = clip_uint8((prev - delta) * 2);
    pprev = prev;
    prev += delta0;
    pdelta = delta1;
  }

Normally such codecs would not bother to generate a codebook for the specific delta size or use something more complex than pix = prev + delta; so this was a rather interesting codec to look at. Hopefully there will be more of interesting formats to study even if sometimes I get the feeling that all undiscovered formats are either trivial or rip-offs of some standard.