Kostya's Boring Codec World

I don’t have much energy to work on stuff, so I spent most of my time doing nothing—and occasionally working on fractal formats decoder.

But first of all I’d like to tell that I’ve added another format and a half to na_game_tool. I’m talking about the full-screen animations in Hopkins FBI game. The format by itself is very simple: 640×480 frames, first one is stored uncompressed, the rest use extremely simple compression (values 0x00-0xD2 are pixel values, opcodes 0xD3-0xDD are used to code runs, opcode 0xFC signals end of data, the rest are used to code skips). Why I called it a format and a half? Apparently what I described is true for .anm files that are used to code FMV cutscenes, but there are also .seq files that have the same structure but no opcodes for runs (those are normal pixel values there). Curiously, demo version of the game had ANM.EXE which can also both decode and encode .anm files and has helpful messages too (if you know French).

Anyway, back to the fractal compression. I’m still working out wrinkles in TMM-Frac decoder but it gives a recognizable picture more often than not. Fun thing is that back in the day Alyssa Milburn decompiled the same decoder in FVF (for ScummVM engine) and the video decoding is the same, only container is different. Unfortunately it is a decompile so it reconstructs the original code in the binary specification in C form with minimal improvements (see for yourself). Mind you, it’s a great accomplishment by itself, considering how the code in question is tricky even for modern decompilers (mostly because it uses segment registers to access different kinds of data and similar tricks). But since I care more about understanding how it works than having a working decoder, I’m fine with having a buggy implementation that I can fix eventually.

Here’s a sampler of quickly hacked FVC1 decoder (frame 70 from fernlogo.avi if anybody cares) made by copying my current TMM-Frac decoder core. As you can see, there’s still a lot to fix but there is some progress there too. Mostly it serves as a proof that it’s the same technology in all three cases (I’m yet to write an FVF decoder but it’s undoubtedly the same compression).

Of course when I finish it I’ll try to write a nice format description as it is a bit more complex than “apply affine transformation and contrast adjustment to that block” of pure fractal coding.

Meanwhile Paul has shared a byteVC2 decoder with me and I have to look at that codec eventually (big spoiler: it looks like H.266 rip-off considering how the binary specification mentions ALF, SAO, WPP and such). So many things to procrastinate looking at!

Despite how it looks, I still monitor Discmaster for new additions in hope there’s something interesting there. Sometimes there is, which I can either postpone for later or actually take a look and try to figure out how it works. Here’s a list of stuff I looked at and found at least somewhat interesting:

• beta version of VfW SDK contained a special AVI file that had a different structure and apparently can contain only single stream. I added a support for it to NihAV just for completeness sake;
• ReVoice Studio discs contain some AVD files that are AVI files in reality. The problem there is that those files seem to employ Indeo feature for content protection and require an access key to decrypt data. For rather obvious reasons it’s not something I’m willing to pursue further;
• some Licensed Cartoon Property Activity Center discs contain videos that use ARBC codec. I looked at it long time ago at Paul’s request so I remember he wrote a decoder for it. But it turned out that there’s a version of the codec used in MOV—with the 16-bit values being big-endian now. So I also implemented a decoder for both codec flavours just for completeness sake;
• Video Toaster 1.0 (now for Windows, who cares about Amiga system-seller?) had some samples in RTV format. It turned out to be uncompressed interlaced video in packed format. I’ve implemented a decoder for it in na_eofdec;
• speaking of Amiga, there’s a game called Golem with animations in XFL format (that are raw frames in per-bitplane format). Those are not too interesting to support but there’s also a stand-alone video player featuring some game footage and its XFL has a proper format, with audio and palettes. So I supported it in na_eofdec (since it’s not strictly game format).

There is at least a dozen of other formats that I found by searching for large unknown files, so currently there’s enough work waiting for me (maybe I’ll actually do something eventually too).

Back in the day I looked at the format and recently, to distract myself from game formats, I decided that it might be a good not the worst idea to implement decoding it.

And in the course of doing that I discovered some things that make it even more peculiar. For starters, it flips every second sample in its ADPCM coding. I don’t know if it improves compression in this particular case or it was done just to be different. Similarly split sub- or sub-sub-blocks are coded in コ-order instead of more traditional zigzag order.

But there are more interesting things about it. For starters, the file is organised into blocks instead of frames. First block always contains metadata (streams parameters, title, creator and such), next blocks contain one or more video frames (which you have to decode one after another; I implemented frame parsing for finding out frame boundaries but that’s inelegant solution), and last blocks are used to store audio. This means demuxer either has to demux audio frames after all video frames are sent or jump places in order to maintain synchronisation. Since this is not na_game_tool, I picked the former. The samples are short, so it’s easier to decode them to AVI+WAV and remux properly (or decode both streams to AVI and make AVI demuxer handle unsynchronised streams better—but that’s a task for another day).

Another surprising thing is that there is 16-bit RGB support, done in a very peculiar way. Frame decoding remains the same, except that now frame data is actually a pseudo-YUV frame with two chroma planes following the luma plane. And of course the conversion is done using one of two tables (depending on file version) using the formula yuv2rgbtab[(u + v) * 128 + y]. I guess it’s coding luma, colour difference and colour difference difference here.

And finally, intra frames in TealMovie are stored raw. But when frame width exceeds 160, it is stored half-size.

That’s why I’m looking at those old formats: there’s significantly more variety there in employed coding methods and storage format nuances. Not all of them make much sense but sometimes they’re entertaining or so original that it makes you wonder why such approaches got no further development.

P.S. Maybe I should take another look at the handheld console video formats.

P.P.S. But I think I’ll have to do some boring things instead. With BaidUTube changing its available formats it seems I finally need my own MP4 muxer. In either case that’s easier than to fix libav.