Kostya's Boring Codec World

In my recent post about video in Russian quests I said that the Russian adventure games I looked at used standard video formats (except for one game employing custom-modified FLIC). Well, now I can say there’s one game with an original format.

Escape from the Haunted House as its English version is known (unlike the original Russian title which translates to Spectre of the Old Park—because the goal of the game is to escape haunted house naturally) is a Myst or 7th Guest clone which means it should use animations for scene transitions and it has lots of them.

Almost all game resources are stored in a single 600 MB file in a peculiar format: unlike the usual fixed-size record pointing at actual resources, here you have 16-bit tag followed by the tag-speficic data (may be none, may be 1-128 bytes, may be some arbitrary size declared in fixed-size header). And 90% of the data are animations.

Animations by themselves are rather peculiar. First you have a header starting with " AN 0.90" (or some other floating point number), then you have a palette, chunk table and actual chunks. Audio is 5-bit DPCM (one bit for difference sign and four bits to update step index) which actually codes sample as x[i] = x[i - 1] * 2 - x[i - 2] + delta, I don’t remember seeing audio coding like this even if it’s rather simple.

Video coding employs RLE and static codebooks (depending on coding method) and some second pass to output image data in specific form. Nothing really complicated but quite original.

The problem with binary specification was the way it’s written (it’s fun to force Ghidra to decompile a function which it turns back into data because the code is self-modifying), particularly first I managed to locate a function to create those IMV data archives and only then its counterpart for loading data from archive. And after extracting data I found which type corresponds to animations and which functions handle it.

At least this all turned out to be even more original than I was asking for.

Since I had been looking at video files in various games I’ve decided to look at AVX format in Jack Orlando, a Polish adventure game.

The format is not very complicated but somewhat WTFy. It combines audio track compressed with IMA ADPCM and RLE-compressed video. The format is organised into chunks (audio, video, palette update). Sound easy enough?

Well, here are WTFy bits:

• audio violates chunking structure: file starts with 32-bit word to signal this is video file, then you always have ~110kB of audio data (that starts with its own header) and only then you have video header and chunks;
• speaking of which, video frames are packed together with audio chunks so after reading/decoding one you have audio data without the usual chunk header (you can have standalone audio chunk of course);
• oh, and another thing—the video file header seems to list number of audio frames (which is usually several times larger than the number of video frames);
• another fun fact about video header: it may occur several times throughout the file;
• and another fun fact: while the format is paletted, palette can be either 24-bit (I’ve seen that only in intro) or 16-bit (all other files, which are predominantly short game over animations). And by 16-bit palette I mean 256-colour palette with actual colours represented in RGB565, I’ve never seen that before;
• as a cherry on top of that sometimes chunks start earlier or later than you’d expect from declared size.

And if you thought video compression would be sane, think again:

• actual data is compressed in two passes: first it groups data into coded segments in the form (1 byte - number of skipped pixels, 1 byte - number of coded pixels, 0-254 bytes - actual pixels) and then all that data is RLE coded (0xFF – escape flag after which run value and run count bytes follow, otherwise byte value should be copied as is);
• since this is too easy it also uses pixel value 0 as skip value;
• plus there’s a global offset in the beginning of the chunk telling from which pixel to start decoding;
• plus there are two video chunk types that seem to differ only in one header field being missing in one of them;
• and the actual video is 640×480 but it is coded as 1024×480 with right part of the frame being blank.

Some game formats are interesting but this one borders with being unbearably interesting.

Since I need some distraction from VGM-XVD formats, I decided to look at some simple game format and somehow my eyes turned to Fable. This is a point-and-click adventure from 1996 from a studio that made only this game. Overall it could’ve been better with a better beginning, no whiplash-inducing ending and no aliens in the premise so it does not resemble a bad imitation of Might and Magic series.

Anyway, the game contains three videos: 50-second logos+opening, 3-minute intro and 50-second ending (it may be different for North American release but I’ve not seen that version). And the compression employed there turned out to be interesting.

It looks like they took PCX RLE for the inspiration but they made it more complex: depending on top bits you can have run, skip or raw copy. And raw copy is where it gets interesting. During decoding of the whole video file the raw pixel sequences are added to the special buffer which can be used later to copy pixels from. This reminds me somewhat of VPx golden frame which could be updated in some versions with fresh decoded blocks and referenced even many frames later.

And here is why I like to look at those old formats—they sometimes offer an interesting coding approach while modern codecs seem to be made from the standard blocks.

As you all know, this is a studio known primarily for the Harvester game and while that’s not a game I’d like to play I still decided to take a look at it. What do you know, the formats for both music and video are documented. The music format is supported by Game Audio Player since ages and the video format was documented on The Wiki more than a decade ago by the guy responsible for reverse engineering most game formats, Vladimir Gneushev (or simply VAG). And yet there are nuances in them…

Music format is based on IMA ADPCM with minimalistic file header and the default predictor. Which means it needs some time to adapt to the actual coming signal amplitudes. In result the old format description based on reverse engineering recommends skipping first 7-57 bytes right after the header because of the garbage sound it produces. When I looked into the binary reference (an adventure that will be described in an upcoming post), it turned out that in certain cases they simply zero out the first 50 decoded samples.

Video format is rather typical of its time: divide frame into 4×4 blocks, depending on coded flags either skip updating it, read full block data, or read two colours and 16-bit pattern telling how to paint them in a block. No additional compression, even audio is stored as PCM. There is one thing though: the lowest row should not be decoded (and the game does not do it either). It is probably an encoder bug but e.g. for skip frames it codes just bit flags that all should be zero and yet last byte there contains some non-zero garbage. Or in a frame containing changes and not just skip blocks you also have some weird block flags that would signal you need to read block data past the end of the input frame data. This does not matter much since most of the videos in the game have not just 320×200 resolution but also large black borders as well. The only exception is credits.fst which is 640×400. Another fun fact is that I got a North American version and (since the company is North American as well) it has virginlogo.fst with Merit Studios logo (plus some videos mention the company as Future Vision and others as DigiFX Interactive). I know they had changed name in the process and had different European publisher but it’s still fun discrepancy.

Meanwhile I’ve also documented DERF format used in Stupid Invaders game (video format there is based on 8×8 DCT for block coding and motion compensation that involved block division and transforms like flipping the source, which may be in the same frame too). While neither of the codecs is complicated it is still a nice change from looking at the mainstream codecs of today (it’s not that there are many non-mainstream codecs survived to look at either).

Here’s a thing that has been bothering me since I tested my Smacker decoder for NihAV.

It decoded various test files fine except for one game: Toonstruck.
(more…)

Now that (as I believe) I’ve fixed remaining reconstruction bugs in VX decoder, why not do a quick comparison of various video codecs developed by Actimagine and see how they differ (if at all).

There seem to be the following codecs:

• Actimagine (VX)
• Mobiclip (Mods)
• Mobiclip (Moflex for 3DS also there’s a version of it for PC known as Mobiclip HD)

And while they all are based on H.264 with finer block partitioning, there are some differences as well.

Proper structure. The original VX codec used quantiser derived from FPS and all frames were encoded in the same way, while the latter codecs have I-frames and quantisers are transmitted for each frame (as delta for non-keyframes).

References and motion compensation. VX had three previous frames as reference ones, later codecs increased that number to five. VX had fullpel motion compensation, later codecs use halfpel MC.

Data coding. VX relied on Elias Gamma’ codes for all codes except coefficient coding, later codecs use codebooks for most coded values. Also while VX coded residue in 4×4 blocks in H.264 way (starting from the end and with tail of ones coded explicitly), newer codecs use separable transforms and the usual (zero run, coefficient level) coding. Additionally only nine coding modes out of twenty four have survived after VX (intra prediction, MC with motion vectors coded and splits).

Overall, while all those codecs are related, there are large differences between VX and later Mobiclip variants and the only differenced between Mobiclip variants are colourspace (Mods uses YCoCg model, HD uses the proper YUV model), quantiser being clipped to 12-52 range, and block mode codebooks being different.

As I mentioned before, somebody has reverse-engineered decoders for Mobiclip (and a quick check on codebooks used tells me that Mobiclip HD and 3DS versions are the same) so if somebody needs them it should not be that hard to write a decoder.

Sometimes I like to play old strategy games from my youth: Civilization II, Settlers II, WarCraft II and Reunion. You probably have never heard about it since it’s not from some famous studio but from some Hungarians and published by rather obscure publisher too.

The idea is about the same as in Settlers II but IN SPACE! In some near future an experimental spaceship somehow gets into an unknown star system, most of technologies are lost and now you have to colonise planets, fight with aliens and find your way back home. This game combines some planet-building with space exploration and ground battles (there are also battles in space but they’re fought without your involvement). And since it has a story you have events like getting a chance to get some technology or break the alliance between your enemies. So it’s an interesting mix overall and it explains why I still return to it time from time. Sadly the game was programmed in traditional Hungarian manner (remember, Hungarians are responsible for such popular software as Windows 95 or MPlayer) and its intro (a separate program) sometimes crashes and sometimes it even makes DosBox segfault. The main game is also prone to corruptions and crashes (yet I still play it sometimes).

Anyway, today I’ve stumbled upon a page of one guy who reverse-engineered image format used in this game just by fiddling with it. It turned out to be compressed with RLE similar to the one used in PCX (0x00-0xBF – normal pixel, 0xC0-0xFF – run of next byte value 0-63 times). Since the game had some animations as well I decided to look at them.

So intro uses mostly still images split into 640×100 strips (so they can fit into one segment if you remember those) that are scrolled and faded in and out. And there’s a special animation format for some in-game animations similar to the picture format (as expected). Animation file is a series of frames (without palette) that are coded with similar RLE but there are some quirks not encountered in still images. First of all, frames are coded as differences and codes in range 0x80-0xBF are used to signal how many pixels to skip. Second, it turns out that codes 0x80 and 0xC0 are actually escape codes and are followed by 16-bit value of actual skip or run length (and in case of 0xC0 code a pixel value after that). Again, since the format is so simple it could be found just by looking inside the animation files and messing with a decoder.

As for the other games mentioned in the beginning, Civ2 has GIF files mostly hidden inside resource .dlls plus Indeo 4 video (with transparency even!) and Settlers II and WarCraft II have videos in Smacker format.

Having said that, my pointless diversion to looking at game formats is over, back to doing nothing!

So I’ve released my decoder for Actimagine VX and it’s far from perfect.

First problem is audio. While the codec itself it not that tricky (it turned out to be some LPC codec that takes 5-10 16-bit words per frame to code pulses and filter for 128-sample frame), but its data is stored right after video frame data so in order to decode audio first you need to decode video frame and feed the remains of input buffer to the audio decoder. Since I can’t do that in a sane way I could not test the decoder either and it’s there just for the informative purposes only.

The second problem is obviously video. I’ve managed to decode bitstream fine but reconstructed images are not bit-exact and in case of plane prediction this leads to ugly artefacts (essentially the target value wraps around and you have gradients from white to black or vice versa instead of almost flat dark or white regions). I’ve introduced a clipping which seems to help but this is not right and maybe I’ll fix it one day. Maybe even before Bink2.

And finally there are some problems with the demuxer. In theory VX files may have multiple tracks but my demuxer might not handle them at all and if it does then it’ll simply ignore anything but the first video stream.

So VX support is far from perfect but it serves its goal of proving that the format works as expected. And if it’s useful to anybody then it’s even better.