Archive for the ‘Game Video’ Category

The most overengineered game codec

Saturday, September 7th, 2024

…as I’ve seen so far, of course, but the chances for it to keep this title are good.

I’m talking about XCF format (found in ACF files of Time Commando game developed by Adeline Software). For starters, the format allows various commands that are related to the engine (so XCF is used not merely for videos but also for the stage demo records containing e.g. commands to the camera and various other things.

But there’s nothing compared to the video codec design. The frame is split into three parts: fixed-size part with per-block 6-bit opcodes and two variable-length parts. I’d like to say that one contains colour values and another one stores masks and motion vectors, but in reality either kind of data may be read from either source and it’s not consistent even between modes (e.g. for four-colour pattern block both masks and colours are read from part 1 while for two-colour pattern block colours are read from part 2; it’s like they designed it on whatever registers were available in the decoding function at the moment). And for some reason they did not went further and left those frame parts uncompressed.

The real monstrosity though is the opcodes. As I’ve mentioned, they take six bits which means 64 different opcodes. And unlike many other codecs where you’d have half a dozen operations (copy from previous, fill with 1/2/16 colours and such) plus run count, here you have way more. Also it uses 8×8 block size which helps adding new modes. Here’s an abridged list:

  • raw block;
  • skip block;
  • motion block with one-byte motion vector (using nibbles for vector components), two-byte absolute offset, two-byte relative offset or two-byte motion vector;
  • motion block with each quarter using the same four modes;
  • single value fill block—and another mode for filling block quarters;
  • 2/4/8/16-colour pattern block;
  • 2/4/8-colour pattern block quarters;
  • special 4-colour pattern subblock coding mode where you can pick only one alternative option and only for half of the pixels;
  • block filled with mostly single colour except where mask tells to read a new value;
  • block with values coded as nibble-size differences to the base colour;
  • block coded as two base colours (high nibble only) plus low nibble of the current value;
  • block using RLE plus one of four scan modes (line scan, top-down scan, zigzag and inverted zigzag);
  • block using the same scan modes but sending only nibbles for the colour values (first nibble is for base colour high nibble, others are used to replace its low nibble).

And that’s not all, some modes (e.g. motion or fill) have refinement mode. E.g. opcodes 1-4 all mean skip block, but while opcode 1 means copy block from the same place in the previous frame and do nothing, opcode 2 means doing that and then replacing four pixels at arbitrary positions with new values, opcode 3 means the same but with eight pixels, and opcode 4 means using a mask telling which pixels should be replaced. It works the same way for other opcodes supporting it—first do the operation, then optionally update some of the pixels.

If you thought that peculiarities end there, you’d be wrong. Motion vectors are a displacement from the centre of the block instead of its top left corner as in any other video codec. And looks like while the format stored video dimensions, the decoder will work with any width as long as it’s 320 pixels.

I don’t envy future me who has to document it for The Wiki.

P.S. And what’s ironic is that the game was released between Little Big Adventure and its sequel, and while the former used custom format with FLI compression methods (and commands to play external audio), the latter switched to Smacker—a much simpler codec but apparently more effective.

P.P.S. After I reverse engineer a codec or two from games published by Psygnosis I should publish a new version of na_game_tool and switch to actually documenting all the formats it supports; and then do something completely different. At least the end is near.

SMS is a new AVI

Wednesday, August 28th, 2024

Since I’m still in a search of video formats for my na_game_tool, I happened to look at Jagged Alliance: Deadly Games and found an SMS file there (and then more for a demo version of Nemesis: The Wizardry Adventure).

The file starts with BLAH magic followed by two sizes, some table and audio data. And after audio data there’s a suspiciously familiar SMK2 signature. And indeed, the file from that point plays fine as Smacker video.

Of course this reminded me of a game Ripper, released in the same 1996 apparently (but by a different developer and publisher), which had AVI format that encapsulated Smacker of FLIC along with an audio track external to the format.

Why such combination needed to exist? I feel that the main actor in Ripper expressed the answer to that the best. After all, Smacker supports up to seven audio tracks and other games of that period played Smacker video and audio just fine; it should allow seeking even if nothing seems to have ever used that feature. Maybe it really needed more cowbell.

Looking at Nova Logic FMV formats

Friday, August 23rd, 2024

So there’s this company known mostly for helicopter simulators and those games use KDV format for animation. That alone is enough to make me look at the formats and what have I found?

  • the original Comanche game used a simple FMV based on PCX RLE. It was enough to just look at the file with a hex viewer to figure out all of it;
  • Armored Fist used KDV file with a different format which I still figured out after looking at it with a hex viewer and correcting my mistakes after image reconstruction somewhat started to work. The main lucky guess was that it uses 2-bit block modes for 4×4 blocks; then figuring out that block modes use 1/2/4/16-colour mode with patterns;
  • Werewolf vs. Comanche has almost the same format but for some reason Armored Fist reads only 3 colours for 4-color block (first colour is explicitly zero) so this one was more natural to RE first;
  • Comanche Gold tweaked format again, with the main change being that mode 3 means skip/fill/paint new block in the same way as the previous block, using a new pattern (for patterned modes) but keeping old colours. For this and the following format I actually had to look into the binary specification;
  • and finally Ukrainian planes simulators (F-16/MiG-29) use 24-bit version of the format above (adding cached colours, more about it below).

The main principle remained the same for all but the first KDV format: split picture into 4×4 blocks, use two-bit mode for each block, pack four modes into one byte followed by block mode data, modes being skip, fill, 2-colour pattern mode, 4-colour pattern mode and raw, colour 0 being used for the pixels that should be left unchanged (or to signal the special mode extension). The details changed though: in first two flavours raw mode was signalled by “fill with 0” mode, in last two flavours it’s “2-colour pattern with the same/zero colours” signals raw block mode and “fill with 0” means skip. An interesting feature of 24-bit format is that it introduced colour cache: if the first colour byte has top bit set, it is really an index in the cache (low 7 bits of that byte are), otherwise it is the first byte of 24-bit colour value which should be also added to the cache (it is organised as a circular buffer, so 128th value replaces 0th, 129th value replaces 1st and so on). And it still uses 0,0,0 as “leave unchanged” value (or a special flag for raw mode when 2-colour pattern uses two zero triplets).

I’ve added support for them all to na_game_tool (a couple more formats and I can make a 0.2 release before switching to a completely different activity) and I want to say some words about format detection.

As mentioned above, the main problem with the format is distinguishing the flavours. It’s very simple for the last two—Comanche Gold uses larger header and 24-bit format uses larger header, lacks palette chunks and stores frame data in K24i chunks (instead of KDVi chunks as the rest). The problem is to distinguish the first two formats without resorting to an ugly hack by trying to decode it both ways and seeing which one sticks. That’s where a new feature of na_game_tool comes into play—detecting format by regular expression. Since I know that most Werewolf vs. Comanche videos start with prefix “w_” I could simply add a rule for it, just like this:

    DetectConditions {
        demux_name: "kdv-wc",
        extensions: ".kdv",
        regex:      "bumper.kdv,w_*.kdv",
        conditions: &[CheckItem{offs: 0, cond: &CC::Str(b"xVDK")},
                      CheckItem{offs: 4, cond: &CC::Eq(Arg::U32LE(0x14))}]
    },

So files with this extension, regex and magic will be detected as this flavour of KDV while files not matching it (but with the same extension and header) will be classified as files from Armored Fist. It’s not something useful for a general multimedia converter but it’s useful for the game files—another case is CRH format that has two flavours for two different games (differing by the hardcoded height and number of bits for image offset) where you also have to rely on the file names to distinguish which one is which.

Of course I have some other changes for the upcoming na_game_tool release (maybe in September, provided I can find enough formats to add by that time) but they’re equally unexciting. Keep staying out of the loop.

Watery format revisited

Saturday, August 10th, 2024

I’m still picking unfinished and half-finished format for na_game_tool and there were some from Access games. While supporting animations from Amazon: Guardians of Eden is rather pointless (since they simply update pixels on some scene and do not have own palette), supporting H2O and PTF formats is more interesting.

I wrote about them back in the day and about H2O specifically. And since it was a quick look and my eyes are not that good, I missed some details. But as I finally implemented a decoder for it, now I can correct that story.

As it turned out, H2O stores video data as Huffman-coded 16-bit values instead of usual bytes. And those values are used for both opcodes and colours. If top four bits of opcode are 0-3 then the top byte codes number of 4×4 blocks to process and low byte is fill value (0 means skip block instead). If top four bits of opcode are 4, then low 12 bits are number of blocks to process minus one and each block data consists of 16-bit pattern and two colours (packed as another 16-bit value). Top four bits of opcode being 5 mean partial update for the specified number of blocks, each block having 16-bit update mask and 0-16 colours to update (packed in 16-bit words).

So while the format is no ground-breaker, it still turned out to be more interesting than I expected it to be.

P.S. I guess I’ll do a bunch of FLI-based formats next. There are some curious variations of it like FLI with audio or RNC ProPack compression.

Implementing Jack Orlando animation support

Monday, August 5th, 2024

As I’m still working on my game tool, I decided to implement video format from the Jack Orlando game. So now na_game_tool has AVX support and I’d like to say something about the format.

The format is a rather weird mix of different things. It starts with a fixed-size audio block and that size depends on whether it’s 8-bit PCM or IMA ADPCM (with stereo samples packed into interleaved 32-bit words), it seems to have block structure where an arbitrary chunk of data is stored as 32-bit size plus block data which includes video data with accidental headers. Yes, the file internally may have several FLX animations with different framerates too, so e.g. a 16-fps piece of animation may be followed by 10- or 24-fps animation. And audio stream is global even if it’s stored in small chunks inside FLX (since you have several seconds of pre-roll audio in the beginning). Oh, and palette data may be stored as RGB565 or RGB24 (which you’re supposed to understand from palette chunk size). Video compression is simple but somewhat weird too, as RLE is used to optionally compress data before it gets decoded.

In either case weird does not mean bad and in my line of hobby an interesting codec is worth more than a popular one.

P.S. And there’s even weirder format known as Knowledge Adventure MOV. I gave up on REing it but apparently video there is split into tiles (4×2 or 2×1 pixels apparently) and frame data is those tile indices that may be stored uncompressed, RLE compressed or, apparently, using an arithmetic coder with static frequencies stored in the file header. The difficulty comes from self-modifying code and not the cute kind that simply modifies couple of instruction constant arguments but one that substitutes function call address with a pointer to a special memory buffer allocated in a context and filled elsewhere. In theory it can be figured out without much hassle with, say, a built-in DosBox debugger but I’m not inclined to do that (at least not now).

Re-revisiting Actimagine VX

Friday, August 2nd, 2024

As Paul said, I probably should do something more useful (and he should do something better than still messing with jbmpeg fork) but we’re all better at giving good advises than at following them.

Recently I was contacted by a guy who asked for my help on dealing with VX format as he needed video part extracted from it and there were no tools available. And he also provided some samples that helped clarify things I got previously wrong, for example the seek table and framerate field as well as stereo audio support.

Also since I bothered to extract video part (the only way to determine its length is to parse it, so now I have a stripped-down version of video decoder doing that) I decided to finally do something about the audio part. So after fixing some things (e.g. missing filter reconstruction or stereo support) now my decoder can decode VX files with recognizable audio (not perfect but who can do better is encouraged to do so). The sad thing is that I could not use my old setup (a console emulator with GDB stub) for debugging so I’m not likely to ever improve it.

At least it was a fun diversion and helped to improve the format knowledge a bit.

P.S. Meanwhile I’m still working on adding new formats for na_game_tool, hopefully I’ll have something to write about another game format soon.

Revisiting Talisman animation format

Sunday, July 28th, 2024

Back in the day I wrote a bit about this format and since it’s rather interesting I decided to implement its support in my game tool.

Eventually I got it working except on two files that seem to have certain frame data unpacked incorrectly (it’s the feature that makes this format interesting in the first place—per-group order-1 Huffman coding). And for every file decoding seems to go past the data boundaries at the end so I insert some additional zeroes at the end of packed data to make it work (another fun fact – apparently last eight bytes of both opcode and colour parts of inter frame seems to be always unused). And it seems audio data is always coded as silence. The actual cutscenes do not make much sense either as it’s usually a static cartoon image superimposed over a 3D rendering of some lady in a tower-like structure (the game does not deserve any associations with Sierra titles so I’m not using other words to describe it). Sometimes her lips are moving (if you can notice it behind the other image), quite often they do not:

All in all, this is a rather weird format for a seemingly weird game but I don’t regret wasting time on implementing a decoder for it.

NihAV Game Tool: the official release

Sunday, July 21st, 2024

I’m finally proud not too ashamed to present a side project I’ve been wasting my time on.

The rationale behind it is simple: I sometimes write throwaway decoders in order to check if I understood format properly or if I really want to see the decoded content. Usually it’s written in C with the same code (usually for dumping output as PPM image sequence or reading 16-bit value) copied over and over again. So I thought to borrow bits from NihAV and finally make a framework for handling output creation and various utilities for handling input (e.g. reading integers of different size and endianness). It’s still better than doing nothing and it may be marginally useful to somebody else.

Not all of the included decoders are completely my own work, some come from ScummVM via the documentation I created for them on The Wiki (but sometimes with improvements, for example they don’t handle compressed sections in PMV) and AV format comes from the Lord of the Rings engine re-implementation (again, via The Wiki).

I should also mention a nasty surprise. Apparently when AVI streams are sufficiently de-synchronised (e.g. the format pre-buffers a second of audio before sending any video frames) libavformat AVI demuxer (which is used by a lot of multimedia tools nowadays) switches to a different mode and treats palette change chunks as normal video data (which works wonders of a stream of course; and apparently nobody had this problem since the time I introduced palette change support there about twenty years ago; also my own player is unaffected). So for such formats I had to introduce manual audio buffering (it’s not nice but the alternatives are worse).

And some words about the releases and release schedule. I don’t want to bother my friend who hosts the public NihAV repositories to add another one and I want to get involved with the usual platforms even less. As the result I’ll simply dump source tarballs with a brief changelog on the site. Releases should happen irregularly, when I accumulate, say, another dozen of formats or have other features implemented (like detecting format by regex instead of just by extension or OpenDML AVI support for being able to output annoyingly large files).

But for now the source code along with some formal Git history is available at a NihAV special page. Grab it while it’s not that stale.

A side project for NihAV

Sunday, July 7th, 2024

Since I still have nothing much to do (and messing with browsers is a bad idea unless the situation is desperate), I decided to make a NihAV-lite project. So announcing na_game_tool.

This is going to be a simple tool to convert various game and image formats (and related) into image sequence, WAV or raw AVI (which then can be played or processed with anything conventional). I’ve begun work on it already but the release will happen when at least when I implement all planned features (which is writing image sequence in BMP format, AVI output and porting two dozen of half-baked decoders I wrote to test if I understood the format).

Why a new project? Because I have nothing better to do, it still may be marginally useful for somebody (e.g. me) and I can do some stuff not fitting into NihAV (for example, decode 3DO version of TrueMotion video split into four files) and I don’t have to bother about other stuff that fits demuxer-decoder paradigm poorly and requires inventing ways to convey format-specific information from the demuxer to the decoder. In my case I simply feed the input name to the input plugin and it returns frames of decoded audio or video data. Some hypothetical Antons might ask a question how to deal with the formats that use variable delay in milliseconds between frames instead (and I’ve implemented one such format already). To which I can reply that one can fit a square peg in a round hole in two ways—by taking a larger hole or by applying more force. The former corresponds to having, say, fixed 1000fps rate and send mostly the same frames just to have constant rate; the latter is forcing a constant framerate and hoping for the best. I chose the latter.

The design is rather simple: there’s a list of input plugins and output plugins. Input plugin takes input name, opens whatever files it needs, outputs information about the streams and then decoded data. Output plugin takes input name, creates whatever files it needs, accepts stream information and then receives and writes frames.

Probably there’s a better alternative with librempeg but you’d better go read about it on Paul’s blog.

REing non-Duck VP X1

Thursday, June 13th, 2024

While I’m still looking for a solution on encoding video files with large differences with TrueMotion, I distract myself with other things.

Occasionally I look at dexvert unsupported formats to see if there’s any new discovery documented there in video formats. This time it was something called VPX1.

I managed to locate the sample files (multi-megabytes ones starting with “VPX1 video interflow packing exalter video/audio codec written by…” so there’s no doubt about it) and an accompanying program for playing them (fittingly named encode.exe). The executable turned out to be rather unusable since it invokes DPMI to switch to 32-bit mode and I could not make Ghidra decompile parts of the file in 386 assembly instead of 16-bit one (and I did not want to bother to decompile it as a raw binary either). Luckily the format was easy to figure out even without the binary specification.

Essentially the format is plain chunk format complicated by the fact that half of the chunks do not have size field (for palette chunk it’s always 768 bytes, for tile type chunk it’s width*height/128 bytes). The header seems to contain video dimensions (always 320×240?), FPS and audio sampling rate. Then various chunks follow: COLS (palette), SOUN (PCM audio), CODE (tile types) and VIDE (tile colours). Since CODE is always followed by VIDE chunk and there seem to be a correlation between the number of non-zero entries in the former and the size of the latter, I decided that it’s most likely a tile map and colours for it—and it turned out to be so.

Initially I thought it was a simple bit map (600 bytes for 320×240 image can describe a bit map for 4×4 tiles) but there was no correlation between the number of bits set and bytes in tile colours chunk. I looked harder at the tile types and noticed that it forms a sane 20×30 picture so it must be 16×8 tiles. After some more studying the data I noticed that nibbles make more sense, and indeed only nibbles 0, 1, 2 and 4 were encountered in the tile types. So it’s most likely 8×8 tiles. After gathering statistics on nibbles and comparing it to tile colours chunk size I concluded that type 2 corresponds to 32 colours, type 4 corresponds to 1 colour and type 1 corresponds to 16 colours. Then it was easy to presume that type 4 is single-colour tile, type 1 is downscaled tile and type 2 is a tile type with doubling in one dimension. It turned out that type 2 tile repeats each pixel twice and also uses interlacing (probably so video can be decoded downscaled on really slow machines). And that was it.

Overall, it is a simple format but it’s somewhat curious too.

P.S. There’s also DLT format in the same game which has similarly lengthy text header, some table (probably with line offsets for the next image start) and paletted data in copy/skip format (palette is not present in the file). It’s 16-bit number of 32-bit words to skip/zero followed by 16-bit number of 32-bit words to copy followed by the 32-bits to be copied, repeat until the end. Width is presumed to be 640 pixels.

P.P.S. I wonder if it deserves a support via stand-alone library named libvpx1 or libvpx and if this name is acceptable for Linux distributions.