Kostya's Boring Codec World

As hinted in my previous posts, I’ve released na_game_tool version 0.2.0. Of course it is available at its dedicated page (and now there’s a link to that page in the “Multimedia Projects” link category). Here I’d like to talk about what’s changed in it and in NihAV.

Probably the most important part is a support of a dozen new video formats—that’s the main task the tool should do after all. I’m not going to duplicate the changelog already presented at its page so those zero people who care may look there. But in the same time it’s worth nothing that I’ve move support for three obscure formats from NihAV (i.e. adapted the code from there and removed it from nihav-game crate). After all, I added support for some game formats there in the first place because I had no other place for that but I did that mainly to test whether I understood their workings right. Now as na_game_tool exists it’s much more appropriate place for such experiments. Maybe I’ll move more formats in the future but not those I care about (like Smacker or VMD) and I might still add new ones like game-specific AVI codec (like IPMA or KMVC).

Now I want to talk about less important technical details. One of the improvements I’m proud about is adding a better detection using regular expression on filename. It is described in a post about Nova Logic FMV formats and it solves a rather annoying problem of certain formats having different flavours that are impossible to detect from the file contents but the naming helps distinguish those.

Also I’ve fixed all spotted bugs in AVI output and added OpenDML support for writing large AVIs. This was needed because some Arxel Tribe games have 24-bit videos in 800×450 resolution and over a minute long so decoded raw AVIs exceed 2GB in certain cases (and intro and advertisement videos from Ur-Quan Masters approached 1GB limit, ending results in ~1.4GB AVI). Additionally I’ve added BMP format for image sequence writer (I don’t think anybody will use it but maybe it’ll come in handy one day for analysing frame palettes).

Having said that, I put the project on pause. As mentioned in the post dedicated to the first release of the tool, my intent is to make releases when it accumulates support for at least a dozen new formats (the ones moved from NihAV do not really count)—and I don’t expect to encounter enough games with undiscovered video formats in the near future. Not that I’d refuse to work on it if it’s not too annoying (see my previous post on Psygnosis games for an example of those) but it may take a year or more to collect enough formats (plus some time to reverse engineer them), so I’d rather concentrate on other things like documenting already implemented formats.

So there’s this company known mostly for helicopter simulators and those games use KDV format for animation. That alone is enough to make me look at the formats and what have I found?

• the original Comanche game used a simple FMV based on PCX RLE. It was enough to just look at the file with a hex viewer to figure out all of it;
• Armored Fist used KDV file with a different format which I still figured out after looking at it with a hex viewer and correcting my mistakes after image reconstruction somewhat started to work. The main lucky guess was that it uses 2-bit block modes for 4×4 blocks; then figuring out that block modes use 1/2/4/16-colour mode with patterns;
• Werewolf vs. Comanche has almost the same format but for some reason Armored Fist reads only 3 colours for 4-color block (first colour is explicitly zero) so this one was more natural to RE first;
• Comanche Gold tweaked format again, with the main change being that mode 3 means skip/fill/paint new block in the same way as the previous block, using a new pattern (for patterned modes) but keeping old colours. For this and the following format I actually had to look into the binary specification;
• and finally Ukrainian planes simulators (F-16/MiG-29) use 24-bit version of the format above (adding cached colours, more about it below).

The main principle remained the same for all but the first KDV format: split picture into 4×4 blocks, use two-bit mode for each block, pack four modes into one byte followed by block mode data, modes being skip, fill, 2-colour pattern mode, 4-colour pattern mode and raw, colour 0 being used for the pixels that should be left unchanged (or to signal the special mode extension). The details changed though: in first two flavours raw mode was signalled by “fill with 0” mode, in last two flavours it’s “2-colour pattern with the same/zero colours” signals raw block mode and “fill with 0” means skip. An interesting feature of 24-bit format is that it introduced colour cache: if the first colour byte has top bit set, it is really an index in the cache (low 7 bits of that byte are), otherwise it is the first byte of 24-bit colour value which should be also added to the cache (it is organised as a circular buffer, so 128th value replaces 0th, 129th value replaces 1st and so on). And it still uses 0,0,0 as “leave unchanged” value (or a special flag for raw mode when 2-colour pattern uses two zero triplets).

I’ve added support for them all to na_game_tool (a couple more formats and I can make a 0.2 release before switching to a completely different activity) and I want to say some words about format detection.

As mentioned above, the main problem with the format is distinguishing the flavours. It’s very simple for the last two—Comanche Gold uses larger header and 24-bit format uses larger header, lacks palette chunks and stores frame data in K24i chunks (instead of KDVi chunks as the rest). The problem is to distinguish the first two formats without resorting to an ugly hack by trying to decode it both ways and seeing which one sticks. That’s where a new feature of na_game_tool comes into play—detecting format by regular expression. Since I know that most Werewolf vs. Comanche videos start with prefix “w_” I could simply add a rule for it, just like this:

    DetectConditions {
        demux_name: "kdv-wc",
        extensions: ".kdv",
        regex:      "bumper.kdv,w_*.kdv",
        conditions: &[CheckItem{offs: 0, cond: &CC::Str(b"xVDK")},
                      CheckItem{offs: 4, cond: &CC::Eq(Arg::U32LE(0x14))}]
    },

So files with this extension, regex and magic will be detected as this flavour of KDV while files not matching it (but with the same extension and header) will be classified as files from Armored Fist. It’s not something useful for a general multimedia converter but it’s useful for the game files—another case is CRH format that has two flavours for two different games (differing by the hardcoded height and number of bits for image offset) where you also have to rely on the file names to distinguish which one is which.

Of course I have some other changes for the upcoming na_game_tool release (maybe in September, provided I can find enough formats to add by that time) but they’re equally unexciting. Keep staying out of the loop.

As Paul said, I probably should do something more useful (and he should do something better than still messing with jbmpeg fork) but we’re all better at giving good advises than at following them.

Recently I was contacted by a guy who asked for my help on dealing with VX format as he needed video part extracted from it and there were no tools available. And he also provided some samples that helped clarify things I got previously wrong, for example the seek table and framerate field as well as stereo audio support.

Also since I bothered to extract video part (the only way to determine its length is to parse it, so now I have a stripped-down version of video decoder doing that) I decided to finally do something about the audio part. So after fixing some things (e.g. missing filter reconstruction or stereo support) now my decoder can decode VX files with recognizable audio (not perfect but who can do better is encouraged to do so). The sad thing is that I could not use my old setup (a console emulator with GDB stub) for debugging so I’m not likely to ever improve it.

At least it was a fun diversion and helped to improve the format knowledge a bit.

P.S. Meanwhile I’m still working on adding new formats for na_game_tool, hopefully I’ll have something to write about another game format soon.

I’m finally proud not too ashamed to present a side project I’ve been wasting my time on.

The rationale behind it is simple: I sometimes write throwaway decoders in order to check if I understood format properly or if I really want to see the decoded content. Usually it’s written in C with the same code (usually for dumping output as PPM image sequence or reading 16-bit value) copied over and over again. So I thought to borrow bits from NihAV and finally make a framework for handling output creation and various utilities for handling input (e.g. reading integers of different size and endianness). It’s still better than doing nothing and it may be marginally useful to somebody else.

Not all of the included decoders are completely my own work, some come from ScummVM via the documentation I created for them on The Wiki (but sometimes with improvements, for example they don’t handle compressed sections in PMV) and AV format comes from the Lord of the Rings engine re-implementation (again, via The Wiki).

I should also mention a nasty surprise. Apparently when AVI streams are sufficiently de-synchronised (e.g. the format pre-buffers a second of audio before sending any video frames) libavformat AVI demuxer (which is used by a lot of multimedia tools nowadays) switches to a different mode and treats palette change chunks as normal video data (which works wonders of a stream of course; and apparently nobody had this problem since the time I introduced palette change support there about twenty years ago; also my own player is unaffected). So for such formats I had to introduce manual audio buffering (it’s not nice but the alternatives are worse).

And some words about the releases and release schedule. I don’t want to bother my friend who hosts the public NihAV repositories to add another one and I want to get involved with the usual platforms even less. As the result I’ll simply dump source tarballs with a brief changelog on the site. Releases should happen irregularly, when I accumulate, say, another dozen of formats or have other features implemented (like detecting format by regex instead of just by extension or OpenDML AVI support for being able to output annoyingly large files).

But for now the source code along with some formal Git history is available at a NihAV special page. Grab it while it’s not that stale.

Since I still have nothing much to do (and messing with browsers is a bad idea unless the situation is desperate), I decided to make a NihAV-lite project. So announcing na_game_tool.

This is going to be a simple tool to convert various game and image formats (and related) into image sequence, WAV or raw AVI (which then can be played or processed with anything conventional). I’ve begun work on it already but the release will happen when at least when I implement all planned features (which is writing image sequence in BMP format, AVI output and porting two dozen of half-baked decoders I wrote to test if I understood the format).

Why a new project? Because I have nothing better to do, it still may be marginally useful for somebody (e.g. me) and I can do some stuff not fitting into NihAV (for example, decode 3DO version of TrueMotion video split into four files) and I don’t have to bother about other stuff that fits demuxer-decoder paradigm poorly and requires inventing ways to convey format-specific information from the demuxer to the decoder. In my case I simply feed the input name to the input plugin and it returns frames of decoded audio or video data. Some hypothetical Antons might ask a question how to deal with the formats that use variable delay in milliseconds between frames instead (and I’ve implemented one such format already). To which I can reply that one can fit a square peg in a round hole in two ways—by taking a larger hole or by applying more force. The former corresponds to having, say, fixed 1000fps rate and send mostly the same frames just to have constant rate; the latter is forcing a constant framerate and hoping for the best. I chose the latter.

The design is rather simple: there’s a list of input plugins and output plugins. Input plugin takes input name, opens whatever files it needs, outputs information about the streams and then decoded data. Output plugin takes input name, creates whatever files it needs, accepts stream information and then receives and writes frames.

Probably there’s a better alternative with librempeg but you’d better go read about it on Paul’s blog.