Archive for the ‘Game Video’ Category

Looking at Tsunami games

Sunday, August 1st, 2021

You may remember Tsunami Media as a company founded by ex-Sierra people that released a couple of games and ceased its existence.

MobyGames lists the following titles (characteristic is mine):

  • Ringworld: Revenge of the Patriarch—an adventure game based on Ringworld novel by Larry Niven. ScummVM supports its but I played it long before that.
  • Wacky Funsters! The Geekwad’s Guide to Gaming—a collection of arcade games with wacky design (I’ve only played its sequel though).
  • Protostar: War on the Frontier—a reportedly good strategy game inspired by the same source as Star Control II so they share a lot of game design (and yes, they have a common ancestor so it’s not a rip-off. I’ve never played it myself but what I saw looks interesting.
  • Blue ForcePolice Quest in anything but name. ScummVM supports it but again, I played it long before that.
  • The Geekwad: Games of the Galaxy—another collection of standard arcade games but with wacky design. I especially liked quizzes there.
  • Flash Traffic: City of Angels—one of the first interactive movies. The Mike has blogged about it.
  • Return to Ringworld—a sequel to the Ringworld obviously. ScummVM supports it (so I can re-play it and check whether that empty platform where you seek for the details is really that horrible).
  • Man Enough—FMV dating game.
  • Silent Steel—yet another interactive movie.
  • Free Enterprise—some business simulator.

As you can see, some of the adventure games are supported by ScummVM already but FMV-based ones are not which makes me wonder why.

Man Enough uses the same engine as the previous games (I checked personally. FMV sections there turned out to be animations in the same format as in Ringworld II (I hacked a quick RLB extractor and animation decoder to check that). Side note: whoever added a support for the engine was tired (for a very good reason given below) and hadn’t recognized that it uses LZW compression (so the decompressor is still slightly beautified REd code). Also in this game you have RLBs with elements following each other and those aligned to 16-byte boundaries. So maybe it’d be better just to check if you have first entry right after the library header or not (they all have TMI- header so you can’t mistake it for anything else).

Flash Traffic is special since while it uses the same engine it has all resources stored separately instead of a single library archiving them all. Additionally while TMI format they use is the same, the compression is different. Here you have LZ77-like method which can copy verbatim, copy 32-bit words from already decoded area, fill region with repeating pattern of two bytes, or simply zero region. I have a working implementation for it so I can unpack various resources from the game just fine (but beside BFI or MPEG files that have been supported since long time why should I bother?).

Silent Steel is their newer FMV game that’s mostly just one large MPEG video file and some logic around it (that’s why there was an interactive DVD re-release of the game later).

And now here’s the most important reason why Flash Traffic and Man Enough are not supported up to this day: Tsunami Media hardcoded game logic into the binaries, so while for SCI or SCUMM games you can write a virtual machine and interpret original bytecodes, here you need to decompile game logic from the DOS executable (it’s not an easy task even today) and re-implement it yourself. That’s why tsage engine in ScummVM is full of ${game_name}/${game_name}_scenesN.cpp files for each of the three games it supports. I’m pretty sure the developers won’t refuse somebody else’s contribution for the other tsage games support so you’re welcome to try.

Final news about H4M and Nightlong FMV

Thursday, July 29th, 2021

In my previous post I said I was looking at them and while some details were unknown, I got some understanding.

But it turns out The Multimedia Mike despite what he claims still hasn’t lost the knack for REing codecs—first you need to look if somebody else has done it already. And indeed, somebody else has spent two years on reverse engineering the codec. So all that was left to me was to improve its description it (which I did).

The video compression there is quite curious. Huffman-coded data chunks are not that rare (Truemotion 2 had them, for example). Huffman-coded data chunks where one chunk has tree description coded in the beginning that is used for several other chunks is more interesting. But the transforms was the juiciest part there. It turns out you have several modes of operation: fill block, raw block, smoothed DC block (that one uses DCs from neighbour blocks on all sides to create a smooth transition between them all) and the actual Adaptive Orthogonal Transform that represents block as 1-5 patterns (scaled), selected from single matrix. It reminds me a bit about SVQ1 where you had a block constructed from 2D patterns of varying size. But even more it reminds me of Matching Pursuit experimental video codec from Berkley University that also performed partial transform and selected only some bases from a large set (does anybody remember this codec at all?).

Nightlong FMV is a bit different. The format is simple but FMV2 has packed frames and there was no code for handling it in 68k version of the binary. Nevertheless I could unpack it simply by dumping frame data and extracting it with unar (PowerPacker is a strange compression format BTW, it starts reading data from the end of buffer and outputs data starting from the end as well). The unpacked data turned out to use the same format except that the frame size is now full 640×360 instead of 320×180 trying to look like twice as large. I documented it as well.

P.S. I still have some stuff to look at but more suggestions are always welcome.

REing codecs from exotic platforms with Ghidra

Wednesday, July 28th, 2021

In a recent post I asked for some stuff to RE and I got some (but more pointers are welcome).

Among those two there was H4M format for Gamecube (kudos to The Mike for providing me with the samples) and FMV format (yes, that’s the extension as well) for Nightlong game on Amiga. This finally gave me a chance to look at how Ghidra 10.0.1 fares with such exotic executables.

First I had to install special loaders for the executable formats (and hack their manifests to claim it’s for 10.0.1 and not 10.0) and also ISA support for special PPC extensions for Gamecube PowerPC CPU. After that it was just loading binaries and analysing them. And there’s nothing else to write about it, it just works.

Now some words about the formats. I’ve not fully REd them (and I don’t know how much time it’ll take either, it all depends on weather and mood) but there are some things I can say about them already.

FMV is a fixed 320×180 paletted video with 4×4 tiles. The compression is very simple: you can either have a raw 4×4 tile, sub-divided 4×4 tile where you can have only some pixels in 2×2 block updated, or skip command (for a single block or for rows). There is also fmv2 where each frame seems to be further (or instead?) compressed with PowerPacker (at least that’s how I interpret frame data starting with PP20).

H4M seems to be a rather advanced format. From what I see it has small frames combined in large blocks with each block having a previous block size stored as well (for seeking backwards like FLV, I suppose). Audio codec looks like a typical speech codec on console. Video codec looks interesting: it has B-frames for starters and it’s definitely not a H.264 rip-off. From what I could gather, it codes several data sources in the frame (i.e. each kind of data is stored separately and not interleaved with the rest of data), some of those data sources contain Huffman-packed data, others are used just to read plain bits and bytes (yes, I saw only functions that read single bit or eight bits, nothing else). Then the data from those sources is used to code coefficients and zero runs for the whole planes (I’m not sure yet if it has blocks at all).

I’ve looked just at I-frame decoding, no reconstruction or P/B-frame decoding yet, but what I’ve seen already makes it a very interesting codec to examine. I was asked to look at it many years ago but I declined that request since I had little experience with both PowerPC assembly and the executable format Gamecube uses (the only time when I looked at PowerPC assembly was Fruity Intermediate Codec and even then IIRC I finished REing it only after x86 binary became available). Now thanks to Ghidra with its decompiler and enthusiasts providing support for such fringe formats I can look at such formats too.

On FMV games video compression

Tuesday, July 13th, 2021

Back in the day I’ve stumbled on a post called Archaic Video Compression Algorithms of FMV Games Era (in Russian) giving a concise review of VGA era video. For a long time I wanted to discuss it but I lost the link to it and managed to locate it just recently.

For those unable to read Russian here’s a gist of it: all the variety of those formats used just three methods to compress video, those methods being lossless redundancy removal via RLE or LZ77-based method (Huffman codes not employed), vector quantisation, and block truncation coding. Inter-frame compression was rather primitive, often without any motion predictions; fade-ins/outs were not handled either. And later, when computers became powerful enough, everybody switched to DCT coding and later to in-engine animation.

It is a very good first approximation but as a person who looked into several formats myself and even documented a couple of them, I have something to add or correct.

First of all, ‘archaic’ means something not merely old but also not in common use—and that does not apply neither to LZ77 (which is still the base of most of the practical compressors and archivers; and there’s WebP using its variant in lossless mode) nor vector quantisation (even if it’s mostly employed in texture compression nowadays).

Second, I consider block truncation coding to be a specialised form of vector quantisation for a case of two colours per block. And considering that the input data could be palettised already it’s more likely the encoder searched for the most fitting two colours in the palette instead of generating two RGB triplets and handing it all to the palettisation process.

Third, I’d argue that back in the day codecs did not need sub-pixel motion compensation and motion vectors were often not needed since you could simply code an offset in LZ77 compression. And some of the codecs allowed copying data from either already decoded parts of the current frame or some not yet touched part (or any part in some cases) of the previous frame. This archaic method of copying from already decoded part of the frame is present in AV1 for example (under IntraBC name).

Fourth, I think that recursive block division deserves to be mentioned there.

A minor nit would be that even if very few codecs supported explicit fading, a lot of them had support for palette change event (partial or full). Yet it mostly depended on people developing the format and the encoder for it how well it was handled.

So in my opinion the final list of the techniques most FMV games of the VGA period used should look like this:

  • lossless data compression with RLE or LZ77-based method
  • vector quantisation—both whole tiles and inside single tile;
  • recursive block splitting (either horizontal/vertical or as a quadtree);
  • and fullpel motion compensation usually done by skipping pixels, blocks, or reusing some already decoded region.

And typical video codec of that era should look like one of those two:

  1. line-based codec employing RLE, probably with skips for inter-frames and optional LZ77 compressing of whole frame data afterwards (or extending LZ77 approach to work with the forward data in the buffer as well);
  2. tile-based codec usually working with 4×4 tiles, employing vector quantisation to paint the tile with 1-8 colours, sometimes sub-dividing tile for better detail preservation, sometimes having a dictionary of tile configurations, sometimes even employing block-based motion compensation; those codecs usually employ 1-4 byte opcodes to code the tile type of whole sequences of tiles of the same type.

Of course there are deviations from those two schemes but the majority of game codecs of that era should follow either of those two designs.

Later, as the original article said, those codecs were supplanted by DCT-based ones (I’ve seen the ones based on MJPEG and MPEG-1/2; and there’s hybrid Bink). On the other hand, DCT-based Bink2 is still in use.

Overall, if you look at game formats section in the wiki you’ll see a lot of various formats and many of them are built on a handful of the idea described above while still being quite different from each other. Originality lies not only in the methods employed but also in how you combine them together. And the modern codecs are the example of how different organisations build essentially the same codec using slightly different methods. MPEG-5 LCEVC was a nice change though.

Revisiting legendary Q format

Sunday, May 30th, 2021

Since I had nothing better to do I decided to look again at Q format and try to write a decoder for NihAV while at it.

It turns out there are three versions of the format are known: the one in Death Gate (version 3), the one in Shannara (version 4) and the one in Mission Control (not an adventure game this time; it’s version 5 obviously). Versions 4 and 5 differ only in minor details, the compression is the same. Version 3 uses the same principles but some coding details are different.

The main source of confusion was the fact that you have two context-dependent opcodes, namely 0xF9 and 0xFB. The first one either repeats the previous block several times or reuses motion vector from that block. The second one is even trickier. For version 5 frames and for version 4 with mode 7 signalled it signals a series of blocks with 3-16 colours in each. But for mode 6 it signals a series of blocks with the same type as the previous one but with some parameters changed. If it was preceded by a fill block, these blocks with have fill value. After a block with patterns you have a series of blocks with patterns reusing the same colours as the original block. For motion-compensated block you have the same kind of motion information transmitted.

But the weirdest thing IMO is the interlaced coding in version 5. For some reason (scalability? lower latency?) they decided to code frame in two part, so frame type 9 codes even rows of the frame and frame type 11 codes odd rows—and in this cases rows are four pixels high as it is one block height. That is definitely not something I was expecting.

All in all, the format turned out to be even weirder than I expected it to be.

The Magic of Animation

Saturday, May 1st, 2021

Since I had nothing better to do, I decided to re-play some old adventure games and one of them was King’s Quest VII (I don’t know why it gets Roberta Williams name attached to it, she’s behind all previous KQ games as well, it’s Mask of Eternity and the 2015-2016 re-imagining that deserve Not Roberta Williams game title). And in my usual habit I also looked at intro/ending animations. As you all remember, there are DOS, Mac and Windows releases of the game and each of them uses its own format. Windows version uses 10-fps MS Video 1 in AVI, Mac version uses 8-fps Cinepak in MOV (with data in a separate resource fork as expected from Mac video), DOS version turned out to use 5-fps RBT. Thanks to Mike Melanson documenting it in the course of his experiment, I was able to write a quick and dirty program to unpack .rbt files (essentially it’s just raw frames compressed with LZStac so if you don’t care about handling errors or less common cases then 3.5kB program in C is enough).

And while doing that I remembered that animating of this game was partly done in-house but most of the work was outsourced to the various animation studios including the infamous Animation Magic. In case you forgot that is a studio with Russian origins that was mostly known for their unforgettable animation of CD-i games. Yes, those CD-i games. To their defence it mostly came from them being inexperienced with the computer animation and slowly the animations in their games became better at the expense of them becoming less memorable (animated fairy tales books are interesting only when Dingo Pictures does them!). But between MAH BOI games that are refusing to be forgotten and rather obscure Magic Tales series there were two edutainment first-person shooters, namely I.M. Meen and Chill Manor, that are still somewhat remembered by their wildly imaginative cutscenes. Of course somebody had to look at the format.

It turned out to be a custom format with intra-only RLE-packed video with the only interesting things about it being the use of up to 128 colours only, the fact it should be drawn over some external background (even the intro or ending), and that it uses run length 0 as “fill until the end of current line” mode. Audio is raw PCM, so nothing remarkable there.

For the comparison here’s the captured image from the intro playback (stolen from Mike’s review of the game):

and a decoded frame from intro.ani (not the same one but close enough):

You can find the missing background among the game files in PCX format though.

This is a bit crazy format but it was fun looking at it.

Looking at Q Format

Saturday, April 24th, 2021

For the lack of anything better to do I took a second look at Shannara game from Legend Entertainment (yes, I was that bored). And while it failed to captivate me once again, at least I have discovered yet another video format.

Actually I like old adventure games of theirs, especially the fact that they use RealSound technology (even if it’s just a way to play PCM on PC Speaker). But Shannara is a hybrid game with all the map travelling and fighting monsters. And I could not get into Terry Brooks’s books either, the first Shannara book reminded me of Lord of the Rings but in post-apocalyptic setting with magic appearing for some reason so I dropped it halfway.

In either case, the game featured full-motion animations and of course I had to look at them. As one would expect, all of them could be found in FLICS subdirectory and some of them even were in FLIC format. The rest were sporting rather rare .q extension and I doubted those were Quantum archives. After looking closer it turned out to be quite interesting format.

Video is compressed by splitting frame into 4×4 blocks, usually coding those blocks either as a block filled with two colours using a pattern or by copying some previous block (it does not try to motion search up to a pixel precision but it can reuse any block from a frame). There is an additional coding mode for coding either raw 4×4 block or block filled with 3-8 colours in a pattern. And additionally 128 of the most commonly used patterns for a group of frames are transmitted in a separate chunk before those frames, in result you can use just one byte to code that index instead of two bytes for a full pattern.

Even if I haven’t managed to figure out all details from it and there may be other flavours of it in other games, it was a surprisingly original format and it was fun looking at it.

And another one weird game

Thursday, March 25th, 2021

Last time I forgot to mention another obscure Russian adventure game called either Adventure History or Sin(d)bad’s Eighth Adventure which has quite interesting selection of video in its resources.

So it has some game sprites in FLIC format (most of the sprites are in its own custom format), half of the game size is occupied by clips from Soviet movie (the one featuring the same hero) in AVI with MJPEG format, and finally logo and ending are in the custom HSA format.

HSA is video-only format that is very simple: 32 bits – video width, 32 bits – video height, then you have video frames data prefixed with 32-bit size (0 means end of file) and followed by 768 bytes of VGA palette. And yes, each video frame has palette following it. And of course each frame is compressed independently (using unmodified classic LZSS.C).

At least that’s more variety than you usually get in the games.

A brief look at various game video formats

Wednesday, March 24th, 2021

Today I’d like to sum up my experience looking at various game video formats, none really finished or worth documenting. In case you wonder I’ve not played (or plan to play) either of these games, I just found a place with some adventure games and looked at them for anything not known to me (and below is what’s left after discarding games using Cinepak, Smacker or Truemotion).

First of all, SIFF format used by Beam Software. It turns out that in their game The Dame Was Loaded has various resources in this container format but even larger ones look more like special game resources than videos (think about .RBT vs .VMD in SCI32 games). But there’s Alien Earth with a newer video compression and audio—both 16-bit now. Since I was unable to find the code that plays it, I’ve found the details by studying the files. The compression seems to remain largely the same except that pixels are 16-bit, pattern coding has changed (and the patterns are coded in the binary) and old “palette present” flag is reused to signal that fill values take one byte instead of two (for black values it’s enough).

Now let’s look at various Cyberflix games like Dust: A Tale of the Wired West or Titanic: Adventure out of Time. From what I could find the .mov files found there are similar in the structure with other kinds of resource files. And even if the game explicitly demands 256-colour video mode I could not see anything resembling a palette inside those files. My conclusion is that they’re not real video files either but rather some game scripts and graphics.

And let’s end it with The Vampire Diaries. This is another game with video files without palette. It seems to store just video frames either uncompressed or compressed with LZSS scheme. It’s not interesting enough to dig further.

I guess this is it and I should finally return to writing proper NihAV video player.

A quick look on movies for handhelds

Sunday, March 21st, 2021

In not-exactly-recent news there was a piece about some guy who decided not to listen to the advice of a director of some blockbuster and instead of going to cinema to watch it he encoded it to watch on Game Boy cartridges instead. While people doing stupid things is hardly news, it sparked a mild interest in me so I looked what are the options on underpowered hardware for storing video.

It turned out there are at least three formats for coding not just cutscenes but whole movies (or at least episodes of various series) to fit into 32MB GBA cartridge. And those three formats seem to be built on vector quantisation and they all embed video into the player program (well, the cartridge in this case does not have segments or filesystem for different resources).

  • GBA Video is probably the most famous and the most official one (there were official releases of couple dozens animated movies and cartoon series that used the format). It’s been developed by Majesco and it seems to use vector quantisation and deflate and since it checks codebook size to be 256*6, it’s most likely to be something like Cinepak using 2×2 YUV 420 codebook entries for compression. Additionally it seems to use left prediction (i.e. code pixel as a difference to the left one);
  • Caiman video codec seemed to come in two flavours, the original one coding 8×8 blocks using either four 4×4 pixel codebooks or just one scaled (that reminds me of Cinepak again for some reason, maybe because it did the same albeit using 2×2 vectors), next version of the codec introduced codebooks of different sizes and 8×8 block could be split recursively for that (also that version got motion compensation);
  • METEO is some Japanese format that seems to be the choice for the GBA enthusiasts since there’s a free encoder for it. I actually looked into it to see what it does (it’s a standalone binary about two hundred kilobytes large) and it turns out to decode input videos using standard Windows interfaces and encode frames with Cinepak encoder and write them into their own container.

All these formats make me think that if I look at other gaming consoles I can find Cinepak there as well. Let’s look what those FMV games used

Curiosity satisfied, I should move to something else.