Kostya's Boring Codec World

Each room has several formats associated with it, some of them are binary, describing various objects but here I’ll describe the following formats:

• IMG
• NOD
• ZZZ
• FTR
• ROM
• GRA

GRA format is a more general format used for animated sprites, some window backgrounds and GUI elements as well as talking character portraits.

NOD

This has nothing to do with C&C is a palette format for the room backgrounds. It starts with QFG5 magic and 32-bit file size but in reality the game engine simply reads 1024-byte RGBA palette starting at the offset 0xA8 and that’s it.

IMG

This is actual room background data. It starts with a header consisting of two 32-bit big-endian words and two 64-bit big-endian floating-point numbers and then the same header repeated in little-endian format. The parameters are height, width, two parameters probably related to the room positioning (probably an offset and full intended room perimeter—the rooms can be circular like the Silmaria main square) and two unknown floating-point numbers.

Then RLE-compressed image follows coded in columns starting from the right edge and with its height doubled (so e.g. 4000×400 image will be decoded as 800×4000 image with the image left edge being at the bottom). The lower part of image is not used (maybe a leftover when it was used for a depth buffer). The actual data is 8-bit indices to the NOD palette with the same resource number.

RLE works in the following way: for each decoded line (or rather column) a signed 8-bit value is read; zero signals the end of line, negative values signal raw data (e.g. 0xFE or -2 means copy two following bytes to the output), positive values mean repeating the next byte the specified number of times.

ZZZ

This is a room depth map. This file contains RLE-compressed depth values (0 = closest to the screen, 255 = farthest) without any header. Data is compressed in the same way as single line of IMG format and has the same dimensions but it is stored in line-by-line format right edge first (i.e. mirrored compared to the background image);

FTR

This is a format defining room regions. It consists of signed 16-bit numbers. First number declares the number of regions in the file, then region data follows consisting of 4-word header (region depth maybe, always zero, an exit portal flag, and the number of points) and the region points (two 16-bit words each).

And here is an example of decoded room images.

Arcane Island background

Arcane Island depth map (and an Easter egg)

Arcane Island region map

ROM

This is a room properties file consisting of two integers and two floats: first field (32-bit integer) seems to be a big-endian version of the third field, second field (32-bit float) is unknown, third field (32-bit integer) declares the number of additional resources that should be loaded for the room (e.g. battle arena has three alternative views), fourth field (32-bit float) seems to specify an angle increase (for circular rooms, I suppose). In either case floating-point numbers seems to be non-zero only for the room 200 (Silmaria main square).

GRA

This is a format used for animated sprites and static images in the various parts of the game.

The file starts with 32-bit image coding mode, 32-bit number of sprite collections, 512-byte palette (in RGB555 format) followed by the 32-bit offsets to the sprite collection data,

Each sprite collection starts with the header containing the following 32-bit values: horizontal sprite position, vertical sprite position, width, height, number of sprites in the collection, delay between frames and flags. It is followed by the offsets (from the sprite collection data start) to the individual frames. Depending on image coding mode frames can be stored in the following way:

1. mode 0—unpacked image data;
2. mode 1—unpacked image data interleaved with depth values (i.e. in the following order: palette index byte, depth byte, palette index byte, depth byte and so on);
3. mode 2—the same RLE compression as in IMG format;
4. mode 3—the same as the previous mode but index 0xFF is used for the transparent pixel;
5. mode 4—similar to the previous mode but value 0xFF signals that the original background pixel should be restored instead.

And here’s an example of a sprite from the same scene (in the same orientation as actual scene background):

I’ll probably try to cover the messages and speech next (font formats, message files and lipsync) but it may take more time. And then only 3D data will be left for figuring out. It’s a pity I don’t know much about 3D though…

There are sixteen formats known:

• 0 – MDL (model format);
• 1 – ANM (model animation);
• 2 – ROM (room parameters, always two integers and floats);
• 3 – NOD (room background palette);
• 4 – IMG (room background image);
• 5 – ZZZ (depth buffer for room background);
• 6 – GRA (sprites);
• 7 – QGM (message format);
• 8 – FTR (some room data);
• 9 – WAV (effects and music);
• 10 – RGD (region data);
• 11 – MOV (intro and cutscenes);
• 12 – QGF (font files);
• 13 – STR (some room data);
• 14 – AUD (speech audio, still in WAV format);
• 15 – SNC (lipsync data to accompany speech).

I’ll try to document all these formats (except for AUD, MOV and WAV).

SPK is the archive format used for storing most of the game resources. There are four known archives: CDA.SPK which seems to contain speech (and lip synchronisation data), CDN.SPK probably with scene-specific data, HDN.SPK contains game music plus world map and some other files, HDNW.SPK contains mostly 3D models and animations (plus some QGF files).

Read the rest of this entry »

So while I have no real breakthroughs, here is some information about the engine in general.

Quest for Glory V seems to be a 2.5D engine (some 3D objects interacting essentially in circular rooms using static background images) rendering output in 15-bit RGB (while using floats internally and paletted assets). The engine logic is hardcoded, partly inside the executable, partly inside dynamically loaded room modules (in Windows or Mac native format). From what I’ve already seen, there are several global objects responsible for various parts of game engine, including the main engine class with around 160 callable methods; room class takes a pointer to it and provides around ten methods that can be invoked by the engine (and which in their turn may invoke engine methods for various actions). So it’s about a megabyte of engine code and over four megabytes of code in room modules. At least I don’t need to decompile them all at once.

The game data is organised in stand-alone files and SPK archives. Cutscenes are Cinepak in MOV (though IIRC my official pirate copy re-compressed them to use SVQ1 instead), audio is MS ADPCM in WAV. Overall there are fifteen resource types known (audio, lipsync data, text messages, panorama background and its palette, 3D models and its textures, GUI decals and so on). Most of the files are contained in SPK archives which are essentially slightly hacked ZIP archives with initial header replaces with custom index per resource type and each PK entry has numbers changed so it won’t be detected as a ZIP archive without header (at least there’s no compression employed as far as I can tell).

So overall I just need to discover how the main engine loop works, what are all those file formats and reconstruct the room logic. That should take a long time (supposing I don’t give up earlier). Either way I’m doing it mostly to pass time and to find out how far I can get REing something more complex than a codec.

As I mentioned previously, NihAV is feature complete so while I may still return to improve it, it is not likely that there will be much work to to be done there. That is why I decided to try my hoof at something different—trying to reverse engineer (and maybe re-create) a game engine. I’ve chosen Quest for Glory V for this task as I still have interest in that game and fine folks from ScummVM are not likely to work on it (as it’s a hardcoded engine, more about it later). I fully understand that I may fail as my knowledge about game engines is about zero and having about five megabytes of code to decompile may be too much so I may give up out of boredom as well. In either case it’s not a big loss.

I’m not sure if I’ll ever release the final code (if there is any final code) but at least I will try to document the formats and inner engine working for the posterity. Otherwise it will be about as bad as with the modding community: I’ve seen people making better 3D models for the game and even patching the game logic to do but there is almost no public information about the formats (except for the trivial ones) let alone about something more serious. I know Mike suggested Xentax Wiki once as a better place for such work but I can’t find it so I’ll just keep blogging here.