Archive for December, 2023

QfG5: leftover formats

Saturday, December 23rd, 2023

Looks like I’ve not said much about the three formats used by the game yet: ANM, RGD and STR. So I guess it’s time to rectify that.

ANM

Apparently there is only one animation format despite file magic being either 8XOV or MIRT. It starts with 4-byte magic, 32-bit header size (always 36 bytes), 16-bit animation name, 32-bit number of animations in the file, 32-bit number of animation blocks in each animation and 32-bit animation delay (usually 33 or 66 milliseconds).

Each animation block consists of two 32-bit integers (seem to be always 1 and 0 correspondingly), translation vector (three 32-bit floats) and rotation matrix (nine 32-bit floats).

I suppose animation sequences are supposed to be applied to the corresponding meshes in the model (each mesh has an animation ID in its header) and maybe I’ll see it eventually if I ever get to the rendering stage.

RGD

This is probably the most annoying format. It describes region data for the room in 3D format (I suppose) and packs a lot of different data that references other parts of the data.

Here’s a brief header description (all items are 32-bit integers):

  • always 0?
  • always 2?
  • total number of regions;
  • region data offset (each region includes among other things a 3-D vector index and an offset to a list of segment IDs);
  • offset to a list of offsets, data those offsets has a list of vector indices;
  • some ignored offset;
  • offset to an array of some region positioning information
  • total number of regions
  • offset to a full list of region IDs
  • total number of region IDs
  • data start offset (seems to be always 0x5C)?
  • number of segments
  • offset to segment data (which includes two point indices and an offset to region ID list);
  • number of points
  • offset to point data (two doubles per point)
  • number of vectors
  • offset to vector data (three doubles per vector)
  • flag signalling that the following fields are meaningful
  • number of special (walkable?) regions
  • connectivity matrix offset (that number of regions squared, -1 and -2 mean there’s no connection)
  • another connectivity matrix (in the same format) offset
  • offset to the list of special region IDs.

As you can see, indirection can get a bit deep. At least until I get my engine reimplementation to the point where I have to worry about pathfinding and hero interaction with things I don’t have to think about it (which is likely never).

STR

This is a special room format that happens only in 30 room (sub)variants. The format by itself is simple: 32-bit number of entries and pairs of 32-bit integers defining points. And as expected from the name, those are used to describe stars (i.e. shiny points in the sky of some locations).

QfG5: (some) cut content

Tuesday, December 19th, 2023

As I keep studying the engine code, I find hints on some planned things that were cut or not fully implemented (because game development).

I’m aware of many things like censored Nawar lines, disabled multiplayer mode, removed Glide spell and some other things being documented already so I’ll try to mention less known things.

For example, the game recognizes about thirty spells but in reality it had about ten more. One was probably deleted Glide spell, there’s something that looks like Dragon Frost spell (which acts like Dragon Fire spell but with a blue dragon head and probably frost damage). I can’t say much about the spells but considering the hints in the code there was supposed to be a completely different category of spells (normal spells have identifiers starting with 101500, paladin abilities have identifiers starting from 101600, those unknown spell IDs start from 101700 and seem to be attack spells only—maybe it’s for multiplayer mode?). For most of those there is no graphics except for one unknown spell:

Or there’s a model 98 called Sparkles which looks like a stone naked woman, I don’t remember seeing that in a game (as well as roaches). Or a blue version of a dragon (which gives only 20-50 drachmas as a loot). If I ever get to rendering stage one day I should provide pictures.

(more…)

RISC-V: still not ready for multimedia?

Friday, December 15th, 2023

A year ago I wrote why I’d like to be excited by RISC-V but can’t. I viewed it (and still view) as slightly freshened up MIPS (with the same problems as MIPS had). But the real question is what does it offer me as a developer of multimedia framework?

Not that much, as it turns out. RISC-V is often lauded as a free ISA any vendor can edit but what does it give me as an end user? It’s not that I can build a powerful enough chip even if hardware specifications are available (flashing an FPGA is an option but see “powerful enough” above) so I’m still dependent on what various vendors can offer me and from that point of view almost all CPU architectures are the same. The notable exceptions are russian Elbrus-2000 where instruction set documentation is under NDA (because its primary application being for russian military systems) and some Chinese chips they refuse to sell abroad (was it Loongson family?).

Anyway, as a multimedia framework developer I care about SIMD capabilities offered by CPU—modern codecs are too complex to write them in assembly and with a medium- or high-level language compiler you don’t care about CPU details much—except for SIMD-accelerated blocks that make sense to write using assembly (or intrinsics for POWER). And that’s where RISC-V sucks.

In theory RISC-V supports V extension (for variable-length SIMD processing), in practice hardly any CPUs support it. Essentially there is only one core on the market that support RISC-V V extension (or RVV for short)—C920 from T-Head and it’s v0.7.1 only (here’s a link to Rémi’s report on what’s changed between RVVv0.7.1 and RVVv1.0). Of course there’s a newer revision of that core that features RVVv1.0 support but currently there’s only one (rather underpowered) board using it and it’s not possible to buy anyway. Also I heard about SiFive designing a CPU with RVVv1.0 support but I don’t remember seeing products built on it.

And before you offer to use an emulator—emulation is skewed and proper simulation is too slow for practical purposes. Here’s a real-world example: when Macs migrated from PowerPC to x86, developers discovered that the vector shuffle instruction that was fast on PowerPC was much slower on Rosetta emulation (unlike the rest of code). Similarly there’s a story about NEON optimisations not giving any speed-up—when tested in QEMU—but made a significant performance boost on real hardware. That’s why I’d rather have a small development board (something like the original BeagleBoard) to test the optimisations if I can’t get a proper box to develop stuff on it directly.

This also rises a question not only about when CPUs with RVV support should be more accessible but why they are so rare. I can understand the problems with designing a modern performant CPU in general let alone with vector extension and on rather short term but since some have accomplished it already, why is it not more common? Particularly SiFive, if you have one chip with RVV what prevents adding it to other chips which are supposedly desktop- and server-oriented? I have only one answer and I’d like to be proven wrong (as usual): while the chip designers can implement RVV, they were unable to make it performant without hurting the rest of CPUs (either too large transistor budget or power requirements; or maybe its design interferes with the rest of the core too much) so we have it mostly on underwhelming Chinese cores and some SiFive CPU not oriented for a general user. Hopefully in the future the problems will be solved and we’ll see more mainline RISC-V CPUs with RVV. Hopefully.

So far though it reminds me of a story about Nv*dia and its first Tegra SoCs. From what I heard, the company managed to convince various vendors to use it in their infotainment systems and those who used it discovered that its hardware H.264 decoder worked only for files with certain resolutions and they somehow used a CPU without SIMD (IIRC the first Tegra lacked even FPU) so you could not even attempt to decode stuff there with a software decoder. As the result those vendors were disappointed and made a pass on the following SoCs (resulting in a rather funny Tegra-powered microwave oven). I fear that RISC-V might lose interest of the multimedia developers with both the need to rewrite code from RVVv0.7.1 to RVVv1.0 and the lack of appealing hardware supporting RVVv1.0 anyway—so when it’s ready nobody will be interested any longer. And don’t repeat again the same words about open and royalty-free ISA. We have free Theora format that sucked and was kept alive because “it’s free”—when it was improved to be about as good as MPEG-4 ASP there was a much better open and free VP8 codec available. Maybe somebody will design a less fragmented ISA targeting more specific use cases than “anything from simple microcontrollers to server CPUS” and RISC-V will join OpenRISC and others (…and nothing of the value will be lost).

P.S. Of course multimedia is far from the most important use case but it involves a good deal of technologies used in other places. And remember, SSE was marketed as something that speeds-up working with Internet (I like to end posts on a baffling note).

QfG5: model format

Tuesday, December 12th, 2023

Even if it looks like I’m not doing anything, that is not completely true. I’m still figuring out various bits of the engine, trying to make a whole picture of its workings. In either case, here’s a bit more of documentation.

Models seem to come in two flavours, distinguished by the first 4 bytes: MIRT (common format) and 8XOV (used in about a dozen of files; the format is almost the same but texture-related fields are absent). Animation files also use the same IDs but with a different format and there 8XOV files are more common than MIRT ones.

Each MDL file begins with the following header:

  • 4 bytes—ID (MIRT/8XOV);
  • 32-bit int—header size;
  • 32-bit int—model ID;
  • 16 bytes—ASCIIZ model name;
  • 32-bit int—number of meshes in the file;
  • 32-bit int—number of textures in the file (MIRT only);
  • 32-bit int—unknown;
  • 1024 bytes—palette in ABGR format;
  • 32-bit int—offset to texture data (MIRT only);
  • Nx32-bit ints—offsets to the each mesh data.

Mesh data has the following header:

  • 16-byte ASCIIZ part name;
  • 20×32-bit floats—unknown;
  • 32-bit int—number of vertices;
  • 32-bit int—number of texture vertices;
  • 32-bit int—number of triangles;
  • 32-bit int—offset to the vertices data (from the mesh data start, should be 0x7C);
  • 32-bit int—offset to the texture vertices data;
  • 32-bit int—offset to the face (triangle) data;
  • 32-bit int—offset to some unknown data.

Vertices data are triplets of 32-bit floats. Texture vertices are pairs of 32-bit floats. Face data consists of a triplet of 32-bit ints with vertex indices, a triplet of 32-bit texture vertex indices, a 32-bit int texture number and a triplet of 32-bit floats with unknown values. Unknown data consists of quads of signed 32-bit integers (with the same number as the vertices) where the last element seems to be always zero.

Texture data is stored in the following format: first there is an array of 32-bit ints with an offset to the start of each texture (e.g. for single texture this offset will be 4), followed by texture header (32-bit float texture width, 32-bit float texture height, 32-bit int width power of two factor, 32-bit int height power of two factor, 32-bit int width mask and 32-bit int height mask) and texture data (palette indices for each pixel) for each of the textures.
E.g. for 128×64 texture the header values will be 128.0, 64.0, 7, 6, 127, 63.

While I’m yet to render it myself, I’ve written a small tool to covert data into Wavefront .obj format that various 3D viewers understand. Here’s a rendered part of 063.MDL as an example:

Beside the head, the model file also contains meshes for: body, tail, up and lower arms (both right and left), hands, shins, feet—13 objects in total. All using parts of the same 512×256 texture.

Another fun FFork

Friday, December 8th, 2023

So apparently Paul B. Mahol had enough and finally split off (considering how his previous claims were usually not followed by immediate actions, I decided to wait for a day to make sure it’s different this time—and it is).

The most curious thing is his friend Nicolas not calling him a Libav spy despite his background and recent commits in librempeg resembling what Libav did back in the day. Maybe he really has a soft spot for Paul.

In either case I wish Paul to achieve his goals unhindered and more developers to follow his example. You don’t have to stick somewhere suffering just because you see no alternative—sometimes it is easy enough to create your own (the statement does not apply to masochists and people doing paid work).