Archive for November, 2025

History versus historical claims

Sunday, November 30th, 2025

Occasionally I watch videos about old games and gaming history (because there might be an old adventure game I have not played or an old game with a custom video format I haven’t looked at). And sometimes I see the claims about somebody doing stuff first which are not true—usually because the competition is gone or only a small group of fanboys cares, so there’s nobody to disprove their claims. One such example would be the cult status of ZX Spectrum in late USSR and post-Soviet countries: it was essentially the only microcomputer that enthusiasts could afford (except for i8080/i8086 clones that were even worse), as the result it was praised for its uniqueness, revolutionary innovations and such; when USSR dissolved, IBM PC has won the market already and the systems from e.g. Commodore were virtually unknown. IMO ZX Spectrum had two advantages: partial compatibility with i8080 thanks to Zilog Z80 CPU and cheap and simple chipset that even USSR could clone and produce. The rest was subpar or shitty (especially its graphics that rivalled CGA in being eye-gouging bad)—but hardly anybody had experience even with Bulgarian Apple II clones let alone something better. So this rant is dedicated to similar situations with both computer hardware and software.

Let’s start with hardware.

For starters, a lot of people believe that CPU was invented by Intel while in reality it’s not just TI that beat them to it, there are many other predecessors as well. See this article for more details and arguments. But considering how many CPU companies have been around and surviving since 1970s it’s no wonder people forget about the no longer existing competitors. Similarly if you know what DEC Alpha did back in the day, Pentium achievements do not look that impressive.

Similarly Nvidia claims to have invented GPU—with a definition tailored to point at their product. Who cares about SGI, 3dfx, or even Argonaut Games with their Super FX chip? Apparently people still remember it but for how long…

And speaking about game consoles, here’s a nice video pointing out some things widely believed to have been invented by N*nt*nd* while they were not.

Let’s move to software and games specifically. There’s often-repeated claim about Lucasfilm Games being a pioneer in adventure games genre by introducing a scripting engine to drive it. To me that is a perfect example of a claim people believe because others were too modest to advertise the fact. While ADVENT(URE) was written in FORTRAN, its re-imagining Zork was later re-written in ZIL—a custom language for its own virtual machine designed specially for text adventures. What’s more, Ken Williams designed a custom engine for his wife’s game (which was the first graphical adventure game BTW) and later (Sierra) On-line adventure games (and sometimes other kinds of games too) have been running on some customisable scripting engine (ADL was replaced with AGI, AGI was replaced with SCI, SCI had been in use until the company was effectively killed). Similarly this post about The Secret of Monkey Island has this passage: “Back then, the randomly generated forest was cutting edge technology. Disk space was at a premium.” One can even get an impression it was something special invented by them—until you read this post from Al Lowe which describes how he’s done it in a games released a year prior to SoMI. And I guess the same approach has been re-invented by the console game creators even earlier.

And of course I can’t go past multimedia. I wrote a post about Opus, how similar CELT design is to G.722.1 especially to its version in RealAudio (mind you, it’s still innovative but not as much as you’d expect it to be); I have not explored it further but overall Opus design resembles USAC a lot and I don’t remember hearing anything explaining that “coincidence”.

Another thing is that the only open-source multimedia player that really came close to “plays it all” was the one released last century and it was called XAnim. I mentioned it before and it deserves to be mentioned again (and again). It had everything you don’t still have e.g. in VLC: frame stepping forward and backward, support for most of the formats of the day (and I suppose a good deal of them were reverse engineered by Mark Podlipec himself; I still sometimes find to my surprise that some obscure format like DL were supported by it). And for certain formats he could not support in open-source form he actually managed to negotiate and release closed-source plugins. For a very long time it served as the source of decoders for such projects as MPlayer or FFmpeg. Even more, its Indeo decoder plugins often were the only way to decode Indeo 4 and 5 on non-x86 Unix systems. After looking at it all achievements from other open-source multimedia players do not look that impressive. And yet its site is gone and it has not got a Wickedpedia page…

Moving on, there’s an “advanced” russian operating system developed by one guy which sounded revolutionary (flat address space encompassing both RAM and disk storage, persistence, inherent security achieved by allowing only interactions between objects). You may believe it unless you actually know a bit of computing history and start asking questions like “why does this persistence and object interaction sound almost exactly like Smalltalk environment?” or “why does this idea of really flat memory space sounds almost exactly like that OS from old IBM mainframes?”. The question “why did he decide to remove all mentions of KeyKOS from the documentation?” does not require an answer IMO.

And for the end let’s talk about a certain display system. Back in the day some people were dissatisfied with X11 and decided to make a modern replacement. That windowing system was supposed to use hardware acceleration where possible, object graph to manage input from different clients (and isolate them from each other), industry-standard ways of passing messages and such. That sounds a lot like Wayland? But that’s not it, I was talking about Berlin (it appears to be memory-holed; the rather generic name does not help searches either. The sad thing is that one of the developers haven’t learned anything from it and later created a programming language with too generic name—see the other repositories of that guy I linked if you still have no idea what I’m talking about).

Why I wrote this post? Definitely not to create a “top N innovative things that were invented elsewhere much earlier”. And it’s not to shame people (let alone companies—those have no shame by definition) either. I was just trying to remind the world that you should take even wide-known claims with a grain of salt since history is written by winners and the voices of those who lost the competition are often get forgotten or remain unheard. So maybe next time you hear about how great, innovative and even revolutionary something is, think about the proof of such claim beside the claimant words themselves.

Ikarion MVI formats

Friday, November 21st, 2025

So I found another pair of rather peculiar game video formats. What’s curious is that while MVI1 and MVI2 are rather different internally, they both are used in Battle Race demo, and that’s not saying anything about their design (yet).

Both MVI1 and MVI2 are codecs used in some old DOS games developed by Ikarion Software. The only common thing in their structure is that is starts with magic containing “MVI” followed by 16-bit version and a table of frame sizes. And that’s where the differences begin. In MVI1 frame sizes are coded as radix-7 (or MIDI encoding, also known as LEB128 nowadays); in MVI2 they didn’t bother with it and left them as 32-bit numbers. In MVI1 frame consists of several optional parts, in MVI2 frame structure is fixed. And there’s another small difference: MVI1 has LZ77-compressed paletted images while MVI2 has vector-quantised 5-bit YUV420.

Of course formats employing LZ77 are dime a dozen but MVI1 still manages to stand out: first, it employs radix-7 encoded offsets, and it also de-interleaves palette by components before compressing (i.e. first all red components are sent, then all green components and finally all blue components). Optionally it can employ motion compensation on 10×10 tiles (which is uncommon already) with motion vectors for each tile being compressed with RLE and also in de-interleaved form (i.e. 768 16-bit offsets are split into high and low bytes and those are compressed one after another). I can’t think of many formats back then that thought about de-interleaving data in order to increase compression.

MVI2 splits frames into 64×64 tiles that are composed of 4×4 blocks selected from a circular buffer of 4096 entries. So frame decoding consists of reading new 4×4 blocks for the codebook and then decoding 12-bit block indices for 64×64 luma plane and two 32×32 chroma planes (all they use the same codebook, which is rather unusual for such codecs). Of course such approach is not particularly new (and reminds me more of 16-bit consoles) but I still can’t remember another VQ-based codec with exactly the same design.

P.S. And there’s also MVI used in Archimedean Dynasty—yet another MVI format from yet another German company. I’ve not looked at it yet (beside a cursory glance in the hex viewer) but it looks different from those two. Let’s see what surprises it has…

Some words about DMV format

Friday, November 14th, 2025

I said it before and I repeat it again: I like older formats for trying more ideas than modern ones. DMV was surprisingly full of unconventional approaches to video compression.

First of all, it uses fixed-size blocks with palette and image data occupying initial part of the block, and audio taking fixed size tail part of that block.

Then, there’s image compression itself: frame is split into 2×2 tiles that can be painted using 1-4 colours using pre-defined patterns, so only colours and pattern indices are transmitted. Those indices are further compressed with LZ77-like method that has only “emit literals” and “copy previous” (with source and destination allowed to overlap but with at least 4-byte gap for more efficient copying).

And even that is not all! I actually lied a bit because in reality it is 2x2x2 tiles since one pattern (using up to 8 colours) codes two 2×2 blocks in two consequent frames simultaneously. Now that’s an idea you don’t see every day. There are many codecs that code colours and patterns separately (like Smacker or System Shock video), some use a variation of LZ77 to compress data even further, but the only other codec I can name that coded two frames at once would be H.263+ with its PB-frames (and it was mostly interleaving data in order to avoid frame reordering, not sharing common coding data between frames).

Searching discmaster.textfiles.com for large unrecognised files returned more hits so I hope that some of the remaining formats will also feature some refreshingly crazy ideas. Now I only need time and mood for reverse engineering.

Quickly about VC2

Wednesday, November 12th, 2025

As I mentioned some time ago, Paul has shared a decoder for bytedanceVC2 codec. From the first glance it looked like H.266 rip-off. After a more thorough looking it seems to be more or less plain H.266.

There are some tell-tale signs of codecs using well-known technologies: simply listing strings in the binary gives you “VPS/SPS/PPS/APS malloc failed” (with APS being adaptation parameter set, which was not present in H.EVC but is present in H.266), “affine merge index is out of range” (again, affine motion was added in H.266), “lmcs_dec” (chroma-from-luma, another feature not picked up by H.265 but which was fine for H.266), “alf_luma_num_filters_signalled_minus1” (a variable straight from the specification). I hope that this is convincing enough to claim that the codec is based on H.266 (or VVC).

Now what would a rip-off codec change? In my experience such codecs keep general structure but replace entropy coding (either completely or at least change codebooks) and DSP routines (e.g. for H.264 rip-offs it was common to replace plane intra prediction mode with something similar but not exactly the same; motion compensation routines are the other popular candidate).

I cannot say I got too deep into it but the overall decoding is rather straightforward to understand and from what I saw at least CABAC was left untouched: it’s exactly the same as in the standard and the model weights are also exactly the same (at least the initial part). Of course that does not preclude it having differences in DSP routines but I seriously doubt it being different from the standard (or some draft of it).

And with that I conclude my looking at it. Those with more motivation and actual content to decode are welcome to try decoding it, I have neither.

Archive extraction support in na_game_tool

Saturday, November 8th, 2025

Back in the day I said that I want 0.5.0 release of na_game_tool to be more than a mere decoder and also support game data archive extraction. And what do you know, less than half a year later I’ve finally started to work on it.

The idea is rather simple: use the same mechanics to detect archive type, create a plugin that will go through archive contents entry by entry, use information provided by it to create an output file and provide a writer for plugin to dump contents to.

Of course such functionality is present in many other tools, but this one will make na_game_tool more than just a decoder for some game multimedia formats, it will support the formats I somewhat care about, and it can perform some format conversion while at it. And it will allow me to offload the throwaway code I wrote for extracting such formats so that knowledge won’t be lost (as well as allowing me to use an existing framework when I need to create another extractor). With this the tool becomes more useful to me.

Here’s a simple example: the first archive format I implemented is for Access Software .ap files. The archive contains merely offsets where next file starts and nothing more. But for convenience I added a bit of code to look at the first bytes of the extracted file and give it a reasonable extension if possible (e.g. for ACC, DBE, H2O, PTF and WAV files).

Here’s another example: third (and currently last) supported archive format is Tsunami Media RLB. Beside mere extraction of all files (which is a necessary step for supporting their animation formats), it also outputs a proper image instead of raw image data for Mehan Enough (that involves remembering last decoded palette and scaling image twice horizontally). In the future I’d like to support images from other games like Ringworld but that would involve a bit more work.

Ultimately I hope to keep such hybrid approach to data extraction: convert simple formats (image and audio) in the extractor plugin if it makes sense and leave more complex formats to the other decoder (which may be na_game_tool, na_encoder or some third-party software). And I have a good candidate for that: Monty Python & the Quest for Holy Grail. You can see a page describing formats from that game linked in the top corner of this blog and even find a couple of posts about it. I guess that means that I care.

Of course it will take more months to make the next release (I’d still like to keep the rule about decoding at least twelve new formats based on my own research) but I’m not in a hurry anyway.

FFpropaganda

Saturday, November 1st, 2025

Recently this tweet was brought to my attention (by somebody else who saw it and found hilarious). While I agree with the message (I also find those OMGsecurity people annoying and rather counter-productive), the premise is bullshit, namely those two lines:

Arguably the most brilliant engineer in FFmpeg left because of this. He reverse engineered dozens of codecs by hand as a volunteer.

So I’ll try to explain what’s wrong with that claim and FFmpeg twitter account (or FFaccount for short) in general.
(more…)