Twenty years

August 14th, 2024

On August 14th 2004 Mike Melanson committed to FFmpeg a decoder for TSCC codec that I sent to him. So on that day my official story of contributing to open-source multimedia has started.

A lot has changed since then and not always to the best. Back then there were a plenty of formats to reverse engineer (with quite a demand for them as well), nowadays it feels like there’s one standard video codec (or two, if you see AV1 as different enough) and Fraunh*fer Alternative-to-Opus Audio Codec (or ***-AAC for short). And the complexity increased so that it’s increasingly harder to implement a codec alone.

From the software side things have changed as well, I’ve ranted about it enough. To put it simply, I’ve not been participating in FFmpeg (or libav) development for the last decade and I don’t regret it at all.

And yet I still have the curiosity about various multimedia formats so I keep investigating them and sharing the results with the public. I’m not sure I’ll find enough material to last for another twenty years but there are still enough obscure games out there. Occasionally my work comes in handy for somebody else and it makes me happy (not as happy as with my first contribution, that happiness lasted for a week—but I was much younger and very inexperienced back then). That’s both the curse and the blessing of being an expert on a very narrow topic: hardly anybody needs you but when they do they have no other option. So I’ll keep providing what I can as long as possible and let’s see how it ends…

Watery format revisited

August 10th, 2024

I’m still picking unfinished and half-finished format for na_game_tool and there were some from Access games. While supporting animations from Amazon: Guardians of Eden is rather pointless (since they simply update pixels on some scene and do not have own palette), supporting H2O and PTF formats is more interesting.

I wrote about them back in the day and about H2O specifically. And since it was a quick look and my eyes are not that good, I missed some details. But as I finally implemented a decoder for it, now I can correct that story.

As it turned out, H2O stores video data as Huffman-coded 16-bit values instead of usual bytes. And those values are used for both opcodes and colours. If top four bits of opcode are 0-3 then the top byte codes number of 4×4 blocks to process and low byte is fill value (0 means skip block instead). If top four bits of opcode are 4, then low 12 bits are number of blocks to process minus one and each block data consists of 16-bit pattern and two colours (packed as another 16-bit value). Top four bits of opcode being 5 mean partial update for the specified number of blocks, each block having 16-bit update mask and 0-16 colours to update (packed in 16-bit words).

So while the format is no ground-breaker, it still turned out to be more interesting than I expected it to be.

P.S. I guess I’ll do a bunch of FLI-based formats next. There are some curious variations of it like FLI with audio or RNC ProPack compression.

Implementing Jack Orlando animation support

August 5th, 2024

As I’m still working on my game tool, I decided to implement video format from the Jack Orlando game. So now na_game_tool has AVX support and I’d like to say something about the format.

The format is a rather weird mix of different things. It starts with a fixed-size audio block and that size depends on whether it’s 8-bit PCM or IMA ADPCM (with stereo samples packed into interleaved 32-bit words), it seems to have block structure where an arbitrary chunk of data is stored as 32-bit size plus block data which includes video data with accidental headers. Yes, the file internally may have several FLX animations with different framerates too, so e.g. a 16-fps piece of animation may be followed by 10- or 24-fps animation. And audio stream is global even if it’s stored in small chunks inside FLX (since you have several seconds of pre-roll audio in the beginning). Oh, and palette data may be stored as RGB565 or RGB24 (which you’re supposed to understand from palette chunk size). Video compression is simple but somewhat weird too, as RLE is used to optionally compress data before it gets decoded.

In either case weird does not mean bad and in my line of hobby an interesting codec is worth more than a popular one.

P.S. And there’s even weirder format known as Knowledge Adventure MOV. I gave up on REing it but apparently video there is split into tiles (4×2 or 2×1 pixels apparently) and frame data is those tile indices that may be stored uncompressed, RLE compressed or, apparently, using an arithmetic coder with static frequencies stored in the file header. The difficulty comes from self-modifying code and not the cute kind that simply modifies couple of instruction constant arguments but one that substitutes function call address with a pointer to a special memory buffer allocated in a context and filled elsewhere. In theory it can be figured out without much hassle with, say, a built-in DosBox debugger but I’m not inclined to do that (at least not now).

Re-revisiting Actimagine VX

August 2nd, 2024

As Paul said, I probably should do something more useful (and he should do something better than still messing with jbmpeg fork) but we’re all better at giving good advises than at following them.

Recently I was contacted by a guy who asked for my help on dealing with VX format as he needed video part extracted from it and there were no tools available. And he also provided some samples that helped clarify things I got previously wrong, for example the seek table and framerate field as well as stereo audio support.

Also since I bothered to extract video part (the only way to determine its length is to parse it, so now I have a stripped-down version of video decoder doing that) I decided to finally do something about the audio part. So after fixing some things (e.g. missing filter reconstruction or stereo support) now my decoder can decode VX files with recognizable audio (not perfect but who can do better is encouraged to do so). The sad thing is that I could not use my old setup (a console emulator with GDB stub) for debugging so I’m not likely to ever improve it.

At least it was a fun diversion and helped to improve the format knowledge a bit.

P.S. Meanwhile I’m still working on adding new formats for na_game_tool, hopefully I’ll have something to write about another game format soon.

Revisiting Talisman animation format

July 28th, 2024

Back in the day I wrote a bit about this format and since it’s rather interesting I decided to implement its support in my game tool.

Eventually I got it working except on two files that seem to have certain frame data unpacked incorrectly (it’s the feature that makes this format interesting in the first place—per-group order-1 Huffman coding). And for every file decoding seems to go past the data boundaries at the end so I insert some additional zeroes at the end of packed data to make it work (another fun fact – apparently last eight bytes of both opcode and colour parts of inter frame seems to be always unused). And it seems audio data is always coded as silence. The actual cutscenes do not make much sense either as it’s usually a static cartoon image superimposed over a 3D rendering of some lady in a tower-like structure (the game does not deserve any associations with Sierra titles so I’m not using other words to describe it). Sometimes her lips are moving (if you can notice it behind the other image), quite often they do not:

All in all, this is a rather weird format for a seemingly weird game but I don’t regret wasting time on implementing a decoder for it.

NihAV Game Tool: the official release

July 21st, 2024

I’m finally proud not too ashamed to present a side project I’ve been wasting my time on.

The rationale behind it is simple: I sometimes write throwaway decoders in order to check if I understood format properly or if I really want to see the decoded content. Usually it’s written in C with the same code (usually for dumping output as PPM image sequence or reading 16-bit value) copied over and over again. So I thought to borrow bits from NihAV and finally make a framework for handling output creation and various utilities for handling input (e.g. reading integers of different size and endianness). It’s still better than doing nothing and it may be marginally useful to somebody else.

Not all of the included decoders are completely my own work, some come from ScummVM via the documentation I created for them on The Wiki (but sometimes with improvements, for example they don’t handle compressed sections in PMV) and AV format comes from the Lord of the Rings engine re-implementation (again, via The Wiki).

I should also mention a nasty surprise. Apparently when AVI streams are sufficiently de-synchronised (e.g. the format pre-buffers a second of audio before sending any video frames) libavformat AVI demuxer (which is used by a lot of multimedia tools nowadays) switches to a different mode and treats palette change chunks as normal video data (which works wonders of a stream of course; and apparently nobody had this problem since the time I introduced palette change support there about twenty years ago; also my own player is unaffected). So for such formats I had to introduce manual audio buffering (it’s not nice but the alternatives are worse).

And some words about the releases and release schedule. I don’t want to bother my friend who hosts the public NihAV repositories to add another one and I want to get involved with the usual platforms even less. As the result I’ll simply dump source tarballs with a brief changelog on the site. Releases should happen irregularly, when I accumulate, say, another dozen of formats or have other features implemented (like detecting format by regex instead of just by extension or OpenDML AVI support for being able to output annoyingly large files).

But for now the source code along with some formal Git history is available at a NihAV special page. Grab it while it’s not that stale.

Web of Bullshit

July 16th, 2024

I’d rather write about the current state of the world in general, how russians proved once again they don’t deserve to be called humans, how only an idiot would trust their word or believe they’re going to keep any agreement, how the general attitude of dealing with russia looks like somebody attempting to cure a disease so that the treatment does not cause any discomfort even if allows that disease to progress until it’s too late to cure it… But I’ve written all about it previously so I’ll write on a related but less crucial topic.

I ranted about the state of Firefox less than two weeks ago. And what do you know, version 128 proved out to be even worse with its attitude to the users. One could wonder how it can get worse but apparently Firefox CTO decided to give a public justification of their decision. So their answer to the war with the annoying advertisements is making sacrifices of the users’ liberties in hope that the aggressor will be satisfied with that (it always worked fine in the real world as can be seen by World War II and the ongoing World War III).

The sad thing is that the advertising is responsible for the current web of bullshit, here’s a short review.

John Wanamaker allegedly said “I am convinced that about one-half the money I spend for advertising is wasted, but I have never been able to decide which half.” It’s hard to disagree with it (except that the share of effective advertising feels much lower these days) and that’s the root of the current problems.

Considering that a lot of the first domains in the Web were belonging to the large companies no wonder ads were present there from the very beginning (a small example: one of the oldest pages on The Wayback Machine is for http://www.ads.digital.com). But the real boom of advertising started when lots of ordinary people started to frequent it and various companies felt that there are money to be made off that. Add rather unscrupulous website designers and you get the (first) dark ages of Internet: annoying Flash banners, pop-ups and pop-unders, blinking text and so on. There’s the first bullshit tendency for you—putting as many ads on a page as the browser can render. And coming with it the second bullshit tendency—inflating content for accommodating more ads. Well, if you give people a way to profit off advertisement placement somebody is going to abuse it to death.

And then somebody came with the main bullshit idea: advertising can be targeted! Theoretically if you know enough about the person (or at least its actions and habits) you can offer that person only the relevant ads thus making the success rate close to 100%. In practice it does not work because people do not work like this at all (banner blindness exists, people usually get too scared when their “smart” device starts recommending them something they talked about in its presence, many people really want different things from what they believe they want and so on; and that’s not counting how the common pattern for recommendations is “you bought an electric stove recently, that means you want to buy another electric stove”). And this bullshit stimulated the growth of privacy violations and social networks. But I repeat myself.

So that’s all fine for the ad networks who can feed this bullshit to the entities placing those ads (as well as another one that those ads will be shown only to the target groups selected by them). Now it was time for people trying to earn money from displaying those ads (voluntarily or not) to learn that earning much from those ads is bullshit. Advertising on streaming platforms gets more and more aggressive but looks like for the content creators the main revenue source is subscriptions and donations but never the share from the ads provided by the platform (partner deals to place specific ads directly in the video may be a different case, you should know those MMOs and VPN services by heart now). Small blogs also seem to live off subscriptions and donations with an occasional native advertisement.

But of course there must be people who decide to automate the process as much as possible to get those vanishingly small amounts of money per ad click for millions of clicks. That’s how we get bullshit generated just to lure people to click on the ads (still talking about the Web and not, say, mobile games BTW) and even bot networks to click ads on bot-generated pages that were placed by the ad-bots. Some call it the Dead Internet Theory, I called it right in the title.


But it’s not all that bad, sometimes things get better: browsers learned to block pop-ups even without a separate plug-in, Flash was killed (maybe because a certain guy could not control it on his phones, or it made them look under-performing—in either case they both are dead now), there are certain legal restrictions for advertising in the Internet even in the USA let alone EU and there are ad-blockers. The main disadvantage is that major browsers are controlled by the companies depending on ad revenue (and A**le, where ads are merely a part of iExperience), Mozilla joining them recently. So it’s natural for them to try offering more data to the advertisers and restrict ad-blockers as much as possible (does anybody believe that things like Manifest V3 have any different intent?). We see the first step done by Mozilla already, crippling uBlock Origin looks like a matter of time. At least it should help Ladybird, Servo and maybe some Firefox forks to develop faster.

A side project for NihAV

July 7th, 2024

Since I still have nothing much to do (and messing with browsers is a bad idea unless the situation is desperate), I decided to make a NihAV-lite project. So announcing na_game_tool.

This is going to be a simple tool to convert various game and image formats (and related) into image sequence, WAV or raw AVI (which then can be played or processed with anything conventional). I’ve begun work on it already but the release will happen when at least when I implement all planned features (which is writing image sequence in BMP format, AVI output and porting two dozen of half-baked decoders I wrote to test if I understood the format).

Why a new project? Because I have nothing better to do, it still may be marginally useful for somebody (e.g. me) and I can do some stuff not fitting into NihAV (for example, decode 3DO version of TrueMotion video split into four files) and I don’t have to bother about other stuff that fits demuxer-decoder paradigm poorly and requires inventing ways to convey format-specific information from the demuxer to the decoder. In my case I simply feed the input name to the input plugin and it returns frames of decoded audio or video data. Some hypothetical Antons might ask a question how to deal with the formats that use variable delay in milliseconds between frames instead (and I’ve implemented one such format already). To which I can reply that one can fit a square peg in a round hole in two ways—by taking a larger hole or by applying more force. The former corresponds to having, say, fixed 1000fps rate and send mostly the same frames just to have constant rate; the latter is forcing a constant framerate and hoping for the best. I chose the latter.

The design is rather simple: there’s a list of input plugins and output plugins. Input plugin takes input name, opens whatever files it needs, outputs information about the streams and then decoded data. Output plugin takes input name, creates whatever files it needs, accepts stream information and then receives and writes frames.

Probably there’s a better alternative with librempeg but you’d better go read about it on Paul’s blog.

Suicide by thousand cuts

July 5th, 2024

Firefox has finally upgraded for me to the localhost version and apparently the developers (or rather some other “creative” people, I suspect) decided to make it more secure by being unusable.

The first thing I noticed is that the last tab refused to close and now you need to close the window—that’s annoying. Then I noticed that bookmarks have disappeared from the toolbar and no matter what you do you can only get an additional blank space shown—at least they’re still accessible through the menu. Then I noticed that downloads may run but they’re not reported—now that’s extremely annoying. And cherry on the top is that closed tabs cannot be restored and recent history remains blank—now that’s borderline unusable.

And apparently the reason is that I’m using the browser wrong. From what I read, they decided to “protect” user data by introducing a session password which you apparently need to enter at the each session start. And considering that I power off (most of) my computers at night and usually launch browser for a quick private session (usually to check news or search for something and not clutter my history with URLs from the search pages and bad results) that means unwanted annoyance many times a day. And of course since I had no reason to launch the browser in non-private mode for many months, the change went completely unnoticed (and when they got rid of XUL even I knew that in advance despite not following the news that much).

Unrequested changes (like changing GUI layout, adding Pocket and so on) build up annoyance and breaking things like this make me consider using another browser. For now I see no real alternative (maybe one of its forks is good without me knowing it, or Servo or Ladybird will become usable for my needs), so I simply downgraded to version 126 for time being and switched off auto-updating but I should use dillo and elinks more.

P.S. One of the reasons why I switched to my own video player was that the previous one I used also decided to “improve” user experience in suspiciously similar ways (by not doing what it did because you apparently don’t know what you’re doing and by interpreting things differently). I definitely don’t want to get into browser development (and I lack hardware for that too) but I need to consider that option…

REing another simple codec

June 29th, 2024

Since I was bored I tried to (ab)use discmaster.textfiles.com to search for interesting (i.e. unsupported) samples once again. The main problem is that if it does not decode contents it does not recognize the format. So e.g. AVI files without video track (yes, those files exist) and those using some unrecognized codec will be both marked as aviAudio format, and if audio stream is absent or unknown as well the file gets demoted to unknown.

So I tried to search AVI and MOV files both by extension and by this audio-only type and here are the categories of the results:

  • actual audio-only files (that’s expected);
  • completely different format (there’s an alternative AVI format and MOV is very popular extension as well);
  • improperly extracted files (rather common with MOV on hybrid Macintosh/PC CDs where resource fork often gets ignored);
  • damaged files (happens with some CDs and very common with AOL file library collection—often AVI data starts somewhere in the middle of the file);
  • too old or poorly mastered files (for example, one AVI file lacks padding to 16 bits between chunks; some MOV files can’t be decoded while they look correct);
  • one Escape 130 that could’ve been supported if libavcodec AVI demuxer would not feed garbage to the decoder (it’s not just my demuxer that can handle it, old MPlayer 2 plays it fine with its own demuxer);
  • some TrueMotion 1 files that were not recognised because of tmot FOURCC;
  • files with some special features of the known codecs (I’ve seen some MOV files containing QDraw codec with JPEG frames);
  • files with the codecs I can decode (like IPMA) but the popular software can’t;
  • files with the known codecs (some documented by me) that nobody bothered to implement (especially Motion Pixels 1 and 2);
  • and finally some AVIs with savi FOURCC and a single file with DKRT FOURCC.

Those “SuperAVI” files turned out to be a rebranded Cinepak which I managed to recognise right away, the remaining file turned out to be a bit baffling. After extracting the frames I figured out that it is raw YV12 video, but for some reason it had 64 bytes of soemthing before the image data and 440 bytes after. It can be located on TNG Klingon Language Disc but it does not look like the software there can decode it anyway.

Overall, nothing hard or interesting (if you don’t count the questions about the origins of that file, that is).