Soon it will be eight years since I’ve (re)started NihAV development in Rust. And for this round date I’d like to present my impressions and thoughts on the language—from the perspective of applicability in my experimental multimedia framework.
My Rust experience after eight years
March 29th, 2025A look on Perfect Clarity Audio
March 28th, 2025Since Paul asked me to look at it (and what’s equally important, provided the binary to look at), I did exactly that.
It turned out to be a rather simple codec, originally known as Sonic Foundry Lossless. The only really interesting detail is that it has intra- and inter-frames. Intraframes start with 0xFFFFFFFF and four last samples for each channel (for filtering purposes). Each frame stores the number of samples (I suppose), coded data size, Rice code parameter (per channel), filter method/order (ditto), LPC coefficients (same, only if global flags enable it though) and probably CRC.
Data is coded as fixed-parameter Rice codes (low bit is used for the sign)—unless it’s filter method 5 which means all zeroes, then optional LPC phase (LPC has order four and 8-bit coefficients) and then, depending on filter order, fixed prediction as in Shorten.
Finally there may be mid/stereo reconstruction (if global config signals it) and clipping for 24-bit mode (but not for 16-bit apparently).
I don’t know if they’ve added some improvements in the newer versions but this looks rather simple compared to other lossless codecs. There’s the final bit of weirdness: it’s usually stored in WAV which means it’s one of those codecs with variable-size frames in WAV. Not something I’d like to support.
P.S. I’ll document it for The Wiki (as well as DVC from the previous post) a bit later.
Revisiting QPEG
March 27th, 2025Since I’ve done all improvements to NihAV that I wanted to do (beside vague “improve Indeo 3 encoder somehow” or “add some interesting format support”), I decided to look at the formats in my backlog and discovered that I have a QPEG player with a couple of DVC samples. Considering that I’ve REd their VfW codec over two decades ago, I had to look at this as well.
It turned out to be a straightforward format with static palette and video frames packed with moderately complex RLE (it has opcodes for run/copy/skip and literals). The most interesting thing there to me was that values without high bit set are treated as literals, or rather as indices in the remapping table (which is the first 128 bytes of a frame). Considering that low 20 colours of the palette seem to be unset, it makes some sense.
The hardest part was to read the binary specification. The executable uses Phar Lap 386 extender, so it’s actually stored in P3 format right after the loader. At least I have some experience with loading such formats when I messed with Linear eXecutable format before Ghidra plugins were available (and sometimes afterward as well, since e.g. neither of two known plugins managed to load Wing Nuts executable). Also I managed to spot that 0xE0 byte happens at the end of packed frame, so I guessed it was the end data marker and searched for the code using it as such. I’ve managed to locate four RLE decompression functions, all probably functionally identical, and after figuring out other details (like where remap table comes from) I ended up with the decoder that works on all four known samples just fine.
Overall, it’s nothing particularly complex but it was still nice to look at.
NihAV: encoder improvements
March 21st, 2025Since I feel I’ve done enough (for a moment) on na_game_tool, I decided to work on the rather (always) neglected NihAV. And specifically on the nihav-encoder tool.
As you can guess from the name, it is a tool for re-encoding input file(s) into something else (or something the same). Since I don’t have much need in transcoding anything, I use it occasionally to test encoders (especially for collecting information by running encoding with different parameters on the selected sample set) or a muxer. Essentially if video is encoded correctly that’s enough, audio de-sync or even drop-outs (because not all encoded packets were actually muxed) is another thing. Finally I’ve decided to improve the situation and make it more usable.
Read the rest of this entry »
A bit of class theory
March 15th, 2025There is a Ukrainian word “жлоб” (zhlob) with unknown origin, but probably coming from Polish żłób via Yiddish (and likely donating that meaning back, since in Ukrainian it means only a person while in Polish it’s not a main meaning and does not connect to it).
It is hard to give a proper translation for that word. It may mean a man large as ox, strong as ox, and with comparable intellect too. One dictionary claims it’s a synonym for a wealthy peasant who employs others. More often though it means a vulgar person with very limited intellectual needs and a miser. Kinda like an average person but without positive traits and with deficiencies magnified. I’m pretty sure that you can recognise such characters even without a strict definition.
And there’s Ukrainian living classic Les’ Poderv’iansky, often known simply as The Artist (he started his career as a painter but got much more famous as a writer and poet; he has given us many catchphrases, the best-formulated version of Ukrainian national idea and I’ve heard people quote his adaptation of Hamlet in full—people who would not normally read classics or memorise poetry).
Once he formulated the main reason why Karl Marx was wrong: people differ not by classes but rather by mindset. So poor zhlob shares interests with the rich zhlob and they can understand each other better (despite being a worker and factory-owner) than, say, a teacher.
Here’s an excerpt from his essay (published in the essay collection “Жлобологія” pp. 245-250 in 2013 if you care):
It would be nice to turn things around so zhlobs can’t get power. Unfortunately that’s utopia.
Sensible people, who are sadly too few, don’t have any influence, so zhlob ideology prospers. The prevalent class is zhlob, who is preserving his values. If this continues, this country will be completely fucked.
That’s because zhlobs are not fit for anything good but like awards nevertheless. […] They require proofs of own importance. They like various tchotchkes, posts and titles. Their hidden dream is to become counts and dukes.
Of course it had been written in Ukraine back in the day before people revolted and threw away one certain zhlob that other zhlobs voted for president, but the principles remain the same and can be applied to other countries. So if you ever wondered why rednecks vote for a hereditary billionaire believing him to be “our guy”, while hating another candidate despite her being “closer” to them—now you know the answer.
NihAV: hardware-accelerated playback revisited
March 10th, 2025Recently I’ve made a mistake of upgrading one of my laptops to systemd 24.04. I can talk about various quality of life improvements there (like brightness control not working any longer so I have to evoke xrandr instead) but it’s so useless rant that it does not deserve to be written. What is worth talking about is hardware acceleration. Previously my player on that laptop had rather unreliable playback with VA-API decoding (i.e. it worked but some videos made it too laggy); now the situation has improved—it’s reliably unusable. Additionally it seems to consume only slightly less CPU than with software decoding. So I finally looked at the way to speed it up (spoiler alert: and failed).
Conceptually it’s simple: after you decode VA-API picture you need to obtain its internal parameters with vaExportSurfaceHandle(), create an EGL picture using eglCreateImage() (or eglCreateImageKHR()), blit it onto OpenGL texture with glEGLImageTargetTexture2DOES() and you’re done. It was not so simple in practice though.
Exporting descriptors is easy, I just modified my fork of VA-API wrapper to do that and it even seemed to produce correct output. The problems started with OpenGL integration. My player uses SDL2 so I spent a lot of time trying to make it work. First of all, it’s not clear how to obtain a proper OpenGL context for the calls, then there’s this problem of it being finicky and not liking multi-threaded execution. And of course you have to load all those functions mentioned above manually (because SDL2 offers only a small subset of all possible OpenGL functions—not surprising, considering how many of those are optionally supported extensions or missing in certain profiles).
Anyway, I threw away most of my player functionality, leaving just loading an input file, decoding it and trying to display only the first frame. It ended with a segfault.
It is probably because of (poorly documented) SDL2 wrapper doing, but it can’t provide a sane OpenGL context probably. So a call to eglGetCurrentDisplay() returns either NULL or a pointer that looks like a small negative value; the same happens with eglCreateImage() (fun thing that eglGetError() returns the same value, -424 if somebody is curious); at glEGLImageTargetTexture2DOES() call it finally segfaults.
At that point I actually tried searching for some alternative crates that would allow me to create an OpenGL window and call those functions—and found none fitting the purpose. They all are either provide rather limited OpenGL subset which is enough for drawing a triangle with shaders (and probably expect you to locate and load the library with the additional functions by yourself) or even simply provide OpenGL bindings, leaving even window creation to you.
In other words, not my kind of fun. I’ll simply disable hardware acceleration for all cases there until I find a better alternative.
P.S. Vulkan is not really an option either. My hardware is too old for ANV driver (and for HASVK driver too).
Random NihAV news
March 8th, 2025Since I can’t do anything about the world largest countries run by over 70-year old dictators, I try to find a distraction elsewhere. It usually works only until the next time I read next. Anyway, here’s something different for a change.
With all the work on na_game_tool I’ve been mostly neglecting the original NihAV (not that it makes much difference). So I’ve tried to improve it a bit and I have something to report.
First of all, I decided to make ARMovie support more complete (after discovering that e.g. the only thing that can play Eidos Escape 122 codec is their own DOS player). So I’ve added all three video codecs with known samples (Escape 122, 124 and 130) as well as their ADPCM codec. Sadly Escape 122 description in The Wiki is somewhat unclear, so I referred to the original DOS player for it. Similarly I looked at ADPCM decoding because what libavcodec produces is not quite right (but I guess Paul can tell you more stories about all those IMA ADPCM flavours). So I guess now my project has the most complete ARMovie formats support out there. There’s only one other third-party format I’m aware of (The Complete Animation film, it may be stand-alone or encapsulated in RPL) and I might get to it eventually.
The other thing is TrueMotion S. When you think there’s nothing unknown left about it, it manages to surprise you anyway. So I was looking at Discmaster search results for potential candidates (“aviAudio” is a good format for that—sometimes it is really AVI with audio track only, sometimes it is AVI with unknown video codec) and found three .duk files that were TM1 in AVI and which could not be decoded. It turned out to use codebook 0—which is not present in the open-sourced version of the decoder. Another thing is that it does not use so-called fat deltas (i.e. larger difference values), so code zero means eight zeroes instead of an escape value. Remarkably, this is demo of The Horde game by Toys For Bob, known to employ yet another exotic version of the codec in their 3DO version of Star Control II. It makes me wonder if they had the same relationship with Duck as JVC NWC with RAD (you know, the company which released a game with Bink-b videos not found anywhere else and bundling Bink-d decoder with one of their other games—all while others started with Bink-f or later).
Hopefully I’ll be able to do more in the future, but I wanted to share these stories while they’re still fresh.
na_game_tool 0.3.0 released
March 4th, 2025This is finally a release I’ve been wanting to make—with a format for each letter of the alphabet! It features about twenty new formats supported, a dozen of them being newly reverse engineered by me and the rest coming from NihAV, various open-source game engines and my older RE work. The games are diverse too, as some of them come from Argentina, Finland, France, Germany, Israel, Japan, Spain—beside the usual English-speaking countries (I’ve not tried to achieve it but that’s a nice thing too).
As usual, it’s available here (but who really cares?).
Considering that I’ve mostly exhausted the games to look at and the goals for the next release are at least twelve new formats and to support over hundred formats in total (i.e. about twelve in total), I’m not sure the next release will happen this year. I’ll find other stuff to do meanwhile.
Cinepak everywhere!
March 2nd, 2025First of all, I’m happy to announce that I’ve finally managed to reverse engineer a format for game called Wing Nuts which means I’ve finally collected formats for the whole alphabet. So I want to implement another format that I’ve REd and I can finally release na_game_tool 0.3.0 in a week.
But what I really want to talk about is the game format itself. There are no real game videos but rather game engine data files containing various kind of data, including video frames. I could also extract PCM audio but I don’t know how to synchronise it properly so I ended up with a video-only decoder. MIDI data was yet another option that I ignored.
What is curious is the video compression format. While REing it I had constant thoughts like “hmm, it stores chunk type in a byte followed by 24-bit big-endian number, just like Cinepak” or “hmm, it uses 2×2 vectors and either scales them or combines four of them in a single 4×4 block, just like Cinepak” or “wait a minute, this bitstream format—chunk markers, flags and vectors—are exactly like Cinepak”. There are some differences from the usual Cinepak we all know though: it is a paletted format (with palette stored in a similar form before actual image data) and it has some additional chunks with codes 0x60-0x62 which seem to contain some auxiliary information used by the engine.
So seeing this shortly after CinEApak and knowing that SEGA company used slightly modified Cinepak format on its consoles, it makes me wonder is the codecs was more widely licensed and tweaked than we are aware of. That would make it even more like Duck TrueMotion (which was also widely used in game videos, on consoles as well, and with a special flavours used at least in one game).
na_game_tool: penultimate letter
February 28th, 2025So now there’s just one letter left, hopefully it won’t take that long to find a suitable format.
Meanwhile two formats for letter ‘o’ have been added, both formats are formally from Origin.
One is FLIC-in-IFF used to store intro and ending for Ultima VII (both parts). Actually there are also fonts and speech stored there but since you need external data (plus game engine) to combine them properly I ended up simply decoding animations from those archives.
Another one is equally flimsy—System Shock movies (which are predominantly just audio tracks used probably for audio logs; also the game reminds me of Paul for some reason). What’s remarkable is that it essentially combines two different coding methods for two sizes: VGA movies are RLE-coded, SVGA movies use 4×4 tile coding with additional Huffman coding of tokens. Essentially tiles are coded using tokens representing opcodes, which may refer to 4-16 colours from the global colour table or some other operation (e.g. “skip N blocks” or “repeat previous operation” or even “fill block with these two colours”); tokens are Huffman-coded, both colours and Huffman table may be transmitted at any time. Luckily I did not have to RE it from scratch as the engine code has been open-sourced some time ago.
Just a couple more formats and I’ll be able to make 0.3.0 release at last.
And I’ve thought for another fun goal for 0.4.0: get a number of supported formats to a hundred (it would be better to hit some round number like 128 but that’s more of a milestone for the improbable 0.5.0 release). Currently I have 70-something formats listed and that will number will increase to 80 for the upcoming release for sure. Finding about eight extra formats to support is easier than finding twelve formats to RE. And to put things into perspective, librempeg lists about 270 video decoders (excluding raw formats and hardware accelerated decoding), some of them being equally incomplete, simple, or being a small variation of another decoder. So having a third of that number for game-only video formats (most of those being unique too) is a respectable feat IMO.