Archive for April, 2023

A quick look at a watery game video format

Monday, April 24th, 2023

As you can guess, my work on Real Video 4 encoder goes so well that I’d rather look at virtually anything else.

Probably some of you are aware of Tex Murphy game series (maybe not even just Mike) and I got reminded of an existence of The Pandora Directive game that supposedly has a lot of videos in it, so I downloaded a demo and decided to explore it a bit.

Game data is packed into extremely simple archive format (just the number of offsets and the offsets) so you need to know what archives to look into (hint: the larger ones).

So what about videos? They start with H2O magic (hence me calling it a watery format) and contain video frames and audio data (in an embedded WAV file). The video compression is both nothing special and somewhat original: it’s Huffman-compressed RLE. Video frame can contain several chunks, first there’s a Huffman tree description, then palette data and finally there’s compressed RLE data. The originality lies in the details: Huffman coding is often used to code single bytes while here single symbol represents both run length and value (also IIUC zero value signals skips), additionally the data is stored as the list of symbols for each codeword length. I don’t think I’ve seen anywhere neither RLE+Huffman used in such manner nor such tree description format.

This was a fun distraction even if I don’t care about the game series (I tried playing their other games like Countdown and Amazon: Guardians of Eden but the interface was too clunky and the action sequences were too annoying). It’s always nice to see some originality in video coding even if it is nothing advanced.

Posted in Game Video | 2 Comments »

rv4enc: limping forward

Tuesday, April 18th, 2023

So far there’s not much to write about: I’ve dealt with the second most annoying thing (writing coefficients) and now my encoder can produce a valid stream of I- and P-frames. Probably it’s a good place to define a road map though.

First of all, there’s still the most annoying thing pending—in-loop filtering. And I can’t reuse that much code from the decoder since it was written back in the day when my knowledge of Rust was even less than now, so I have to partly rewrite some of the parts to make them fit my current approaches to the interfaces (that means a lot of DSP functions among other things). At least it can be disabled for now but I’ll have to return to it sooner or later.

Then there’s 4×4 intra prediction mode still waiting to be implemented. Again, I know how to pick the good enough prediction modes, it’s context-dependent coding of those modes that is going to be annoying.

Another thing missing is estimating the number of bits required to encode a single block. There are about five codebooks involved and those are selected depending on macroblock type, quantiser, coefficient block type (luma, chroma or luma DCs) and the global set index. I guess I’ll resort to gathering statistics for all possible coding modes and seeing if I can make some heuristic estimation out of it. And there’s still a task of selecting the best coding set for a slice…

After all that it will be mostly B-frame related things left to implement. I also need to make sure the muxer will write them out properly (for historical reasons in that case it mangles the timestamp to be last frame timestamp plus one). That’s not counting all possible enhancements like deciding the frame type for coding. Most likely I’ll be annoyed by it and keep the fixed coding order instead.

There are too many things to do and considering my pace it may keep me busy for a good part of the year—I mean a large part of a calender year, the current date is still February 24.

Posted in rv4encfail | 3 Comments »

Starting yet another failure of an encoder

Thursday, April 6th, 2023

As anybody could’ve guessed from Cook encoder, I’d not want to stop on that and do some video encoder to accompany it. So here I declare that I’m starting working on RealVideo 4 encoder (again, making it public should prevent me from chickening out).

I can salvage up some parts from my VP7 encoder but there are several things that make it different enough from it (beside the bitstream coding): slices and B-frames. Historically RealMedia packets are limited to 64kB and they should not contain partial slices (grouping several slices or even frames in the packet is fine though), so the frame should be split during coding. And while Duck codecs re-invent B-frames to make them still be coded in sequence, RealVideo 4 has honest B-frames that should be reordered before encoding.

So while the core is pretty straightforward (try different coding modes for each macroblock, pick the best one, write bitstream), it gives me enough opportunity to try different aspects of H.264 encoding that I had no reason to care about previously. Maybe I’ll try to see if automatic frame type selection makes sense, maybe I’ll experiment with more advanced motion search algorithms, maybe I’ll try better heuristics for e.g. quantiser selection.

There should be a lot to keep me occupied (but I expect to spend even more time on evading that task for the lack of inspiration or a sheer amount of work to do demotivating me).

Posted in NihAV, rv4encfail | 4 Comments »