Back in February I wrote about my failed attempt to write TrueMotion 1 encoder. And since I was bored and really had nothing better to do, I tried my hoof at it again.
Last time it was 24-bit encoding, now I tried to approach 15-bit encoding instead and got some results. I guess the moral of the story is that you should not overthink it and use the simplest approach to coding.
(more…)
Since I still have nothing better to do, I decided to look at some obscure video codec I had laying around for really long time. And it turned out to be simple yet rather original.
Unlike many other codecs, this one codes YUV 4:2:2 (at least it looks like that) line per line in chunks of 24, 16 or 8 elements (essentially 24-pixel chunks plus shorter tail and the line width being a multiple of eight). Each chunk can be coded using one of four modes (leave it as is, decode and apply delta, copy chunk from the previous line with or without delta). Delta values are coded as chunk quantiser, delta values (-q, 0, q, escape) plus escape values. And since those mode/delta values can fit into two bits, they’re packed together into 16-bit words.
If you look at it, there’s nothing really inventive: short slices are present in many codecs, lossy delta coding is common too. But together they create a combination that I’ve not seen anywhere else. And that’s why looking at older codecs is pleasing: beside seeing rip-offs of the standard codecs sometimes you also encounter such original ideas as well.
I was asked to look at it and since I have nothing better to do, I finally did.
OSQ was a format used by WaveLab mastering software for storing lossless compressed audio (they seem to have dropped support for it in the more recent versions). Internally it’s a clone of ages-old Shorten which does not even have LPC mode, only fixed predictors.
Of course the format somewhat differs and there are more prediction modes but they still use maximum three previously decoded samples and filter coefficients are constant. There are other original additions such as stereo decorrelation or adaptive Rice coding with the parameter selected based on previous statistics and some floating-point calculations that look more complex than they should be. Overall though it looks like somebody took Shorten and decided not to do the hard part (LPC), maybe in order to make files to save faster, and tried to compensate it by a couple of simple tricks.
I’ll probably document it at my leisure but overall it’s a rather silly format.
Since OSQ format requires obtaining a copy of some expensive software of unknown old version, I’ll leave it to somebody else. Meanwhile I’ve looked closer at the AVC format mentioned in the previous post and its relatives.
For those of you who don’t recognize the name immediately, Digital Pictures is the company responsible for some FMV-based action games, including the infamous Night Trap. As I rediscovered previously, the AVC files they use are really SGA files with compression method 0x81 but what about the other formats?
About half of the games I could look at contain an archive occupying most of the CD space, inside that archive are the same AVC files. Other half of the games usually has one or two AVC files with a company logo and one megamovie in various formats. And after some research it turned out to be the same 0x81 compression format but with audio data and varying headers.
And since nobody bothered to document it for The Wiki, I’ll explain it here.
(more…)
Since I have nothing better to do as usual, I decided to look at some game formats.
For instance, there’s a game called The Fuel Run promoting a product from a Swiss company supporting russian war crimes. This game has animations in VDO format. VDO file header starts with the string “Artform” and it employs RLE compression, each line of the frame prefixed with its size. What’s funny is that its RLE uses only bottom 6 bits for the run length and top bit is completely unused. For obvious reasons the format is not worthy documenting further.
Or there’s a game called Double Switch, this one has AVC videos. It uses RGB555 palette and 8×8 tiles that depending on opcode may be either coded with one of 25 predefined patterns and 1-7 colours or split into tiles of the smaller size. And only afterwards I decided to look into The Wiki and it seems to match SGA format description (except that this particular format variant is not documented). I don’t know if I should bother writing a decoder for it but with the lack of PC codecs to RE I might try my hoof at console ones instead.
Since a certain friend of mine keeps asking what features I need in Rust and then forgets the answer, here I decided to write it all down. Hopefully it will become outdated sooner than later.
And I’d like to start with some explanations and conditions. I develop a certain multimedia project so I have certain common flows (e.g. processing 16×16 macroblocks in a frame) and I’d like to be able to optimise for them. Also I do not like to use the nightly/unstable version of Rust (as those unstable features may take an extremely long time to hit stable and they change in the process, as it happened to asm!{} support to give one example). And finally I do not accept the answer “there’s a crate X for that”—out of design considerations I prefer to avoid external dependencies (short explanation: those get out of control fast; my encoder and player projects depend only on my own crates for doing everything but the player additionally pulls sdl2 dependency and suddenly it’s 33 crates instead of 19; IIRC with a newer version of sdl2 crate the total number gets to fifty).
Anyway, here are the features I miss with some explanations why that should be relevant probably not just to me.
(more…)
So I decided to look at yet another game at archive.org with an FMV tag. This time it was The Lawnmower Man. It had NXL format that should be a rather unremarkable container with raw video and audio except for one fact: it is EGA full-motion video (320×200, 16 colours, custom palette per each frame). It is the first time I see video being stored in planar form (i.e. for each line first the lowest bits of each colour index value are stored, then bits 1, bits 2 and finally bits 3). Now consider that the format is big-endian (even if DOS version came out reportedly earlier than Mac or SEGA CD versions) and it feels positively weird.
Some of the details I bothered to document are at the usual place (somebody should keep The Wiki alive beside Mike after all).
Sometimes I watch reviews of various old video games (usually of adventure type but not necessarily). This time it was Conquest Earth RTS from 1997 and the reviewer said magic words: “I was not able to extract or play videos with anything”. Okay, you got my attention.
I could find some .rpl files there using Escape 124 codec (yes, Eidos was the publisher, how could you tell?), some standalone .flh files and some flicN.wad archives. Despite the name, the archives turned out to have a footer, i.e. all metadata is stored at the end for a change. The video files seem to be some .flc but mostly .flh.
As one could reasonably expect, it turned out to be yet another hack of the old venerable FLIC format, now with RNCv2 compression. I did not look further to find out whether it’s raw frame or RLE-compressed data there but it does not matter much as the main features are already discovered.
There are many various FLIC extensions, like high(er)-bitdepth video, custom RLE coding schemes and even audio support. And that’s not counting the fun things like FLIC-in-AVI or FLIC-in-AVI. But I leave documenting all those format variations to The Multimedia Mike, gaming and documenting FLIC are his passions.
I’ve remembered that back in the day there was an encoder for RoQ format (the format that uses a codebook with 2×2 YUV vectors, what a coincidence!) called Switchblade and it was using NeuQuant before it was integrated into FFmpeg where it started to use ELBG. So I decided to give it a try.
If you have forgotten, NeuQuant is an application of Kohonen neural network to the task of generating palette for an image. I’ve implemented that kind already so I tried my hoof at adapting it for a larger vector size. Good thing: it works and it’s reasonably fast (2-3 times slower than median cut, faster than partitioned ELBG—and that’s the code that uses doubles for the majority of its calculations). Bad thing: the result quality is mediocre. The results obviously can be improved by adjusting various factors (wait, am I talking about neural network or string theory?) and changing the pseudo-random order in which the candidates are sampled but I don’t feel enthusiastic about tweaking all those parameters and see which ones work good for the widest selection of video sequences.
So I’m drawing a line here. It was a quick and failed experiment, I should find something better to do.
Since I had nothing better to do I was browsing FMV games at archive.org and in one of them I found rather peculiar sample: avconv has wrong palette for the first half of it and nihav-tool has wrong palette in the second half of the clip. And I thought that MOV is not supposed to have palette changes at all.
It turned out they used a multiple sample descriptors trick: it’s possible to provide several codec descriptions for one track and use one or another for different frames. That file has two descriptors for the video track with different palettes. Mystery solved.
And it also solved another mystery with a different file from that game where some frames are not decoded properly. It turned out that it also has two sample descriptors for the video track: one is A**le Graphics and another one is Cinepak.
Back in the day I ranted that MOV is too flexible and this proves once again how true that is. Good thing I don’t have to care about supporting such files properly.