Kostya's Boring Codec World

So while russia demonstrates once again what a hysterical führer and his similarly minded generals would do, I’m trying to do something to distract myself from the thoughts about the losses Ukraine suffered in last couple of days (and previous couple hundreds of days too).

Accidentally I’ve managed to find a sample in IVF format—the original streaming format for Indeo codecs. Despite the expectations it was not a renamed AVI file which made it even more interesting. So I took Ghidra and binary specification and started implementing it.

The format by itself is rather simple and the only interesting thing is that it transmits video data in passes: first it’s just bare minimum keyframes and all other frames as drop frames, then some inter frames are sent, then more data is sent and more. Unfortunately there’s no obvious field to tell where each part starts so I simply assemble frames in memory (which is not effective but still better than the original creating temporary files for assembling fragments).

Then there was a problem with decoding them: my existing decoder expected a frame with sequential structure while in this case fragments are transmitted out of order. Normally you’d get all four bands for luma plane and then a band for each chroma planes but in this case it may be one or two bands for luma plane, then chroma plane bands, then some zero padding and then the rest of the bands for luma (if they were transmitted at all). In the reference implementation the demuxer seems to parse and re-assemble the frame while I ended up writing a slightly different decoder for this scalable mode. It seems to work fine so now NihAV support for Indeo family of formats is even more complete than ever.

P.S. The Wiki had only the entry for Duck test samples format (that has the same extension) so somebody had to correct that. Which also makes me think about things like Duck AVC, AVS being an FMV game format or Chinese non-Duck AVC rip-off, IEEE AAC and so on. This confusion deserves to be written about but don’t expect me to do that.

The war has shifted to a terrorist operation against Ukrainian civilians (with no change for my home city, it gets several strikes from russian territory every day regardless) and instead of threatening the world with nuclear war russia threatens the world with nuclear terrorism using the captured Ukrainian nuclear power plant. So here’s yet another attempt to distract myself from thoughts about it.

It seems that AVS3 has been standardised already (and nobody cares). So out of idle curiosity I’ve downloaded the spec from avs.org.cn (in Chinese of course, and it requires you to fill some information but accepted any garbage). So let’s look at this completely original format that has not borrowed anything neither from H.266 nor from AV1.
(more…)

I wanted to reverse-engineer and implement some wavelet codec just for the sake of it. And finally I’ve managed to do that.

Initially I wanted to finish Rududu Video codec (I’ve looked at it briefly and one of the funny things is that the opensource release of Rududu Image codec does not match the actual binary specification, even arithmetic coder is different), but it turns out there’re no samples in the usual place so I just picked something that has some samples already.

The codec turned out to employ some tricks so I had to resort to collecting debug information in order to understand band structure (all band dimensions are implicit, you need to know them and the order to decode it all successfully). Then it turned out that band data is coded in boustrophedon order instead of the usual raster scan. And finally there’s fun with scaling: vertical transform is the same as horizontal one but the output is scaled by 128. Beside that it’s rather unremarkable.

Anyway, I got slightly deeper knowledge about the inner workings of wavelet codecs and it should not bother me any longer. It’s time to slack off before doing something else.

A recent question about buggy SMC decoding led me deep into QuickTime specification to look at the codec missing opcode. And there are some noteworthy things here as well.

Back in the day there was the multimedia player for Unix called XAnim. Its last release was in 1999—before other opensource multimedia player projects have started! It was both feature-rich (e.g. it could step frames forward and backwards, something that not all current media players can do) and had an excellent codec support for the time.

Somehow its author reverse engineered (long before the era of decompilers too) a lot of codecs and somehow managed to obtain the sources for e.g. Indeo and while he could not provide them, he offered them for a wide variety of architectures—Alpha, MIPS, Sparc, PowerPC, x86. It was a treasure trove for formats and lots of the decoders were ported to other projects (even I did that for one or two codecs) and binary codecs were a great help in reverse-engineering efforts as well.

Now to SMC itself. Formally it’s QuickTime Animation codec but people call it after its FOURCC which is “smc “, probably after the author’s initials.

Opensource SMC decoders come from the same source (I based mine on the description in The Wiki but you can guess what that description is based on; and yes, back in the day e.g. MPlayer and Xine had their own decoders for various codecs before relying on libavcodec for everything). After looking at the binary specification I can say it looks exactly like it was reverse engineered from it directly (it has the same logic and data types but lacks sensible names). Anyway, the thing is that it does not handle opcode 0xF0 and I finally had an occasion to look at it.

I took QuickTime 6.3 binary specification for Windows (somehow the decoder ended in QuickTimeInternetExtras.qtx) and looked inside. It turns out that there are several decoding functions there (for different output formats) but they all do the same: handle 0xF0 opcode in exactly the same way as 0xE0 opcode (raw blocks), there are no differences there whatsoever.

That’s one mystery less, even if the answer is a bit disappointing. At least I could reminisce about good old times hardly anybody else remembers.

Since the time I first looked at XVD-related codecs I dug deeper and at one point considered implementing it all for NihAV. But every time I look at Muzip or some logic inside video decoders I lose all interest. So finally I’ve documented my finds on The Wiki and now I can forget about it and move to something else.

Some of it was easy to investigate since VGM demuxer along with Muzip CTP06/CTP07, Domen and VT2k decoders are present in Java applets that can be easily decompiled. Some like V2K-II or XVD can be easily decompiled with Ghidra and produce mostly understandable code (except for wavelet decoding part in XVD). Muzip4 and VT on the other hand have hard to follow logic. And VGM2 demuxer is available only as DirectShow splitter which is a pain to search for the COM object responsible for the demuxing itself.

Funny enough now Alaris VGPixel looks more related since VT codec has similar mode of compression. Additionally both the official player and demo programs from VGM-XVD developer site use the same trick—they put all .dlls in compressed form (the standard SZDD compression) at the end of executable, which decompresses and loads them at start.

Also it’s worth mentioning that all decoders (except for VGPixel) have the same interface via the functions UCF_InitCodec, UCF_ProcessFrame and such. Anybody interested enough can write his own program that demuxes VGM or VGM2, loads the proper decoder libraries and does something with the result. At least I’ve documented it as much as I could (or cared) so there’s some foundation to start from.