Kostya's Boring Codec World

Vad kan man göra på nyårsafton om han dryck inte alkohol och han är FFmpeg developer också? (Translation: what can man do on evening before New Year if he does not drink alcohol and he is FFmpeg developer?).

Naturally, I spent it hacking at some codec. One of the most annoying issues with it (at least for some people like me) is that it does not support some features, like interlaced VC-1 and decodes only I-frames in old version of WMV3. So I’ve tried to fix the latter.

There is a flag called RES_RTM which tells whether it’s final bitstream format (a.k.a. “release to manufacturer”, hence the name) or not. I’ve tried tracing its effect through binary decoder and it turned out that it only alters transform pattern decoding.

Here’s a brief outline of transform concept: WMV2, WMV3 and VC-1 may partition P or B blocks into 4×8, 8×4 or 4×4 subblocks; each subblock, obviously, may be coded or not, so decoder also needs to extract subblock coding pattern unless transform type specifies it (like “8×4 transform, left half coded”).

It turns out that older version uses different condition for subblock coding pattern reading (for 8×4 and 4×8 transforms only), without checking whether this is transform specified only for current block or for whole macroblock (i.e. group of 6 blocks). Quick hacking has not made FFmpeg decode those old file correctly though, so there is still some work left to figure out correct condition.

To my great surprise there are people working with reference decoder and asking for files with some features and digestable by it. Well, here is an ultimate answer – RCV muxer for FFmpeg (hey, writing a muxer and patching FFmpeg build system were two things I haven’t done yet).

You can get FFmpeg sources, apply this patch, compile and try ffmpeg -i file.wmv -vcodec copy out.rcv. Good luck!

BIG FAT WARNING. It is not guaranteed to work with reference decoder on all files. Try files with smaller dimensions.
Or hack reference decoder to ignore image dimensions (ffplay would play those files correctly while reference decoder complains on something about “image size is too big for this level”).

Now we have X8intra frames support in ffmpeg! Mike has already expressed his joy in his blog. I think anime fans who tried to play WMV3 in AVI would also be glad.

Why this has not been done earlier? Well, sheer disgust. The person who invented this scheme should be either fired or promoted to M$ CTO. Here is the list of reasons:

• It is used for some keyframes coding so it cannot be skipped
• Design is totally unlike anything standard
• You should perform bitexact decoding. If your DCT produces slightly different results or you forgot about loop filtering then you won’t be able to decode picture properly. Hey, that’s utter crap by _any_ standard
• It has made it to WMV3 too.

To put it mildly, X8Intra is an illegal offspring of JPEG and some early H.264 draft. It is mainly Huffman-coded 8×8 DCT-transformed blocks with spatial prediction and loop filtering. It does not have macroblocks like decent codecs. Spatial prediction has some directions and relies on previously decoded blocks and bits read depending on that. I have not seen anything with such a bad bitstream-transform dependency (in other codecs you can decode coefficients first and then perform image reconstructions but not here). X8Intra excrementum est(pardon my Latin).

Still I very grateful to the person named “someone” who did this. Otherwise I should clean this cesspool.

Looks like the best way to collect a lot of samples is to become a developer and maintain demuxer/decoder. You may receive a lot of samples that do not decode with it 😉

I’d like to thank three major contributors (from Dania, Netherlands and Japan) of VC-1 in HD-DVD/Blu-Ray samples. It is because of them VC-1 video playback is less buggy than it was before (I know it still is but at least it’s improving).

This and last weekend I fixed some bugs in my decoder so now watching HD-DVD/BluRay movies will be a bit more pleasant ;-). But for the whole watching and listening experience you should wait for E-AC3 decoder (maybe it will be implemented during Google Summer of Code).

As for me, my systems neither are fast enough to decode it in realtime (1.42Ghz PPC G4 and PII 266Mhz) nor can even display full picture (max. resolution is 1280×1024 but I’m fine with it). So you may conclude I’m not interested in watching HD and you will be right. But I care for decoder and will fix bugs in it as I do already.

I’ve just added some support for WMV3 Complex Profile (aka old Advanced Profile).
I don’t know why most samples (users complaining about) I’ve met are anime. It’s a good chance they will work now.

And here is updated list of CP features explained:

• RES_X8 – additional bit at the end of I-frame header pointing if this is normal I-frame or this is special coded frame (I’m sure that’s exactly WMV2/8 J-frame with different header)
• RES_FASTTX – somehow interferes with P-frames but bitstream is not changed
• RES_RTM – old P-frame format (header should be the same but some additions on macroblock level). This is also quite common in old WMV3 files. Not RE’d yet

Now FFmpeg and MPlayer support VC-1 in MPEG2-TS streams which can be found on Blu-Ray discs.

Thanks to Martin for providing sample and Nico Sabbi for implementing handling of this in both FFmpeg and MPlayer.

For those whom it may interest.
What are the things needed to play HD-DVD movies in open-source multimedia player?

• Decrypted movie (use this tool)
• Demuxer support
• Video decoder
• Audio decoder

EVO demuxer is really just slightly modified MPEG stream demuxer. FFmpeg and MPlayer fresh SVN versions support it (MPlayer was there first). I don’t know about other multimedia players but that’s quite easy to implement.

Video decoding: there are three possible choices – MPEG-2, H.264 and VC-1. First two should work fine, the last one should work but with some bugs (I will fix them eventually).

Audio decoding: is also tricky – standard AC3 audio will work, DTS is supported via libdca and E-AC3 is still to be figured out.

Conclusion – try current SVN of MPlayer and see if it works to you. And if you care about Blu-Ray support (and it does not work) – please provide a sample.