Kostya's Boring Codec World

See this picture?

I don’t know why it appears in quite a few VC-1 Advanced Profile official test samples (the rest I could decode are football trainings as in SP/MP samples).

Yesterday I’ve written VC-1 Elementary Stream demuxer and now have enough material to test interlaced decoding and other AP features (when I implement them).

And another task – make work VC-1 embedded in .EVO – official format of BlueRay/HD-DVD.

Known fourccs:

• WMV3
• WMVP
• WVP2
• WMVA
• WVC1

Known meaning of flags:

• RES_SM: really that’s two flags, RES_SM&2 indicated 411 interlaced mode and RES_SM&1 indicates sprite mode(or so it seems). These flags are mostly used by WMVP and WVP2
• RES_X8: I-frames may be packed with X8 algo. This also means that each I-frame has additional bit in frame header signalling if current frame should be encoded with X8 or with standard encoding scheme. Used in WMV3 Complex Profile.
• RES_FASTTX: still unknown, somehow interferes with motion compensation (at least it does not affect bitstream)
• RES_TRANSTAB: each macroblock should have its own DC/AC table index, not one global stored in the frame header.
• RES_RTM: Old version of WMV3 with different P-frame coding mode

X8 frames use their own Huffman codes (code lengths are stored as nibbles somewhere in bitstream) and also blocks looks like to be decoded in this way:

for(y=0;y

for(x=0;x< mb_width*2;x++){

decode luma block

if( x & y & 1)

decode both chroma blocks

}

}

Ever heard of codec with fourcc WMVP? That is another Windows Media variant designed for slideshows, video albums, ets. Why I have got interested in it? WMV3 Complex Profile (aka Ye Olde Undocumented and Incompatible Advanced Profile) uses exactly the same coding method to store I-frames when flag RES_X8 is set (if it is not set then VC-1 Main Profile decoder may decode that movies).

While standard decoder output mostly garbage, some blocks are decoded correctly thus I hope it’ll be possible to add support for both WMVP and WMV3 CP.

OK, Now B-frames for simple and main profile work. Now there are only some exotic features left to implement as spatial downsampling and sync markers. And a lot of RE’ing to implement old WMV3 P-frames decoding and complex profile.

As for advanced profile, B-frames are still not implemented.

For those who interested, here is a very experimental patch to enable decoding of WVC1 codec in FFmpeg. I’m currently searching and downloading sample files but it is already tested a bit.

Instructions:

1. Get FFmpeg sources (r614x should work)
2. Download and apply patch (cd ffmpeg; patch -p0 < vc1-advanced.patch)
3. Run configure and make

As always, non-working samples are welcome.

If you look in VC-1 standard specification it’s hard to find header specification for Simple/Main Profile (at least I was unable to do so in comittee draft available – only some hints in bitstream container format). Was that intensional or not?

FFmpeg old and not working vc9.c contained header parsing code and reference decoder does this too. This header contains a lot of ‘reserved’ flags which hidden meaning become clearer if you meet with them:

• RES_RTM_FLAG – should be set to ‘1’ but all old WMV3 files have it set to zero. For now you can call it ‘P-frames will be decoded correctly’ flag as old WMV3 has I-frames decoding identical to standard
• PROFILE=2. This is not allowed profile but in vc9.c it was called “Complex Profile” and really is Advanced Profile for old WMV3. I don’t know why but one sample was decoded when I changed profile value to “Main”, other Complex Profile samples produced garbage (but they also had RES_X8 and RES_FASTTX flags set).

There is also new M$ codec – WVC1 which is not decoded by reference decoder.

So here is a small list of huge tasks:

1. Find out how to decode P-frames when RES_RTM_FLAG=0 thus enabling decoding of many old WMV3 videos
2. Find out how to parse WVC1 movies
3. Support Complex Profile (not really important)

Some notes on forward transform.

For 4×4 transform matrix is:

17 17 17 17
22 10 -10 -22
17 - 17 - 17 17
10 -22 22 -10

And suggested norm is (8/289 ; 8/292; 8/289; 8/292)
The forward transform for (A B C D ) will be:

A1 = (17 * A + 17 * B + 17 * C + 17 * D) * 8 / 289;
B1 = (22* A + 10 * B - 10 * C - 22 * D) * 8 / 292;
C1 = (17 * A - 17 * B - 17 * C + 17 * D) * 8 / 289;
D1 = (10 * A - 22 * B + 22 * C - 10 * D) * 8 / 292;

Of course, don’t forget to round result.
I tested it on Octave and it proved to be correct.

Optimization is also straightforward: compute (A+D) , (B+C), (B-C) and use their sums/differencies.

Yesterday I finished B-frames parsing support and to my surprise all of B-frames in test streams were decoded correctly even without motion compensation. I don’t know why all of them are composed from intra-blocks only. And they are not MS-invented BI-frames either.
While I understand that’s legal I still cannot understand why it was done so.

I’ve written a small and quick demuxer for VC-1 Simple/Main profile test bitstreams (source available Here).

Most of the samples are recordings of Football training. You can see “Windows Media” logotypes on some players’ uniform.

And rarely clips with some singer occur.

Anybody can guess why?