Well, this information should’ve been posted by someone else but those people seem to be lazier than me. In return I’m not going to use XViD or FLIC for encoding my content.
So, REing VP4 is rather easy – you just download original VP3.2 decoder source (still available at Xiph SVN servers) and compare it to the structure in vp4vfw.dll. There are differences in structures and a bit in code layout but mostly it’s the same code with new additions.
So, VP4 is based on VP3 (surprise!) and introduces a new bitstream version (which is 3 for some reason). Here’s an incomplete list of differences I’ve spotted:
- Base frame header has some additional fields (I didn’t care enough to decipher their meaning though);
- Superblock coding uses a bit different scheme with new universal codes resembling exp-Golomb but with VP4 quirk;
- Frame data decoding differs for frame types;
- Motion vector component extraction uses Huffman tables and sign from the previous block.
And yet it uses the same coding principles and even token coding seems to be left untouched. It was suspected for a long time that even-numbered On2 codecs were simply an improvements over previous version while odd-numbered On2 codecs were more innovative but not much was known about VP4 to prove it:
- Duck TrueMotion 1 — a new codec;
- Duck TrueMotion 2 — mostly like TrueMotion 1 but with Huffman encoding;
- Duck/On2 TrueMotion VP3 — DCT + static Huffman coding;
- On2 TrueMotion VP4 — VP3 with some bitstream coding changes;
- On2 TrueCast VP5 — DCT + arithmetic coder;
- On2 VP6 — VP5 with some bitstream changes;
- On2 VP7 — H.264 ripoff with their own arithmetic coder;
- On2 VP8 — VP7 with some small changes;
- Baidu VP9 — H.265 ripoff with their own arithmetic coder;
- rumoured Baidu VP10 — since there’s no H.266 in the works for now…
It’s all kinda Intel CPUs but without confusing codenames (and Xiph hasn’t produced too many codecs to confuse whether Daalawell came before Theorabridge or after).
P.S. Many thanks to big G for releasing no information on that codec or any other codecs from On2. Oh, and is VP9 “specification” still under NDA?
P.P.S. I should really work on a game codec named after chemical warfare instead.
Nice. Glad someone finally got down to doing this since I will probably never get around to it.
I don’t think we can blame Google for not releasing details on old codecs like VP4. When I met a bunch of the former On2 people in 2010 after they had joined Google, I asked them about the possibility of releasing VP4. They said that it was unlikely that they could even locate the source code for the codec anymore. Knowing how software organizations operate, I have no reason not to believe them.
Good idea regarding differential analysis between the VP3 and VP4 DLLs. How did you do it? Straight binary comparison and identifying common functions between the 2?
Also, if you examine the released VP3 code, there are some unused portions. IIRC, there was unused code for even more insane DC prediction algorithms. I always assumed that was a prototype for what would could in VP4.
Nah, I’ve not looked at VP3 binary decoder, I just saw that VP4 can decode VP3 as well, assumed that it’s based on the same source code and it turned out to be right. Of course there are differences but overall it’s close enough. In most cases you simply have inserts in form if (pbi->version > 2) …
Oh, and when someone asked those people to release VP7 sources they refused it on the ground it takes time and they’d rather work on improving VP8. I heard they had lost their old source code but equally they can be not even trying to locate it at all.
i asked the big Godzilla to release on2 codec source code, response was “not now, too busy with Godzilla SoC or VP8 or (insert project here) and “dont want to release source that may contain secrets or lies”.
oh well.
Awesome work Koysta. Your theory about odd-version numbers checks out! I was secretly hoping VP4 would be different. 哦我期待百度VP10哈哈