Kostya's Boring Codec World

I remember playing game “Lost Vikings” by Silicon&Synapse (who was later renamed to Blizzard).

Lost vikings (again) and additional creatures. Endgame scene.

Who knew that it had a sequel? It was done by Beam Software and looks less appealing to me than original. But it has cutscenes and audio in its own format, which makes it a bit more interesting. A contributor, who wishes to remain anonymous, sent me his description of this format so I was able to implement demuxer and decoder for their video and it will be committed soon (I hope). Details of this format are already here.

As a follow-up to the theme set by Michael here are my few random shots of flowers (made in the Western Ukraine, several thousands kilometers from my home).

(more…)

Extremely big thanks to Mike, who sent me a dozen of DVDs with samples and they eventually got delivered (Roberto Togni also had sent me some disks with samples but they disappeared somewhere). It took about a week to deliver them to Ukraine. And a whole month to pass customs control.

But nevertheless it’s here and now I have enough samples to test my future and current decoders.

Here I want to present my thoughts on decompiler techniques I’d like to see. Maybe a lot of this is implemented somewhere but I haven’t seen working decompiler.

• Possibility to load disassembly instead of disassembling by itself.
• Good flow analyzer. REC, for example, produces a lot of silly gotos. Is it so hard to build directed graph for blocks, separate out conditional code and loops? IDA does so. And it’s pretty easy to recognize typical schemes like do{}while;
if(){}else{} and detect break; in loop.
• Watching ‘live’ registers. Each instruction may affect some registers and flags but some of them won’t be needed later (for example, sometimes substraction is used to modify some value and sometimes also result flags are checked too). And block of instructions may depend on some register values set before (if they are not modified before using). Boomerang had something like this but resulting code was too LISPy.
• Reiterations – if decompiler finds out that function uses registers for passing parameters then code must be changed to reflect this.
• Pattern recognition – it would be very nice if decompiler could recognize the same patterns over the code (in form: A = B+ constant; B = A | constant; ). And if it also could automatically label bitreading functions… But I fear that this is AI-complete problem.

Well, my rant ends here. Back to work.

Let’s see what I have done in 2006:

1. Fraps (FPS1) version 2/4 decoder
2. Intel Music Coder decoder (some cleaning and lavc’ization)
3. KMVC (video codec for FMV used in Worms games) decoder
4. Musepack SV7 demuxer and decoder
5. Smacker demuxer and decoder implementation (no, I haven’t RE’d it)
6. VMWare capture codec (VMnc) decoder (implementation and some RE work)
7. Wavpack decoder
8. ZMBV (DosBox capture codec) decoder and encoder (encoder for palettized format only)
9. Some image formats reading support (TGA, TIFF)
10. 16-bit grayscale support for some image formats
11. And finally – VC-1 decoder (with some WMV3 support)

And here are my plans for this year. Warning! This is not the final list and may be changed in process.

• Make some lossless audio decoders (MPEG-4 ALS and Monkey’s Audio)
• Implement missing VC-1 Simple/Main profile features (there are few of them which nobody uses but who knows)
• Implement VC-1 Advanced profile interlaced mode (looks like it will be needed in future)
• Help with some other projects – DCA implementation and maybe even finish RE for Xan v4 (looks like Mike forgot to do this)
• And ultimate goal – take participation in GSoC 2007 and write JPEG-2000 decoder (in very distant future – even encoder)

Here is a quick list of what has been done recently:

• LZW decoding in TIFF
• 16-bit grayscale limited support (decoding PNG, read/write PGM and JPEG-LS but the latter is a bit buggy)
• Some progress on Intel Music Codec decoder

Ah, I still remember those days when FFmpeg got one new decoder once week or two.

Mike once mentioned here that one coder searches for ideas to implement. My situation is rather complimentary – I hardly manage to fullfill all requests from other people. Recently I’ve added two image decoders to FFmpeg – Targa (.tga) and TIFF. Both of them were done by request. The same situation was with Worms Video.

WavPack decoder was written by my own wish but there are already some people asking to write MPEG-4 ALS and Monkey’s Audio decoders. While the later is not possible to implement right now, I’ll implement both MP4 ALS decoder and encoder.

I just had to express that. Recently I’ve opportunity to examine two DVD disks – one with 8 films-in-one and another one is 7 films-in-one (our pirates usually pack at least two films on each disk). Every DVD has a proud label ‘Real DVD Quality’ (but sometimes even VHS tapes are marked so).
As expected, those films had slightly lesser resolution than 720×576, one disk had 352×576 films, but another one… 352×288 !!! If disk was labeled ‘Real VCD Quality’ – that should be fair.

I’ve committed today another decoder – for KMVC codec, used in Worms games. This is the final step to the completion of my task.
Multimedia Mike has a simple goal – to have a player which will play any multimedia file you like. My task for now is much simpler – have a decoder for each letter of alphabet.
With the latest addition there are only two letters left: E and Y. But considering that TSCC is now called Ensharpen and only known formats on Y are YUV variants, I declare this task complete. Future RE’ing will be just for fun 😉

Here is a list of codecs I reverse engineered in some way.

Codecs RE’d by format analyzing

• Apple QuickDraw
• Autodesk RLE
• Sierra On-line Audio
• TechSmith capturing codec (aka Ensharpen)
• Pinnacle/Miro XL

Most of those are RLE variants (actually 2 of 3 are MS RLE variants).

Sources->Description->New Code

• LOCO codec
• Duck TrueMotion 2
• IBM Ultimotion
• Winnov WNV1

The last one is re-created from tuner driver.

Binary RE’d

• Creative ADPCM
• Intel Indeo 2
• Q-team QPEG
• DSP Group TrueSpeech