Kostya's Boring Codec World

I just had to post this – our Philharmonic presented own harpsichord. Several years ago when I first visited it I’ve listened to concert music with harpsichord but it was borrowed one and there was nothing comparable since that time.

The presentation went well and we were enjoying different music – from sonatas by Handel and Telemann, Johann Sebastian Bach concertos to Mozart to jazz improvisations and modern Ukrainian music (well, when composer plays on one of the instruments himself I consider it modern).

Looking forward for further listening (with hope that it would take less than a couple of years of waiting).

I presume those interested in multimedia coding have heard of “Data Compression: The Complete Reference” by David Salomon. Personally I consider this book very good but maybe we should write our own book concentrating on multimedia only. Why? I have not seen books where video (and audio) compression is not merely outlined (like in most books on general data compression) or is not solely dedicated to some standard
(MPEG usually).

I gladly remember this book as it’s quite outdated but at least it covers many codec, container and even implementation issues (unfortunately, sound only))!

My proposal for book outline:

• General multimedia concept (pixels, samples, PCM, DCT)
• Audio compression
  • Simple time-domain codecs (DPCM, ADPCM)
  • Complex time-domain codecs (lossless mostly)
  • Speech codecs
  • MDCT-based codecs and friends
  • How to write a fast decoder and good encoder (or otherwise)
• Image Compression
• Video compression
  • Lossless coding
  • Game video codecs (who will write this?)
  • Modern standard and non-standard codecs
  • Implementation tips and tricks
  • Known codecs (implementation-wise) overview
• Containers
  • Why making codecs dependent on custom container is idiotic 🙂
  • File-based containers
  • Streaming containers

And I know where to get information ;-). Well, let’s see if this catches up.

I remember playing game “Lost Vikings” by Silicon&Synapse (who was later renamed to Blizzard).

Lost vikings (again) and additional creatures. Endgame scene.

Who knew that it had a sequel? It was done by Beam Software and looks less appealing to me than original. But it has cutscenes and audio in its own format, which makes it a bit more interesting. A contributor, who wishes to remain anonymous, sent me his description of this format so I was able to implement demuxer and decoder for their video and it will be committed soon (I hope). Details of this format are already here.

As a follow-up to the theme set by Michael here are my few random shots of flowers (made in the Western Ukraine, several thousands kilometers from my home).

(more…)

Extremely big thanks to Mike, who sent me a dozen of DVDs with samples and they eventually got delivered (Roberto Togni also had sent me some disks with samples but they disappeared somewhere). It took about a week to deliver them to Ukraine. And a whole month to pass customs control.

But nevertheless it’s here and now I have enough samples to test my future and current decoders.

Here I want to present my thoughts on decompiler techniques I’d like to see. Maybe a lot of this is implemented somewhere but I haven’t seen working decompiler.

• Possibility to load disassembly instead of disassembling by itself.
• Good flow analyzer. REC, for example, produces a lot of silly gotos. Is it so hard to build directed graph for blocks, separate out conditional code and loops? IDA does so. And it’s pretty easy to recognize typical schemes like do{}while;
if(){}else{} and detect break; in loop.
• Watching ‘live’ registers. Each instruction may affect some registers and flags but some of them won’t be needed later (for example, sometimes substraction is used to modify some value and sometimes also result flags are checked too). And block of instructions may depend on some register values set before (if they are not modified before using). Boomerang had something like this but resulting code was too LISPy.
• Reiterations – if decompiler finds out that function uses registers for passing parameters then code must be changed to reflect this.
• Pattern recognition – it would be very nice if decompiler could recognize the same patterns over the code (in form: A = B+ constant; B = A | constant; ). And if it also could automatically label bitreading functions… But I fear that this is AI-complete problem.

Well, my rant ends here. Back to work.

Let’s see what I have done in 2006:

1. Fraps (FPS1) version 2/4 decoder
2. Intel Music Coder decoder (some cleaning and lavc’ization)
3. KMVC (video codec for FMV used in Worms games) decoder
4. Musepack SV7 demuxer and decoder
5. Smacker demuxer and decoder implementation (no, I haven’t RE’d it)
6. VMWare capture codec (VMnc) decoder (implementation and some RE work)
7. Wavpack decoder
8. ZMBV (DosBox capture codec) decoder and encoder (encoder for palettized format only)
9. Some image formats reading support (TGA, TIFF)
10. 16-bit grayscale support for some image formats
11. And finally – VC-1 decoder (with some WMV3 support)

And here are my plans for this year. Warning! This is not the final list and may be changed in process.

• Make some lossless audio decoders (MPEG-4 ALS and Monkey’s Audio)
• Implement missing VC-1 Simple/Main profile features (there are few of them which nobody uses but who knows)
• Implement VC-1 Advanced profile interlaced mode (looks like it will be needed in future)
• Help with some other projects – DCA implementation and maybe even finish RE for Xan v4 (looks like Mike forgot to do this)
• And ultimate goal – take participation in GSoC 2007 and write JPEG-2000 decoder (in very distant future – even encoder)

Here is a quick list of what has been done recently:

• LZW decoding in TIFF
• 16-bit grayscale limited support (decoding PNG, read/write PGM and JPEG-LS but the latter is a bit buggy)
• Some progress on Intel Music Codec decoder

Ah, I still remember those days when FFmpeg got one new decoder once week or two.

Mike once mentioned here that one coder searches for ideas to implement. My situation is rather complimentary – I hardly manage to fullfill all requests from other people. Recently I’ve added two image decoders to FFmpeg – Targa (.tga) and TIFF. Both of them were done by request. The same situation was with Worms Video.

WavPack decoder was written by my own wish but there are already some people asking to write MPEG-4 ALS and Monkey’s Audio decoders. While the later is not possible to implement right now, I’ll implement both MP4 ALS decoder and encoder.

I just had to express that. Recently I’ve opportunity to examine two DVD disks – one with 8 films-in-one and another one is 7 films-in-one (our pirates usually pack at least two films on each disk). Every DVD has a proud label ‘Real DVD Quality’ (but sometimes even VHS tapes are marked so).
As expected, those films had slightly lesser resolution than 720×576, one disk had 352×576 films, but another one… 352×288 !!! If disk was labeled ‘Real VCD Quality’ – that should be fair.