Checkpoint

January 6th, 2008

Let’s see what was done with my plans for 2007:

  • Make some lossless audio decoders – no luck (well, I’ve helped a bit to include Monkey’s Audio decoder to ffmpeg)
  • Implement missing VC-1 Simple/Main profile features – no luck (at least WMV3 Complex Profile is almost completely supported)
  • Implement VC-1 Advanced profile interlaced mode – no luck
  • Help with some other projects – DCA implementation and maybe even finish RE for Xan v4 – partially done
  • Write JPEG-2000 decoder – no luck, some people convinced me to write RV3/4 decoder instead and I’m still waiting for the specs 🙁

So it was extremely unlucky year in fulfillment of goals.

What have been done instead:

  • Musepack SV8 support
  • Helped with some projects (DCA decoder, Monkey’s Audio decoder, few game formats)
  • Some WMV3/VC-1 fixes
  • A bit of work on RV3/4 decoder

I hope this year will be better.

Posted in Uncategorized | 2 Comments »

Some Thought on Future FFmpeg Audio API

November 22nd, 2007

After some discussions on IRC I’ve participated I’d like to present here for future discussion.

  1. Audio API should reflect video API as much as possible. Now decoder outputs 16-bit native-endian audio into raw buffer.
  2. Introduce audio formats. I’d like to be able to decode old 8-bit codec into bytes, newer 24-bit audio into 32-bit ints, floats for other codecs if they need it, etc.
  3. Planar format for multichannel codecs. It will simplify downmixing and channel reordering. (This is not my idea but it is worth mentioning)
  4. Swscaler-like structure for format handling and negotiations between audio filters.
  5. Block-based audio processing. Each audio should be operated as a multiple of blocks with fixed number of samples (like video is operated by frames and rarely by slices). Why not always by single block? Because some formats throw chunks with multiple blocks to decode (Monkey Audio, Musepack SV8) and some have too small blocks that cause too much overhead to process them by one at time (most speech codecs and (AD)PCM). This is just a bit stricter than current scheme.

Now, who wants to implement this?

Posted in CEmpeg | 3 Comments »

X8intra is there!

November 10th, 2007

Now we have X8intra frames support in ffmpeg! Mike has already expressed his joy in his blog. I think anime fans who tried to play WMV3 in AVI would also be glad.

Why this has not been done earlier? Well, sheer disgust. The person who invented this scheme should be either fired or promoted to M$ CTO. Here is the list of reasons:

  • It is used for some keyframes coding so it cannot be skipped
  • Design is totally unlike anything standard
  • You should perform bitexact decoding. If your DCT produces slightly different results or you forgot about loop filtering then you won’t be able to decode picture properly. Hey, that’s utter crap by _any_ standard
  • It has made it to WMV3 too.

To put it mildly, X8Intra is an illegal offspring of JPEG and some early H.264 draft. It is mainly Huffman-coded 8×8 DCT-transformed blocks with spatial prediction and loop filtering. It does not have macroblocks like decent codecs. Spatial prediction has some directions and relies on previously decoded blocks and bits read depending on that. I have not seen anything with such a bad bitstream-transform dependency (in other codecs you can decode coefficients first and then perform image reconstructions but not here). X8Intra excrementum est(pardon my Latin).

Still I very grateful to the person named “someone” who did this. Otherwise I should clean this cesspool.

Posted in VC-1 | Comments Closed

Multimedia-unrelated news

November 9th, 2007

I just had to post this – our Philharmonic presented own harpsichord. Several years ago when I first visited it I’ve listened to concert music with harpsichord but it was borrowed one and there was nothing comparable since that time.

The presentation went well and we were enjoying different music – from sonatas by Handel and Telemann, Johann Sebastian Bach concertos to Mozart to jazz improvisations and modern Ukrainian music (well, when composer plays on one of the instruments himself I consider it modern).

Looking forward for further listening (with hope that it would take less than a couple of years of waiting).

Posted in Uncategorized | Comments Closed

A Book on Multimedia

November 8th, 2007

I presume those interested in multimedia coding have heard of “Data Compression: The Complete Reference” by David Salomon. Personally I consider this book very good but maybe we should write our own book concentrating on multimedia only. Why? I have not seen books where video (and audio) compression is not merely outlined (like in most books on general data compression) or is not solely dedicated to some standard
(MPEG usually).

I gladly remember this book as it’s quite outdated but at least it covers many codec, container and even implementation issues (unfortunately, sound only))!

My proposal for book outline:

  • General multimedia concept (pixels, samples, PCM, DCT)
  • Audio compression
    • Simple time-domain codecs (DPCM, ADPCM)
    • Complex time-domain codecs (lossless mostly)
    • Speech codecs
    • MDCT-based codecs and friends
    • How to write a fast decoder and good encoder (or otherwise)
  • Image Compression
  • Video compression
    • Lossless coding
    • Game video codecs (who will write this?)
    • Modern standard and non-standard codecs
    • Implementation tips and tricks
    • Known codecs (implementation-wise) overview
  • Containers
    • Why making codecs dependent on custom container is idiotic 🙂
    • File-based containers
    • Streaming containers

And I know where to get information ;-). Well, let’s see if this catches up.

Posted in Uncategorized | 8 Comments »

Audiovisual debugger

November 4th, 2007

I have never though about FFplay in that way but it had struck me today that waveform visual display is one of the best ways to debug it.
Why?


FFplay

One of C.P.E. Bach’s Wurttemberg sonatas (a small excerpt, really)

Because it gives you those advantages:

  1. Noise hurts your eyes less than ears
  2. Some inaudible artifacts (like DC bias) are easily spottable
  3. Clipping and volume change is easily spottable too
  4. Stereo differences are easy to find
  5. …
  6. It may give you some aesthetic pleasure 😉

I must also add that most audio player have visualizers but they lack simplicity and usability of this 640×480 clean waveform rendering.

Posted in Audio | 1 Comment »

Another game format

October 17th, 2007

I remember playing game “Lost Vikings” by Silicon&Synapse (who was later renamed to Blizzard).


Lost Vikings 2
Lost vikings (again) and additional creatures. Endgame scene.

Who knew that it had a sequel? It was done by Beam Software and looks less appealing to me than original. But it has cutscenes and audio in its own format, which makes it a bit more interesting. A contributor, who wishes to remain anonymous, sent me his description of this format so I was able to implement demuxer and decoder for their video and it will be committed soon (I hope). Details of this format are already here.

Posted in Uncategorized | 1 Comment »

Flowers

October 9th, 2007

As a follow-up to the theme set by Michael here are my few random shots of flowers (made in the Western Ukraine, several thousands kilometers from my home).

Read the rest of this entry »

Posted in Uncategorized | Comments Closed

Musepack SV8 is almost ready

September 26th, 2007

Judging from this post there is an eight steam version of Musepack in beta testing (but the spec is not frozen yet).

What distinguishes it from previous versions? Now it is container-aware. Previous versions store just audio frame in the continuous bitstream with no defined behavior on seeking nor demuxing. It was almost as fun as Monkey’s Audio container.

Now Musepack has a chance to spread in the wild (i.e. in other containers than .MPC). That mostly depends if there will be standard way to store them in the other containers (that’s where Ogg Vorbis failed). Well, good luck.

Posted in Audio | Comments Closed

Some details on RV30/40

September 12th, 2007

RV8 or RV30 is close to earlier H.264 drafts and Sorenson Video 3, because they both use one variable-length code (Golomb codes in case of SVQ3, something special in RV30/RV40) to represent macroblock information (type, intra prediction modes). RV40 uses this code only to represent motion vectors and number of macroblocks to skip, macroblock type and intra prediction modes are coded with variable-length codes chosen from the set of them depending on context.

Here are the main differences from H.264:

  1. Different bitstream. Special codes for elements and own substitution of Golomb code.
  2. Intra prediction types are specified only for luma subblocks, chroma subblocks use some of them for prediction too.
  3. Intra prediction modes are slightly different and some of them require additional context (down left neighbour).
  4. Bidirectionally predicted blocks in B-frames do not use motion vectors from the previous/next frame in motion vector prediction while using them in motion compensation.
  5. Intra prediction for intra blocks in interframes is performed even when neighbouring blocks are not intra blocks.

Main differences between RV30 and RV40:

  1. Bitstream syntax and different codes (RV30 is easier in this matter).
  2. RV30 does not have 8×16 and 16×8 motion compensation modes (while 8×8 mode exists).
  3. Different motion vector prediction algorithm.
  4. B-frames in RV30 do not contain some variant of bidirectionally predicted blocks which RV40 has.

And now here is the list of things I didn’t like in RV30/40:

  1. Slice header does not contain number of macroblocks coded in this title. While it saves whopping 6-13 bits per slice (and frames usually have less than a dozen of slices, usually two or three), it gives unnecessary pain to implementor.
  2. Vertical left intra prediction method uses down left neighbour pixels in calculation for one insignificant pixel (it’s insignificant because it does not affect further intra prediction).
  3. Motion vector prediction is a bit complicated too.
  4. And, of course, the lack of good documentation

Posted in RealVideo | 5 Comments »

« Older Entries
Newer Entries »