Kostya's Boring Codec World

Few days ago FFmpeg finally got WMA3 decoder. This event gives me an opportunity to look at our achievements.

1. Popular and/or standard codecs — supported except for the newest stuff (AAC-HE[2], H.264 interlaced modes, VC-1 interlaced modes).
2. Windows Media — WMV1-WMV3 are supported (except for beta version of WMV3 and other WMV3 spinoffs). WMA1-WMA3 is supported too. We still have WMA Lossless and WMA Voice to RE and our top men are working on it (did you remember “Raiders of the Ark” ending? Neither did I).
3. Real Media — RV1-RV4 are supported, from the variety of audio codecs only Sipro and Real Lossless support are missing. Sipro is in the works and nobody (including RealNetworks itself) cares about RALF.
4. Intel codecs — Indeo 1-3 is supported, patch for Indeo 4-5 is available, IMC is supported, IAC is not REd (and not in queue).
5. RAD codecs — REd, there are still some issues with Bink to sort out before inclusion.
6. AVI codecs — that’s a mess. There are simply too many very codecs and new ones still continue to appear. Some are supported, most are not.
7. Lossless audio codecs — some are supported, some are not. Again, looks like everybody writes own lossless audio or video codec. I’d like to get support for TAK though.
8. Game video codecs — we still have a lot of them to RE. Personally I want Discworld III video (BMV, but it differs from the format used in Discworld II) support. *sigh*

If you think there’s some codec we definitely should support, please tell us (preferably with specification or decoder sources 😉 If you just want to have some codec support in FFmpeg — make us interested in it, some codecs support appeared in FFmpeg after somebody had said “can play that file?”.

Looks like I’ve neglected my blog for some time. In order to improve situation a bit here are some assorted pieces of news:

• FFmpeg release — probably we will have one Really Soon Now. Previous release was before I started developing for FFmpeg.
• RV3/4 is improving bit by bit. For now most troubles lie in incorrect motion vector predictions for B-frames. I hope to fix it one day (or preferably that someone else will fix it but that’s even more unlikely).
• SwScaler is slowly moving to be usable under LGPL. Probably it will be only x86 SIMD code that will be left under GPL.
• PB-frames support was added. So the only one who cares about Intel codecs (Benjamin, son of Lars) can watch i263 with lavc decoder now.
• I took some time to understand ELBG code in lavc and wrote simple 15bit MS Video1 encoder. Patch pending.
• I’ve tried to RE BMV (video format employed in Discworld II and Discworld Noir). Discworld II decoder is in ScummVM sources, so I give a shot on DW Noir which is unlikely to be supported by any opensource engine. While figuring out header and container format was piece of cake, finding out audio compression scheme was easy (boy, they do like SWAR!), I have troubles determining which function is used for video decoding.

More news will follow eventually.

Following Mike’s example and with intention to spread the word, I welcome you to look at our Hall of Shame where you can find those, who use FFmpeg and probably other products but fail to comply to (L)GPL, giving no credit and/or sources.

So, if you want a bit of publicity follow their example. But don’t be surprised later.

I think I ought to tell about the idea that fellow FFmpeg participant Robert Swain (I still wait for him to complete AMR-NB decoder) tries to champion. I mean encoding profiles.

The idea is quite simple – allow users use convenient presets instead of command line black magic, so simple commands like

ffmpeg -i input.vob -preset dvdrip -preset hq_mp3 output.avi
ffmpeg -i input.avi -preset flash_youtube output.flv
ffmpeg -i input.vob -preset ipod_vid output.mp4

can be used without bothering about bitrate/flags/gop size/whatever.

I also think this will be useful. It will be easier to change options meaning without making additional complications (we still get questions from users that do not know recent ffmpeg expects bitrate to be specified in bits, not in kilobits as before). Just don’t make frontend for it named ffclippy, okay?

After some discussions on IRC I’ve participated I’d like to present here for future discussion.

1. Audio API should reflect video API as much as possible. Now decoder outputs 16-bit native-endian audio into raw buffer.
2. Introduce audio formats. I’d like to be able to decode old 8-bit codec into bytes, newer 24-bit audio into 32-bit ints, floats for other codecs if they need it, etc.
3. Planar format for multichannel codecs. It will simplify downmixing and channel reordering. (This is not my idea but it is worth mentioning)
4. Swscaler-like structure for format handling and negotiations between audio filters.
5. Block-based audio processing. Each audio should be operated as a multiple of blocks with fixed number of samples (like video is operated by frames and rarely by slices). Why not always by single block? Because some formats throw chunks with multiple blocks to decode (Monkey Audio, Musepack SV8) and some have too small blocks that cause too much overhead to process them by one at time (most speech codecs and (AD)PCM). This is just a bit stricter than current scheme.

Now, who wants to implement this?