Kostya's Boring Codec World

Well, I think it’s time to look at FFmpeg achievements for this year.
Main achievement is without any doubt breeding Sus Scrofa Avionica — FFmpeg 0.5 release. It took only 5 or so years.

What has been added (mostly from Changelog):

1. FunCom ISS demuxer and decoder
2. various Electronic Arts formats support
3. Gopher protocol
4. MXF D-10 muxer
5. Ogg muxer improvements
6. VQF decoding support
7. PCX encoder
8. Cook multichannel support
9. DPX decoder
10. ALS decoder
11. WMAPro decoder
12. Atrac 1 decoder
13. many other smaller additions and enhancements

What I did:

1. out WavPack decoder now support hybrid coding as well
2. PB-frames support for H.263 decoder, so Intel I263 files are played more or less fine
3. some RTMP support
4. Aura 1 and 2 decoders
5. fixed long-standing bug in Interplay Video MVE — now it detects 16-bit variant of video (and decodes it)
6. some improvements on SwScaler — 48-bit RGB support and rewriting our YUV to RGB conversion code, so SwScaler could be used under LGPL

But mostly this year gave us hopes for a lot of even more important things:

1. there is hope that our AAC encoder will work
2. there is hope that our AAC decoder will support SBR (and we can remove libfaad2 support)
3. there is hope that RTMP input will work with all commercial servers as supposed
4. Bink support is at least 90% complete, it’s mostly cleanup and bugfixing left to do
5. Indeo 4 and Indeo 5 decoders. Boy, how I miss them!
6. Apple codecs (ProRes, Intermediate codec).
7. Sipro decoder will be hopefully committed
8. hopefully, Windows Media codec family support will be almost full too, the only things left to RE are their speech codec, lossless audio codec, screen capture codecs and some WMV3 leftovers
9. There is hope that some other unfinished work will be completed. For example, I have almost working LucasArts SMUSH video player based on some older work by somebody from ScummVM, MS Video 1 encoder. Other people for sure have some unfinished stuff too.
10. I have a faint hope that Discworld 3 FMV will be playable with something opensourced. And then by FFmpeg.
11. Finally, I hope Mike will finish his Xan WC4 decoder.

Since I’m not going to work at this soon (have more stuff to do), I publish that stuff I did. Grab
tgzipped sources here. Most of it does not give any significant speedup because of internal Loongson structure, so it’s just proof of a concept.

I’ve been following the steps of Måns and got myself Gdium too. Since it’s no fun just owning less-spread computer architecture and not writing anything on it, I’ve tried SIMDifying the easiest operations one can do on it — vector sum, vector subtraction and vector scalar product. And I have a decoder that uses those operations extensively, so why not try to benchmark it a bit?

Test sample was first 26 seconds of Monkey Audio file with insane compression since this mode uses longest filters and benefits from SIMD most (and is slow enough even for short samples ;). In all cases I’m the one who has written SIMD code, so it’s fair 🙂

PowerPC (Freescale 7447A 1.42 GHz): 25 seconds and 6 seconds
MIPS (Loongson 2F 900 MHz): 37 seconds and 7 seconds
ARM (Cortex A8 600 MHz): 138 seconds and 22 seconds
x86 (Intel Atom N270 800 MHz): 50 seconds and 9 seconds

Mind you, SIMD instructions in Loongson are custom for that CPU and modelled after MMX (64-bit registers, actually reusing FPU regs, similar names) but at least they are done in RISC fashion, i.e. you can store result in some other register.

I’ve also looked out of interest at binary representation of SIMD. On x86 the principle is to prefix SIMD instruction (usually with 0x66 “opcode for CPU with half of current bits” byte) so SSE7 instructions will look like instructions for 1-bit FPU on Intel 4004 predecessor and will take 8-16 bytes to represent.

Other architectures use simple 32-bit word for any instruction. NEON (on ARM) and AltiVec (PowerPC) use some opcodes in general instruction space, Loongson 2 SIMD are custom calls to the second co-processor.

Talking about instruction sets I cannot omit the fact that IDA 5.2 sucks at disassembling PowerPC code (not only AltiVec but some of the core instructions too) and objdump sucks at disassembling MacOSX format (it ignores internal structure and disassembles it as raw file), that looks like the reason why we don’t have Apple Intermediate Codec RE’d yet.

P.S. Jag vill gärna få AVR32, BlackFin, ColdFire och andra exotisk CPU:ar. Alpha eller Sparc är bra ochså men det är bara orealistisk, tror jag.

Few days ago FFmpeg finally got WMA3 decoder. This event gives me an opportunity to look at our achievements.

1. Popular and/or standard codecs — supported except for the newest stuff (AAC-HE[2], H.264 interlaced modes, VC-1 interlaced modes).
2. Windows Media — WMV1-WMV3 are supported (except for beta version of WMV3 and other WMV3 spinoffs). WMA1-WMA3 is supported too. We still have WMA Lossless and WMA Voice to RE and our top men are working on it (did you remember “Raiders of the Ark” ending? Neither did I).
3. Real Media — RV1-RV4 are supported, from the variety of audio codecs only Sipro and Real Lossless support are missing. Sipro is in the works and nobody (including RealNetworks itself) cares about RALF.
4. Intel codecs — Indeo 1-3 is supported, patch for Indeo 4-5 is available, IMC is supported, IAC is not REd (and not in queue).
5. RAD codecs — REd, there are still some issues with Bink to sort out before inclusion.
6. AVI codecs — that’s a mess. There are simply too many very codecs and new ones still continue to appear. Some are supported, most are not.
7. Lossless audio codecs — some are supported, some are not. Again, looks like everybody writes own lossless audio or video codec. I’d like to get support for TAK though.
8. Game video codecs — we still have a lot of them to RE. Personally I want Discworld III video (BMV, but it differs from the format used in Discworld II) support. *sigh*

If you think there’s some codec we definitely should support, please tell us (preferably with specification or decoder sources 😉 If you just want to have some codec support in FFmpeg — make us interested in it, some codecs support appeared in FFmpeg after somebody had said “can play that file?”.

Looks like I’ve neglected my blog for some time. In order to improve situation a bit here are some assorted pieces of news:

• FFmpeg release — probably we will have one Really Soon Now. Previous release was before I started developing for FFmpeg.
• RV3/4 is improving bit by bit. For now most troubles lie in incorrect motion vector predictions for B-frames. I hope to fix it one day (or preferably that someone else will fix it but that’s even more unlikely).
• SwScaler is slowly moving to be usable under LGPL. Probably it will be only x86 SIMD code that will be left under GPL.
• PB-frames support was added. So the only one who cares about Intel codecs (Benjamin, son of Lars) can watch i263 with lavc decoder now.
• I took some time to understand ELBG code in lavc and wrote simple 15bit MS Video1 encoder. Patch pending.
• I’ve tried to RE BMV (video format employed in Discworld II and Discworld Noir). Discworld II decoder is in ScummVM sources, so I give a shot on DW Noir which is unlikely to be supported by any opensource engine. While figuring out header and container format was piece of cake, finding out audio compression scheme was easy (boy, they do like SWAR!), I have troubles determining which function is used for video decoding.

More news will follow eventually.

Following Mike’s example and with intention to spread the word, I welcome you to look at our Hall of Shame where you can find those, who use FFmpeg and probably other products but fail to comply to (L)GPL, giving no credit and/or sources.

So, if you want a bit of publicity follow their example. But don’t be surprised later.

I think I ought to tell about the idea that fellow FFmpeg participant Robert Swain (I still wait for him to complete AMR-NB decoder) tries to champion. I mean encoding profiles.

The idea is quite simple – allow users use convenient presets instead of command line black magic, so simple commands like

ffmpeg -i input.vob -preset dvdrip -preset hq_mp3 output.avi
ffmpeg -i input.avi -preset flash_youtube output.flv
ffmpeg -i input.vob -preset ipod_vid output.mp4

can be used without bothering about bitrate/flags/gop size/whatever.

I also think this will be useful. It will be easier to change options meaning without making additional complications (we still get questions from users that do not know recent ffmpeg expects bitrate to be specified in bits, not in kilobits as before). Just don’t make frontend for it named ffclippy, okay?