Archive for the ‘FFmpeg’ Category

A Tale of Two Failed Projects

Saturday, June 25th, 2016

Yes, it’s about FFmpeg and Libav again. And yes, I consider them both to be failed projects (not that their basic goal is failed and they provide even less multimedia support than GStreamer with no external libraries used), I mean the state of the project as living and developing entity.

Even if I mostly emulate Derek nowadays—i.e. unsubscribed from FFmpeg and Libav mailing lists, do nothing productive, wait for somebody to reverse engineer codecs I care about somewhat (that would be ClearVideo, thank you very much). Yet I peruse development-related resources for both projects (mostly for finding laughs) and sometimes I see gems like this (it was pointed at in the comments as well since it answers some questions I’ve asked before).

First, I’d like to outline how large projects are organised and what to expect in general. So, if you have a large and used project you’ll have at least these components:

  • codebase (normal projects have some code to run after all);
  • developers (to add features, fix bugs and such);
  • users (to annoy developers and once in a while to provide sensible bugreport or feature request);
  • infrastructure (hosting for code, means to communicate for developers, maybe even support for users).

Developers can be also divided into three main categories:

  1. core developers—the ones who do main work on the codebase and do it in regular manner (they might intersect with the next category too);
  2. corporate developers—the ones who do work mostly on behalf of their companies (e.g. add a feature they need internally so they don’t have to maintain it themselves);
  3. contributors—developers who add some feature or provide some bugfix because they needed it themselves, they do it irregularly or even just once (again, they might intersect with the previous category).

This division is by no means perfect but it shows the main forces behind development: those who treat is as a hobby, those who do it for their benefit (i.e. making money with/from it) and those who use it and just want to be it a bit more suited to their personal needs.

So, with that all in mind let’s look at the projects:

FFmpeg

Codebase. It’s a complete mess. And its git history is even worse. The running joke is that who cares what that piece of code does, it’s FFeature so it must be kept at whatever cost (that’s how you get double decoders, demuxers and encoders; an outstanding example there would be libutvideo wrappers—refer for the details to ffmpeg-devel mailing list).

Developers. Because of the merging policy (that is likely to be codified soon—see this document again) many developers of FFmpeg code are not FFmpeg developers. And yet they are dictating API to be used in FFmpeg: the first example that also involves me—I’ve proposed side data for packets in Libav, FFmpeg hesitated for a bit yet included it with such flattering message; the most of examples include Anton’s work from introducing refcounted buffers to splitting codec parameters into separate structure—in any case FFmpeg simply takes it and converts their code to comply with a new practice (even if it has to include some horrible hacks). If that doesn’t cry out loud “a failed project” I don’t know what does.

Also (even if I’m stepping onto minefield) some FFmpeg developers are completely unfit for collective work because of their personal qualities. People may make jokes about providing full console output of ffmpeg command but it’s not Carl who’s the main problem in FFmpeg (yes, people who didn’t work on MPlayer might think otherwise; I still believe he’d be a decent leader for FFmpeg—mostly because he doesn’t focus just on technical side and he’s unlikely to be treated as a technical god who can’t make any mistake or write less than perfect code). Here it’s more about Michael and Clément—the former never really understood what being a leader really is or what resigning from a leader means (anyone disagreeing please ban yourself from a mailing list of your choice for 24 hours), the latter does not understand people at all (neither does Michael)—I’m not going to paste the link to the same document for the third time, I’ll simply quote the relevant part:

Any Libav developer is of course welcome anytime to contribute directly to the
FFmpeg tree. Of course, we fully understand and are forced to accept that very
few Libav developers are interested in doing so, but we still want to recognize
their work.

Here’s an excerpt from Michael’s mail:

> Don’t you think you should remove Diego, Måns, Kostya, … as well?

They didnt ask me to remove them, they didnt remove themselfs even
though they could, they didnt post a patch to remove themselfs.
No contributor said that he contacted them and they no longer maintain
the code they are listed for. (or i missed that)

Well, if it’s hard to realize that Libav developers don’t want to contribute to FFmpeg and don’t want to do anything with it even though it’s been over five years then you really have a problem. And I’ve expressed my thought on reuniting both projects already.

Users. You know, there’s a difference between catering to your users and selling out completely (to put it mildly). When you see some changes being done in interests of some third party often without mentioning it that looks suspicious. I’m not against making money off your work but when it’s not even mentioning the fact it looks strange; when you have a decoder with a copyright assigned to some company it’s fine, but when you have fixes for files nobody has seen or FFv1 features added because it was all paid by somebody (see here slide 12) it looks not completely honest even if there’s nothing wrong with it.

Infrastructure. From what I understood FFmpeg services are now hosted on various boxes with no plan or idea (i.e. if somebody could provide a box for something they took it) and there’s no system administrator for these boxes. Again, as I understand it, they were kicked out of Hungary for some reason and even though they got a free server and hosting in Bulgaria they cannot use that box properly because there’s nobody to set it up properly and maintain afterwards. Sounds like fail to me.

Libav

This project is failed for the different reasons but failed nevertheless.

Codebase. While it’s mostly fine sadly new features hardly come in. Just two examples—there have been talks about replacing libswscale since ages, two years ago they’d started to design it (and it went nowhere), then I offered my design with a PoC (yes, piece of that) code to test it (that’s how NAScale was born), people work on integrating it into Libav a bit and that’s all—nothing has happened yet; the second example is bitstream reader replacement—since its submission in April nothing has come out of it as all traction was lost in bikeshedding. Is it failure or what?

Developers. Here we have two problems—some FFmpeg folks and some core developers. I’ve written about the former before so let’s talk about the latter. Surprisingly or not there are counterparts for Austrian FFmpeg developers in Libav. Where in FFmpeg you have Carl Eugen, in Libav there’s Diego and I guess many have suffered from his perfectionism (in form of proper formatting). And instead of Michael there’s Anton. While he is not that leadery in general sense, he’s the one introducing big changes in API that are hardly discussed before. And even worse thing—he tries to make all nontrivial code go through him, QSV support is a good example: Maxym Dmytrychenko had submitted initial support but it was not deemed good enough so Luca Barbato had to rework it into proper form. And what do you know? It turned out to be not good enough for Anton so he worked on it himself with the result not being much different from Luca’s. And since nothing is being done about that I consider it to be a failure.

Users. Sadly, there seems to exist not so many of them which is a fail. On the other hoof they don’t need to deal with distros and Baidu and that’s a blessing by itself. Though there is still an issue with FFmpeg users who bother (ex-)developers for features present in FFmpeg but not in Libav (or present in different form), like Blackmagic card support or prores_ks encoder (hint: there’s no encoder with such name in Libav and it’s my personal pleasure to ignore mails about it).

Infrastructure. From what I heard thanks to Attila and Janne everything is working fine.


Well, maybe I should continue with Actimagine VX codec at last and forget about multimedia outside work matters afterwards (insert the obvious joke about this not hurting NihAV development at all).

Schizophrenia in Open-source Projects

Saturday, May 14th, 2016

Disclaimer: the word “schizophrenia” is used here as it’s perceived by majority, not to denote a certain psychological condition. Feel free to be offended.

I’ve wasted about a decade working at two multimedia projects (plus a patch or two to unrelated projects) and what I’ve seen there leads me to the conclusion they both suffer from schizophrenia, albeit in different forms.

FFmpeg

FFmpeg features two forms of schizophrenia—developers and code.

Developers schizophrenia can be seen in how some developers believe they are also Libav developers. Mostly they brag because they’ve sent a patch or two to Libav and now can use it as a free review service. While I dislike Carl Eugen he’s at least honest and acts to his beliefs (here, an amended Elenril’s Law fulfilled; in case you’ve forgotten it says “Every FFmpeg-related discussion ends up mentioning Michael. Or Carl Eugen.”).

Code schizophrenia is more celebrated. The most prominent example is ProRes support—they offer two decoders and two encoders for it. There are two ASF demuxers as well. And two audio resampling libraries. And there are talks about adding second libswscale (*shudder*). The best part is that if you ask why it will probably go like this:

— Why do you have feature X in two versions?
— Because Libav has it.
— But why do you take it if you have your own version?
— To make merging Libav codebase easier.
— But why do you need to merge it at all?
— To make merging Libav codebase easier.

Please please tell me I’m wrong and provide proper reasons why FFmpeg keeps merging Libav stuff and keeps several versions of the same feature.

Libav

Here it’s somewhat more interesting—you have developers with physical multiple personalities disorder. Unlike the case in FFmpeg here you have people that work solely on Libav but as several different people. Most prominent examples are Luca (known as lu_zero and koda on IRC) who is really several instances not always agreeing with themselves (and if you subscribe to Lu_zianism then he’s also both Michael and Carl Eugen too, if you don’t believe it—you should because it annoys him/them). And there’s also Alexandra (aka beta elenril) and Anton (aka sasshka 2.0). And that’s the majority of core Libav developers anyway.

But at least the project seems to be more happy with itself and probably has a dream similar to the Ukrainian dream (which is “bugger off you damned Russians” in case you didn’t know).


It was somewhat fun to watch the fate of proposed bitstream reader replacement. Alexandra (she’s also Top Libav Blogger ?2 by the way—simply because she blogs) proposed a new bitstream reader to replace the old horror (which is a good idea), and that new bitstream reader turned out to be faster than the old mess too, and what was the result? If I’d be British I’d call it sheep-worrying.

Those developers from FFmpeg that believe they also should have a say in Libav process started to express their opinions. While there were independent benchmarking proving the new implementation is indeed faster (which is a good thing to provide) those benchmarks were also run on the decoder not present in Libav, with badly converted functions for code reading, and that turned out to have some problems because of the encoder used (also not in Libav) that produced nonconforming stream and screwed multi-threaded decoding benchmarks (that one can be seen as both trollish and arrogant—kinda like judging The Beatles performance from an excerpt sung by your not very talented neighbour).

But mostly it was bikeshedding and asking why it was not using old get_bits interface. The answer for the latter is simple—because it was built from horrible macros used in half of the places directly so you should either to make everything follow those macros design or convert the old UPDATE_CACHE(); LAST_SKIP_BITS(); ... CLOSE_READER(); into saner get_bits(); skip_bits(); anyway. And Libav developers decided it’s better to have fully new interface anyway and to make it consistent with bytestream reading while at it.

So why did people who should have nothing to do with it bikeshedded that much? Probably because they know in their hearts that as soon as it hits Libav the work on copying it into FFmpeg starts and sooner or later it will reside in FFmpeg codebase probably along with old get_bits.h with most decoders switched to the new bitstream reader anyway. Why? See the theoretical conversation above. I’d like to know the answer why merges are really done but I guess I’ll get it no sooner than this bitstream reader is accepted into Libav master (i.e. never).

Some Thoughts on Reuniting

Thursday, April 28th, 2016

Before I move to the point I’d like to give some historical examples based on countries.

Ukraine

Well, as you remember in 1917-1918 there were several Ukrainian republics, most known are Ukrainian People’s Republic, West Ukrainian People’s Republic and Ukrainian Soviet Socialistic Republic. There were some other small states like anarchic republic but they are not relevant here.

So, Ukrainian People’s Republic and West Ukrainian People’s Republic willingly united in 1919 and that day is a national holiday now (later it was obviously occupied by Soviet Russia, Poland, Romania and Czechoslovakia). But why did that union happen? Because people wanted it and there was a dream for the united Ukraine since ages.

Germanies

You should’ve learned about it at school (or witnessed if you’re old enough). Why did the unification happened? Because people from both sides wanted it and the Soviet union could not prevent it any more.

Moldova and Romania

These countries share common history, have the same language and people like the idea of single country. While unification has not happened yet it might happen even in this century.

Chinese Republics

Here the situation is funnier. People’s Republic of China doesn’t recognize Republic of China yet they somehow co-exist and probably in distant future they will be the one again. Why? Because PRC is changing and it’s not what it had been during Chairman ? times.

Korea

Here the situation is even funnier. There are two governments who think they are the only True UpstreamKorean state, it’s just half of it is still occupied. And while there are constant talks about reunification, neither state really wants it. One country has suffered under homebrew Socialism (just look up what ‘juche’ means) for too long so it will take an enormous amount of time and money to make both parts equal (even funnier if you consider that before 1960s North Korea was industrially developed and South Korea was an underdeveloped agrarian region). Germanies got it easier (as a person paying Solidaritätszuschlag I know that). So will the reunion ever happen? I wouldn’t bet on it.


And now, to the our favourite projects.

Time from time somebody outside projects or from FFmpeg side asks about projects reunification. There are talks about it at VDDs. And yet there are no results. Have you noticed that I mentioned no such talks initiated by Libav. Why? Probably because Libav does not want to merge back. And there you have it—reunification cannot happen peacefully because you don’t have majority on both sides wanting it.

And that raises two questions: why FFmpeg wants reunification and why Libav doesn’t want it (or as a single question—what prevents it).

It seems that for some reason not clear to me FFmpeg keeps merging all stuff from Libav (feel free to enlighten me, otherwise FFmpeg developers themselves might forget it and it’ll turn into tradition) and having both projects together will solve two problems: the need for merge and the lack of skilled developers (it’s always the issue).

What does Libav gain from the merging? Relief from constant merges? Unlikely since it’s not being done there. More developers? That’s nice but Libav project seems to be happy as is. Return to the known brand and distributions? See above and here.

Let’s assume the projects decided to play nice out of nowhere and please people who’d want them to reunite. What would happen then? Multiple discussions about development process (that lead to the split in the first place), including but not limited to: reviewing process (relaxed and not applicable to some people or mandatory for any change), code standards (especially formatting), what features to have in the united tree (flat history or merges, one native decoder for certain format or two, use the code snippet like it was done in FFmpeg or in Libav). And on this stage it will all start to fall apart again.

So there you have it: clash of different development ideologies and more benefits for one side than the other. Also it’s rather hard to force people to work on the project they don’t like (and now they can choose at least).

And since this discussion cannot avoid certain names, here it is: I believe that Carl Eugen Hoyos deserves to be the next FFmpeg leader. Obviously my opinion doesn’t matter there and I could not convince anybody at VDD’15 but I firmly believe so. He’s the one with passion for the project, he cares for codec support (even fringe formats), he likes to follow guidelines, he respects Michael and is unlikely to go and ruin what he created. And at VDD he looked kinda like the most responsible adult too, so he can be the project face. Again, this is merely my opinion that won’t change anything.

Sincerely yours, NihAV project developer (it’s still vapourware, thanks for asking).

Why FFmpeg is “better” than Libav by Numbers but not in Reality

Saturday, October 15th, 2011

Time from time there are mails from FFmpeg developers to different distributions saying “Oh, we are better, have more features and more secure, pick us and not libav”. In case somebody wonders here are two examples: Ubuntu and Debian.

So let’s walk through and see the claims. Comparison was made from git snapshots made 2011-10-15 11:00 CEST.

More codecs

FFmpeg claims to have more codecs. In numbers that’s true. Let’s see what are those codecs though:

  • VDPAU accels for MPEG-1 and MPEG-2.
  • AMV encoder. Really just a quick hack to flip picture before using standard JPEG encoding.
  • 8SVX raw audio. Really just a hack to support one particular case of raw planar audio.
  • Flash Screen Video 2 encoder. If you look at the discussion here and the committed file you’ll see that encoder was committed by the same person who had some objections but they never were addressed. Also committing encoder for the codec without any working opensource decoder (well, no decoder beside Adobe Flash client could decode the output properly) is not the brightest idea IMO.
  • CrystalHD MPEG-2, MPEG-4, H.264 and VC-1 support.
  • JPEG-2000 codec. Known to be very buggy and yet included.
  • M$ Video 1 encoder. That was the last review for it – link on gmane and then it was just committed verbatim. Such hypocrisy is one of the reasons I don’t contribute to FFmpeg directly (they pull all patches from libav anyway).
  • DTS encoder. It was considered not to be good enough by the person who was responsible for its development (Benjamin Larsson) but not bad enough to not be included to FFmpeg.
  • G.723.1 codec. The same story.
  • G.729 decoder. It was abandoned halfway by original developer so it’s not complete and buggy.
  • Sonic codec. An experimental codec that was not touched since 2004. It’s not supported by anything and my tests show that it often cannot even compress audio losslessly (output size is bigger than original). That’s why we finally threw it out at libav.
  • Subtitle and text codecs.
  • Stagefright H.264 decoder. It’s too ugly to be included to libav (C++ ABI issues, internal bitstream filter usage).
  • Speex encoder wrapper and Celt encoder wrapper.
  • libaacplus encoder wrapper. It’s based on 3GPP source code and hence not redistributable.

In conclusion, those additional codecs supported by FFmpeg are either hardware acceleration (VDPAU, CrystalHD, stagefright), trivial or some old patches previously rejected but now picked up to maintain impression of superiority.

More muxers and demuxers

Now to muxers and demuxers.

  • ACT and BIT demuxers. Containers for G.729 data.
  • ADF/IDF/bintext/xbin demuxers Containers for text formats.
  • CAF muxer.
  • G.723 demuxer. Container for G.723 data
  • LOAS demuxer. It’s just a hack for probing raw AAC stream.
  • MicroDVD subtitles muxer and demuxer.
  • PMP demuxer.
  • WTV muxer.
  • libmodplug Module tracker file support (libmodplug decodes the files, demuxer outputs decoded raw sound and optional text information).

Most of (de)muxers are either for formats we don’t support or trivial. But I admit there are maybe two demuxers and one muxer that can be included into libav after some cleanup.

Filters

I’m not fully familiar with filter subsystem but it still has too many problems and having an ugly wrapper for MPlayer filters (which are also not of the highest code quality) was one of the points that drove FFmpeg-libav split.

Features, security and overall

When FFmpeg claims to have more features than libav it’s true — they merge everything from libav nowadays. And FFmpeg gets more side features for two reasons: the name is more known and it’s easy to get a feature in because nobody seems to bother with proper reviews nowadays. As for security, it’s the same. For example, when there was a report for some minor security flaw in Chinese AVS decoder, original patch from security investigators (applied in FFmpeg) just made decoder more secure but it still crashed on that file, in libav we created our own patch that dealt with both of these problems. FFmpeg often reminds me of a saying “Socialistic system successfully overcomes the problems it creates itself”.

In conclusion, my opinion is that FFmpeg is trying to compete with libav by lowering standards on accepted stuff. Hence the name of this post.

And I’m glad we can’t have commits like this. For those who have not so good C understanding: it compares addresses of two different strings and it’s an undefined operation that works mostly by luck. So my advise is simple: if you want quality then choose libav, if you want some special features — look for suitable fork at github.

What happened to FFmpeg

Wednesday, March 30th, 2011

This is my look on what happened but I’ll try to remain objective.

A bit about me (in the very unlikely case you don’t know already and care). I’ve learned about FFmpeg in 2004 or so, just saw it along with other packages in Mandrake release. For several months I downloaded source snapshots at cybercafe (even dial-up was impossible then). I had long interest in general data compression methods and some interest in codecs sparkled by XAnim and desire to play M$ ADPCM files on Linux and FFmpeg got new decoders every week or so (mostly for packed YUV formats but nevertheless quite useful).

One day I tried to reverse-engineer some codec (just for fun), looked at sample produced by TechSmith Camtasia and realized that it’s packed with zlib and after some time guessed correctly they use M$ RLE. In order to test it I wrote a decoder and hacked it into FFmpeg. Eventually it worked and I send my decoder to Mike Melanson. On the 14th of August 2004 it was committed to FFmpeg codebase and it made me proud for my work for a week (those were the times!). After another decoder or two I’ve learned and started to read ffmpeg-devel mailing list (it was on SourceForget then). I think I started submitting my patches there with Indeo2 decoder or so.

After a while I was offered CVS commit access which I refused because of technical limitations. Finally in March 2006 I got display for my MacMini and I was ready for more active development. Google Summers of Code gave me opportunity to dedicate a bit more time for FFmpeg since I could say “hey, I’m payed for it!”. I still try to contribute even if I’m no longer student, have job and too little free time.

And now to the business.

As you may know, most active group of developers had disagreements on how FFmpeg was managed. First that resulted into an attempt to move old-style development elsewhere and reinstating new development under old name. Since Fabrice was in favour of old group and controls ffmpeg.org DNS entry, new model development group was forced out and now is known under a name of Libav.

But what is the root of disagreements? The “legendary” leader, Michael Niedermayer. Legendary in the sence that it’s a legend and not a reality.

At least since 2004 (when I joined the project) FFmpeg was rather a self-organizing community of developers, each with his own goals. Somebody wanted to play movie trailers encoded with QuickTime (hi there, Mike!), somebody wanted to play obscure game formats, somebody just wanted to support anything that he could reverse engineer (that’s me and probably Mike and other people as well). Diego Biurrun tried to bring project in shape by introducing formatting conventions (in early days nobody cared about style much), he and later Måns Rullgård made FFmpeg build system almost perfect, also Måns and Baptiste Coudurier (and many other people) worked on improving or introducing support for common formats.

Later when FFmpeg started participating in GSoCs, at first it was handled by Mike and now by Ronald Bultje. Our test system — FATE started as Mike’s experiment for automated testing regressions for many parts. Later it was completely redesigned and rewritten by Måns who also used a lot of his own hardware to provide test results so FFmpeg was tested on variety of platforms and compilers (most non-x86 things at our FATE are because of his work).

Bug tracking system was set up by Luca and he also found a hosting for it. A lot of services for FFmpeg were run on hardware of Attila Kinali (and even bandwidth and hosting for main server was his achievement). And recent Subversion -> Git transition with merging history from SwScaler is mostly done by Janne Grunau.

So, what’s the role of leader in FFmpeg? None! Almost every significant action was done by somebody else. Were they following some roadmap devised by him? There is no such thing either. Maybe it’s his social skills that kept community together? Wrong again, he caused some people to leave project (and not only the last year, Baptiste would serve good example) and different service maintainers too — by forcing his idiosyncrasies on project (like long-standing DTS guessing issue) or ordering service maintainers around.

And his role as lead developer has been diminishing probably since 2004. I can’t deny he did outstanding work on optimising H.263-based encoders and decoders and writing H.264 decoder, writing and developing some other stuff and providing reviews for patches. But what does he do in recent time? I can’t name anything significant. And from technical point he can’t serve as example: he never cared much about architectures beside x86 nor about his code being easily understandable.

Thus, some developers had had enough and forked. It’s still self-organized community with people contributing to what they seem important and nobody to order around (and not that much stalling on patch reviewing like in times of designated maintainers either).

This fork seems to moved murky waters and some trolls (mostly from MPlayer project that have no relation to FFmpeg at all) reappeared after long time; I cannot directly blame Michael on it but it seems suspicious for me. And the messages I’ve read on ffmpeg-devel between forking and creating Libav made me mostly disgusted, so I’ve unsubscribed from FFmpeg mailing lists and don’t participate in FFmpeg anymore. What goes there is not my concern anymore and I’m happy with Libav.

P.S. Also since most of new things in FFmpeg were introduced despite of him (like Git transition and releases), I can’t forget one historical analogy. In German “the leader” is “der Führer”, but that word is rarely used nowadays because there was another Austrian who completely spoiled its meaning.

Almost full Bink support

Sunday, February 13th, 2011

Thank to the efforts of Peter Ross we finally got Bink version b video support. Now I can stop nagging him about it and he may work on something else. I think it’s almost a year since we had game decoders added to FFmpeg, we should do that more often.

P.S. Why I say almost full? There is still some issue with audio in Bink-b, after Peter resolves it FFmpeg will have complete Bink decoding support.

Update: thanks to Peter’s effort we have full Bink-b support now, including both video and audio.

All codecs roads lead to FFmpeg

Saturday, February 6th, 2010

This is written mainly as a response to some flamewars.

All codecs may be divided into two categories — mature codecs and developing codecs. In the first case we have frozen bitstream format and not so many enhancements to codebase supporting that codec. In the second case we have codec that may change bitstream format and (what is quite important) encoder features.

FFmpeg itself went that way — from highly experimental H.26x encoder and decoder to rather stable set of almost all decoders available around and several encoders. Since there are some coding rules and conventions and existing framework in it, it makes it very convenient place to implement decoders — you can reuse a lot of code optimised for many platforms (so you don’t have to care about DCT speed, for example) and users don’t have to worry about adding new decoding interface and new lines in configure script, it’s all handled inside libavcodec. And “NIH syndrome” also gives a benefit here — you don’t have to worry about additional libraries (and original codec devs will have their codec specs tested as well).
You know the other advantages of this approach too.

In the same time those features make FFmpeg a bad place for having still evolving encoders for they are not likely to fit into existing framework so easy. The best this tension could be viewed in our interaction with certain encoder. They constantly modify this encoder, so existing FFmpeg options and presets are not good for them and it’s hard to tell how well it will work. Now let’s see what happens if x264 code will be merged into FFmpeg. It will put a rather harsh constraint on x264 developers because it’s hard to tell what change breaks other codecs (changes behaviour, whatever) or vice versa. The same applies to codec-specific features (like muxer using some encoder information, think H.264+MPEG-TS).

On the other hand, it is much easier to incorporate into FFmpeg an encoder not changing so much — some compromises should be made on common interface, some parts replaced with standard FFmpeg routines and voila!

I think that’s the reason we have a lot of decoders and not so many lossy encoders (especially not so many lossy encoders with good quality) in last N years. And it’s the reason why encoder should be originated as standalone projects and merged when they are stable. I’d also like to note that FFmpeg has standing issues with providing better framework for non-H.261 based codecs and descendants (where is codec-independent rate control and motion estimation?), maybe this affected Snow development as well. Anyway, let’s live and see how all these things will be resolved.

2009 — a Year of Hopes

Thursday, December 31st, 2009

Well, I think it’s time to look at FFmpeg achievements for this year.
Main achievement is without any doubt breeding Sus Scrofa Avionica — FFmpeg 0.5 release. It took only 5 or so years.

What has been added (mostly from Changelog):

  1. FunCom ISS demuxer and decoder
  2. various Electronic Arts formats support
  3. Gopher protocol
  4. MXF D-10 muxer
  5. Ogg muxer improvements
  6. VQF decoding support
  7. PCX encoder
  8. Cook multichannel support
  9. DPX decoder
  10. ALS decoder
  11. WMAPro decoder
  12. Atrac 1 decoder
  13. many other smaller additions and enhancements

What I did:

  1. out WavPack decoder now support hybrid coding as well
  2. PB-frames support for H.263 decoder, so Intel I263 files are played more or less fine
  3. some RTMP support
  4. Aura 1 and 2 decoders
  5. fixed long-standing bug in Interplay Video MVE — now it detects 16-bit variant of video (and decodes it)
  6. some improvements on SwScaler — 48-bit RGB support and rewriting our YUV to RGB conversion code, so SwScaler could be used under LGPL

But mostly this year gave us hopes for a lot of even more important things:

  1. there is hope that our AAC encoder will work
  2. there is hope that our AAC decoder will support SBR (and we can remove libfaad2 support)
  3. there is hope that RTMP input will work with all commercial servers as supposed
  4. Bink support is at least 90% complete, it’s mostly cleanup and bugfixing left to do
  5. Indeo 4 and Indeo 5 decoders. Boy, how I miss them!
  6. Apple codecs (ProRes, Intermediate codec).
  7. Sipro decoder will be hopefully committed
  8. hopefully, Windows Media codec family support will be almost full too, the only things left to RE are their speech codec, lossless audio codec, screen capture codecs and some WMV3 leftovers
  9. There is hope that some other unfinished work will be completed. For example, I have almost working LucasArts SMUSH video player based on some older work by somebody from ScummVM, MS Video 1 encoder. Other people for sure have some unfinished stuff too.
  10. I have a faint hope that Discworld 3 FMV will be playable with something opensourced. And then by FFmpeg.
  11. Finally, I hope Mike will finish his Xan WC4 decoder.

Gdium optimizations

Tuesday, November 24th, 2009

Since I’m not going to work at this soon (have more stuff to do), I publish that stuff I did. Grab
tgzipped sources here. Most of it does not give any significant speedup because of internal Loongson structure, so it’s just proof of a concept.

F2 40 01 2A (some notes on SIMD, instruction sets and everything)

Tuesday, September 15th, 2009

I’ve been following the steps of Måns and got myself Gdium too. Since it’s no fun just owning less-spread computer architecture and not writing anything on it, I’ve tried SIMDifying the easiest operations one can do on it — vector sum, vector subtraction and vector scalar product. And I have a decoder that uses those operations extensively, so why not try to benchmark it a bit?

Test sample was first 26 seconds of Monkey Audio file with insane compression since this mode uses longest filters and benefits from SIMD most (and is slow enough even for short samples ;). In all cases I’m the one who has written SIMD code, so it’s fair 🙂

PowerPC (Freescale 7447A 1.42 GHz): 25 seconds and 6 seconds
MIPS (Loongson 2F 900 MHz): 37 seconds and 7 seconds
ARM (Cortex A8 600 MHz): 138 seconds and 22 seconds
x86 (Intel Atom N270 800 MHz): 50 seconds and 9 seconds

Mind you, SIMD instructions in Loongson are custom for that CPU and modelled after MMX (64-bit registers, actually reusing FPU regs, similar names) but at least they are done in RISC fashion, i.e. you can store result in some other register.


I’ve also looked out of interest at binary representation of SIMD. On x86 the principle is to prefix SIMD instruction (usually with 0x66 “opcode for CPU with half of current bits” byte) so SSE7 instructions will look like instructions for 1-bit FPU on Intel 4004 predecessor and will take 8-16 bytes to represent.

Other architectures use simple 32-bit word for any instruction. NEON (on ARM) and AltiVec (PowerPC) use some opcodes in general instruction space, Loongson 2 SIMD are custom calls to the second co-processor.

Talking about instruction sets I cannot omit the fact that IDA 5.2 sucks at disassembling PowerPC code (not only AltiVec but some of the core instructions too) and objdump sucks at disassembling MacOSX format (it ignores internal structure and disassembles it as raw file), that looks like the reason why we don’t have Apple Intermediate Codec RE’d yet.


P.S. Jag vill gärna få AVR32, BlackFin, ColdFire och andra exotisk CPU:ar. Alpha eller Sparc är bra ochså men det är bara orealistisk, tror jag.