Yes, it’s about FFmpeg and Libav again. And yes, I consider them both to be failed projects (not that their basic goal is failed and they provide even less multimedia support than GStreamer with no external libraries used), I mean the state of the project as living and developing entity.
Even if I mostly emulate Derek nowadays—i.e. unsubscribed from FFmpeg and Libav mailing lists, do nothing productive, wait for somebody to reverse engineer codecs I care about somewhat (that would be ClearVideo, thank you very much). Yet I peruse development-related resources for both projects (mostly for finding laughs) and sometimes I see gems like this (it was pointed at in the comments as well since it answers some questions I’ve asked before).
First, I’d like to outline how large projects are organised and what to expect in general. So, if you have a large and used project you’ll have at least these components:
- codebase (normal projects have some code to run after all);
- developers (to add features, fix bugs and such);
- users (to annoy developers and once in a while to provide sensible bugreport or feature request);
- infrastructure (hosting for code, means to communicate for developers, maybe even support for users).
Developers can be also divided into three main categories:
- core developers—the ones who do main work on the codebase and do it in regular manner (they might intersect with the next category too);
- corporate developers—the ones who do work mostly on behalf of their companies (e.g. add a feature they need internally so they don’t have to maintain it themselves);
- contributors—developers who add some feature or provide some bugfix because they needed it themselves, they do it irregularly or even just once (again, they might intersect with the previous category).
This division is by no means perfect but it shows the main forces behind development: those who treat is as a hobby, those who do it for their benefit (i.e. making money with/from it) and those who use it and just want to be it a bit more suited to their personal needs.
So, with that all in mind let’s look at the projects:
FFmpeg
Codebase. It’s a complete mess. And its git history is even worse. The running joke is that who cares what that piece of code does, it’s FFeature so it must be kept at whatever cost (that’s how you get double decoders, demuxers and encoders; an outstanding example there would be libutvideo wrappers—refer for the details to ffmpeg-devel mailing list).
Developers. Because of the merging policy (that is likely to be codified soon—see this document again) many developers of FFmpeg code are not FFmpeg developers. And yet they are dictating API to be used in FFmpeg: the first example that also involves me—I’ve proposed side data for packets in Libav, FFmpeg hesitated for a bit yet included it with such flattering message; the most of examples include Anton’s work from introducing refcounted buffers to splitting codec parameters into separate structure—in any case FFmpeg simply takes it and converts their code to comply with a new practice (even if it has to include some horrible hacks). If that doesn’t cry out loud “a failed project” I don’t know what does.
Also (even if I’m stepping onto minefield) some FFmpeg developers are completely unfit for collective work because of their personal qualities. People may make jokes about providing full console output of ffmpeg command but it’s not Carl who’s the main problem in FFmpeg (yes, people who didn’t work on MPlayer might think otherwise; I still believe he’d be a decent leader for FFmpeg—mostly because he doesn’t focus just on technical side and he’s unlikely to be treated as a technical god who can’t make any mistake or write less than perfect code). Here it’s more about Michael and Clément—the former never really understood what being a leader really is or what resigning from a leader means (anyone disagreeing please ban yourself from a mailing list of your choice for 24 hours), the latter does not understand people at all (neither does Michael)—I’m not going to paste the link to the same document for the third time, I’ll simply quote the relevant part:
Any Libav developer is of course welcome anytime to contribute directly to the
FFmpeg tree. Of course, we fully understand and are forced to accept that very
few Libav developers are interested in doing so, but we still want to recognize
their work.
Here’s an excerpt from Michael’s mail:
> Don’t you think you should remove Diego, Måns, Kostya, … as well?
They didnt ask me to remove them, they didnt remove themselfs even
though they could, they didnt post a patch to remove themselfs.
No contributor said that he contacted them and they no longer maintain
the code they are listed for. (or i missed that)
Well, if it’s hard to realize that Libav developers don’t want to contribute to FFmpeg and don’t want to do anything with it even though it’s been over five years then you really have a problem. And I’ve expressed my thought on reuniting both projects already.
Users. You know, there’s a difference between catering to your users and selling out completely (to put it mildly). When you see some changes being done in interests of some third party often without mentioning it that looks suspicious. I’m not against making money off your work but when it’s not even mentioning the fact it looks strange; when you have a decoder with a copyright assigned to some company it’s fine, but when you have fixes for files nobody has seen or FFv1 features added because it was all paid by somebody (see here slide 12) it looks not completely honest even if there’s nothing wrong with it.
Infrastructure. From what I understood FFmpeg services are now hosted on various boxes with no plan or idea (i.e. if somebody could provide a box for something they took it) and there’s no system administrator for these boxes. Again, as I understand it, they were kicked out of Hungary for some reason and even though they got a free server and hosting in Bulgaria they cannot use that box properly because there’s nobody to set it up properly and maintain afterwards. Sounds like fail to me.
Libav
This project is failed for the different reasons but failed nevertheless.
Codebase. While it’s mostly fine sadly new features hardly come in. Just two examples—there have been talks about replacing libswscale since ages, two years ago they’d started to design it (and it went nowhere), then I offered my design with a PoC (yes, piece of that) code to test it (that’s how NAScale was born), people work on integrating it into Libav a bit and that’s all—nothing has happened yet; the second example is bitstream reader replacement—since its submission in April nothing has come out of it as all traction was lost in bikeshedding. Is it failure or what?
Developers. Here we have two problems—some FFmpeg folks and some core developers. I’ve written about the former before so let’s talk about the latter. Surprisingly or not there are counterparts for Austrian FFmpeg developers in Libav. Where in FFmpeg you have Carl Eugen, in Libav there’s Diego and I guess many have suffered from his perfectionism (in form of proper formatting). And instead of Michael there’s Anton. While he is not that leadery in general sense, he’s the one introducing big changes in API that are hardly discussed before. And even worse thing—he tries to make all nontrivial code go through him, QSV support is a good example: Maxym Dmytrychenko had submitted initial support but it was not deemed good enough so Luca Barbato had to rework it into proper form. And what do you know? It turned out to be not good enough for Anton so he worked on it himself with the result not being much different from Luca’s. And since nothing is being done about that I consider it to be a failure.
Users. Sadly, there seems to exist not so many of them which is a fail. On the other hoof they don’t need to deal with distros and Baidu and that’s a blessing by itself. Though there is still an issue with FFmpeg users who bother (ex-)developers for features present in FFmpeg but not in Libav (or present in different form), like Blackmagic card support or prores_ks encoder (hint: there’s no encoder with such name in Libav and it’s my personal pleasure to ignore mails about it).
Infrastructure. From what I heard thanks to Attila and Janne everything is working fine.
Well, maybe I should continue with Actimagine VX codec at last and forget about multimedia outside work matters afterwards (insert the obvious joke about this not hurting NihAV development at all).
I do not mind that a feature takes long time to appear in Libav as long:
– it does appear eventually
– the end outcome is good
– the largest amount of people I care about is happy about it
That said I take your critics seriously and I’ll try harder to get avscale in within the year =)
I want you back in the project =)
libutvideo wrapper have been removed in FFmpeg.
And how much time and effort did it take? And all the changes to make libutvideo wrapper still work somehow?
libutvideo wrapper was useful back then when native utvideo encoder was developed and when lib was still functional and portable. Later it was kept because of single reason, named Carl.
Well, isn’t that single reason a good warning there’s something wrong with the project? To put it into oversimplified form, developers leave Libav because they can’t do all what they want and developers leave FFmpeg because other developers can do all what they want. And there are no working methods to fix that (unless you consider voting and following 24h ban a success—it that case please tell me why do you think so).
Nobody wanted to vote for 4 month ban….
How much actual features does LibAV develop these days?
I don’t see everything naturally, but your writeup kinda confirms the impression that it is mostly rewrites of stuff, API changes, and refactors (and Diego who only did cosmetics IIRC). Little actual new things, speedups or improvements that would concern the end user of ffmpeg or software that uses it as backend. (Feel free to correct me.)
That might be part of the reason there is fewer users, if the development mainly placates the developers themselves?
Beside all the hardware acceleration work, all the work on the parsers and bitstream filters, the new bitstream reader work, that Kostya wants in sooner than later, and some more stuff from yours truly I’ll publish once the current batches of code settles in =)
Looks to me that you do not want to see. Well, you are an egregious fan of FFmpeg after all =)
The bitstream stuff is just utility component that replaces work done by older implementation of the same thing, isn’t it? Hardware acceleration of important formats AFAIK also worked for years before? Although maybe that was partially player/client-side implementations, IDK.
It’s true that there probably aren’t that many tangible improvements of the sort I would want in FFmpeg either, I guess. I can think of BBB’s speedups of VP9 few years ago – thanks to which apparently you can decode 4K VP9 in 60fps on 4GHz AMD quad – not bad, plus he also added some 32bit/SSE2 SIMD after some critique 🙂
Otherwise there is DTS-MA decoding, but that is external library.
You guys have points for HEVC SIMD, but afaik your decoder is incompatible with a lot of other optimizations in ffmpeg, which means that until your version of decoder overtakes it in overally speed, any partial improvement is useless (unless it can be combined with things in ff side).
New scaler would be a plus because swscale is terrible even from users’s pov, so rewrite would count as new functionality, but OTOH there is zimg/zscale too (dunno how fast it is though).
——————————-
I’m not an egregious fan BTW. I admit that from what I know (I was looking at irc logs and MLs of FFmpeg from time to time even before the schism, I guess maybe as far back as 2009/2010) I think you guys (LibAV) don’t have a fair view of the other side. OTOH some of people active or inactive in ffmpeg (there are 3 names) are also unfair towards you, or even act like assholes (there are two obvious names, everybody knows I guess). In LibAV/FF flames, you guys had asshole moments too, though.
Ah, looks like I forgot to remove the piece about “3 names”, ignore that. I think one of those people isn’t that bad, that’s why meant to change it to 2 names but failed to edit it properly.
Anyway, I have somewhat more sympathise for FFmpeg (also they were the underdog after the coup business which was shady IMHO), but otherwise the whole business and you guys’ grudges, animosities and demonized views of each other – that is VERY CLEARLY kindergarten-tier bullshit and idiocy, to me. And of course, not helping us end users, as I said last time.
@Mandarinka if nothing of value has been developed one should wonder why merges keep happening 🙂
And if I learned something from my mistakes, stay away from the ffmpeg/libav debate, there is nothing of value you can add if not more drama and old flames.
You are an egregious fan, and trying to pass as a neutral party is quite silly (anybody can just google your name and makes his mind easily).
No the new bitstream has some interesting differences beside the fact it is more rational API wise (e.g. speed boosts on x86_64, speed regression on x86_32 for mezzanine formats) and we are spending time on it to make so that the speed regression are contained or nulled even in those cases.
You seem to not know that hardware acceleration bridges with some hardware and there are multiple SDK that have to be covered. But again, you are a fan so you will keep you bias, ignore what we do in Libav or downplay it.
And as Koda said: why are they so interested in take our code if nothing of value is being developed?
@Luca Barbato
Note that I didn’t say there is nothing of value at all. If you look to my first post, it is actually a question about this.
Dunno what can be googled by my nick*, it’s not like I actually post about this stuff much besides the few blogposts here.
Actually I think I got tempted to respond on another blogposts of yours, IIRC before I had the impression that it a bit inflated mistakes of the other side and painting libav in saint colors, basically I only minded that it was quite black and white. I think that comment never made it out of the moderation, though (I checked 4-5 months later, so maybe it just took time?). And no, I don’t want to take sides on your thingy, I just thought that if you saw that outsiders have a bit different view that you have, such input could help you a bit.
* Hmm, maybe on Doom9? I recall I had some snarky comment about you guys being lazy and not optimizing HEVC playback. I recall that, maybe it ended up reading harsher to libav than to ffmpeg (which is guilty too after all), in that case sorry.
I also recall that I wrote there that your refactors of the H264 decoder slowed things down. I though that was not untrue – am I wrong there?
Hmm, now when I googled myself, I discovered your comment from May: http://codecs.multimedia.cx/?p=1026#comment-216777
For the record, I don’t think I was ever in #ffmpeg-devel and I think I only went to #ffmpeg itself once, many years ago, to report something. I have only followed #x264, #x265, #VP8, #theora and #daala on Freenode. And #MPlayer in 2008 or so?
I only look at their publicly available logs from time to time, here: http://mplayerhq.hu/mailman/listinfo/ffmpeg-devel-irc
(I think I spend too much time on that too, at some point I was interested in what development takes place in decoders, particularly at the start of HEVC time in 2013-2014, and it became a bad habit. Also I guess some some of the weird drama that happens from time to time was appealing to morbid curiosity.)