Today I’d like to talk about how opensource projects are supported by “community” on example of FFmpeg/Libav.
Obviously I’ve chosen this example because I know some facts about internal politics and the fact libavcodec
is de facto multimedia decoding standard used on every platform and by most multimedia processing tools out there.
What good did it bring to the project(s)? Some fame but that’s probably it—the largest users don’t even bother to acknowledge in public that they use it (cough, BaidUTube, cough). There’s an enormous amount of code (that serves as a good compiler suite too) but it’s maintained mostly by volunteers and people who have to use it at work (as they were hired because they worked on it in the first place). That’s it: the best material gain is an employment because you’ve showed off your skills or you can take occasional consulting work with varying quality of tasks and pay. Some people are paid to improve or write new decoder. Some are hired to work on improving protocol support and hardly paid at all (true story). Some simply sigh at yet another “please implement this for my app” mail. That’s good but where are the money to pay for task the project itself (e.g. refactor old horrible code, add tests, implement new feature or fix some old problem)?
There was an attempt to set up a foundation to gather money and use them for the project but it didn’t work well even before the split and got completely derailed after; and of course it was not a good idea to set it up in the USA since IRS refused to recognize it as non-profit organisation as other open-source projects have experienced (including X.org). The best part is how much money it could raise—I don’t remember the actual sum but it was relatively low like less than $20,000 (please correct me if I’m wrong) and it came mostly from caught (L)GPL violators IIRC.
Let’s take a successful opensource project that uses libavcodec
, that would be VLC. For example, last VideoLAN Developer Days definitely costed tens and tens of thousands ${proper_currency}
— for accommodating about hundred of people, reimbursing (at least some of) their travel costs, food etc etc; the event was sponsored by the largest Internet advertising company, the largest French advertising company and the largest French VideoLAN advertising company. I remember talks that they could even employ one developer full-time. And would VLC be any useful without libavcodec
? And while they probably are the biggest opensource supporter of FFmpeg (they host their Git repository after all) and their developers write some code time from time they hardly do anything else—there is a bounty program but it’s a complete joke since it lists mostly done tasks nobody will claim reward for (and despite me pointing them to that fact nothing has been done). And obviously it doesn’t have tasks that would be beneficial for FFmpeg/Libav but not for VLC directly.
Let’s take a look at some commercial user. All those video hosting sites are available mostly because there’s enough bandwidth to stream video and because there is free code to support most of the formats uploaded by users so they don’t have to bother about details. And that free code is… (no points for guessing). And the biggest video hosting site was mentioned above. They don’t provide any support nor even acknowledge that they use certain opensource projects. Of course one can point to Baidu Summer of Code program but it’s for involving students into opensource (and probably finding fresh meat for themselves) and it doesn’t work that good for projects—you mostly have students willing to get money and/or credit for résumé. They tend to do the task and disappear completely. I’m sure that projects would prefer to have financial freedom instead to sponsor specific tasks that could be undertaken by anyone (not just students) and last whatever time is needed (not just summer). If the project gets some support from large companies it’s usually because a developer from that project working in that company convinced the management to do so.
Even worse is the situation with distributions because they tend to demand free technical support from you: fixing your own bugs, fixing other projects that use your code and such. What do you get in return? Nothing. The recent Debian and Xscreensaver messup is not some special example, it’s too typical.
It’s funny how the best sponsor for Libav might be Lu_minem.it—a small Italian company whose founder is Libav developer and thus knows what the project needs.
How do I fit into all of this? I was FFmpeg and then Libav developer, later a multimedia related work has found me because of my work on decoders (I’d been trying to find a good job myself but failed and had to accept the best offer I got). I was around core developers of the projects and thus could learn the facts written above. What have I got beside developing experience, acquaintances and pessimistic outlook on life? BSoC 2006-2009 participation (where I managed to finish only one project in time really despite the formal passing of it) plus some smaller sum of money for rewriting a component of swscale
to make it LGPL. And some free dinners from VideoLAN foundation. So I’m fine but seeing that the project cannot afford paying a developer for some internal project (like writing a new RealMedia demuxer) is still very sad.
Yes, that it is very sad side of story/money.
Hello Kostya,
Well that’s not true:
“And obviously it doesn’t have tasks that would be beneficial for FFmpeg/Libav but not for VLC directly.”
We’re also hosting fate instance operated by Diego.
That sentence was about bounty program so it’s true. The fact you host FATE instance is nice though.
Yup it’s sad that companies doesn’t donate to those projects either money or developers (for working on already reported bugs). This software could be so much better by now, but mostly there is a stagnation (as I can see).
Well speaking as one of the evil corporate overlords, the problem is more a case of providing nice fundable projects. Projects with uncertainty are much harder to fund because it’s essentially a punt in the dark. There are not many simple fundable projects remaining in my opinion.
There is plenty to do that I can see.
Obviously radical changes get more inertia to overcome.
What changes would you consider commercially valuable (I.e doing something faster or higher quality etc)
– the bitstream reader (done, the 10% speedup for mezzanine formats is quite cute, isn’t it?)
– current and future codecs development
– improved format support
– better and uniform API
There is plenty to do as you can read on Kostya’s and my blog.
@Kieran: And why do commercial entities like you only consider speed and image quality commercially valuable? What about general maintenance and bug fixing? One would expect that the existence and viability of a project you use also has commercial value in and of its own.
@Diego
Because history shows that general maintenance and bugs which don’t directly affect the commercial entity won’t get fixed. That’s the fundamental problem and it’s why only some kind of consortium like the Linux Foundation is the only working method of actively maintaining projects.
You have to look at it from the commercial viewpoint (because those companies won’t look at it from a code viewpoint): “What will the work we are funding let us do that we couldn’t do before”.
@Kieran
“Because history shows that general maintenance and bugs which don’t directly affect the commercial entity won’t get fixed.”
I have trouble parsing that sentence, I see multiple possible meanings that you may be trying to convery.
What troubles me is the freeloader mentality I notice in many commercial entities. Getting something extra is always nice, but complex software requires continuous maintenance to keep running as the environment changes and quality requirements increase. What commercial entities get out of it is the ability to (continue to) run their businesses.
Yes, the free rider problem is a known problem – I won’t pretend to be an armchair economist so I point you to wikipedia: https://en.wikipedia.org/wiki/Free_rider_problem
Anyway, it’s like arguing for businesses to voluntarily pay extra taxes, they won’t do it unless there’s something in it for themselves. Until there is a problem that will break X on day Y and cost Z in lost money, it’s very difficult to sell the idea of maintenance of a component.
History shows that the only way to fund this is via something like the Linux Foundation which essentially is a marketing tool that funds development. Or via a direct support contract where companies get bugs fixed that affect them (they are not going to fund whitespace changes in game codecs as an extreme case) in a given time window. The problem is FFmpeg/libav works too well and so breakage is quite rare.