Kostya's Boring Codec World

Inspired by recent events.

It is no secret that sometimes (or rather often, I’d say) political and business considerations prevail over technical ones. The persistent rumour said that MP3 format was not so bad originally but during the standardisation phase it had been changed to contain QMF in addition to MDCT because a certain company still help a patent on it. We have a couple of video codec standards developed not for any technical merit but rather for trying to create a patent-free formats (and failing at that). We see how many modern formats (not just audio or video, but streaming protocols as well) are essentially “one of everything” because each company tries to put its own technology there (probably for patent considerations)—and then even more companies appear with a claim to own a patent on the same technology (some of them form a patent pool, some act on their own). And of course we see Nokia (not the dead phone company and not the tyre producing one either) trying to become the SCO of this decade.

You know, the modern patent system was formed with the intent of sustaining development of new inventions: an inventor brings benefit to society with new inventions, society repays by granting that inventor a protection on exclusive rights for those inventions allowing to get profit from them. In theory a mutually beneficial scheme but people always find a way to game system and here we are. IMO the best patch to the legal system would be to strip those abusing their rights of that right, be it copyright (material part), industrial property rights or anything else. But as an optimist I expect the legions of lawyers to find a workaround for it rather fast.

Anyway, I wanted to demonstrate how political and financial interests spoiled already undead (I’ll elaborate below why I think so) project. And how a certain Frenchman paved a road with good intentions there. Of course I’m talking about FFmpeg (or jbmpeg as I name it after the current most influential person).

I think the story begins around 2015 when at VideoLAN Developer Days a certain Jean-Baptiste Kempf (known simply as j-b or sometimes jbk, a president of VideoLAN and so on) expressed an idea of re-organising the split FFmpeg and libav into something more attractive for the potential customers and making those large companies fund the development as well. I have no solid proofs but I think his efforts resulted in FFl*bs (not giving a full company name because one of its co-founders reacts strangely at the mentions) where he was one of the co-founders along with certain other prominent developers. And also around 2020 there was an attempt made to re-organise the project to have clear decision and conflict resolution process (in my terminology a move from CEmpeg to jbmpeg). As a sceptic I did not expect it to work out and looks like I was proven right.

As I understand it, the idea was to have a General Assembly consisting of active developers who could elect people to Community Committee (which resolves personal conflicts) and Technical Committee (which resolves disputes around the e.g. design approaches in the submitted patches). Surprisingly enough there was something formed. I did not expect it to work though and it failed to disappoint me. Anybody familiar with the project knows that there are certain people there with rather abrasive personalities—and as expected nobody was banned for their behaviour (if you know a case when a complaint to CC led to a ban please let me know; I heard that somebody was banned but not using that procedure). TC passing out rulings for controversial technical decisions did not seem to work out: a friend of mine asked there if her set of patches to replace bitstream reader macros set with a cleaner design was acceptable only to be said by a TC member who was opposing such patches that it should be decided by GA vote instead. Again, if you know the cases when TC actually worked as supposed please let me know.

Since it was 2020 for a couple of years, nobody cared about those committees and their names were more as a joke. Eventually the mandates expired but nobody cared to elect new committees.

This year for some reason some people decided it is a good idea to restart the whole system, it was discussed at VDD and a vote on clarifying some aspects of the system was called. Drama ensued.

First there was a long squabble about the terms, then there was an actual vote, then the person in control of voting server declared the outcome invalid and ran it a second time without anybody else’s consent. That made some of the participants protest by making their private voting token public as a mean to make their (potential) votes invalid. So while the dust has not settled down, it is not clear what to expect next. But so far the credibility of this and consequent elections seems to be low and not likely to instil certainty in the project’s future in the third parties which may consider supporting it.

That’s all politics but where are money involved, you may ask. Let’s get to that part.

Shortly before that there was another, slightly less ugly drama. There was a proposal to use donations the project accumulated to kick-start a fund-raising campaign by performing some paid work (with a specific example for such work for the person who gave the proposal) in hope it will draw attention of potential sponsors. The problem, as described, is that the amount of available money is enough to, say, sponsor some trips and an occasional new hardware for the developers but not enough to pay a decent salary for even a couple developers for a year (let alone donations are not coming in a decent amount either). Eventually the idea was shot down for the reasons that should be apparent immediately.

But you know what common denominator unites these stories? j-b. It was him who proposed the initial system with General Assembly and committees, it was him who attempted to restart it recently and (according to one guy on publicly logged IRC channel) it was his threat to leave FFl*bs (and pull his funds out of it) that made another developer make that proposal about using donation money for development grants. I am convinced that he acted with the best intentions for the project and other people’s interests (highly likely more than in his own interests and maybe even at his own expense). But you know where the road paved with good intentions leads…

Do I think that something bad will happen to the project after all this drama? Not at all, I consider it to be undead already and it can remain in that state for indefinitely long time. Why undead? Because it’s definitely not healthy, not growing (in terms of developers) yet it’s active and instils dread in the outside observers. It has not passed away (or into obscurity) only because too many software projects are based on it or use it—and yet unlike Linux kernel corporations are not eager to dedicate engineers to improve it overall (pushing wrappers for their libraries and small fixes for commonly used components is not it; CPU vendors may submit some platform-specific optimisations but it’s essentially the same thing).

So can anything be done to improve the situation? IMO yes and no. The project itself can’t be saved, partially because one person (not j-b this time) has Fabrice’s ear (who still owns the domain name and IIRC a trademark as well) which is an equivalent of nuclear option: you can’t kick such person from the project (as libav participants discovered) and thus there is no real way to make such person to abide the rules. That erodes the trust because then anybody else can do everything and nobody gets kicked out. And if jbmpeg had a leader I’d say that the leader tries to keep everything under his control by having his friends in key positions and opposing any real democratic process by discussing nits of the rules to death (especially if they are related to the voting process). But since jbmpeg has no leader I don’t know what causes the events to go the way they do.

And what about the ways to improve situation? I think a new project should be started along with a foundation to serve its interests, preferably in Switzerland or some other country that offers (perceived) protection against patent extortionists (so those do not come for Danegeld after it gets its first money). It should have a clearly defined (and enforced) set of rules right from the start so there are no untouchable developers who can do whatever they want. For similar reasons there should be no developers there known to cause drama in the past even (or especially) if they are perceived as rock stars (I think though it should be possible to pay Paul B. Mahol for one-time tasks but not to keep him permanently—he’ll get too bored and will be kicked out for trying to liven up atmosphere). The lack of well-known names should be compensated by the project reputation doing honest work: timed releases for the various OS distributions, streamlined process for (paid) implementation of the features that other companies may want and such. It will be boring, probably with no exciting features like fringe formats support, but big businesses like boring, predictable and lack of drama (or scandals of the kind libav had because of the forceful split). A typical paid job in other words (at least with a higher chance to be paid). Oh, and it would make sense to shrink down the forked codebase by throwing away all the features that are rarely used if at all but increase potential attack surface (like all the in-house formats that nobody else uses).

I don’t care much whatever happens to **mpeg. I don’t need it in my everyday use (I don’t watch videos in the browser normally and I use my own player with my own decoders for stand-alone playback), while I sometimes enjoy the drama it generates I can find another sources of entertainment, and I don’t have to deal with it at work either. So I propose purely a theoretical solution for a problem I see and I would be surprised if it is not ignored. Consider it to be a mental exercise.

This entry was posted on Tuesday, November 14th, 2023 at 4:30 pm and is filed under FFhistory, Useless Rants. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.