Recently this tweet was brought to my attention (by somebody else who saw it and found hilarious). While I agree with the message (I also find those OMGsecurity people annoying and rather counter-productive), the premise is bullshit, namely those two lines:
Arguably the most brilliant engineer in FFmpeg left because of this. He reverse engineered dozens of codecs by hand as a volunteer.
So I’ll try to explain what’s wrong with that claim and FFmpeg twitter account (or FFaccount for short) in general.
Ghidra has been available since 2019, making decompilation available for everybody. Before that for a decade we didn’t have IDA Pro with Hex-Rays decompiler plug-in (because it’s next to impossible to buy even if you could afford it—they vetted each potential buyer for eligibility) but there were enough leaks and cracked versions around to use it anyway. So unless there was somebody not aware of these tools who started contributing “dozens of codecs” after, say, 2015 and then leaving (which I find implausible; especially considering that non-trivial codec may easily take months to understand from the disassembly) I should be aware of him.
Let’s see: currently the list of supported decoders lists about 450 various audio and video codecs (minus hardware-accelerated decoders, external libraries and raw formats). If you discard external contributions and codecs implemented after specification (including the original codec source code), that number is more than halved. So overall I counted only under two hundred decoders that might be based on reverse engineering (or ported from the source code elsewhere), which narrows it to ten people at most, in reality fewer than that. I can name only four people who are responsible for reverse engineering more than ten codecs:
- Michael Niedermayer. Maybe a bit surprising but he did some RE back in the day, mostly of JPEG and H.263-based proprietary flavours (and in that time there were no good decompilers at all). But he can leave only with the project itself, so he can’t be that person;
- Fabrice Bellard. Same story but he got inactive because he had other interests to pursue (and that happened in mid-2000s);
- Peter Ross. From what I know he’s still active, working on awesome stuff and not caring if it takes years to complete;
- Paul B Mahol. I actually asked him directly and he replied that he left because his work was obstructed —not by dealing with CVEs but rather because his submissions for new filters were not accepted (and that the tweet is question is not about him). Since then he maintains librempeg where nobody tells him that his filters are conceptually flawed and he does not know anything about
libavfilterat all. I wish him all the best.
The other people I can think about either do not come close in the number of reverse engineered codecs or worked in collaboration with others on them, so their brilliance is less apparent. That is why I call this bullshit—the original claim was either half-truth bent to serve narrative or outright invented for the same purpose.
And then there was this video from some guy named Theo (now that video is hidden for some reason) where he complained about a person responsible for FFaccount to be rather unprofessional—and that’s putting it mildly (unlike how he did it). From what I saw he had some legit complaints that could be either left to the mentioned party (i.e. VLC) or explained why it’s hard implementing what he wants (maybe followed by “…that’s why we need your help”). Instead he got a nice answer boiling down to “you’re a freeloading user, you can’t know anything in principle, so shut up”. Apparently that was annoying enough for him to record that video and insult others (I don’t approve such behaviour either). The situation degenerated further when fans of both sides started doing their usual toxic stuff. To me the saddest part of it all is that FFmpeg community members refuse to see anything wrong with such behaviour and rather encourage it (because it helps account growth and such).
So I decided to dive into FFaccount and see what were the tweets in the recent time (let’s take full October—there’s enough material there and I doubt it was an exceptional month for them). Since I don’t have an account (never had, and not going to get one either), the following list is what xcancel service provides:
- October 1—some retweet of a guy praising the tool;
- October 2—a rather non-sequitur reply to a tweet advising to keep tests running in under three minutes unless you have a serious business. All just to remind about FATE (a thing which was invented, named and implemented by people who’re not with
FFmpegsince 2011 BTW); - October 3—proudly showing a comment from HN by one of their own;
- October 4—retweeting the same person talking about SIMD (in connection to
libavcodec, just as in the comment above); - October 5—a retweet of a picture reminding how modern IT infrastructure is based on the work of unpaid open-source volunteers;
- October 8—a post about removing some MMX code;
- October 9—an announcement about upcoming presentation and a retweet of a certain company praising the tool (the same company is known to not publicly admit that it uses
FFmpegBTW); - October 10—a reply to criticism about issues related to integrating codecs essentially with “you haven’t contributed to us so shut up”. One of the things that inspired that video I mentioned earlier. Another retweet of somebody praising
FFmpeg. Yet another retweet of some random thing remarkable only as being near the end of another discussion involving FFaccount (where it was not fully correct). Retweet of a donation to them being made. Another post about removing more MMX code (while reminding they keep writing assembly code by hand); - October 11—a reply to somebody endorsing low-level programming. A random repost of some reply in a discussion about GPU-based encoding. A reminder for (a new) bugtracker. Couple of retweets (three, to be more precise) praising
FFmpeg; - October 12—a link to an interesting and good post of one
FFmpegdeveloper (who wasn’t one back when he wrote it BTW). A reminder about VideoLAN Dev Days. Retweet of somebody praising the author of the post linked earlier (for his non-FFmpegwork); - October 13—a retweet praising
FFmpeg(in Portuguese, I think). A comment aboutTeamSpeakusingFFmpeg. Another retweet praisingFFmpeg(with an animated picture); - October 15—a reminder about NTTW conference. An announcement about YesWeHack participation. A comment on the sad situation with security demands. A retweet of the talk happening at NTTW about GPU FFV1 en/decoder. Another retweet praising
FFmpeg; - October 16—a reply explaining why
libavcodecH.264 decoder not being the fastest one does not matter. A take on whydav1dis faster thangav1ddespite the latter coming from Google instead of bunch of enthusiasts (and it manages to spoil it by claiming that “dav1d was written by people in their basements” which is stretching truth at least). Bullshit tweet that has started this post. A retweet from that discussion praising low-level programming. A tweet saying how some people don’t appreciate their posts about hand-written assembly while it’s what makes it fast (maybe it’s because of the sensationalist style of them like “OMG this code runs 1000x faster in AVX512 hand-written assembly than Python”—slightly exaggerated for clarity). More retweets with praise. Rather dubious claim “Indeed, we have several legendary codec reverse engineers that do mind blowing work!” (I can think only of Peter Ross, the rest have left or don’t do any mind blowing work). A rather non-sequitur reply advertising FATE (in response to somebody complaining not being amazed by new software). A retweet of insightful comment by Ronald Bultje explaining whygav1dis not so fast. One rather interesting anecdote from the discussion; - October 17—“For the avoidance of doubt security issues are taken extremely seriously in FFmpeg, but fixes are written by volunteers.” (take it as you will). A comment on random post in some discussion dissing on Rust. Thanks to the archiving community for funding work on multimedia. A retweet supporting dissing of safe languages (some people don’t know about FORTRAN). An answer why
FFmpeghas fallback C function and why that matters. More retweets with praises. A retweet with a reminder how RCT was written all in assembly. A post reminding thatFFmpeguses IRC for live chat. - October 18—a couple of posts explaining why assembly is better than intrinsics. An answer explaining support of AVX-512 subsets. Retweets of people agreeing that assembly is better than intrinsics;
- October 19—a tweet advising angry people to touch grass. Retweets of praise;
- October 20—”[FFmpeg-devel] [RFC] C++ Oh boy” (do I sense mild disapproval here?);
- October 22—retweets of praise;
- October 23—A retweet of an interesting experiment with the compilers. More retweets of praise;
- October 24—a single retweet of praise;
- October 25—a post about 3DNow! optimisations removal. A retweet of some curious experiment with
FFmpegcode. A post about patchset speeding up CRC calculation. More praise. A post about ProRes decoding on GPU with Vulkan; - October 26—retweet of the yesterday news as reported by Phoronix. An answer why new CRC calculation is more than just single SIMD instruction for calculating specific CRC32. A retweet of somebody correcting a statement about
mpv(I think it still relates to that video). Thanks donations. A reminder about touching grass (repost from 19th of October). A reminder about Demuxed conference. “How about a nice game of chess?” FSF membership endorsement repost. Some retweets of praise. A rather misleading statement about what FFaccount does (it’s more about what it should do but in reality it’s half that and half something else entirely); - October 27—a link to an interesting repository with code for fast CRC calculation (plus a mention how compilers interfere with that). Some random animated image (a response to the video-related scandal?). An answer about what VDD is. Retweets of thanks to the project. A repost of one
FFmpegdeveloper about GSoC Mentor Summit. Praises to the archiving community as an example of proper partnership with open-source multimedia. A post about an interesting patchset. A (rather hypocritical IMO but still somewhat reasonable) post calling for calming down and not attack community members after recent drama. A retweet of a response about another fast CRC implementation. Retweet of some Grok answer about how muchFFmpegis used. A post claiming that they’re returning to the regular schedule (which does not fully match what I’ve seen here so far even forgetting all the dramas). Some AI-generated cringe; - October 28—retweets of praise and support (including donations). A retweet and reply on an interesting link about certain kind of bit tricks. A post about SIMD in Formula 1;
- October 29—tweet about $100k donation. Some post about the FFaccount people remaining the same and attempts to damage control after that video-related scandal. Some reports from Demuxed conference (with focus on
FFmpegrole of course). Some retweets with praises; - October 30—another post about removing MMX code. A repost of a link to some useful information. Some random reply The lion does not concern himself with the opinion of sheep (talking about themselves apparently, it does not sound arrogant at all). An encouragement to send
FFmpegmemes. Some more reposts with those memes, praise and reports from Demuxed conference. A post about new patch that improves Smush decoding with rather deceptive comment thatFFmpegaims to play every video file (the deception is that they don’t do much to achieve that goal beside waiting for somebody to do the work and submit patches—unlike, say,Librempeg); - October 31—more reposts with memes. A complaint about AI-generated vulnerability report on the Smush codec from yesterday (it would be better if the ending “A 1990s game codec is not the same as a mainstream codec like H.264.” would not collide with “We take security very seriously” from the beginning; probably “…but selectively” was missed somewhere). A retweet with praise from somebody who believed the claim about “every video file ever made” (Bink2 when BTW?). A retweet of Grok-generated answer that
FFmpegtakes security seriously (according to their own tweets at least). A retweet of some replies from other people in that discussion. Couple of thanks for the conference organisers and somebody sending a patch; - November 1—more retweets of people agreeing with their position in that continuing discussion. “This account also aims to raise awareness of the challenges volunteer run projects have.” (It does, but rather clumsily IMO—see the very start of this post). And here’s a good place to stop.
Overall, I can split this into several categories: retweeting praise from others saying how awesome FFmpeg is or how the life would not be good without it (there’s nothing wrong with that though I found it tiring and rather pointless—keeping just the tweets telling how exactly FFmpeg helps them would be much better IMO); posts and retweets about how C and hand-written assembly for SIMD is the winning combination (it would be fine if not for being overly aggressive to anybody having a different opinion); posts and retweets about unpaid open-source volunteer work being crucial to the modern IT (which is true if they would not try to stretch and bend truth to serve the narrative); general retweeting of responses where people agree with their position on whatever topic is currently being discussed; boasting about their work (uncalled for on more than one account); an occasional post or answer with technical information; project news; inside jokes; zero tweets about reverse engineering.
I got an impression that people behind the FFaccount (maybe it’s just one guy, maybe it’s several people—the body count does no matter, the overall mindset does) are jaded from the lack of corporate support and constant barrage of user requests (exactly like “please implement this feature for us for free so we can earn money by using FFmpeg in our product/service”) and either have a superiority complex (“we talk of technical stuff you don’t understand”) or believe that an arrogant way is the best way to deal with everybody (“scandal means engagement means popularity growth”). In that sense they reaped what they sowed.
Of course my words will be ignored but here’s what I’d change:
- drop unsolicited advertising—if the talk was not remotely about what you do then don’t get involved. That includes “defending” other projects on their behalf when nobody asked you about that;
- drop untruthful boasting. Considering video codecs,
FFmpegdevelopers have not produced native decoders for anything more advanced than VP9 and no encoders for anything after MPEG-4 ASP (with audio codecs the situation is better but still not ideal). H.265 and H.266 decoders were developed by people outside the project and rather donated to it, AV1 decoding requires an external library (not fromFFmpegdevelopers despite how much FFaccount wants to paintdav1das their own success), H.264 encoder as well. And it looks like you’re eager to attribute e.g. current Paul’s achievements to yourself as well. Sooner or later somebody important (unlike me) will call you on all this bullshit; - respect the others! Attacking programming languages, projects and individual people makes you seem petty and insecure in your current position. Telling users they can’t have nice things because there’s nobody to work on that thing is fine, replying “you haven’t contributed in one and only particular way we see appropriate” is rude and often insulting. You seem to enjoy something akin to monopoly but maybe one day your arrogance will repel enough users to consider something else. Or somebody will cobble up something using just the most popular formats and that will be enough for the majority of users (for example, browsers use cut-down versions of
FFmpegalready and would be happy to replace it with something more lightweight). And if you think this can’t work—MPlayerstarted this way and it’s still alive and strong in its descendants; - think about the whole picture before demanding patches. When you demand patches you can show a middle finger instead with the same result. Can you guarantee that somebody who got hyped and decided to submit a patch will not receive a complete lack of feedback or, which is worse, some George Nichols (just a random made-up name with no deeper meaning) will tell how much it sucks and needs to be redone (along with some other unrelated subsystem), turning potential contributor off forever? Are there people willing to put such patch in a proper form acceptable by a project if the original submitter does not know how? Are there people willing to teach others what and how needs to be done? Maybe it’s submit your own reviewers along with your patches? A couple of lessons on assembly and new shiny forge do not solve all these problems.
If you want respect you need to show respect yourself, if you sling shit around don’t act surprised when you get shit-storms in return.
P.S. And in case you’d try to apply the usual excuses from the FFaccount to me:
- I don’t use
FFmpeg—neither as a tool nor as a component of a player. My $DAYJOB does not depend on it either; - there are over 800 commits to my name in the
FFmpegcodebase (unless they got all removed recently—go ahead and do that, please); - some of those commits are introducing SIMD optimisations, for different architectures too (both hand-written assembly and intrinsics);
- other commits introduce decoders written by me for formats also reverse-engineered by me, from simple DPCM codecs to H.264 rip-offs;
- even now some of my work gets back by osmosis (e.g. Paul implementing decoders based on my description that gets picked up or Peter Ross basing his RV6 decoder on my description of the format and using my decoder as a reference).
There’s a reason why I left FFmpeg for libav back in 2011 (before CVEs became such a nuisance BTW) and never wanted to get back. And hopefully I’ll never have to write about FFmpeg ever again (not even an obituary).
P.P.S. I wrote this post with an ulterior motive—I don’t want to feel shame when remembering that I took part in that project. So far though the more I hear about it the more disgusted I become. Looking at its social networks activity was no improvement.