Kostya's Boring Codec World

Since 2006 FFmpeg (and libav when it existed and was active) has been participating in the Summer of Code program. Essentially it’s students working on tasks for different project with one corporation paying for the successful completion of the task during summer (back in the day it was $4.5k, no idea what happens now). While the sum is not remarkable by American standards, it was high by Central European and Asian standards. And of course there were students who were after money and disappeared after they got them (in one case with unbelievable claims about why the health problems prevented him from the completion). In one case there was a student who essentially plagiarised another project, in another case a student dumped a lot of code with unknown functionality so it was hard to understand or review. But here I want to talk about the students who actually stuck around after the program was over and even did something else.

The first such student would be Robert Swain. He had been around before and worked mostly on the x264 wrapper. He tried to implement AMR-NB decoder but failed to complete it (considering how convoluted those speech codecs are it’s hard to blame him) but he later redeemed himself by working on AAC decoder (which was a task another SoC student failed). Eventually he got hired by Collabora.

Reynaldo H. Verdejo Pinochet worked on QCELP decoder (and it took a long time to complete). IIRC he went to work for Samsung and tried to support FFmpeg from there.

Alexander Strange was probably the best student. He made some H.264 decoder optimisations before and was chosen to introduce multi-threaded decoding in libavcodec. IIRC he forked it into ffmpeg-mt, created the framework and implemented it for some codecs. So now you know whom to thank for multithreaded decoding.

Jai Menon is responsible for ALAC encoder. Afterwards he submitted random patches, often for libopenjpeg wrapper.

Zhentan Feng wrote MXF muxer and worked on MMS later.

Thilo Borgmann did MPEG-4 ALS decoder. Also he implemented CorePNG as his qualification task that required some API changes (the codec relies on keyframe status set by the container and that information was not passed to the decoder before). As I mentioned before, he’s still active in the project.

Marcelo Galvão Póvoa wrote AMR-WB decoder.

Alberto Delmás reverse engineered MSS2 which was not an easy task as it combines a non-trivial arithmetic coding for lossless parts with regions coded as WMV3 frames. He also REd Windows Media Image/Presentation codec which was WMV3-coded frames with various transforms applied to them.

And of course I can’t forget Mashiat Sarker Shakkhar. This student from Bangladesh worked on WMA Lossless decoder and improving VC-1 decoder. When he searched for a job he asked if I can give recommendations to his potential employer—which I did to my best and he got employed. When we met at VDD15 he was still happy to see me.

There was Guillaume Martres who single-handedly wrote most of the working H.265 decoder. It spawned three forks with different optimisations: beside the expected two there was also OpenHEVC (which was later merged).

From newer students I should mention Rostislav Pehlivanov who is responsible for creating AAC encoder for libavcodec (previously it was just bitstream writer as the quality was abysmal, as expected from yet another failed SoC project) and later he also wrote Opus encoder as well as Dirac encoder (this one on behalf of Kieran’s company).

Here I mentioned just newcomers, there were some long-standing developers participating in the program with varying degrees of success. For example, Diego Biurrun was selected to write Flash Screen Video 2 decoder using decompiled source as the reference (so it was mostly the matter of refactoring it) but he got passionate about other things and failed to do the tasks. Jan Ekström wrote Ut Video encoder. My projects were hit-and-miss (RTMP was okay, VC-1 decoder worked but not in full, RV4 decoder worked eventually but it took years to finish, AAC encoder failed).

Overall, in many cases even if the student failed to complete the task in time and left, the results could be salvaged and completed later. That’s how FFmpeg got some of its decoders and even libavfilter after all.

This entry was posted on Monday, January 16th, 2023 at 3:42 am and is filed under FFhistory. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.