Archive for the ‘Useless Rants’ Category

Call for a new container format!

Friday, January 17th, 2025

Sometimes I remember that Matroska exists and today I also remembered how it came to existence. Its author proudly admits mixing all the buzzwords of Web 2.0 era like XML, Semantic Web etc etc and coming up with that format. Since we’re in Web 3.0 era we should have something more modern.

That is why I’m calling for a modern multimedia container format to supplant the outdated formats of the old. It should encompass all the features that make the modern Web great:

  • binary JSON as the backbone of the format;
  • central repository for the downloadable descriptions of the parts of the format (but not codecs themselves! Think of it as of MXF specification if it helps);
  • blockchain (as well as clusterchain and framechain);
  • metaverse integration;
  • decentralised storage (so that the container may refer to some data in the cloud as well as on the local disk; even MOV could do something like this);
  • and of course AI!

Some of you may ask where AI can be applied in this scenario. The answer is obvious—transforming input data for better compression (let alone generating metadata or enabling better integration with other Web 3.0 products). A good model should be able to do the same savings like Matroska did by e.g. shaving off common headers bytes from each frame but without special mapping. An excellent model may generate the content by the embedded description instead of transmitting AV2 video. And of course the central repository will contain the description of models and parameters to be used (in addition to the descriptions of better representation of container parts layout). The possibilities are limitless!

Proposals should be sent to the Alliance for Open Media , I have worse things to deal with.

On the sorry state of opensource multimedia

Wednesday, December 25th, 2024

I’ve been wanting to write this post for a long time, with a focus on the difference between hobby project and product and about NihAV only. But a recent FFdrama made me re-think both the structure and the conclusions.

Apparently there’s another surge of developer’s discontent in jbmpeg for having mushroom treatment (not for the first time and probably not for the last one). IMO they need to realize the project is as free and democratic as Soviet Union and you need simply to agree to the things proposed by the General Secretary (definitely not the leader)—that would save time and nerves of everybody involved. As I wrote countless times before, I do not fear for the future of that project as it can keep such existence indefinitely long and here I’ll try to present my reasons why.

First of all, revolution a la libav won’t work—Michael has learned the lesson and he won’t be kicked out again (not that it really worked in 2011 but now there are no chances for that).

Second, if you split out and form an alternative, it has not so many chances to replace it. And if you decide to write anything from scratch your chances are next to zero. The rest of this post is dedicated to answering why.

Recently I re-read The Mythical Man-Month which tells not only about the author’s experience designing RedHat OS/360 but also presents more general observations and ideas. And right in the beginning he talks about the difference between a program, a programming product, and a programming systems product. Essentially, a program something a programmer writes and that it works for him on his system; a programming product is a program with documentation and support; and a programming system product is that works as a component in a larger system. And moving from one stage to another requires an effort several times larger than the previous one (I’m simplifying a lot and probably misremember something—so you’d better read the original book, it’s a worthy reading anyway).

Here we have a similar situation: writing a tool just to do things for you is straightforward, even I have managed to do it with NihAV; making it into a product requires offering much wider support for different platform configurations (for example, my videoplayer has VA-API hardware decoding enabled by default while it’s not available, say, on Windows and you need to switch that feature off there in order to build it) and different features (e.g. nihav-encoder works for testing encoding per se, but lacks ability to encode input into a good intermediate format supported by other players and encoders). And it gets even worse if you try to make it into a library ready to be used by others—beside the usual things like documentation you’re expected to guarantee some API stability and a certain level of quality. So while I may not care that my app panics/crashes in certain circumstances, it’s significantly less forgivable for a library. And of course achieving such quality level requires a lot of unexciting work on small details. Debugging is even worse.

Suppose you decided to create a fork and work from that. Then you still have a much worse position—you may have the same codebase but there are no killer features you can offer and you don’t have recognition. libav managed to succeed for a while since it was supported by some distribution maintainers—and even then users complained because the de facto brand name was replaced with some unknown thing. And I guesstimate 40% of current jbmpeg developers contribute to it in order to upstream the changes they make while using it in their employer’s product or pipeline. So how can you convince those companies to use your fork instead? And that’s not taking patent situation into account which makes a substantial support from any large company for your project rather improbable.

Good thing I’ve never intended NihAV to be a competition, but what about other projects? rust-av died because of lack of interest (Luca claims that he started it mostly to learn Rust and see how performant it can get—mission accomplished, no further development required). librempeg fares better but I doubt that Paul wants to deal with all the demands that other parties make for the honour of your stuff being included into their distribution (or being used without even a credit).

Another thing that needs to mentioned is that multimedia is no longer an attractive field. Back when I started to dabble in the field, it was rather exciting: there were many different formats around—in active use as well, and people wanted to play them not only with the proprietary players. There were libraries and players supporting only a specific subset of different formats, like avifile or libquicktime or DVD-only player. Nowadays it’s usually a combination of H.26x+AAC in MP4 or VP9/AV1+Opus in WebMKV, all formats having specifications (unless you lack Swiss Francs to pay for the ones from ISO) and new formats are not introduced that often either. Of course, we might have H.267 standardised soon but who uses even H.266? When was the last time you heard AV2 development news? The codec was supposed to be released a couple of years ago, did I miss it along with AV3? Do you remember Ghost audio codec from Xiph? Of course Fraunhofer will keep extending AAC patent lifetime by inventing new formats and naming them like OMGWTFBBQ-AAC but who really cares?

That is why I believe that no matter how dysfunctional jbmpeg is, it will keep existing in this undead state indefinitely long as it’s good enough for the most users and there’s no compelling reason (e.g. new popular formats or radically different ways to process data) to switch to anything else. The only winning move is not to play.

To NIH or not to NIH

Sunday, December 22nd, 2024

Paul of librempeg fame informs me about his achievements occasionally (and in my turn I try to remind the world time from time that this project exists and may provide you functionality hardly found elsewhere, like various filters or console formats support). And his recent work was implementing inflate routine specifically for the multimedia needs. This made me think if it makes sense to have a custom implementation for deflate decompressor and packer in multimedia project when zlib exists and I think it makes perfect sense. Of course in NihAV I NIHed it because the project concept demands it and it was a nice exercise, but in more serious projects it makes sense too and below I’ll try to explain the reasons.

Zeroth of all, a quick reminder about flexibility. RFC 1951 merely specifies the format, so the implementations can output varying bitstreams that will correctly decompress the data. Back in the day when I worked on it I mentioned how you can compress data in different ways, dedicating more time to achieve better compression. And there are more tricks not mentioned there like parallel compression.

Now, the first reason why to have your own implementation is that you can adapt it to your custom needs. As Paul demonstrated, inflating data into a custom format might be beneficial. If all you need in 99% cases is unpacking data from one buffer into another or to a frame buffer (which has padding so you have to output, say, 27 bytes, then skip 5 bytes, output 27 bytes and so on) you can do without all additional functions and not bother with a sequence of calls to partially inflate data.

Then, there’s a question of consistency. You cannot be sure that a new version of zlib will produce the same output as its previous version (I vaguely remember that there was a small scandal when GitHub releases were re-generated using different deflate implementation and resulted in a lot of headache because of old archive hashes being no longer valid; or there’s a story of some Linux distros replacing zlib with zlib-ng resulting in failed tests; and even “no compression” format may change apparently). The case with liblzma is probably a good demonstration of other reasons why it’s not always wise to rely on third-party components.

And finally, you can not merely adapt interface to your needs, you can tune it to handle your data better too. There’s a reason why there exist compressors targeting e.g. genome data. So when you compress image data, it may be beneficial to search for matches around the position right above the current line first, and the presets for the compression levels may be tuned to the different sets of trade-offs. After all, deflate is often used in screen capture codecs where real-time performance is more important than the compression ratio. But who can imagine people tinkering with the encoder trying to improve its performance in a multimedia project?

I hope this helped to convince you that there are circumstances where NIHing something may prove worthy. As a rule of thumb, if it’s easier to implement yourself than to re-use an existing library then maybe you should do so. That it the right way, and the other way lies left-pad.

Some words about Alien Isolation CDA2

Wednesday, December 11th, 2024

Some of you might’ve heard about a Finnish adventure game called Alien Incident. Apparently it had a CD release with intro and final cutscenes re-done as an animation format. Of course that’s enough to draw my attention.

It turned out that it has several features making it one of the most advanced game formats: multiple soft subtitles and checksums for each frame. The file header is followed by the text block with subtitles in several languages that are rendered by the player over the video (and the font itself is RLE-compressed). Then there’s a table of 32-bit checksums for each frame, and only after that there’s frame size table followed by actual frame data.

Video is coded in 32×40 tiles with one of several coding modes: read full tile, update marked pixels in the tile, update 2×2 blocks in the tile, or update 4×4 blocks in the tile. That data is further compressed with slightly modified LZSS. Additionally frames carry audio data and some additional commands such as fading or setting frame duration. By default player operates on 70 or 75Hz clock and each frame is displayed for several ticks (usually 5-8). So a special command in the frame data tells player to display each frame for N ticks until further notice.

And now it is time for a rant. There is an issue uncovered by decoding these files. Both files are long (finale is over 11 minutes at 18.75 fps) and have lots of palette changes (because of both scene changes and fading)—and those two things helped uncover a problem with FFmpeg. Its AVI demuxer is the problem: it scans index, finds palette change chunks (which I explicitly mark with AVIIF_NO_TIME flag), adds them to the video stream statistics (despite the flag), concludes that video stream has significantly more frames than audio stream and switches to non-interleaved mode; in that mode it disregards actual index contents and treats palette change chunks as video data. Personally I don’t care because I have my own set of tools for decoding, transcoding or playing videos that does not have this problem, but considering that virtually every other piece of software uses libavformat for handling input data, that may pose a problem for everybody else (I can afford to not care but somebody else would have to change perfectly valid output just to work around third-party bug). This is a fun case demonstrating that monopoly is evil, even if it’s a monopoly of open-source software.

P.S. Probably it’s a good occasion to remind that librempeg exists and probably both it and you can benefit from your attention to it.

So long, USA, it was a disappointment knowing you

Wednesday, November 6th, 2024

I’m going to publish this after the elections are over so no idiot can call me interfering in the elections, even if I don’t argue in favour of either candidate or political system reform. This post is simply summing up the frustrations about the mismatch of how the country views and positions itself against what it really is.

First of all, the title is about USA slipping out of relevance and not e.g. about me emigrating from the country (I’ve never been to it and it’s highly unlikely I’ll ever go there) nor about USA stopping to exist in the near future. My point of view is that while USA has done to undermine its own position, it has built enough inertia to keep going for a while—like large baobabs that can’t physically fall down and simply rot away standing.

The upcoming downfall does not make me happy as I realise how bad it will be for the whole civilised world (remember 2008? ever heard of Black Tuesday?). So consider it as an extremely late (a decade or two late) and useless warning. In theory some things still can be done to rectify the situation but does anybody believe anything substantial will be done?

Below I’ll try to present my thoughts in a logical form: what made USA great in the first place, what it did wrong in last decades (and why it hurts USA), and how it will likely to end.

What made USA a global superpower?

Let’s start with the foundation on which USian power was built. Without defining it it’s impossible to argue what actions undermine that foundation.

My short list would look like this:

  • having one of the largest pieces of land stolen from others (rivalled only by China and russia); and if you wonder why I haven’t included Canada to the list—with those countries land was not merely something seized from uncivilised tribes (which may be unthinkable now but was though more as a God-given right back then) but from other countries as well (ever heard of internationally recognised Hawaiian Kingdom and how it ended being a state of USA? do you want to visit United States (of Mexico) and tell them how Texas and California have always been a part of USA?);
  • that land being suitable for agriculture and being rich in natural resources (russia has been always complaining about the quality of its lands—even after large oil reserves were discovered; from what I heard China also complains about its territory being mostly mountains);
  • people willing to work and the state not hindering them (you won’t believe how many russian proverbs are there about the futility of working; also back in the day russia and Austro-Hungary were two empires stifling technical progress in fear of losing power—you can guess how it helped them during World War I);
  • being isolated by oceans from most other countries and not experiencing a full-scale war at its own territory in XX century—sure, USA was involved in wars but it was mostly sending troops overseas instead of, say, Japanese troops occupying half of Oregon or German bombers levelling down Bridgeport, Philadelphia and Jacksonville. If you manage to have your civil population safe, industry and economics mostly intact and working, not needing money for post-war reconstruction—that definitely gives you an edge against the countries devastated by war;
  • even more, you can help other countries in time of need tying them to you economically;
  • as the result you can play Switzerland and organise things to go by your rules that will benefit all the players involved (like free trade, technological co-operation, unified defence and so on);
  • and finally, you can build large and good equipped military force that helps keeping the rules.

And thus, by the end of XX century USA had:

  • a powerful economy;
  • the standard currency for international trade (surviving even de-pegging from the gold standard in 1971);
  • well-developed industry and financial sector;
  • being a leader in scientific research and hi-tech developments;
  • leading the most powerful military alliance, with its main antagonist temporarily withdrawing from the playing-field.

In theory every thing on that list should only ensure that other things keep getting better, so what went wrong?

Losing advantages

In one word—idiocy. In more words—greed, short-sightedness and idiocy. Almost every advantage USA had was lost either via uncontrollable greed, decisions that prioritised short-term gains over long-term consequences, and refusal to see something that does not fit your comfortable image of the world. Let’s see how each and every strong point got undermined.

A powerful economy. I’d point at 2008 financial crisis as the best demonstration of what went wrong. The financial market became so strong and successful that it managed to shatter the bonds of reality and as the result it could build something imaginary—a derivative paper on derivative paper that tries to hide a risk of suspicious credit (or mortgage of dubious value). And the total amount of CDSes issued was comparable to the whole world economy of the day (Wikipedia quotes the number of over $62 trillion of CDS amount by the end of 2007 and about $109 trillion for estimated size of world economy in 2024). You should be able to guess what reasons were at play here. I’d also add that dealing with it demonstrated short-sightedness as it seems nobody serious was punished (because they all are Too Big To Fail) which incentivises others to keeps such dangerous games—after all, you get your bonus now and the government will bail you out later if needed. And such hiccup in the economy lowered the trust in its robustness abroad.

The standard currency. While US dollar is still the main international trade currency, both the behaviour of US government and the way it imposed sanctions made other countries search a way to avoid relying on it too much. Terrorist states (like russia) do it to ensure they can function even under sanctions cutting off access to the international banking and payment processing, states supporting terrorism (like China) take preventive measures against possible sanctions, and even the EU wants to ensure it’ll manage to function no matter what ham-fisted decisions US government will make (in recent news there was a bit about EU startup backed by a lot of large European banks trying to provide an independent payment system).

Well-developed industry. (no reason to discuss financial sector again) Guess what is losing its reputation rather fast. For example, I still remember the days when Boeing was a company producing quality air planes, Intel making cutting-edge CPUs and Google search returning relevant results. Nowadays Boeing is known for its faulty planes, Intel for failing to keep with the progress and the overall state of Internet search has degraded. Why? I’d blame two main things: having duopoly (or monopoly) which makes you care less about the innovations (after all, what alternatives do your customers have?) and the usual prioritisation of showing current growth over long-term improvements (Intel is the most apparent case of not trying to invest money into semiconductor fabrication research and losing, in other cases it’s more like small cuts that result in a company not caring about the quality of its products). I’d argue this is also a consequence of companies becoming too large and divorced from reality (especially when managed by those with MBS degree instead of people who at least have an idea what the company should make and how). After all, investors mostly care about the current price of their stocks. And offloading production to China also lowers perception of USian goods being high-quality—after all, most of them are not made in the USA (and quite often, not even designed there either).

Scientific research and hi-tech developments. Those seem to have suffered in two ways: first, from the companies spending less on the research (because not all companies do production themselves anymore and thus don’t care about improving e.g. production processes—they can simply move to the lowest bidder in China instead); second, the unhealthy atmosphere in academia. I’m not going to claim knowledge what exactly is wrong with it, but from an outsider point of view there are two main problems: degree inflation (so that everybody and their dog tries to get a degree even if it’s mostly good for whining about a student debt later let alone the pressure succinctly put in formula “publish or perish” resulting in a large amount of low-quality papers) and abusing ethical and moral principles in the best Soviet traditions (I’m not going to discuss if there’s anything wrong with those principles, I would simply like to point out that there are people abusing those principles either to get rid of successful competition or merely secure a place for themselves so you can’t kick them out as it would be treated as an offence even if there’s no merit to keep them at that position in the first place). And in both those aspects you can’t compete with China which has larger population and thus can engage more students producing ideologically correct papers.

Leading the most powerful military alliance. The recent events demonstrated the lack of leadership there. In the first days of 24 February there was a joke that if russia would invade Poland, Poland would be kicked out from NATO. Now it seems more like Poland would be kept in NATO but it would be forbidden for it to retaliate—after all russian invasion would pose no threat to the alliance and retaliating means escalation of the conflict. The current war (essentially resulting from the Budapest Memorandum that USA pressured Ukraine to sign—with no guarantees in return) demonstrated that USA is unreliable partner. Even worse, recent events show that USA has OSCE-grade of blindness. I need to elaborate this a bit.

There was a joke in Ukrainian emigrant magazine of 1948—”USA will send $100 million relief funds to European countries, one half immediately, for the second half they’ll have to wait too.” This applies to the way USA provided its military help as well. And it not only delayed its own help, it sabotaged attempts of other countries by e.g. not giving a permission to train Ukrainian pilots on F-16 planes in other countries. And of course that help was so limited that it reminds me somebody taking an antibiotic to combat the disease in such half-hearted way that it mostly helps breeding antibiotic-resistant bacteria than actually curing a patient. If you don’t see a similarity—thanks to the weapons being limited in at least two senses (range and amount) russia has learned how to deal with them (same with economic sanctions). And as a cherry on top, you should remember USian officials asking Ukraine not to hit russian refineries—because they value oil prices as a more important thing than safety. And unless you’re a USian official you should know that at least one common feature among China, Iran, North Korea and russia is that they all regard USA as their number one enemy and would gladly hit it if the occasion presents itself. Another fun fact: while Ukraine managed to damage or destroy several russian strategic objects used in their nuclear defence without any retaliation (despite russian doctrine claiming that such actions would cause an immediate nuclear strike on the offender), USA is still afraid of that paper bear.

Speaking of which, you might’ve wondered what “OSCE-grade blindness” expression means. You can read more about the organisation in Wikipedia but suffice to say that after the russian invasion in Ukraine in 2014 it proved itself to be worse than useless—it completely ignored russian war crimes and, as Ukrainian soldiers wrote, after OSCE inspector visits they had to relocate the forces because russian shells would start hitting the place they visited. And now we have USA reacting to the reality of one terrorist state sending military help to the other terrorist state:

  1. Those are just the rumours.
  2. Okay, maybe not the rumours but still there’s no reason to do anything yet.
  3. We’ll consider our actions when it comes to the direct engagement.
  4. Ukraine has the right to defend itself against Koreans (why, thank you very much!).
  5. We’ll go as far as not forbidding Ukraine to use our weapons against them.

I believe this post was not far from the truth (translated for those who don’t understand Ukrainian):

Bloomberg writes that potential NATO answers to the DPRK actions are limited, yet they may include:

  • concern
  • deep concern
  • playing dumb
  • making a busy appearance
  • shaking head in disbelief
  • saying “tsk-tsk” with a sad expression
  • giving russians some Patriot systems to protect their oil refineries

And while various experts and “experts” say it’s a desperate step for russia, they forget to mention that desperate buyer pays any price—and Iran and DPRK both would gladly wrestle some technologies from it so they can nuke USA (or at least Israel) later. Of course USA sees no threat in that.

As the result, other NATO members (as well as other countries currently depending on USian help) see this inaction and finally prepare to be able to defend themselves even without USA. The claims of one candidate about leaving NATO (as well as hindering the trade with other countries) if he wins—which he may or may not follow—do not help with trusting USA either.

Overall, it seems like the advantages USA had were so successful that they went out of control and started to hurt themselves. To give an example, in ancient times stock prices were determined a lot by dividends they paid, nowadays dividends are treated as something archaic and it’s presumed that stock owners should get wealth from the increase of stock prices—which are determined by a metric “how much a lot of suckers would agree to pay” (and not some silly things as assets and profits). Similarly the companies are run by professional managers that care mostly about their bonuses—which usually means implementing short-term measures like sacking fifth of staff right before the financial report, so savings can be shown as profits, growth (and bonus for the successful manager). And if somebody is not like this—where to find enough people with a similar mindset to run a large corporation? Hubris and monopoly may make you think that it’ll last forever—but then the conditions change and your position is not so sure anymore (you might’ve heard about gas guzzlers and Detroit being the industrial heart of USA—before oil prices soared and USians started to buy Japanese cars instead).

There were many attempts to undermine Pax Americana but this time it seems USA is helping in doing that itself.

What’s next?

Nobody knows. IMO the most realistic scenario is that USA will wither while pretending that everything is nice and Enron-y—and it may work for them until the reality hits.

There are countries that attempted to concentrate on ideological issues instead of more practical ones, usually building communism (socialism, or Islam revolution). This ended badly for the population of those countries even if they must feel happy living in such moral place (or face consequences of wanting more bread instead of high moral ground).

There are two countries known for going into isolation and having extremely limited trade with the outside world—China and Japan. In case of Japan most countries politely avoided interacting with it until some rude USians came uninvited on their black warships and said “let’s trade!” In case of China it ended with a period they proudly call “century of humiliation”. You know, when the country that was proud to have most population and the most advanced industry (for some time) lost couple of wars to small groups of foreign barbarians, having to open trade, paying tribute and ceding territories. I guess that’s the future some want for USA.

If somebody thinks that voting for a right candidate will solve everything, I advise you to watch South Park episode about choosing a school mascot. I’d compare the current alternatives as Boeing and The-Company-Formerly-Known-as-Twitter: in one case you have a respectable outlook that hides degradation of the things—until the planes start to crash or self-disassemble in the air; in another case you have a company managed by a chaotic personality that manages to lose half of the users and advertisers while severing ties with the other half itself. IMO the main problem comes from most people being functionally idiots i.e. having a working brain but refusing to use it (thinking is hard after all—and there are too many helpful people around offering to think for you).

I don’t have much hope in other countries either—many things I’ve written above apply to the EU as well (but I’d argue it’s more about bureaucracy and having too many parties to come to unanimous decisions). Even China while being effectively a dictatorship (and thus lacking several of the problems) seems to experience problems (because dictators don’t always make good decisions and there’s nobody to stop them) and it is not clear if it will survive the downfall of USA (though it would welcome such event regardless).

Welcome to the shitty world, finding new ways to get worse since 2020. Enjoy it while there’s still something left to enjoy.

Woes of implementing MPEG-4 ASP decoder

Friday, October 11th, 2024

So, for a last month or even more (it feels like an eternity anyway) I was mostly trying to force myself to write a MPEG-4 ASP decoder. Why? Because I still have some content around that I’d like to play with my own player. Of course I have not implemented a lot of the features (nor am I going to do that) but even what I had to deal with made me write this rant.
(more…)

Looking at Winamp codebase

Friday, October 4th, 2024

Breaking news from the Slowpoke News Channel™: a source code base for Winamp has been released (just last month). So it’s a good occasion to talk about it and what interesting (for me) things can be found in the third-party libraries.

I think I used the software a bit back in the day when MP3 was still rage (and there were CDs sold proudly featuring MP3 at 256kbps) and I still was using Windows (so around 1998-1999). I even briefly tried K-Jofol player (does anybody remember that?) but I didn’t see much point in that. About at that time I finally switched to Linux as my main OS and started using XMMS and XMMS2 (I actually met one of its developers at FOSDEM once—and saw a llama or two when I visited a zoo but that’s beside the point). Also there was a plugin for XMMS2 that added VQF support (again, nowadays hardly anybody remembers the format but it was an interesting alternative; luckily Vitor Sessak reverse engineered it eventually). But with the time I switched to MPlayer for playing music and nowadays I use my own player with my own decoders for the formats I care about (including MP3).

But I wanted to talk about the code, not about how little I care about the program.

First fun thing is that the source code release looks like somebody was lazy and thinking something similar to “let’s just drop what we have around and tell not to do much with it—it’ll create some hype for us”.

Second fun thing is that it fails to live up to the name. As it should be obvious, the name comes from AMP—one of the earliest practical MP3 decoders (the alternatives I can remember from those times were dist10 or the rather slow Fraunh*fer decoder). And of course WinAMP uses mpg123 for decoding instead (I vaguely remember that they switched the decoding engine to the disappointment of some users but I had no reason to care even back then).

But the main thing is that they’ve managed to do what Baidu failed to do—they’ve made VP5 decoder and VP6 codec open-source. Of course it may be removed later but for now the repository contains the library with the traditional Truemotion structure that has VP5 decoder as well as VP6 decoder and VP6 encoder. So those who wanted an open-source VP6 encoder—grab it while it’s still there (and I still have my own implementations for either of those things).

Out of curiosity I looked at the encoder and I was not impressed. It has more features (like two-pass encoding) and more refined rate control but it does not look that much better. I wonder what Peter Ross could say about it, being a developer of a popular and well-respected encoder for a codec with rather similar structure.

Overall, the code base looks like a mess with no clear structure, with most libraries shoved into one directory without any further attempt to separate them by purpose. But it does not matter as it was not intended for the large collaborative efforts and two or three programmers could live with it.

Still, let’s see if something good comes from this source release.

Update from October 17th: looks like the original owners had enough fun and removed the repository. So those who’re still willing to study the code need to locate one of countless copies of it first.

On over- and under-engineered codecs

Tuesday, September 10th, 2024

Since my last post got many comments (more than one counts as many here) about various codecs, I feel I need to clarify my views on what I see as over-engineered codec as well as under-engineered codec.

First of all, let’s define what does not make a codec an over-engineered one. The sheer number of features alone does not qualify: the codec may need those to fulfill its role—e.g. Bink Video had many different coding modes but this was necessary for coding mixed content (e.g. sharp text and smooth 3D models in the same picture); and the features may have been accumulating over time—just look at those H.26x codecs that went a long way, adding features at each revision to improve compression in certain use cases. Similarly it’s not the codec complexity per se either: simple methods can’t always give you the compression you may need. So what is it then?

Engineering in essence is a craft of solving a certain class of problems using practical approaches. Yes, it’s a bit vague but it shows the attitude, like in the famous joke about three professionals in a burning hotel: an engineer sees a fire extinguisher and uses it to put out fire with the minimum effort, a physicist sees a fire extinguisher, isolates a burning patch with it and studies a process of burning for a while, a mathematician sees a fire extinguisher, says that there’s a solution and goes to sleep.

Anyway, I can define an over- or under-engineered codec by its design effectiveness i.e. the amount of features and complexity introduced in relation to the achieved result as well as the target goal. Of course there’s rarely a perfect codec so I’ll use a simpler metric: a codec with several useless features (i.e. those that can be thrown out without hurting compression ratio or speed) will be called over-engineered and a codec which can be drastically improved without changing its overall design will be called under-engineered. For example, an RLE scheme that allows run/copy length of zero can be somewhat improved but it’s fine per se (and the decoder for it may be a bit faster this way); an RLE scheme that uses zero value as an escape with real operation length in the following byte is under-engineered—now you can code longer runs but if you add a constant to it you can code even longer runs and not waste half of the length on coding what can be coded without an escape value already; and an RLE scheme that allows coding the same run or copy in three different ways is over-engineered.

And that’s exactly why XCF is the most over-engineered format I’ve even seen. Among other things it has three ways to encode source offset with two bytes: signed X/Y offsets, signed 16-bit offset from the current position or an unsigned 16-bit offset from the start. And the videos come in either 320×200 or 320×240 size, so unless you have some weird wrap-around video you don’t need all those addressing modes (and actually no video I’ve tried had some of those modes used). Also since the data is not compressed further you can’t claim it improves compression. Speaking of which, I suspect that wasting additional bits on coding all those modes for every block in every frame negates any potential compression gains from specific modes. There are other decision of dubious usefulness there: implicit MV offsets (so short MVs are now in range -4,-4..11,11 for 8×8 blocks and -6,-6..9,9 for 4×4 sub-blocks), randomly chosen data sources for each mode, dedicated mode 37 is absolutely the same as mode 24 (fill plus masked update) and so on.

Of course there are more over-engineered codecs out there, I pointed at Indeo 4 as a good candidate in the comments and probably lots of lossless audio codecs qualify too. But my intent was to show what is really an over-engineered codec and why I consider XCF to be the worst offender among game video codecs.

As for under-engineered codecs, for the reasons stated above it’s not merely a simple codec, it’s a codec where a passerby can point out on a thing that can be improved without changing the rest of the codec. IMO the most fitting example is Sonic—an experimental lossy/lossless audio codec based on Bonk. Back in the day when we at libav discussed removing it, I actually tried evaluating it and ended with encoded files larger than the original. And I have strong suspicion that simply reverting coding method to the original Bonk or selecting some other sane method for residue coding would improve it greatly—there’s a reason why everybody uses Rice codes instead of Elias Gamma’. Another example would be MP3—there’s a rumour that FhG wanted it to be purely DCT-based (as AAC) but for patent holder’s consideration it had to keep QMF, making the resulting codec more complex but less effective.

P.S. The same principles are applicable to virtually everything, from e.g. multimedia containers and to the real world devices like cars or computers, but I’ll leave exploring those to the others.

Self-fulfilling excuses

Saturday, August 31st, 2024

There’s this old but annoying (and sometimes infuriating) thing called self-fulfilling excuses. Essentially it’s when somebody uses a deferred reasoning “this thing will not work, because I’ll be stonewalling it until it fails, and then I can proudly claim that I was right”. Here I’ll give three examples of it.

First, there’s the tired “masks won’t work against spreading COVID-19”. From the facts I know even WHO finally admitted that the virus is airborne, so using a mask to protect yourself against it by filtering it out of the air you breathe sounds like a reasonable precaution (especially since virus size and masks filtering capabilities are known). And yet indeed masks were not an effective measure because of so many opinionated idiots who refused to not wear them. Just this year alone two random people shouted at me because they saw me wearing a mask (I don’t know who loves wearing them but I keep doing that in public confined places like shops or public transport because my health is not good enough as it is already).

Second, there’s this recent drama about Rust in Linux kernel. I can’t tell whether Rust really belongs there or if it was a mistake admitting it there in the first place—let the technical points be discussed by the experts—but the recent resignation of one of the maintainers demonstrated that the main reason why Rust code can’t work in Linux kernel because certain maintainers are against it. And you know what, if the rest of Rust kernel writers would leave and create their own kernel it may be for the best—I always supported having an alternative (this would be a good place to insert an advertisement for librempeg but Paul hasn’t provided one).

Third, there’s the situation with American help for Ukraine. Just yesterday russian plane sent bombs to my home city, killing civilians and destroying living houses. The most reasonable action would be to allow Ukraine use the provided missiles to strike those planes right at their airbases, eliminating or reducing the threat both directly and indirectly (as russians would have to operate the remaining planes from farther airbases, making their further strikes less effective). Instead USA forbids using its missiles for that purpose. Among the reasons quoted were that there are too few missiles USA can give anyway (right, nobody knows how to allocate scarce resources) and that there’s no strategic advantage to that (link provided as a proof that I’m not the one making it up). And considering that russians are taking such threat into account and have started building plane shelters on the airbases closer to Ukrainian borders, that will turn out true.

Apropos, this is an appropriate place to spotlight this saying:

This will probably keep being true for a long time.

If you’re looking for any solutions, you won’t find them here. This is an innate thing to many humans so I don’t believe it can be eliminated without cardinal changes to human nature (and we don’t know what and how to change), and while theoretically the consequences can be mitigated by the reasonable discussions and rational decisions, those are not common things either (and what I said above the human nature applies here as well). In other words, this world is imperfect and sucks a lot. I have more trite things to say but that’s enough for now.

On some German town names

Monday, August 19th, 2024

One of my hobbies is travelling around and during my travels I see names that I find amusing like Geilhausen which can be translated as “Gay/kewl housing” or Waldfischbach (which translates to “forest fish brook” and it makes me think about what special kinds of fishes live in the forest). But today I want to talk about more IT-related town names.

Travelling from here to Switzerland first you encounter Vimbuch (“das Buch” is “book” in German and the town is named after St. Find. Really!). The only problem is that they had to wait over seven centuries in order for people to understand that it’s not named after an editor manual.

In twenty kilometres from it you can find Urloffen (“offen” is “open” in German). One can only wonder what its inhabitants did for over eight centuries before hyperlinks were invented.

Another thirty kilometres to the south you can find Rust.

And if you continue following the road you may end in Switzerland and see Speicherstraße in canton St. Gallen (“der Speicher” is “storage” e.g. HDD in German; the road is not impressive at all even compared to the old PATA buses). But let’s not go there, it’s a neutral place. After all, there’s Speicher near Bitburg and you know it’s a very safe storage if they build a castle for a single bit.

But speaking of the programming languages, there’s a place called Perl in the remotest rural place of Saarland (a federal land often regarded to be exactly the remotest rural piece of Germany in the eyes of people not living there). The fun thing is not only the fact that it borders Schengen Area (yes, that town is right across the river) but also that the rail line continues to Apache in France (since French famously suck at spelling, they forgot a letter in the web server’s name).

And that’s all names I can remember immediately not counting the mountain range Eifel but who remembers a programming language named after it anyway?