Staying neutral

March 31st, 2023

I hate these filthy Neutrals, Kif. With enemies, you know where they stand but with Neutrals, who knows? It sickens me.

Back in the day this quote from Futurama could be seen only at its face value: something only a paranoid jingoistic moron would say. Nowadays we see how the organisations and countries with neutral status use it mostly as an excuse to avoid responsibility for their acts.

Let’s start with various international organisations. Various international sports federations and the IOC say they’re outside politics and welcome money participants from all countries. All while forgetting that accepting participants from the countries that violate their principles of fair sportsmanship is going against those principles (“to oppose any political or commercial abuse of sport” from the Olympic Charter sounds especially ironic). And we have the ICRC that is in theory should act as a neutral mediator between fighting parties. In practice it has destroyed its reputation by not willing to organise anything for Ukraine (I can understand that they could not organise green corridors for Ukrainians because russians were against that but not performing their duties by visiting Ukrainian POWs in russia and essentially participating in war crimes by supporting russian deportation centres—that I cannot neither understand nor forgive). I suspect than in both cases corruption is at play: a dictatorship can always offer money or “gifts” that law-abiding democratic countries can’t.

And of course there’s Useless Nations organisation (still officially called United Nations by mistake). Considering how russia ignores its General Assembly decisions without any repercussions one should wonder about its usefulness already. And then they (as well as certain other international organisations “for all good against everything bad”) release reports blaming both parties: russia for aggression and Ukraine for defending itself. Including the latest scandal with russian “orthodox” church (FSB patriarchy)—it does not matter that in every church and monastery in Ukraine they occupy (in both senses) there were found stashes of russian propaganda, weapons and often russian agents in hiding; it does not matter how the church officials and lower clergy still repeat russian propaganda theses and act against Ukraine—the church is obviously a neutral organisation and should be treated as such.

On the same note, even individuals who repeat the same things about how both parties are wrong and you should stay neutral invoke only disgust in me. In the best case they’re idiots who listen to russian propaganda, in the worst case they’re narcissists who believe only they can be right and the rest of the world is stupid and wrong (place the current Pope into a category you see fit).

Now to the countries with neutral or “neutral” status. Often such country stays neutral because it is too small and cannot afford to anger its neighbours. The classic example of that would be Switzerland (just ignore its army regularly invading Liechtenstein during trainings), Sweden or Israel. Finland remained neutral for a sadder reason: it found out how useless were those alliances for defending it against Soviet aggression (similarly there was a joke in the beginning on 2022 that if russia would invade Poland or Estonia then NATO would react by excluding that country from the alliance). And we also see “neutral” Hungary and China. Let’s take a closer look at all those countries.

Switzerland started out as a small poor country that had fears of being invaded by its stronger neighbours—which happened only once, in Napoleon times (and because of their connections Swiss ended up with more territory when the war was over; those who try to mention Austrian invasion should look first who those Habsburgs were). Nevertheless they decided to remain heavily armed neutrals so that they don’t get involved in wars themselves and when an invader comes a good deal of the population will fight those armies in the mountains (having mountains is a good bonus to defence indeed). That’s why they have the law against transferring their weapons and munitions to the countries in the state of war (I can respect that. Update: after reading this article I’m inclined to re-evaluate their neutrality as to “do whatever we want as long as it does not anger powerful countries much”). In the same time they used their neutral status to conduct unscrupulous trade with all countries of the world (especially during WWII) and to get a reputation of a banking safe haven. But thanks to the pressure from the USA and various data leaks from its own banks it is no longer so. Hopefully in the future the country will survive and prosper on its high-tech chocolate industry and other respectable businesses instead of keeping stolen goods and money from dictators all around the world (just see this list to understand why one of their major banks went belly-up and may take another one with it).

And now to Israel. This is somehow an inverted Switzerland—a unitary state created more or less as a safe haven for a certain nation, it has constant tensions with its neighbours but remains neutral because it does not want its citizens (and potential ones) living in other countries to be harmed (a noble goal really). So in the beginning of the war they remained neutral because they did not want russia to get vengeful and supply a neighbouring terrorist state with something that will harm Israel. russia responded to this with various anti-Semitic statements and actions (read about the Sokhnut court case) and went to arm Iran anyway. Also it helps that the current government is very pro-russian and acts in rather russian ways (see their recent laws and the heavily-opposed legal reform; just another case why you should not have politicians serve indefinitely long terms). Here the neutral status is used to evade certain actions and also to evade responsibility for the other actions. Though the way it goes either the current government will be kicked out or they’ll lose international support.

Let’s move to really neutral countries like Sweden. In XX century it let Norway free after it decided to dissolve the union and the only war-like act during WWI was occupation of Åland that was more of a territorial dispute that was settled more or less peacefully (between Sweden and Finland, the other parties involved were not so nice). During WWII Sweden had to yield to pressure from Germany (like letting its troops pass through Northern Sweden—which those troops regretted doing) but in general it remained neutral because of its usefulness (like the iron ore and products from it) that would be lost in case of occupation. The current war made Sweden reconsider their position and it has finally applied for NATO membership.

And now for another NATO applicant, Finland. It has been suffering from the same problem as Ukraine—bordering with russia. In the first half of XX century it had been constantly struggling to keep its territory and sovereignty from Soviet Union (and the League of Nations proved out to be as useful as UN now, only Sweden and Norway helped it at that time plus some political manoeuvring). That’s why after 1945 it remained neutral but the recent events proved that if you border with russia and you’re not large enough then russia will try to occupy you. So Finland reconsidered its status as well.

Temporarily Occupied West Taiwan (or China for short) is a country that declares itself to be neutral while it semi-openly supports russia (and does not do that in the open because of the fear of sanctions). To which russia repays with its usual ingratitude: just three days after Xi Jinping visited it and had his roadmap for peace accepted there there’s a claim that they’ll go against it by threatening the world with their nukes and nuclear proliferation. This is a poster case of neutrality used as an excuse to do whatever they want since they’re not restricted by some political alliance.

And finally there’s a country that conveniently forgets about its treaties and alliances and proclaims itself neutral to do business with sanctioned countries. Of course I’m talking about Hungary. It’s the country that constantly blocks EU and NATO decisions either because it was paid for that (that’s why the amount of EU sanctions against russian oligarchs and entities is much lower than it should be) or because they want to have a bargaining chip (EU sanctions and other unanimous decisions again and blocking NATO membership for Finland, Sweden and Ukraine are good examples of that). And when I said they forget about previous treaties at their convenience I meant their claims about increasing trade with Iran (despite EU sanctions) or not honouring the indictments from the International Criminal Court because the (signed and ratified) Rome Statute “is not integrated into our legal system”. I believe somebody should help them become truly neutral by kicking them out from both EU and NATO. In the recent news though it’s reported that russia has added them to their list of unfriendly countries. Looks like there’s a common pattern in dealing with russia.

I probably should’ve mentioned Austria here as it was made a neutral country after 1945 but considering how it turned into serving russian interests I don’t see it showing any new aspects of “neutrality” and thus contributing to the discussion.

In conclusion I want to say that there are three different kinds of neutrality that different countries (organisations, people) often mix and pass for another kind: “don’t touch me” neutrality, “I can do whatever I want” neutrality and “you can’t make me do it” neutrality. The first kind is essentially a country staying aside and remaining not involved in any wars so that other countries do not have a reason to invade it either; sadly this works well only for countries with good defence (and not for Ukraine). The second kind is when you claim to be neutral in order not to have any restrictions on dealing with all other possible parties (it works only so well as you can withstand sanctions and pressure i.e. China has better chances doing that than Switzerland). The third kind is complementary to the second kind and the same reasoning applies.

So next time you hear claims about somebody or something staying neutral and not wanting to leave anybody or anything behind, ask yourself about the real goal of that neutrality (hint: there are dirty money involved quite often in this case).

The disturbing similarities between France and russia

March 11th, 2023

Dedicated to yet another piece of news about French company deciding to increase its business presence in russia instead of leaving the “market”.

In general I enjoy learning a bit about various foreign languages but I never had a desire to learn French language and when asked why I always answered that I know russian language already. But with the time I saw many more common things between France and russia than their language policy. This year of 1939 (will it end already?) brought even more examples of russia behaving like France and the French Empire(s) of old. Which makes me wonder if it is a certain type of country and what makes it such.

To me it looks the root of all problems is that both countries grew too large in territory, incorporating different nations in the process and relying too much on the central power (come to think about it, Temporarily Occupied West Taiwan comes to mind as well). The only way they could keep it going was to create “single nation”, by means of the cultural or physical genocide if required, and stealing the history while at it.

Below I’ll try to group things that are similar in the both countries by category.

History.

Both countries originated as the outskirts of some larger country that were overrun by the invaders. As the result they ended up as something semi-fictional: why do you think territory formerly known as Gallia (and proud of Galls) is called after some Bavarians (and IIRC it even had a war with them over the name, so one ended as France and another one as Franconia)? Similarly why a country in a place originally inhabited by Finno-Ugric people, where only the nobility was of Slavic origin (with their blood heavily diluted by Mongols) would boast their completely Slavic lineage (forgetting about all those Finno-Ugric, Turkic and Caucasian nations living on the majority of its territory)? And of course both countries claimed the title of the One True Heir of Roman Empire. Let’s skip most of the Medieval history, remarking only such details as St. Bartholomew’s Day massacre and the genocide of Novgorod Republic, and move to the more modern days. The French Revolution (the first and most famous one)—and the revolutionary terror of course—set the standard of how to do such things (which russia copied later). It’s worth noting that for the majority of XXth century France and russia remained friends: first the Entente, after WWII France willingly returned to the Soviet Union people who fled russia even before USSR was created (or it was russian territory at all), then (after France decided to leave NATO) Soviet-French friendship prospered (to the point possible by the foundational principles of Soviet Union), even now the French president tries hard to be russian advocate (the same can be said about Germany but despite some common things the countries were not similar throughout the majority of the history).

Country structure.

I notice that both countries occupy a large territory but despite that a significant part of the population lives in the metropolitan area of the capital: about one fifth for France, and one seventh for russia (for comparison, for Germany, Ukraine or Kazakhstan those numbers are below one tenth). Additionally the capitals in both countries are considered to be the only place to live (in case of russia their former capital is regarded in the same way too) and locals (whose ancestors probably lived far away from that capital) sneeze down at all those newcomers who don’t live in the capital and don’t speak the capital dialect either. Then there’s the whole question of the empire…

The main difference here is that russia was never good at maintaining overseas colonies (Fort Ross and Alaska were sold and the pirate nest of Septinsular Republic could not be hold) so it resorted to occupying and controlling neighbouring countries. For example, during Soviet times Mongolia was controlled by the Irkutsk division of The Party. Maybe that’s why they always wanted more oceanic ports.

France, on the other hand, had its colonies all around the world and keeps some of them under the name of overseas departments. For more exquisite things there’s Andorra. And of course there are “military advisors” in various African countries (former French colonies by coincidence). Now that France does not want or can’t maintain its presence, those are withdrawn and are usually replaced by the troops of the infamous russian military company (hopefully it’ll find its end at Bakhmut soon).

And finally, if you think that russia is the only country that tried to create puppet republics in Europe in order to annex them later you’d be wrong. When in 1945 several countries agreed to occupy parts of Germany in order to make it a decent civilised country, USSR almost immediately made its zone of responsibility into a socialistic puppet state that had been the best friends to its end and even after that (see Angela Merkel). France, meanwhile, tried to pull a similar trick with Saarland. They tried to convert Saar Protectorate into a puppet People’s Republic of Saarland but (since they could not falsify elections) the result was unfavourable for them and in 1957 Saarland joined Germany for good. Considering how Saarland plays about the same role as Donetsk and Luhansk regions in Ukraine it’s hard not to see parallels to russia’s actions in 2014 and 2018.

Language and cultural policies.

Here I’m not going to talk about the influence of the for-export version of the country culture (writers, composers, ballet and so on) on other nations but rather what they did inside their own territory.

I observe the same approach: cultural genocide and forcing the One True Language. russia did that to many of its colonised nations, see Ems decree as one of such example (I’ve been there recently BTW and spat and the bust of Alexander II installed in the local park). Nobody talks about France in this aspect but it did rather well to eliminate Occitan and Basque languages from its territory (and if you think what Basques have to do with it—learn about the history of Gascony fist). Similarly it forced French as the first language in the schools—unlike, say, British Empire where people could learn their native language first and English later. As the result many people from “former” French colonies speak perfect French while people from British Commonwealth have all kinds of accents and dialects.

Of course (in both countries) you have language chauvinism extended past beyond that when they refuse to learn and speak the language of any other country (one philologist who emigrated from USSR once said that the purpose of studying foreign languages there was mostly to incite hate to that language). And now both countries have laws for protecting their language purity against foreign influences too.

Another aspect of such language chauvinism is demanding to support their language on the state level (you might’ve heard about the situation in Canada where exists essentially a language police from a certain province that checks that all inscriptions in this predominantly English-speaking country have translations in French next to it in the same size; and of course even the most liberal russians demand the same from any country they take residence in).

And the last thing about the language is its writing complexity. By that I mean the fact that there’s not a very strong connection between the way the words are written and spoken (to the point where sometimes you need to know the context in order to pronounce the word properly). And reforms are often met with hard resistance (see France, 2016 or russia, 1918) probably because they cherish the elite status of the language and do not want to make it more available to the common folk. There are other languages with similar characteristics, but in case of English it was essentially arbitrarily picked from various dialects (of several languages) for printing convenience (i.e. it was not something forced by a government) and people curse William Caxton to this day; in case of Chinese it’s used to hide the fact that different parts of the country speak different languages so if they don’t write in the same way they would be completely disconnected (which goes against the imperial narrative of a single Han nation); in case of Japanese it’s mostly a consequence of adapting Chinese writing system for their language that is built on very different principles (it has a completely different grammar and mostly different lexis even if it has a good share of Chinese loanwords); and Hebrew as a Semitic language has little regard for vowels so they’re omitted where possible and usually indicated in full only in the texts for children and the religious texts. In either case it’s not done just to make scribes feel more important.

I could also mention how they all like to overlook the fact that a good deal of their famous people and inventions have a different origin (starting with that guy from freshly conquered Italian territory by the name of Napoleone Buonaparte or the famous russian explorer from Danemark Vitus Bering) but IMO that’s pretty normal when you’re building a unified empire based on a single culture forced onto everybody as the mean of unification. A certain country in the East comes to mind as well with its Uyghur re-education camps.

And when I talk about cultural genocide I mean not only forcing the master language and culture onto the colonised nations but also destroying (or at least hiding) the remnants of native civilisation so that the nations won’t get wrong ideas. In Soviet Union the representatives of state security always accompanied archaeological expeditions and denied publishing the findings if they were contradicting the state narratives (for example, the real age of Trypillia or Carpathian settlements as they undermined the narrative about russians being the oldest nation). Similarly I read a story over two decades ago that in French Indochina the explorers discovered an impressive city or temple complex (was it Angkor Wat?) that was kept in obscurity so it would not inspire any national liberation movements and it was effectively rediscovered in 1960s or so.

Cruelty.

Here’s a quote from Mark Twain:

Yet even France rose at last — and would have retired to its warren again quite contented with a cuff and a bonbon if the foolish King had offered them, but it was not his style to do the needful thing at the needful time, so the chance went by. Then the nation cast its rabbit skin and put on its other national garment, the tiger skin; being closely pressed by Europe in arms, it went a step further and asserted its manhood, and was doubtless surprised to find how much it had of it. Napoleon, the great foreigner, brought the people’s soldiership up to the last summit of perfection; and when he got ready he dressed the nation in their rabbit skins again and put his foot on their necks, and they glorified him for it. Napoleon III accommodated them in the same way, to their vast satisfaction.

The same largely applies to russians as well: they never had real uprisings against the sovereign, only protests addressed to him (the only real uprisings were started by other nations, like Ukrainians or Bashkorts) but when they were allowed to display their repressed feelings we got East Prussia in 1945 or Yahidne, Mariupol’ or Izyum this year. And if you look at russians they’re meekly going to war if they can’t avoid it and their relatives blame Ukraine for their subsequent deaths (instead of the führer who sent them there to die).

And if you look at what historical figures russians worship and despise, they mostly worship tyrants who oppressed them and wasted them in countless wars and despise rulers who tried to have more peaceful and liberal politics (because it was usually a sign of them being weak). I don’t know much about the French but considering the Penguin Island novel I expect about the same. At least nowadays they have strikes to vent off their negative feelings…

Conclusions

As I mentioned in the beginning, I suspect there’s a reason why both countries have acted similarly throughout the ages. Maybe it was the absolute power of the monarch and the abundance of initial territories, maybe it was something else. For example, British Empire grew up a lot thanks to the private initiative (names like Cecil Rhodes or East India Company come to mind immediately); USA (with such “unincorporated territories” as Puerto Rico or American Samoa) seem to retain its satellites a lot thanks to the fact that it’s better to be part of USA and get involved in its economic processes—and for the rest there’s U.S. Navy (which reminds me a lot of Roman Empire). With russia it was the opposite—the conquered countries tried hard to get independent from it (all of the Baltic states immediately and Ukraine, even if it took it more years after 1991 to realize why). French colonies are known to have been treated some of the worst (not as bad as Belgian king’s and later simply Belgian Congo though)—Haiti is the poorest country of Americas and I urge you to compare how many of the world poorest countries are former French colonies.

So it is no wonder that the countries with similar mindset would keep doing business together as long as the public opinion permits. At least France seems to improve albeit slowly while I see only two realistic ways for russia—balkanisation or turning into another North Korea. In either case maybe the future historians will explain what was wrong with both countries and why they went different ways eventually.

NihAV: towards RealMedia encoding support

March 10th, 2023

Since I have nothing better to do with my life, I’ve decided to add a rudimentary support for encoding into RealMedia. I’ve written a somewhat working RMVB muxer (it lacks logical stream support that is used to describe the streaming capabilities, it also has some quirks in audio and video support but it seems to produce something that other players understand) and now I’ve made a Cook audio encoder as well.

Somebody who knows me knows that I fail spectacularly at writing non-trivial lossy audio encoders but luckily here we have a format which even I can’t botch much. It is based on parametric bit-allocation derived from band energies. So all it takes is to perform slightly modified MDCT, calculate band energies, convert them to scale factors, perform bit allocation based on them, pack band contents depending on band categories and adjust the categories until all bands fit the frame. All of these steps are well-defined (including the order in which bands should be adjusted) so making it all work is rather trivial. But what about determining the frame size and coupling mode?

As it turns out, RealMedia supports only certain codec profiles (or “flavors” in its terminology), so Cook has about 32 different flavours defined for different combinations of sample rates, number of channels and bitrates. And each flavour has an internally bitrate parameters (frame size and which coupling parameters to use) for each channel pair so you just pick a fitting profile and go on with it. In theory it’s possible to add a custom profile but it’s not worth the hassle IMO.

And now here are some fun facts about it. Apparently there are three internal revisions of the codec: the original version for RealMedia G2, version 2 for RealMedia 8 and multichannel encoder (introduced in RealMedia 10, when they’ve switched to AAC already). Version 2 is the one supporting joint-stereo coding. The codec is based on G.722.1 (the predecessor of CELT even if Xiph folks keep denying that) but, because Cook frames are 256-1024 samples instead of fixed 320-sample frames, they’ve introduced gains to adjust better for volume changes inside the frame (but I haven’t bothered implementing that part). That is probably the biggest principal change in the format (not counting the different frame sizes and joint stereo support in the version 2). And finally I should probably mention that there are some flavours with the same bitrate that differ by the frequency range and where the joint stereo part starts (some of those are called “high response music” whatever that means).

Time to move to the video encoder…

FFhistory: the most annoying format

March 3rd, 2023

Looks like the series was misunderstood by the public, especially by those who did not read the prologue and were disappointed by the conclusion. Oh well, I can still post random bits of FFhistory with some inconvenient truths even if nobody is going to read them.

There are many codecs and container formats that are annoying to support: “industrial” formats like MXF have their own Internet of documentation (i.e. lots of various documents referring to other documents, most of them are paywalled as well), other formats suffer from being too flexible or too bloated that it’s next to impossible to implement support for all possible features (e.g. JPEG-2000 or H.264 scalable and multi-view extensions). There are formats that are abused to death (MPEG-TS and MP3 come to mind), there are formats that are annoying to reverse engineer and there’s too little interest going there (you would not believe how much time it took to support Windows Media 3, from basic decoder to the interlaced mode support, something tells me we won’t see completed Bink 2 decoder any time soon either), there are formats that require writing an emulator for some system (like CGDI codec that recorded GDI commands), but there’s yet another candidate that I consider the most annoying one in the whole FFhistory since it annoyed the project in many different ways.
Read the rest of this entry »

366th day of February 24

February 24th, 2023

So it’s been a calendar year since February 24 started along with year 1939 (the dates since February 2020 got really messed up). Let’s see how it went.

russia decided to mimic Nazi Germany and started the war “against NATO threat” by using its own Gleiwitz incident (which they first executed a couple days before the full-scale invasion and then back-pedalling—they can’t do anything right). They committed every possible war crime, if they have missed some that’s only because they either overlooked it or didn’t know how to do it. The occupied territories suffered from the genocide (direct and cultural), not occupied territories suffer from their terrorist strikes (mostly on civilian infrastructure). And all of this happens because of some deranged führer in the bunker with a resentment about russia losing the Cold War. And while many world leaders try their hardest to not see the parallels, Hungary was eager to become its ally (it’s the only Axis country country where a 1944 coup changed pro-Nazi government with even more pro-Nazi government after all).

In either case, in the first days of February many believed russian propaganda about its power yet it turned out to be not a colossus with feet of clay but rather a colossus made from shit and sticks on brick legs. Their plan was to bribe various officials, make some precise strikes and during the confusion seize the control of various crucial positions and use them for fast occupation of Ukraine. The first step had some limited success since while certain people are willing take bribes, not all of them are eager to work for it or use those received money for further bribes (it is said that medvedchuk received a billion dollars for that purpose and he used them for his own benefit). The second step also had issues since russian weaponry was not that precise as they hoped. The third step largely failed since Ukrainian authorities were prepared so people acted on their own even without the communications to the centre so e.g. in my home city the head of local state security division acted by russian orders and yet he failed to cause enough confusion and seize control, so the city did not fall to russia forces. Sadly there were enough traitors in Kherson and some parts of Kharkiv region to surrender local towns to russian forces.

Yet the main thing that allowed russians to occupy large portions of Ukraine were their reserves of Soviet weapons and munition plus modernisations of them. Neither of their newer wunderwaffen have proved to be any good (does anybody remember how their deployed some laser weapons for shooting down drones in May? has anybody seen seen their new Armata tanks in action? what about independently verified proofs of their Su-57 usage?). Initially they could use the usual World War II military tactics but then Ukrainian air defence (which they claimed to have fully obliterated in the first days) brought down enough of their planes and Ukrainian soldiers with Javelins and NLAWs destroyed enough of their tanks so they had to resort to the World War I tactics (shelling by artillery and then sending their hordes in) but nowadays with HIMARS “cottoning” their front line munition depots and the overall depleting of the reserves (they can’t produce enough and have to hope that DPRK and China will send more) nowadays it’s mostly sending their poorly-equipped hordes plus terrorist attacks on the civilian infrastructure by various missiles they have (who cares that a good deal of them malfunctions or explodes near the launch site?) and Iranian drones.

And there’s one thing they’ve threatened the world with—nuclear weapons. From what I heard, their de-escalation strategy includes escalation by exploding a small nuke somewhere to demonstrate that they have them and they won’t hesitate to use them. So far nothing like that has happened, I heard several reasons why: USA threatening to obliterate russian forces on Ukrainian territory by conventional means, russian allies (China and India) not willing to accept that either (because everybody will lose then), or even that they tried it but either it was sabotaged on some lower level or their devices simply failed to work. In either case, in the first days of the 24th of February it was easy to believe in their nuclear threats, nowadays nobody takes them seriously. And even them taking nuclear power plants as hostages is now seen mostly just as another war crime they’ll have to pay for later (and somewhat tricky place to liberate).

War is a very horrible thing but it also shows true colours of the people and tests your claims against reality. russia has boasted its army as the second most powerful in the world and its weapons as unparalleled. When they’ve finally decided to continue the 2014 war it turned out that their tanks lose to Ukrainian tractors, none of their advertised weapons are that good (and can’t work without Western chips) and “we’ll take Kyiv in three days and whole Ukraine in a week” turned into “we need to fight for preserving the integrity of russia”. It also turned out that making Hungary a member of anything was a big mistake and that the most of European politicians lack spine. Ukrainians turned out to be the nation that unites against the common threat and demonstrates unexpected strength while russians turned out to be non-thinking thieves and liars deeply infected by imperialistic chauvinism (it’s hard to hide behind the great artists of the past, most of whom are not of russian origin anyway, when all your soldiers do is loot, rape and shitting and their relatives are mostly fighting over compensations for their relatives killed in action).

Meanwhile though they’re seemingly trying to turn it into a religion like building communism before (and with the same rhetoric)—it will be all good sometime in the future when we achieve the goal (but we’re not going to disclose it), so you should stand up to the difficulties created by our self-imposed isolation (the country is doing great by the way and anybody claiming otherwise will be prosecuted as a heretic) and be ready to lay down your life for the country in Afghani… err Syr… err Ukraine.

I hope this day will end soon, and russia will follow the suit. Hopefully then the enslaved minor nations of russia will have a chance to build their own states (now they’re mostly sent to die in Ukraine which russians approve with the notion “why should we go to die when those exist”) and the carbuncle of this world won’t be able to threaten other countries ever again (also maybe China will get a lesson from this but I shan’t bet on it).

A failed attempts on writing Duck TrueMotion S encoder

February 23rd, 2023

So, my attempt to write a semi-decent TrueMotion 1 encoder has failed (mostly because I’m too annoyed to continue it). Here I’ll describe how the codec works and what I’ve implemented.

In essence, Horizons Technology PVEZ is a rather simple delta-compression based codec that codes RGB15 (or ARGB20) and RGB24 using prediction from three neighbours and Tunstall codes (I’m aware only of one other codec, CRI P256, that employs it). For convenience there are three possible fixed sets of deltas and three fixed codebooks as well. Videos from (3DO version IIRC) Star Control II: The Ur-Quan Masters used custom codebooks (data for cutscenes was stored in several files and one of them was the codebook specification), and later TM2X allowed using per-frame custom codebooks and deltas but nobody remembers it. The second revision of the codec (do not confuse it with TrueMotion 2 though) introduced inter frames where some of the 2×4 blocks could be marked as skipped.

Initially I had no idea on how to do it properly so I tried brute forcing it by creating a search tree limited to maximum of 256 nodes at each level but as you can expect it took about a minute to encode two frames in VGA resolution. Thus I decided to look at the codebook closer and eventually found out that it’s a prefix one (i.e. for each chain of codes there’s its non-empty prefix in the codebook as well) so I can use greedy approach by simply accumulating codes in a sequence and writing codebook entry ID when the sequence can’t be extended further (or when adding the next code forms a sequence not in the codebook). Which leaves the question of deltas.

There are two kinds of deltas there, both occurring in pairs: C-deltas that update red and blue components (depending on coding parameters there may be 1-4 deltas per 2×4 block of pixels) and Y-deltas that update all pixel components (and for all pixels as well). The problem here was to keep deltas in order so they produce sane pixel values (i.e. without wraparounds) and that’s where I failed. I used the same approach as the decoders and grouped delta pairs into single value. The problem is that I could not keep the result value from causing overflows even if I tried all possible deltas and did not check C-deltas result (as Y-deltas are added immediately after that). And I also made a mistake of using pixel value with its components stored separately (the deltas apparently exploit carries and subtracting with borrows for higher components). I suppose I’d have better luck if I use 32-bit pixel value (converting it to bytes for checking the differences and such) and if I use individual deltas and probably with a trellis search for 4-8 deltas to make sure the result does not deviate much from the original pixel values…—but I was annoyed enough at this point so I simply gave up. And that is before getting to the stage when I have to figure out how to select delta values set (probably just calculate the deltas for the whole frame and see what set fits there the best), what codebook to pick and how to control bitrate (by zeroing small deltas?).

Oh well, I’m sure I’ll find something else to work at.

P.S. I’ve also tried to look at the reference encoder but CODUCK.DLL was not merely a horrible pun but an obfuscated (you were supposed to pay for the encoder and use serial numbers and such after all) 16-bit code that made Ghidra decompiler commit suicide so I gave up on it as well.

P.P.S. I might return to it one day but it seems unlikely as this codec is not that interesting or useful for me.

Indeo 3 encoder: done

February 16th, 2023

After fixing some bugs I think my encoder requires no further improvement (there are still things left out there to improve but they are not necessary). The only annoying problem is that decoding some videos with the original VfW binary gives artefacts. Looks like this happens because of its peculiar way to generate and then handle codebooks: corrector pairs and quads are stored as single 32-bit word and its top bit is also used to signal if it’s a dyad or a quad. And looks like for some values (I suspect that for large negative ones) the system does not work so great so while e.g. XAnim module does what is expected, here it mistakes a result for another corrector type and decodes the following bytestream in a wrong way. Of course I’m not going to deal with that annoyance and I doubt anybody will care.

Also I’ve pruned one stupid bug from my MS Video 1 encoder so it should be done as well. The third one, Cinepak, is in laughable state (well, it encodes data but it does not do that correctly let alone effectively); hopefully I’ll work on it later.

For now, as I see no interesting formats to support (suggestions are always welcome BTW), I’ll keep writing toy encoders that nobody will use (and hopefully return to improving Cinepak encoder eventually).

Indeo 3, the MP3 of video codecs

February 14th, 2023

I know that MPEG-4 ASP is a better-known candidate for this role, but Indeo 3 is a strong contender too.

Back in the day it was ubiquitous, patented, and as I re-implemented a decoder for it I discovered another fun similarity with MP3: checksums.

Each Indeo 3 frame has a checksum value embedded in it, calculated as XOR of all pairs of pixels. I had an intent to use it to validate the decoder output but after playing a bit I’ve given up. Some files agree on checksums, others disagree while the output from the reference decoder is exactly the same, in yet another files checksums are correct but byte-swapped and one file has only zeroes for checksums. This is exactly like MP3, and like there Indeo 3 decoders ignore that field.

Also I’ve encountered other fun quirks. For example, one Indeo file is 160×120 but its frame header claims it’s 160×240 (but you still have to decode it as 160×120). You’d think it’s the rule but I know some VMD files from Urban Runner game where the first or last frame are double the size. Another file errors out on the first frame because of the inappropriate opcode encountered (essentially “skip to the line 2” right after “skip to the line 2”) but it turns out that VfW decoder does not check that case and simply uses that opcode as a codebook index.

At least my new decoder should be good enough to iron out the obvious bugs from the encoder and after that I shall forget about that codec again for a long time.

Revisiting MSVideo1 encoder

February 8th, 2023

Recently somebody asked me a question about my MS Video 1 encoder (not the one in NihAV though) and I’ve decided to look if my current encoder can be improved, so I took Ghidra and went to read the binary specification.

Essentially it did what I expected: it understands quality only, for which it calculates thresholds for skip and fill blocks to be used immediately, clustering is done in the usual K-means way and the only curious trick is that it used luminance for that.

So I decided to use that idea for improving my own encoder. I ditched the generic median cut in favour of specially crafted clustering in two groups (I select the largest cluster axis—be it luma, red, green or blue—split 4 or 16 pixels into two groups by being above average or not and calculate the average of those two groups). This made encoding about two times faster. I’ve also fixed a bug with 8-colour blocks so now it encodes data properly (previously it would result in a distorted block). And of course I’ve finally made quality affect encoding process (also by generating thresholds, but with a different formula—unlike the original my encoder uses no floating-point maths anywhere).

Also I’ve added palette mode support. The idea is simple: internally I operate on pixel quads (red, green, blue, luma) so for palette mode I just need to replace an actual pixel value with the index of the most similar palette entry. For that task I reused one of the approaches from my palettiser (it should be faster than iterating over the whole palette every time). Of course the proper way would be to map colour first to have the proper distortion calculated (because the first suitable colour may be far from perfect) but I decided not to pursue this task further, even if it results in some badly-coded features sometimes. It’s still not a serious encoder after all.

Now this member of the early 90s video codecs ruling triumvirate should be good enough. Cinepak encoder is still rather primitive so I’ll have to re-check it. Indeo 3 encoder seems to produce buggy output on complex high-motion scenes (I suspect it’s related to the number of motion vectors exceeding the limit) but it will have to wait until I rewrite the decoder. And then hopefully more interesting experiments will happen.

Indeo 3 encoder: done

February 6th, 2023

I’ve done what I wanted with the encoder, it seems to work and so I declare it to be finished. It can encode videos that other decoders can decode, it has some adjustable options and even a semblance of rate control.

Of course I’ll return to it if I ever use it and find some bugs but for now I’ll move to other things. For instance, Indeo 3 decoder needs to be rewritten now that I understand the codec better. Also I have some ideas for improving MS Video 1 encoder. And there’s TrueMotion 1 that I wanted to take a stab at. And there are some non-encoder things as well.

There’s a lot of stuff to keep me occupied (provided that I actually get myself occupied with it in the first place).