Archive for September, 2022

German PEGs

Thursday, September 29th, 2022

Having said everything I could on the current political situation, I returned to looking at random codecs and I found one with a curious name.

The name is DPEG and it’s used in at least one random game I’ve never heard of. It turned out to be a rather simple tile-based codec with raw intra frames and inter frames that employ RLE and motion compensation.

So nothing interesting but then it struck me: wait a bit, I remember REing a codec with almost the same name, block-based RLE coding approach and also from a German company (a different one though).

And there’s this Fraunhofer society involved in MPEG Video and MPEG Audio (different branches though)…

So what’s the German fascination with naming codecs [A-Z]PEG and how many out of 22 possible codecs are really implemented?

Why it’s pointless to expect anything good from russians

Monday, September 26th, 2022

So we have two major events going in parallel in two deeply undemocratic countries: Iranians having enough of their leaders and russians fleeing mobilisation. I can’t even pretend to have any knowledge on Iran so IMO the uprising there may lead to something or may be squashed like the previous ones, yet Iranians still have my respect for doing that. As for russians, even if I’d be not from Ukraine but know their history and current events, I’d still not expect anything good from them.

Here I’ll try to give a short review on russian history, culture and society to show how artificial it is, how those parts do not connect with each other, and why the population of russia mostly remained inert.

History

russian propaganda often claims that other countries (usually those having misfortune to border it) are artificial while in reality russia is the one artificial country more than any of its neighbours. If you look at e.g. Ukraine, Kazakhstan or Baltic States, the countries are formed around the nations which had been living in the corresponding regions since ages. If you look at russia, you’ll see a lot of territory grabbed from the natives by the invaders (the same applies to the USA but there are certain differences mentioned below that make them sufficiently different).

If you look at the origin of Ukraine, you’ll see that it formed itself on the trade route “from the Varangians to the Greeks” (i.e. from Scandinavia to East Roman Empire) and spread naturally around it assimilating various Slavic tribes living nearby. russia, on the other hand, takes it origin from a remote outpost on a completely different trade route in a territory populated by Finno-Ugric tribes (fun fact: Moscow originally was just a town in the principality of Suzdal and you won’t get an answer when it actually became the capital). For a long time it was a part of the Golden Horde (another fun fact russians don’t want to know: until XVIIIth century russia paid tribute to the Crimean Khanate, the successor of the Golden Horde, as its client) until it decided to become independent.

You probably have heard the title “czar”, some of you probably know that it comes from Caesar’s name but how many of you know that it was a self-proclaimed title not supported by anybody else? For comparison, Kings of Galycia-Volyn (now Western Ukraine) got their title from Pope Innocent IV as any other European monarchs did at that time. The ruling of the first czar Ivan IV was the example that set the course for russia since then: self-proclaimed titles (and history retconning), creating puppet states recognized by nobody else, limiting personal freedoms, hiding behind nominal figurehead, oprichnina, territorial conquests (and wars to squash democracies and gain seaports) and so on. This topic deserves at least a separate book so I’ll leave it at this.

To sum up the following centuries, russia behaved like an empire with the colonies being attached to it instead of being located overseas (because they could not control overseas colonies, that’s why they sold Alaska and Californian colonies for a rather measly sum). And running the empire required a lot of trained professionals—which usually were Ukrainians and Germans (usually from Baltic States area). Fun fact: even the transition of Myscovy Kingdom to russian empire was invented and proposed by a Ukrainian. So here’s one difference from the USA already: USA has not tried to pretend that it was a successor of British Empire or that its history spawned millennia.

Culture

Modern russian language is invented mostly by Ukrainians and sustained by Germans. Yes, russian empire started to invent russian language culture in XIXth century and had to rely on non-russians for that (before that elites spoke in French and russian-speaking common folk had various dialects closer to modern Ukrainian than to modern Russian; that’s why it’s easier to understand older russian or Old Slavic language if you know Ukrainian or Belarusian). And of course they tried to strangle down Ukrainian culture (the Valuev Circular and the Ems Decree are the most notorious examples) so that talented Ukrainians would have to write in russian in order to achieve anything. So when russians criticise Ukrainian language for being not as developed as russian it’s like mocking a person that you hit in a knee that he is lame.

It was no better in Soviet times: Ukrainian culture was developed by enthusiasts and the official government cared mostly about curbing the development. For example, there was a Ukrainian grammar developed in 1920 (so-called Skrypnykivka) which was less than a decade later replaced with something closer to russian language (and with subsequent reforms to make it even closer, many of those coincident with famines). Ukrainian movies were quite often forbidden to be released or were re-dubbed in russian before release (and Ukrainian originals were lost in the archives). And stealing things from the other cultures while passing them as their own is a very russian thing.

Of course borrowing is a normal thing that enriches culture. But when you take something not belonging to you, pretend that it’s your own and try to get rid of the original creator—that’s definitely bad (and only Di$ney is allowed to do so, just read the story about Kimba the White Lion). Fun thing is that is you look at any symbol associated with russia you’ll find that it’s been stolen from elsewhere with no credit given (except for the original words of USSR anthem).

Overall, russian culture is created for the needs of uniting the empire instead of coming from the common folk and thus it’s rather artificial one that still does not connect with the people (for the reasons clear from their society structure). So whatever is presented for the export is what you call classics (“something that everybody wants to have read and nobody wants to read”) while what is popular is of a completely different type. The main source of their culture seems to be prison, so virtually everybody knows at least some of prison argot and traditions, and шансон (or блатняк) is their unique genre of music universally loved there (it’s about criminals as you can guess and quite often about them getting to prison too). Side note: while American culture is not that great either but at least it’s not ashamed about its burgers, rap and typical Hollywood action movies as they’re both consumed internally and exported worldwide.

Society

Anyway, now it’s time to talk about their society structure. As I said in the beginning, the principality of Suzdal was founded by Slavic people on Finno-Ugric lands which meant not so good treatment for the locals. The same applied to other conquered nations and no wonder that serfdom was common (and if it’s not slavery it’s pretty damn close). You had all varieties of serfs—private serfs (belonging to some landlord), state serfs (belonging to the state which was not much better) and even industry serfs (serfs that were supposed to work at some mine, fabric or plant without a right to leave it). By some estimations by 1861 three quarters of the russian empire population were serfs of some kind. After “emancipation” their state has not improved much. There was a great divide between elites (who spoke French before 1812 like in many other European countries), literate people living in cities and illiterate peasants (with legal barriers preventing them from getting higher education). Returning to the culture, no wonder that whatever citizens and rich landlords created was not sticking to the majority of the population.

Additionally there’s a theory that russians living on not very fertile lands had to create a community, but it was a kind of community where everybody had nothing (except for something you’ve managed to hide) so it could be a very dynamic community to which members belonged or not depending on the current goals. That’s why a russian may expect a help from a stranger since he’s included him into his community (while the stranger usually did not). That’s why they can claim they don’t leave their own behind while actually leaving people behind—they’ve been excluded from the community.

During Soviet times there was an attempt to build classless society of the future, so it was mostly the society of workers and peasants, a strata of engineers, scientists and teachers/librarians/whatever (who earned less than workers for more skilled work) and nomenklatura (people occupying high posts in The Party who managed everybody and everything else). Of course the ideology ruled everything so almost every unpopular decision like rising prices (there was no inflation in the USSR!) were justified by “the popular demands”. And the protests were put down in rather secretive way (Novocherkassk massacre is the best known example). So people lived in the nice state of doublethink: while they saw that not everything was good they were believing it’s just them and the rest of the country was fine. And of course state propaganda claimed in all possible ways (there’s an expression “from every [electric clothes] iron” to describe exactly how ubiquitous it was) that people were living in the best possible country with the best possible order of things and minor deficiencies are caused by enemy countries that are envious of the Soviet way of things. Of course decades of such brainwashing stuck to many people, especially poor ones.

And it’s worth noticing that there was a process of divorcing reality from the reports. Since Soviet Union had plan economy, on the one hand there was a shortage of goods demanded by the population (because they were either not put in plan or the quantity was mispredicted) and on the other hand the plants and factories had to meet the plan and go slightly over the top (you’d get punished for failing to deliver the demanded amount of goods and if you do too well the plans for the next period would be increased which might be an even worse punishment). So people started to build a parallel economy based on personal connections and bartering (e.g. you know somebody who works at shoe shop, he can reserve a nice pair of shoes for you when they’re available for something he can’t easily buy himself) with the various forms of theft and forgery required to cover up for the goods sold illegally (like diluting milk to have the required amount after your friends got couple of litres of fresh fat milk) and organisations started to lie in their reports about the amounts of goods they produced (because they were rewarded or punished based on the reports).

Plus during Andropov times there was an inflation of KGB and police numbers (my grandpa told once that back in his youth they had one policeman and he was formidable, later they had dozens policemen and they hardly got any respect), and those KGB/police personnel who had to maintain internal order were bored and started to get connections with illegal manufacturers or goods, smugglers and such leading to the famous fusion of criminals and law enforcement (especially in the 1990s). Other institutions were decaying as well, in the army soldiers were used as a free labour to build houses for the commanders and lying there was ubiquitous as well (for example, many memoirs and recollections tell how it was more important to make an impression of doing something big instead of doing something small properly; it is also said that a plane escaped USSR because anti-air defence reported plane movements after its regular schedule, i.e. always giving the same numbers instead of actually looking at the radar).

Modern times

After dissolution of the USSR in 1991, the situation was grim for many post-Soviet republics because of the destroyed economical ties (and the fact that most of the goods produced were acceptable only inside of the USSR and only for the lack of the alternative). In either case, 1990s in russia are nicknamed the cursed nineties and people don’t want to realise those were the times that made russia stronger and wealthier. But considering how the first president of russia had to protect himself and certain people around him from possible (well-grounded) legal persecution, he passed the post in semi-democratic way (i.e. there were elections but with mass media under proper control and virtually no time for the other candidates to prepare) to the selected successor, who was a former KGB agent and a member of Tambov Gang. During his rule all strategic companies were put under control of his friends, cronies or underlings. And of course since the führer missed USSR, he tried to reboot it including switching to the different economic model where everything important belongs to the state (i.e. his people acting in his and their own interests) and the rest of people live essentially in trickle-down economy where oligarchs get money from the state usually by being allowed to mine and sell natural resources or to fulfil state contracts, other companies get their share of money from serving those oligarchs, other companies get their money from doing work for them and so on. And law enforcement forces get their share for turning blind eye to it or directly participating in the illegal activity.

This of course corrupted people as they see the quickest way to get rich is to either have proper connections so you can participate in cut-and-kickback scheme or to join law enforcement (or some safety inspection) and get you money from bribes and “protection” money. Doing business in such condition is possible only if you have right connections to protect you from the people above (though people protecting you might want to take your business for themselves if it gets profitable enough). And if you’re too poor you just have to rely on the pity sustenance provided to you by the state (which paradoxically makes you love it).

So what are russians and what to do with them?

And that is how you get russians—not a real nation but rather a gathering of people (if you can call them people) who care only about themselves. Even with the current protests against “partial” mobilisation you can see that protesters in Moscow do nothing for the others and let them be detained while in e.g. Dagestan protesters fight against the police to prevent people from being arrested (similarly to the current Iranian uprising or Ukrainian Euromaidan of 2014). Similarly russians have never protested for defending their freedoms, there were several uprisings in the past but they were mostly a way to tell the czar that their living conditions are bad (or sometimes it was against a reform introducing something new).

Thanks to their imperial past they feel superior to other nations and despise them while still being afraid of any real nation—like Ukrainians or their Caucasian nations (Dagestani, Chechen and so on). This probably happens because of their disunity since they know nobody will fight for them while others may get help from their compatriots. And of course such people would be worse than professional executioners if they get a chance to get even on somebody else. The findings and testaments from places like Bucha, Yahidne or Izyum prove that. Similarly there are many accounts when russians hearing about mobilisation are outraged by the fact it affects them as they’d prefer somebody else to die instead of them, especially national minorities (“russian tank driver from Buryatia” has become a meme during the first phase of this war in 2014 already).

Probably another affect of their disunity is their volatility and unreliability. In the past centuries they’ve been trained well enough to support whatever is forced unto them by the authorities (either out of fear or in faint hope of reward) but they do it half-heartedly. That is why after the parade with photos of fallen WWII heroes (or whatever passes for them) as their current state religion demands all those photos gets thrown away to the nearest garbage heap. That is why they were putting half-swastikas onto their cars after the start of the war and that’s why they’re removing it after hearing the rumours that those wearing it would be drafted first (all while complaining “I’m a patriot so let all those prisoners and traitors fight instead”).

That is why I don’t think that accepting russian “refugees” (especially in large quantities) is a good idea neither from the strategic point of view (as those troops won’t make a large difference anyway, modern wars are fought not by large masses of unequipped people) nor for the receiving side. Thanks again to the imperial past, they hate other nations and refuse to learn local language let alone to fully integrate—Baltic states can tell a lot of stories and here in Germany it’s no better. Diaspora by itself is not a bad thing at all, but not respecting a host country while still expecting it to provide for you is. Now “good russians” (an oxymoron IMO) point at Mongolia offering Buryats a refuge and saying other countries should do the same but in this case we have a country offering a shelter to an oppressed national minority with a culture similar to its own and not somebody fleeing from the mess he helped to create (or she, if they start drafting women at large too). It’s like the saying I heard here that Austrians consider Austria to be the first victim of Hitler while Germans say it’s Germany (but guess who voted him to power).

I’m not a nice man and I believe that you should not try to save adults from their own mistakes. Of course it’s different for children but that’s exactly because they don’t know much and don’t realize what consequences their actions might have. If you’re a legal adult you don’t have such excuse and you should bear full responsibility. Giving those fleeing russians a shelter won’t stop the war, won’t eliminate the risk of a deranged dictator using nuclear weapons as the last resort and it won’t help russians to realize that what they’re doing is wrong. The opposite will probably have the same effect but would require significantly less efforts and resources. Save them for the occasion when you can finally make difference by teaching them that their ideology is bad and why. It had worked for the West Germany after all.

Why I’m not excited about RISC-V

Tuesday, September 20th, 2022

So while russia is trying to commit political suicide by recognizing its war officially and making it a criminal offence to evade it (or disagreeing with the official course in general), here’s a text I wanted to write for a long time but finished only recently.

In general I’m eager to look at some computer architecture and see how multimedia software can be run there (even if I’m no Måns). For a long time my primary development machine was PowerPC-based (before I could buy some decent x86_64-based laptop), I played with ARMv7 and NEON years before Raspberry Pi was created (but who remembers Beagle Board nowadays?), I had two laptops with Chinese MIPS inside (and tried optimising for Loongson 2 SIMD too), I own ARM64-based box too (and did some work on it as well). I’d like to try RISC-V hardware but the state of RISC-V gives me no excitement and here I’ll try to explain why.
(more…)

Looking at a-Pac

Monday, September 19th, 2022

Since currently there’s a preparation phase for the next Ukrainian counteroffensive and I don’t know what other features to add to NihAV (beside improving video player), I was bored enough to do this.

After a comment from Paul, I’ve looked at some random lossless audio codec from 90’s, namely a-Pac. While the codec by itself it very simple (no LPC or IIR filter, no variable-length codes except in block header, no arithmetic coder either) as you can see from its description in the Wiki, it was a bit tricky to RE.

The main issues being: apac.exe is in NE format (which means 16-bit code and Ghidra sucks at figuring out which segment registers should be used when) and, which is worse, written in Delphi with all the quirks of its object model implementation. So it’s after some search I’ve found the virtual table corresponding to the APAC format handler (both coding and decoding) and after trying one function after another I’ve finally found the one responsible for encoding—after which finding a decoder function was easy.

The code itself (beside the weirdness introduced by Delphi compiler like using both positive and negative offsets in vtable calls) is very old-fashioned too: bit reading is done by keeping bitreader state as global variables, reading n-bit values by reading a single bit in a loop and refilling that bit buffer every byte (with a callback function).

It’s still fun to see how we’ve moved from simple formats like this to overcomplicated formats like LA or OptimFrog while settling down on in-between complexity FLAC, Monkey’s Audio and WavPack.

russia and friends

Monday, September 12th, 2022

I’ve been wanting to write this for a long time and the russian terrorist attack on Sunday was the last straw.

There a saying: tell me who your friends are, and I will tell you who you are. Of course it’s not always true but it hints on the usual way of choosing friends to have a lot of common with you. The same stands true not just for people but for whole countries as well and you can guess which countries we’re going to look at today.

First, here’s a list of countries that support or openly befriend russia (the list is composed from the countries that russia openly called friends or those behaving in that way):

  • Afghanistan
  • Belarus
  • Burundi
  • Cameroon
  • Chad
  • China
  • Eritrea
  • Ethiopia
  • Guinea
  • Hungary
  • Iran
  • Mali
  • Myanmar
  • North Korea
  • Serbia
  • South Africa
  • Sudan
  • Türkiye
  • Uganda
  • Venezuela
  • Zimbabwe

So what’s common between those countries? They can be divided into two major groups—former colonies with a dictator ruling it (or demokratur, see that post for the explanation) or leftovers of former empires that dream of restoring their old glory (and have about the same government). The first group befriends russia mostly for the benefits (the same way many African countries befriended USSR), the countries from the second group usually went the same way as russia itself and thus they have some class solidarity, though it does not prevent them from (ab)using russia to their own benefit. Now let’s look at them closer.

Negotiable affection friends

Afghanistan. If you haven’t forgotten yet, this is a country where a corrupt weak government was forcefully replaced by a terrorist organisation recognized in the whole world (even russia, which led to a rather funny situation where they invited Taliban representatives to various forums and diplomatic talks while still not lifting a status of terrorist organisation). So you have the same terrorist mindset to keep them close. And in somewhat recent news they asked for a large oil shipments in return for something undisclosed. Considering how good Afghanistan is economically and that russia won’t be able to mine for natural resources there effectively it looks like typical Soviet-African friendship.

Belarus. A former Soviet colony (sometimes called “most Soviet of all Soviet republics”) with a dictator ruling it since 1994. Lukashenko had been using russia for his own benefits (like buying oil cheap from russia and selling petroleum not-so-cheap to Europe or even getting non-returnable loans for various purposes) for a long time, usually under pretence of making Union State something different from a bureaucratic fiction. But after the coup against Belarusian people in 2020 he has to rely on russia for everything and that’s why Belarus serves as a large military base for russian forces and that’s why he makes various bellicose claims (while not actually trying to engage his own forces because they’re more likely to turn against him).

Burundi, Cameroon, Chad, Eritrea, Ethiopia, Guinea, Mali, Sudan, Uganda, Zimbabwe. I can’t say I know much about those countries so let’s look what’s common there. Former colonies (mostly French ones)—check, rather poor countries—check for many of them, dictatorship/demokratur—also check for many of them (or military junta like in Myanmar). So it seems to me they’re friends with russia because they behave the same or just use it to get cheap resources or military equipment.

Myanmar.—this country seem to go the same route as some African countries: former colony, low quality of life and civil war, recent military coup (after which it befriended russia).

North Korea

I think this country makes a category of its own. On the one hand it enjoys benefits from “befriending” russia as it can sell its resources there and send workers to earn good money (by their standards), on the other hand it’s russia that tries to resemble DPRK. And let’s not forget about the recent news about russia buying artillery munition from it (there are unverified claims that in the first months of war russian minister of war went there to beg for other Soviet equipment like missiles). Though the dynamic between North and South Korea reminds a lot about the current situation including the constant threats of using nukes to destroy Americans (in South Korea).

Countries resembling russia

China. This country has history resembling russian one a lot. Having an long-running proud history that’s mostly fake—check, being conquered by Mongols—check, getting rid of an emperor in the beginning of XXth century—check,this event being followed by creating a Western-type government that was deposed by communists—check, communist terror—check, reforms that tried to make the country more open to the West —check, a dictator who wants to bring country back to the glorious past—check. Territorial claims for the (pieces of) neighbouring countries—check. Rather passive population that does not care about anything—check. Not keeping their word, taking whatever businesses they like from the foreigners and not allowing them to de facto own anything—check (yes, China does it in a more refined way but you should remember how your business there should have at least 51% partnership by Chinese and how the stock of Chinese companies traded overseas is not actually a right of ownership in a company and what had happened to ARM China). And of course copying or stealing ideas and design from the West while pretending it’s their own (again, China is doing it much better and actually develops on it). Here’s a personal story: my great-grandfather worked at a local locomotive-building plant (mostly famous for creating the T-34 tank) which was founded by a French society (a fact not often mentioned in the USSR) and in 1950s he was sent to China to help them build their own industry (a fact not mentioned often in China).

Though you should not forget that some of the territories that China wants to return are those taken by russian empire. And you should not forget that China acts pragmatically so when russia threatened to sell its natural resources to China instead of Europe and make reserves in yuan, it essentially made itself dependent on China which can now dictate the prices for oil and gas (and doing operations with yuan also requires an approval of Chinese government). Currently China pretends to not support russia because Western sanctions hurt. But we’ll see if this stops them from attempting to conquer Taiwan.

Hungary. Former Kingdom of Hungary, later part of Austro-Hungarian Empire. Current regime relies on the same principle as russia—praising “traditional values” and hating everything not from the country. And of course they want to get back territories “lost” after 1918: you should remember how orbán asked for Croatian port and pipeline during the European sanctions discussions. And they’ve been giving away Hungarian passports to Ukrainians living in Subcarpathian Ukraine for years along with trying to dictate what Ukraine should do in its region (russia had been doing the same in various places that later it turned into puppet states like Abkhazia).

The related thing is that how the government tries to gloss over its wartime atrocities (for example, this, even if the English version of the article does not mention them) and have installed a hypocritical monument to Hungary being a “victim” to Nazi Germany (being its willing ally until 1944 does not count apparently). The same way russia always accuses Germany attacking it in 1941 while not mentioning that it happened in the Poland which both of them occupied in 1939 (and their co-operation went further than that).

And of course you should not forget that Hungary gets most of its energy from russia: oil, gas and even fuel for nuclear power. Seeing how Hungary sells Serbia gas from an unknown source and then their minister going to Moscow to beg for increased shipment of gas is disgusting. Seeing how a plane delivering nuclear fuel from Moscow lands at NATO base in Hungary or how Hungary allows russia to build a new nuclear power plant right after what they’ve done to Zaporizhzhia nuclear power plant is revolting.

Iran. While I have not heard about them having claims on trying to restore an old empire (which would be some extremely old empire in their case), they’ve been trying to conquer their neighbours (at least Syria) and destroy certain state for purely ideological reasons. They also have nominal democracy (i.e. you can vote for public officials but it’s certain people with religious authority who decide what will be done) and their rhetoric is the same as in russia (at least when russia does not repeat the speeches from Nazi Germany). And considering the recent news about their battle UAVs shipped to russia and a good deal of them being broken right from the start, Iran manufactures the goods (and lies about their quality) about as good as russia.

Serbia. This is a rather sad case. On one hand russian empire was seen there as the defender of Slavic nations in different lands (which has started the World War I among other things) and their historically significant territory Kosovo turned into Kosova. On the other hand Yugoslavia (both kingdom and SFRJ) was a mix of nations with significant differences (mostly in the religion) and typical highlander character that makes peaceful resolving of the conflicts less possible (just look how well various nations in Caucasian nations get along). So when the external pressure that kept them together was gone, the internal tensions made dissolution of Yugoslavian Federation a very bloody process that required an internation intervention to make the bloodbath end more or less. And yet Serbia seems to train its army for invasion instead of defence and considering recent moves on Kosov(a/o) and fuelling tensions in Bosnia and Herzegovina there may be yet another tragedy brewing.

Türkiye. To condense its history, first there had been Ottoman Empire that grew around the Mediterranean Sea plus Central Europe and Mesopotamia, but after WWI it collapsed and was succeeded by Turkey, a country where army made military coups in order to preserve democracy since Mustafa Kemal himself (but sooner or later people elected autocrats anyway). (The) last coup in 2016 has failed and now we have Erdoğan trying to restore the old empire (and since he demanded to change the official country name to Türkiye I’m using it to denote the country under his rule and not the previous state). Of course he also wants to be a sultan and bring back the various pieces of land lost since 1922, from Albania to Yemen.

It’s hard to call him a complete friend of russia as he’s mostly using it for his own needs, but while he had closed Bosporus for their military ships and Turkish airspace for their military planes (mostly because he wants them out of Syria and to stop protecting Armenia as well), at the same time he does not support sanctions against russia and enjoys being an offshore. Recently it was agreed by two dictators that Türkiye will help russia to evade sanctions by allowing russian money in Turkish banks and russian oil to Turkish refineries (so later you can’t track the origin of oil and oil products). It is obvious that he’ll break any agreement with them as soon as it stops being profitable to him—but russia would do the same so they’re a perfect match.

South Africa. This one I can’t tell about but from random news I hear about it, it sounds a lot like russia—a corrupt government, main wealth coming from natural resources, and hardly any of this wealth reaches common people that have to live in shitty conditions.

Venezuela. Another country that resembles russia (and even more it resembles Soviet Union). You have corrupt non-democratic government that stifled all of the opposition, you have wealth coming from the natural resources (oil and gold) and thanks to the mismanagement the population is extremely poor and depends on the dole their government gives them.

Conclusion

I’ve not mentioned sympathisers like France or Italy that receive only russian tourists and oligarchs buying villas and yachts there so they don’t want to do anything against them and would rather have some peaceful solution, those countries are like a lighter version of negotiable affection friends. I’ve not mentioned current Germany that enjoyed cheap oil and gas for its industry and the position of main russian gas distributor for Europe and how this corrupted it on the border with high treason. I’ve not mentioned Armenia that has to be friends with russia because it “defends” it from Azerbaijan (while russia shipped weapons there as well). I’ve not mentioned Georgia that has a government elected on the idea “if you don’t elect us there will be a war with russia again, and here’s a bribe to make you feel less miserable”. I’ve decided to concentrate on the countries that are helping russia willingly (even if that will is compensated).

As you could see, most of russian friends are the countries expecting something (or a lot) in return for their friendship and (with one exception) none of them can be called prosperous. Misery loves company or birds of a feather flock together?

Looking at some FMV formats

Wednesday, September 7th, 2022

It’ll take a long time to liberate Ukraine from the invaders, so while it goes on I have to entertain myself somehow.

Since I had nothing better to do I looked at two FMV games—Deadly Tide and Hot Wheels: Stunt Track Driver. Both of them have their levels packed as video data with some additional control data.

Deadly Tide is more complex one with its L96 format: it defines a resource file with several possible sub-levels and three different video streams (something like 320×240, 160×120 and 80×80) compressed with Cinepak plus half a dozen of various control resources and audio stream.

Hot Wheels uses JAM format which is a series of compressed chunks. 0 – frame data, 2 – palette update (strangely enough it updates palette not from the start but rather to the end), 3 – some control data. Frames are stored either unpacked, packed with LZSS or packed with differential LZSS (in this case copy operation takes bytes from the previous frame). Oh, and it uses obfuscated WAV files for audio that I didn’t bother to look at.

I’m still looking for an interesting format to RE, meanwhile those two were at least somewhat interesting.

Rust inline assembly experience

Saturday, September 3rd, 2022

Since I need something less exciting than the series about IAEA willingly ignoring the terrorists occupying the largest nuclear power plant in Europe (even during its mission visit there) or the series called “what russia destroyed in my home city today”, I’ve tried inline assembly support in recent stable Rust compiler, here’s a short report.

Since I’m working on a multimedia framework, my primary interest about inline assembly is how well I can add SIMD code for various codecs. My previous attempt was optimising the adaptive filter in Monkey’s Audio decoder and while it worked it looked ugly because of the way Intel named its intrinsics (if you like the names like _mm_madd_epi16 then our tastes are very different) and the verbosity (constant need to cast vectors to different types. So I decided to wait until non-experimental inline assembly support is ready.

This time I’ve decided to look how easy it is to make SIMD optimisations for my own H.264 decoder (and it needs them in order to be usable when I finally switch to my own video player). Good things: I’ve managed to speed up overall decoding about 20%. Bad things: a lot of things can’t be made faster because of the limitations.

For those who are not familiar, H.264 decoder contains a lot of typical operations performed on blocks with sizes 2×2, 4×4, 8×8 or 16×16 (or rectangular blocks made by splitting those in half) with operations being copying, adding data to a block, averaging two blocks and so on.

Writing the code by itself is nice: you can have a function with a single unsafe{ asm!(..); } statement in it and you let the compiler to figure out the details (the rather famous x86inc.asm is mostly written to deal with the discrepancies between ABIs on different platforms and for templating MMX/SSE/AVX code). Even nicer is that you can specify arguments in a clear way (which is much better than passing constraints in three groups for GCC syntax) and used named arguments inside the code. Additionally it uses local labels in gas form which are a bit clearer to use and don’t clutter debug symbols.

Now for the bad things: inline assembly support (as of rustc 1.62.1) is lacking for my needs. Here’s my list of the annoyances with ascending severity:

  • the problem with sub-registers: I had to fill XMM register from a GPR one so I wrote movd xmm0, {val} with val being a 32-bit value. The compiler generated a warning and the actual instruction in the binary was movq xmm0, rdx (which is copying eight bytes instead of four). And it’s not immediately clear that you should write it as movd xmm0, {val:e} in that case (at least Luca has reported this on my behalf so it may be improved soon);
  • asm!() currently supports only registers as input/output arguments while in reality it should be able to substitute some things without using registers for them—e.g. when instruction can take a constant number (like shifts, it’s very useful for templated code) or a memory reference (there are not so many registers available on x86 so writing something like paddw xmm1, TABLE[{offset}] would save one XMM register for loading table contents explicitly). I’m aware there’s a work going on that so in the future we should be able to use constant and sym input types but currently it’s unstable;
  • and the worst issue is the lack of templating support. For instance, I have functions for faster averaging of two blocks—they simply load a certain amount of pixels from each line, average them and write back. For 16×16 case I additionally unroll the loop a bit more. It would be nice to put it into single macro that instantiates all the variants by substituting load/store instruction and enabling certain additional code inside the loop when block width is sixteen. Of course I work around it by copy-pasting and editing the code but this process is prone to introducing errors (especially when you confuse two nearly identical functions—and they tend to become long when written in assembly). And I can’t imagine how to use macro_rules!() to either construct asm!() contents from a pieces or to make it cut out some content out of it. Having several asm!() blocks one after another is not always feasible as nobody can guarantee you that the compiler won’t insert some code between them to juggle the registers used for the arguments.

All in all, I’d say that inline assembly support in Rust is promising but not yet fully usable for my needs.

Update. Luca actually tried to solve templating problem and even wrote a post about it. There’s a limited way to do that via concat!() instead of string substitution and a somewhat convoluted way to fit some blocks inside one assembly template. It’s not perfect but if you’re desperate enough it should work for you.

Ripper: Smackered AVIs

Thursday, September 1st, 2022

Recently I’ve encountered a review of an old FMV game called Ripper and my professional interest was to look what formats it uses. Well, it turned out to use Smacker and AVI. And those AVI files are their own format that uses Smacker in perverse way: it stores a sequence of commands with 14-byte header that control video playback. The first dozen of commands are usually “fill audio buffer with this data” commands and there’s one command for initialising video that includes full Smacker header inside its payload. From what I see it abuses Smacker decoder by constructing file on the fly using the data from the commands (I’m pretty sure the following commands have video frames in their payload).

I’m not sure if I’d want to pursue it further and document it let alone adding support for it in NihAV but that format was definitely something.