Archive for February, 2013

Monkey’s Audio: noted differenced between versions

Thursday, February 28th, 2013

While preparing for working on old APE versions support I finally got courage to try and trace all changes for different versions. So here’s the list of internal versions and the changes they introduced:

  • 0000 — the reference version for all prehistoric version. Before version 0000 it was fine, then it all got worse IMO.
  • 3320 — changes in the filters
  • 3600 — changes in the filters
  • 3700 — changes in the filters
  • 3800 — blocks per frame changed for extra high compression level; changes in the filters (yawn)
  • 3810 — frame start at byte boundaries now
  • 3820 — special codes extension (signalled by top bit of CRC set to one)
  • 3830 — filter lengths and some implementation details changed
  • 3840 — CRC calculation algorithm changed a bit
  • 3870 — significant changes in residue coding
  • 3890 — small changes in residue coding
  • 3900 — residue coding format has changed seriously.
  • 3910 — small change in the residue coding (more than 16 bit values can be coded now)
  • 3930 — significantly changed format introduced (both filtering and coding scheme were changed)
  • 3950 — filter format changed a bit (and insane compression mode is added somewhere after that), blocks per frame is changed too
  • 3960 — some small and compatible change in the bitstream (consuming two last bytes or not)
  • 3980 — file format is changed a bit; filtering process has changed a bit too.
  • 3990 — the latest (known) format. Residue coding has changed.

Do you still wonder why I strongly dislike this format?

Preserving extinct formats

Wednesday, February 27th, 2013

By the request of one guy (he has provided samples as well) I shall work on supporting old Monkey’s Audio versions (before 3.95).

Why? Because the latest official version of Monkey’s Audio has dropped support for those files, because I wanted to support such files since really long time (just didn’t have a good opportunity to do that) and because I definitely need a distraction from Go2Insanity codec (I shan’t blog about it anymore).

Well, let’s see what the old versions of the worst (known) designed lossless audio codec have to offer me.

Now (hopefully) the last post about Go2Disassembly details

Monday, February 18th, 2013

I’ll update codec details on our wiki and hopefully can forget about it. Let somebody French-speaking complete it.

There are many codecs out there to reverse engineer, including worthier ones. On the second thought that would define almost every codec.

Update: filled all information I had, the rest is up to somebody with the debugger and motivation to do it.

Another piece of digital archeology news

Thursday, February 14th, 2013

After investigating the smallest available pile of fossilised dung called Go2Cesspit (only 2.5MB instead of 15MB for the current version) the G2M1 can be reconstructed. It had the same tiled structure as its successors but it coded all tiles with ELS only, no combining with JPEG data.

And the full history of Go2UnwantedPlaces evolution (there definitely was no intelligent design for obvious reasons):

  1. Citrix licenses image coding from Accusoft (ELS-based), uses it to release G2M1.
  2. They want to improve compression and try to code some blocks in the different way. JPEG to the rescue! That’s how G2M2 was born (compression method 2).
  3. G2M3 looks like marketing bump since image coding has not been changed.
  4. For some reasons they replace ELS part with simple deflated raw bitmap. That’s G2M4 now (and compression method 3).

Further findings may correct this system of course but so far it looks like this.

P.S. If you want to have G2M1 supported then send samples and support requests to VideoLAN. They will be grateful for sure.

Now the final words about Go2UselessTalk

Wednesday, February 13th, 2013

Now I can officially say that G2M4 is essentially reverse engineered. It indeed uses zlib-compressed image for sharp details (it’s called “synthetic layer” internally) and JPEG compression for smooth detail (it’s called “natural layer” internally). The only catch — it’s not plain JPEG coding, it codes only some block with JPEG and uses a special way to restore a sparse image.

The idea of this J-BKoding (not the same as J-BKoder in Go2Coven!) is simple: only the blocks referenced from top layer are coded and to save space and improve compression they are coded continuously. So how you know how to restore the picture back? Easy! JPEG data is preceeded by number of macroblocks coded and flags (in bytes, LSB first) telling whether the block is decoded one or skipped. I suspect that something similar might be true for the previous versions of the codec too, because quite often decoded JPEG data showed that its width is less than expected.

Here’s the output of the previous version with synthetic layer only (demonstrated at FOSDEM to the close circle of trolled people):

G2M4 decoder output

New version (quick hack):

0001

Now any VideoLAN guy should be able to implement a decoder for it.

A few words about Go2WasteTime version 2/3

Saturday, February 9th, 2013

As some people know, there was Go2UselessActivity decoder for Libav showcased at FOSDEM (on an ARM-based laptop even). It decoded all known variants of the codec with varying degress of success (i.e. output ranged from being completely garbage to being slightly recognizable). Some guy nicknamed “j-b” was really happy for some unknown reason.

Let’s consider a purely theoretical situation that somebody needs Go2EncountersOfTheWorstKind 2/3 and wants to know how it works (usually it’s either one or another or none). As it’s known already it combines ELS-coded images with JPEG-coded data for some pixels that are coded transparent in ELS. JPEG data, to quote j-b is boring, so let’s look at ELS image coding.

The coder used is the augmented ELS coder by Douglas M. Withers (still available somewhere in Internet insize OSAUGELS.ZIP) with the same tables (36 jots per byte). The only interesting thing is how the image itself is coded with this binary coder.

  • Unsigned values are decoded as unary prefix for number of bits to follow and then the actual value, signed values are coded as unsigned with an usual scheme x -> 2*x, -x -> 2*x - 1 (zero is explicitly signalled by number of bits being zero of course).
  • Pixels are coded as RGB triplets, using median prediction from top, top left and left neighbours if possible. If prediction is used then pixels are coded as differences to (G, R-G, B-G) predicted value.
  • General image coding is conceptually simple: the image is coded as runs of RGB triplets if possible; in some cases if it’s possible to decode pixel from cache it’s done so. If that’s not possible too then one pixel is coded as described above.

Here’s slightly more detailed image decoding description:


for (y = 0; y < height; y++) {   x = 0;   while (x < width) {    if (x > 1 && y > 0 &&
     rgb[x - 1][y] != rgb[x - 2][y] &&
     rgb[x - 1][y] != rgb[x ][y - 1] &&
     rgb[x - 1][y] != rgb[x - 1][y - 1] &&
     rgb[x - 1][y] != rgb[x - 2][y - 1] &&
     pixel_in_cache(rgb[x - 1][y]) {
    rgb[x][y] = decode_pixel_with_prediction(x, y);
    x++;
    continue;
   }
   decode_run(x, y, &run_length, &pix);
   if (run_length > 0) {
    // pixel value may get changed here
    decode_modified_pix(x, y, run_length, &pix);
    while (run_length--)
     rgb[x++][y] = pix;
   } else if (decode_from_cache(&pix)) {
    rgb[x][y] = pix;
    x++;
   } else {
    rgb[x][y] = decode_pixel_with_prediction(x, y);
    x++;
   }
 }
}

Hopefully no more information will follow soon.

How one marks useless activity

Wednesday, February 6th, 2013

There are many things Ukraine is infamous for, most of them are related to the state. Today I’m going to review the name for one of its main soap opera places (my frient from Kanthalland wanted posts like this).

So, “parliament”. The place where useless people gather to make useless, idiotic or outright harmful legislation (rare exceptions are not listed).

In English and some other languages it’s called by the word coming from French “parle” — “to speak”, the other nations have completely different names for it, let’s see what they are.

  • Talks ­— see above.
  • Gathering — very common too, especially under the names “assembly” or “congress”, quite often local word for “gathering together” is used as well, like “скупштина”, “zgromadzenie/sejm” or “国会”(the latter one is more like “state meeting” though).
  • (Daily) work — even though some argue it had nothing to do with the word “day”, in Germanic languages it became so. And besides ever-so-funny Diet of Worms(which had nothing to do with either diet or invertebrates) there are Riksdag (in Stockholm) and Bundestag (in Sumpfstadt).
  • Giving a tip — in theory if people consult with each other before passing a decision you can call them a council. In practice such legislative organ is called Верховный Совет (Supreme Council in Russian, the name of Soviet parliament) or Верховна Рада (the same in Ukrainian though Ukrainians think Верховна Зрада or Supreme Betrayal is more appropriate since it serves either not national or not Ukrainian interests) or something like Bundesrat.
  • Old geezers — quite popular too, mostly in the form of “senate”.
  • A thing — an old Scandinavic word “ting” (which is still used for local government in Sweden for instance) meant “a thing” (okay, not only that but it’s funnier this way) and in some places it is still used in parliament’s name, e.g. Denmark, Norway and Isle of Man.
  • House — or chamber. Too many obvious examples.

Of course there are some names that do not fit into this category:

  • in Netherlands they still believe in class system and call it “all estates” or Staten-Generaal. The same is true for some English islands. In France they had also tried it and it ended with the traditional French outcome (revolution).
  • in Russia they call it “Государственная Дума” (from the word “думать” — to think) but as it’s common in Russia the name has nothing to do with reality.
  • in Vatican they have a comission with an extra long name.

Moral of the story: no matter how you name your legislative organ it’s still possible to laugh at it.

On Sweden and small details

Monday, February 4th, 2013

Probably Sweden was never a really influental in science besides chemistry but it has contributed a lot of small things that really made our world better: high-precision measuring tools, or official human rights representative, or Trocadero, even flatpacked furniture.

Here’s an example: in the swedishest corner of Sweden there was a guy who really changed our world with two simple things. One of them is centrifugal milk separator. Another one is the nozzle. Yes, a simple nozzle that is used in rocket and jet engines. Were those inventions that great and important? Maybe not. Were they useful and have they changed the world a bit to the better? Definitely yes.

And that’s how the place where he came from looks nowadays:

IMG_1107

IMG_1109

IMG_1119

(in case it was not obvious — I’ve been to Orsa and I’m proud about that).