Kostya's Boring Codec World

I’ll update codec details on our wiki and hopefully can forget about it. Let somebody French-speaking complete it.

There are many codecs out there to reverse engineer, including worthier ones. On the second thought that would define almost every codec.

Update: filled all information I had, the rest is up to somebody with the debugger and motivation to do it.

After investigating the smallest available pile of fossilised dung called Go2Cesspit (only 2.5MB instead of 15MB for the current version) the G2M1 can be reconstructed. It had the same tiled structure as its successors but it coded all tiles with ELS only, no combining with JPEG data.

And the full history of Go2UnwantedPlaces evolution (there definitely was no intelligent design for obvious reasons):

1. Citrix licenses image coding from Accusoft (ELS-based), uses it to release G2M1.
2. They want to improve compression and try to code some blocks in the different way. JPEG to the rescue! That’s how G2M2 was born (compression method 2).
3. G2M3 looks like marketing bump since image coding has not been changed.
4. For some reasons they replace ELS part with simple deflated raw bitmap. That’s G2M4 now (and compression method 3).

Further findings may correct this system of course but so far it looks like this.

P.S. If you want to have G2M1 supported then send samples and support requests to VideoLAN. They will be grateful for sure.

Now I can officially say that G2M4 is essentially reverse engineered. It indeed uses zlib-compressed image for sharp details (it’s called “synthetic layer” internally) and JPEG compression for smooth detail (it’s called “natural layer” internally). The only catch — it’s not plain JPEG coding, it codes only some block with JPEG and uses a special way to restore a sparse image.

The idea of this J-BKoding (not the same as J-BKoder in Go2Coven!) is simple: only the blocks referenced from top layer are coded and to save space and improve compression they are coded continuously. So how you know how to restore the picture back? Easy! JPEG data is preceeded by number of macroblocks coded and flags (in bytes, LSB first) telling whether the block is decoded one or skipped. I suspect that something similar might be true for the previous versions of the codec too, because quite often decoded JPEG data showed that its width is less than expected.

Here’s the output of the previous version with synthetic layer only (demonstrated at FOSDEM to the close circle of trolled people):

New version (quick hack):

Now any VideoLAN guy should be able to implement a decoder for it.

As some people know, there was Go2UselessActivity decoder for Libav showcased at FOSDEM (on an ARM-based laptop even). It decoded all known variants of the codec with varying degress of success (i.e. output ranged from being completely garbage to being slightly recognizable). Some guy nicknamed “j-b” was really happy for some unknown reason.

Let’s consider a purely theoretical situation that somebody needs Go2EncountersOfTheWorstKind 2/3 and wants to know how it works (usually it’s either one or another or none). As it’s known already it combines ELS-coded images with JPEG-coded data for some pixels that are coded transparent in ELS. JPEG data, to quote j-b is boring, so let’s look at ELS image coding.

The coder used is the augmented ELS coder by Douglas M. Withers (still available somewhere in Internet insize OSAUGELS.ZIP) with the same tables (36 jots per byte). The only interesting thing is how the image itself is coded with this binary coder.

• Unsigned values are decoded as unary prefix for number of bits to follow and then the actual value, signed values are coded as unsigned with an usual scheme x -> 2*x, -x -> 2*x - 1 (zero is explicitly signalled by number of bits being zero of course).
• Pixels are coded as RGB triplets, using median prediction from top, top left and left neighbours if possible. If prediction is used then pixels are coded as differences to (G, R-G, B-G) predicted value.
• General image coding is conceptually simple: the image is coded as runs of RGB triplets if possible; in some cases if it’s possible to decode pixel from cache it’s done so. If that’s not possible too then one pixel is coded as described above.

Here’s slightly more detailed image decoding description:


for (y = 0; y < height; y++) {   x = 0;   while (x < width) {    if (x > 1 && y > 0 &&
     rgb[x - 1][y] != rgb[x - 2][y] &&
     rgb[x - 1][y] != rgb[x ][y - 1] &&
     rgb[x - 1][y] != rgb[x - 1][y - 1] &&
     rgb[x - 1][y] != rgb[x - 2][y - 1] &&
     pixel_in_cache(rgb[x - 1][y]) {
    rgb[x][y] = decode_pixel_with_prediction(x, y);
    x++;
    continue;
   }
   decode_run(x, y, &run_length, &pix);
   if (run_length > 0) {
    // pixel value may get changed here
    decode_modified_pix(x, y, run_length, &pix);
    while (run_length--)
     rgb[x++][y] = pix;
   } else if (decode_from_cache(&pix)) {
    rgb[x][y] = pix;
    x++;
   } else {
    rgb[x][y] = decode_pixel_with_prediction(x, y);
    x++;
   }
 }
}


Hopefully no more information will follow soon.