Kostya's Boring Codec World

I should have written this earlier if not for non-FFmpeg work I have to do here. BTW, are some linguists around there that can explain a relation between bureaucratic and textile (“bureaucracy” comes from a sort of cloth used to cover tables, “red tape” is rather obvious, Russian “????????”, “????????” and “??????????” are also related to a process of obtaining thin threads). Ahem.

AAC coding has two computationally costly operations — MDCT and coefficient quantisation. While the former takes more cycles per one call, the latter is usually called several times for each frame, so those times tend to sum up and outweigh MDCT in bad encoders (like mine). From rate distortion theory we know how to determine proper quantizers for AAC – distortion caused by that quantisation multipled by lambda plus number of bits needed to code that band with this quantiser should be minimal for given value of lambda.

How could we achieve this? Well, use one of three approaches:

1. Assign some fixed quantizers
2. Use some ad hoc rule to determine quantiser and then refine its value a bit (aka heuristic, since it gives good speed, it is widely used)
3. Try all possible quantisers by brute force or Viterbi method (optimal but very slow)

With heuristic you have one catch: if your primary guess on quantiser is not good then refining either takes a lot of time or gives you far from optimal result. Trellis-based search is implemented in my decoder and results in around 20x slower than realtime encoding speed (i.e. encoding one second of audio takes 20 seconds of CPU time) on modern CPUs. I’m playing with something heuristical and fast.

Now to quantising itself.

Each coefficient is quantised as out = (int)pow(in / quantiser, 0.75);. Division of floating-point numbers is slow, taking power of a number is even slower. You can convert MDCT coefficients to the power of three fourths (and quantisers are also converted in precomputed table), thus getting rid of power. FAAC also multiplies coefficients so they are always quantised except for taking integer part. My decoder just multiplies possible codebook vectors by that quantiser and compares it with input coefficients leaving them intact. I also had an idea to present MDCT coefficients in base pow(2, 0.25) making it easy to manipulate but someone still has to test it where base conversions won’t eat all of the gain. I have also tried several optimisations like not trying to match coefficients against all codebook vectors using only close enough vectors. More approaches to try.

(I hope these notes will form “How I Wrote the Best Opensource AAC Encoder Around (to Accompany x264)” memoirs :-S )

Well, Summer of Code 2008 goes to end. I think I’ve completed my task – creating simple AAC encoder with psychoacoustics
(here are my application link and a rough task description). I’ve written AAC encoder with psychoacoustic model made after 3GPP TS26.403 and it seems to work fine (on multichannel sources too).

Now there’s a small catch – to be included into FFmpeg it should be optimal in the terms of coding. That means minding rate distortion, choosing optimal quantizers and codebooks, etc. Now it’s a bit harder task, so I think final encoder will be in SVN this autumn or later. Any pointers on information how to optimally encode some part of stream are welcome (for miscellaneous stuff like pulses, temporary noise shaping, etc.).

Stay tuned (and use Flac instead 😉 ).

If there are still people interested in my work on AAC encoder (I doubt though), here is a status report for you.

Tasks completed:

• Make encoder provide lookahead samples
• Make psy model use these samples to produce block switching decision
• M/S case handling (at least I hope so)
• Multichannel coding (from mono to 5.1, seems to work fine)

So there are only two bits left: tune model to produce better sound and commit that all to FFmpeg SVN.

The problem with current implementation is that it follows given bitrate too closely, so some frames may be coded too badly resulting in audible artifacts. I will add ABR mode and quality-based mode to make it better. Another issue is how to use codec options to turn them on, so Robert Swain can add profiles for them ;).

In related news: AAC decoder is slowly migrating to FFmpeg SVN, so I should be able to submit my encoder for review and inclusion as well.

I’ve not abandoned work on AAC encoder completely.
For example, I think now my psy model calculates and handles bitrate better. Now the only goals I set to myself are:

• Make encoder provide psy model lookahead samples for block switching
• Finetune psy model, or more specifically:
  • block switching decision
  • finish M/S case handling (for now it does not update all psy model information and result in artifacts on several test samples)
  • adjust scalefactors to reduce quantisation distortion
  • some other tricks from 3GPP TS26.403?

After that there are several ways to go: improve quality until it beats other encoders (at least libfaac), implement SBR/PS encoding (the latter is easier to do), introduce multichannel coding.

I suspect that at least one person will suggest to do it all.

I’m battling with psychoacoustic model. 3GPP TS26.403 seemed clear at the start but then problems have begun.

The main problem is perceptual entropy estimation. Since MDCT coefficients differ by magnitude in FFmpeg and 3GPP, perceptual entropy estimated by formulae from 3GPP differs much from actual number of bits to code. Also 3GPP encoder always makes scalefactor lie in range 104…164, beats me why (and its center does not correspond to scale = 1.0 either).

I have to investigate further before continuing work on psy model. I also hope to see AAC decoder in FFmpeg SVN soon and push my work on encoder there as well.