Archive for the ‘RealVideo’ Category

That is not dead which can eternal buffer, And with strange aeons RALF may be implemented.

Sunday, August 7th, 2011

And now for something completely different, a post about our favourite eldritch abomination (the word “buffering” should tell it all).

I’ve decided to spend some time on RealVideo codecs.

  • RM demuxer — the one in Libav is based on some scarce guesswork and does many things incorrectly (reporting incorrect FPS, reporting PTS while container stores only DTS, ignoring interleavers, selecting video codec by version reported in its extradata, etc.). I hope improve it a bit or kill trying.
  • RV1/2 — our decoder is based mostly on guesswork, I’ve looked at it and tried to correct header parsing at least. For these codecs internal version number actually matters for bitstream reading. Another funny fact: Real didn’t develop RV2 fully by themselves, they based it on Intel ILVC sources (even some header from Helix distribution says it and I wondered why some functions in RV2 have Hive* names like in Indeo 4 and 5). Also decoder sports some artefacts related to motion vectors outside picture boundaries, maybe it will be fixed too.
  • RV3 — there was a problem with chroma drift, it’s finally fixed.
  • RV4 — there’s well-known problem caused by lack of weighted MC. I’ve finally implemented it, after some cleaning and related work on RV3/4 decoder it should appear in Libav soon.

P.S. So far there are two codecs not supported by Libav, RALF (yet another pointless lossless audio format in its own special container too, or in slightly twisted RM at least) and ClearVideo (yes, it was possible to have non-RV there, anybody remembers that fractal codec?). While RALF is unlikely to be implemented ever (I think I wrote about it once), ClearVideo might be supported eventually but don’t hold your breath on it.

RV40 is in

Tuesday, December 2nd, 2008

As you may know from other place, FFmpeg got RV40 decoder. There’s still some hope that FFmpeg will get RV30 decoder as well (it needs loop filter and squishing some bugs).

Some notes about performance:

  • PPC G4 1.42GHz — on par
  • Celeron 600MHz (inside ASUS Eee) — significantly slower (2 minutes of the same source decoded in 64 and 82 seconds by binary and native decoders respectively)

When I switch motion compensation functions from C implementations to optimised H.264 counterparts (they are slightly different so the picture quality gets worse) native decoder becomes faster than binary one by several percents on x86 and even faster on PPC. Conclusion: if you want fast decoding then submit SIMD versions of motion compensation functions.

RV: a small update

Sunday, November 23rd, 2008

Hereby I declare that my RV40 decoder changed its status from “Well, it’s better than nothing” to “Good enough”. While there are still problems with chroma and jitter in B-frames due to wrong motion vectors prediction, luma decoding is bitexact on I- and P-frames.
I hope to weed them out and have decoding enabled in FFmpeg before next year. Maybe RV30 too.

For those who ask specs on RealVideo:


I hope the message is clear enough.

RV3/4 decoders present state: stalled again

Tuesday, November 4th, 2008

I’ve been very busy with the things outside FFmpeg yet I’ve managed to do something on RV3/4 decoders too:

  • Found and fixed an old bug with quantisation for DC coefficients.
  • Cleaned a bit RV4 loop filter.
  • Fixed chroma MC bug in RV3 decoder.
  • B-frames motion vectors are now closer to the reality in RV3.

What is missing:

  • RV3 loop filter
  • correct RV3 motion vectors calculation
  • RV4 motion compensation incompatibilities

The main problem is that I don’t quite understand why it’s working in the way it works and (in some parts) how it works. Hopefully it will be clearer next time I’ll look at it.

A bit more

Friday, August 8th, 2008

With low-pass filter my AAC encoder is more or less feature-complete. Of course there’s still more room for improvements but it’s pretty fine now. I’d like to submit it for review but it depends on some parts of AAC decoder and it’s still under review :-(. So I don’t have much to do until then.

So I switched to last GSoC task and hacked again at RV40 loop filter. Well, filter invoking pattern is almost there and I’ve fixed several bugs in actual filtering code. Bit it’s not there yet. Maybe in a month it will be so if AAC encoder won’t take all my time again.

News + Extra

Sunday, August 3rd, 2008

AAC front: to compete with other encoders I have to implement low-pass filter. Benjamin suggested Butterworth filter, so I will try it next week. Hopefully that will be the last big feature to do.

RV front: looks like deblocking pattern is generated from comparing motion vectors, if the difference for subblocks is greater than 3, then edge between them is scheduled for loop filtering. Don’t expect working loop filter implementation too soon though, I still have to deal with AAC encoder and it’s more important.

Extra: I’ve finally decided to buy ASUS Eee, it was easy thing to do – there’s only one model (Eee 701 4G with Win XP installed) for about the same price of four hundred bucks (maybe $450 in greedy shops). So the first thing I did with it was installing Linux and tearing down that stupid “Designed for Windows” label (which was surprisingly easy thing to do and left no marks on laptop surface).

Now here are complains about Ubuntu Eee (I don’t have USB DVD drive and Xandros hasn’t worked from USB flash drive for me): it requires some hacking of system configuration to make it work (like shutdown properly) but that I can live with, but the braindead thing is that gcc is installed (why?) without any development header or library, so you can’t compile even “Hello, world!” program. Both of those issues are resolved, so I just need to make this toy more useful to me 🙂

Turtles All the Way

Sunday, July 20th, 2008

Just in time I though I’ve fully understood RV4 loop filter. It uses both coded block pattern and some other pattern. I thought it was CBP from the previous frame, but it turned out to be some special deblocking pattern calculated for each block in interframes after decoding that block. That calculation is easy – it just selects a set of subblocks to check, compare some values and if the difference is less than 3 then set a bit in deblock pattern. Now the only thing left is to find out is where those values come from.

Again and again on RV40 loop filter

Tuesday, July 15th, 2008

I’ve mostly understood how RV40 loop filter works.

Just not to forget main principles I document it here (this blog was created for such things after all).

  • CBPs from left, top and bottom neighbours are used in filter, and if frame type is interframe then CBPs for those blocks in that reference frame are used as well.
  • There are two actual filter types – weak and strong, both are described in H.264 drafts.
  • Edges in subblock are filtered in the next order: bottom, left, top. Top edge is filtered only for the subblocks in the first row.
  • There are many filter parameters passed: dither argument (for strong filter, depends on subblock position), two thresholds taken from ClipTable, threshold taken from alpha_tab, threshold taken from beta_tab and the same value multiplied by 3 or 4 (four is for Y plane filters in not extremely big pictures).
  • The problem was to determine what ClipTable parameters should be used, as it has an additional dimension, more on it below.

There are seven values taken from ClipTable total:

  1. ClipTable[0][current block quantiser]
  2. ClipTable[2][current block quantiser]
  3. ClipTable[2][global quantiser set in header]
  4. ClipTable[x][current block quantiser]
  5. ClipTable[x][top neighbour quantiser]
  6. ClipTable[x][left neighbour quantiser]
  7. ClipTable[x][bottom neighbour quantiser]

That x value is 2 for the intra block types and P-frame interblock with DCs coded separately, 1 otherwise.
As I understand, ClipTable[x][current block quantiser] is used by default and other valuer are used for corner cases (subblock on the side of the edge is uncoded, belongs to another macroblock or does not exist at all).

I should look at H.264 loop filter description (thanks to all who sent me the pointers to the book by Iain Richardson), it seems suspiciously similar.

Again on RV40 loop filter

Thursday, July 10th, 2008

While work on AAC encoder is slowly progressing (now it’s mostly psychoacoustics left to do and maybe HE-AAC if somebody will convince me), I’m looking at side tasks to make my life a bit more colourful.
For now those tasks are writing SSE2 optimization for Monkey’s Audio decoder (and that is the first piece of SIMD assembly I’ve ever written) and working on RV40 loop filter.
To give people false hope, it’s more understandable by now. Only one function argument is not obvious. And Dark Shikari, you were wrong – RV40 is 99,5% alike with H264 draft (not 99% you said), as loop filter is suspiciously similar to H264 one.

RV: present state

Sunday, May 18th, 2008

If you are interested in what’s going with my RV decoder from GSoC 2007 then here are your answers.

What works:

  • RV30 decoding mostly works
  • RV40 decoding mostly works
  • Pictures are quite recognizable

What needs to be resolved:

  • RV40 loop filter
  • RV30 loop filter (a bit easier)
  • RV30 motion vectors in B-frames (sometimes they are a bit jumpy)
  • RV30 chroma problems (colours are always moving to the upper left corner of the frame – incorrect rounding?)
  • RV30 slice uniting problem (some splitted slices should be united by decoder – at least I know how and when to do this)

If you want to help with loop filter then loop at
loop filter work scheme (SVG, ~128Kb) and give your proposals on how it works.
Legend (macroblock is 4×4 subblocks, no borders as they will ruin this scheme):

  • numbers at the top and left eddge – macroblock numbers
  • black lines – subblock edges where loop filtering took place
  • hex number at the top left corner of macroblock – coded block pattern, it’s red for intra types macroblocks and for P macroblocks with DC coeffs coded separately
  • blue square – coded subblock

Any suggestions (and pointers to the information about H.264 loop filtering explained clearer than in standard) are welcome.

I’d like to finish it before starting my work on AAC encoder…

BTW, you can use ffmpeg-rv.patch from soc/rv40 repository to enable RV30/40 decoding in ffmpeg.