In my last post I mentioned I need to deal with MP3 in AVI and multi-threaded decoding. The former turned out to be a simple bug (I should’ve not trusted AVI header reporting 12-bit audio), and I gave up on the latter.
The main reason for that is what seems to be the main contribution of MPEG to the world of video coding, namely B-frames. While the idea behind them is reasonable (to code scene transitions or smooth movements as an interpolation between two keyframes), practical implementation brings headaches because those frames are coded in an order different from the display order (after all, you can’t interpolate between two frames if you haven’t decoded both of them). And of course it got worse in H.264 and later codecs where B-frames can reference other B-frames so you need to code information about the frame structure (references and how to update them).
And the problem with MPEG-4 ASP is that while it can have B-frames, its popularity it tied more to AVI container which lacks means to signal frame reordering (fun fact: the MPEG-4 ASP video files in MOV that I have would be perfect candidates for B-frames but lack them entirely). Of course later there other containers gaining popularity like Matroska or OGM (or even MP4 occasionally) but the gilded age seems to be tied to AVI. And of course that created difficulties.
If you have I- and P-frames only, there’s nothing to care about—but multi-threading won’t be that effective either. Newer implementations (Xvid 1.3.7 is rather fresh BTW) output B-frames as is so good luck knowing that in advance and performing reorder. In this case I see if the coded timebase is the same as the one reported by the container and simply re-assign timestamps from the bitstream (and if this does not work—well, tough luck). But there was a funnier intermediate solution with one frame containing data for both P- and B-frame and the following frame being a skip frame, so a decoder could replace it with an appropriate frame. This reminds of Indeo 4 which performed the same trick. And making that a multi-threaded decoding would be a mess requiring either saving frame data and scheduling it for later decoding or scheduling both frames and then trying to tie it to the upcoming frame decoding request. And playing back typical video takes about 20% of CPU load…
Thus I’ve committed what I find to be good enough for my needs and I shall forget about it—at least until some decoding artefact will annoy me enough. There’s more boring and unremarkable stuff I want to do on NihAV, working on this decoder reminded me that it can always be worse (or uglier).
P.S. For some reason repository cloning or updating from git.nihav.org still does not work (but the web interface is fine). I’ve reported the problem and hopefully it will be resolved soon. I suspect that the provider blocked it because of too many synchronisation requests from other sites trying to mirror the repositories. In either case I’m still grateful for the hosting.