NihAV — A New Approach to Multimedia Pt. 8

Demuxers

First of all, as I’ve mentioned in the beginning, codecs should have string IDs. Of course if codec tag is present it can be passed too. Or stream handler in AVI, it’s just optional. This way AVI demuxer can report codec {NIH_TYPE_VIDEO, "go2meeting", "G2M6"} or {NIH_TYPE_VIDEO, "unknown", 'COL0"} and an external program can guess what the codec is that and handle it specially.

Second, demuxers should return two types of data — packets and streams. E.g. MPEG-TS (the best container ever by popular vote) does not care about frame boundaries, so it should not try to be smart and return a stream that can be fed to a parser and that parser will produce proper packets.

Third, parsers. There are two kinds of them — ones that split stream into frames and ones that parse frame data to set some properties to the packet. They should be two separate entities and invoked differently, one after another if needed.

Something similar for muxers — everybody knows that one can mux absolutely any codec into AVI. Or put H.264 into MPEG-PS (hi Luca!). Muxers just should allow callers to do that when possible instead of failing because codec is unrecognised.

P.S. If I’m ever to implement this all it will take a lot of time and Trocadero.

This entry was posted on Monday, May 11th, 2015 at 12:28 pm and is filed under NihAV. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

One Response to “NihAV — A New Approach to Multimedia Pt. 8”

  1. Luca Barbato says:
    May 11, 2015 at 2:46 pm

    o/

    On the user-API side I’m sure that most people is happy to just get a nice packet that the decoder could just use directly to decode to some data.

    That obviously requires to have on the low level api separated framers and parsers and (for the joy of many) bitstream converters (Hi h264 annexb vs mp4!).

    The fun gets even funnier when you mux since have to juggle stuff around so the right chunk of data appears at the right position according to whichever lovely idea somebody had regarding the minimum client-side buffer.

    So you take the data in proper packets fresh out of the encoder (or the splitter, parser and bitstream converter) and shove it down different layers of lovely madness.

    Will be fun but indeed you’ll need Trocadero and I’ll need enough Cola from Baladin or some mint and liquorice drink you hadn’t tried yet.