I’m not yet ready to release it but it’s a good time to tell what I’ve been working on (and intend to keep working for a while).
There are many valid QT MOV files on discmaster that are not decoded (or recognised for that matter, probably exactly for the reason they cannot be decoded). Looking manually into them often shows some exotic content not understood by any program (not even by QuickTime itself these days): Flash, MIDI, PDF, PPT, QT version of Flash aka sprites and so on. And in order to get some useful content out of such files I’m developing nihav-movrender. In theory it should be able to do at least this:
- extract various kinds of binary data;
- decode some exotic formats;
- assemble QTVR or navigable MOV panoramas from individual frames;
- merge tracks with multiple descriptors into one.
Currently I have first two items done and the other two should be easier to deal with (unless I hit some unforeseen complication of course). Below I want to talk more about different content that MOV files in the wild were discovered to have and how my tool deals (or hopefully will deal) with them.
Binary files
MOV files can embed various binary files, in different forms too. So far I’ve encountered three formats—namely Flash, PDF and PPT—and they all been stored in different ways. Flash files have their own stream type handler and it’s single file in single frame (but there may be two or more tracks with Flash files). The only sample with PDF came as many video streams, each of type “pdf” and containing a single-page PDF file. PPT, on the other hand, came in a generic data stream of sub-type “@ppt” with document split into several frames. In either case extracting them is trivial.
Media files
Occasionally a whole multimedia file of other format gets wrapped into MOV, the main examples being DV (I’ve never seen such a file and I’m fine this way), MPEG-1 (rather common) and ASF/WMF (generated by Flip4Mac). I’m mentioning those mostly for the completeness sake as I’m not sure if I want to do anything with them.
Auxiliary streams
There are several of those, for instance there’s timecode stream (considering that usually it contains just one frame I doubt it’s of any real value). There’s Tween stream (short for “between”) which is used to provide smoother transitions for other tracks (usually sprites).
I also encountered skin streams (which seem to contain single PICT file) and Quartz Composer media, where data is represented as binary .plist (with embedded PNG images if that helps).
Sprite system
Initially I assumed it’s a simple object-based animation system (like Flash Animation) but in reality it’s an interactive system involving multiple tracks i.e. usually one stream contains sprite data for content animation and another one handles controls. And the text streams (which are also complex constructions of QT atoms) seem to contain action handlers and relate to the sprites. So I ended up simply dumping decoded or extracted sprite images plus text-based description of what goes on there.
The process of decoding the samples I collected was rather fun. Sprite images may come in various formats, including GIF, JPEG, PNG, TIFF, Macintosh PICT as well as QT-compressed (usually RLE) frames. What’s more, those PICT files may contain RLE data of various kinds (sometimes even with larger dimensions than the image itself) as well as QT-compressed data. So apparently some presentation frames are coded as a command set like “set foreground colour to blue; paint a rectangle with it; and now call QT RLE decoder to paint the text”. Currently I don’t handle this situation well, so either the rectangle or overlay text is returned but it’s still better than nothing. And there are funny situations like 1100×800 JPEG image embedded inside 640×480 PICT.
MIDI
This data is stored in QT Music Architecture format (with each event data aligned to 32-bit word for convenience), sample descriptor having instrument descriptions and packets simply containing 1KB of data, whatever actual time period it may cover. I wrote a simple converter to SMF format and the result sounds OK to me.
QTVR
This format is actually stored in two or three tracks: image data split into strips and coded as individual frames, QTVR data and (optionally) panorama data. Since looking at a sequence of scenery pieces is rather boring, I want to make my tool assemble one large panorama from them.
Oh, and before QTVR there were Navigable movies (the same thing but simpler and with panorama data stored in track user data). I’m aware of four such files, one of them being the only known file employing CD-Graphics codec.
Other
If you look into QuickTime File Format Specification (it’s like ISO-IEC 14496-12 but older and available for free), it mentions even more auxiliary stream formats like hot sport image track, hint tracks and such. And of course it mentions 3D images, which must’ve been a killer feature at the time.
That’s why I alluded to this format as both common (since its derivative called MP4 is still the major multimedia container, even MXF can’t rival it; and MOV itself was extremely widespread back in the day) and exotic—you can’t claim that most things I listed here are used in modern multimedia workflows on daily basis (beside embedding font files in Matroska of course).
And who knows, maybe this will help me discover some more formats hidden in those files. The whole trove of QuickDraw flavours alone made it worthy for me.