I got free Monday so I decided to spend some time on any small codec and took VMware codec used by VMware emulator to capture screen output. As Alex discovered, this is merely recorded session of RFB protocol (used in VNC applications) but slightly mangled.
My further investigation show that bytes 1-2 in big-endian order contain number of rectangles updated and if that value is equal to 8 then the first chunk is ServerInitialization struct with marker ‘WMVi’ inserted before pixel format data. Otherwise frame may contain one or several FramebufferUpdate messages with standard HexTile encoding.
I’ve committed decoder to FFmpeg and I hope to find more samples and clarify some details about it (there are seven markers from ‘WMVd’ to ‘WMVh’ but what they mark is known only for ‘WMVi’, palette mode is not tested too).
OOPS, I’ve just discovered three more samples at MPHQ and they are not decoded correctly, so I have some more work to do.
Well this was one way of taking a break from the VC-1 task 🙂 Great job.