After having several attempts on trying to add hardware-accelerated decoding support for NihAV I’m giving up, the reason being the sorry state of it in general.
I’m aware of two major APIs for hardware-accelerated video decoding for Linux, those are VDPAU and VA-API. Plus there are some specific toolkits e.g. from Intel but from what I remember those are even more complicated.
So, VDPAU has only bare-bone documentation without actual explanation what is expected for each codec in order to decode it. VA-API turned out to be even worse: it points out to 01.org for documentation which no longer exists (and redirects to some Intel’s page blurbing how great they are at open source). And web.archive.org shows that that page essentially contained a link to libva and libva-utils repositories plus some references to the projects that have VA-API support implemented. “…so shut up and go away” was not written but implied.
At least VA-API has three crates implementing its bindings in Rust and not just one not updated in four years like VDPAU but how usable are those? There’s FeV that seems to support JPEG decoding only (and has a strict warning against AMD GPUs), there’s libva-sys that is a pile of auto-generated bindings and there’s cros-libva. The latter seems to be the cleanest one and most actively developed (too actively developed to my taste as it changes base APIs every couple of months). Unfortunately it’s still not exactly clear how to use it for H.264 decoding (and the cros-codecs crate provides equally confusing API). And the final straw is that it seems to be intended for single-thread use only by design, which means it’s not possible to use with my library (e.g. my player uses separate threads for audio and video decoding, so I can’t use the standard decoder interface for hardware-accelerated decoding without some ugly hacks).
Oh well, I’ll work on improving my own H.264 decoder performance—while it’s not much fun either at least it’s clear what I can do with it and how it can be done.
P.S. This reminds me of the situation with ALSA. From what I heard it’s one of the worst documented subsystems in Linux with too flexible interface, to the point that it took help from ALSA developers to make at least MPlayer support ALSA output. The most probable reason is that it’s common to smoke weed in Czechia (where ALSA was developed), but what is the excuse for the other libraries?
Hi! Just wondering about H.264: Is your stance on the myriad of corresponding patents, and the risks of their infringement, basically an “IDGAF :shrug emoji:”?
Essentially yes. You can’t make sure none of them is infringed so why bother? Also I’ve used the original version of the standard from 2003 so the original patents should be expired anyway.
Got it, thanks! I get your point. I wish lawyers, and soulless, greedy corporate entities also did… :/
I’m sure you are aware, but just for the sake of spreading and recording (questionably) useful info, there’s this handy collection of all the (sometimes many times extended) patents _they_ consider relevant for H.264:
https://www.mpegla.com/wp-content/uploads/avc-att1.pdf
And the same for H.265:
https://www.mpegla.com/wp-content/uploads/hevc-att1.pdf
From there, use G**gle Patents (or similar) to look up details of each patent.
Thanks for the reference. I’ve briefly looked at some of them and out of four random patents: one expires this month (despite not being marked as such), one concerns film grain (not a thing I remember seeing in H.264), another one concerns tiling (again, more of H.265 thing) and the last one is about computing direct vectors without division by zero (moo!).
It also reminds of AAC, that’s a constantly moving target so while the patents for baseline AAC and even HE-AAC should be expired, they keep piling up patents for the newer technologies in tangentially related things (like xHE-AAC) and are still licensing it as a whole. As you can guess, I’m not going implement those newer bits or care about them and their patents.
[…] mentioned before couple of times that NihAV has its own functioning H.264 decoder. And after my failed attempts to use hardware accelerated decoding instead, I spend some time trying to optimise it but eventually gave up. On one hand it’s […]
How about Vulkan video decoding? Rust has good Vulkan crates and it supports hardware decoding on all platforms. Supports h264 & 265 for now, with more to come:
VK_EXT_video_decode_h264
VK_EXT_video_encode_h264
VK_EXT_video_decode_h265
VK_EXT_video_encode_h265
That’s a good idea, thanks! I’ll give it a try to see how it works with my hardware.
You can check your hardware support by running these samples i guess:
https://github.com/nvpro-samples/vk_video_samples
Also there’s a good introduction presentation:
https://vulkan.org/user/pages/09.events/vulkanised-2023/vulkanised_2023_vulkan_video_core_apis.pdf
This information is way better than VA-API (the only weird thing is that the video decoding example has more semaphores than Neckartalbahn). I’ve tried
vulkanocrate to list capabilities at it was surprisingly easy.Now the bad news—looks like my hardware is too old. My main laptop from 2010 seems to be out of question entirely and the other one, while having a chance of being supported (it has 7th gen Core i5 CPU), in reality lacks support as well. Probably it can be resolved with upgrading Mesa to the latest version but it’ll likely involve whole OS upgrade so I’ll have to wait for a better occasion.
Which is a shame, Vulkan-based decoding definitely looks better documented and somewhat easier to use.