Soon it will be eight years since I’ve (re)started NihAV development in Rust. And for this round date I’d like to present my impressions and thoughts on the language—from the perspective of applicability in my experimental multimedia framework.
Why Rust?
I love C and would use it to write a new tool if needed, but it has two major deficiencies: lack of composability and different concepts of the language held by different parties. The former means that you start to get an increasing amount of problems if you software grows in complexity and has to manage interactions between various components, countless interfaces and data structures (mind you, it is possible to overcome it, as Linux kernel shows, but even they have to resort to various tricks and tools to keep development running). The latter means that different people and entities treat C completely differently: ordinary developers still believe in “portable assembly” concept and expect the compiled code to do what they meant; ISO committee members believe C to be a portable high-level language so anything that is not covered by their very abstract machine (e.g. adding two signed integers resulting in an overflow) is called undefined behaviour (or platform/implementation-specific behaviour) and gives compilers a license to creatively misinterpret code to do less work; and of course there are compiler writers (large companies or organisations, not small groups or individual enthusiasts) who treat C as a nuisance that has to be supported in their C++ compiler. I’ve ranted about that in my old post enough.
So, what other programming language to try? C++ is good mainly for two things: keeping developers well-employed and demonstrating how not to do things (just show me a modern and relatively popular language that formats output like iostream and uses diamond inheritance and I may reconsider my words). Java? No, thanks. I had enough experience with it to write anything for fun. And since e.g. decoding video requires some performance, interpreted languages are not an option.
My criteria were subjective but simple to understand: the language should introduce new concepts (as Principia Discordia puts it, ’tis an ill wind that blows no minds), the language should be more or less stable (i.e. not still at an experimental stage), no dick (or walrus) operator (for me it’s been a good sign that I’m not going to like the language, so far there were no false positives). This means that Go and Nim are out (maybe I need a rule of thumb about programming languages named after games?), Zig is too experimental (I also looked at it back in the day and found it not exciting and pointlessly strict on handling returned errors), D is too obscure (and I think it still had licensing problem back then), .NET is too tied to the platform I don’t care about (same with Swift).
And since Rust was more or less stable and known back then—and some of my friends had experience with it—I decided to try it.
What works in Rust
Probably I should’ve mentioned it more clearly in the previous section as it affects my language choice but here’s my wish-list for the language features (in no particular order):
- composability—I’d like to be able to structure code, hiding private functions and binding functionality to the data structure (e.g. being able to sum two motion vectors directly instead of adding each component manually, or not polluting global namespace with functions that work only on motion vectors);
- being able to write performant code (i.e. compiler should be able to perform optimisations without too many hints from my side);
- being able to perform other low-level operations (like reinterpreting value in memory as other type);
- being able to use assembly (inline or standalone);
- reasonably easy build system.
As you can see, there’s not that much to it, and in theory any language improving on C should be able to provide all of it. I don’t care about, say, async features but I care a lot about, say, inline assembly.
Let’s look what works from that list.
Composability is definitely there. Beside crates and nested modules with fine-grained visibility you can also define structures with only some fields or methods being public, you can use structure namespaces for the specific methods as well as traits for implementing common operations (like add/subtract/negate for motion vectors).
Performant and low-level code is possible to write too. Beside the normal operations (like .chunks_exact()) that should provide enough hints for the compiler, it is possible to resort to low-level tricks like allocating uninitialised memory for temporary buffer that you know will be written to or transmuting one type to another (and conversion between floating-point number and its integer representation is a part of the standard library). The functions to perform pointer magic like in C are present in the standard library as well, so it’s well-covered IMO.
Inline assembly support in Rust is even better than in C in some aspects but still lacking a couple of things to be perfect (I’m not sure that both sym and const made it to the stable Rust and there’s still no standard way to generate names e.g. when you want to generate a function working on both 8- and 16-pixel block from a macro). Standalone assembly support is much worse: you have to resort to build.rs and calling an external program (coincidentally there does not seem to be a decent assembler written in Rust either).
Build system is rather good: you have cargo with the standard package registry, you can specify custom locations for dependencies (and even patches to apply before building them) as well as custom build commands in build.rs. The main annoying thing is that there are too many crates available so if you update crate index only occasionally it takes quite a while.
Rust provides other features that I did not ask for explicitly but found useful nevertheless:
- increased safety—by having stricter type and borrow checking. Sometimes it creates inconveniences when I have to split a structure into smaller parts in order to be able to use the associated functions, but at least I can be sure that I can’t accidentally modify data I’m currently working with by calling such function. And
unsafe{}keyword is used to mark idiots and disingenuous people: if somebody complains that Rust is not safe because such keyword exists, you know with whom you’re dealing; - lifetime rules free you from memory management without introducing all the drawbacks of automatic garbage collection. Of course they require some learning but if you’re not willing to learn the base concepts of the language then you don’t have to complain about it at all;
- simple but convenient error handling. Maybe you like the way Go does it (or how it’s virtually impossible to ignore returned result in Zig) but I appreciate
()?andResult<T,E>; - macro system is nice too even if it has some serious deficiencies worth mentioning in the following section;
- language versioning—you have editions of the language so you can leave your crate on, say, 2018 edition without worrying that the compiler will start interpreting code according to the rules of 2024 edition;
- built-in functionality for unit tests;
- rather helpful compiler error messages and linter (even if I not always agree with the latter).
What does not work
…or at least does not work that great.
Zeroeth of all, the suspicious situation around Rust Foundation—scandals, maintainers resigning and such. It has not affected me in any way and if you look at the rest of the world it’s nothing special either. But you should be ready for some breach of trust and/or crazy demands on par with USian government (thankfully there are alternatives).
First of all, language specification. Back in the day I wrote a post why Rust is not a mature programming language and the lack of specification was the main point. Of course you can argue that Rust recently adopted language specification, but if you take a second to learn more about it you’ll find out that it comes essentially from outside. There’s a company called Ferrocene that specialises on making Rust usable for more serious uses (i.e. where you need to pass certification because it is recognised that multi-million assets and/or human lives may be lost on your software malfunctioning). So Ferrocene had to work on the language specification in order to make it acceptable for certifiable use. Meanwhile Rust Foundation at first ignored it, then made a token gesture of starting a process of producing language specification and ended up simply adopting what Ferrocene has produced. It’s definitely a good outcome but it leaves a strange aftertaste…
Then, the whole situation with traits. They provide a lot of useful functionality but somehow stop halfway. I mean the situation with automatic selection of proper type. For instance, arr[42u8.into()] depending on compiler version and phase of the moon can work, fail to compile or fail to compile with completely cryptic message (yes, I’ve experienced all three on the variations of the same code). I understand why this is hard and accept it, nevertheless it’s annoying (I keep hearing for the last several years that Polonius should solve that eventually). Even more annoying is that I have to duplicate the iterator code: as I often work with frames that may be flipped bottom-up I want to write code like
let mut iterator = if !flipped {
data.chunks_mut(stride)
} else {
data.chunks_mut(stride).rev()
};
for line in iterator { ... }
but I can’t since objects have different type (and probably size) and there’s no realistic way to coerce them to the same interface.
And finally, proc macros. I understand what problem they’re solving, how useful they are and why other zero compilation cost abstractions can’t replace them. But I also understand that they slow compilation time significantly and feed paranoia of any security specialist (because you end up building a binary—or downloading a precompiled one, like Serde did for some time—that runs on your machine modifying your code and theoretically doing anything else. This does not affect me directly but it does not look nice either.
Conclusion
Overall, Rust was a good choice for my project (but that does not mean it’ll necessarily be for yours: screwdriver can be used to hammer nails and carve wood but a specialised tool is still better for such cases). Of course there are things that did not work for me well, but others did and I’m satisfied with the results.
Probably the main factor is that I approached the language with the mindset of “let’s see what it can offer and if I can use it to build what I want”. It turned out I can. Many people seem to treat it as a replacement for their current language and get burnt when it turns out that Rust (surprise!) does not map one-to-one in the concepts.
As for me, I learned to accept that some things should be done differently (like iterators instead of traditional loops), some things have different meaning (e.g. let operator is not merely for variable declaration, it can also do de-structuring and that’s why if let makes sense) and so on. I appreciate the new concepts I learned here as it enlarged my understanding of programming and (hopefully) made me a better programmer.
Hopefully there will be another interesting language to learn (or maybe it’s already there and I’m simply not aware of it). Without such languages I’d probably use something more akin to C3 and my life would be more boring.