Defer: Resource cleanup in C with GCCs magic
oshub.org55 points by joexbayer a day ago
55 points by joexbayer a day ago
Testing with Jen's macro this was based on, and found that the always_inline was redundant under even -O1 (https://godbolt.org/z/qoh861Gch via the examples from N3488 as became the baseline for the TS for C2y, which has recently a new revision under N3687), so there's an interesting trade-off between visibly seeing the `defer` by not not-inlining within the macro under an -O0 or similar unoptimised build, since with the inlining they are unmarked in the disassembly. But, there's an interesting twist here, as "defer: the feature" is likely not going to be implemented as "defer: the macro", since compilers will have the keyword (just `defer` in TS25755, or something else that uses a header for sugared `defer`) and may see the obvious optimised rewrite as the straightforward way of implementing it in the first place (as some already have), meaning we can have the benefit of the optimised inline with the opportunity to also keep it clearly identifiable, even in unoptimised and debug builds, which would certainly be nice to have!
Then there is the proposal to add standard `defer` to C2y[0]
[0] https://thephd.dev/c2y-the-defer-technical-specification-its...
It's been implemented in GCC(in review)[1], onramp[2] and my own slimcc[3]!
[1] https://patchwork.ozlabs.org/project/gcc/list/?series=470822
A good example of adding stuff to the standard with community feedback from a preview implementation.
Jen's macro that this was based on was an implementation of his own proposal (N3434) for `defer`, which was one of a few preceding what finally became TS25755! So, yes, C2y is lined up to have "defer: the feature", but until then, we can explore "defer: the macro" (at least on GCC builds, as formulated).
Nested functions are cool, although not supported by clang.
However they rely on Trampolines: https://gcc.gnu.org/onlinedocs/gccint/Trampolines.html
And trampolines need executable stack:
> The use of trampolines requires an executable stack, which is a security risk. To avoid this problem, GCC also supports another strategy: using descriptors for nested functions. Under this model, taking the address of a nested function results in a pointer to a non-executable function descriptor object. Initializing the static chain from the descriptor is handled at indirect call sites.
So, if I understand it right, instead trampoline on executable stack, the pointer to function and data is pushed into the "descriptor", and then there is an indirect call to this. I guess better than exec stack, but still...
They only need trampolines when they access their local environment and you take their address. Without optimization a trampoline was generated whenever an address was taken, but I recently changed this in the development version of GCC to only do this when needed, so hopefully in the next released version you will not get a trampoline for many more cases. Here, there is no address being taken anyway, so you do not get a trampoline.
(and I hope we get a solution without trampolines for the remaining cases as well)
They only need an executable stack when they're not inlined.
The always_inline keyword takes care of that here.
Slightly off-topic, but:
The fact that go "lifts" the deferred statement out of the block is just another reason in the long list of reasons that go shouldn't exist.
Not only is there no protection against data-races (in a language all about multithreading), basically no static checking for safety, allocation and initialization is easy to mess up, but also defer just doesn't work as it does in C++, Rust, Zig, and any other language that implements similar semantics.
What a joke.
I'm always suspicious of exotic features that could fail in surprising ways.
Well, each `defer` proposal for C agreed that it shouldn't be done the way Go does it, and should just be "run this at the end of lexical scope", so it'll certainly be less surprising than the alternative... and far easier to implement correctly on the compiler side... and easier to read and write than the corresponding goto cleanup some rely on instead. Honestly, I feel like it becomes about as surprising as the `i++` expression in a `for` loop, since that conceptually is also moved to the end of the loop's lexical scope, to run before the next conditional check. Of course, a better way of representing and visualising the code, even if optionally, would help show where and when these statements run, but a standard feature (especially with some of the proposed safety mechanisms around jumps and other ways it could fail in surprising ways) it would hardly seem exotic, and inversely is quite likely to expose things currently fail in surprising ways precisely because we don't have a simple `defer` feature and so wrote something much more complicated and error-prone instead.
So, I completely understand the sentiment, but feel that `defer` is a feature that should hopefully move in the opposite direction, allowing us to rely on less exotic code and expose & resolve some of the surprising failure paths instead!
Small blog post exploring a defer implementation using GCC’s cleanup + nested functions, looking at the generated assembly and potential use cases.
I don't understand why people insist on simulating a poor substitute for RAII with a feature that is itself almost decent RAII.
> If malloc fails and returns NULL, the cleanup function will still be called, and there’s no simple way to add a guard inside free_ptr.
free(NULL) is a no-op, this is a non-issue. I don't know what's so hard about a single if statement anyway even if this were an issue.
> I don't understand why people insist on simulating a poor substitute for RAII with a feature that is itself almost decent RAII.
RAII doesn't make sense without initialization.
Are you proposing C should add constructors, or that C should make do without defer because it can't add constructors?
> RAII doesn't make sense without initialization.
Rust has RAII and does not have constructors.
Rust mandates that every field in a user-defined type is initialized at once. How do you propose to retrofit that into C without "constructors"?
C has had designated initializers since C99, if you want you can initialise every struct field at once.
I encourage you to read (at least) this section of this blog before making simplistic suggestions: https://thephd.dev/just-put-raii-in-c-bro-please-bro-just-on...
How do you mandate initialization, handle copies, move objects, prevent double frees? What's RAII without any of that?
poor?
If I use RAII I'd need to have a struct/class and a destructor.
If I use defer I'd just need the keyword defer and the free() code. It's a lot more lean, efficient, understandable to write out.
And with regards to code-execution timing, defer frees me from such a burden compared to if-free.
> If I use defer I'd just need the keyword defer and the free() code.
Yeah, and not accidentally forgetting to call it. That's the big part. And before "True Scotsman will always free/close/defer!" - No, no they won't.
Unless the compiler screams at them, or its enforced via syntax constructs, it will always slip through the cracks.
Well I'd have to pay all the friction of writing up a new type, and in some cases the type gets cubersome. Doubly so if your codebase requires extra some friction like 1 header for each type.
Also get over it. We got post-processor things like static analyzers, etc, and whatever AI code reminders/fixers that are coming up next. I'd prefer those over muddying up the code base.
> Also get over it. We got post-processor things like static analyzers, etc, and whatever AI code reminders/fixers that are coming up next.
Sure. But unless it's part of compiler, someone will not run it, or will run out of resources (no net or no tokens).
Defaults matter a ton.
> I don't understand why people insist on simulating a poor substitute for RAII with a feature that is itself almost decent RAII.
Because it’s nowhere near “almost decent RAII” and RAII requires a lot more machinery which makes retrofitting RAII complicated, especially in a langage like C which is both pretty conservative and not strong on types:
- RAII is attached to types, so it’s not useful until you start massively overhauling code bases e.g. to RAII FDs or pointers in C you need to wrap each of them in bespoke types attaching ownership
- without rust-style destructive moves (which has massive langage implications) every RAII value has to handle being dropped multiple times, which likely means you need C++-style copy/move hooks
- RAII potentially injects code in any scope exit, which I can’t see old C heads liking much, if you add copy/move then every function call also gets involved
- Because RAII “spreads” through wrapper types, that requires surfacing somehow to external callers
Defer is a lot less safe and “clean” than RAII, but it’s also significantly less impactful at a language level. And while I very much prefer RAII to defer for clean-slate design, I’ve absolutely come around to the idea that it’s not just undesirable but infeasible to retrofit into C (without creating an entirely new language à la C++, you might not need C++ itself but you would need a lot of changes to C’s semantics and culture both for RAII to be feasible).
https://thephd.dev/just-put-raii-in-c-bro-please-bro-just-on... has even more, mostly from the POV of backporting C++ so some items have Rust counterpoints… with the issue that they tend to require semantics changes matching Rust which is also infeasible.
Not having RAII is precisely the reason I prefer C over C++ or Rust. I WANT to be able to separate allocation from initialization.
I'm currently working with Arduino code and the API is a mess. Everything has a second set of manual constructor/destructor, which bypasses type-safety entirely. All only to shoehorn having existing, but uninitialized objects into C++.
> I WANT to be able to separate allocation from initialization.
Which hardly ever makes sense, and is possible with clean C++ anyway...
Both C++ and Rust allow that? Having niche behaviour not be the default makes sense, but both know it's needed and therefore allow it?
(C++ lets you malloc and then placement new (just casting the pointer like C does is UB, but it's being fixed for trivial types) and Rust has both plain alloc and Box<MaybeUninit<T>>)
There are a lot of other reasons not to use them, but yours is a made up strawman.
This isn't what people are talking about, you aren't understanding the problem
With RAII you need to leave everything in an initialized state unless you are being very very careful - which is why MaybeUninit is always surrounded by unsafe
{
Foo f;
}
std::vector<T> my_vector(10000);
Even if I just want a std::vector<uint8_t> to use as a buffer, I can't - I need to manually malloc with `(uint8_t)malloc(sizeof(uint8_t)*10000)` and fill that
So what if the API I'm providing needs a std::vector? well, I guess i'm eating the cost of initializing 10000 objects, pull them into cache + thrash them out just to do it all again when I memcpy into it
This is just one example of many
another one:
with raii you need copy construction, operator=, move construction, move operator=. If you have a generic T, then using `=` on T might allocate a huge amount of memory, free a huge amount of memory, or none of the above. in c++ it could execute arbitrary code
If you haven't actually used a language without RAII for an extended period of time then you just shouldn't bother commenting. RAII very clearly has its downsides, you should be able to at least reason about the tradeoffs without assuming your terrible strawman argument represents the other side of the coin accurately
`free(NULL);` will crash on some platforms that gcc supports, I believe.
Then it's in violation of the C standard, at least as of C11 (I didn't check C99 or C89).
> The free function causes the space pointed to by ptr to be deallocated, that is, made available for further allocation. If ptr is a null pointer, no action occurs. Otherwise, if the argument does not match a pointer earlier returned by a memory management function, or if the space has been deallocated by a call to free or realloc, the behavior is undefined.
Emphasis mine
It shouldn't https://pubs.opengroup.org/onlinepubs/7908799/xsh/free.html
>If ptr is a null pointer, no action occurs.
While I agree it shouldn't, that particular document is the UNIX specification, not the C specification, so it does not apply to C compilers on non-UNIX platforms.
> `free(NULL);` will crash on some platforms that gcc supports, I believe.
I'm pretty certain that `free(NULL)` is part of the C99 standard, so compiler vendors have had 25 years to address it.
If your `free(NULL)` is crashing on a certain platform, you probably have bigger problems, starting with "Compiler that hasn't been updated in 25 years".
That feels like a "citation needed", since that would be very clear violation of the C spec and thus a rather serious bug in the standard library for that platform.
can we just do `if(*ptr == NULL) return;` ?
No, because optimizing compilers are free to elide the check. https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#ind...
I'm not quite familar with this flag, but this
>so that if a pointer is checked after it has already been dereferenced, it cannot be null.
sound to me that if i've never deref the pointer anytime before(e.g the null check is at the beginning of function), the compiler won't remove this check.
Since the compiler will merge/fold what it appears to be a different logic sections of your code into a single one, you can never be sure what the release build codegen looks like unless you read the assembly.
If you check for null pointer before you dereference, then no the compiler cannot elide the check.
If you check after dereferencing it, yes it can. But in this case why would you not check before dereferencing? It's the only UB-free choice.
Yes, it can. Why would you be checking the pointer for nullptr after you have dereferenced it? It makes no sense at all, so, compiler indeed can elide the nullptr check before dereferencing the ptr exactly because it is free to _always_ assume that the program is free of UB.
To be more precise GCC says "eliminate useless checks for null pointers" and what I am saying that you can never be sure what in your code ended up being "useless check" vs "useful check" according to the GCC dataflow analysis.
Linux kernel is a famous example for disabling this code transformation because it is considered harmful. And there's nothing harmful with the nullptr check from your example.
> Yes, it can.
I don't think so. If it could, then this code would reliably crash:
char *mystr = strdup (oldstr);
if (mystr)
*mystr = 0; // Truncate string
> Why would you be checking the pointer for nullptr after you have dereferenced it? It makes no sense at all
Right. It's UB. And that's why the optimization in question is about removing that check. The only reason the optimization is valid for a C compiler to do, is that it can assume dereferencing a null pointer lands you in UB land.
I'm sorry, either you are terrible at trying to explain things, or you have thoroughly misunderstood what all this is about. GCC cannot, under any circumstances or with any flags, remove an "if (ptr == NULL)" that happens before dereferencing the pointer.
What this flag is about, and what the kernel bug you mentioned (at least I think you're referring to this one) is about, was a bug that went "int foo = ptr->some_field; […] if (ptr == NULL) { return -EINVAL; }". And GCC removed the post-deref null pointer check, thus making the bug exploitable.
From the help text:
> if a pointer is checked after it has already been dereferenced, it cannot be null.
after. Only applies after. A check before dereferencing can never be removed by the compiler.
Obviously.
Not on all platforms! If you’re writing portable code targeting a lot of embedded platforms then you don’t want to rely on this optimization.
It's a platform-agnostic optimization in case of GCC so if your embedded Linux toolchain is based on GCC, and most of them are, it's pretty much the case that it will have this optimization turned on by default.
> This option is enabled by default on most targets. On AVR and MSP430, this option is completely disabled.
Yes and if you’re targeting AVR, an extremely popular 8 bit micro, then it’ll be turned off.
If «ptr» is not a valid pointer, an attempt to dereference it (i.e. *ptr) will most assuredly crash the process with a SIGSEGV.
But when would it not be a valid pointer, and yet also not a null pointer? A null pointer we can check for easily.
A null pointer is not a valid pointer in a predominant number of systems in existence. If malloc (3) has returned a NULL, *ptr will cause a SIGSEGV.
Embedded systems are an exception, though. They may not have a MMU, and in such a case the operation will succeed.
1. No, dereferencing a null pointer will not "cause a sigsegv". It causes UB. In practice, in unix user space, yes it'll probably be SIGSEGV. 2. A null pointer is not a valid pointer: Yeah… Once again my question was "But when would it not be a valid pointer, and yet also not a null pointer? A null pointer we can check for easily."
This code will NEVER deference a null pointer. Not under any compiler, not with any compiler options:
if (ptr != NULL) { *ptr = 0; }
No, that's not quite pedantically accurate. A null pointer is not a valid pointer in the C programming language. Address zero may or may not be, that's outside the scope of the C language. Which is why embedded and kernel work sometimes has to be very careful here.
> They may not have a MMU, and in such a case the operation will succeed.
Lack of MMU does not mean address zero is valid. It definitely* doesn't make a null pointer valid. In fact, a null pointer may not point to address zero.
A zero (0, not NULL!) pointer is a valid pointer in C/C++. It is not a UB, and it means one simple thing: «give me the contents of a memory cell (a byte, a word, a long word etc) at the address of 0». Old hardware designs used the address of 0 to store a jump address of the system boot-up sequence (i.e. firmware), and I personally wrote the code in C to inspect / use it in the unpriviledged hardware mode.
The prevailing number of modern systems do not map the very first virtual (the emphasis is on virtual) memory page (the one that starts from zero) into the process address space for pragmatic reasons – an attempt to dereference a zero pointer is most assuredly a defect in the application. Therefore, an attempt to dereference a zero pointer always results in a page fault due to the zeroeth memory page not being present in the process' address space, which is always a SIGSEGV in a UNIX.
Embdedded systems that do not have a MMU will allow *ptr where «ptr» is zero to proceed happily. Some (not all) systems may even have a system specific or a device register mapped at the address being 0.
You are conflating several unrelated things, and there is no pedantry involved – it is a very simple matter with nothing else to debate.
[dead]
Just use C++, it's its main feature on top of C.
> Just use C++, it's its main feature on top of C.
If you want to and/or can, then go ahead. This is for those people who either don't want to, or can't, use C++.
Are you suggesting only use C++ over C in all situations?
> on top of C.
If we're referring to the "C is a subset of C++" / "C++ is a superset of C" idea, then this just hasn't been the case for some time now, and the two continue to diverge. It came up recently, so I'll link to a previous comment on it (https://news.ycombinator.com/item?id=45268696). I did reply to that with a few of the other current/future ways C is proposing/going to diverge even further from C++, since it's increasingly relevant to the discussion about what C2y (and beyond) will do, and how C code and C++ code will become ever more incompatible - at least at the syntactic level, presuming the C ABI contains to preserve its stability and the working groups remain cordial, as they have done, then the future is more "C & C++" rather than "C / C++", with the two still walking side-by-side... but clearly taking different steps.
If we're just talking about features C++ has that C doesn't, well, sure. RAII is the big one underpinning a lot of other C++ stuff. But C++ still can't be used in many places that C is, and part of why is baggage that features like RAII require (particularly function overloading and name mangling, even just for destructors alone)... which was carefully considered by the `defer` proposals, such as in N3488 (recently revised to N3687[0]) under section 4, or in other write-ups (including those by that proposal's author) like "Why Not Just Do Simple C++ RAII in C?"[1] and under the "But… What About C++?" section in [2]). In [0] they even directly point to "The Ideal World" (section 4.3) where both `defer` and RAII are available, since as they explain in 4.2, there are benefits to `defer` that RAII misses, and generally both have their uses that the other does not cleanly (if at all) represent! Of course, C++ does still have plenty of nice features that are sorely missing in C (personally longing for the day C gets proper namespaces), so I'm happy we always have it as an option and alternative... but, in turn, I feel the same about C. Sadly isn't as simple to "just use C++" in several domains I care about, let alone dealing with the "what dialect of C++" problem; exceptions or not, etc, etc...
[0]: https://www.open-std.org/JTC1/SC22/WG14/www/docs/n3687.htm [1]: https://thephd.dev/just-put-raii-in-c-bro-please-bro-just-on... [2]: https://thephd.dev/c2y-the-defer-technical-specification-its...
> this just hasn't been the case for some time now
Which I find sad actually. The idea of C++ as a superset of C is really powerful, especially when mixing C and C++. A while ago I had a C project (firmware for a microcontroller) and wanted to bake the version and the compilation time into the firmware. I didn't find a way to do this in plain C, but in C++ you can initialize a global struct and it gets statically linked into the output. This didn't even use constexpr, just preprocessor trickery. Then it was just a matter of renaming the c file to cpp and recompiling. I guess you could also do that with C, but there are things like RAII or constexpr or consuming a C++ library that you can't do without.
> wanted to bake the version and the compilation time into the firmware. I didn't find a way to do this in plain C, but in C++ you can initialize a global struct and it gets statically linked into the output. This didn't even use constexpr, just preprocessor trickery.
I might be misunderstanding here, but if you are okay with preprocessor trickery, then it's doable.
I do this routinely in the Makefile, which (very tediously) generates a build_info module (header and implementation) that is linked into the final binary: https://github.com/lelanthran/skeleton-c/blob/8e04bed2654dac...
> But C++ still can't be used in many places that C is
Unless we are speaking about PICs or similar old school 8 and 16 bit CPUs, with compilers like those from MIKROE, there is hardly a platform left were the vendor compiler isn't C and C++ (even if it doesn't go beyond C++11).
And if it must be deployed as freestanding, there are still enough improvements to take advantage of.
In the end it boils down to human factor in most cases, however as Dan Saks puts "If you're arguing, you're losing.", taken from
CppCon 2016: “extern c: Talking to C Programmers about C++”
> personally longing for the day C gets proper namespaces
I think in the spirit of C, this should go into the linker, not in the compiler.
Seems good, but I do not care about cleanup memory since I started to use arenas.
This is still useful for cleaning up file descriptors, unlocking mutexes, etc.
Cleanup can be very useful if you depend on a library that does not support arenas.