Tracking issue for alloc_system/alloc_jemalloc

[alexcrichton]

Introduced in RFC 1183 these two crates lie underneath the facade with their tracking issue hooked up to #27783. I'd like to open a separate issue for just these two crates.

We have a request in Gecko to stabilize the ability to route allocations to specifically malloc (on all platforms). This includes Windows, where currently alloc_system routes to functions like HeapAlloc.

Essentially, one of these crates, or perhaps a new crate like them, may want to have an accelerated path to stabilization before allocators themselves. For example I wouldn't want to stabilize #![allocator] or #![needs_allocator], but we could stabilize the semantics of extern crate alloc_malloc redirects all allocations to malloc (for example).

Nominating for discussion!

cc @rust-lang/libs
cc @froydnj

[Aatch]

Whatever the behaviour of alloc_malloc is, it'll impact the behaviour of the other allocator traits, though this probably isn't a worry in practice, just need to make sure we don't stabilize ourselves into a corner.

I think a malloc allocator crate is useful and general enough that we could probably make it a compiler switch of some sort, anything that supports Rust should have a functioning malloc available.

[sfackler]

Just to be clear, the only thing that would be stabilized here would be the fact that a crate named alloc_malloc exists and causes allocation calls to hit malloc/realloc/free, and it would have no stable items?

[aturon]

cc @nikomatsakis @pnkfelix

I think it's reasonable to fast-track stabilization of some way of getting pure malloc-based allocations. I'm slightly uneasy with extern crate having these kind of behind-the-scenes effects, but I also don't immediately see a better mechanism.

[alexcrichton]

@Aatch, @sfackler

To be clear, I'm thinking we could stabilize only the name extern crate alloc_malloc and no other APIs. Semantically you would essentially be guaranteed that if you link to that crate then all allocations will be implemented via malloc and friends (standard C APIs, not Windows-specific ones where possible).

Also to be clear, it is not clear to me that this is actually what Gecko wants and that it would suffice for them.

@aturon

I'm slightly uneasy with extern crate having these kind of behind-the-scenes effects

Yeah we've always had a nice property that you can basically concatenate any crates together, but unfortunately allocators and panic runtimes erodes this down a little more :(

[nikomatsakis]

@aturon

I'm slightly uneasy with extern crate having these kind of behind-the-scenes effects, but I also don't immediately see a better mechanism.

It seems reasonable to have the idea of "global language services" that can be required and which can admit only one implementation. I wouldn't want this to be an expandable, user-defined set, since it's kind of anti-modular (unlike say a trait), but we do need SOME answer that permits someone to say "gimme memory" or "panic" without having to have the implementor in scope. Tying it to very special crates seems... ok to me. At least I too cannot directly think of an alternative.

I guess I could imagine that merely adding the crate as an extern crate isn't enough, the app or somebody has to "bless it" for it to serve in its special role as a provider of that mechanism. But I'm not convincced that's valuable and it requires us to think up how it gets declared.

In any case, total composability of crates isn't really possible. For example, we already have the notion of #[no_mangle] functions and so forth. That is more-or-less the same thing, no?

[sfackler]

To make things a bit more explicit, we could take a similar approach as to what we do now for plugins:

#![allocator(alloc_malloc)]

That approach would also be nice in that you could have one allocator crate that calls into another and e.g. adds some instrumentation or whatever.

[froydnj]

Also to be clear, it is not clear to me that this is actually what Gecko wants and that it would suffice for them.

The proposed changes here would work for us in the short term. We'd still probably write our own allocator when that API was stabilized so we could take advantage of posix_memalign etc. being available on all our platforms, but this would fill in the gaps nicely.

[alexcrichton]

Talking with @aturon offline, some thoughts we have are:

• Something like #![allocator] here seems plausible. This would likely want to be an RFC amendment with the semantics:
  • Final artifacts can mention #![allocator], and they will use that allocator (e.g. no rlibs)
  • We whitelist maybe only a handful of names for now. I'd probably go for system and malloc, where system is what alloc_system does today and malloc is "use malloc everywhere, including windows".
• We can likely fast-track this for stabilization, but it won't be available before 1.11 (August 10, 2016).
• We'll likely deprecate this once it's stable to write your own custom allocator as that's a strict superset of the functionality (and most of the rest of the Rust community probably doesn't want the malloc crate, just the system crate).
• Is this absolutely necessary for Gecko just yet? This should only matter once Rust has a significant memory footprint in Gecko in theory?

[sfackler]

To be clear, the thing to deprecated would be #[allocator(malloc)], not the entire #[allocator] API, right??

[alexcrichton]

Indeed!

[froydnj]

Is this absolutely necessary for Gecko just yet? This should only matter once Rust has a significant memory footprint in Gecko in theory?

This is not absolutely necessary for Gecko currently; as you say, the amount of Rust code is rather small and doesn't allocate much memory anyway. But at some point, this will start to become a problem.

I guess it's a question of how soon custom allocator support is likely to stabilize, versus providing this stopgap, with some considerations to how much Rust code is going to get integrated into Gecko in the intervening timeframe. (I'm assuming Gecko can only use stable Rust, and not something like the beta releases, which may not be a great assumption.) Is that stabilization liable to be sometime in 2016, or is it more likely coming in 2017?

[alexcrichton]

The earliest this can be stable (mentioned above) is August 2016, and I'm relatively confident we can shoot for that date.

[froydnj]

The earliest this can be stable (mentioned above) is August 2016, and I'm relatively confident we can shoot for that date.

To be clear, I was asking about full-blown custom allocator stabilization, not merely an alloc_malloc crate. Does that change the answer any?

[alexcrichton]

Aha yes indeed! I suspect that would be 2017 at the earliest, but even that would be generous most likely as it's not a super high priority just yet. We can always shift some priorities, however, to move it up.

Does that mean, though, that an allocator which always calls malloc isn't what Gecko wants in the long run either?

[aturon]

@froydnj By "full-blown custom allocator", do you mean:

• Changing the global/default allocator
• Configuring allocators per data structure

[froydnj]

Does that mean, though, that an allocator which always calls malloc isn't what Gecko wants in the long run either?

It is what Gecko wants; my thought on asking when custom allocators were going to be ready was whether they'd be ready soon enough that they'd converge with a raft of Rust code being added to Gecko. In that case, we could just deal with the alloc_system crate and not have to add alloc_malloc, just to have it deprecated later.

It is still possible that a raft of Rust code and custom allocators being ready would converge, even with a 2017 schedule, though! I'm not aware of what timeframe everybody wants their Rust code landing in Gecko. I should go find out.

@aturon I meant changing the global/default allocator. I'm not familiar with the per-data structure allocator work; folks mentioned it in #29802 for possibly providing fallible operations on containers, but it's not clear to me whether the per-data structure allocator work addresses the concerns in #29802.

[alexcrichton]

Ok, thanks for the clarification @froydnj!

[froydnj]

cc @nnethercote

[nnethercote]

I'm late to the party here and I might be missing some context and details. But it might be helpful to stake out a maximalist position in terms of Firefox requirements, independent of specific Rust mechanisms.

• I want lots of Rust code in Firefox as soon as possible. So I'd like to minimize any tolerance for shortcomings that arise from the fact that there is little Rust code in Firefox right now.
• I want C++ code in Firefox and Rust code in Firefox to use the same global heap allocator.
• I want that global heap allocator to be Firefox's version of jemalloc, be it mozjemalloc (Firefox's old and much hacked-up fork of jemalloc which is still the default) or jemalloc4 (whatever version we currently have in Firefox, which can be enabled when Firefox is configured).
• I want Firefox's "replace-malloc" functionality -- which lets us write heap analysis tools that wrap the heap allocator -- to keep working. This is required for DMD (https://developer.mozilla.org/en-US/docs/Mozilla/Performance/DMD) to keep working. replace-malloc involves some hairy stuff.
• I want all of the above on all Tier 1 platforms, and all platforms treated equally.
• I want to be able to compile Firefox with a stable version of rustc.

As I said, it's a maximalist position :) I understand that hitting all these requirements may be challenging.

cc @glandium in case he has anything to add.

[alexcrichton]

@nnethercote just as another data point, how many of your requirements are also true for third-party components that Gecko includes? I'm just curious in the sense of figuring out how different the requirements are from "Rust is some extra stuff linked in" vs "Rust is the exact same as C++".

The basic gist is that with custom global allocators we can hit all your requirements except for the stable version of rustc (which seems like a pretty good requirement!). I'm not sure if the "route everything to malloc on all platforms" style allocation hits your requirements, though.

[nnethercote]

@nnethercote just as another data point, how many of your requirements are also true for third-party components that Gecko includes?

As far as I know all existing third-party components are C or C++ and just use malloc or new. So they end up calling into mozjemalloc thanks to our mozjemalloc-is-the-heap-allocator setup. I don't know of any that implement their own heap allocator, except perhaps as a layer on top of malloc.

I'm not sure if the "route everything to malloc on all platforms" style allocation hits your requirements, though.

It sounds like it would, but I know the devil is in the details with this stuff. The tricks we have to play to get mozjemalloc as the heap allocator are non-trivial, and vary from platform to platform. @glandium is the expert there.

[alexcrichton]

Ok! It'd be helpful for me at least to learn some of these details about how malloc/new are wired up to mozjemalloc. It may also be helpful for understanding whether "route to malloc" will be sufficient or not. I'll see what @glandium has to say though :)

[glandium]

I think I wrote about this in some other related rust issue. Anyways, from the perspective of linking static libraries into gecko and getting them to use mozjemalloc like everything else, it's only a matter of the static libraries having (in ELF-speak) undefined symbol references to malloc, free, calloc, realloc. That's valid on all platforms Gecko builds on.

Now, there's another concern from Gecko perspective: malloc is fallible, and most allocations in Gecko are infallible. Being that if they fail, we MOZ_CRASH(). For C++, that happens because we have an inline definition of operator new that uses moz_xmalloc, which does malloc and MOZ_CRASH() if malloc returns NULL. (C++ code can also opt-in to fallible allocation with new (fallible) Type())

So, in practice, we have malloc, free, calloc, realloc, moz_xmalloc, moz_xcalloc, moz_xrealloc.

I'd expect most rust code in Gecko would want the infallible-malloc type of behavior. So this raises the question: what does rust code do when malloc/calloc/realloc returns NULL to it? And the corollary: what can rust code do to gracefully handle memory allocation failure?

[alexcrichton]

Ok, thanks for the explanation @glandium! Out of curiosity, could the same treatment apply to HeapAlloc, HeapFree, and friends? I'm not sure if that'd wreak havoc with other libraries, however.

what does rust code do when malloc/calloc/realloc returns NULL to it?

For us any situation where an allocation returns NULL will end up calling alloc::oom which has an unstable method to configure what happens on OOM. The default handler configured by the standard library will simply print "out of memory" and then trigger an abort via a ud2 instruction on Unix and on Windows.

If it's necessary to route that to something else we can also look into stabilizing a method to handle OOM (basically something with this signature, although it'd likely be unsafe. Is that also a requirement for Gecko?

what can rust code do to gracefully handle memory allocation failure?

Currently, in stable Rust, nothing. All collections and pointers in the standard library end up calling that oom routine if a memory allocation fails. We plan on stabilizing the ability to handle this eventually, but that's pretty far off.

[glandium]

Ok, thanks for the explanation @glandium! Out of curiosity, could the same treatment apply to HeapAlloc, HeapFree, and friends? I'm not sure if that'd wreak havoc with other libraries, however.

We could, and in fact, maybe we should... last time I looked there are a couple third-party things in Gecko that end up using HeadAlloc/HeapFree...

For us any situation where an allocation returns NULL will end up calling alloc::oom which has an unstable method to configure what happens on OOM. The default handler configured by the standard library will simply print "out of memory" and then trigger an abort via a ud2 instruction on Unix and on Windows.

Whatever we end up doing for rust "exceptions" (was that called panic? I don't remember), we may want to have kind of the same for OOM. At the very least, we need to ensure this triggers the crash reporter like OOM in Gecko.

All collections and pointers in the standard library end up calling that oom routine if a memory allocation fails.

I haven't touched rust in a long while, but what does this mean for code that would, for example, try to allocate a multi-hundred megabytes buffer to decode a large image or something? Is there any way it can gracefully fail on its own or would allocating that buffer call the oom routine in the first place because it's handled by something in the standard library that does?

[jonas-schievink]

@nnethercote Yes, it's RSS. I have no idea how to interpret the maps, but I put them in a gist in case they're useful for you.

I though that higher RAM usage is a pretty well-known problem with jemalloc tbh :P

[glandium]

The contents of /proc/self/smaps would be more useful.

[jonas-schievink]

@glandium Here you go!

[glandium]

I'm going to go out on a limb here, but I guess your application is doing large allocations. The system allocator is likely unmapping them when they're deallocated, while jemalloc keeps them around for later reuse. It also doesn't necessarily clean them up straight away.
You should probably take a look at the maximum RSS during the process lifetime (/usr/bin/time -v yourapp will give you that) in both cases, and compare that. I suspect the difference will be less important in that case.
You can also tweak jemalloc's behavior for various things, through the MALLOC_CONF environment variable. Try lg_dirty_mult:10 to maybe trigger more memory purging. Jemalloc also keeps a thread cache, which can take some space, so you can also try tcache:false. You can combine the values with a comma (e.g. MALLOC_CONF=lg_dirty_mult:10,tcache:false).
Finally, you can get some stats about your allocations with print_stats:true. (btw, a gist with the output from print_stats:true would be interesting to see)

[jonas-schievink]

@glandium Thanks, that's quite helpful. For now it's much easier to just use alloc_system (I already have to use some unstable features), so I might revisit this later.

[est31]

It seems that the tor project would really like to be able to use their own allocators in stable Rust: https://www.reddit.com/r/rust/comments/62o9rx/tordev_tor_in_a_safer_languagerust_network_team/dfovrkx/

[sfackler]

rust-lang/rfcs#1974

[alexcrichton]

We're unlikely to really do anything to fix this, so I'm going to close this in favor of #27389. It's highly likely that all programs will start to link to jemalloc the system allocator by default once we stabilize that feature.