Tracking issue for PR 40537 (hint_core_should_pause)

[mstewartgallus]

Tracking #40537 (comment)

[clarfonthey]

Reposting my comment from #41207: the spin crate calls this function cpu_relax; maybe we should do something similar?

I'd much prefer spin_loop_hint or something similar to the current function name. We could even potentially have a warning that suggests using this function, or even better, have the compiler automatically add these instructions inside busy loops.

[ghost]

I like the spin_loop_hint name, but would be okay with spin_wait as well.
Java calls this onSpinWait, in .NET it's SpinWait, and Linux kernel has cpu_relax.

Would it be a good time to propose stabilization after some name bikeshedding?
I've wanted this function in stable Rust for quite a while. :)

[alexcrichton]

@rfcbot fcp merge

Sounds like a great issue to consider for stabilization!

Proposed name

spin_loop_hint

[rfcbot]

Team member @alexcrichton has proposed to merge this. The next step is review by the rest of the tagged teams:

• @BurntSushi
• @Kimundi
• @alexcrichton
• @aturon
• @dtolnay
• @sfackler

No concerns currently listed.

Once these reviewers reach consensus, this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[alexcrichton]

To be clear I also don't have much of a preference on names, @stjepang's preference of spin_loop_hint also seems fine by me

[sfackler]

spin_loop_hint or spin_wait both seem reasonable.

[dtolnay]

I am on board with stabilizing a function with this behavior. But in general, the more a function's behavior may be confusing, the more careful we need to be in selecting a name.

• 👎 spin_wait because the spin loop code would read as though this call internally does a spin wait, rather than identifies the spin wait containing the call.
• 👎 cpu_relax because the word "relax" is already claimed in the context of atomics for Ordering::Relaxed.
• 👎 hint_core_should_pause because "pause" means do nothing for a while, where doing something else for a while may be more what you want.
• 👍 spin_loop_hint which is what the Intel® 64 and IA-32 Architectures Software Developer’s Manual names the pause instruction.

I also sort of like the naming that WebKit uses -- pause is called YIELD_PROCESSOR and thread::yield_now is called YIELD_THREAD. This frames them as a pair of lower/higher level instructions. Yield processor is a hint to the processor, and yield thread is a hint to the OS's scheduler. Go uses the same naming -- pause is called runtime.procyield. So I'll add another name:

• 👍 yield_processor.

[clarfonthey]

👍 to spin_loop_hint

[dtolnay]

Looking at this again, I prefer yield_processor over spin_loop_hint. It seems like both would need to be documented equally carefully, but yield_processor results in code that is more pleasant to read. Unless yield_processor mischaracterizes how people conceptualize this instruction, I would go with that one. Also, precedent in Go and WebKit is encouraging.

It could also work as std::sync::atomic::yield. Is everything that runs Rust a "processor"?

[petrochenkov]

Is everything that runs Rust a "processor"?

"Processor" sounds kinda wrong in this context IMO, since the word is so often used for the whole package with multiple cores, caches, etc, while the instruction applies to a "core" or "thread".

[SimonSapin]

std::sync::yield or std::thread::yield sounds fine to me. (It’s not really about atomic instructions, more about thread scheduling.)

[kennytm]

Isn't yield a keyword?

[SimonSapin]

Oh, right.

[clarfonthey]

I think that we should be cognisant of the case where someone imports both the processor and the thread yield functions at the same time. How would someone differentiate these functions?

Additionally, what this function does isn't quite a "yield;" on x86 specifically, it simply delays the execution of the next instruction by a small amount to avoid a memory order violation, reducing power consumption but not really reducing execution time. It's less the program "yielding" to the CPU and more the program telling the CPU "I'm just going to loop again, so, take your time."

Additionally, the thread version of a yield will definitely yield, whereas this will simply do nothing on architectures which have no such function. It makes a lot more sense to call this a hint than a yield.

[clarfonthey]

Also this is a general question, but how reasonable would it be for the compiler itself to notice these cases and simply insert the instruction into busy loops? For example, something like loop {} should also contain this instruction. I wonder if LLVM has an optimisation like that.

[ghost]

@clarcharr

Also this is a general question, but how reasonable would it be for the compiler itself to notice these cases and simply insert the instruction into busy loops?

I don't think the compiler can automatically decide when and where to insert the instruction into a loop. This is something only the programmer knows, really.

Inserting the instruction into a loop is a very delicate matter. Consider how parking_lot does it:

if self.counter <= 10 {
    cpu_relax(4 << self.counter);
} else {
    thread_yield();
}

Here, cpu_relax(t) will execute the instruction t times. Each successive step of the loop the instruction is executed twice as many times. There's a lot going on in that snippet - and is something that's very hard for the compiler to do correctly.

There's no recipe for how to properly use the instruction. You just have to carefully tweak the parameters, profile the program, and see what works best. There's no other way. :)

[Ixrec]

I'm not familiar with this function, but after spending a few minutes reading up on it, I believe spin_loop_hint is the only name suggested so far that would not be misleading to me. All of the others contain a word like yield, pause, wait, relax, etc that implies significantly more impact on execution semantics than a mere hint to the CPU.

[main--]

From the Intel Optimization Manual, section 8.4.7:

The PAUSE instruction is typically used with software threads executing on two logical processors located in the same processor core, waiting for a lock to be released. Such short wait loops tend to last between tens and a few hundreds of cycles, so performance-wise it is more beneficial to wait while occupying the CPU than yielding to the OS. When the wait loop is expected to last for thousands of cycles or more, it is preferable to yield to the operating system by calling one of the OS synchronization API functions, such as WaitForSingleObject on Windows* OS.

The PAUSE instruction is intended to:

• Temporarily provide the sibling logical processor (ready to make forward progress exiting the spin loop) with competitively shared hardware resources. [...]
• Save power consumed by the processor core compared to executing equivalent spin loop instruction sequence [...]

The latency of PAUSE instruction in prior generation microarchitecture is about 10 cycles, whereas on Skylake microarchitecture it has been extended to as many as 140 cycles.

The increased latency (allowing more effective utilization of competitively-shared microarchitectural resources to the logical processor ready to make forward progress) has a small positive performance impact of 1-2% on highly threaded applications. [...]

As the PAUSE latency has been increased significantly, workloads that are sensitive to PAUSE latency will suffer some performance loss.

The instruction's original function (hint to avoid memory order violation) has been extended quite a bit. So yes, it does yield resources.

[alexcrichton]

I'm going to go ahead an check @BurntSushi's checkbox as it's been awhile and I think he's quite busy!

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period is now complete.

[bstrie]

I marginally prefer spin_loop_hint over yield_processor, if only because it has zero chance of confusion with, er, whatever the yield keyword ends up doing.

[ghost]

In #43751, there has been some talk of stabilizing module std::intrinsics or introducing std::hints. I wonder if such a module would be a better fit for spin_loop_hint than std::sync::atomic - what does everyone think?

(just a reminder: this feature is currently in beta, scheduled for stabilization in 1.24)

[ghost]

This feature was stabilized. Let's close the issue.