Re-think CpuBoundWork implementation and usage

[Al2Klimov]

Is your feature request related to a problem? Please describe.

With the new network stack we start CORES x 2 I/O threads, for accept(2), SSL handshake, ... and for actual messages processing. Especially the latter may consume quite some CPU and time. So we've scoped such with CpuBoundWork "semaphores", so that those happen only in (CORES x 2) x 0.75 threads in parallel. This way there are always some threads available for... "I/O stuff".

CpuBoundWork uses RAII, just like std::unique_lock. On construction, as long as there's no available slot, it yields the current coroutine so that I/O stuff can happen. (Kinda async spin.) Once there's a slot, it occupies that and continues. On destruction it frees the slot.

That "async spin" is neither optimal (Hello unfair scheduling!), nor necessary.

Describe the solution you'd like

As CpuBoundWork is scoped anyway, let's put the stuff to do in a callback and in the regular thread pool. It's their turn when it's their turn. Once that finished, the caller coroutine can be awakened via the existing AsioConditionVariable.

Describe alternatives you've considered

Also, if no slots are available, such an AsioConditionVariable can be put into a queue. This way nothing spins, but no callbacks need to be transferred across thread pools.

Additional context

Also, @yhabteab said I've used CpuBoundWork a bit more than necessary across lib/remote.

ref/IP/48306

[yhabteab]

There is absolutely no need to acquire a CPU slot here as there's no lock involved in anyway, that would entirely block the current thread.

icinga2/lib/remote/httpserverconnection.cpp

Lines 540 to 542 in 01a6c4c

	CpuBoundWork fetchingAuthenticatedUser (yc);
	authenticatedUser = ApiUser::GetByAuthHeader(std::string(request[http::field::authorization]));

icinga2/lib/remote/httpserverconnection.cpp

Lines 243 to 245 in 01a6c4c

	CpuBoundWork allowOriginHeader (yc);
	auto allowedOrigins (headerAllowOrigin->ToSet<String>());

This only wastes CPU slots and slows down the message handling unnecessarily, as you can update the stats within the try/catch above it while already holding a CPU slot for other (better) things.

icinga2/lib/remote/jsonrpcconnection.cpp

Lines 92 to 94 in 01a6c4c

	CpuBoundWork taskStats (yc);
	l_TaskStats.InsertValue(Utility::GetTime(), 1);

Although there is locking involved here, I don't think it should take too long for the thread to actually acquire it, since there aren't that many threads dealing with endpoint clients. It's just wasting pointless time trying to obtain a CPU slot.

icinga2/lib/remote/jsonrpcconnection.cpp

Lines 203 to 209 in 01a6c4c

	CpuBoundWork removeClient (yc);
	if (m_Endpoint) {
	m_Endpoint->RemoveClient(this);
	} else {
	ApiListener::GetInstance()->RemoveAnonymousClient(this);
	}