HikariCP Fairness: Independent Analysis
We would like to thank James Pickering for his technical analysis of HikariCP's ConcurrentBag, which came in our response to his Battle of the Connection Pools blog post.
We asserted a claim of queueing 'fairness', to which James had reasonable questions about susceptibility to 'barging' behavior by threads.
The following his James' analysis:
I had a go at testing the fairness of ConcurrentBag. Turns out fairness is tricky to test - you need some kind of external synchronization to actually gauge fairness, but then you've changed the result by measuring it!
Still, I put together a simple framework to roughly measure the fairness of ConcurrentBag, which is on GitHub. In the test, I included ConcurrentBag, a fair queue from the JVM (ArrayBlockingQueue with fairness enabled), and a hand-rolled queue that was designed to be as unfair as possible without being unrealistic (it's a single element queue, that uses fast-path barging, and JVM synchronized/notify for slow-path acquisition - I thought about implementing a stack-based "queue", but decided it would be a lot of work for no benefit).
The best illustration of fairness seems to be the acquire-time graphs, below:
I ran two worker threads, each trying to acquire resources from a bag or queue with one element, for 100000 iterations each. The graph shows the "time" at which successive resource acquires succeeded. "Time" is in quotes, since the framework comes with three "timestampers", only one of which is actually based on wall-clock time. The timestamper used in these tests was based on a counter protected by a fair lock (although I found similar results with the other timestampers).
For a completely fair queue, you'd expect the two lines to overlap. However, they don't quite overlap for the fair queue, probably due to quirks of OS scheduling. ConcurrentBag shows slightly more deviation, but seems to fit the bill of being "mostly fair". It's certainly more fair than my made-up, really unfair queue, which is all over the place.
Obviously, this test is massively unrealistic. There's no work done between acquire and release, which massively increases the chances of a barge attempt succeeding. In real life, I can't imagine you'd be able to distinguish between HikariCP and a fair pool - other than the fair pool being slower.
The fairness harness was run against HikariCP 2.6.0's ConcurrentBag implementation and this is the result:
