Skip to content

Commit

Permalink
Implement a lock-free work-stealing deque
Browse files Browse the repository at this point in the history
This adds an implementation of the Chase-Lev work-stealing deque to libstd
under std::rt::deque. I've been unable to break the implementation of the deque
itself, and it's not super highly optimized just yet (everything uses a SeqCst
memory ordering).

The major snag in implementing the chase-lev deque is that the buffers used to
store data internally cannot get deallocated back to the OS. In the meantime, a
shared buffer pool (synchronized by a normal mutex) is used to
deallocate/allocate buffers from. This is done in hope of not overcommitting too
much memory. It is in theory possible to eventually free the buffers, but one
must be very careful in doing so.

I was unable to get some good numbers from src/test/bench tests (I don't think
many of them are slamming the work queue that much), but I was able to get some
good numbers from one of my own tests. In a recent rewrite of select::select(),
I found that my implementation was incredibly slow due to contention on the
shared work queue. Upon switching to the parallel deque, I saw the contention
drop to 0 and the runtime go from 1.6s to 0.9s with the most amount of time
spent in libuv awakening the schedulers (plus allocations).

Closes rust-lang#4877
  • Loading branch information
alexcrichton committed Nov 29, 2013
1 parent 08f4d1f commit a70f9d7
Show file tree
Hide file tree
Showing 7 changed files with 723 additions and 141 deletions.
13 changes: 7 additions & 6 deletions src/librustuv/net.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1076,21 +1076,22 @@ mod test {
use std::rt::task::Task;
use std::rt::task::UnwindResult;
use std::rt::thread::Thread;
use std::rt::work_queue::WorkQueue;
use std::rt::deque::BufferPool;
use std::unstable::run_in_bare_thread;
use uvio::UvEventLoop;

do run_in_bare_thread {
let sleepers = SleeperList::new();
let work_queue1 = WorkQueue::new();
let work_queue2 = WorkQueue::new();
let queues = ~[work_queue1.clone(), work_queue2.clone()];
let mut pool = BufferPool::init();
let (worker1, stealer1) = pool.deque();
let (worker2, stealer2) = pool.deque();
let queues = ~[stealer1, stealer2];

let loop1 = ~UvEventLoop::new() as ~EventLoop;
let mut sched1 = ~Scheduler::new(loop1, work_queue1, queues.clone(),
let mut sched1 = ~Scheduler::new(loop1, worker1, queues.clone(),
sleepers.clone());
let loop2 = ~UvEventLoop::new() as ~EventLoop;
let mut sched2 = ~Scheduler::new(loop2, work_queue2, queues.clone(),
let mut sched2 = ~Scheduler::new(loop2, worker2, queues.clone(),
sleepers.clone());

let handle1 = Cell::new(sched1.make_handle());
Expand Down
Loading

1 comment on commit a70f9d7

@alexcrichton
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=pcwalton

Please sign in to comment.