Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use vmsplice(2) to read/write multiple requests at a time #35

Closed
tv42 opened this issue Aug 15, 2014 · 3 comments
Closed

use vmsplice(2) to read/write multiple requests at a time #35

tv42 opened this issue Aug 15, 2014 · 3 comments

Comments

@tv42
Copy link
Member

tv42 commented Aug 15, 2014

depends: #33

@tv42 tv42 changed the title use vmsplice(2) to read/write multiple requests at a time use vmsplice(2) to read/write multiple requests at a time [depends:#33] Aug 15, 2014
@tv42 tv42 mentioned this issue Aug 17, 2014
@tv42
Copy link
Member Author

tv42 commented Jun 30, 2016

Oh, wow. After all this time, I come to the realization that there was no such thing as length-prefixed multiple requests in one memory area. I wonder where I even got that idea? Maybe from reading l-k back in the day and seeing a patch that never got accepted. Dang.

vmsplice trickery can still help in decreasing copies, but it's not very interesting for avoiding gc churn. I'm bummed now. Well, dropping maxWrite down for Linux helped there a ton, if that still needs more, next up we'll use "avoid sync.Pool for 2" trick from #120 and I think it was @jacobsa who suggested copying small requests out to avoid pinning a whole large buffer. Also BATCH_FORGET will help with one common request storm.

Right now a pretty simple lots of small requests benchmark says one request roundtrip takes half a millisecond and allocates <4kB (profiling blames context.WithCancel).

BenchmarkCreate-12    20000     647504 ns/op        3591 B/op         44 allocs/op

If you have benchmarks you care about, please try to share them! Feel free to create separate issues for that, to avoid drowning this one in those.

@tv42
Copy link
Member Author

tv42 commented Jun 30, 2016

A more flattering, and relevant, benchmark:

BenchmarkLookup-12    200000        181849 ns/op        1096 B/op         14 allocs/op

This avoids the NodeID bookkeeping that Create has, and is just a loop over a stat(2) getting ENOENT (no negative dentry cache).

@tv42 tv42 changed the title use vmsplice(2) to read/write multiple requests at a time [depends:#33] use vmsplice(2) to read/write multiple requests at a time Apr 15, 2020
@tv42 tv42 removed the not-ready label Dec 9, 2022
@tv42
Copy link
Member Author

tv42 commented Dec 9, 2022

Yup, vmsplice can't do what was thought of here. io_uring could do something similar, but that depends on so much Go infrastructure that it's not really up to us.

@tv42 tv42 closed this as not planned Won't fix, can't repro, duplicate, stale Dec 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant