Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove ArrayVec dependency #414

Merged
merged 6 commits into from
Sep 17, 2019

Conversation

Shnatsel
Copy link
Contributor

@Shnatsel Shnatsel commented Sep 7, 2019

ArrayVec is used in just one place, and even there crossbeam uses a small part of its API surface. This PR replaces ArrayVec with 100% safe code using a plain old array and ditches the ArrayVec dependency altogether.

This removes 170 unsafe expressions from the dependency tree according to cargo-geiger. With less unsafe code to worry about there's less chance of memory safety bugs, and safety/security audits will be cheaper and easier to perform.

An unfortunate casualty of this change is #[derive(Debug)] on internal structures because there is no Debug implementation for [T; 64] - these impls are only generated up to length of 32. It will be possible to re-enable it once const generics are stabilized. For now they're implemented manually.

cargo bench results are unaffected - or might be within experimental noise of 2% or so, hard to tell without statistical analysis on benchmark results.

Copy link

@Lokathor Lokathor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing Debug from anything feels unacceptable, particularly since it's a breaking change. You should just write the impl manually using a loop over 0..len

@Shnatsel
Copy link
Contributor Author

Shnatsel commented Sep 7, 2019

CI failed with "error: Could not compile lazy_static." - apparently 1.26 is too old to compile lazy_static!. Nightly has passed.

@jeehoonkang
Copy link
Contributor

I'm personally preferring to use ArrayVec, because it's specifically designed for such a scenario as Bag. The number of unsafe expressions of Crossbeam is not a big concern, I think, as (1) ArrayVec is an established project, and (2) its safety is as much scrutinized as Crossbeam's.

@Shnatsel
Copy link
Contributor Author

Shnatsel commented Sep 7, 2019

@Lokathor I've implemented Debug manually

@Shnatsel
Copy link
Contributor Author

Shnatsel commented Sep 7, 2019

@jeehoonkang the selling point of Rust is that you can trust the compiler to verify safety for you instead of trusting humans to get it right.

ArrayVec might be sound now (although I haven't checked, and you wouldn't believe what I've found in some other popular Rust crates recently), but having an unsafe dependency is not just about now, it's a continuous burden. Not only you have to audit all unsafe blocks in the the current version, you also have to commit to auditing all unsafe blocks in all future versions.

Between taking on the burden of manually auditing all the unsafe in all future versions of ArrayVec and rewriting this small bit as safe code without even incurring a performance penalty, the choice is pretty obvious.

Deferred::new(no_op_func), Deferred::new(no_op_func), Deferred::new(no_op_func), Deferred::new(no_op_func),
Deferred::new(no_op_func), Deferred::new(no_op_func), Deferred::new(no_op_func), Deferred::new(no_op_func),
Deferred::new(no_op_func), Deferred::new(no_op_func), Deferred::new(no_op_func), Deferred::new(no_op_func),
Deferred::new(no_op_func), Deferred::new(no_op_func), Deferred::new(no_op_func), Deferred::new(no_op_func)]
Copy link

@jadbox jadbox Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious on two things over why are this many particular Deferred objects created here and is there a static way to populate the list than to hardcode each new obj construction (to make it DRY'er)?

Copy link
Member

@Vtec234 Vtec234 Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because Deferred isn't copy, there is only the unsafe option to start with mem::uninitialized and then fill the array with ptr::write, but that would defeat the point of this PR. I don't believe macro_rules macros can pattern match on natural numbers (so that we could insert the same expression n times with a macro), and a compiler plugin would be overkill, so it seems we have to tolerate this until the mentioned rust-lang issue is resolved.
@Shnatsel could you mark this as TODO?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can match on a number but you're just matching on the token form of a number, so if you match on 3 you know that you're in the 3 branch but can't automatically use the 3 in any way. You'd need to write a macro that expanded each case by hand anyway.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could definitely do this with a macro, maybe just not in the most natural way. For example you could hardcode 0-10 fold repetition and write 64-fold repetition as repeating by 6 * 10 + 4 🙂

@Vtec234
Copy link
Member

Vtec234 commented Sep 12, 2019

@jeehoonkang If there is no performance penalty (and I don't see why there should be one, as we'd still be using a staticly-allocated array) and we end up with fewer unsafe dependencies, I think the slightly ugly array initialization is acceptable, so I'd support merging this.

@Shnatsel
Copy link
Contributor Author

Ugly array initialization marked TODO as requested. Is there anything else to be done, or is it ready for merging?

@Vtec234 Vtec234 merged commit ae148de into crossbeam-rs:master Sep 17, 2019
@Shnatsel Shnatsel deleted the remove-arrayvec-dependency branch September 17, 2019 15:45
for deferred in self.deferreds.drain(..) {
for i in 0..self.len {
let no_op = Deferred::new(no_op_func);
let deferred = mem::replace(&mut self.deferreds[i], no_op);
deferred.call();
}
Copy link

@bluss bluss Sep 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be an iteration over &mut self.deferreds[..self.len]

bors bot added a commit that referenced this pull request Oct 27, 2019
442: Do not perform superfluous bounds checks on dropping crossbeam-epoch::Bag r=jeehoonkang a=Shnatsel

Addresses [this comment](#414 (comment)) that I didn't get a chance to act upon before merging #414. 

This way bounds checks should be performed only once. However, this does not measurably impact benchmarks.

Co-authored-by: Sergey "Shnatsel" Davidoff <[email protected]>
exrook pushed a commit to exrook/crossbeam that referenced this pull request Oct 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

6 participants