Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify that copied allocators must behave the same #105096

Merged
merged 1 commit into from
Mar 26, 2023

Conversation

LegionMammal978
Copy link
Contributor

@LegionMammal978 LegionMammal978 commented Nov 30, 2022

Currently, the safety documentation for Allocator says that a cloned or moved allocator must behave the same as the original. However, it does not specify that a copied allocator must behave the same, and it's possible to construct an allocator that permits being moved or cloned, but sometimes produces a new allocator when copied.

Contrived example which results in a Miri error
#![feature(allocator_api, once_cell, strict_provenance)]
use std::{
    alloc::{AllocError, Allocator, Global, Layout},
    collections::HashMap,
    hint,
    marker::PhantomPinned,
    num::NonZeroUsize,
    pin::Pin,
    ptr::{addr_of, NonNull},
    sync::{LazyLock, Mutex},
};

mod source_allocator {
    use super::*;

    // `SourceAllocator` has 3 states:
    // - invalid value: is_cloned == false, source != self.addr()
    // - source value:  is_cloned == false, source == self.addr()
    // - cloned value:  is_cloned == true
    pub struct SourceAllocator {
        is_cloned: bool,
        source: usize,
        _pin: PhantomPinned,
    }

    impl SourceAllocator {
        // Returns a pinned source value (pointing to itself).
        pub fn new_source() -> Pin<Box<Self>> {
            let mut b = Box::new(Self {
                is_cloned: false,
                source: 0,
                _pin: PhantomPinned,
            });
            b.source = b.addr();
            Box::into_pin(b)
        }

        fn addr(&self) -> usize {
            addr_of!(*self).addr()
        }

        // Invalid values point to source 0.
        // Source values point to themselves.
        // Cloned values point to their corresponding source.
        fn source(&self) -> usize {
            if self.is_cloned || self.addr() == self.source {
                self.source
            } else {
                0
            }
        }
    }

    // Copying an invalid value produces an invalid value.
    // Copying a source value produces an invalid value.
    // Copying a cloned value produces a cloned value with the same source.
    impl Copy for SourceAllocator {}

    // Cloning an invalid value produces an invalid value.
    // Cloning a source value produces a cloned value with that source.
    // Cloning a cloned value produces a cloned value with the same source.
    impl Clone for SourceAllocator {
        fn clone(&self) -> Self {
            if self.is_cloned || self.addr() != self.source {
                *self
            } else {
                Self {
                    is_cloned: true,
                    source: self.source,
                    _pin: PhantomPinned,
                }
            }
        }
    }

    static SOURCE_MAP: LazyLock<Mutex<HashMap<NonZeroUsize, usize>>> =
        LazyLock::new(Default::default);

    // SAFETY: Wraps `Global`'s methods with additional tracking.
    // All invalid values share blocks with each other.
    // Each source value shares blocks with all cloned values pointing to it.
    // Cloning an allocator always produces a compatible allocator:
    // - Cloning an invalid value produces another invalid value.
    // - Cloning a source value produces a cloned value pointing to it.
    // - Cloning a cloned value produces another cloned value with the same source.
    // Moving an allocator always produces a compatible allocator:
    // - Invalid values remain invalid when moved.
    // - Source values cannot be moved, since they are always pinned to the heap.
    // - Cloned values keep the same source when moved.
    unsafe impl Allocator for SourceAllocator {
        fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError> {
            let mut map = SOURCE_MAP.lock().unwrap();
            let block = Global.allocate(layout)?;
            let block_addr = block.cast::<u8>().addr();
            map.insert(block_addr, self.source());
            Ok(block)
        }

        unsafe fn deallocate(&self, block: NonNull<u8>, layout: Layout) {
            let mut map = SOURCE_MAP.lock().unwrap();
            let block_addr = block.addr();
            // SAFETY: `block` came from an allocator that shares blocks with this allocator.
            if map.remove(&block_addr) != Some(self.source()) {
                hint::unreachable_unchecked()
            }
            Global.deallocate(block, layout)
        }
    }
}
use source_allocator::SourceAllocator;

// SAFETY: `alloc1` and `alloc2` must share blocks.
unsafe fn test_same(alloc1: &SourceAllocator, alloc2: &SourceAllocator) {
    let ptr = alloc1.allocate(Layout::new::<i32>()).unwrap();
    alloc2.deallocate(ptr.cast(), Layout::new::<i32>());
}

fn main() {
    let orig = &*SourceAllocator::new_source();
    let orig_cloned1 = &orig.clone();
    let orig_cloned2 = &orig.clone();
    let copied = &{ *orig };
    let copied_cloned1 = &copied.clone();
    let copied_cloned2 = &copied.clone();
    unsafe {
        test_same(orig, orig_cloned1);
        test_same(orig_cloned1, orig_cloned2);
        test_same(copied, copied_cloned1);
        test_same(copied_cloned1, copied_cloned2);
        test_same(orig, copied); // error
    }
}

This could result in issues in the future for algorithms that specialize on Copy types. Right now, nothing in the standard library that depends on Allocator + Clone is susceptible to this issue, but I still think it would make sense to specify that copying an allocator is always as valid as cloning it.

@rustbot
Copy link
Collaborator

rustbot commented Nov 30, 2022

r? @joshtriplett

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Nov 30, 2022
@rustbot
Copy link
Collaborator

rustbot commented Nov 30, 2022

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

  • Stabilizing library features
  • Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
  • Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
  • Changing public documentation in ways that create new stability guarantees
  • Changing observable runtime behavior of library APIs

@LegionMammal978
Copy link
Contributor Author

r? libs-api

@rustbot rustbot assigned Amanieu and unassigned joshtriplett Mar 25, 2023
@Amanieu
Copy link
Member

Amanieu commented Mar 25, 2023

@bors r+ rollup

@bors
Copy link
Contributor

bors commented Mar 25, 2023

📌 Commit 57e12f9 has been approved by Amanieu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 25, 2023
@LegionMammal978
Copy link
Contributor Author

@rustbot label +T-libs-api -T-libs

(just for consistency's sake)

@rustbot rustbot added T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. and removed T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Mar 25, 2023
@bors
Copy link
Contributor

bors commented Mar 26, 2023

⌛ Testing commit 57e12f9 with merge 48ae1b3...

@bors
Copy link
Contributor

bors commented Mar 26, 2023

☀️ Test successful - checks-actions
Approved by: Amanieu
Pushing 48ae1b3 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 26, 2023
@bors bors merged commit 48ae1b3 into rust-lang:master Mar 26, 2023
@rustbot rustbot added this to the 1.70.0 milestone Mar 26, 2023
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (48ae1b3): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.9% [1.9%, 1.9%] 1
Regressions ❌
(secondary)
4.0% [1.1%, 6.2%] 3
Improvements ✅
(primary)
-1.2% [-1.2%, -1.2%] 1
Improvements ✅
(secondary)
-0.7% [-0.7%, -0.6%] 2
All ❌✅ (primary) 0.4% [-1.2%, 1.9%] 2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
0.6% [0.4%, 0.8%] 8
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

@LegionMammal978 LegionMammal978 deleted the copied-allocators branch May 15, 2023 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants