Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Major Change] Enforcing tensor alignment #148

Merged
merged 4 commits into from
Feb 21, 2023
Merged

[Major Change] Enforcing tensor alignment #148

merged 4 commits into from
Feb 21, 2023

Commits on Feb 8, 2023

  1. [Major Change] Enforcing tensor alignment 

    - Now the header will automatically align itself to 8 bytes (f64) with
  appending extra spaces as necessary.
- This will allow extra fast memory mapping by reinterpreting bytes as
  f32/f64 etc.. Unaligned bytes do not allow for this. https://www.reddit.com/r/rust/comments/tanaxm/mutating_a_buffer_of_u8s_as_f32s_in_place/
- This does not change contiguousness of tensors
- This does not change the actual spec (we're just putting extra valid bytes
  in the header and using a different serialization ordering)
- Readers should still be able to read old files, they would just need
  to be copied before being cast as their final destination when using
  mmap
- This has no effect for GPU since copy is already necessary (*I think*,
  depends on the cuda API actually if it allows filling f32 addresses
  from raw unaligned bytes).

This change will only be interesting if things like https://github.com/Narsil/fast_gpt2
actually pick up. And even with the copy, load times are still vastly
superior to `pytorch`.

We need to be able to read old files.
    @Narsil
    Narsil committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    1cf3343 View commit details
    Browse the repository at this point in the history

  2. Fixup.

    @Narsil
    Narsil committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    f5d27a4 View commit details
    Browse the repository at this point in the history

  3. Clippy fix.

    @Narsil
    Narsil committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    fb9869b View commit details
    Browse the repository at this point in the history

  4. Cargo fmt (clippy --fix broke it ? :( )

    @Narsil
    Narsil committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    f5ef88d View commit details
    Browse the repository at this point in the history