Fix detection of duplicate torch tensors

[fxmarty]

As per title. Two tensors with different data may share the same data_ptr in case they are views of an other tensor. For example:

import torch

a = torch.rand(8, 10, 50)

b = a[2:4]
c = a[4:6]

print(a.untyped_storage().data_ptr())
print(b.untyped_storage().data_ptr())
print(c.untyped_storage().data_ptr())

prints

93888779151936
93888779151936
93888779151936

[Narsil]

I think this line is correct.

It's trying to see which tensors have shared memory, which is indicated by storage.
offset will make you miss some shared tensors.

[fxmarty]

In my understanding, what this functions does is finding which tensors in the state_dict are "identical". Relying on data_ptr() is in this case not enough, as data_ptr is the same for views of a tensor - views that may represent different data. For example, bar = a[2:4] is not the same tensor as foo = a[4:6].

I'll try to come up with a minimal example to showcase what I mean.

[fxmarty]

@Narsil @LysandreJik Here is a minimal repro of the issue:

import torch
import torch.nn as nn
from safetensors.torch import _find_shared_tensors

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.qkv = nn.Linear(20, 30)
        
        self.q = nn.Linear(20, 10)
        self.q.weight = torch.nn.Parameter(self.qkv.weight[:10])
    
        self.v = nn.Linear(20, 10)
        self.v.weight = torch.nn.Parameter(self.qkv.weight[10:20])

    def forward(self, x):
        return x

model = MyModel()
shared_params = _find_shared_tensors(model.state_dict())
print("shared_params", shared_params)

printing: shared_params [{'v.weight', 'qkv.weight', 'q.weight'}, {'qkv.bias'}, {'q.bias'}, {'v.bias'}]

[fxmarty]

Okay but I understand that you argue that _find_shared_tensors is meant to group tensors sharing memory, no matter whether some of them are views and the actual data is different. Is that the case?

Despite the fix huggingface/transformers#27314, Transformers still calls safetensors.torch.save_file, and complains that tensors still have shared memory: https://github.com/fxmarty/safetensors/blob/f04d064884be5ede7b6f7d844ce22b793607d091/bindings/python/py_src/safetensors/torch.py#L474. Couldn't Transformers call save_model as the error suggests? Though I doubt this would help as save_model calls _remove_duplicate_names that in turns calls _find_shared_tensors.

[Narsil]

Fixed directly in transformers.