blockstore: give a direct access to the index for read operations #387
Conversation
| @@ -178,6 +179,12 @@ func (b *ReadWrite) initWithRoots(v2 bool, roots []cid.Cid) error { | |||
| return carv1.WriteHeader(&carv1.CarHeader{Roots: roots, Version: 1}, b.dataWriter) | |||
| } | |||
|
|
|||
| // Index gives direct access to the index. | |||
There was a problem hiding this comment.
do we need this on ReadWrite - that's where it's dangerous.
The worry with ReadWrite access is you add another record, and then when you go to finalize the blockstore the extra record creeps in.
on ReadOnly, the index you access isn't going to get written back out into the underlying car file.
There was a problem hiding this comment.
I do need that on ReadWrite, but I can add a wrapper to enforce the read-only access.
There was a problem hiding this comment.
do you need it before you finalize?
could we have finalizeReadOnly give you a ReadOnly variant that you can then get the index from?
There was a problem hiding this comment.
I only need it after finalize, but I sort of see why some might.
|
I'm fine with this. Tests please, on both readonly and readwrite to prove that it's doing what you think it is. readwrite gives you InsertionIndex which is quite a different beasty than the typical Index. A wrapper to protect write operations might be a good idea, although I tend to lean on the caveat emptor approach—if you figure out how to hack it to do your crazy thing then that's your problem; we already have plenty of warnings and safety procedures in place wrt "untrusted" indexes from third parties and this library isn't going to stop bad indexes getting published. |
MichaelMure
force-pushed
the
blockstore-index
branch
from
80cfe8d
to
a9d8485
Compare
|
I added some tests. |
MichaelMure
force-pushed
the
blockstore-index
branch
from
a9d8485
to
979c38d
Compare
|
needs a rebase @MichaelMure, I'll cut a semver-minor after this gets merged |
MichaelMure
force-pushed
the
blockstore-index
branch
from
979c38d
to
a3e9d48
Compare
|
@rvagg it's rebased now. |
This can be useful for advanced read-only operation, like indexing.