-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mmap_array segfaults #11351
Comments
Does this also happen with the current |
Yes, this is current behavior on 0.3/0.4. |
Here's another fun f = open("C:/Users/karbarcca/test.jl", "r")
r = mmap_array(Uint8, (10,), f, -1)
# segfault By trying to do a negative offset. (Yay for more testing!) |
Ugh... Is |
I think this is one of the design flaws of |
(related: On Tue, May 19, 2015 at 1:51 PM, Jacob Quinn [email protected]
|
#6877, see in particular the last sentences of my introductory post. |
Also, I suppose this is a case where #10503 made things less clear. |
So I know this is bleeding into #11280, but what do people think about getting rid of |
My workflow often involves creating objects that have array-typed fields, saving them to disk, and then reading the objects back in using JLD's However, if we want a separate type, I'm not sure if it should be |
Yeah, I was just looking through rewriting without |
From a certain perspective it's just a block of memory; it's a detail that it happens to be backed by a disk file. Seems much more array-like than IO-like to me. |
Yes, the more I think about it, the more it makes sense to think about it like an Array, but a special Array that has to deal with file permissions 😨 Alright, I think I'll try to make |
If we can merge #10525, almost nothing is needed other than scalar indexing and support for Personally, until #10525 is merged I think your best strategy is to keep |
It's not just file permissions... for example, what if you are using shared memory (does your mmap allow for that?), and you only want one process to be able to modify the data, and many to be able to read it... |
Yes, SharedArrays use this to share memory. I don't think we have anything implemented to give different workers different permissions, aside from having the workers open the "file" with different permissions. We don't currently have a concept of a "read-only" array in base. (Such a thing could easily be implemented in a package, however.) |
Yet another fun one: f = open("/Users/jacobquinn/test")
m = mmap_array(UInt8, (11,), f, 0)
# returns successfully
Libc.munmap(pointer(m), 11) # run munmap() manually
m[1]
# segfaults |
@quinnj If you call x = zeros(10)
Libc.free(pointer(x))
# segfaults |
Yeah, that's a fair point. I'd also argue that this is something that could be guarded easily against by having an actual |
So in doing some more testing, I've found another weird case of segfaulting: if I run a = mmap_array(...) # read-only array
a[5] = 2 # try to setindex! from the command line, I correctly get an I just reverted #11280 back to returning a regular |
…ndows and creating a new ReadOnlyMemoryError type
Fixed via #11491. |
While battle-testing #11280, I discovered the following for Windows:
On OSX, I get an
OutOfMemoryError()
, but I don't think that's very informative.I'm not entirely sure what to do here. Restrict
Mmap.Array
to file's where we have read & write permissions? Is that overly restrictive? Put a warning in the docs to be careful with read-only files? Rethink theMmap.Array
interface entirely?The text was updated successfully, but these errors were encountered: