Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

portable matrix file #87

Open
mikejiang opened this issue Aug 6, 2018 · 4 comments
Open

portable matrix file #87

mikejiang opened this issue Aug 6, 2018 · 4 comments

Comments

@mikejiang
Copy link

Currently it is saved as a filebacked matrix file and the descriptor file. I assume the matrix file is written by boost interprocess, which should be portable across language as long as the language binding for interprocess lib is available. (e.g. https://github.com/ESSS/pyboost_ipc). But the descriptor file is saved as rds file, will it be more portable to save it as plain text so that the on-disk matrix can be loaded into other non-R environment?
Also, How easy (or difficult) is it to write an equivalent python package to interchange the memory-mapped matrix file from bigmemory?

@privefl
Copy link
Contributor

privefl commented Aug 6, 2018

Currently, the default is to write the descriptor as a plain text .desc file that basically contains the result of dput() of the descriptor.
Don't know Python well, but that shouldn't be too hard to parse I guess.

@mikejiang
Copy link
Author

Great. So how about the binary matrix file itself? Does it contain bigmemory-specific information or simply the generic boost interprocess format and thus readable by other platform independent from bigmemory R package?

@privefl
Copy link
Contributor

privefl commented Aug 6, 2018

The binary file containing the data is just nrow * ncol * sizeof(type) bytes.

@mikejiang
Copy link
Author

Thanks for the information!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants