-
Notifications
You must be signed in to change notification settings - Fork 985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Segfault in [.data.table
#4892
Comments
This comment has been minimized.
This comment has been minimized.
There are probably smaller datasets that could trigger the same segfault, but I haven't figured out how to make one. Here's a version that uses CSV instead of RData input if that helps; it does at least show that the problem isn't with binary loading or saving. Here are the two CSVs in a gzipped tarball.
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Here is an option to lookup
I think using |
As @chinsoon12 notes, the problem is in the library(data.table)
x = data.table(id = 1:4, key = 'id')
y = data.table(id = 2:5, key = 'id')
x[y, by = .(1:4), .SD, verbose = TRUE]
## next line will segfault
x[y, by = .(c(2L, 1L, 2L, 1L)), .SD, verbose = TRUE]
The requirements are 1) a keyed join; 2) a non-column I have some other things to follow-up on, but if this is still around I'll look deeper. |
FWIW data.table has been crashing with segfault errors on my 128 Gb Centos machine that defaults to 128 threads. The input data file to the script has just over 99 million edges. The same script runs fine on a Fedora machine with 16 threads so I tried setDT(16) in the Centos machine and the error doesn't seem to have recurred since although there is, unsurprisingly, a performance hit.
Matrix products: default locale: attached base packages: other attached packages: loaded via a namespace (and not attached): Error Message: [path to dbid]$ nohup Rscript ~/repos/spring_2022_research/george/fmoy_test.R & [path to dbid]$ tail -f nohup.out |
@chackoge note that this issue was closed by a commit in the current dev version 1.14.3, can you try installing that and making sure it solves your issue? |
@MichaelChirico thank you. This commit seems to have addressed the problem- I should have tried the dev version first,
other attached packages: |
glad it solved your issue!! |
The following code (see attached file for the data)
causes this segfault:
sessionInfo
is:The text was updated successfully, but these errors were encountered: