Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Work around the infamous locked DLL issue #368

Closed
gaborcsardi opened this issue May 16, 2019 · 28 comments
Closed

Work around the infamous locked DLL issue #368

gaborcsardi opened this issue May 16, 2019 · 28 comments
Labels
feature a feature request or enhancement

Comments

@gaborcsardi
Copy link
Member

gaborcsardi commented May 16, 2019 •
edited
Loading

This issue is currently a place that I'll dump my notes about the investigation of this problem, and also thoughts about possible workarounds.

What R does

R 3.6.0, install binary from remote repo, or from a local ZIP file

  • Staged installation does not matter here.

  • If the package is loaded in the current R session, installations is refused with a warning:

> install.packages("curl")
Installing package into ‘C:/Users/Gabor/Documents/R/win-library/3.6’
(as ‘lib’ is unspecified)
Warning: package ‘curl’ is in use and will not be installed
  • Now if we unload the package, but not the DLL, then we get a warning and a broken package is left around, i.e. only the dll is there, the rest is removed:
> unloadNamespace("curl")
> install.packages("curl")
Installing package into ‘C:/Users/Gabor/Documents/R/win-library/3.6’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/bin/windows/contrib/3.6/curl_3.3.zip'
 length 2996751 bytes (2.9 MB)
downloaded 2.9 MB

package ‘curl’ successfully unpacked and MD5 sums checked
Warning: cannot remove prior installation of package ‘curl’

The downloaded binary packages are in
        C:\Users\Gabor\AppData\Local\Temp\Rtmp2JeKkE\downloaded_packages
> library(curl)
Error in library(curl) : there is no package called ‘curl’
  • The same happens in the package is loaded in another R session. Note that even if we install the exact same binary, we'll end up with a broken installation.

R 3.6.0, install source from remote repo, or from local tar.gz file, or from a local directory

  • If the package is loaded in the current session, R still tries to install it. With the default staged install, moving the package to its final place produces a warning, but the installation still continues, and then the load-test fails, and the originally installed package is restored. So for some time the library contains a broken package. Finally, the install.packages() call gives a warning that the installation had a non-zero exit status:
** testing if installed package can be loaded from temporary location
*** arch - i386
*** arch - x64
WARNING: moving package to final location failed, copying instead
Warning in file.copy(instdir, dirname(final_instdir), recursive = TRUE,  :
  problem copying C:\Users\Gabor\Documents\R\win-library\3.6\00LOCK-curl\00new\curl\libs\x64\curl.dll to C:\Users\Gabor\Documents\R\win-library\3.6\curl\libs\x64\curl.dll: Permission denied
** testing if installed package can be loaded from final location
*** arch - i386
*** arch - x64
Error: package or namespace load failed for 'curl' in FUN(X[[i]], ...):
 no such symbol R_new_file_writer in package C:/Users/Gabor/Documents/R/win-library/3.6/curl/libs/x64/curl.dll
Error: loading failed
Execution halted
ERROR: loading failed for 'x64'
* removing 'C:/Users/Gabor/Documents/R/win-library/3.6/curl'
* restoring previous 'C:/Users/Gabor/Documents/R/win-library/3.6/curl'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
  problem copying C:\Users\Gabor\Documents\R\win-library\3.6\00LOCK-curl\curl\libs\x64\curl.dll to C:\Users\Gabor\Documents\R\win-library\3.6\curl\libs\x64\curl.dll: Permission denied

The downloaded source packages are in
        ‘C:\Users\Gabor\AppData\Local\Temp\Rtmp4EIpFM\downloaded_packages’
Warning message:
In install.packages("curl", type = "source") :
  installation of package ‘curl’ had non-zero exit status

  • The same happens if the package is loaded in another session.

  • If the already installed package is of the same version as the one being installed, then the load-test succeeds and you only get the warnings about the failed move and the failed copy:

** testing if installed package can be loaded from temporary location
*** arch - i386
*** arch - x64
WARNING: moving package to final location failed, copying instead
Warning in file.copy(instdir, dirname(final_instdir), recursive = TRUE,  :
  problem copying C:\Users\Gabor\Documents\R\win-library\3.6\00LOCK-curl\00new\curl\libs\x64\curl.dll to C:\Users\Gabor\Documents\R\win-library\3.6\curl\libs\x64\curl.dll: Permission denied
** testing if installed package can be loaded from final location
*** arch - i386
*** arch - x64
** testing if installed package keeps a record of temporary installation path
* DONE (curl)
  • If the already installed package is not of the same version as the one being installed, but they are close enough for the load-test to succeed, then you get a package that looks OK, but essentially has undefined behavior: it may crash, it may freeze, it may fail, or produce the wrong results. :(

R 3.6.0, non-staged install, source from repo

The end result is the same as for staged install, although the warnings are slightly different. If the load-test fails, then we only get the warning about the failed copy of the DLL file.

** testing if installed package can be loaded
*** arch - i386
*** arch - x64
Error: package or namespace load failed for 'curl' in FUN(X[[i]], ...):
 no such symbol R_new_file_writer in package C:/Users/Gabor/Documents/R/win-library/3.6/curl/libs/x64/curl.dll
Error: loading failed
Execution halted
ERROR: loading failed for 'x64'
* removing 'C:/Users/Gabor/Documents/R/win-library/3.6/curl'
* restoring previous 'C:/Users/Gabor/Documents/R/win-library/3.6/curl'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
  problem copying C:\Users\Gabor\Documents\R\win-library\3.6\00LOCK-curl\curl\libs\x64\curl.dll to C:\Users\Gabor\Documents\R\win-library\3.6\curl\libs\x64\curl.dll: Permission denied

The downloaded source packages are in
        ‘C:\Users\Gabor\AppData\Local\Temp\RtmpIlhhOz\downloaded_packages’
Warning message:
In install.packages("curl", type = "source") :
  installation of package ‘curl’ had non-zero exit status

What pkginstall does currently

pkginstall only installs binary packages. To install a source package, needs to be R CMD build-t first, and then pkginstall extracts the binary, and install the extracted binary:

This is the current code:
https://github.com/r-lib/pkginstall/blob/0ca61acc1028e1f8d399bbb175824219bcee12e5/R/install_binary.R#L51-L92

In a nutshell:

  1. First move the existing package dir to a temporary place. This works even if a process has the DLL open. If the move fails, it aborts.
  2. Then tries to delete the moved dir, which may fail if another process as the file open. If it fails, it gives a warning.
  3. Then it tries to move the extracted new package into the library. If this fails, then it aborts.
@gaborcsardi
Copy link
Member Author

gaborcsardi commented May 16, 2019 •
edited
Loading

Here is what should happen ideally, from the user's point of view. (It might not be possible to achieve this, though.)

  • The user should get an error that the installation is aborted. Maybe this could be restricted to the packages that have a DLL, but even w/o the DLL the installation might break the current session, via the lazy-load DB changes. So is is better to do it for all packages.

  • The user should get some hints about whether the package is loaded into the current session or another one, and possibly also which one (e.g. its PID).

  • If the current session is interactive, and the package is loaded in another session, then the user has the choice to

    • abort and try again
    • try the installation anyway, knowing that this might break the other session
    • kill the other R process(es)
    • install the package(s) in a temporary library, that is added to .libPaths(), so they can still be used in the current session, without breaking the other session.

@jimhester
Copy link
Member

jimhester commented May 16, 2019 •
edited
Loading

Then it tries to move the extracted new package into the library. If this fails, then it gives a warning.

Is this right? Shouldn't we throw an error if the new package can't be moved into the library?

Edit: it looks like it errors for this as well (to me)

https://github.com/r-lib/pkginstall/blob/0ca61acc1028e1f8d399bbb175824219bcee12e5/R/install_binary.R#L82-L83

@gaborcsardi
Copy link
Member Author

@jimhester yes, sry, updated above!

@jimhester
Copy link
Member

Having more diagnostic info would be nice, but in general what is wrong with what pkginstall does?

@gaborcsardi
Copy link
Member Author

I think pkginstall is basically doing the right things. It could provide more info, and maybe even help you resolve the issue, see the edited #368 (comment) above.

I should have said it at the beginning, but there are two goals here, one is having a good way to handle this in remotes/pak/renv/etc. via a standalone file, the other is to see how we could write a patch to install.packages() that improves the situation.

@gaborcsardi
Copy link
Member Author

@jimhester what do you think about #368 (comment) ?

@jimhester
Copy link
Member

I think it sounds pretty good overall

@gaborcsardi
Copy link
Member Author

One question is whether we want to do sg about source installation. pkginstall always runs R CMD build first, and maybe we could just do the same in remotes, but I am not sure.

Working around source installation is much more difficult, because we need to call install.packages().

@mattdowle
Copy link

mattdowle commented Jun 11, 2019 •
edited
Loading

More background to gather in one place :
Rdatatable/data.table#3056 (data.table issue)
Rdatatable/data.table#3237 (data.table permanent solution using compiled constant but each package would have to do this too and therefore isn't a general solution)
https://twitter.com/MattDowle/status/1084528873549705217 (twitter discussion)
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17478 (R bug report)
https://tdhock.github.io/blog/2019/windows-dll/ (recent article by Toby Dylan Hocking)
http://r.789695.n4.nabble.com/R-pkg-install-should-fail-for-unsuccessful-DLL-copy-on-windows-tp4757281p4757302.html (problem on Bioconductor attributed to this)

@mattdowle
Copy link

mattdowle commented Jun 11, 2019 •
edited
Loading

I'm willing to help with a fix. But I don't like starting to work on something only to find that R-core are already working on it, or know of failed approaches they've tried before. I'm hoping for some kind of interaction with R-core on it before I look myself. But they haven't replied and the bug is still unconfirmed status.

@gaborcsardi
Copy link
Member Author

@kalibera is working on it already, apparently:
wch/r-source@0434ff2
wch/r-source@ebef820

@mattdowle
Copy link

Seems like R-devel fixes it. 17478 has been closed.
https://twitter.com/MattDowle/status/1141443694735704065
I suppose I should retest latest R-devel on Windows then.

@kalibera
Copy link

FYI, the fixes have been ported to R-patched as well.

@gaborcsardi gaborcsardi mentioned this issue Jun 20, 2019
Closed
jimhester added a commit that referenced this issue Jun 20, 2019
@jimhester
Use locking when installing packages
ae93c2e 
Part of #368
@jimhester jimhester added the feature a feature request or enhancement label Jun 25, 2019
@msevestre msevestre mentioned this issue Jul 25, 2019
Merged
@krlmlr
Copy link
Member

krlmlr commented Mar 30, 2020

Independently of that, do we want to offer a helper that checks all files under the package library if they are in use, and by what process, and offers a way to kill?

@gaborcsardi
Copy link
Member Author

@krlmlr Yeah, that would be useful, but it is not at all trivial to find the processes that are locking a file.

@krlmlr krlmlr mentioned this issue Mar 31, 2020
Closed
@dracodoc
Copy link

dracodoc commented May 5, 2020

Is this problem already fixed with R 4.0 and current version of remotes?

I met this problem with devtools, which looks like the case 1:

  • rlang 0.4.5 binary is installed in windows, and I always chose not to install 0.4.6 from source when asked
  • devtools::install_github will try to install rlang, after the question and user choice, still install 0.4.5, which result an error.
  • remotes::install_github did the same, but don't have problem because rlang is loaded by devtools, not by remotes.
  • In all cases, I restarted R session right before the command, didn't load devtools/remotes package itself.

If the problem mentioned in this issue has been fixed in R and remotes, the issue I met above should not appear, right?

Alternatively, at least for my case, I think some logic can be added to avoid install a package with same version again, which doesn't make sense anyway. So when user chose not to install from source, there should be some check before installing binary version, because the plan of installing this package was based on the source version is newer, but now the binary version is same and the installation should be skipped.

I think this should be easy to implement as there is no locking involved, just add some check logic is enough. I'd happy to try to implement and test it.

@dracodoc
Copy link

dracodoc commented May 5, 2020

It seemed the timing of each choice is a little bit tricky:

  • remotes checked package sha with remote_sha, which will report rlang to be 0.4.6 in remote, thus go ahead and install it.
  • install.packages was called, then ask user whether to install from source. User may select no, but it will just go ahead install binary version because there is no version checking/avoid re-installation in this step.

The better approach might be change in install.packages side, otherwise it's hard for remotes to determine right version first without user input on binary/source, unless it's specified from the beginning as parameters.

@jimhester
Copy link
Member

@dacodoc, I don't think you should try to fix this issue.

@dracodoc
Copy link

dracodoc commented May 9, 2020

I agree, this is not as straightforward as I thought.

@MichaelChirico MichaelChirico mentioned this issue May 19, 2020
Closed
@mirh
Copy link

mirh commented Feb 12, 2021

Can't you trigger/ask for a reload of the session like RStudio does?
(they are not always successful, but still that could get me to update xfun at the end of the day)

@gaborcsardi
Copy link
Member Author

We could unload the packages that are loaded in the same session, before the installation. But we cannot detect the DLLs that are locked by other R processes, without the ps package or compiled code.

In any case, this is now solved nicely in pak, so it is not very urgent to improve remotes:
r-lib/pak#174
r-lib/pak#206

@kalibera
Copy link

Unloading packages cannot be expected to work reliably (see e.g. ?dyn.unload, one of the problems are packages registering finalizers, but there are more).

@gaborcsardi
Copy link
Member Author

Yes, that is a risk that we take. It does work for most packages. Reloading does fail often, so pak suggests a restart after the installation:

> library(testthat)
> pak::pak("testthat")
v Updated metadata: 3.59 MB in 7 files.
v Updating local metadata database ... done

> Will update 2 packages.
> Will download 1 CRAN package (26.23 kB).
> Will download 1 package with unknown size.
+ testthat 3.0.1 > 3.0.2 [bld][cmp][dl]
+ waldo    0.2.3 > 0.2.4 [bld][dl] (26.23 kB)

! testthat is loaded in the current R session and has locked files in your
  library. The installation will probably fail, unless pak unloads it, and also
  all other packages importing it.

  What do you want to do?

  1. Have pak unload them before the installation. (Safest option.)
  2. Try the installation without unloading them.
  3. Abort the installation.

? Your choice [1]: 1
  Unloading testthat...

i Getting 1 pkg (26.23 kB) and 1 pkg with unknown size
v Got waldo 0.2.4 (source) (26.23 kB)
v Got testthat 3.0.2 (source) (680.37 kB)
v Downloaded 2 packages (706.59 kB) in 960ms
i Building waldo 0.2.4
v Built waldo 0.2.4 (3.9s)
v Installed waldo 0.2.4  (147ms)
i Building testthat 3.0.2
v Built testthat 3.0.2 (50s)
v Installed testthat 3.0.2  (381ms)
v 1 + 33 pkgs | kept 32, updated 2, new 0 | downloaded 2 (706.59 kB) 1m 27s

! pak had to unload some packages before installation, and the
  current R session may be unstable. It is best to restart R now.

@jimhester
Copy link
Member

As mentioned we have code in pak to help with this, and doing a similar thing in remotes would be challenging because we cannot use external packages. We don't plan on addressing this further in remotes, so I am going to close this issue.

@hdvinod
Copy link

hdvinod commented Aug 5, 2021

I also got \R\win-library\4.0\haven\libs\x64\haven.dll: Permission denied
problem when I try to install or update packages from R-GUI
However, the problem somehow disappears when I use the install / update buttons on R-studio, This is an easy workaround.

@gaborcsardi
Copy link
Member Author

gaborcsardi commented Aug 5, 2021 •
edited
Loading

@hdvinod for the record, that must have been a coincidence. E.g. maybe you started a fresh R session before pressing the install/update button. I don't think that just pressing the button works in all circumstances. Surely not if other R sessions are locking the files.

What surely works are:

  • closing down all R sessions, and opening up a new clean R session for the installation, without any loaded packages, and using the standalone mode of remotes, and
  • using pak.

@hdvinod
Copy link

hdvinod commented Aug 5, 2021 via email

@gaborcsardi
Copy link
Member Author

what is pak?

Not a stupid question at all, I am sorry for not including a link:
https://github.com/r-lib/pak#installation

Install as

install.packages("pak", repos = "https://r-lib.github.io/p/pak/dev/")

This is how it deals with the DLLs:
https://twitter.com/MilesMcBain/status/1364031217293684740

@allenbaron allenbaron mentioned this issue Feb 25, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature a feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@jimhester @gaborcsardi @dracodoc @kalibera @krlmlr @hdvinod @mattdowle @mirh