Download only subdirectory for GitHub packages when appropriate

[jashapiro]

When installing a package from GitHub where the package is in a subdirectory of the main repository, renv::install() downloads the entire repository. For some repositories this may be a non-issue, but if the repository is large (and the R package is only a small part of the repository), this may result in a long, slow download.

Might it be possible to implement a sparse-checkout or equivalent to reduce downloads and improve install times in this situation?

[kevinushey]

It should be possible, but I'm not sure if there's anything made available by the GitHub API. We could implement support for this when installing packages via git remotes though, e.g.

renv::install("git::[email protected]:user/repo.git:subdir")

For reference, here's the git command we generate to check out sources from a particular repository.

renv/R/retrieve.R

Lines 499 to 504 in 1c8c64c

	template <- heredoc('
	git init ${QUIET}
	git remote add origin "${ORIGIN}"
	git fetch ${QUIET} --depth=1 origin "${REF}"
	git reset ${QUIET} --hard FETCH_HEAD
	')

All that said -- it's also possible that R packages within a project sub-directory might still depend on files outside of that sub-directory, so I'm not sure if this is something we could do by default.

[jashapiro]

It should be possible, but I'm not sure if there's anything made available by the GitHub API. We could implement support for this when installing packages via git remotes though, e.g.

renv::install("git::[email protected]:user/repo.git:subdir")

For reference, here's the git command we generate to check out sources from a particular repository.

renv/R/retrieve.R

Lines 499 to 504 in 1c8c64c

	template <- heredoc('
	git init ${QUIET}
	git remote add origin "${ORIGIN}"
	git fetch ${QUIET} --depth=1 origin "${REF}"
	git reset ${QUIET} --hard FETCH_HEAD
	')

All that said -- it's also possible that R packages within a project sub-directory might still depend on files outside of that sub-directory, so I'm not sure if this is something we could do by default.

Yes, after some investigation on other fronts, I am not sure it is a great idea, or even a particularly useful one. When I was trying to figure out how this might actually work, I found that the .git directory for the repository I was working with which prompted this request was quite large on its own. So the best solution ended up being to break out the R package to its own repository anyway.

[kevinushey]

Thanks! In that case, I think this issue can be closed? Let me know if there's yet anything else I can improve on the renv side.