Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/cue: the directory tree structure of the modules cache is inconsistent #3139

Closed
mvdan opened this issue May 10, 2024 · 1 comment
Closed

cmd/cue: the directory tree structure of the modules cache is inconsistent #3139

mvdan opened this issue May 10, 2024 · 1 comment
Assignees
@mvdan
Labels
modules Issues related to CUE modules and the experimental implementation

Comments

@mvdan
Copy link
Member

mvdan commented May 10, 2024

After cleaning my entire ~/.cache/cue and running cue mod tidy on a module which depends on another module, I see:

$ find .cache/cue -type f
.cache/cue/cache/download/github.com/mvdan/test-repo/@v/v0.0.11.lock
.cache/cue/cache/download/github.com/mvdan/test-repo/@v/v0.0.11.zip
.cache/cue/cache/download/github.com/mvdan/test-repo/@v/v0.0.11.mod
.cache/cue/github.com/mvdan/[email protected]/cue.mod/module.cue
.cache/cue/github.com/mvdan/[email protected]/foo.cue
  • ~/.cache/cue/cache/download is unnecessarily repetitive; we should not insert a "cache" directory inside... CUE's cache directory.
  • The unzipped modules are extracted straight inside CUE's cache directory, causing namespace issues for future kinds of caches we might want to use.

My thinking is that we should use ${CUE_CACHE_DIR}/mod/download/${modpath} for downloaded artifacts (e.g. zip files), and ${CUE_CACHE_DIR}/mod/extract/${modpath} for the contents of extracted archives. This way they are nicely grouped together, and there is some hierarchy that describes their contents as well.
@mvdan mvdan added the modules Issues related to CUE modules and the experimental implementation label May 10, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Release v0.9 May 10, 2024
@mvdan mvdan mentioned this issue May 11, 2024
Open
@myitcv myitcv moved this from Backlog to v0.9.0-rc.1 in Release v0.9 May 12, 2024
@myitcv myitcv moved this from Backlog to Planned in Modules Roadmap May 12, 2024
@myitcv myitcv mentioned this issue May 12, 2024
Closed
@myitcv myitcv moved this from v0.9.0-alpha.5 to v0.9.0-rc.1 in Release v0.9 May 15, 2024
@myitcv
Copy link
Member

myitcv commented May 28, 2024

My thinking is that we should use ${CUE_CACHE_DIR}/mod/download/${modpath} for downloaded artifacts (e.g. zip files), and ${CUE_CACHE_DIR}/mod/extract/${modpath} for the contents of extracted archives. This way they are nicely grouped together, and there is some hierarchy that describes their contents as well.

This sounds good to me.

@mvdan mvdan assigned mvdan and unassigned rogpeppe May 30, 2024
cueckoo pushed a commit that referenced this issue May 30, 2024
@mvdan
cmd/cue: check the structure of CUE_CACHE_DIR after downloading modules
81522e2 
This acts as a nice regression test and visualization, and we will see
the impact of future changes like https://cuelang.org/issue/3139.

Note that ./mod/modcache already covers the presence of specific cache
files via its Go unit tests, but this added testscript check is a nice
complement which covers all files and shows the entire structure.

For #3139.

Signed-off-by: Daniel Martí <[email protected]>
Change-Id: I2ffa32f7ccb44b52c0bfad1bfe56f93a450d5a70
cueckoo pushed a commit that referenced this issue May 30, 2024
@mvdan
cmd/cue: use a consistent directory structure for the modules cache
8a29fb4 
Downloads used to go to ${CUE_CACHE_DIR}/cache/download,
such as ~/.cache/cue/cache/download, which often repeated "cache"
and wasn't clear about it relating to the caching of modules.

Extracted module archives went directly under ${CUE_CACHE_DIR},
which also didn't clearly hint at a modules cache.
Worse, by not using a namespace, it made the top-level ${CUE_CACHE_DIR}
less useful and more confusing for other caching needs in the future.

Now both are under ${CUE_CACHE_DIR}/mod:

* ${CUE_CACHE_DIR}/mod/download for downloaded modules

* ${CUE_CACHE_DIR}/mod/extract for extracted modules

Note that this does make absolute file paths for extracted modules
slightly longer by twelve characters, but that seems an OK tradeoff.

Update the testscript and unit tests accordingly.

Fixes #3139.

Signed-off-by: Daniel Martí <[email protected]>
Change-Id: I12d5926b9afc9242d056839855ff51b58545295a
cueckoo pushed a commit that referenced this issue May 30, 2024
@mvdan
cmd/cue: check the structure of CUE_CACHE_DIR after downloading modules
07e1c14 
This acts as a nice regression test and visualization, and we will see
the impact of future changes like https://cuelang.org/issue/3139.

Note that ./mod/modcache already covers the presence of specific cache
files via its Go unit tests, but this added testscript check is a nice
complement which covers all files and shows the entire structure.

For #3139.

Signed-off-by: Daniel Martí <[email protected]>
Change-Id: I2ffa32f7ccb44b52c0bfad1bfe56f93a450d5a70
cueckoo pushed a commit that referenced this issue May 30, 2024
@mvdan
cmd/cue: use a consistent directory structure for the modules cache
adf9cb2 
Downloads used to go to ${CUE_CACHE_DIR}/cache/download,
such as ~/.cache/cue/cache/download, which often repeated "cache"
and wasn't clear about it relating to the caching of modules.

Extracted module archives went directly under ${CUE_CACHE_DIR},
which also didn't clearly hint at a modules cache.
Worse, by not using a namespace, it made the top-level ${CUE_CACHE_DIR}
less useful and more confusing for other caching needs in the future.

Now both are under ${CUE_CACHE_DIR}/mod:

* ${CUE_CACHE_DIR}/mod/download for downloaded modules

* ${CUE_CACHE_DIR}/mod/extract for extracted modules

Note that this does make absolute file paths for extracted modules
slightly longer by twelve characters, but that seems an OK tradeoff.

Update the testscript and unit tests accordingly.

Fixes #3139.

Signed-off-by: Daniel Martí <[email protected]>
Change-Id: I12d5926b9afc9242d056839855ff51b58545295a
cueckoo pushed a commit that referenced this issue May 30, 2024
@mvdan
cmd/cue: check the structure of CUE_CACHE_DIR after downloading modules
d94b670 
This acts as a nice regression test and visualization, and we will see
the impact of future changes like https://cuelang.org/issue/3139.

Note that ./mod/modcache already covers the presence of specific cache
files via its Go unit tests, but this added testscript check is a nice
complement which covers all files and shows the entire structure.

For #3139.

Signed-off-by: Daniel Martí <[email protected]>
Change-Id: I2ffa32f7ccb44b52c0bfad1bfe56f93a450d5a70
Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/1195517
Unity-Result: CUE porcuepine <[email protected]>
Reviewed-by: Paul Jolly <[email protected]>
TryBot-Result: CUEcueckoo <[email protected]>
@github-project-automation github-project-automation bot moved this from Planned to Done in Modules Roadmap May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mvdan mvdan

Labels
modules Issues related to CUE modules and the experimental implementation
Projects
Modules Roadmap
Archived in project
Release v0.9
Status: v0.9.0-rc.1
Milestone
No milestone
Development

No branches or pull requests
3 participants
@rogpeppe @myitcv @mvdan