Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dvc: command to list file/dir versions/history #3616

Closed
nabil-aitbaha opened this issue Apr 9, 2020 · 4 comments
Closed

dvc: command to list file/dir versions/history #3616

nabil-aitbaha opened this issue Apr 9, 2020 · 4 comments
Labels
feature request Requesting a new feature p3-nice-to-have It should be done this or next sprint

Comments

@nabil-aitbaha
Copy link

Actually to see versions of a DVC file, we need to use "git log" but this command need to clone the repository before. So if I need to import a specific version of a dataset on an another project, I need to clone the entire repository.
A better way should to use a command like "dvc list --versions url".

@triage-new-issues triage-new-issues bot added the triage Needs to be triaged label Apr 9, 2020
@nabil-aitbaha nabil-aitbaha changed the title Add a new option for "--versions" for the command dvc list. This option will permit to list all git tags associated to a DVCFile (tracked on Git) Add a new option "--versions" for the command dvc list. This option will permit to list all git tags associated to a DVCFile (tracked on Git) Apr 9, 2020
@shcheklein shcheklein added the feature request Requesting a new feature label Apr 9, 2020
@triage-new-issues triage-new-issues bot removed the triage Needs to be triaged label Apr 9, 2020
@efiop
Copy link
Contributor

efiop commented Apr 9, 2020

For the record: would probably reconsider adding this to the list itself, feels like it doesn't belong there. Maybe we need a different command for such functionality.

@efiop
Copy link
Contributor

efiop commented Apr 9, 2020

For now git log my.dvc does the job. Discord context: https://discordapp.com/channels/485586884165107732/563406153334128681/697826620601532496

@efiop efiop added the p3-nice-to-have It should be done this or next sprint label Apr 9, 2020
@efiop efiop changed the title Add a new option "--versions" for the command dvc list. This option will permit to list all git tags associated to a DVCFile (tracked on Git) dvc: command to list file/dir versions/history Apr 9, 2020
@tall-josh
Copy link

On a similar note, I've been doing the following. Say I have some Tensorboard logs I wish to track with dvc. That's great, but Tensorboard also allows you to specify many separate logdirs so you can visualize multiple runs at once. What I have been doing recently is generating a history directory that contains symlinks to files in .dvc/cache.

ie, If I had a directory structure like this:

myproject/
  artefacts/
    tf-logs/        # Track with dvc
    metrics/
      train.yaml    # Track file with dvc
      eval.taml     # Track file with dvc

Then assuming I have 2 git tags run1 and run2 with corresponding tracking information in .dvc files I generate a history dir:

history/
  artefacts/
    tf-logs/
      run1/    # Recreate directory structure of tf-log dir where the bottom 
               # of the tree consists of symlinks to files in `.dvc/cache` from
               # git tag `run1`
      run2/    # Same but for git tag `run2`
    metrics/
      train.yaml/
        run1   # Symlink to `.dvc/cache` file from `run1`
        run2   # Symlink to `.dvc/cache` file from `run2`
      eval.yaml/
        run1   # Symlink to `.dvc/cache` file from `run1`
        run2   # Symlink to `.dvc/cache` file from `run2`

I've been doing this with git show $TAG --name-only | grep .dvc and git show $TAG:$DVCFILE

@nabil-aitbaha
Copy link
Author

@tall-josh Yes it is a good idea thanks 👍

@efiop efiop closed this as not planned Won't fix, can't repro, duplicate, stale Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Requesting a new feature p3-nice-to-have It should be done this or next sprint
Projects
None yet
Development

No branches or pull requests

4 participants