Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query for big repo is too slow. You can use AQL for get ALL folder\files in one get-requests #2

Open
allburov opened this issue Aug 25, 2017 · 2 comments

Comments

@allburov
Copy link

Thanks for the tool!

Query for big repo is too slow. You can use Artifactory-AQL for get ALL folder\files in one get-requests, and process after.

With https://github.com/devopshq/artifactory:

from artifactory import ArtifactoryPath
aql = ArtifactoryPath( "http://my-artifactory/artifactory") # path to artifactory, NO repo

# dict 
resp = aql.aql("items.find", {"repo": "myrepo", "type": "any"})

# process after
@reversefold
Copy link
Owner

I use this script as a Jenkins job which runs regularly so we always have a mostly up-to-date version of the output at any given time.

I'd be wary of using an API call which returns all artifacts at once as it could be very large and potentially cause timeouts or memory issues.

@reversefold
Copy link
Owner

We also run this on a repository with more than 100,000 artifacts totalling more than 1TB and it tends to finish fairly quickly. You can increase the number of workers with -n to try to make it faster too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants