cache with concurrency

[po3rin]

Introduce concurrency because cache will be slower by the number of projects.

You can check the progress of parallel processing with the progressbar.

$ bqiam cache
[1/3][project-1] caching datasets... 23 % [======>-----------------------]  10s
[2/3][project-2] caching datasets... 28 % [=======>----------------------]   9s
[3/3][project-3] caching datasets... 27 % [=======>----------------------]   8s

demo

performance

Measured by the amount of practical data.

• before add progress bar #22 (total: 103s)
• after (total: 46s)

check

Because parallel processing is included, I checked it with -race just in case

$ go run -race main.go cache

[hirosassa]

What a short life, haha.

[po3rin]

schollz/progressbar mantainer not supports multi line bar ...
schollz/progressbar#6

[hirosassa]

LGTM!