Archive file backward compatibility

[lafriks]

When restoring archive files created with :latest tag with :master-ef42beb version there is error when extracting archive file:

level=info name=drone-cache ts=2020-04-08T09:36:06.836811454Z caller=restorer.go:95 component=plugin component=restorer msg="extracting archived directory" remote=xxx_7f8f9564d852af06d95e5156aa50a2ed_f024307c4da35ea0c40eb17b8730c470/src/xxx/web/node_modules local=src/xxx/web/node_modules
level=error name=drone-cache ts=2020-04-08T09:36:07.372433303Z caller=main.go:581 err="[IMPORTANT] restore cache, restore failed, download from <xxx_7f8f9564d852af06d95e5156aa50a2ed_f024307c4da35ea0c40eb17b8730c470/src/xxx/web/node_modules> to <src/xxx/web/node_modules>, extract files from downloaded archive, pipe reader failed, relative name, relative path <>, base <node_modules>, Rel: can't make /go/src/src/xxx/web relative to src/xxx/web/node_modules\n"

To me it looks like previously files in archive was stored with full path?

WORKAROUND: override with rebuild or delete cached files

[kakkoyun]

Let's not block the release with this issue which is already delayed considerably. @lafriks already provided a workaround.
We're also planning the introduce a cache clear/delete/flush functionality to easily work around this issue.
I have also added a known issue tag to this to keep around.

Thanks a lot for reporting this @lafriks

[cmays20]

@kakkoyun I am getting this same error with the current :latest tag: sha256:b52a859bebdb55d14b85cd966ffcca1c86353ade93ce58603fc7d0c3ea233026. My archive was just rebuilt with the same image, so I am not sure this only related to backwards compatibility. Here are the ENV variables I am using (running this in k8s):

        - name: AWS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: s3-config
              key: AWS_ACCESS_KEY_ID
        - name: AWS_SECRET_ACCESS_KEY
          valueFrom:
            secretKeyRef:
              name: s3-config
              key: AWS_SECRET_ACCESS_KEY
        - name: PLUGIN_PATH_STYLE
          value: 'true'
        - name: PLUGIN_MOUNT
          value: /root/.m2
        - name: PLUGIN_BUCKET
          value: artifacts
        - name: PLUGIN_ENDPOINT
          value: 'http://s3-us-east-1.dispatch'
        - name: PLUGIN_REGION
          value: us-east-1
        - name: PLUGIN_CACHE_KEY
          value: hello-world
        - name: PLUGIN_RESTORE
          value: 'true'

And here is the error message:

level=info name=drone-cache ts=2020-06-12T06:10:45.873690358Z caller=restorer.go:55 component=plugin component=restorer msg="restoring directory" local=/root/.m2 remote=hello-world/root/.m2
level=info name=drone-cache ts=2020-06-12T06:10:45.873727095Z caller=restorer.go:96 component=plugin component=restorer msg="extracting archived directory" remote=hello-world/root/.m2 local=/root/.m2
level=info name=drone-cache ts=2020-06-12T06:10:45.873752928Z caller=restorer.go:87 component=plugin component=restorer msg="downloading archived directory" remote=hello-world/root/.m2 local=/root/.m2
level=error name=drone-cache ts=2020-06-12T06:10:45.876894565Z caller=main.go:587 err="[IMPORTANT] restore cache, restore failed, download from <hello-world/root/.m2> to </root/.m2>, extract files from downloaded archive, pipe reader failed, relative name, relative path <>, base <.m2>, Rel: can't make root relative to /root/.m2\n"

Let me know if you see anything obvious or if you need more debug information. FYI, this all worked in previous versions.

[cmays20]

I can confirm it works if I specify tag 1.0.4 (obviously I should be specifying a tag to be safe anyway).

2020/06/12 06:18:58 [IMPORTANT] using aws s3 as backend
2020/06/12 06:18:58 using provided cache key template
2020/06/12 06:18:58 restoring directory </root/.m2> from remote cache <hello-world/root/.m2>
2020/06/12 06:18:58 dowloading archived directory <hello-world/root/.m2>
2020/06/12 06:18:58 extracting archived directory <hello-world/root/.m2> to </root/.m2>
2020/06/12 06:18:58 cache restored in 165.1148ms

[kakkoyun]

@cmays20 Thanks for the detailed report. Yesterday, I have released the a new version v1.1.0 so latest points this version, which actually includes this known issue. Since there's an easy, known fix I've decided to go ahead and release it. Sorry for the inconvenience.
If the work around doesn't work or it's not manageable please let me know, I can work on a remedy.

[cmays20]

@kakkoyun can you explain the workaround better? I already tried deleting the cached files and rebuilding them. However, I can't get a restore to work with my settings.

[kakkoyun]

Hey @cmays20, I realized when I've checked your logs the problem you've is different than what described here. You are trying to cache a directory that is outside the workspace. Which was accidentally made possible with the previous the previous release, we had to fix as a remedy to another problem.

Let's open an another issue for this and if the path starts with /, we can introduce a new absolute path mode to enable this behaviour.

For now, the best workaround would be to copy those files to workspace before rebuilding cache and move them their appropriate place after rebuild.

Sorry for the inconvenience.