Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jf scan ./image.tar reporting no vulnerabilities in GitHub action #1645

Open
timdittler opened this issue Aug 10, 2022 · 17 comments
Open

jf scan ./image.tar reporting no vulnerabilities in GitHub action #1645

timdittler opened this issue Aug 10, 2022 · 17 comments
Labels
bug Something isn't working

Comments

@timdittler
Copy link

timdittler commented Aug 10, 2022

Describe the bug

  • If I scan image.tar directly after creation, it's recognized as Generic and doesn't show any vulnerability.
  • If I load it into docker and save it again, it'll work

To Reproduce

name: Scan

on:
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: setup buildx
        uses: docker/setup-buildx-action@v2

      - name: build image
        uses: docker/build-push-action@v3
        with:
          context: .
          file: ./Dockerfile
          platforms: linux/amd64
          build-args: |
            BUILDCACHE_BASEURL_ARG=${{ secrets.BUILDCACHE_BASEURL }}
            BUILDCACHE_PUSH_ENABLED_ARG=false
            BUILDCACHE_USER_ARG=${{ secrets.BUILDCACHE_USER }}
            BUILDCACHE_PASSWORD_ARG=${{ secrets.BUILDCACHE_PASSWORD }}
          pull: true
          cache-from: type=gha
          cache-to: type=gha,mode=max
          outputs: type=docker,dest=image.tar

      # works
      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/[email protected]
        with:
          input: image.tar
          trivy-config: trivy.yaml

      - uses: jfrog/setup-jfrog-cli@v2
        with:
          version: 2.24.1
        env:
          JF_ENV_1: ${{ secrets.JF_ENV_1 }}

      # doesn't work
      - run: |
          jf scan ./image.tar
        env:
          JFROG_CLI_LOG_LEVEL: DEBUG

      - uses: docker/login-action@v2
        with:
          registry: ${{ secrets.DOCKER_REGISTRY }}
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: load and push image
        run: |
          docker image load --input image.tar
          docker push ${{ secrets.DOCKER_REGISTRY }}/private/image:test

      # works
      - run: |
          jf docker scan ${{ secrets.DOCKER_REGISTRY }}/private/image:test
        env:
          JFROG_CLI_LOG_LEVEL: DEBUG

      - run: |
          container-diff diff daemon://${{ secrets.DOCKER_REGISTRY }}/private/image:test ./image.tar --type=history --type=file --type=size --type=apt

      - name: save
        run: |
          docker save ${{ secrets.DOCKER_REGISTRY }/private/image:test -o ./image2.tar

      # works
      - run: |
          jf scan ./image2.tar
        env:
          JFROG_CLI_LOG_LEVEL: DEBUG

results in

Run jf scan ./image.tar
  jf scan ./image.tar
  shell: /usr/bin/bash -e {0}
  env:
    JFROG_CLI_ENV_EXCLUDE: *password*;*secret*;*key*;*token*;*auth*;JF_ARTIFACTORY_*;JF_ENV_*
    JFROG_CLI_OFFER_CONFIG: false
    JFROG_CLI_BUILD_NAME: Vulnerability Scan
    JFROG_CLI_BUILD_NUMBER: 595
    JFROG_CLI_BUILD_URL: https://github.com/company/image/actions/runs/2840639249
    JFROG_CLI_USER_AGENT: setup-jfrog-cli-github-action/2.3.0
    JFROG_CLI_LOG_LEVEL: DEBUG
14:47:58 [Debug] Sending HTTP GET request to: https://company.jfrog.io/xray/api/v1/system/version
14:47:58 [Debug] Usage Report: Sending info...
14:47:58 [Debug] Sending HTTP GET request to: https://company.jfrog.io/artifactory/api/system/version
14:47:58 [Debug] Artifactory response: 200 OK
14:47:58 [Debug] The Artifactory version is: 7.41.7
14:47:58 [Debug] Sending HTTP POST request to: https://company.jfrog.io/artifactory/api/system/usage
14:47:58 [Info] JFrog Xray version is: 3.52.4
14:47:58 [Debug] Creating lock in:  /home/runner/.jfrog/locks/xray-indexer
14:47:58 [Info] JFrog Xray Indexer 3.52.4 is not cached locally. Downloading it now...
14:47:58 [Debug] Sending HTTP GET request to: https://company.jfrog.io/xray/api/v1/indexer-resources/download/linux/amd64
14:47:58 [Debug] Usage Report: Artifactory response: 200 OK
14:47:58 [Debug] Usage Report: Usage info sent successfully.
14:48:03 [Info] The downloaded Xray Indexer version is 3.52.4
14:48:03 [Debug] Releasing lock:  /home/runner/.jfrog/locks/xray-indexer/jfrog-cli.conf.lck.3298.1660229278636011461
14:48:03 [Info] [Thread 2] Indexing file: ./image.tar
14:48:07 [Info] 2022-08-11T14:48:04.17901381Z [jfxia] [DEBUG] [] [wire_gen:45                   ] [main                ] Initializing filtering service
2022-08-11T14:48:05.001930349Z [jfxia] [DEBUG] [] [indexer-app:43                ] [main                ] Indexing standalone file ./image.tar using artifactory folder /tmp/jfrog.cli.temp.-1660229283-497546448
2022-08-11T14:48:05.002024149Z [jfxia] [DEBUG] [] [indexer_app:109               ] [main                ] Local path: /tmp/jfrog.cli.temp.-1660229283-497546448/6b42c647-e44f-456e-5704-1e4bf803459f/166022928500202044/image.tar
2022-08-11T14:48:05.002052049Z [jfxia] [DEBUG] [] [indexer_app:109               ] [main                ] Scanning file from Artifactory with mimetype 'application/x-gzip'
2022-08-11T14:48:07.603531673Z [jfxia] [DEBUG] [] [indexer_app:109               ] [main                ] Found archive file. Performing deep scan for file /tmp/jfrog.cli.temp.-1660229283-497546448/6b42c647-e44f-456e-5704-1e4bf803459f/166022928500202044/image.tar
2022-08-11T14:48:07.693722778Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/1152061f1151c742af79b176c806bf8e72bfbfd110835efd6317fb8bb4d254e9
2022-08-11T14:48:07.697623578Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/16f15939bf55211a98fafa76f9e247e9d319e76c6e6d81c7a91b82becb0c00ba
2022-08-11T14:48:07.697675978Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/1fe172e4850f03bb45d41a20174112bc119fbfec42a650edbbd8491aee32e3c3
2022-08-11T14:48:07.706308678Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/44d3aa8d076675d49d85180b0ced9daef210fe4fdff4bdbb422b9cf384e591d0
2022-08-11T14:48:07.706817778Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/53d8f3c0b37abd925ba94b581a31d167ddaa2b3c5687aa8a7ceeca150e15496a
2022-08-11T14:48:07.706857378Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/6ce99fdf16e86bd02f6ad66a0e1334878528b5a4b5487850a76e0c08a7a27d56
2022-08-11T14:48:07.758768281Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/9a27270b63ac3a43a90311711576840fe278679291b26993abbe581e8c466f93
2022-08-11T14:48:07.758833181Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/c0ab546c23d0497649e47056be2521d9211721303e9487e8aacbe7aec6d7a747
2022-08-11T14:48:07.800972883Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable blobs/sha256/cf0532f0204bdb5f0d5a35e14592233e9db15d5f1ca9fb001a44095ba8c98b31
2022-08-11T14:48:07.801297183Z [jfxia] [DEBUG] [] [archive_mgr:1264              ] [main                ] Found archive file. Performing deep scan for file /tmp/jfrog.cli.temp.-1660229283-497546448/6b42c647-e44f-456e-5704-1e4bf803459f/166022928780111878/manifest.json
2022-08-11T14:48:07.801337683Z [jfxia] [DEBUG] [] [archive_mgr:232               ] [main                ] No classification found for manifest.json, classified as generic
2022-08-11T14:48:07.801358383Z [jfxia] [DEBUG] [] [archive_mgr:232               ] [main                ] manifest.json was classified as Generic
2022-08-11T14:48:07.801383483Z [jfxia] [DEBUG] [] [archive_mgr:232               ] [main                ] total running time for indexing tree construction of  manifest.json: 4.61e-05 seconds
2022-08-11T14:48:07.801440383Z [jfxia] [DEBUG] [] [archive_mgr:1245              ] [main                ] checking if the file is supported executable oci-layout
2022-08-11T14:48:07.801489283Z [jfxia] [DEBUG] [] [archive_mgr:232               ] [main                ] No classification found for image.tar, classified as generic
2022-08-11T14:48:07.801508783Z [jfxia] [DEBUG] [] [archive_mgr:232               ] [main                ] image.tar was classified as Generic
2022-08-11T14:48:07.801526683Z [jfxia] [DEBUG] [] [archive_mgr:232               ] [main                ] total running time for indexing tree construction of  image.tar: 5.29e-05 seconds
2022-08-11T14:48:07.801545683Z [jfxia] [DEBUG] [] [archive_mgr:195               ] [main                ] total running time for indexing image.tar: 0.19794911 seconds

14:48:07 [Debug] Sending HTTP POST request to: https://company.jfrog.io/xray/api/v1/scan/graph?scan_type=binary
14:48:08 [Info] Waiting for scan to complete...
14:48:08 [Debug] Sending HTTP GET request to: https://company.jfrog.io/xray/api/v1/scan/graph/84e868a0-f872-4530-76fb-b84f7e0bcb79?include_vulnerabilities=true
The full scan results are available here: /tmp/jfrog.cli.temp.-1660229288-2224644868
Note: no context was provided, so no policy could be determined to scan against.
14:48:08 [Info] Scan completed successfully.
You can get a list of custom violations by providing one of the command options: --watches, --repo-path or --project.
Read more about configuring Xray policies here: https://www.jfrog.com/confluence/display/JFROG/Creating+Xray+Policies+and+Rules
Below are all vulnerabilities detected.
+-------------------------------------+
| ✨ No vulnerabilities were found ✨ |
+-------------------------------------+

Expected behavior
Show all vulnerabilities, as on workstation

Versions

  • JFrog CLI version: 2.24.1
  • JFrog CLI operating system: ubuntu-latest
  • Artifactory Version: jfrog.io

Additional context

  • aquasecurity/trivy-action works as expected on the same image.tar
@timdittler timdittler added the bug Something isn't working label Aug 10, 2022
@sverdlov93
Copy link
Contributor

@timdittler,
Thanks for reaching out.
Few questions:

  • Do you run on the same OS and with the same Jfrog Platform server?

  • Can you please provide the JFrog CLI version? The version: 3.52.4 is the JFrog Xray version. CLI version can be found using: jf --version.

  • Can you run the git action with the latest CLI version? Can be achieved by:

- uses: jfrog/setup-jfrog-cli@v2
  with:
    version: latest
  • Can you add the env - JFROG_CLI_LOG_LEVEL: DEBUG before running the scan command to show more log information?
   - uses: jfrog/setup-jfrog-cli@v2
        env:
          JF_ENV_1: ${{ secrets.JF_ENV_1 }}
         JFROG_CLI_LOG_LEVEL: DEBUG

@sverdlov93
Copy link
Contributor

Also just to let you know, we recently introduced the jf docker scan, which creates a tar file and scans it with one command: jf docker scan centos:latest

And another nice and easy way to scan docker images is using our new JFrog Docker Desktop Extension, available on your local docker desktop app.

@timdittler
Copy link
Author

Thanks for your comment @sverdlov93 . I tried many different things. Right now, I believe something is off with my image creation process. I'll investigate and re-open this ticket if necessary.

@timdittler
Copy link
Author

I dug a bit deep and come up with the example above. It's actually not about GH Actions vs. Workstation. I really don't know what's the problem. But jf scan won't detect anything in the first try, but on all the ones after importing it to docker. So I guess it's a problem of jf. Sadly, I can't share my image.tar.

@timdittler
Copy link
Author

This is beginning of the log of the second run with jf scan:

2022-08-11T14:50:30.0059600Z ##[group]Run jf scan ./image2.tar
2022-08-11T14:50:30.0059915Z �[36;1mjf scan ./image2.tar�[0m
2022-08-11T14:50:30.0113850Z shell: /usr/bin/bash -e {0}
2022-08-11T14:50:30.0114118Z env:
2022-08-11T14:50:30.0114471Z   JFROG_CLI_ENV_EXCLUDE: *password*;*secret*;*key*;*token*;*auth*;JF_ARTIFACTORY_*;JF_ENV_*
2022-08-11T14:50:30.0114850Z   JFROG_CLI_OFFER_CONFIG: false
2022-08-11T14:50:30.0115159Z   JFROG_CLI_BUILD_NAME: Vulnerability Scan
2022-08-11T14:50:30.0115478Z   JFROG_CLI_BUILD_NUMBER: 595
2022-08-11T14:50:30.0115859Z   JFROG_CLI_BUILD_URL: https://github.com/company/image/actions/runs/2840639249
2022-08-11T14:50:30.0116298Z   JFROG_CLI_USER_AGENT: setup-jfrog-cli-github-action/2.3.0
2022-08-11T14:50:30.0116633Z   JFROG_CLI_LOG_LEVEL: DEBUG
2022-08-11T14:50:30.0116896Z ##[endgroup]
2022-08-11T14:50:30.0269934Z 14:50:30 [Debug] Sending HTTP GET request to: https://company.jfrog.io/xray/api/v1/system/version
2022-08-11T14:50:30.0283730Z 14:50:30 [Debug] Usage Report: Sending info...
2022-08-11T14:50:30.0429139Z 14:50:30 [Debug] Sending HTTP GET request to: https://company.jfrog.io/artifactory/api/system/version
2022-08-11T14:50:30.4992161Z 14:50:30 [Info] JFrog Xray version is: 3.52.4
2022-08-11T14:50:30.4999911Z 14:50:30 [Debug] Creating lock in:  /home/runner/.jfrog/locks/xray-indexer
2022-08-11T14:50:30.5000571Z 14:50:30 [Debug] Releasing lock:  /home/runner/.jfrog/locks/xray-indexer/jfrog-cli.conf.lck.3542.1660229430499120246
2022-08-11T14:50:30.5001021Z 14:50:30 [Info] [Thread 2] Indexing file: ./image2.tar
2022-08-11T14:50:30.5114204Z 14:50:30 [Debug] Artifactory response: 200 OK
2022-08-11T14:50:30.5114887Z 14:50:30 [Debug] The Artifactory version is: 7.41.7
2022-08-11T14:50:30.5116634Z 14:50:30 [Debug] Sending HTTP POST request to: https://company.jfrog.io/artifactory/api/system/usage
2022-08-11T14:50:30.8857105Z 14:50:30 [Debug] Usage Report: Artifactory response: 200 OK
2022-08-11T14:50:30.8857952Z 14:50:30 [Debug] Usage Report: Usage info sent successfully.
2022-08-11T14:50:56.8747634Z 14:50:56 [Info] 2022-08-11T14:50:30.697780082Z �[33m[jfxia]�[0m [DEBUG] [] [wire_gen:45                   ] [main                ] Initializing filtering service
2022-08-11T14:50:56.8748627Z 2022-08-11T14:50:31.525523231Z �[33m[jfxia]�[0m [DEBUG] [] [indexer-app:43                ] [main                ] Indexing standalone file ./image2.tar using artifactory folder /tmp/jfrog.cli.temp.-1660229430-862749059
2022-08-11T14:50:56.8749640Z 2022-08-11T14:50:31.525635931Z �[33m[jfxia]�[0m [DEBUG] [] [indexer_app:109               ] [main                ] Local path: /tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943152563193/image2.tar
2022-08-11T14:50:56.8750534Z 2022-08-11T14:50:31.525666231Z �[33m[jfxia]�[0m [DEBUG] [] [indexer_app:109               ] [main                ] Scanning file from Artifactory with mimetype 'application/x-gzip'
2022-08-11T14:50:56.8751504Z 2022-08-11T14:50:35.314667199Z �[33m[jfxia]�[0m [DEBUG] [] [indexer_app:109               ] [main                ] Found archive file. Performing deep scan for file /tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943152563193/image2.tar
2022-08-11T14:50:56.8753260Z 2022-08-11T14:50:38.824982961Z �[33m[jfxia]�[0m [DEBUG] [] [tar:82                        ] [main                ] Docker image manifest scanning File: [Id=7246466352339916359, name=/***/private/image/timdittlertest/manifest.json, path=/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943882472426/, mime=application/x-docker, sha256=9dbd75b60387daa107389f7d67ce1e491b876e1ae2c150ff6da946ae9f57df54, parent=9dbd75b60387daa107389f7d67ce1e491b876e1ae2c150ff6da946ae9f57df54, childrens=0]
2022-08-11T14:50:56.8763332Z 2022-08-11T14:50:38.825214061Z �[33m[jfxia]�[0m [DEBUG] [] [tar:82                        ] [main                ] docker layers on message {"messageId":"bcf7deea-4476-4386-6d2a-9472bae16341","eventType":"","downloadUrl":"onDemand","artifactoryId":"","repoKey":"","repoPkgType":"","path":"/***/private/image/timdittlertest/manifest.json","checksums":{"md5":"1fb52bb5821f4174cf9a2ce14488467c","sha1":"50ac4d340ee9071331ec56ce207503434bfb3fa8","sha256":"9dbd75b60387daa107389f7d67ce1e491b876e1ae2c150ff6da946ae9f57df54"},"archivePath":"/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943882472426/manifest.json","downloadedDockerArchive":{"onDemand":{"DockerArchivesArray":[{"ArchiveName":"sha256__82b15c83930edda93acf7653ecdbad3507061f40d704bb1e5ada03a2c7a29e80.tar","MediaType":"","Sha":"82b15c83930edda93acf7653ecdbad3507061f40d704bb1e5ada03a2c7a29e80","DownloadLink":"","SavedFilePath":"/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943531482009/45cb19a6236cea8ff70ec30070835a33a6946fd3962706f792455aa35cae1b6e/sha256__82b15c83930edda93acf7653ecdbad3507061f40d704bb1e5ada03a2c7a29e80.tar"},{"ArchiveName":"sha256__e8df80afc57d1d3442ee9c057b6c90d7750d1cee47c9eb3e1c8b5c462fd7de35.tar","MediaType":"","Sha":"e8df80afc57d1d3442ee9c057b6c90d7750d1cee47c9eb3e1c8b5c462fd7de35","DownloadLink":"","SavedFilePath":"/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943628902755/612270a8219d297da7713f799e39f8c7c12a3499893a3bc20c686cb72fc7652c/sha256__e8df80afc57d1d3442ee9c057b6c90d7750d1cee47c9eb3e1c8b5c462fd7de35.tar"},{"ArchiveName":"sha256__9c1b6dd6c1e6be9fdd2b1987783824670d3b0dd7ae8ad6f57dc3cea5739ac71e.tar","MediaType":"","Sha":"9c1b6dd6c1e6be9fdd2b1987783824670d3b0dd7ae8ad6f57dc3cea5739ac71e","DownloadLink":"","SavedFilePath":"/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943639807257/6e3692f03eefee9a819cd5ef747d7331ec6a1e20b65d5eb648b923469e74377e/sha256__9c1b6dd6c1e6be9fdd2b1987783824670d3b0dd7ae8ad6f57dc3cea5739ac71e.tar"},{"ArchiveName":"sha256__1390fc6b2a9fa968c8d267d5cb10dcba10c44616f2d221fcefcf495499dfa570.tar","MediaType":"","Sha":"1390fc6b2a9fa968c8d267d5cb10dcba10c44616f2d221fcefcf495499dfa570","DownloadLink":"","SavedFilePath":"/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943689760635/77a833cd5f5c2b28b07eb05fc731f6afdf19996ebba3cd2a12ea719adf4c6115/sha256__1390fc6b2a9fa968c8d267d5cb10dcba10c44616f2d221fcefcf495499dfa570.tar"},{"ArchiveName":"sha256__13a34b6fff7804cf7f6e8f52a4cf25ceb2e32fc35a6f39e8158074c64831ebf0.tar","MediaType":"","Sha":"13a34b6fff7804cf7f6e8f52a4cf25ceb2e32fc35a6f39e8158074c64831ebf0","DownloadLink":"","SavedFilePath":"/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943689811575/884857678af1292ad90fdf1a9c84a468d220fb6b5d5133692d3810da5274dd56/sha256__13a34b6fff7804cf7f6e8f52a4cf25ceb2e32fc35a6f39e8158074c64831ebf0.tar"},{"ArchiveName":"sha256__6be690267e47ddcfd965449d2af70a9eca9879f9436948ee83d7f4ad473b8e64.tar","MediaType":"","Sha":"6be690267e47ddcfd965449d2af70a9eca9879f9436948ee83d7f4ad473b8e64","DownloadLink":"","SavedFilePath":"/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-9472bae16341/166022943692851615/db9e8c3bbea76eab9355035aa6bbe9453249192305f7e97756bbd67e943cd698/sha256__6be690267e47ddcfd965449d2af70a9eca9879f9436948ee83d7f4ad473b8e64.tar"},{"ArchiveName":"sha256__fba45aa3e67564317c0e0d31e5c7cad5a2e2b01a672251cb73b1ea0bbeb62423.tar","MediaType":"","Sha":"fba45aa3e67564317c0e0d31e5c7cad5a2e2b01a672251cb73b1ea0bbeb62423","DownloadLink":"","SavedFilePath":"/tmp/jfrog.cli.temp.-1660229430-862749059/bcf7deea-4476-4386-6d2a-

Differences begin after it detects a different mime type. Could this be the cause?!

@sverdlov93
Copy link
Contributor

@timdittler
Are you sure that the output from your build command is image.tar tar file and not a directory with the name image.tar?

@timdittler
Copy link
Author

timdittler commented Aug 11, 2022 via email

@timdittler
Copy link
Author

Really is an archive. Output from runner:

ls -l ./image.tar
file ./image.tar
  shell: /usr/bin/bash -e {0}
-rw-r--r-- 1 runner docker 384894976 Aug 15 07:23 ./image.tar
./image.tar: POSIX tar archive

@sverdlov93
Copy link
Contributor

Hi @timdittler ,
Can you reproduce that also using docker build on docker cli on your local machine?
I am trying to understand the difference between the tar and the manifest.json created by 'docker save' that we support correctly,
and the tar created by the docker build with output flag.

@timdittler
Copy link
Author

Sorry, my jfrog trial ran out and I have no possiblity to test this anymore.

@sverdlov93
Copy link
Contributor

Hi @timdittler,
You can always create a free tier account https://jfrog.com/start-free/#saas without any time limits.
You can also create one from your CLI using the following command:
curl -fL "https://getcli.jfrog.io?setup" | sh

@timdittler
Copy link
Author

I'm now running into the same problem with version 2.5.0

jf scan --watches service service.tar
  shell: /usr/bin/bash -e {0}
  env:
    JAVA_HOME: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.8-7/x64
    JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.8-7/x64
    LD_PRELOAD: /usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4
    JFROG_CLI_ENV_EXCLUDE: *password*;*secret*;*key*;*token*;*auth*;JF_ARTIFACTORY_*;JF_ENV_*;JF_URL;JF_USER;JF_PASSWORD;JF_ACCESS_TOKEN
    JFROG_CLI_OFFER_CONFIG: false
    JFROG_CLI_BUILD_NAME: Vulnerability Scan
    JFROG_CLI_BUILD_NUMBER: 9137
    JFROG_CLI_BUILD_URL: https://github.com/Staffbase/service/actions/runs/5854750056
    JFROG_CLI_USER_AGENT: setup-jfrog-cli-github-action/3.3.0
10:44:05 [Info] JFrog Xray version is: 3.79.11
10:44:05 [Info] JFrog Xray Indexer 3.79.11 is not cached locally. Downloading it now...
2023/08/14 10:44:10 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
10:44:10 [Info] The downloaded Xray Indexer version is 3.79.11
10:44:10 [Info] [Thread 2] Indexing file: service.tar
10:44:16 [Info] 2023/08/14 10:44:10 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
2023-08-14T10:44:14.980Z [jfxia] [WARN ] [] [docker_tar:74                 ] [main                ] Failed to index tar file as container image, continue to generic tar indexer. Error: failed to analyze OCI tar archive
 --- at /go/src/jfrog.com/xray/service/indexer/indexer_core/docker_tar.go:144 (DockerTarOpener.analyzeTarAsContainer) ---
Caused by: failed to parse and validate manifests list: index.json
 --- at /go/src/jfrog.com/xray/service/indexer/indexer_core/oci_tar.go:53 (DockerTarOpener.handleIndexFile) ---
Caused by: manifest does not contain annotation: org.opencontainers.image.ref.name
 --- at /go/src/jfrog.com/xray/service/indexer/indexer_core/oci_tar.go:89 (DockerTarOpener.parseAndValidateManifestsList) ---
2023-08-14T10:44:15.575Z [jfxia] [WARN ] [] [archive_mgr:282               ] [main                ] Archive manifest.json exceeded internal depth limitation, extraction stopped.

10:44:16 [Info] Waiting for scan to complete on JFrog Xray...

 The full scan results are available here: /tmp/jfrog.cli.temp.-1692009857-247050634
10:44:17 [Info] Scan completed successfully.

+-----------------------------------+
| No security violations were found |
+-----------------------------------+
+---------------------------------------------+
| No license compliance violations were found |
+---------------------------------------------+

The service.tar was build with docker/build-push-action.

      - name: Build image
        uses: docker/build-push-action@v4
        with:
          context: .
          file: ./Dockerfile
          platforms: linux/amd64
          pull: true
          cache-from: type=gha
          cache-to: type=gha,mode=max
          outputs: type=docker,dest=service.tar

@yahavi
Copy link
Member

yahavi commented Aug 14, 2023

@timdittler
Could you please try to add the --bypass-archive-limits flag?

jf scan --watches service service.tar --bypass-archive-limits

@timdittler
Copy link
Author

Sadly, no change in behavior

@yahavi
Copy link
Member

yahavi commented Aug 14, 2023

What is your JFrog CLI version, @timdittler?
We added the support for this flag in 2.28.2.

@timdittler
Copy link
Author

I'm now loading the image to work around the problem

      - name: Build image
        uses: docker/build-push-action@v4
        with:
          context: .
          file: ./Dockerfile
          platforms: linux/amd64
          pull: true
          cache-from: type=gha
          cache-to: type=gha,mode=max
          outputs: type=docker,dest=service.tar
          tags: service

      - name: Load image
        run: docker load -i service.tar

      - name: Setup JFrog CLI
        uses: jfrog/setup-jfrog-cli@v3
        env:
          JF_ENV_1: ${{ secrets.JF_ENV_1 }}

      - name: Run vulnerability scanner
        run: jf docker scan --watches service --format json service

@guyshe-jfrog
Copy link

Related: PR to support docker scan from tar directly:
jfrog/jfrog-cli-security#30

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants