Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support package-manifest with over 1k packages #237

Closed
scottford-lw opened this issue Oct 30, 2020 · 2 comments · Fixed by #285
Closed

support package-manifest with over 1k packages #237

scottford-lw opened this issue Oct 30, 2020 · 2 comments · Fixed by #285
Assignees
@afiune
Labels
cli Something related to the Lacework CLI feat New feature or request

Comments

@scottford-lw
Copy link
Contributor

The api/v1/external/vulnerabilities/scan api has a limit of 1k per package manifest. It would be great if generate-pkg-manifest command could automatically break up the manifest into multiple files.

NOTE: Calls to this operation are rate limited to 10 calls per hour, per access key. If this rate limit is exceeded, an exception is thrown.
Also, note that this operation is limited to 1k of packages per payload. If you require a payload larger than 1k, you must make multiple requests.
For more information about creating an API access key and token to run this operation and using this operation with organization resources,
@afiune afiune changed the title Feature Request: generate-pkg-manifest should create multiple files when exceeding 1k packages generate-pkg-manifest should create multiple requests when exceeding 1k packages Dec 7, 2020
@afiune afiune changed the title generate-pkg-manifest should create multiple requests when exceeding 1k packages support package-manifest with over 1k packages Dec 7, 2020
@afiune afiune added feat New feature or request cli Something related to the Lacework CLI labels Dec 7, 2020
@afiune
Copy link
Contributor

afiune commented Dec 7, 2020

I think that it would be better if we make the scan-pkg-manifest command to support a single manifest file with over 1k packages. I feel that making the generate-pkg-manifest create multiple files will cause a bit more difficulties mainly because then the user would have to parse every file and submit multiple scans. Thoughts @scottford-lw ?

@scottford-lw
Copy link
Contributor Author

@afiune I agree that would be a better user experience. It would be great to get some feedback from a customer on this as well. Can you ask in customer slack with the customer that inspired this one?

@afiune afiune self-assigned this Jan 7, 2021
afiune added a commit that referenced this issue Jan 7, 2021
@afiune
feat(cli): support manifest bigger than 1k packages
eebddb9 
**User Story**

As a user of the Lacework CLI,
I would like to be able to submit scans of package manifests bigger than
1,000 packages,
So I don't have to implement a splitting mechanism and run multiple CLI
commands for a single manifest.

**Implementation Details**

The CLI will now check if the package manifest has more than the maximum
number of packages, if so, it will split the package manifest into
multiple chunks and trigger multiple API requests.

**NOTE:** We disallow more than 10 parallel requests (workers), which
are more than 10,000 packages on a single manifest/system.

Closes #237

Signed-off-by: Salim Afiune Maya <[email protected]>
@afiune afiune mentioned this issue Jan 7, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@afiune afiune

Labels
cli Something related to the Lacework CLI feat New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(cli): support manifest bigger than 1k packages lacework/go-sdk
2 participants
@afiune @scottford-lw