-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: include metrics about number of custom rules [CFG-1133] #121
base: main
Are you sure you want to change the base?
Conversation
34bec46
to
0fa0b4a
Compare
internal/build.go
Outdated
@@ -30,6 +32,19 @@ type BuildCommandParams struct { | |||
func RunBuild(args []string, params *BuildCommandParams) error { | |||
buf := bytes.NewBuffer(nil) | |||
|
|||
var metadataFile = "" | |||
rules, err := util.RetrieveRules(args) | |||
if err == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there is an error we don't want to cause problems at build
time since this is just metadata
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As long as we account for this in the CLI and dashboards, I think we're fine.
0fa0b4a
to
798cc13
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The rules section in the data.json file is generated automatically by OPA if there are json files, which there are under the fixtures/ folder used for testing
This made me stop, we don't want to be including fixture data in the data.json. These should be added to the list of file ignores when we do the build. If these fixtures are not present does this mean the data.json is not created, regardless of the precense of the metadata file? If so we might want to consider another approach.
In the same vein, can we think of any usecases where someone would want their own metadata.json file as part of their own custom rules? In which case we're going to override it.
I wonder if we should just consider writing a .snyk-metadata.json file and adding that to the tarball generated by OPA build?
internal/build.go
Outdated
if metadataFile != "" { | ||
_ = os.Remove(metadataFile) | ||
} | ||
|
||
return out.Close() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both this os.Remove()
call and the out.Close()
I think would be better served to live in a defer
block, that way we ensure they're run regardless of whether this function returns early or not.
util/inspect.go
Outdated
for _, module := range result.Modules { | ||
fileName := filepath.Clean(module.Name) | ||
if !strings.Contains(fileName, "_test.rego") { | ||
fileNames = append(fileNames, fileName) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this filepath not also contain the rule id? Would this be a cleaner alternative to trying to extract the rule from the Rego file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does, but there is nothing stopping customers from changing the publicId
while keeping the name of the folder unrelated to it. It depends on how strict we want to be but it seemed very likely to me that this could happen
util/inspect.go
Outdated
} | ||
|
||
func extractPublicIdFromRego(rego string) string { | ||
re := regexp.MustCompile("\"publicId\"\\s*:\\s*\"(.*?)\"") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On first read this felt fragile to me, I was going to suggest parsing the Rego file here and then traversing the AST to find this value but honestly I reckon this will work most of the time.
internal/build.go
Outdated
@@ -30,6 +32,19 @@ type BuildCommandParams struct { | |||
func RunBuild(args []string, params *BuildCommandParams) error { | |||
buf := bytes.NewBuffer(nil) | |||
|
|||
var metadataFile = "" | |||
rules, err := util.RetrieveRules(args) | |||
if err == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As long as we account for this in the CLI and dashboards, I think we're fine.
Thanks @aron, I think naming this file |
My understanding is that data.json is intended to be loaded alongside the bundled rules and available in the rules. We don't currently support this in our CLI but might find we need to as custom rules are developed to use more of the OPA functionality. This would mean that fixture data would be injected into the rego evaluation context and available via
Same reason as above really, we'd be potentially injecting a |
I think only data that must be evaluated should be in |
I've had a look at using that
|
108ce61
to
a925f51
Compare
What this does
This PR implements a way to get the rules that were written by a customer using the custom rules SDK. The
publicId
of the rule is what dictates the uniqueness/name of the rule, so we needed a way to parse the Rego files.Notes for the reviewer
inspect
which helps us find all the rego rules in a folder: https://github.com/open-policy-agent/opa/blob/main/cmd/inspect.goinspect
command only looks at files and not the contents - but we need a way to get thepublicId
publicId
is by reading each file and using a regex to extract thepublicId
which would be configured in apublicId
key - the regex allows for any spacing so that no matter what tabbing and newline characters the customer uses, we will retrieve itmetadata.json
file, which OPA includes then in thedata.json
as seen in the screenshotrules
section in thedata.json
file is generated automatically by OPA if there are json files, which there are under thefixtures/
folder used for testingMore information