Entity Work Jan 4 - Manny #9718
Conversation
Required items, please complete Change(s): - Working on some of the detections Shain asked us to focus on, mainly adding fullName identifiers Reason for Change(s): - Changes needed listed on Shain's dashboard Version Updated: - yes Testing Completed: - yes Checked that the validations are passing and have addressed any issues that are present: - no
|
Hello how are you I am GitHub bot |
![contentautomationbot[bot]](https://avatars.githubusercontent.com/u/33221666?s=60&v=4){kind=small}
There was a problem hiding this comment.
KQL Best Practices Suggestions for: ExternalUserAddedRemovedInTeams.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Consider using the materialize() function for the 'TeamsAddDel' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using the materialize() function for the 'TeamsAdd' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: SharePoint_Downloads_byNewIP.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
- Consider using the materialize() function for the 'szSharePointFileOperation' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using the materialize() function for the 'szOperations' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using the materialize() function for the 'endtime' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: Mail_redirect_via_ExO_transport_rule.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: AVdetectionsrelatedtoUkrainebasedthreats.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: PotentialBuildProcessCompromiseMDE.yaml
- Consider using the materialize() function for the 'timeframe' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using the materialize() function for the 'time_window' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: RareOfficeOperations.yaml
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: Malicious_Inbox_Rule.yaml
- Use the 'has' operator instead of 'contains' for string operators.
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'contains_cs' operator instead of 'contains' for case-sensitive comparisons.
- Don't use '*' for searching text. Look in a specific column.
- Consider using the materialize() function for the 'Keywords' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: ADFSDomainTrustMods.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: Office_MailForwarding.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Consider using Col =~ "lowercasestring" instead of tolower(Col) == "lowercasestring" for case-insensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: AVSpringShell.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
- Consider using Col =~ "lowercasestring" instead of tolower(Col) == "lowercasestring" for case-insensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: SharePoint_Downloads_byNewUserAgent.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
- Consider using the materialize() function for the 'szSharePointFileOperation' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using the materialize() function for the 'szOperations' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using the materialize() function for the 'endtime' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using the materialize() function for the 'Baseevents' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: office_policytampering.yaml
- Use the 'has' operator instead of 'contains' for string operators.
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
- Use the 'contains_cs' operator instead of 'contains' for case-sensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: PossiblePhishingwithCSL&NetworkSession.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: External User added to Team and immediately uploads file.yaml
- Use the 'has' operator instead of 'contains' for string operators.
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'contains_cs' operator instead of 'contains' for case-sensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: exchange_auditlogdisabled.yaml
- Use the 'has' operator instead of 'contains' for string operators.
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
- Use the 'contains_cs' operator instead of 'contains' for case-sensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: CorrelateIPC_Unfamiliar-Atypical.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Consider using the materialize() function for the 'Alert_UnfamiliarSignInProps' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using the materialize() function for the 'Alert_AtypicalTravels' variable if its assignment involves computation or calculation. This can improve performance.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: sharepoint_file_transfer_folders_above_threshold.yaml
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: SUNSPOTHashes.yaml
- Consider using the materialize() function for the 'SUNSPOT_Hashes' variable if its assignment involves computation or calculation. This can improve performance.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: AVTarrask.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Use the 'in' operator instead of 'in~' for case-sensitive comparisons.
- Consider using Col =~ "lowercasestring" instead of tolower(Col) == "lowercasestring" for case-insensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
- Consider using hint.strategy=broadcast when the left side is small and the right side is large.
- Consider using the lookup operator instead of join when the right side is small and the left side is large.
- Consider using hint.shufflekey= when both sides are too large.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: sharepoint_file_transfer_above_threshold.yaml
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
There was a problem hiding this comment.
KQL Best Practices Suggestions for: MultipleTeamsDeletes.yaml
- Use the '==' operator instead of '=~' for case-sensitive comparisons.
- Consider using hint.shufflekey=key with the summarize operator when group by keys have high cardinality.
Disclaimer: These suggestions are offered to enhance query efficiency and adherence to best practices. It is recommended to consider applying them for improved code quality, but their application is optional and context-dependent.
shainw
added
the
CoreEntityFix
| @@ -54,10 +54,12 @@ query: | | |||
| | where FileAccessCount > fileAccessThrehold | |||
| ) on $left.OfficeObjectId == $right.OfficeObjectId | |||
| )on $left.UPN == $right.UserId | |||
| | extend AccountName = tostring(split(UserWhoAdded, "@")[0]), AccountUPNSuffix = tostring(split(UserWhoAdded, "@")[1]) | |||
| | extend AccountName = tostring(split(UPN, "@")[0]), AccountUPNSuffix = tostring(split(UPN, "@")[1]) | |||
There was a problem hiding this comment.
We should parse and bring through UserWhoAdded and UserWhoDeleted to the entity mappings along with the UPN (which is the MemberAdded) I recommend we rename UPN to MemeberAdded.
| @@ -40,9 +40,11 @@ query: | | |||
| entityMappings: | |||
There was a problem hiding this comment.
We should parse and bring through UserWhoAdded and UserWhoDeleted to the entity mappings along with the UPN (which is the MemberAdded) I recommend we rename UPN to MemeberAdded.
Additionally, we need to bring through ClientIP to IPAddress entity mapping
| @@ -31,9 +31,11 @@ query: | | |||
| entityMappings: | |||
There was a problem hiding this comment.
Is there an opportunity to bring through ClientIP mapping here?
| @@ -107,12 +107,16 @@ query: | | |||
| entityMappings: | |||
| - entityType: Account | |||
| fieldMappings: | |||
| - identifier: FullName | |||
| columnName: InitiatingProcessAccountName | |||
There was a problem hiding this comment.
This will not work, this is just the Name, need to bring through InitiatingProcessAccountUpn from DeviceEvents.
Also, InitiatingProcessAccountDomain is available so bring that through and there is no need to do this extend - tostring(split(InitiatingProcessAccountName,'@',0)[0]), UPNSuffix = tostring(split(InitiatingProcessAccountName,'@',1)[0])
Lastly, we should bring through RecipientEmailAddress which will need to be parsed into Name and UPNSuffix as this is a UPN.
| - identifier: DnsDomain | ||
| columnName: DnsDomain | ||
| version: 1.0.1 | ||
| - identifier: FullName |
There was a problem hiding this comment.
We should bring through Account here, we have it so it needs to be added. You will need to modify DeviceFileEvents query to include InitiatingProcessAccountDomain and InitiatingProcessAccountUpn, then you can map all the fields.
| - identifier: DnsDomain | ||
| columnName: DnsDomain | ||
| version: 1.0.1 | ||
| - identifier: FullName |
There was a problem hiding this comment.
We should bring through Account here, we have it so it needs to be added. You will need to modify DeviceImageLoadEvents query to include InitiatingProcessAccountName, InitiatingProcessAccountDomain, InitiatingProcessAccountUpn, then you can map all the fields.
| | extend CreatedByApp = tostring(InitiatedBy.app.displayName) | ||
| | project Creation_TimeGenerated, Deletion_TimeGenerated, TimeDelta, UserPrincipalName, DeletedByUser, DeletedByIPAddress, DeletedByApp, CreatedByUser, CreatedByIPAddress, CreatedByApp, Creation_AdditionalDetails = AdditionalDetails, Creation_InitiatedBy = InitiatedBy, Creation_TargetResources = TargetResources, Deletion_AdditionalDetails, Deletion_InitiatedBy, Deletion_TargetResources | ||
| | extend timestamp = Deletion_TimeGenerated, Name = tostring(split(UserPrincipalName,'@',0)[0]), UPNSuffix = tostring(split(UserPrincipalName,'@',1)[0]) | ||
| | extend InitiatingAppName = iff(isnotempty(InitiatedBy.user.userPrincipalName), InitiatedBy.user.userPrincipalName, InitiatedBy.app.displayName) |
There was a problem hiding this comment.
This one needs a more complex addition. We need to bring through the account that created, the account that deleted and the account that was created/deleted. I will do this once you have done the other fixes.
| @@ -88,9 +88,11 @@ entityMappings: | |||
| columnName: AppDisplayName | |||
| - entityType: Account | |||
| fieldMappings: | |||
| - identifier: FullName | |||
There was a problem hiding this comment.
Same on this one, it needs Initiated by and the Target. The Target also needs to be fixed, bringing through displayName is not likely correct. I will fix once you address the other items.
|
Hello @mmelndezlujn, Shain has requested some changes please take a look at it |
|
Hello @mmelndezlujn, Shain has some requested changes. Kindly please have a look at it. |
|
Hello @mmelndezlujn, |
|
Hello how are you I am GitHub bot |
Hello @v-prasadboke, I am not sure if I can provide write access as I am not the repo owner. @shainw could you help me with this, please? :) |
Hi @shainw, Any help on this. |
|
Hello @mmelndezlujn, can you update the branch from master |
done :) |
|
Hello @mmelndezlujn, |
|
Hello how are you I am GitHub bot |
My bad, I hadn't pushed it, but it's updated now |
@v-prasadboke and @mmelndezlujn - We are not going to complete this PR. There are too many file changes in one PR and there are more changes that are needed based on my comments. I am going to take the changes and break them down into 5-6 file submissions and extend the mappings I called out as needed. I am closing this and will reference this in the new PRs. |
Required items, please complete
Change(s):
Reason for Change(s):
Version Updated:
Testing Completed:
Checked that the validations are passing and have addressed any issues that are present: