Adding Non-Interactive User Sign-In Logs and Service Principal Sign-In Logs to filebeat azure module

[npanone]

Enhancement

What does this PR do?

Adds Non-Interactive User and Service Principal configurations to the azure module. Practically a clone of the Sign-In event hub, the log formats are identical except for the category which is SignInLogs, NonInteractiveUserSignInLogs or ServicePrincipalSignInLogs.

Why is it important?

Ability to track all login types into Azure AD. Sign-In Logs was only capturing 1/3rd of the authentications.

Checklist

• My code follows the style guidelines of this project
  - [ ] I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

Related issues

Relates #23653

Use cases

Screenshots

Logs

[cla-checker-service]

💚 CLA has been signed

[npanone]

I've signed the Contributor Agreement

[elasticmachine]

❕ Build Aborted

The PR is not allowed to run in the CI yet

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: Pull request #24532 opened

• Reason: The PR is not allowed to run in the CI yet

• Start Time: 2021-03-14T18:22:24.852+0000

• Duration: 4 min 51 sec

• Commit: 1cff336

Trends 🧪

Steps errors

Expand to view the steps failures

Load a resource file from a shared library

• Took 0 min 0 sec . View more details on here
• Description: approval-list/elastic/beats.yml

Error signal

• Took 0 min 0 sec . View more details on here
• Description: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permissions can do so.)

Log output

Expand to view the last 100 lines of log output

[2021-03-14T18:24:18.542Z] Checking out Revision 1cff33600039dfe193c52552f44ec9fd29722d41 (PR-24532)
[2021-03-14T18:24:18.515Z]  > git remote # timeout=10
[2021-03-14T18:24:18.519Z]  > git config --get remote.origin.url # timeout=10
[2021-03-14T18:24:18.522Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2021-03-14T18:24:18.526Z]  > git merge 4f72a77cd9d04702f8ff70c3141e4b63b27bb80f # timeout=10
[2021-03-14T18:24:18.536Z]  > git rev-parse HEAD^{commit} # timeout=10
[2021-03-14T18:24:18.543Z]  > git config core.sparsecheckout # timeout=10
[2021-03-14T18:24:18.546Z]  > git checkout -f 1cff33600039dfe193c52552f44ec9fd29722d41 # timeout=15
[2021-03-14T18:24:22.832Z] Commit message: "Adding Non-Interactive User and Service Principal Sign-ins to Azure module"
[2021-03-14T18:24:22.832Z] First time build. Skipping changelog.
[2021-03-14T18:24:22.832Z] Cleaning workspace
[2021-03-14T18:24:22.993Z]  > git --version # timeout=10
[2021-03-14T18:24:22.996Z]  > git --version # 'git version 2.17.1'
[2021-03-14T18:24:23.000Z] fatal: bad object 83043640419f537f6bf9b3a689d25de46bac2014
[2021-03-14T18:24:22.833Z]  > git rev-parse --verify HEAD # timeout=10
[2021-03-14T18:24:22.837Z] Resetting working tree
[2021-03-14T18:24:22.838Z]  > git reset --hard # timeout=10
[2021-03-14T18:24:22.918Z]  > git clean -fdx # timeout=10
[2021-03-14T18:24:23.768Z] Timeout set to expire in 3 hr 0 min
[2021-03-14T18:24:23.795Z] The timestamps step is unnecessary when timestamps are enabled for all Pipeline builds.
[2021-03-14T18:24:24.212Z] [INFO] Number of builds to be searched 10
[2021-03-14T18:24:24.900Z] [INFO] 'shallow' is forced to be disabled when running on PullRequests
[2021-03-14T18:24:24.928Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24532/src/github.com/elastic/beats
[2021-03-14T18:24:24.971Z] [INFO] gitCheckout: Checkout SCM PR-24532 with default customisation from the Item.
[2021-03-14T18:24:25.014Z] [INFO] Override default checkout
[2021-03-14T18:24:25.099Z] Sleeping for 10 sec
[2021-03-14T18:24:35.134Z] The recommended git tool is: git
[2021-03-14T18:24:35.195Z] using credential f6c7695a-671e-4f4f-a331-acdce44ff9ba
[2021-03-14T18:24:35.254Z] Wiping out workspace first.
[2021-03-14T18:24:35.263Z] Cloning the remote Git repository
[2021-03-14T18:24:35.263Z] Using shallow clone with depth 10
[2021-03-14T18:24:35.263Z] Avoid fetching tags
[2021-03-14T18:24:35.277Z] Cloning repository [email protected]:elastic/beats.git
[2021-03-14T18:24:35.303Z]  > git init /var/lib/jenkins/workspace/Beats_beats_PR-24532/src/github.com/elastic/beats # timeout=10
[2021-03-14T18:24:35.309Z] Fetching upstream changes from [email protected]:elastic/beats.git
[2021-03-14T18:24:35.309Z]  > git --version # timeout=10
[2021-03-14T18:24:35.314Z]  > git --version # 'git version 2.17.1'
[2021-03-14T18:24:35.314Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2021-03-14T18:24:35.319Z]  > git fetch --no-tags --progress -- [email protected]:elastic/beats.git +refs/heads/*:refs/remotes/origin/* # timeout=15
[2021-03-14T18:24:56.831Z] Cleaning workspace
[2021-03-14T18:24:56.843Z] Using shallow fetch with depth 10
[2021-03-14T18:24:56.843Z] Pruning obsolete local branches
[2021-03-14T18:24:57.993Z] Merging remotes/origin/master commit 4f72a77cd9d04702f8ff70c3141e4b63b27bb80f into PR head commit 1cff33600039dfe193c52552f44ec9fd29722d41
[2021-03-14T18:24:56.817Z]  > git config remote.origin.url [email protected]:elastic/beats.git # timeout=10
[2021-03-14T18:24:56.820Z]  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
[2021-03-14T18:24:56.827Z]  > git config remote.origin.url [email protected]:elastic/beats.git # timeout=10
[2021-03-14T18:24:56.832Z]  > git rev-parse --verify HEAD # timeout=10
[2021-03-14T18:24:56.836Z] No valid HEAD. Skipping the resetting
[2021-03-14T18:24:56.836Z]  > git clean -fdx # timeout=10
[2021-03-14T18:24:56.845Z] Fetching upstream changes from [email protected]:elastic/beats.git
[2021-03-14T18:24:56.845Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2021-03-14T18:24:56.849Z]  > git fetch --no-tags --progress --prune -- [email protected]:elastic/beats.git +refs/pull/24532/head:refs/remotes/origin/PR-24532 +refs/heads/master:refs/remotes/origin/master # timeout=15
[2021-03-14T18:24:57.995Z]  > git config core.sparsecheckout # timeout=10
[2021-03-14T18:24:57.998Z]  > git checkout -f 1cff33600039dfe193c52552f44ec9fd29722d41 # timeout=15
[2021-03-14T18:24:59.551Z] Merge succeeded, producing 1cff33600039dfe193c52552f44ec9fd29722d41
[2021-03-14T18:24:59.552Z] Checking out Revision 1cff33600039dfe193c52552f44ec9fd29722d41 (PR-24532)
[2021-03-14T18:25:00.357Z] Commit message: "Adding Non-Interactive User and Service Principal Sign-ins to Azure module"
[2021-03-14T18:25:00.357Z] Cleaning workspace
[2021-03-14T18:25:00.510Z]  > git --version # timeout=10
[2021-03-14T18:25:00.515Z]  > git --version # 'git version 2.17.1'
[2021-03-14T18:25:00.518Z] fatal: bad object 83043640419f537f6bf9b3a689d25de46bac2014
[2021-03-14T18:24:59.526Z]  > git remote # timeout=10
[2021-03-14T18:24:59.530Z]  > git config --get remote.origin.url # timeout=10
[2021-03-14T18:24:59.534Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2021-03-14T18:24:59.537Z]  > git merge 4f72a77cd9d04702f8ff70c3141e4b63b27bb80f # timeout=10
[2021-03-14T18:24:59.546Z]  > git rev-parse HEAD^{commit} # timeout=10
[2021-03-14T18:24:59.553Z]  > git config core.sparsecheckout # timeout=10
[2021-03-14T18:24:59.556Z]  > git checkout -f 1cff33600039dfe193c52552f44ec9fd29722d41 # timeout=15
[2021-03-14T18:25:00.359Z]  > git rev-parse --verify HEAD # timeout=10
[2021-03-14T18:25:00.362Z] Resetting working tree
[2021-03-14T18:25:00.362Z]  > git reset --hard # timeout=10
[2021-03-14T18:25:00.447Z]  > git clean -fdx # timeout=10
[2021-03-14T18:25:01.532Z] Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD
[2021-03-14T18:25:02.191Z] + git fetch https://****:****@github.com/elastic/beats.git +refs/pull/*/head:refs/remotes/origin/pr/*
[2021-03-14T18:26:10.062Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24532/src/github.com/elastic/beats/.git
[2021-03-14T18:26:10.221Z] Archiving artifacts
[2021-03-14T18:26:10.968Z] + git rev-parse HEAD
[2021-03-14T18:26:11.308Z] + git rev-parse HEAD
[2021-03-14T18:26:11.817Z] + git rev-parse origin/pr/24532
[2021-03-14T18:26:11.871Z] [INFO] githubEnv: Found Git Build Cause: pr
[2021-03-14T18:26:12.127Z] Masking supported pattern matches of $GITHUB_TOKEN
[2021-03-14T18:26:12.917Z] [WARN] githubApiCall: The REST API call https://api.github.com/repos/elastic/beats/pulls/24532/reviews return 0 elements
[2021-03-14T18:26:12.981Z] [INFO] githubPrCheckApproved: Title: Adding Non-Interactive User Sign-In Logs and Service Principal Sign-In Logs to filebeat azure module - User: npanone - Author Association: FIRST_TIME_CONTRIBUTOR
[2021-03-14T18:26:13.295Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet
[2021-03-14T18:26:13.295Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permissions can do so.)
[2021-03-14T18:26:13.371Z] [INFO] Let's stop build #1. The PR is not allowed to run in the CI yet
[2021-03-14T18:26:13.394Z] Sleeping for 5 sec
[2021-03-14T18:26:14.399Z] Stage "Lint" skipped due to earlier failure(s)
[2021-03-14T18:26:14.486Z] Stage "Build&Test" skipped due to earlier failure(s)
[2021-03-14T18:26:14.563Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-03-14T18:26:14.684Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24532/src/github.com/elastic/beats
[2021-03-14T18:26:15.243Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-24532
[2021-03-14T18:26:15.375Z] [INFO] getVaultSecret: Getting secrets
[2021-03-14T18:26:15.474Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-03-14T18:26:16.462Z] + chmod 755 generate-build-data.sh
[2021-03-14T18:26:16.462Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24532/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24532/runs/1 ABORTED 231336
[2021-03-14T18:26:16.712Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24532/runs/1/steps/?limit=10000 -o steps-info.json
[2021-03-14T18:26:17.263Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24532/runs/1/tests/?status=FAILED -o tests-errors.json
[2021-03-14T18:26:17.263Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2021-03-14T18:26:18.173Z] Retry 2/3 exited 22, retrying in 2 seconds...

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[npanone]

I'd like to resubmit with some additional changes.

[renini]

I'd like to resubmit with some additional changes.

@npanone was this PR incomplete or what was the reason for the closure?