Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add .mocharc to create test XML results. #1588

Merged
merged 5 commits into from
Jan 18, 2020
Merged

Add .mocharc to create test XML results. #1588

merged 5 commits into from
Jan 18, 2020

Conversation

ace-n
Copy link
Contributor

@ace-n ace-n commented Jan 11, 2020

Part 2 of #1587

DO NOT MERGE until all test results are (somehow) visible in the Kokoro UI.

@ace-n ace-n requested a review from fhinkel as a code owner January 11, 2020 00:26
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jan 11, 2020
@ace-n ace-n requested a review from bcoe January 13, 2020 20:25
Copy link
Contributor

@bcoe bcoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So here's what I wish we had, for client libraries too, is the ability to have the default mocha output, in conjunction with the xunit output, and then we could just always have xunit outputting to a well known location, e.g. .test/all-tests.xml, which we don't check in.

We could then enable this on all our repos in a standard way, without moving a file around. This would allow us to take advantage of some of the work @tbpg has done with a "build cop bot".

.kokoro/.mocharc.yml Show resolved Hide resolved
@ace-n ace-n added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jan 14, 2020
@ace-n
Copy link
Contributor Author

ace-n commented Jan 16, 2020

@fhinkel FYI - @bcoe requested that I apply this change to all tests (as opposed to just the already-labeled tests in functions and appengine).

This PR will cause two major changes:

  1. mocha runs on Kokoro will only output XUnit (i.e. the default human-readable mocha output will not show up)
  2. Kokoro automatically breaks up XML test results into this nice tree-view UI format, which helps compensate for the lack of human-readable mocha output. (The XUnit logs will still be output as an XML artifact.)

@fhinkel - @bcoe was fine with this for the client libraries, but I want to make sure you're OK with this as well before merging.

Copy link
Contributor

@bcoe bcoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach, as we'd like to move towards performing analysis on test failures across all of our client libraries -- and it seems like xunit is probably a good way to do so.

My only concern is if the kokoro UI ends up not being easy enough to delve into on failures, it looked fairly reasonable from what @ace-n showed me though.

@ace-n ace-n added kokoro:force-run Add this label to force Kokoro to re-run the tests. and removed kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Jan 17, 2020
@ace-n ace-n removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jan 18, 2020
@ace-n
Copy link
Contributor Author

ace-n commented Jan 18, 2020

Merging this now, as it's necessary for a demo next week.

@fhinkel if this change becomes an issue for you, please ping me so we can roll back and/or work out an alternative solution. 🙂

@ace-n ace-n merged commit 5343102 into master Jan 18, 2020
@ace-n ace-n deleted the mocharc branch January 18, 2020 02:10
@fhinkel
Copy link
Contributor

fhinkel commented Jan 21, 2020

Sounds good, let's see if it causes any problems (also with the build cop bot).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants