Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only output test logs on failure in CI #27604

Merged
merged 2 commits into from
Jul 29, 2021

Conversation

ijjk
Copy link
Member

@ijjk ijjk commented Jul 29, 2021

This updates to only output a test's logs when it fails to reduce the noise in the CI's logs to allow easier investigating a failure. This also updates azure to leverage the run-tests script when testing ie11 to allow retrying similar to our other tests.

@ijjk ijjk added the created-by: Next.js team PRs by the Next.js team. label Jul 29, 2021
@ijjk

This comment has been minimized.

@ijjk ijjk marked this pull request as ready for review July 29, 2021 14:45
run-tests.js Outdated Show resolved Hide resolved
@ijjk
Copy link
Member Author

ijjk commented Jul 29, 2021

Stats from current PR

Default Build (Decrease detected ✓)
General
vercel/next.js canary ijjk/next.js update/ci-test-output Change
buildDuration 13.1s 13.2s ⚠️ +14ms
buildDurationCached 3s 3s ⚠️ +16ms
nodeModulesSize 50.3 MB 50.3 MB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary ijjk/next.js update/ci-test-output Change
/ failed reqs 0 0
/ total time (seconds) 2.242 2.234 -0.01
/ avg req/sec 1115.16 1119.24 +4.08
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.232 1.261 ⚠️ +0.03
/error-in-render avg req/sec 2029.19 1981.81 ⚠️ -47.38
Client Bundles (main, webpack, commons)
vercel/next.js canary ijjk/next.js update/ci-test-output Change
745.HASH.js gzip 180 B 180 B
framework-HASH.js gzip 42.2 kB 42.2 kB
main-HASH.js gzip 22.8 kB 22.8 kB
webpack-HASH.js gzip 1.49 kB 1.49 kB
Overall change 66.8 kB 66.8 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary ijjk/next.js update/ci-test-output Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB
Overall change 31.1 kB 31.1 kB
Client Pages
vercel/next.js canary ijjk/next.js update/ci-test-output Change
_app-HASH.js gzip 980 B 980 B
_error-HASH.js gzip 194 B 194 B
amp-HASH.js gzip 312 B 312 B
css-HASH.js gzip 329 B 329 B
dynamic-HASH.js gzip 2.52 kB 2.52 kB
head-HASH.js gzip 350 B 350 B
hooks-HASH.js gzip 903 B 903 B
image-HASH.js gzip 4.1 kB 4.1 kB
index-HASH.js gzip 261 B 261 B
link-HASH.js gzip 1.66 kB 1.66 kB
routerDirect..HASH.js gzip 319 B 319 B
script-HASH.js gzip 387 B 387 B
withRouter-HASH.js gzip 320 B 320 B
bb14e60e810b..30f.css gzip 125 B 125 B
Overall change 12.8 kB 12.8 kB
Client Build Manifests
vercel/next.js canary ijjk/next.js update/ci-test-output Change
_buildManifest.js gzip 490 B 490 B
Overall change 490 B 490 B
Rendered Page Sizes
vercel/next.js canary ijjk/next.js update/ci-test-output Change
index.html gzip 531 B 531 B
link.html gzip 544 B 544 B
withRouter.html gzip 524 B 524 B
Overall change 1.6 kB 1.6 kB

Webpack 4 Mode (Increase detected ⚠️)
General
vercel/next.js canary ijjk/next.js update/ci-test-output Change
buildDuration 10.6s 10.6s ⚠️ +14ms
buildDurationCached 4.2s 4.1s -47ms
nodeModulesSize 50.3 MB 50.3 MB
Page Load Tests Overall increase ✓
vercel/next.js canary ijjk/next.js update/ci-test-output Change
/ failed reqs 0 0
/ total time (seconds) 2.304 2.292 -0.01
/ avg req/sec 1084.99 1090.94 +5.95
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.298 1.293 -0.01
/error-in-render avg req/sec 1925.55 1933.47 +7.92
Client Bundles (main, webpack, commons)
vercel/next.js canary ijjk/next.js update/ci-test-output Change
17.HASH.js gzip 185 B 185 B
677f882d2ed8..HASH.js gzip 13.8 kB 13.8 kB
framework.HASH.js gzip 41.9 kB 41.9 kB
main-HASH.js gzip 10.6 kB 10.6 kB
webpack-HASH.js gzip 1.19 kB 1.19 kB
Overall change 67.7 kB 67.7 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary ijjk/next.js update/ci-test-output Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB
Overall change 31.3 kB 31.3 kB
Client Pages
vercel/next.js canary ijjk/next.js update/ci-test-output Change
_app-HASH.js gzip 965 B 965 B
_error-HASH.js gzip 3.74 kB 3.74 kB
amp-HASH.js gzip 552 B 552 B
css-HASH.js gzip 333 B 333 B
dynamic-HASH.js gzip 2.71 kB 2.71 kB
head-HASH.js gzip 2.97 kB 2.97 kB
hooks-HASH.js gzip 911 B 911 B
index-HASH.js gzip 231 B 231 B
link-HASH.js gzip 1.64 kB 1.64 kB
routerDirect..HASH.js gzip 298 B 298 B
script-HASH.js gzip 2.94 kB 2.94 kB
withRouter-HASH.js gzip 294 B 294 B
e025d2764813..52f.css gzip 125 B 125 B
Overall change 17.7 kB 17.7 kB
Client Build Manifests
vercel/next.js canary ijjk/next.js update/ci-test-output Change
_buildManifest.js gzip 499 B 499 B
Overall change 499 B 499 B
Rendered Page Sizes
vercel/next.js canary ijjk/next.js update/ci-test-output Change
index.html gzip 577 B 577 B
link.html gzip 589 B 589 B
withRouter.html gzip 569 B 569 B
Overall change 1.74 kB 1.74 kB
Commit: 9b9170d

@kodiakhq kodiakhq bot merged commit f5ee9ed into vercel:canary Jul 29, 2021
@ijjk ijjk deleted the update/ci-test-output branch July 29, 2021 15:36
flybayer pushed a commit to blitz-js/next.js that referenced this pull request Aug 19, 2021
This updates to only output a test's logs when it fails to reduce the noise in the CI's logs to allow easier investigating a failure. This also updates azure to leverage the `run-tests` script when testing ie11 to allow retrying similar to our other tests.
@vercel vercel locked as resolved and limited conversation to collaborators Jan 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
created-by: Next.js team PRs by the Next.js team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants