Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JENKINS-47378# Switch to JDK8 and core 2.73.2 #1479

Merged
merged 17 commits into from
Oct 23, 2017
Merged

Conversation

vivek
Copy link
Collaborator

@vivek vivek commented Oct 11, 2017

Description

Upgrade Java level to JDK8 and Jenkins core to 2.60.3.

See JENKINS-47378.

Submitter checklist

  • Link to JIRA ticket in description, if appropriate.
  • Change is code complete and matches issue description
  • Appropriate unit or acceptance tests or explanation to why this change has no tests
  • Reviewer's manual test instructions provided in PR description. See Reviewer's first task below.

Reviewer checklist

  • Run the changes and verified the change matches the issue description
  • Reviewed the code
  • Verified that the appropriate tests have been written or valid explanation given

@i386
Copy link
Contributor

i386 commented Oct 11, 2017

Workflow SCM step and Workflow Job can be upgraded too BTW

@vivek
Copy link
Collaborator Author

vivek commented Oct 11, 2017

@i386 ok will update them.

@fwilhe
Copy link

fwilhe commented Oct 11, 2017

I think the FROM statement in the Dockerfile needs to be updated, too in this PR.

@vivek
Copy link
Collaborator Author

vivek commented Oct 11, 2017

@fwilhe Thanks for bringing it to attention. Looks like we forgot to change it last time we upgraded core to 2.46.3 as well. cc: @michaelneale

@vivek
Copy link
Collaborator Author

vivek commented Oct 11, 2017

@i386 @michaelneale We can not use workflow-job > 2.12.2 as these later versions only work with jenkins 2.62 and higher. There are new APIs added in 2.62 that they use so unless we upgrade to LTS 2.73.1 or later we have to stick with workflow-job 2.12.2.

@i386
Copy link
Contributor

i386 commented Oct 11, 2017

@vivek ahh gotcha.

@michaelneale
Copy link
Member

ah I misremembered the workflow-job version

@vivek vivek changed the title JENKINS-47378# Switch to JDK8 and core 2.60.3 JENKINS-47378# Switch to JDK8 and core 2.73.2 Oct 13, 2017
@vivek
Copy link
Collaborator Author

vivek commented Oct 13, 2017

@michaelneale @imeredith Something strange with ATH test runs. All of these failing tests work fine locally.

Most of them are failing with timeout during call to Jenkins APIs. Since I can't reproduce locally, don't know how to go about debugging it. Can you guys try out locally and see if it fails for you? Or if there is something else needs to be changed to upgrade jenkins core?

Caused by: io.blueocean.ath.AcceptanceTestException: Error while waiting for something
Caused by: org.openqa.selenium.TimeoutException: 

I suspect something to do with jenkins-client. Calls are timing out during jenkins api calls, e.g. CustomJenkinsServer.deleteUserDomainCredential() possible at client.get() based on the log of GithubCreationTest.testTokenValidation_failed(), see https://ci.blueocean.io/blue/organizations/jenkins/blueocean/detail/task%2FJENKINS-47378_jdk8/14/tests.

@michaelneale
Copy link
Member

@vivek https://ci.blueocean.io/job/blueocean/job/task%252FJENKINS-47378_jdk8/14/artifact/acceptance-tests/target/screenshots/io.blueocean.ath.offline.dashboard.DashboardTest_checkExecutorPluginLoaded.png to me implies blue ocean isn't loading properly, or perhaps compatibility with something broke.

I am not sure if the hpi sizes are off - if they are it means a bundling problem (ie different from a passing master branch).

@vivek
Copy link
Collaborator Author

vivek commented Oct 13, 2017

@michaelneale bundling etc. are fine. Try running locally, works great. I suspect some other factor introduced during run on dogfood? Most likely its exposing ATH test flakiness introduced with upgrade to core 2.73.2.

@michaelneale
Copy link
Member

@vivek that 404 means something isn't loading into the ATH from blue ocean (ie something has broken re backward compatability) - maybe the JDK version on CI is not the same as locally? There hasn't been a change to the environment.

@vivek
Copy link
Collaborator Author

vivek commented Oct 13, 2017

@michaelneale Right but don't know whats causing because the same test works well locally. Try locally see if it fails, we need to find whats causing this plugin to not load. In other instances its just timeout, maybe related.

@michaelneale
Copy link
Member

@vivek locally I see problems.

Did you try running the whole suite locally?

checkExecutorPluginLoaded fails for me the same as CI.
Also look in your log (scroll up):

[INFO] ------------------------------------------------------------------------
[INFO] Building Fake realm plugin 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Fake realm plugin .................................. FAILURE [  5.670 s]
[INFO] ATH Dependencies Aggregator ........................ SKIPPED
[INFO] Blue Ocean Parent .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.084 s
[INFO] Finished at: 2017-10-12T16:42:20+00:00
[INFO] Final Memory: 26M/278M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project fake-realm: Could not resolve dependencies for project org.jenkins-ci.plugins:fake-realm:hpi:1.0-SNAPSHOT: Could not find artifact org.jenkins-ci.main:jenkins-war:jar:war-for-test:2.73.2 in repo.jenkins-ci.org (https://repo.jenkins-ci.org/public/) -> [Help 1]

you have to look up the log a bit to see it when it starts the ATH. @imeredith and I looked at this the other day, sometimes errors get swallowed there and things aren't ok.

Did you run the whole suite locally? as it isn't starting up correctly in ATH locally for me, I see that error, there is no executor plugin installed etc... so something isn't right.

Look for the failure in the log: https://ci.blueocean.io/blue/organizations/jenkins/blueocean/detail/task%2FJENKINS-47378_jdk8/14/artifacts

I expect this would be a fairly heavy change, not surprised things arent' smooth. I think the runtime-deps of the ATH will need adjustment, but I can't remember how/where it assembles the plugins.

@vivek
Copy link
Collaborator Author

vivek commented Oct 13, 2017

@michaelneale thanks, ok had to scroll up a bit to notice that error. Probably my local ath had stale bit and for some reason running with -Dtest=DashboardTest was passing. Anyways, I have fixed and can confirm running checkExecutorPluginLoaded is passing now. Lets wait for CI run, https://ci.blueocean.io/blue/organizations/jenkins/blueocean/detail/task%2FJENKINS-47378_jdk8/16/pipeline.

@vivek
Copy link
Collaborator Author

vivek commented Oct 13, 2017

@michaelneale now one test is failing, but it passes for me locally. I am running it as:

nightwatch src/test/js/edgeCases/folder.js

However it failed in CI with this error:

&amp#27;[0;31m???&amp#27;[0m Timed out while waiting for element <form[name="config"]> to be present for 20000 milliseconds. &amp#27;[1;37m - expected &amp#27;[0;32m"found"&amp#27;[0m&amp#27;[0m but got: &amp#27;[0;31m"not found"&amp#27;[0m
06:58:12 INFO  - &amp#27;[0;90m    at createFolder (/home/jenkins/slave/workspace/ean_task_JENKINS-47378_jdk8-YUP4IDGCYBA4YZOHDUPWFZGBV4IP4AOMCUCCX6C5IRJCLP3HYONA/acceptance-tests/src/main/js/page_objects/classic_jenkins/folderCreate.js:57:18)
06:58:12 INFO  -     at Object.module.exports.commands.createFolders (/home/jenkins/slave/workspace/ean_task_JENKINS-47378_jdk8-YUP4IDGCYBA4YZOHDUPWFZGBV4IP4AOMCUCCX6C5IRJCLP3HYONA/acceptance-tests/src/main/js/page_objects/classic_jenkins/folderCreate.js:80:13)
06:58:12 INFO  -     at Object.module.exports.step 01 (/home/jenkins/slave/workspace/ean_task_JENKINS-47378_jdk8-YUP4IDGCYBA4YZOHDUPWFZGBV4IP4AOMCUCCX6C5IRJCLP3HYONA/acceptance-tests/src/test/js/edgeCases/folder.js:60:21)&amp#27;[0m

We should re-write these tests to use API to create these folders, no point running selenium test on classic UI, thats not something we intend to test anyways.

@michaelneale
Copy link
Member

michaelneale commented Oct 13, 2017 via email

@vivek
Copy link
Collaborator Author

vivek commented Oct 13, 2017

@michaelneale It looks legit but nothing to do with encoded folder names. Its failing doing selenium testing of classic UI to create folder. Please give it try your Monday morning, see if you can reproduce it because I can't reproduce it locally.

@michaelneale
Copy link
Member

@vivek yes that probably could explain it.

Creating jobs should be via other means. This change will take quite a lot more work perhaps than expected. Is this urgent?

@vivek
Copy link
Collaborator Author

vivek commented Oct 14, 2017

@michaelneale yeah I think we must do it first thing if we want to switch to jdk8 for 1.4, of course if there is something urgent then we can do this later. By postponing it won’t buy anything, will only delay problems to be fixed later. Also we want to code in Java 8:)

@vivek
Copy link
Collaborator Author

vivek commented Oct 14, 2017

@michaelneale work wise, disagree it needs lots of work. we are pretty close, the last set of ath nightwatch tests are failing. Matter of figuring why. It works fine locally so must be something minor somewhere.

@michaelneale
Copy link
Member

michaelneale commented Oct 16, 2017

@vivek ok cool, yeah just wasn't sure if this would be one of those huge time sinks, but if you sound confident, that is good ;)

FWIW that edgeCases/folder.js test passes for me locally too - try it again? Otherwise my guess is some encoding thing (although the missing form element usually points to classic behavior)

@vivek
Copy link
Collaborator Author

vivek commented Oct 16, 2017

@michaelneale replayed. I don't know how big this issue is, fact that it works locally for us, could be something in CI env causing it. I will keep it going in parallel...

@michaelneale
Copy link
Member

05:09:14 INFO  -  �[1;31mTEST FAILURE:�[0m  �[0;31m1�[0m assertions failed, �[0;32m777�[0m passed. (16m 56s)
05:09:14 INFO  - 
05:09:14 INFO  - �[0;31m ??? edgeCases/folder�[0m
05:09:14 INFO  - 
05:09:14 INFO  -    - step 01 �[1;33m(23.226s)�[0m
05:09:14 INFO  - �[0;31m   Timed out while waiting for element <form[name="config"]> to be present for 20000 milliseconds. �[1;37m - expected �[0;32m"found"�[0m�[0m but got: �[0;31m"not found"�[0m�[0m
05:09:14 INFO  - �[0;90m       at createFolder (/home/jenkins/slave/workspace/ean_task_JENKINS-47378_jdk8-YUP4IDGCYBA4YZOHDUPWFZGBV4IP4AOMCUCCX6C5IRJCLP3HYONA/acceptance-tests/src/main/js/page_objects/classic_jenkins/folderCreate.js:57:18)
05:09:14 INFO  -        at Object.module.exports.commands.createFolders (/home/jenkins/slave/workspace/ean_task_JENKINS-47378_jdk8-YUP4IDGCYBA4YZOHDUPWFZGBV4IP4AOMCUCCX6C5IRJCLP3HYONA/acceptance-tests/src/main/js/page_objects/classic_jenkins/folderCreate.js:68:9)
05:09:14 INFO  -        at Object.module.exports.step 01 (/home/jenkins/slave/workspace/ean_task_JENKINS-47378_jdk8-YUP4IDGCYBA4YZOHDUPWFZGBV4IP4AOMCUCCX6C5IRJCLP3HYONA/acceptance-tests/src/test/js/edgeCases/folder.js:60:21)�[0m

So not sure what that is about - only thing different would be that the JDK versions might not be exactly the same locally, or the time taken to load that form (or something like that) - but it is odd. Consistently fails there.

@vivek vivek force-pushed the task/JENKINS-47378_jdk8 branch 2 times, most recently from d622238 to cf627fb Compare October 20, 2017 06:16
@vivek
Copy link
Collaborator Author

vivek commented Oct 23, 2017

@michaelneale yeah not up to date with master. Ok, will wait for #1484 to merge.

@vivek
Copy link
Collaborator Author

vivek commented Oct 23, 2017

@michaelneale rebates on latest mater is green. I think master is stable now and it can be merged?

" sh('sleep 60') " +
" stage ('Test1'); " +
" stage ('Build'); " +
" semaphore 's' " +
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat

@@ -16,9 +16,9 @@
<name>Blue Ocean Parent</name>

<properties>
<java.level>7</java.level>
<java.level>8</java.level>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we're fully Java 8 now? can use all java 8 features?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, once it merges java8 it is:)

@@ -490,6 +478,13 @@
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>git</artifactId>
<version>${git.version}</version>
<exclusions>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Annoying to have to exclude this everywhere...

if (task.getCauseOfBlockage() != null) {
cause = task.getCauseOfBlockage().getShortDescription();
CauseOfBlockage causeOfBlockage = task.getCauseOfBlockage();
if ( causeOfBlockage != null) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: whitespace

Copy link
Collaborator

@kzantow kzantow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, can we finally use Java 8 features?

@vivek vivek merged commit 86068d3 into master Oct 23, 2017
@vivek vivek deleted the task/JENKINS-47378_jdk8 branch October 23, 2017 13:47
vivek added a commit that referenced this pull request Nov 13, 2017
* JENKINS-47378# Switch to JDK8 and core 2.60.3

* Updated scm sep and workflow job plugins

* workflow-job version > 2.12.2 is compatible with core 2.62 and higher

* Test fix to reliably stop a pipeline run

Also avoids race condition where CPS execution is interrupted while its
in the middle of writing node data to xstream resulting in result
reported as FAILURE due to underlying IOException vs expected ABORTED.

* Bump up core version to 2.60.3 for ATH as well

* Changed jenkins core versions for ath

* Use 2.73.2 LTS instead

* Findbugs fixes

* Log error message in ath test

* Fix build failure in ath runtime-plugins module and upgrade them to
2.73.2

* Added 10 more sec before timeout and some logging

* Increasing timeout to 1 min, lets see if its just slow load of config
page.

* enable nightwatch screenshots by placing "screenshots" config element in the right place

* suppress nightwatch screenshot during "command errors" since these seem superfluous

* Set utf-8 locale in the ubuntu container

* locale-gen and then set locale

* install locales
@vgaidarji vgaidarji mentioned this pull request Sep 20, 2020
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants