changes to overcome performance issues with huge fitnesse result files #10

ricointersys · 2013-12-05T10:23:08Z

(several MP)

Hi,
our company had an issue with performance of this plugin due to huge fitnesse results. Since the fitnesse results are stored in build.xml this file had a size of several MB. Details are explained here: https://issues.jenkins-ci.org/browse/JENKINS-13936.

We overcome the issue as follows:

do not store the fitnesse-results in build.xml, put them in separate files (in same directory as build.xml)
in addition we added a new column to directly jump to the fitnesse history page

We would like to give back the changes if you are interested in. there is still some work to do

update test cases
documentation

Please let me know if you are interested
Rico

(several MP)

cloudbees-pull-request-builder · 2013-12-05T10:25:48Z

plugins » fitnesse-plugin #10 UNSTABLE
Looks like there's a problem with this pull request

ricointersys · 2013-12-05T10:29:55Z

Hi,
as stated in the pull request, if you are interested in the changes I will fix the tests as well.
Regards
Rico

Von: CloudBees pull request builder plugin [mailto:[email protected]]
Gesendet: Donnerstag, 5. Dezember 2013 11:26
An: jenkinsci/fitnesse-plugin
Cc: Rico Gaudlitz
Betreff: Re: [fitnesse-plugin] changes to overcome performance issues with huge fitnesse result files (#10)

plugins » fitnesse-plugin #10 https://jenkins.ci.cloudbees.com/job/plugins/job/fitnesse-plugin/10/ UNSTABLE
Looks like there's a problem with this pull request

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10#issuecomment-29886703.

jenkinsadmin · 2013-12-05T13:01:50Z

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

lessonz · 2013-12-06T05:14:10Z

src/main/java/hudson/plugins/fitnesse/FitnesseResultsRecorder.java

+			logger.println("all-content: " + pageCounts.getAllContents().size());
+			logger.println("resultsFile: " + getFitnessePathToXmlResultsIn());
+			// logger.println("content: " +
+			// pageCounts.getAllContents().values().iterator().next());


Let's not leave commented-out code.

lessonz · 2013-12-06T05:20:20Z

I think this would be a good addition to the project. Thanks for your efforts. Yes, please correct the unit tests. Also, having only had a chance to skim the code, why remove so much envVar references?

cloudbees-pull-request-builder · 2013-12-06T09:52:50Z

plugins » fitnesse-plugin #11 SUCCESS
This pull request looks good

ricointersys · 2013-12-06T10:02:51Z

Thanks for the review.
I did the requested updates and provided some documentation. Hope it is helpful.

I’d like to raise your attention to some ugly detail. To restore the saved fitnesse results (which are now in separate files) I had to store the filename (with directory) in the Jenkins-result. Is there a better way to retrieve the path to the current build in hudson.plugins.fitnesse.ResultsDetails.getDetailsHtml()?

why remove so much envVar references?
Sorry I do not get your question. I hope that my changes didn’t touch any envVar references. Could you please point me to the code?

Best regards
Rico

Von: lessonz [mailto:[email protected]]
Gesendet: Freitag, 6. Dezember 2013 06:21
An: jenkinsci/fitnesse-plugin
Cc: Rico Gaudlitz
Betreff: Re: [fitnesse-plugin] changes to overcome performance issues with huge fitnesse result files (#10)

I think this would be a good addition to the project. Thanks for your efforts. Yes, please correct the unit tests. Also, having only had a chance to skim the code, why remove so much envVar references?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10#issuecomment-29964264.

lessonz · 2013-12-08T06:03:09Z

src/main/java/hudson/plugins/fitnesse/FitnesseBuilder.java

@@ -73,17 +71,6 @@ private String getOption(String key, String valueIfKeyNotFound) {
    	return valueIfKeyNotFound;
    }

-    private String getOption(String key, String valueIfKeyNotFound, EnvVars environment) { 


This method, and calls to it, are what I was referring to with regards to your removing EnvVars references.

Just seen the changes. This is maybe an issue with the Fork I created. Those changes are by mistake. Shall I restore the previous version?
Sorry
Rico

Von: lessonz [mailto:[email protected]]
Gesendet: Sonntag, 8. Dezember 2013 07:03
An: jenkinsci/fitnesse-plugin
Cc: Rico Gaudlitz
Betreff: Re: [fitnesse-plugin] changes to overcome performance issues with huge fitnesse result files (#10)

In src/main/java/hudson/plugins/fitnesse/FitnesseBuilder.java:

@@ -73,17 +71,6 @@ private String getOption(String key, String valueIfKeyNotFound) {

return valueIfKeyNotFound; }

private String getOption(String key, String valueIfKeyNotFound, EnvVars environment) {

This method, and calls to it, are what I was referring to with regards to your removing EnvVars references.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10/files#r8183926.

Yes, please try to ensure only the files you intended to change are actually modified. Thanks!

Compared all files now with the history. Hope that I reverted all unintentional changes.

Von: lessonz [mailto:[email protected]]
Gesendet: Dienstag, 10. Dezember 2013 07:12
An: jenkinsci/fitnesse-plugin
Cc: Rico Gaudlitz
Betreff: Re: [fitnesse-plugin] changes to overcome performance issues with huge fitnesse result files (#10)

In src/main/java/hudson/plugins/fitnesse/FitnesseBuilder.java:

@@ -73,17 +71,6 @@ private String getOption(String key, String valueIfKeyNotFound) {

return valueIfKeyNotFound; }

private String getOption(String key, String valueIfKeyNotFound, EnvVars environment) {

Yes, please try to ensure only the files you intended to change are actually modified. Thanks!

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10/files#r8223222.

I'm sorry, but I don't see your most recent changes. Perhaps you forgot to push them?

That is right, I forgot to push it … . Did it now.

Von: lessonz [mailto:[email protected]]
Gesendet: Mittwoch, 11. Dezember 2013 07:07
An: jenkinsci/fitnesse-plugin
Cc: Rico Gaudlitz
Betreff: Re: [fitnesse-plugin] changes to overcome performance issues with huge fitnesse result files (#10)

In src/main/java/hudson/plugins/fitnesse/FitnesseBuilder.java:

@@ -73,17 +71,6 @@ private String getOption(String key, String valueIfKeyNotFound) {

return valueIfKeyNotFound; }

private String getOption(String key, String valueIfKeyNotFound, EnvVars environment) {

I'm sorry, but I don't see your most recent changes. Perhaps you forgot to push them?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10/files#r8258257.

lessonz · 2013-12-08T06:06:53Z

Please understand I only relatively recently took over this plugin. I am only responsible for about a grand total of 5 lines of the code base so far. With regards to retrieving the path to the current build, this is something I'd have to look into further.

cloudbees-pull-request-builder · 2013-12-11T06:26:16Z

plugins » fitnesse-plugin #12 SUCCESS
This pull request looks good

changes to overcome performance issues with huge fitnesse result files

lessonz · 2013-12-12T04:26:49Z

Thanks so much for working through the issues. It's greatly appreciated.

ricointersys · 2013-12-12T06:23:58Z

Thank you as well for your patience and acceptance of the change. It was a pleasure to work on this.

Von: lessonz [mailto:[email protected]]
Gesendet: Donnerstag, 12. Dezember 2013 05:27
An: jenkinsci/fitnesse-plugin
Cc: Rico Gaudlitz
Betreff: Re: [fitnesse-plugin] changes to overcome performance issues with huge fitnesse result files (#10)

Thanks so much for working through the issues. It's greatly appreciated.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/10#issuecomment-30388600.

changes to overcome performance issues with huge fitnesse result files

ade15d6

(several MP)

lessonz reviewed Dec 6, 2013
View reviewed changes

ricointersys added 3 commits December 6, 2013 08:29

forgot to commit it in pull request

7c875cb

code cleanup and documentation

0e8a0af

fixed tests

6f29eb6

lessonz reviewed Dec 8, 2013
View reviewed changes

restore previous changes. My Checkin was wrong.

d39e331

lessonz added a commit that referenced this pull request Dec 12, 2013

Merge pull request #10 from ricointersys/master

5834f5f

changes to overcome performance issues with huge fitnesse result files

lessonz merged commit 5834f5f into jenkinsci:master Dec 12, 2013

antoine-aumjaud mentioned this pull request Aug 29, 2014

Store HTML content in file during the XML parsing #16

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

changes to overcome performance issues with huge fitnesse result files #10

changes to overcome performance issues with huge fitnesse result files #10

ricointersys commented Dec 5, 2013

cloudbees-pull-request-builder commented Dec 5, 2013

ricointersys commented Dec 5, 2013

jenkinsadmin commented Dec 5, 2013

lessonz Dec 6, 2013

lessonz commented Dec 6, 2013

cloudbees-pull-request-builder commented Dec 6, 2013

ricointersys commented Dec 6, 2013

lessonz Dec 8, 2013

ricointersys Dec 9, 2013

lessonz Dec 10, 2013

ricointersys Dec 10, 2013

lessonz Dec 11, 2013

ricointersys Dec 11, 2013

lessonz commented Dec 8, 2013

cloudbees-pull-request-builder commented Dec 11, 2013

lessonz commented Dec 12, 2013

ricointersys commented Dec 12, 2013

changes to overcome performance issues with huge fitnesse result files #10

changes to overcome performance issues with huge fitnesse result files #10

Conversation

ricointersys commented Dec 5, 2013

cloudbees-pull-request-builder commented Dec 5, 2013

ricointersys commented Dec 5, 2013

jenkinsadmin commented Dec 5, 2013

lessonz Dec 6, 2013

Choose a reason for hiding this comment

lessonz commented Dec 6, 2013

cloudbees-pull-request-builder commented Dec 6, 2013

ricointersys commented Dec 6, 2013

lessonz Dec 8, 2013

Choose a reason for hiding this comment

ricointersys Dec 9, 2013

Choose a reason for hiding this comment

lessonz Dec 10, 2013

Choose a reason for hiding this comment

ricointersys Dec 10, 2013

Choose a reason for hiding this comment

lessonz Dec 11, 2013

Choose a reason for hiding this comment

ricointersys Dec 11, 2013

Choose a reason for hiding this comment

lessonz commented Dec 8, 2013

cloudbees-pull-request-builder commented Dec 11, 2013

lessonz commented Dec 12, 2013

ricointersys commented Dec 12, 2013