Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cached build of repos when doing TravisCI build #223

Closed
wants to merge 15 commits into from

Conversation

travissluka
Copy link
Collaborator

@travissluka travissluka commented Oct 18, 2019

Pay no attention to this for now, testing ideas for doing caching on TravisCI

but if you must know...

Previously, running the tests in debug and with code coverage was timing out because it took so long (well over an hour). Now a TravisCI test take ~7 minutes if the cache is already populated, ~25 minutes otherwise for a full rebuild. 🎉

Instead of running the tests on TravisCI by compiling everything that would normally be in the bundle, each upstream repository is built separately, and then installed to a cached location doing make install. (This required changes to quite a few repos to make sure the install works correctly). On subsequent builds this cached install directory is used instead of rebuilding. The repos are rebuilt if the scripts detect any of several changes (different repo branch name, different git hash tag, an upstream repo was rebuilt, change in order of repo dependencies.... ). The soca repo is always rebuilt every time.

The ccache utility is used to greatly accelerate compiling the C files even when an entire repo needs to be rebuilt. This of course only accelerates the C files, not Fortran, but it's still useful.

By doing the compilation separately this way, the build type can be changed for each repo. Now, the upstream repos are all built with RelWithDebInfo to help run the tests faster, and soca is built with Debug to catch more errors. Code coverage instrumentation usually slows down the code, but now it can be enabled for soca only, and not all the upstream repos. We can choose to instead build all repos, or a subset of the repos, in debug for a longer nightly/weekly cron run.

Requires:

@codecov
Copy link

codecov bot commented Oct 18, 2019

Codecov Report

❗ No coverage uploaded for pull request base (develop@e3452f3). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             develop    #223   +/-   ##
=========================================
  Coverage           ?   91.8%           
=========================================
  Files              ?      84           
  Lines              ?    3671           
  Branches           ?       0           
=========================================
  Hits               ?    3370           
  Misses             ?     301           
  Partials           ?       0
Impacted Files Coverage Δ
src/soca/Model/soca_mom6.F90 90% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e3452f3...fbd3e44. Read the comment docs.

Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@travissluka travissluka force-pushed the bugfix/cmake_install branch 3 times, most recently from b07385e to 8fabc1f Compare October 22, 2019 00:07
@travissluka travissluka deleted the bugfix/cmake_install branch October 22, 2019 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants