Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Travis build matrix to do parallel builds for (make, CMake) x (with CUDA, without CUDA) #943

Merged
merged 1 commit into from
Aug 18, 2014

Conversation

jeffdonahue
Copy link
Contributor

With so many different build options -- use make or CMake, build with CUDA, build/test without CUDA -- doing them all serially with one Travis worker is starting to take a pretty long time (30+ minutes each, even at non-peak hours with good CUDA download speeds). This adds a build matrix to separate them into four builds, each taking < 15 minutes.

@akosiorek do you have any thoughts on why the CMake build tends to take significantly longer? I noticed that some files were built multiple times, e.g. test_caffe_main.o is built for every test rather than just once, but I'm not sure if that's significant. edit: actually it looks like it might just be variance...on this build CMake beat make on the CUDA build.

and CUDA-ful.  Move bits of functionality into scripts under scripts/travis
for readability. Only generate CUDA compute_50 for perfomance.
@bhack
Copy link
Contributor

bhack commented Aug 17, 2014

How much time we will maintain Makefiles? Any change that impact the build system need a double effort now that cmake branch is merged.

@jeffdonahue
Copy link
Contributor Author

I don't know.

@jeffdonahue
Copy link
Contributor Author

@bhack given that the existing, more manual, Makefile.config-based build currently works well for many users (as much as they may have been frustrated and struggled throughout the process of setting it up initially), and the CMake build is relatively less tested and adds extra installation steps for many users with a currently working build environment (e.g., Ubuntu 12.04), we will keep it in place indefinitely. In the meantime, users and private developers are free to use whichever build system they feel more comfortable using.

For contributions to the official BVLC/Caffe repo, yes, developers will have to not break either of the two build systems, but I honestly think that will be a pretty minimal burden for the majority of contributions. And if someone makes a PR that is really useful but is failing in Travis only using CMake or only using make, one or more of the main developers at BVLC will probably be able to provide some guidance on how to make it work; I don't think this is going to be a serious problem.

Later, if we agree that we're comfortable that the new CMake-based build system works just as well as the existing system and we (as the core BVLC developers) are comfortable providing support for using and troubleshooting it, we can think about the process of deprecating the Makefile. Currently I think I'm the only person here at Berkeley who's tried the CMake build system (which means that, e.g., I don't even know if it works for OSX users), and the build system is honestly not really anyone's highest priority here, so we are not in a hurry to start deprecating the existing build system at this time.

jeffdonahue added a commit that referenced this pull request Aug 18, 2014
add Travis build matrix to do parallel builds for (make, CMake) x (with CUDA, without CUDA)
@jeffdonahue jeffdonahue merged commit 0a9a43c into BVLC:dev Aug 18, 2014
@bhack
Copy link
Contributor

bhack commented Aug 18, 2014

I hope that maintaining a duplicate build system will not let one of two to be less tested by users. I hope that in the long run we can "make" makefile deprecated :)

@jeffdonahue jeffdonahue deleted the parallel-travis-builds branch August 18, 2014 02:10
@akosiorek
Copy link
Contributor

test_caffe_main.o is built multiple times due to test_cafe_main.cpp being included in multiple targets. Since CMake have no way of knowing that's the same file, it builds it every time. CMake 2.8.8 introduced an Object Library which handles this issue. However, it didn't work well last time I checked. I'll try to work it out.

mitmul pushed a commit to mitmul/caffe that referenced this pull request Sep 30, 2014
add Travis build matrix to do parallel builds for (make, CMake) x (with CUDA, without CUDA)
RazvanRanca pushed a commit to RazvanRanca/caffe that referenced this pull request Nov 4, 2014
add Travis build matrix to do parallel builds for (make, CMake) x (with CUDA, without CUDA)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants