Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add tbb to travis-ci #210

Closed
wants to merge 2 commits into from
Closed

add tbb to travis-ci #210

wants to merge 2 commits into from

Conversation

acxz
Copy link
Contributor

@acxz acxz commented Jan 12, 2020

Addresses #209 (comment)
Not completely sure if I added the TBB compilation where you guys wanted it, but I used my best judgement. Feel free to change this PR.


This change is Reviewable

@acxz
Copy link
Contributor Author

acxz commented Jan 12, 2020

hmm interestingly clang with release mode errors out with TBB: https://travis-ci.com/borglab/gtsam/jobs/274705966#L1506

Copy link
Member

@dellaert dellaert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, but this will significantly increase CI time. I propose to make every Debug build be TBB=OFF, and just add one “special” build with TBB and Debug to make sure stuff compiles.

@dellaert
Copy link
Member

/home/travis/build/borglab/gtsam/gtsam/linear/VectorValues.h:201:22: error: no member named 'emplace' in 'gtsam::ConcurrentMap<unsigned long, Eigen::Matrix<double, -1, 1, 0, -1, 1> >'
1507      return values_.emplace(j, value);
1508             ~~~~~~~ ^
15091 error generated.

Only on clang on Linux. Very bizarre. I propose - if you agree - to make the changes to Travis.yml as I suggested which will also trigger a new Travis build. This could be a Travis fluke, as it seems completely orthogonal to the changes.

@acxz
Copy link
Contributor Author

acxz commented Jan 12, 2020

I’ll make the changes, but it may not be a fluke. I’ll compile on my machine with clang and see if the output matches. Basically tbb has its own version of ‘concurrent_unordered_map’, in which ‘emplace’ has a very slight difference. But maybe ‘emplace’ is not available in the TBB version shipped with Ubuntu 16.

@chrisbeall
Copy link
Member

chrisbeall commented Jan 12, 2020

We ran into this issue with clang + Ubuntu 16.04 + TBB 4.4 at work. The problem is that TBB only enables C++ 11 features such as emplace if __LIBCPP_VERSION is set, and clang does not set it. (Everything works fine with gcc). A forum post with details:
https://software.intel.com/en-us/forums/intel-threading-building-blocks/topic/607423
I suspect this is fixed in newer versions of TBB, but didn't check yet.

A possible hack/workaround:

#ifdef GTSAM_USE_TBB
      values_.insert(std::make_pair(key, x.segment(j, n)));
#else
      values_.emplace(key, x.segment(j, n));
#endif

Could also ifdef on TBB version number if it's fixed in recent version of TBB.

@acxz
Copy link
Contributor Author

acxz commented Jan 12, 2020

@chrisbeall interesting. I just tested clang with TBB 2020 on my machine and I did not get any errors. We should do ifdef based on TBB_VERSION, so future versions with the fix can use it. We know that TBB 4.4 does not work and TBB 2020 works. I'm a bit too lazy to find which version they fixed it in, so should we guard with TBB_VERSION <= 4.4 or TBB_VERSION >= 2020?

@dellaert
Copy link
Member

TBB_VERSION >= 2020 seems safer

@acxz
Copy link
Contributor Author

acxz commented Jan 12, 2020

Should I add the commit to this PR or make another one? The travis build won't succeed otherwise.

@dellaert
Copy link
Member

This PR, as indeed otherwise we can't land it...

@acxz
Copy link
Contributor Author

acxz commented Jan 13, 2020

Sweet we got it building. LGTM!

Copy link
Member

@dellaert dellaert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great that it builds! But, two comments to this PR:

  1. what happens if we are not compiling with TBB? I think maybe emplace - which would exist then? - is not used?
  2. looking at this pattern, which is copy/pasted 4 times, it seems the right solution is to add an "emplace" to ConcurrentMap if it's missing.

@acxz
Copy link
Contributor Author

acxz commented Jan 13, 2020

Yep, you're right. Adding emplace to ConcurrentMap would be the cleaner solution. It would also resolve the first point, since ConcurrentMap is only used if TBB is found, otherwise FastMap is used, which will already have an emplace function. I'll add the change either tomorrow or later today.

EDIT: I have some deadlines coming up, so I won't be able to tackle this issue until then.

@acxz
Copy link
Contributor Author

acxz commented Feb 11, 2020

I know it has been a while since I have responded, but I cannot seem to resolve the issue in a smart way. Since I do not have a Ubuntu 16 machine to test with either, this is making it hard to resolve this Clang issue. I had thought overriding the emplace function with insert functionality would work, but sadly it does not. Adding the virtual didn't help either. I'll have to stop working on this for the time being or if someone could resolve this issue it would be much appreciated.

@varunagrawal varunagrawal added the feature New proposed feature label Feb 23, 2020
@varunagrawal
Copy link
Collaborator

#250 fixes all the issues that were attempted to be solved in this PR and works great (including fixing the Clang issue). Closing this. Thanks @acxz!

@acxz acxz deleted the tbb_ci branch May 6, 2020 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New proposed feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants