Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Known macOS build issues with Protobuf version 3.19 and 3.20 #1936

Closed
hamptonm1 opened this issue Dec 20, 2022 · 8 comments
Closed

Known macOS build issues with Protobuf version 3.19 and 3.20 #1936

hamptonm1 opened this issue Dec 20, 2022 · 8 comments
Assignees

Comments

@hamptonm1
Copy link
Collaborator

hamptonm1 commented Dec 20, 2022

When upgrading protobuf from 3.18.3 to 3.20.3, I noticed the macOS build failed with the following:

Undefined symbols for architecture x86_64:
[720](https://github.com/onnx/onnx-mlir/actions/runs/3743651637/jobs/6356066681#step:10:721)
  "google::protobuf::internal::InternalMetadata::~InternalMetadata()", referenced from:
[721](https://github.com/onnx/onnx-mlir/actions/runs/3743651637/jobs/6356066681#step:10:722)
      google::protobuf::MessageLite::~MessageLite() in libonnx_proto.a(onnx-ml.pb.cc.o)
[722](https://github.com/onnx/onnx-mlir/actions/runs/3743651637/jobs/6356066681#step:10:723)
      google::protobuf::MessageLite::~MessageLite() in libonnx_proto.a(onnx-data.pb.cc.o)
[723](https://github.com/onnx/onnx-mlir/actions/runs/3743651637/jobs/6356066681#step:10:724)
ld: symbol(s) not found for architecture x86_64

This is actually a known issue in the protobuf community, please see the following issues:
protocolbuffers/protobuf#9947
facebookincubator/velox#2029

This problem is fixed in protobuf version 3.21.3, however onnx community has not upgraded to that version. Instead they are on version 3.20.3 which is where the failure lies. There is a workaround if we still want to upgrade to version 3.20.3 by adding the following to the CMake.txt file:

add_definitions(-DNDEBUG)

@hamptonm1 hamptonm1 self-assigned this Dec 20, 2022
@hamptonm1
Copy link
Collaborator Author

hamptonm1 commented Dec 20, 2022

Should we upgrade to protobuf 3.20.3 and include the workaround or should we wait for onnx to upgrade to 3.21.3 which fixes the issue and therefore we do not need the workaround. @AlexandreEichenberger @gongsu832 @chentong319 @tungld ? Any comments about this topic :)

@gongsu832
Copy link
Collaborator

In general, only upgrade when there is a specific reason to do so. Stay in sync with onnx is desirable but if the new version has known problem like the one above, I'd wait until things get sorted out.

@chentong319
Copy link
Collaborator

To my understanding, protobuf 3.20.3 is required by onnx opset 18. protobuf 3.21 is needed for Mac to fix the issue with protobuf 3.20.3 on Mac. Can protobuf 3.21.3 work with other components in onnx-mlir? Open a draft PR to try?

@hamptonm1
Copy link
Collaborator Author

To my understanding, protobuf 3.20.3 is required by onnx opset 18. protobuf 3.21 is needed for Mac to fix the issue with protobuf 3.20.3 on Mac. Can protobuf 3.21.3 work with other components in onnx-mlir? Open a draft PR to try?

So the onnx community upgraded to 3.20.2 because 3.18.3 is out of maintenance window as shown in the following PR : onnx/onnx#4629. So it is not required to use opset 18 from what I read. I can create a draft PR and test what happens while using protobuf 3.21. Then everyone can decide based on my results.

@chentong319
Copy link
Collaborator

Thanks for your efforts. Your draft PR ran through on Mac, AMD and PPC Linux, but failed on S390 and Windows. It seems that Protobuf 3.20.2 is not available for Windows and can not be built on S390.

@hamptonm1
Copy link
Collaborator Author

Thanks for your efforts. Your draft PR ran through on Mac, AMD and PPC Linux, but failed on S390 and Windows. It seems that Protobuf 3.20.2 is not available for Windows and can not be built on S390.

@chentong319 Yes I tested Protobuf 3.20.2, 3.20.3 and 3.21.3 which failed for some checks and not for others. So I guess we will have to remain at the version we are currently on.

@chentong319
Copy link
Collaborator

Yes, remain at the version we are currently on. Is it possible to use a different version for Mac only? Even if the git build for Mac fails for a while due to the Protobuf 3.18, I am building onnx-mlir locally on my Mac using Protobuf 3.20.3.

@hamptonm1
Copy link
Collaborator Author

Yes, remain at the version we are currently on. Is it possible to use a different version for Mac only? Even if the git build for Mac fails for a while due to the Protobuf 3.18, I am building onnx-mlir locally on my Mac using Protobuf 3.20.3.

Interesting!! Okay let me try that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants