You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What language does this apply to?
Python
If it's a proto syntax change, is it for proto2 or proto3? N/A
If it's about generated code change, what programming language? Python
Describe the problem you are trying to solve.
Trying to use protobuf with cpp_implementation in the same python process as a library that's also using protobuf under the covers.
This seems to be a widespread issue in the tensorflow community. My issue isn't with tensorflow but the problem is identical. Tensorflow seems to have fixed this by suggesting people pin to an older version of protobuf before the wheels started shipping the native extension.
Describe the solution you'd like
An option to install a wheel which relies on dynamic linking instead of static linking, allowing both libraries to reference protobuf without causing segfaults and undefined behavior.
I think this could take a few forms:
We could add the native extension as a an extra (e.g. users would pip install protobuf[cpp] to get the cpp extension).
We could ship libprotobuf.*.{dylib, so, dll} as package_data and explicitly load it at import time with cdll which would allow other libraries to also find and load them.
We could publish another package all together (e.g. protobufdynamic)
Describe alternatives you've considered
Tried loading protobuf symbols from _message.*.so instead of libprotobuf`- not all symbols were baked in and seems overall risky/unstable
What language does this apply to?
Python
If it's a proto syntax change, is it for proto2 or proto3? N/A
If it's about generated code change, what programming language? Python
Describe the problem you are trying to solve.
Trying to use protobuf with cpp_implementation in the same python process as a library that's also using protobuf under the covers.
This seems to be a widespread issue in the tensorflow community. My issue isn't with tensorflow but the problem is identical. Tensorflow seems to have fixed this by suggesting people pin to an older version of protobuf before the wheels started shipping the native extension.
(e.g. tensorflow/tensorflow#8394,
tensorflow/tensorflow#9525 (comment)
tensorflow/tensorflow#24976,
tensorflow/tensorflow#35573,
https://github.com/tensorflow/tensorflow/blob/v2.0.0/tensorflow/contrib/makefile/rename_protobuf.sh,
tensorflow/tensorflow#16104)
Describe the solution you'd like
An option to install a wheel which relies on dynamic linking instead of static linking, allowing both libraries to reference protobuf without causing segfaults and undefined behavior.
I think this could take a few forms:
extra
(e.g. users wouldpip install protobuf[cpp]
to get the cpp extension).libprotobuf.*.{dylib, so, dll}
aspackage_data
and explicitly load it at import time withcdll
which would allow other libraries to also find and load them.protobufdynamic
)Describe alternatives you've considered
_message.*.so
instead of libprotobuf`- not all symbols were baked in and seems overall risky/unstableAdditional context
OSX defaults to RTLD_GLOBAL when loading libraries -https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/dlopen.3.html
The text was updated successfully, but these errors were encountered: