Fix tiff lib and remove stub ffmpeg libraries for macOS support #119
Conversation
The video ops used to be linked to a stub (empty) library so that in compile time it is possible to build tensorflow-io without actually providing ffmpeg dynamic library. This causes issues in bazel's sandboxed mode because sandboxed one need to know all files in advance. So far we have been using standalone mode to address this issue. It is still desirable to use sandbox mode as there might be some unintended effects. For example, when we compile tiff we assume jpeg is not supported, but in Linux it is almost certain it will include a header from system library. Which is one of the reasons causeing MacOS to fail (include path different from linux). In short, we should use sandboxed mode for Bazel. This PR fixes the TIFF issue, and also uses dlopen(RTLD_GLOBAL) to resolve ffmpeg symbols in run time, effectively avoids the stub (empty) libary we have. This PR is part of the effort to spport macOS for tensorflow-io. Signed-off-by: Yong Tang <[email protected]>
Signed-off-by: Yong Tang <[email protected]>
|
We will need this fix for macOS support in #116 |
|
@terrytangyuan Yes sandboxed mode is the preferred (and default) mode in bazel and cross aboard. As sandboxed mode gives you a more repeatable result without the impact of the machine you compile (e.g., having some package installed on your linux or not). We were forced to use standalone mode + stub ffmpeg .so because bazel does not support soname like so.57. I created a bug/feature request in Bazel (bazelbuild/bazel#7059) some time ago. It is labeled with In our previous setup, we use a bare-bone Ubuntu 14.04 to build the binary so the binary is (to an extent repeatable). However, because our Ubuntu 14.04 also comes with python and python consists many modules, a lot of header files from other packages are exposed in system directory. This is causing the tiff/jpeg issue when the same code is compiled in macOS. This fix will help make sure header files from installed packages like |
|
After this, should I remove |
|
@BryanCutler Yes there is no need to run with |
Fix tiff lib and remove stub ffmpeg libraries for macOS support
The video ops used to be linked to a stub (empty) library so that in compile time it is possible to build tensorflow-io without actually providing ffmpeg dynamic library. This causes issues in bazel's sandboxed mode because sandboxed one need to know all files in advance. So far we have been using standalone mode to address this issue.
It is still desirable to use sandbox mode as there might be some unintended effects. For example, when we compile tiff we assume jpeg is not supported, but in Linux it is almost certain it will
include a header from system library. This is one of the reasons causing MacOS to fail (include path different from linux).
In short, we should use sandboxed mode for Bazel.
This PR fixes the TIFF issue, and also uses dlopen(RTLD_GLOBAL) to resolve ffmpeg symbols in run time, effectively avoids the stub (empty) libary we have.
This PR is part of the effort to spport macOS for tensorflow-io.
Signed-off-by: Yong Tang [email protected]