Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImportError: dynamic module does not define module export function (PyInit__caffe) #6054

Open
monajalal opened this issue Nov 13, 2017 · 15 comments

Comments

@monajalal
Copy link

[jalal@goku caffe]$ pwd
/scratch/caffe

$make clean
$mkdir build
$cd build 
$cmake -DBLAS=open ..
$make install all -j 12
$sudo make install

(I get no error for above). The only change I have made in the code I got from git repo is copying Makefile.config.example to Makefile.config and changing a few things to reflect I am using opencv, Anaconda Python and also cudnn

[jalal@goku caffe]$ python
Python 3.5.4 |Anaconda custom (64-bit)| (default, Nov  9 2017, 00:39:12) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/scratch/caffe/python/caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
  File "/scratch/caffe/python/caffe/pycaffe.py", line 13, in <module>
    from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: dynamic module does not define module export function (PyInit__caffe)
>>> quit()
[jalal@goku caffe]$ tail -5 ~/.bashrc
#export PYTHONPATH=$HOME/caffe/python
export PYTHONPATH=/scratch/caffe/python
#export CAFFE_ROOT=$HOME/caffe

export PATH=/usr/local/anaconda3/bin/python:$PATH


I basically need to import caffe in the Anaconda Python 3.
ImportError: dynamic module does not define module export function (PyInit__caffe)

Here is the Makefile.config

[jalal@goku caffe]$ cat Makefile.config
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 1
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#	You should not set this flag if you will be reading LMDBs with any
#	possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
		-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_52,code=sm_52 \
		-gencode arch=compute_60,code=sm_60 \
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /usr/lib64/atlas #added by Mona
BLAS_LIB := /usr/lib64/atlas #added by Mona

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
#		/usr/lib/python2.7/dist-packages/numpy/core/include


# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := /usr/local/anaconda3
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		 $(ANACONDA_HOME)/include/python3.5m \
		 $(ANACONDA_HOME)/lib/python3.5/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
#PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @
[jalal@goku caffe]$ 

Please suggest what is wrong and how to fix it?

[jalal@goku caffe]$ uname -a
Linux goku.bu.edu 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

[jalal@goku caffe]$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.4.1708 (Core)
Release: 7.4.1708
Codename: Core



>>> print(cv2.getBuildInformation())

General configuration for OpenCV 3.1.0 =====================================
  Version control:               unknown

  Platform:
    Host:                        Linux 4.8.0-46-generic x86_64
    CMake:                       3.6.3
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/gmake
    Configuration:               Release

  C/C++:
    Built as dynamic libs?:      YES
    C++ Compiler:                /opt/rh/devtoolset-2/root/usr/bin/c++  (ver 4.8.2)
    C++ flags (Release):         -I/cs/software/anaconda3/include    -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -I/cs/software/anaconda3/include    -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /opt/rh/devtoolset-2/root/usr/bin/cc
    C flags (Release):           -I/cs/software/anaconda3/include    -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fopenmp -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -I/cs/software/anaconda3/include    -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fopenmp -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):
    Linker flags (Debug):
    Precompiled headers:         YES
    Extra dependencies:          /cs/software/anaconda3/lib/libjpeg.so /cs/software/anaconda3/lib/libpng.so /cs/software/anaconda3/lib/libtiff.so /cs/software/anaconda3/lib/libhdf5.so /usr/lib64/librt.so /usr/lib64/libpthread.so /cs/software/anaconda3/lib/libz.so /usr/lib64/libdl.so /usr/lib64/libm.so dl m pthread rt
    3rdparty dependencies:       libwebp libjasper IlmImf libprotobuf

  OpenCV modules:
    To be built:                 core flann hdf imgproc ml photo reg surface_matching video dnn fuzzy imgcodecs shape videoio highgui objdetect plot superres xobjdetect xphoto bgsegm bioinspired dpm face features2d line_descriptor saliency text calib3d ccalib datasets rgbd stereo structured_light tracking videostab xfeatures2d ximgproc aruco optflow stitching python3
    Disabled:                    world contrib_world
    Disabled by dependency:      -
    Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev java python2 ts viz cvv matlab sfm

  GUI: 
    QT:                          NO
    GTK+:                        NO
    GThread :                    NO
    GtkGlExt:                    NO
    OpenGL support:              NO
    VTK support:                 NO

  Media I/O: 
    ZLib:                        /cs/software/anaconda3/lib/libz.so (ver 1.2.8)
    JPEG:                        /cs/software/anaconda3/lib/libjpeg.so (ver 80)
    WEBP:                        build (ver 0.3.1)
    PNG:                         /cs/software/anaconda3/lib/libpng.so (ver 1.6.27)
    TIFF:                        /cs/software/anaconda3/lib/libtiff.so (ver 42 - 4.0.6)
    JPEG 2000:                   build (ver 1.900.1)
    OpenEXR:                     build (ver 1.7.1)
    GDAL:                        NO

  Video I/O:
    DC1394 1.x:                  NO
    DC1394 2.x:                  NO
    FFMPEG:                      NO
      codec:                     NO
      format:                    NO
      util:                      NO
      swscale:                   NO
      resample:                  NO
      gentoo-style:              NO
    GStreamer:                   NO
    OpenNI:                      NO
    OpenNI PrimeSensor Modules:  NO
    OpenNI2:                     NO
    PvAPI:                       NO
    GigEVisionSDK:               NO
    UniCap:                      NO
    UniCap ucil:                 NO
    V4L/V4L2:                    YES/YES
    XIMEA:                       NO
    Xine:                        NO
    gPhoto2:                     NO

  Parallel framework:            OpenMP

  Other third-party libraries:
    Use IPP:                     9.0.1 [9.0.1]
         at:                     /opt/conda/conda-bld/opencv_1491943704081/work/opencv-3.1.0/3rdparty/ippicv/unpack/ippicv_lnx
    Use IPP Async:               NO
    Use VA:                      NO
    Use Intel VA-API/OpenCL:     NO
    Use Eigen:                   YES (ver 3.2.8)
    Use Cuda:                    NO
    Use OpenCL:                  NO
    Use custom HAL:              NO

  Python 2:
    Interpreter:                 (ver 3.5.3)

  Python 3:
    Interpreter:                 /cs/software/anaconda3/bin/python (ver 3.5.3)
    Libraries:                   /cs/software/anaconda3/lib/libpython3.5m.so (ver 3.5.3)
    numpy:                       /cs/software/anaconda3/lib/python3.5/site-packages/numpy/core/include (ver 1.12.1)
    packages path:               /cs/software/anaconda3/lib/python3.5/site-packages

  Python (for build):            

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Matlab:                        NO

  Tests and samples:
    Tests:                       NO
    Performance tests:           NO
    C/C++ Examples:              NO

  Install path:                  /cs/software/anaconda3

  cvconfig.h is in:              /opt/conda/conda-bld/opencv_1491943704081/work/opencv-3.1.0/build
-----------------------------------------------------------------


>>> cv2.__version__
'3.1.0'
>>> 



@nn-ww
Copy link

nn-ww commented Jan 23, 2018

Hi monajalal,
I also get the same error:
import error: dynamic module does not define module export function (PyInit__caffe)
Have you solved the error?

@sunblade
Copy link

sunblade commented Feb 8, 2018

I have the same issue, but in my case it was working just after compiling source. I have exited a shell, and after opening it again problem started.

.bashrc

export PYTHONPATH=$HOME/caffe/python:$PYTHONPATH

@nn-ww
Copy link

nn-ww commented Feb 9, 2018 via email

@pencilzhang
Copy link

@nn-ww Hi, how did you solve this issue? Thanks!

@WUhailing
Copy link

@pencilzhang Did you fix it? I ran into the same dynamic error when import caffe.

@nn-ww
Copy link

nn-ww commented Mar 23, 2018 via email

@WUhailing
Copy link

@nn-ww Thanks. It turns out to be 'cmake' issue, so switch to 'make', it works.

@nn-ww
Copy link

nn-ww commented Mar 23, 2018 via email

@pencilzhang
Copy link

Thank you @nn-ww . I switch to cmake then everything works fine on a Mac OS system. @WUhailing

@AllenYLJiang
Copy link

I've solved the problem. In my case, in the first anaconda envs, the protoc is not compatible with pycaffe (caffe), so I entered another envs.

@mrgloom
Copy link

mrgloom commented Feb 24, 2019

This problem happens when you use python2 build with python3

Just uncomment:

# Uncomment to use Python 3 (default is Python 2)
PYTHON_LIBRARIES := boost_python3 python3.5m
PYTHON_INCLUDE := /usr/include/python3.5m \
                 /usr/lib/python3.5/dist-packages/numpy/core/include

fanghuaqi added a commit to foss-for-synopsys-dwc-arc-processors/synopsys-caffe that referenced this issue Aug 15, 2019
@selvakarna
Copy link

Hi @mrgloom ,
I got following error when i import cadquery package, this package are installed but i got error bellow,

import freecad
import cadquery
Traceback (most recent call last):
File "/home/hp/.local/lib/python3.6/site-packages/cadquery/freecad_impl/init.py", line 151, in
import FreeCAD
ModuleNotFoundError: No module named 'FreeCAD'

During handling of the above exception, another exception occurred:

import FreeCAD
ImportError: dynamic module does not define module export function (PyInit_FreeCAD)

@mjrlgue
Copy link

mjrlgue commented Dec 17, 2019

for futur readers, this error is caused because of the default python version is set to 3.x.x, I set it to python 2 using pyenv global system and then import caffe worked !

@alex9311
Copy link

for what it's worth, I got this error because I had compiled Caffe for python2.7 but I'm running python 3. So I was in the inverse situation that @mjrlgue mentions above. The solution was to add -D python_version=3 flag to the cmake call.

git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git . && \
    pip install --upgrade pip && \
    cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. && \
    mkdir build && cd build && \
    cmake -DCPU_ONLY=1  -D python_version=3 .. && \
    make -j2

(note that the above is for a CPU build)

@kangchunhung
Copy link

@alex9311 This works for me, really appreciate that. My platform is Nvidia Jetson AGX Xavier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests