Caffe build failed on Ubuntu 16.04 because of protobuf errors

[spurani]

Basically I have followed your guide to install and build caffe entirely but it seems that it fails giving this error
AR -o .build_release/lib/libcaffe.a
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/ld: cannot find -lboost_python3-py35
collect2: error: ld returned 1 exit status
Makefile:582: recipe for target '.build_release/lib/libcaffe.so.1.0.0' failed
make: *** [.build_release/lib/libcaffe.so.1.0.0] Error 1

So I did my own research of building caffe https://medium.com/@atinesh/caffe-installation-on-ubuntu-18-04-lts-python-3-6-e76375f0d353
BVLC/caffe#6720 => Similar issue
Not sure if this issue is caused by step in your guide install caffe on ubuntu python3
9 Now rename boost_python3 in PYTHON_LIBRARIES to boost_python-py35. PYTHON_LIBRARIES should look like this.

PYTHON_LIBRARIES := boost_python-py35

It would be really helpful if you can guide me. Please feel free to reach out should you have any questions

My Makefile.config

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#	You should not set this flag if you will be reading LMDBs with any
#	possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH :=# -gencode arch=compute_20,code=sm_20 \
#		-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_52,code=sm_52 \
		-gencode arch=compute_60,code=sm_60 \
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
#		/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)
 PYTHON_LIBRARIES := boost_python3-py35 python3.5m
 PYTHON_INCLUDE := /usr/include/python3.5m /usr/lib/python3.5/dist-packages/numpy/core/include /home/sp/.local/lib/python3.5/site-packages
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib/x86_64-linux-gnu/
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
 WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial /usr/lib/x86_64-linux-gnu/

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

[spurani]

@adeelz92 A quick update on this issue I solved the previous error by making minor changes in my makefile.config as per your guide but now I am another issue which is based on protobuf. Can you please take a at my issue and help me guide through this. I am still learning and trying my best to solve this issue. But it would be great if you can drop any suggestions or advice which will help me learn new concept.Thanks and cheers

AR -o .build_release/lib/libcaffe.a
LD -o .build_release/lib/libcaffe.so.1.0.0
CXX tools/upgrade_net_proto_text.cpp
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedInputStream::DecrementRecursionDepthAndPopLimit(int)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::WireFormatLite::UInt32Size(google::protobuf::RepeatedField const&)'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::WireFormatLite::WriteFloatArray(float const*, int, google::protobuf::io::CodedOutputStream*)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedInputStream::ReadVarint64Fallback()'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedOutputStream::default_serialization_deterministic_' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::Arena::AddListNode(void*, void ()(void))'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::WireFormat::ReadPackedEnumPreserveUnknowns(google::protobuf::io::CodedInputStream*, unsigned int, bool (*)(int), google::protobuf::UnknownFieldSet*, google::protobuf::RepeatedField<int>*)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedInputStream::IncrementRecursionDepthAndPushLimit(int)'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::WireFormatLite::Int32Size(google::protobuf::RepeatedField<int> const&)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedInputStream::ReadVarint32Fallback(unsigned int)'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedOutputStream::WriteVarint64SlowPath(unsigned long)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::RegisterAllTypes(google::protobuf::Metadata const*, int)'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedOutputStream::WriteVarint32SlowPath(unsigned int)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::InitProtobufDefaults()'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::Arena::AllocateAligned(std::type_info const*, unsigned long)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedInputStream::ReadLengthAndPushLimit()'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::fixed_address_empty_string[abi:cxx11]' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedInputStream::CheckEntireMessageConsumedAndPopLimit(int)'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::WireFormatLite::WriteDoubleArray(double const*, int, google::protobuf::io::CodedOutputStream*)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedInputStream::ReadVarintSizeAsIntFallback()'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::io::CodedInputStream::ReadTagFallback(unsigned int)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::AssignDescriptors(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, google::protobuf::internal::MigrationSchema const*, google::protobuf::Message const* const*, unsigned int const*, google::protobuf::MessageFactory*, google::protobuf::Metadata*, google::protobuf::EnumDescriptor const**, google::protobuf::ServiceDescriptor const**)'
.build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::RepeatedPtrFieldBase::InternalExtend(int)' .build_release/lib/libcaffe.so: undefined reference to google::protobuf::internal::ArenaStringPtr::AssignWithDefault(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const*, google::protobuf::internal::ArenaStringPtr)'
collect2: error: ld returned 1 exit status
Makefile:635: recipe for target '.build_release/tools/upgrade_net_proto_text.bin' failed
make: *** [.build_release/tools/upgrade_net_proto_text.bin] Error 1

[spurani]

@adeelz92 Another quick update on this issue. Finally I was able to successfully build and install caffe with python 3.

make all
CXX/LD -o .build_release/examples/cpp_classification/classification.bin

make runtest
[----------] Global test environment tear-down
[==========] 2059 tests from 272 test cases ran. (361672 ms total)
[ PASSED ] 2059 tests.

make pycaffe
CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
touch python/caffe/proto/init.py
PROTOC (python) src/caffe/proto/caffe.proto

But now its giving me errors when i import it
Traceback (most recent call last):
File "", line 1, in
File "/home/sp/caffe/python/caffe/init.py", line 1, in
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
File "/home/sp/caffe/python/caffe/pycaffe.py", line 15, in
import caffe.io
File "/home/sp/caffe/python/caffe/io.py", line 2, in
import skimage.io
File "/usr/lib/python3/dist-packages/skimage/io/init.py", line 15, in
reset_plugins()
File "/usr/lib/python3/dist-packages/skimage/io/manage_plugins.py", line 91, in reset_plugins
_load_preferred_plugins()
File "/usr/lib/python3/dist-packages/skimage/io/manage_plugins.py", line 71, in _load_preferred_plugins
_set_plugin(p_type, preferred_plugins['all'])
File "/usr/lib/python3/dist-packages/skimage/io/manage_plugins.py", line 83, in _set_plugin
use_plugin(plugin, kind=plugin_type)
File "/usr/lib/python3/dist-packages/skimage/io/manage_plugins.py", line 253, in use_plugin
_load(name)
File "/usr/lib/python3/dist-packages/skimage/io/manage_plugins.py", line 297, in _load
fromlist=[modname])
File "/usr/lib/python3/dist-packages/skimage/io/_plugins/matplotlib_plugin.py", line 1, in
import matplotlib.pyplot as plt
File "/usr/lib/python3/dist-packages/matplotlib/pyplot.py", line 36, in
from matplotlib.figure import Figure, figaspect
File "/usr/lib/python3/dist-packages/matplotlib/figure.py", line 40, in
from matplotlib.axes import Axes, SubplotBase, subplot_class_factory
File "/usr/lib/python3/dist-packages/matplotlib/axes/init.py", line 4, in
from ._subplots import *
File "/usr/lib/python3/dist-packages/matplotlib/axes/_subplots.py", line 10, in
from matplotlib.axes._axes import Axes
File "/usr/lib/python3/dist-packages/matplotlib/axes/_axes.py", line 22, in
import matplotlib.dates as _ # <-registers a date unit converter
File "/usr/lib/python3/dist-packages/matplotlib/dates.py", line 126, in
from dateutil.rrule import (rrule, MO, TU, WE, TH, FR, SA, SU, YEARLY,
File "/home/sp/.local/lib/python3.5/site-packages/dateutil/rrule.py", line 55
raise ValueError, "Can't create weekday with n == 0"
^
SyntaxError: invalid syntax

Can you please guide me? Though I am trying my best and update if anything comes up thanks

[spurani]

Fixed the issue by reinstalling dateuitls python3 package based on caffe requirement pip3 install python-dateutil --upgrade