Include non-free algorithms

[sinecode]

Hello,

with the recent update, in Opencv 3.4.3, the non-free algorithms such as SIFT and SURF are no more available.

>>> import cv2
>>> cv2.__version__
'3.4.3'
>>> cv2.xfeatures2d.SIFT_create()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cv2.error: OpenCV(3.4.3) /io/opencv_contrib/modules/xfeatures2d/src/sift.cpp:1207: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'

>>> cv2.xfeatures2d.SURF_create()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cv2.error: OpenCV(3.4.3) /io/opencv_contrib/modules/xfeatures2d/src/surf.cpp:1016: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'

As the error messages say, the library should be recompiled with -D OPENCV_ENABLE_NONFREE=1.
Would it be possible to reintroduce these algorithms in the pip package?

[skvark]

Those algorithms have been included erroneously before because they were not properly protected in the upstream by the OPENCV_ENABLE_NONFREE flag.

I am not a lawyer. I'm not sure if I can redistribute those algorithms.

Edit: See: opencv/opencv_contrib#1668

[LauLauThom]

I'm not sure if I can redistribute those algorithms.

I understand your concern. The problem is that SIFT is the gold standard algorithm, it is highly used in research and other non-commercial applications.
It does not mean that it should be freely available and redistributed, but to my mind it is the responsibility of the programmer that make use of it, to make sure he is respecting the patent (which is apparently only covering the USA by the way).
Which implementation he is using and how he had access to it probably does not matter.

The SIFT implementation is anyway available from the C++ repository of opencv, and I am sure that if the patent owner wanted to ban SIFT from opencv they would have done it before.
The thing is that they would have no interest doing that because no one would be able to use SIFT because no handy implementation would be available.

Removing SIFT from this repo is first probably breaking a lot of scripts that were relying on the previous version, and prevent people to test this solution for their problem.
Of course there are others free alternatives but if SIFT is the gold standard, it's for a good reason !

What about adding a warning when a SIFT detector/descriptor object is created ? Like for deprecated stuff but here mentioning the patent.

[skvark]

I know that both SIFT and SURF are used widely. I agree that the programmer or a company is responsible for checking that the patents and licenses of the software are respected.

However, "probably does not matter" is not enough. These pre-built packages are freely available around the world so everything must be considered even if it concerns only USA. There have been other interesting discussions about this very same matter: https://answers.ros.org/question/34557/opencv-patent/

Snippet from https://github.com/opencv/opencv_contrib/blob/master/modules/xfeatures2d/src/sift.cpp:

//    Note that restrictions imposed by this patent (and possibly others)
//    exist independently of and may be in conflict with the freedoms granted
//    in this license, which refers to copyright of the program, not patents
//    for any methods that it implements.  Both copyright and patent law must
//    be obeyed to legally use and redistribute this program and it is not the
//    purpose of this license to induce you to infringe any patents or other
//    property right claims or to contest validity of any such claims.  If you
//    redistribute or use the program, then this license merely protects you
//    from committing copyright infringement.  It does not protect you from
//    committing patent infringement.  So, before you do anything with this
//    program, make sure that you have permission to do so not merely in terms
//    of copyright, but also in terms of patent law.
//    Please note that this license is not to be understood as a guarantee
//    either.  If you use the program according to this license, but in
//    conflict with patent law, it does not mean that the licensor will refund
//    you for any losses that you incur if you are sued for your patent
//    infringement.

I didn't remove anything from this repo. OpenCV developers moved SIFT and SURF behind the non-free build flag in the most recent release which is the way they should have been always but, for some reason, weren't. If I turn the non-free build flag on, it's not only SIFT and SURF that get included in the wheels. There are other non-free modules in OpenCV.

I can't add any warnings in this repository to the OpenCV code (without very cumbersome patches) since this repository just pulls a certain vanilla OpenCV version from the upstream and builds it.

If you need the non-free modules you can build the Python bindings yourself and flip the non-free flag on. After all, that's the official way. These packages are unofficial and not endorsed by OpenCV.

Best solution would be probably to create proper support for sdist for these packages and allow users to force the rebuild of the packages themselves. This requires of course compilers and other tools to be installed on the target system and the installation would take about ~10-15 minutes.

[waveform80]

but to my mind it is the responsibility of the programmer that make use of it, to make sure he is respecting the patent (which is apparently only covering the USA by the way). Which implementation he is using and how he had access to it probably does not matter.

I'm afraid it does matter. Or to be more precise, it's not in the "definitely doesn't matter" category, or even in the "probably does not matter" category. More like in the "might well matter" category! Here's some background on the interesting question of why distributing source code might be in the "probably does not matter" category, but distributing built binaries isn't:

https://patents.stackexchange.com/questions/13992/in-the-usa-is-it-legal-to-distribute-source-code-that-when-run-would-violate

(which in turn answers why the code is still in the upstream OpenCV repository, just guarded with a compiler flag)

Best solution would be probably to create proper support for sdist for these packages and allow users to force the rebuild of the packages themselves. This requires of course compilers and other tools to be installed on the target system and the installation would take about ~10-15 minutes.

More like 2 hours on a Pi :)

Still, as the person who builds the wheels for opencv-python on the piwheels platform (with the relevant bits from skvark's repo here), I'm entirely in agreement with skvark here and will be following upstream's lead. IANAL (too), and while it's interesting (to some?) to read patent-related SE posts and the odd legal paper I wouldn't trust my conclusions to defend me from a pack of rabid patent attorneys.

[ubaierbhat]

Is it possible to install opencv-python to include these non-free algorithms with pip ?

[skvark]

Old versions have SIFT and SURF included due to the reasons explained above. The latest release does not include them. Any upcoming release will not include them (unless OpenCV developers move the algorithm to the free side of OpenCV after patent expiration). It's better not to rely on these packages if you wish to use non-free algorithms.

[native-api]

@Jindil http://answers.opencv.org/question/18259/is-surf-algorithm-used-in-opencv-patented/ <- it says there are better, free alternatives. Judging by https://www.quora.com/Computer-Vision-They-dont-use-SIFT-at-Microsoft-they-use-MOPS-Meaning, they seem to only differ by some reference dataset or something.

http://answers.opencv.org/question/73877/opencv-free-for-commercial-use/ <- this says they are free for non-commercial use only, so if they are included, opencv-python cannot be distributed under the MIT license as it is now, or a commercial user or @skvark could be held liable for patent infringement (depending on jurisdiction).

@ubaierbhat you can tweak CMake command line (cmake_args) in setup.py and make a custom build.

[skvark]

I added a note about this to the README. Closing because this is not going to happen.

[ducha-aiki]

Any upcoming release will not include them

Even after the expiration? :)

https://patents.google.com/patent/US6711293
"2020-03-06 Anticipated expiration"

[skvark]

If the patent expires and the algorithm is moved to free side in OpenCV it will be included in the builds automatically. We will see next year.

[YJonmo]

I got the same problem of the nonfree error when using the cv2.kinfu function:

sudo pip2 install opencv-python==4.1.2.30
sudo pip2 install opencv-contrib-python==4.1.2.30

then in python

cv2.kinfu.KinFu_create(params)

error Traceback (most recent call last)
in
----> 1 cv2.kinfu.KinFu_create(paras)

error: OpenCV(4.1.2) /io/opencv_contrib/modules/rgbd/src/kinfu.cpp:314: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'

The errors also happens in C++ code even after I recompiled the opencv with the -D OPENCV_ENABLE_NONFREE=1

Any solution?
Cheers

[nairshobhit94]

I got the same problem when using cv2.detectAndCompute

pip install opencv-python==4.1.2.30
pip2 install opencv-contrib-python==4.1.2.30

---

AttributeError Traceback (most recent call last)
in ()
9
10 #finding out the keypoints and their descriptors
---> 11 keypoints1,descriptors1 = cv2.detectAndCompute(image1,None)
12 keypoints2,descriptors2 = cv2.detectAndCompute(image2,None)
13

AttributeError: module 'cv2.cv2' has no attribute 'detectAndCompute'

Can someone help?

[parujp]

@nairshobhit94 : SIFT is not available in open CV versions > 3.4.2 and you don’t need opencv-python to use SIFT. You only need the correct version of opencv-contrib-python. So try

pip uninstall opencv-python

and

pip install —user opencv-contrib-python==3.4.2.17

[SystemDiagnosticss]

@skvark Can you show How to changed setup.py for build with SIFT/SURF ?
Where and What exactly I need add to this file for correct buid?
It's "-DOPENCV_ENABLE_NONFREE=1" in cmake_args or what?

[skvark]

This sould do it:

export CMAKE_ARGS="-DOPENCV_ENABLE_NONFREE=ON"
python setup.py bdist_wheel

[mamur9]

@skvark I have build opencv 4.2 with nonFree enabled on Windows 64x, it is working fine.
But in order to use this built opencv I need to package it somehow. Can you please explain how to make wheel file from it ('if it is possible more detailed, I could not find any information about how opencv wheel made').
In which directory I need to run setup.py bdist_wheel

[bhagyas731]

@skvark please tell in which directory do we have to run python setup.py bdist_wheel in anaconda environment of ubuntu 19.04 without rebuilding and compiling the whole source and how this adding of non-free modules process goes on...?
your this help would be much appreciated...

[skvark]

If you want to enable non-free modules (or other modules which are not enabled in the pre-built wheels) and package this project you must compile OpenCV from sources. To do that with the toolchain provided by this repository:

git clone --recursive https://github.com/skvark/opencv-python.git
cd opencv-python
export CMAKE_ARGS="-DOPENCV_ENABLE_NONFREE=ON"
python setup.py bdist_wheel

[tuanardouin]

@skvark The patent for SIFT expired 10 days ago.
https://patents.google.com/patent/US6711293

Are you going to include it now ?

[skvark]

Please open an issue to the OpenCV repository. I cannot enable it in this repository, the C++ macro which prevents it from being included in the builds is in there.

[skvark]

See: opencv/opencv#16736

[asknask]

If you want to enable non-free modules (or other modules which are not enabled in the pre-built wheels) and package this project you must compile OpenCV from sources. To do that with the toolchain provided by this repository:

git clone --recursive https://github.com/skvark/opencv-python.git
cd opencv-python
export CMAKE_ARGS="-DOPENCV_ENABLE_NONFREE=ON"
python setup.py bdist_wheel

Completely new to this so excuse the ignorance. I cloned your repository. I downloaded OpenCV's latest version from theirs and pasted it into the opencv folder in your repo's folder. I set the environment variable using

set CMAKE_ARGS "-DOPENCV_ENABLE_NONFREE=ON"

I then ran the python command mentioned by you above. It gives the following error:

D:\opencv-python-master>python setup.py bdist_wheel Traceback (most recent call last): File "setup.py", line 397, in <module> main() File "setup.py", line 64, in main package_version = get_opencv_version() File "setup.py", line 351, in get_opencv_version runpy.run_path("find_version.py") File "C:\Program Files\Python38\lib\runpy.py", line 263, in run_path return _run_module_code(code, init_globals, run_name, File "C:\Program Files\Python38\lib\runpy.py", line 96, in _run_module_code _run_code(code, mod_globals, init_globals, File "C:\Program Files\Python38\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "find_version.py", line 26, in <module> git_hash = subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD']).splitlines()[0].decode() File "C:\Program Files\Python38\lib\subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "C:\Program Files\Python38\lib\subprocess.py", line 489, in run with Popen(*popenargs, **kwargs) as process: File "C:\Program Files\Python38\lib\subprocess.py", line 854, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Program Files\Python38\lib\subprocess.py", line 1307, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] The system cannot find the file specified

[skvark]

It seems that you do not have Git installed / Python does not find the git executable. Please open a new issue if you have further problems, this issue has been closed and it is not related to your issue.

[YouYue123]

Remember to add ENABLE_CONTRIB=1. And the latest 4.3.0 opencv-contrib includes SIFT without OPENCV_ENABLE_NONFREE flag.

[rooseveltrp]

Here's how I got it working on macOS Catalina (10.15.4).

brew install qt
cd ~/Downloads
git clone --recursive https://github.com/skvark/opencv-python.git
cd opencv-python
export CMAKE_ARGS="-DOPENCV_ENABLE_NONFREE=ON -DENABLE_CONTRIB=1 -DOPENCV_EXTRA_MODULES_PATH=/Users/<User>/Downloads/opencv-python/opencv_contrib/modules/ -DCMAKE_PREFIX_PATH=/usr/local/Cellar/qt/5.14.2/"
python3 setup.py bdist_wheel
cd dist
sudo pip3 install opencv_python-4.3.0+3073e9e-cp37-cp37m-macosx_10_9_x86_64.whl

A test script:

import cv2
import numpy as np

img = cv2.imread("123.jpg")
gray= cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
sift = cv2.xfeatures2d.SURF_create()
kp = sift.detect(gray,None)
img=cv2.drawKeypoints(gray,kp,img)
cv2.imshow("sift_keypoints",img)
cv2.waitKey(0)

On a mac, SIFT is kinda slow. But if you compile OpenCV with GPU computing support on a Linux box, it's ridiculously fast!

Here's how you compile on Ubuntu with GPU support.

cd /tmp
git clone --recursive https://github.com/skvark/opencv-python.git
cd opencv-python
export CMAKE_ARGS="-DOPENCV_ENABLE_NONFREE=ON -DWITH_TBB=ON -DWITH_CUDA=ON -DWITH_CUDNN=ON -DOPENCV_DNN_CUDA=ON -DCUDA_ARCH_BIN=7.5 -DBUILD_opencv_cudacodec=OFF -DENABLE_FAST_MATH=1 -DCUDA_FAST_MATH=1 -DWITH_CUBLAS=1 -DWITH_V4L=ON -DWITH_QT=OFF -DWITH_OPENGL=ON -DWITH_GSTREAMER=ON -DOPENCV_GENERATE_PKGCONFIG=ON -DOPENCV_ENABLE_NONFREE=ON -DOPENCV_EXTRA_MODULES_PATH=/tmp/opencv-python/opencv_contrib/modules"
python3 setup.py bdist_wheel
cd dist
pip3 install opencv_python-4.3.0+3073e9e-cp36-cp36m-linux_x86_64.whl

[lober-io]

@rooseveltrp
I'm hitting the following error. did it occur to you also and have you found a solution?

Version:  4.3.0+3073e9e
setup.py:85: DeprecationWarning: SO is deprecated, use EXT_SUFFIX
  ['python/cv2[^/]*%(ext)s' % {'ext': re.escape(sysconfig.get_config_var('SO'))}],
Traceback (most recent call last):
  File "/home/fabian/anaconda3/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 560, in setup
    cmkr = cmaker.CMaker(cmake_executable)
  File "/home/fabian/anaconda3/lib/python3.7/site-packages/skbuild/cmaker.py", line 95, in __init__
    self.cmake_version = get_cmake_version(self.cmake_executable)
  File "/home/fabian/anaconda3/lib/python3.7/site-packages/skbuild/cmaker.py", line 82, in get_cmake_version
    "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable)

Problem with the CMake installation, aborting build. CMake executable is cmake

[dokuboyejo]

@lober-io ensure you have cmake installed. I'll also recommend upgrading setuptools if not up-to date. You can get that done using
pip3 install --upgrade pip cmake setuptools

[lorenz-h]

Here's how I got It working on for Python 3.7.5 on Windows 10 Version 1903 (Warning: hacky):
You need Visual Studio 2015 and the corresponding VC++ development tools.
First clone the repo:
git clone --recursive https://github.com/skvark/opencv-python.git
cd opencv-python
While it is not the most elegant solution I manually edited setup.py:

1. line 18: build_contrib = True
2. line 110: add "-DOPENCV_ENABLE_NONFREE=ON" to cmake_args

Finally:
python setup.py bdist_wheel
After building locate the generated wheel in the opencv-python/dist/ folder and install it using pip:
pip install NAME_OF_YOUR_WHEEL.whl

[MME9]

Really helpful comments,
@dokuboyejo Your update was required
@lorenz-h I followed your steps - I think a comma is required at the end of line 110
@skvark thank you for maintaining this page and helpfully sharing a simple method

[shivasj]

I was able to get a successful build with the above solutions when i do not set the enabled contrib flag , but when i set the env variable to ENABLE_CONTRIB=1 then i get the below error. Any thoughts on what I could be missing?

Error:
Scanning dependencies of target opencv_python3 [100%] Building CXX object modules/python3/CMakeFiles/opencv_python3.dir/__/src2/cv2.cpp.o In file included from /app/opencv-python/_skbuild/linux-x86_64-3.7/cmake-build/modules/python_bindings_generator/pyopencv_custom_headers.h:7, from /app/opencv-python/opencv/modules/python/src2/cv2.cpp:1894: /app/opencv-python/opencv_contrib/modules/phase_unwrapping/misc/python/pyopencv_phase_unwrapping.hpp:2:13: error: ‘phase_unwrapping’ in namespace ‘cv’ does not name a type typedef cv::phase_unwrapping::HistogramPhaseUnwrapping::Params HistogramPhaseUnwrapping_Params; ^~~~~~~~~~~~~~~~ In file included from /app/opencv-python/_skbuild/linux-x86_64-3.7/cmake-build/modules/python_bindings_generator/pyopencv_custom_headers.h:8, from /app/opencv-python/opencv/modules/python/src2/cv2.cpp:1894: /app/opencv-python/opencv_contrib/modules/surface_matching/misc/python/pyopencv_ppf_match_3d.hpp:3:40: error: ‘ppf_match_3d’ was not declared in this scope template<> struct pyopencvVecConverter<ppf_match_3d::Pose3DPtr >

[konradmb]

@shivasj Try rm -r _skbuild. It compiled successfully for me after removing previous build data.

[shivasj]

Thank you @konradmb removing the previous build folder fixed the issue

[sczerniawski]

Turns out pip can build the package with the NONFREE flag set.

CMAKE_ARGS="-DOPENCV_ENABLE_NONFREE=ON" pip install --no-binary=opencv-contrib-python opencv-contrib-python

There's no need to compile anything separately.

[buildgreatthings]

Since SURF is now off patent, can we include that back into the pypi builds of OpenCV?

[native-api]

@awcchungster It's up to the upstream to move it into the free section of the build logic -- then any of OpenCV-Python builds will include it automatically.

[skvark]

As @native-api wrote and as it has been explained in this issue also previously, all non-free modules are protected by a C++ macro in the upstream repositories where the actual OpenCV C++ code is located. This repository will never turn on the OPENCV_ENABLE_NONFREE flag by default because that will cause all patented / non-free modules to be included in the builds.

For example, for SURF the macro is here:

https://github.com/opencv/opencv_contrib/blob/a26f71313009c93d105151094436eecd4a0990ed/modules/xfeatures2d/src/surf.cpp#L118

The correct place for these requests is the OpenCV contrib repository.

[MJAHMADEE]

We have identified an issue some of you are encountering when using the Surf algorithm, which results in the following error message:

error: OpenCV(4.7.0) /io/opencv_contrib/modules/xfeatures2d/src/surf.cpp:1026: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'

This error occurs because the Surf algorithm is patented and requires specific configuration to be enabled.

To resolve this issue and make it possible to use the Surf algorithm in the Google Colab environment, please refer to the following GitHub repository and the accompanying Google Colab notebook:

https://github.com/MJAHMADEE/SURF

The repository provides a comprehensive solution to this problem, ensuring that the Surf algorithm can be used without encountering the aforementioned error. The Google Colab notebook within the repository contains detailed instructions and the necessary setup to get you started.

We hope this helps you overcome the issue and continue your work seamlessly.