Possibility to add extra python packages

[pellegoeg]

Is it not possible to add extra python packages to the QGIS installation?

If I eg. wants to install geopandas in my normal python installation i would
$ pip3 install geopandas,

In QGIS3.10 I have tried this:
$ /Applications/QGIS3.10.app/Contents/Frameworks/Python.framework/Versions/Current/bin/python3 -m pip install geopandas

my terminal gives me this:

Collecting geopandas
  Using cached https://files.pythonhosted.org/packages/83/c5/3cf9cdc39a6f2552922f79915f36b45a95b71fd343cfc51170a5b6ddb6e8/geopandas-0.7.0-py2.py3-none-any.whl
Collecting pyproj>=2.2.0 (from geopandas)
  Using cached https://files.pythonhosted.org/packages/a4/a3/df9cf74ce8cc1b44e83260d74f285fb8dd3b06f4063d6cdc6c404e0deb8b/pyproj-2.5.0-cp37-cp37m-macosx_10_9_x86_64.whl
Collecting fiona (from geopandas)
  Using cached https://files.pythonhosted.org/packages/b2/9a/3cb5728fca91a642fdb1107dcb30c97ecffe25c841a7a4cfece680fd802a/Fiona-1.8.13-cp37-cp37m-macosx_10_6_intel.whl
Requirement already satisfied: pandas>=0.23.0 in /Applications/QGIS3.10.app/Contents/Resources/python (from geopandas) (0.24.2)
Requirement already satisfied: shapely in /Applications/QGIS3.10.app/Contents/Resources/python (from geopandas) (1.6.4.post2)
Collecting click-plugins>=1.0 (from fiona->geopandas)
  Using cached https://files.pythonhosted.org/packages/e9/da/824b92d9942f4e472702488857914bdd50f73021efea15b4cad9aca8ecef/click_plugins-1.1.1-py2.py3-none-any.whl
Collecting cligj>=0.5 (from fiona->geopandas)
  Using cached https://files.pythonhosted.org/packages/e4/be/30a58b4b0733850280d01f8bd132591b4668ed5c7046761098d665ac2174/cligj-0.5.0-py3-none-any.whl
Collecting munch (from fiona->geopandas)
  Using cached https://files.pythonhosted.org/packages/cc/ab/85d8da5c9a45e072301beb37ad7f833cd344e04c817d97e0cc75681d248f/munch-2.5.0-py2.py3-none-any.whl
Collecting click<8,>=4.0 (from fiona->geopandas)
  Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Requirement already satisfied: attrs>=17 in /Applications/QGIS3.10.app/Contents/Resources/python (from fiona->geopandas) (19.1.0)
Requirement already satisfied: six>=1.7 in /Applications/QGIS3.10.app/Contents/Resources/python (from fiona->geopandas) (1.12.0)
Requirement already satisfied: pytz>=2011k in /Applications/QGIS3.10.app/Contents/Resources/python (from pandas>=0.23.0->geopandas) (2019.1)
Requirement already satisfied: python-dateutil>=2.5.0 in /Applications/QGIS3.10.app/Contents/Resources/python (from pandas>=0.23.0->geopandas) (2.8.0)
Requirement already satisfied: numpy>=1.12.0 in /Applications/QGIS3.10.app/Contents/Resources/python (from pandas>=0.23.0->geopandas) (1.16.3)
Installing collected packages: pyproj, click, click-plugins, cligj, munch, fiona, geopandas
Successfully installed click-7.0 click-plugins-1.1.1 cligj-0.5.0 fiona-1.8.13 geopandas-0.7.0 munch-2.5.0 pyproj-2.5.0

but afterwards

$ /Applications/QGIS3.10.app/Contents/Frameworks/Python.framework/Versions/Current/bin$ ./python3 -m geopandas
/Applications/QGIS3.10.app/Contents/Frameworks/Python.framework/Versions/3.7/bin/python3: No module named geopandas

[PeterPetrik]

A) we can add the package to the official list of packages OR
B) you can package it as wheel in your plugin
C) try to install it as described #32 (comment)

[joaoponceleao]

I am unable to install extra python packages using #32
For some reason pip installed that way gets installed in /usr/local/bin and stops working. I imagine it is conflicting with homebrew. Adding the python.framework bin folder from QGIS to the top of the path (before and after installing pip), does not solve the problem.

Would adding pip as a dependency to the official qgis mac packager not solve the problem? We could then call that qgis version of pip directly.

Otherwise, I remember that the, now unavailable, boundlessgeo qgis packager (which was quite similar to this), came with a bash shortcut which would essentially launch a terminal with all the correct paths to install packages via pip or even homebrew for use by qgis. That could also be a way.

[scku208]

A) we can add the package to the official list of packages OR
B) you can package it as wheel in your plugin
C) try to install it as described #32 (comment)

Hi, @PeterPetrik. I also have the requirement, may I ask some info/tutorial about plan B): package it as wheel in your plugin. thanks.

[palmerj]

For some reason pip installed that way gets installed in /usr/local/bin and stops working.

This also happens for me.

[PeterPetrik]

There is not pip3 as part of the bundle in location QGIS.app/Contents/MacOS/bin/pip3 . One can simply run pip3 command from this location and install whatever likes which will end up in QGIS.app/Contents/Resources/python/ directory (bundle's site-packages)

available so far only in nightly2 builds

[AndrewAnnex]

@PeterPetrik I am running into this issue at the moment. There are packages like the IPython plugin and my own custom plugins that require pip to work correctly. Is there a work around or is this only supported in the nightly builds?

[PeterPetrik]

it will be available in nightly2 and from PR 3.16 on

[joaoponceleao]

Thanks so much for the pip. Just an update on this though.
Python packages that include C extension modules (such as triangle) can't be installed through pip in QGIS. They complain about missing header file "Python.h". On Linux, this normally requires an install of "python-dev". On Mac (and Windows?), this is normally included in the python package from most sources. Would it be possible to include this in QGIS? Or even better, is there an alternative way to install such packages or include them in plugins?

Update: At the moment I'm adding a python install that has that header file to C_INCLUDE_PATH in the system envs. That seems to work for me. But this would make distributing plugins that rely on such packages a bit complicated.

[PeterPetrik]

Thanks so much for the pip. Just an update on this though.
Python packages that include C extension modules (such as triangle) can't be installed through pip in QGIS. They complain about missing header file "Python.h". On Linux, this normally requires an install of "python-dev". On Mac (and Windows?), this is normally included in the python package from most sources. Would it be possible to include this in QGIS? Or even better, is there an alternative way to install such packages or include them in plugins?

Update: At the moment I'm adding a python install that has that header file to C_INCLUDE_PATH in the system envs. That seems to work for me. But this would make distributing plugins that rely on such packages a bit complicated.

Hi, best to create extra ticket to be able to do this to discuss. with C extensions it would be more diffucult, but I can think about it

[joaoponceleao]

You are absolutely right of course. Just created the ticket.