pip upgrade/global proxy failed via master on minion (with system proxy)

[itopsatheg]

Description of Issue/Question

I will update the pip package. But the minion isn't use the (system) proxy settings
On the minion locally is all working. Via master only on new packages the minion is using the (system) proxy settings

Setup

cat generic.pip
{% if grains.pythonversion[0] == 2 %}
pip_pkg:
  pkg.installed:
    - pkgs:
      - python-pip
      - python3-pip
    - reload_modules: true
{% endif %}
{% if grains.pythonversion[0] == 3 %}
pip_pkg:
  pkg.installed:
    - pkgs:
      - python3-pip
    - reload_modules: true
{% endif %}

pip_pip:
  pip.installed:
    - name: pip
    - upgrade: True
    - proxy: proxy.internal:8080
    - use_vt: True
    - require:
      - pkg: pip_pkg

Steps to Reproduce Issue

master# salt minion.internal' state.apply generic.pip
minion.internal:
----------
          ID: pip_pip
    Function: pip.installed
        Name: pip
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python3/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python3/dist-packages/salt/loader.py", line 1939, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/states/pip_state.py", line 765, in installed
                  **kwargs)
                File "/usr/lib/python3/dist-packages/salt/states/pip_state.py", line 274, in _check_if_installed
                  desired_version = available_versions[-1]
              TypeError: 'NoneType' object is not subscriptable
     Started: 14:34:50.316884
    Duration: 19101.443 ms
     Changes:   

Summary for minion.internal
------------
Succeeded: 1
Failed:    1
------------
Total states run:     2
Total run time:  19.168 s
ERROR: Minions returned with non-zero exit code

minion# lsof +p 27203
python3 27203 root    3u  IPv4 28260955      0t0      TCP minion.internal:49514->151.101.128.223:https (SYN_SENT)

Versions Report

master:
Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: 3.5.0
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 12 2018, 14:36:49)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-46-generic
         system: Linux
        version: Ubuntu 16.04 xenial

minion:
Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.7 (default, Oct 22 2018, 11:32:17)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5
 
System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-44-generic
         system: Linux
        version: Ubuntu 18.04 bionic

minion: Repo: deb [arch=amd64] https://repo.saltstack.com/py3/ubuntu/18.04/amd64/latest bionic main
minion# echo $http_proxy http://proxy.internal:8080
minion# echo $https_proxy http://proxy.internal:8080

local is working:

minion# salt-call state.apply generic.pip
local:
----------
          ID: pip_pkg
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 14:38:13.626207
    Duration: 66.384 ms
     Changes:   
----------
          ID: pip_pip
    Function: pip.installed
        Name: pip
      Result: True
     Comment: Python package pip was already installed
              All specified packages are already installed and up-to-date
     Started: 14:38:13.917877
    Duration: 1642.391 ms
     Changes:   

Summary for local
------------
Succeeded: 2
Failed:    0
------------
Total states run:     2
Total run time:   1.709 s

master # salt minion.internal' pip.install pip proxy=http://proxy.internal:8080 upgrade=true
minion.internal:
    ----------
    pid:
        29531
    retcode:
        0
    stderr:
    stdout:
        Requirement already up-to-date: pip in /usr/local/lib/python3.6/dist-packages (19.0.3)

minion# pip3 install -U pip
Requirement already up-to-date: pip in /usr/local/lib/python3.6/dist-packages (19.0.3)

If I'm using a package this isn't exist, the minion use the proxy

generic.pip:
pip_pip:
  pip.installed:
    - name: blabla
    - proxy: proxy.internal:8080

python3   30160            root    3u  IPv4 28271917    0t0  TCP minion.internal:53326->proxy.internal:8080 (ESTABLISHED)

[garethgreenaway]

@itopsatheg Thanks for the report. I wonder if the PIP state isn't using the proxy parameter correctly. On the local minion, where it appears to be working as expected, can you unset those http_proxy environment variables and attempt to run that state or run the execution module using a package that isn't installed?

[itopsatheg]

@garethgreenaway no change

minion$ echo $http_proxy
minion$ echo $https_proxy
minion$ systemctl restart salt-minion.service
master$ salt 'minion.internal' state.apply generic.pip
minion$ lsof +p 27203
python3 27203 root    3u  IPv4 28260955      0t0      TCP minion.internal:49514->151.101.128.223:https (SYN_SENT)
minion$ ls -l /tmp/
drwx------ 2 root   root      6 Mar 14 14:36 pip-ephem-wheel-cache-5hyd4sou
drwx------ 2 root   root      6 Mar 18 11:31 pip-ephem-wheel-cache-kzgi5cj7
drwx------ 2 root   root      6 Mar 14 14:36 pip-install-34pgwnbk
drwx------ 2 root   root      6 Mar 18 11:31 pip-install-d4uznf9f
drwx------ 2 root   root      6 Mar 14 14:36 pip-req-tracker-4g37rxnt
drwx------ 2 root   root      6 Mar 18 11:31 pip-req-tracker-hx40yn8l
...

[itopsatheg]

a package that isn't installed is using the proxy parameter

[itopsatheg]

same with git

git-config:
  git.config_set:
    - name: http.proxy
    - value: http://proxy.internal:8080
    - global: true
mariadb-sys:
  git.latest:
    - name: https://github.com/good-dba/mariadb-sys.git
    - target: /srv/db/mariadb-sys

COMMAND     PID USER   FD   TYPE   DEVICE OFFSET NODE NAME
git-remot 21124 root    3u  IPv4 23035012    0t0  TCP 10.80.18.140:46648->192.30.253.113:443 (SYN_SENT)

[Poil]

Hi,
I confirm proxy on pip is no more working with 2019.2.0 (was working before), here the proxy is not taken into account :

[root@tlsslt-p01-mst ~]# salt 'xxx-sonarqube' state.show_sls  docker saltenv=base
    docker-py:
        ----------
        __env__:
            base
        __sls__:
            docker
        pip:
            |_
              ----------
              name:
                  docker
            |_
              ----------
              reload_modules:
                  True
            |_
              ----------
              proxy:
                  22.0.6.8:3128
            |_
              ----------
              require:
                  |_
                    ----------
                    pkg:
                        docker-package-dependencies
            - installed
            |_
              ----------
              order:
                  10008

The module works, not the state, this is working :

salt 'xxx-sonarqube' pip.install docker-py proxy=22.0.6.8:3128

Best regards

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.