[BUG] apt pkg.install with HTTP URL as source fails, if URL contains a port

[hurzhurz]

Description

When trying to install a .deb package that is located on an internal webserver listening on a special port, it fails without an visible error.

The issue seems to be that aptpkg uses cp.cache_file to download the deb file and the resulting cache file path contains the http host and port with a colon in between.
I think somewhere in aptpkg the colon in the path is an issue.

Setup

Ubuntu 20.04 focal
Salt 3006.1

• VM (Virtualbox, KVM, etc. please specify)
• container (Kubernetes, Docker, containerd, etc. please specify)
• onedir packaging

Steps to Reproduce the behavior

It can be tested by using any package from a public repositroy by just adding the default port in the URL, e.g. for nano:

• working: http://archive.ubuntu.com/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb
• not working: http://archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb

# apt-get remove nano
# salt-call pkg.install sources='[{"nano":"http://archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb"}]'

debug output

root@salttest:~# salt-call pkg.install -l debug sources='[{"nano":["http://archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb"](http://archive.ubuntu.com/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb)}]'
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: salttest
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[DEBUG   ] Configuration file path: /etc/salt/minion
[DEBUG   ] Grains refresh requested. Refreshing grains.
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] The functions from module 'core' are being loaded by dir() on the loaded module
[DEBUG   ] The functions from module 'disks' are being loaded by dir() on the loaded module
[DEBUG   ] The functions from module 'extra' are being loaded by dir() on the loaded module
[DEBUG   ] The functions from module 'lvm' are being loaded by dir() on the loaded module
[DEBUG   ] The functions from module 'mdadm' are being loaded by dir() on the loaded module
[DEBUG   ] The functions from module 'minion_process' are being loaded by dir() on the loaded module
[DEBUG   ] The functions from module 'opts' are being loaded by dir() on the loaded module
[DEBUG   ] Override  __utils__: <module 'salt.loaded.int.grains.zfs' from '/opt/saltstack/salt/lib/python3.10/site-packages/salt/grains/zfs.py'>
[DEBUG   ] The functions from module 'zfs' are being loaded by dir() on the loaded module
[DEBUG   ] The `lspci` binary is not available on the system. GPU grains will not be available.
[DEBUG   ] The functions from module 'zfs' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded zfs.is_supported
[DEBUG   ] Connecting to master. Attempt 1 of 1
[DEBUG   ] "salt" Not an IP address? Assuming it is a hostname.
[DEBUG   ] Master URI: tcp://127.0.1.1:4506
[DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'salttest', 'tcp://127.0.1.1:4506')
[DEBUG   ] Generated random reconnect delay between '1000ms' and '11000ms' (7325)
[DEBUG   ] Setting zmq_reconnect_ivl to '7325ms'
[DEBUG   ] Setting zmq_reconnect_ivl_max to '11000ms'
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] salt.crypt._get_key_with_evict: Loading private key
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] SaltEvent PUB socket URI: /var/run/salt/minion/minion_event_1bc1a361f1_pub.ipc
[DEBUG   ] SaltEvent PULL socket URI: /var/run/salt/minion/minion_event_1bc1a361f1_pull.ipc
[DEBUG   ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG   ] Decrypting the current master AES key
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG   ] salt.crypt.verify_signature: Loading public key
[DEBUG   ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG   ] salt.crypt.verify_signature: Verifying signature
[DEBUG   ] <salt.crypt.AsyncAuth object at 0x7f77a66f8d00> Got new master aes key.
[DEBUG   ] Closing AsyncReqChannel instance
[DEBUG   ] Connecting the Minion to the Master publish port, using the URI: tcp://127.0.1.1:4505
[DEBUG   ] <salt.transport.zeromq.PublishClient object at 0x7f77a66f8e50> connecting to tcp://127.0.1.1:4505
[DEBUG   ] Sending event: tag = __master_connected; data = {'master': 'salt', '_stamp': '2023-06-21T13:36:30.073434'}
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] Closing IPCMessageClient instance
[DEBUG   ] Determining pillar cache
[DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'salttest', 'tcp://127.0.1.1:4506')
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] salt.crypt.verify_signature: Loading public key
[DEBUG   ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG   ] salt.crypt.verify_signature: Verifying signature
[DEBUG   ] Closing AsyncReqChannel instance
[DEBUG   ] The functions from module 'jinja' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] The functions from module 'yaml' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] The functions from module 'pkg' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded pkg.install
[DEBUG   ] The functions from module 'direct_call' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded direct_call.execute
[DEBUG   ] The functions from module 'pkg_resource' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded pkg_resource.parse_targets
[DEBUG   ] The functions from module 'config' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded config.valid_fileproto
[DEBUG   ] The functions from module 'cp' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded cp.cache_file
[DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'salttest', 'tcp://127.0.1.1:4506')
[DEBUG   ] Requesting URL [http://archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb](http://archive.ubuntu.com/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb) using GET method
[DEBUG   ] Using backend: tornado
[DEBUG   ] Closing AsyncReqChannel instance
[DEBUG   ] The functions from module 'cmd' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded cmd.run_stdout
[INFO    ] Executing command dpkg-query in directory '/root'
[DEBUG   ] The functions from module 'lowpkg' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded lowpkg.bin_pkg_info
[INFO    ] Executing command dpkg in directory '/root'
[INFO    ] Executing command dpkg in directory '/root'
[INFO    ] Executing command systemd-run in directory '/root'
[INFO    ] Executing command dpkg-query in directory '/root'
[DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'salttest', 'tcp://127.0.1.1:4506')
[DEBUG   ] Closing AsyncReqChannel instance
[DEBUG   ] The functions from module 'nested' are being loaded by dir() on the loaded module
[DEBUG   ] LazyLoaded nested.output
local:
    ----------

When using cp.cache_file to download the file as an extra step, the result is the same:

# salt-call cp.cache_file http://archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb
local:
    /var/cache/salt/minion/extrn_files/base/archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb
# salt-call pkg.install sources='[{"nano":"/var/cache/salt/minion/extrn_files/base/archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb"}]'

But renaming the folder works:

# salt-call cp.cache_file http://archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb
local:
    /var/cache/salt/minion/extrn_files/base/archive.ubuntu.com:80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb
# mv /var/cache/salt/minion/extrn_files/base/archive.ubuntu.com\:80 /var/cache/salt/minion/extrn_files/base/archive.ubuntu.com-80
# salt-call pkg.install sources='[{"nano":"/var/cache/salt/minion/extrn_files/base/archive.ubuntu.com-80/ubuntu/pool/main/n/nano/nano_4.8-1ubuntu1_amd64.deb"}]'

Versions Report

salt --versions-report

Salt Version:
          Salt: 3006.1

Python Version:
        Python: 3.10.11 (main, May  5 2023, 02:31:54) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
        relenv: 0.12.3
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: ubuntu 20.04.6 focal
        locale: utf-8
       machine: x86_64
       release: 5.4.0-148-generic
        system: Linux
       version: Ubuntu 20.04.6 focal

[Ch3LL]

closed by #64945