Fix ZSH implementation - missing completion (kislyuk#433)

* Fix ZSH implementation - missing completion

Initially, `COMP_WORDS` was employed for matching modules and interpreter
arguments. However, `COMP_WORDS` isn't defined in ZSH; `words` is the correct
variable to use instead. To resolve this, an argument vector with base-1
indexing was created to work with both shells.

Secondly, the __python_argcomplete_scan_head function included a defective read
command in ZSH during interpreter matching. Using -k with read in ZSH reads a
specified num of bytes, regardless of a newline presence. This issue was
addressed by introducing a secondary read command in both shells to retrieve the
line. Additionally, in ZSH, regex matches are not stored in BASH_REMATCH unless
a compatibility option is activated. This was addressed by enabling this option
locally. The difference between base-0 and base-1 indexing was mitigated by
incrementing `BASH_REMATCH` when operating in Bash and indexing correctly.

A ZSH-compatible version of __python_argcomplete_expand_tilde_by_ref was also
introduced as the prior version utilized zero indexing and Bash-specific
indirection syntax.

Moreover, the indexing and variable expansion of the argument array in the
`__python_argcomplete_run` function were correctly adjusted.

Lastly, the path to a Python script was expanded using tilde notation to
enable auto-completion when employing:

```zsh
python ~/some/script.py
```

* FIX: missing bash_completion.d -> Darwin install (kislyuk#431)

When installing the global completion script, the non-existence of the
`/etc/bash_completion.d` directory mistakenly led argcomplete to install
the completion scripts in `/usr/local/etc/bash_completion.d`. This directory
is the default location on MacOS and is not typically sourced by Linux distributions.

* Global completion for Bash 5.0 and ZSH 5.7

This commit introduces several changes to ensure compatibility with older
environments. One significant modification was made to address an issue
with `BASH_REMATCH` in older versions of Bash. In these versions, modifying
`BASH_REMATCH` would cause the script to silently exit. To prevent this, a
new local variable named `_BASH_REMATCH` has been defined to hold the shifted
match values.

Another problem identified was that the secondary capture group, which
previously identified the interpreter, did not capture the version of the
Python interpreter within the regex. For example, it didn't capture `python3`
or `python3.9`. This resulted in the incorrect interpreter being used. To
resolve this, I decided to use the previous capture string, which captures
the entire grouping after the shebang. With this change, interpreter lines
like `/bin/env pythonX` will work correctly with the new implementation.

Additionally, the `describe` command,
`_describe "$executable" completions -o nosort`, produced some incorrect
suggestions on `zsh 5.7`. To rectify this, I removed the `-o nosort` option,
which appeared to fix the issue and restore the expected behavior.

argcomplete/bash_completion.d/python-argcomplete

@@ -4,13 +4,20 @@
 # Licensed under the Apache License. See https://github.com/kislyuk/argcomplete for more info.
 
 # Copy of __expand_tilde_by_ref from bash-completion
+# ZSH implementation added
 __python_argcomplete_expand_tilde_by_ref () {
-    if [ "${!1:0:1}" = "~" ]; then
-        if [ "${!1}" != "${!1//\/}" ]; then
-            eval $1="${!1/%\/*}"/'${!1#*/}';
-        else
-            eval $1="${!1}";
-        fi;
+    if [ -n "${ZSH_VERSION-}" ]; then
+        if [ "${(P)1[1]}" = "~" ]; then
+            eval $1="${(P)1/#\~/$HOME}";
+        fi
+    else
+        if [ "${!1:0:1}" = "~" ]; then
+            if [ "${!1}" != "${!1//\/}" ]; then
+                eval $1="${!1/%\/*}"/'${!1#*/}';
+            else
+                eval $1="${!1}";
+            fi;
+        fi
     fi
 }
 
@@ -38,17 +45,58 @@ __python_argcomplete_run_inner() {
     fi
 }
 
-# Scan the beginning of an executable file ($1) for a regexp ($2). By default,
-# scan for the magic string indicating that the executable supports the
-# argcomplete completion protocol. By default, scan the first kilobyte;
-# if $3 is set to -n, scan until the first line break up to a kilobyte.
+__python_argcomplete_upshift_bash_rematch() {
+    if [[ -z "${ZSH_VERSION-}" ]]; then
+        _BASH_REMATCH=( "" "${BASH_REMATCH[@]}" )
+    else
+        _BASH_REMATCH=( "${BASH_REMATCH[@]}" )
+    fi
+}
+
+# This function scans the beginning of an executable file provided as the first
+# argument ($1) for certain indicators, specified by the second argument ($2),
+# or the "target".  There are three possible targets: "interpreter",
+# "magic_string", and "easy_install". If the target is "interpreter", the
+# function matches the interpreter line, alongside any optional interpreter
+# arguments. If the target is "magic_string", a match is attempted for the
+# "PYTHON_ARGCOMPLETE_OK" magic string, indicating that the file should be
+# searched.  If the target is "easy_install", the function matches either
+# "PBR Generated" or any of the "EASY-INSTALL" scripts (either SCRIPT,
+# ENTRY-SCRIPT, or DEV-SCRIPT). In all cases, only the first kilobyte of
+# the file is searched. The regex matches are returned in BASH_REMATCH,
+# indexed starting at 1, regardless of the shell in use.
 __python_argcomplete_scan_head() {
+    local file="$1"
+    local target="$2"
+
     if [[ -n "${ZSH_VERSION-}" ]]; then
-        read -s -r -k 1024 -u 0 < "$1"
+        read -r -k 1024 -u 0 < "$file";
     else
-        read -s -r ${3:--N} 1024 < "$1"
+        read -r -N 1024 < "$file"
     fi
-    [[ "$REPLY" =~ ${2:-PYTHON_ARGCOMPLETE_OK} ]]
+
+    # Since ZSH does not support a -n option, we
+    # trim all characters after the first line in both shells
+    if [[ "$target" = "interpreter" ]]; then
+        read -r <<< "$REPLY"
+    fi
+
+    local regex
+
+    case "$target" in
+            magic_string) regex='PYTHON_ARGCOMPLETE_OK' ;;
+            easy_install) regex="(PBR Generated)|(EASY-INSTALL-(SCRIPT|ENTRY-SCRIPT|DEV-SCRIPT))" ;;
+            interpreter) regex='^#!(.*)$' ;;
+    esac
+
+    local ret=""
+    if [[ "$REPLY" =~ $regex ]]; then
+        ret=1
+    fi
+
+    __python_argcomplete_upshift_bash_rematch
+
+    [[ -n $ret ]]
 }
 
 __python_argcomplete_scan_head_noerr() {
@@ -64,44 +112,79 @@ __python_argcomplete_which() {
 }
 
 _python_argcomplete_global() {
+
+    if [[ -n "${ZSH_VERSION-}" ]]; then
+        # Store result of a regex match in the
+        # BASH_REMATCH variable rather than MATCH
+        setopt local_options BASH_REMATCH
+    fi
+
+    # 1-based version of BASH_REMATCH. Modifying BASH_REMATCH
+    # directly causes older versions of Bash to exit
+    local _BASH_REMATCH="";
+
+    local executable=""
+
+    # req_argv contains the arguments to the completion
+    # indexed from 1 (regardless of the shell.) In Bash,
+    # the zeroth index is empty
+    local req_argv=()
+
     if [[ -z "${ZSH_VERSION-}" ]]; then
-        local executable=$1
+        executable=$1
+        req_argv=( "" "${COMP_WORDS[@]:1}" )
         __python_argcomplete_expand_tilde_by_ref executable
     else
         # TODO: check if we should call _default or use a different condition here
         if [[ "$service" != "-default-" ]]; then
             return
         fi
-        local executable=${words[1]}
+        executable="${words[1]}"
+        req_argv=( "${words[@]:1}" )
     fi
 
     local ARGCOMPLETE=0
     if [[ "$executable" == python* ]] || [[ "$executable" == pypy* ]]; then
-        if [[ "${COMP_WORDS[1]}" == -m ]]; then
-            if __python_argcomplete_run "$executable" -m argcomplete._check_module "${COMP_WORDS[2]}"; then
+        if [[ "${req_argv[1]}" == -m ]]; then
+            if __python_argcomplete_run "$executable" -m argcomplete._check_module "${req_argv[2]}"; then
                 ARGCOMPLETE=3
             else
                 return
             fi
-        elif [[ -f "${COMP_WORDS[1]}" ]] && __python_argcomplete_scan_head_noerr "${COMP_WORDS[1]}"; then
-            local ARGCOMPLETE=2
-        else
-            return
+        fi
+        if [[ $ARGCOMPLETE == 0 ]]; then
+            local potential_path="${req_argv[1]}"
+            __python_argcomplete_expand_tilde_by_ref potential_path
+            if [[ -f "$potential_path" ]] && __python_argcomplete_scan_head_noerr "$potential_path" magic_string; then
+                req_argv[1]="$potential_path" # not expanded in __python_argcomplete_run
+                ARGCOMPLETE=2
+            else
+                return
+            fi
         fi
     elif __python_argcomplete_which "$executable" >/dev/null 2>&1; then
         local SCRIPT_NAME=$(__python_argcomplete_which "$executable")
+        __python_argcomplete_scan_head_noerr "$SCRIPT_NAME" interpreter
         if (__python_argcomplete_which pyenv && [[ "$SCRIPT_NAME" = $(pyenv root)/shims/* ]]) >/dev/null 2>&1; then
             local SCRIPT_NAME=$(pyenv which "$executable")
         fi
-        if __python_argcomplete_scan_head_noerr "$SCRIPT_NAME"; then
-            local ARGCOMPLETE=1
-        elif __python_argcomplete_scan_head_noerr "$SCRIPT_NAME" '^#!(.*)$' -n && [[ "${BASH_REMATCH[1]}" =~ ^.*(python|pypy)[0-9\.]*$ ]]; then
-            local interpreter="$BASH_REMATCH"
-            if (__python_argcomplete_scan_head_noerr "$SCRIPT_NAME" "(PBR Generated)|(EASY-INSTALL-(SCRIPT|ENTRY-SCRIPT|DEV-SCRIPT))" \
-                && "$interpreter" "$(__python_argcomplete_which python-argcomplete-check-easy-install-script)" "$SCRIPT_NAME") >/dev/null 2>&1; then
-                local ARGCOMPLETE=1
-            elif __python_argcomplete_run "$interpreter" -m argcomplete._check_console_script "$SCRIPT_NAME"; then
-                local ARGCOMPLETE=1
+        if __python_argcomplete_scan_head_noerr "$SCRIPT_NAME" magic_string; then
+            ARGCOMPLETE=1
+        elif __python_argcomplete_scan_head_noerr "$SCRIPT_NAME" interpreter; then
+            __python_argcomplete_upshift_bash_rematch
+            local interpreter="${_BASH_REMATCH[2]}"
+
+            if [[ -n "${ZSH_VERSION-}" ]]; then
+                interpreter=($=interpreter)
+            else
+                interpreter=($interpreter)
+            fi
+
+            if (__python_argcomplete_scan_head_noerr "$SCRIPT_NAME" easy_install \
+                && "${interpreter[@]}" "$(__python_argcomplete_which python-argcomplete-check-easy-install-script)" "$SCRIPT_NAME") >/dev/null 2>&1; then
+                ARGCOMPLETE=1
+            elif __python_argcomplete_run "${interpreter[@]}" -m argcomplete._check_console_script "$SCRIPT_NAME"; then
+                ARGCOMPLETE=1
             fi
         fi
     fi
@@ -116,8 +199,8 @@ _python_argcomplete_global() {
                 _ARGCOMPLETE=$ARGCOMPLETE \
                 _ARGCOMPLETE_SHELL="zsh" \
                 _ARGCOMPLETE_SUPPRESS_SPACE=1 \
-                __python_argcomplete_run "$executable" "${words[@]:1:${ARGCOMPLETE}-1}"))
-            _describe "$executable" completions -o nosort
+                __python_argcomplete_run "$executable" "${(@)req_argv[1, ${ARGCOMPLETE}-1]}"))
+            _describe "$executable" completions
         else
             COMPREPLY=($(IFS="$IFS" \
                 COMP_LINE="$COMP_LINE" \
@@ -127,7 +210,7 @@ _python_argcomplete_global() {
                 _ARGCOMPLETE=$ARGCOMPLETE \
                 _ARGCOMPLETE_SHELL="bash" \
                 _ARGCOMPLETE_SUPPRESS_SPACE=1 \
-                __python_argcomplete_run "$executable" "${COMP_WORDS[@]:1:${ARGCOMPLETE}-1}"))
+                __python_argcomplete_run "$executable" "${req_argv[@]:1:${ARGCOMPLETE}-1}"))
             if [[ $? != 0 ]]; then
                 unset COMPREPLY
             elif [[ "${COMPREPLY-}" =~ [=/:]$ ]]; then