-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python3.9 Support #174
Merged
Merged
Python3.9 Support #174
Changes from 14 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
6e5cda7
add py39 into matrix
yunhaoling ed35fad
test latest stable cython
yunhaoling 1bf04be
test 3.9 support
yunhaoling e0470f0
update path
yunhaoling 654c094
update macos version to match
yunhaoling a519b97
revert back macos version
yunhaoling abf0013
test tweak macos python version
yunhaoling 0c5530f
update to use latest stable cython
yunhaoling 437b6c5
update python39 version as it has been officially release
yunhaoling da03280
Merge remote-tracking branch 'central/master' into python39support
yunhaoling 8c76d49
bump version and update changelog
yunhaoling 7aa91eb
update python 3.9 version
yunhaoling 36896df
update travis to use py3.9-dev as py3.9 is not available on travis yet
yunhaoling 45d3aa7
update changelog
yunhaoling 66e0d3a
rename files to match the ones in the mono repo
yunhaoling File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,158 @@ | ||
import platform | ||
import json | ||
import argparse | ||
import urllib | ||
import urllib.request | ||
from subprocess import check_call, CalledProcessError | ||
import sys | ||
import os | ||
import zipfile | ||
import tarfile | ||
import time | ||
|
||
from packaging.version import Version | ||
from packaging.version import parse | ||
from packaging.version import InvalidVersion | ||
|
||
import pdb | ||
|
||
# SOURCE OF THIS FILE: https://github.com/actions/python-versions | ||
# this is the official mapping file for gh-actions to retrieve python installers | ||
MANIFEST_LOCATION = "https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json" | ||
|
||
MAX_INSTALLER_RETRY = 3 | ||
CURRENT_UBUNTU_VERSION = "18.04" # full title is ubuntu-18.04 | ||
|
||
UNIX_INSTALL_ARRAY = ["sh", "setup.sh"] | ||
WIN_INSTALL_ARRAY = ["pwsh", "setup.ps1"] | ||
|
||
|
||
def download_installer(remote_path, local_path): | ||
retries = 0 | ||
|
||
while True: | ||
try: | ||
urllib.request.urlretrieve(remote_path, local_path) | ||
break | ||
except Exception as e: | ||
print(e) | ||
retries += 1 | ||
|
||
if retries >= MAX_INSTALLER_RETRY: | ||
print( | ||
"Unable to recover after attempting to download {} {} times".format( | ||
remote_path, retries | ||
) | ||
) | ||
exit(1) | ||
time.sleep(10) | ||
|
||
|
||
def install_selected_python_version(installer_url, installer_folder): | ||
current_plat = platform.system().lower() | ||
|
||
installer_folder = os.path.normpath(os.path.abspath(installer_folder)) | ||
if not os.path.exists(installer_folder): | ||
os.mkdir(installer_folder) | ||
local_installer_ref = os.path.join( | ||
installer_folder, | ||
"local" + (".zip" if installer_folder.endswith("zip") else ".tar.gz"), | ||
) | ||
|
||
download_installer(installer_url, local_installer_ref) | ||
|
||
if current_plat == "windows": | ||
with zipfile.ZipFile(local_installer_ref, "r") as zip_file: | ||
zip_file.extractall(installer_folder) | ||
try: | ||
check_call(WIN_INSTALL_ARRAY, cwd=installer_folder) | ||
except CalledProcessError as err: | ||
print(err) | ||
exit(1) | ||
|
||
else: | ||
with tarfile.open(local_installer_ref) as tar_file: | ||
tar_file.extractall(installer_folder) | ||
try: | ||
check_call(UNIX_INSTALL_ARRAY, cwd=installer_folder) | ||
except CalledProcessError as err: | ||
print(err) | ||
exit(1) | ||
|
||
|
||
def get_installer_url(requested_version, version_manifest): | ||
current_plat = platform.system().lower() | ||
|
||
print("Current Platform Is {}".format(platform.platform())) | ||
|
||
if version_manifest[requested_version]: | ||
found_installers = version_manifest[requested_version]["files"] | ||
|
||
# filter anything that's not x64. we don't care. | ||
x64_installers = [ | ||
file_def for file_def in found_installers if file_def["arch"] == "x64" | ||
] | ||
|
||
if current_plat == "windows": | ||
return [ | ||
installer | ||
for installer in x64_installers | ||
if installer["platform"] == "win32" | ||
][0] | ||
elif current_plat == "darwin": | ||
return [ | ||
installer | ||
for installer in x64_installers | ||
if installer["platform"] == current_plat | ||
][0] | ||
else: | ||
return [ | ||
installer | ||
for installer in x64_installers | ||
if installer["platform"] == "linux" | ||
and installer["platform_version"] == CURRENT_UBUNTU_VERSION | ||
][0] | ||
|
||
|
||
if __name__ == "__main__": | ||
parser = argparse.ArgumentParser( | ||
description="Takes the incoming request and attempts to resolve the version spec." | ||
) | ||
|
||
parser.add_argument( | ||
"version_spec", | ||
nargs="?", | ||
help=("The version specifier passed in to the UsePythonVersion extended task."), | ||
) | ||
|
||
parser.add_argument( | ||
"--installer_folder", | ||
dest="installer_folder", | ||
help=( | ||
"The folder where the found installer will be extracted into and run from." | ||
), | ||
) | ||
|
||
args = parser.parse_args() | ||
max_precached_version = Version("3.8.6") | ||
try: | ||
version_from_spec = Version(args.version_spec) | ||
except InvalidVersion: | ||
print("Invalid Version Spec. Skipping custom install.") | ||
exit(0) | ||
|
||
with urllib.request.urlopen(MANIFEST_LOCATION) as url: | ||
version_manifest = json.load(url) | ||
|
||
version_dict = {i["version"]: i for i in version_manifest} | ||
|
||
if version_from_spec > max_precached_version: | ||
print( | ||
"Requested version {} is newer than versions pre-cached on agent. Invoking.".format( | ||
args.version_spec | ||
) | ||
) | ||
install_file_details = get_installer_url(args.version_spec, version_dict) | ||
install_selected_python_version( | ||
install_file_details["download_url"], args.installer_folder | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
parameters: | ||
versionSpec: '' | ||
|
||
steps: | ||
# use python 3.8 for tooling. packaging. platform. | ||
- task: UsePythonVersion@0 | ||
displayName: "Use Python 3.8" | ||
inputs: | ||
versionSpec: 3.8 | ||
|
||
- pwsh: | | ||
pip install packaging | ||
displayName: Prep Environment | ||
# select the appropriate version from manifest if present | ||
- task: PythonScript@0 | ||
displayName: 'Check Version Spec Against Known Versions' | ||
inputs: | ||
scriptPath: '.azure-pipelines/resolve_installer_location.py' | ||
arguments: '${{ parameters.versionSpec }} --installer_folder="../_pyinstaller' | ||
|
||
# use | ||
- task: UsePythonVersion@0 | ||
displayName: "Use Python $(PythonVersion)" | ||
inputs: | ||
versionSpec: ${{ parameters.versionSpec }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to handle any permissions issues on the installation directory? Or just let them raise?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know the answer, the scripts are provided by Scott. (to be more precise, scripts are here in our mono repo: Azure/azure-sdk-for-python#14160)
but I think he should have already taken it into account :P cc @scbedd