feature(hydra): move to formal python:3.8 image #2178

fruch · 2020-05-21T11:17:45Z

centos7 isn't formally support python3
and were getting python3.6 out of iusrepo
and
iusrepo/announce#18
seem like a good reason to move to formal python image

PR pre-checks (self review)

I followed KISS principle and best practices
I didn't leave commented-out/debugging code
I added the relevant backport labels
New configuration option are added and documented (in sdcm/sct_config.py)
I have added tests to cover my changes (Infrastructure only - under unit-test/ folder)
All new and existing unit tests passed (CI)
I have updated the Readme/doc folder accordingly (if needed)

docker/env/Dockerfile

docker/env/version

unit_tests/test_seed_selector.py

fruch · 2020-05-21T20:26:22Z

Status:
docker provision test passing locally
all provision tests in jenkins seem to be failing,
AWS:

[2020-05-21T20:01:44.358Z] < t:2020-05-21 20:01:43,847 f:log.py          l:20   c:sdcm.utils.log       p:ERROR >   File "/sct/sdcm/utils/common.py", line 1073, in get_my_ip
[2020-05-21T20:01:44.358Z] < t:2020-05-21 20:01:43,847 f:log.py          l:20   c:sdcm.utils.log       p:ERROR >     ip = socket.gethostbyname(hostname)
[2020-05-21T20:01:44.358Z] < t:2020-05-21 20:01:43,847 f:log.py          l:20   c:sdcm.utils.log       p:ERROR > socket.gaierror: [Errno -2] Name or service not known

GCE: ssh issue to connect to loader, seem like not related
docker:
import yaml fails when running configen.py, very weird since local with hydra it was running good

ghost · 2020-05-24T06:44:57Z

Congratulations 🎉. DeepCode analyzed your code in 0.49 seconds and we found no issues. Enjoy a moment of no bugs ☀️.

👉 View analysis in DeepCode’s Dashboard | Configure the bot

fruch · 2020-05-24T12:43:50Z

Status:
docker and AWS provision tests are passing
GCE is failing, doesn't sounds related, trying again to be sure.

bentsi · 2020-05-24T14:13:35Z

remove

@cached_property
sdcm/utils/fromisoformat.py

bentsi · 2020-05-31T10:49:54Z

@fruch please add documentation of how to install Python 3.8 on your local env

enaydanov

Two thumbs up to have Python 3.8 in hydra.

Overall it looks good.

I don't see any issues with list-resources.

docker/env/Dockerfile

enaydanov

Looks good.

fruch · 2020-06-11T09:36:57Z

@bentsi anything else holding this one ?

bentsi · 2020-06-11T10:17:58Z

@fruch Thursday :D
Let's merge on Sunday

fruch · 2020-06-11T10:51:36Z

@bentsi Friday is a good day for those kind of merges 👍

fruch · 2020-06-15T08:01:29Z

@bentsi we agreed we'll merge this week, right ?

bentsi · 2020-06-15T09:29:15Z

yes

bentsi · 2020-06-15T09:29:40Z

I will check it with the new Hyra and SCT runners

fruch · 2020-06-24T08:30:40Z

@bentsi a reminder, it's been two week since we agreed to merge this one...

centos7 isn't formally support python3 and were getting python3.6 out of iusrepo and iusrepo/announce#18 seem like a good reason to move to formal python image

in some case seems like this function isn't working if the hostname is wrong, or configured manually and isn't found in /etc/hosts

prepend `/usr/local/bin` to path to make sure we use the correct python3 when running with docker backend

screen isn't directly use, and when needed be be installed manually it's failing to install for docker backend on the loader "node" which is local run, now that we switch a formal python image for hydra

now that we are moving to python 3.8 we don't need this backport code anymore

this backport isn't needed now that we move to python3.8

fruch added test-provision-docker test-provision-aws Run provision test on AWS test-provision-gce Run provision test on GCE labels May 21, 2020

fruch commented May 21, 2020

View reviewed changes

docker/env/Dockerfile Outdated Show resolved Hide resolved

fruch commented May 21, 2020

View reviewed changes

docker/env/Dockerfile Outdated Show resolved Hide resolved

fruch commented May 21, 2020

View reviewed changes

docker/env/version Outdated Show resolved Hide resolved

fruch force-pushed the move_to_py3.7 branch 2 times, most recently from 8a71fe3 to 228dac5 Compare May 21, 2020 12:49

bentsi reviewed May 21, 2020

View reviewed changes

unit_tests/test_seed_selector.py Show resolved Hide resolved

fruch force-pushed the move_to_py3.7 branch from 228dac5 to 5f9ece7 Compare May 21, 2020 19:46

fruch removed the test-provision-aws Run provision test on AWS label May 24, 2020

fruch force-pushed the move_to_py3.7 branch from 6650cd0 to 8c4f9e0 Compare May 24, 2020 10:00

fruch added backport/none Backport is not required Ready for review and removed test-provision-docker test-provision-gce Run provision test on GCE labels May 24, 2020

fruch force-pushed the move_to_py3.7 branch from ea4f473 to ebcdf94 Compare May 24, 2020 13:08

fruch force-pushed the move_to_py3.7 branch from ebcdf94 to 8dbaab8 Compare May 24, 2020 14:35

fruch marked this pull request as ready for review May 24, 2020 14:35

fruch assigned bentsi May 24, 2020

fruch added the test-provision-gce Run provision test on GCE label May 24, 2020

fruch marked this pull request as ready for review May 24, 2020 17:11

fruch force-pushed the move_to_py3.7 branch from 8dbaab8 to 794da2a Compare May 25, 2020 07:00

fruch added test-provision-gce Run provision test on GCE and removed test-provision-gce Run provision test on GCE labels May 25, 2020

bentsi requested a review from enaydanov May 25, 2020 20:03

fruch assigned bentsi and unassigned fruch May 27, 2020

enaydanov requested changes May 31, 2020

View reviewed changes

docker/env/Dockerfile Outdated Show resolved Hide resolved

docker/env/Dockerfile Show resolved Hide resolved

docker/env/Dockerfile Show resolved Hide resolved

docker/env/Dockerfile Outdated Show resolved Hide resolved

fruch force-pushed the move_to_py3.7 branch from fde4c33 to 68a9bba Compare May 31, 2020 13:53

fruch requested a review from enaydanov May 31, 2020 13:54

fruch force-pushed the move_to_py3.7 branch from 68a9bba to af751aa Compare June 2, 2020 12:27

fruch removed the test-provision-gce Run provision test on GCE label Jun 2, 2020

enaydanov approved these changes Jun 7, 2020

View reviewed changes

fruch force-pushed the move_to_py3.7 branch from af751aa to d9d52c1 Compare June 11, 2020 09:34

fruch force-pushed the move_to_py3.7 branch from d9d52c1 to 94f6390 Compare June 11, 2020 10:50

fruch force-pushed the move_to_py3.7 branch from 94f6390 to 3c7411e Compare June 24, 2020 08:28

fruch force-pushed the move_to_py3.7 branch 2 times, most recently from 0fc88a4 to cde8556 Compare July 5, 2020 11:39

fruch added 6 commits July 5, 2020 14:44

feature(hydra): move to formal python:3.8 image

108ef25

centos7 isn't formally support python3 and were getting python3.6 out of iusrepo and iusrepo/announce#18 seem like a good reason to move to formal python image

fix(get_my_ip): work when hostname isn't configured correctly

bc85ccb

in some case seems like this function isn't working if the hostname is wrong, or configured manually and isn't found in /etc/hosts

fix(genconfig.py): make sure we run correct python

a132d76

prepend `/usr/local/bin` to path to make sure we use the correct python3 when running with docker backend

fix(cluster_install): remove install of screen

6870cf8

screen isn't directly use, and when needed be be installed manually it's failing to install for docker backend on the loader "node" which is local run, now that we switch a formal python image for hydra

fix(fromisoformat): remove python 3.7 backport

a337504

now that we are moving to python 3.8 we don't need this backport code anymore

refactor(cached_property): remove python3.8 backports

edfd9ef

this backport isn't needed now that we move to python3.8

fruch force-pushed the move_to_py3.7 branch from cde8556 to edfd9ef Compare July 5, 2020 11:44

bentsi approved these changes Jul 5, 2020

View reviewed changes

bentsi merged commit 6d69f9f into scylladb:master Jul 5, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature(hydra): move to formal python:3.8 image #2178

feature(hydra): move to formal python:3.8 image #2178

fruch commented May 21, 2020

fruch commented May 21, 2020

ghost commented May 24, 2020 •

edited by ghost

Loading

fruch commented May 24, 2020

bentsi commented May 24, 2020

bentsi commented May 31, 2020

enaydanov left a comment

enaydanov left a comment

fruch commented Jun 11, 2020

bentsi commented Jun 11, 2020

fruch commented Jun 11, 2020

fruch commented Jun 15, 2020

bentsi commented Jun 15, 2020

bentsi commented Jun 15, 2020

fruch commented Jun 24, 2020

feature(hydra): move to formal python:3.8 image #2178

feature(hydra): move to formal python:3.8 image #2178

Conversation

fruch commented May 21, 2020

PR pre-checks (self review)

fruch commented May 21, 2020

ghost commented May 24, 2020 • edited by ghost Loading

👉 View analysis in DeepCode’s Dashboard | Configure the bot

fruch commented May 24, 2020

bentsi commented May 24, 2020

bentsi commented May 31, 2020

enaydanov left a comment

Choose a reason for hiding this comment

enaydanov left a comment

Choose a reason for hiding this comment

fruch commented Jun 11, 2020

bentsi commented Jun 11, 2020

fruch commented Jun 11, 2020

fruch commented Jun 15, 2020

bentsi commented Jun 15, 2020

bentsi commented Jun 15, 2020

fruch commented Jun 24, 2020

ghost commented May 24, 2020 •

edited by ghost

Loading