Update pylint in the test worker image not compatible with f style strings

[jlewi]

I get different behavior when running pylint in the test worker image then when running it locally.

Example PR
https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubeflow_testing/559/kubeflow-testing-presubmit/1214777447178833920/

When run in the worker image I get

util.py                     72 INFO     ************* Module kubeflow.testing.cd.update_kf_apps
util.py                     72 INFO     E: 45, 0: invalid syntax (<string>, line 45) (syntax-error)
util.py                     46 INFO     Running: pylint --rcfile=/mnt/test-data-volume/kubeflow-testing-presubmit-py-unittests-559-d51aded-3920-83d3/src/kubeflow/testing/.pylintrc /mnt/test-data-volume/kubeflow-testing-presubmit-py-unittests-559-d51ade
d-3920-83d3/src/kubeflow/testing/py/kubeflow/testing/cd/update_kf_apps_test.py

I do not get that error when running locally.

Locally I'm using version

pylint3 --version
pylint3 2.2.2
astroid 2.1.0
Python 3.7.5rc1 (default, Oct  2 2019, 04:19:31) 
[GCC 8.3.0]

in gcr.io/kubeflow-ci/test-worker@sha256:fbfb9141c02e9a50e3679f84637fc2d7c142fe9eaa6b9f55a3c9f8a710000837

pylint --version
No config file found, using default configuration
pylint 1.8.2, 
astroid 1.6.1
Python 3.5.2 (default, Oct  8 2019, 13:06:37) 
[GCC 5.4.0 20160609]

I tried installing pylint3 in the container and got the same error

pylint3 --version
Using config file /git_kubeflow-testing/.pylintrc
pylint3 1.8.2, 
astroid 1.6.1
Python 3.5.2 (default, Oct  8 2019, 13:06:37) 
[GCC 5.4.0 20160609]

It looks like the syntax error is caused by f-style strings.

e.g.

  raise ValueError(f"Repository {r['name']} is missing param url") # pylint: disable=syntax-error

I suspect to fix this we may need to upgrade the python version to 3.7

[jlewi]

I think we can do something like the following to install the later version of python3

RUN apt-get update -y && \
    apt-get install -y python3.8 python3-pip && \
    ln -sf /usr/bin/python3.8 /usr/bin/python

RUN python3.8 -m pip install fire watchdog

I think we want to use python3.8 -m pip to install any pip packages as that ensures they are installed in the dist packages location corresponding to python3.8

[jlewi]

Example Dockerfile
https://github.com/kubeflow/testing/blob/master/apps-cd/Dockerfile

I tried updating our existing worker images to ubuntu:18.04 and some of the existing steps failed.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in one week if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been closed due to inactivity.