Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: no det version parameter via continued config #10151

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

ci: no det version parameter via continued config #10151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
a94ed82
ci: no det version parameter via continued config
JComins000 Oct 29, 2024
d2851e2
change path
JComins000 Oct 29, 2024
8125381
install git
JComins000 Oct 29, 2024
1c291e6
try installing git from this orb?
JComins000 Oct 29, 2024
65d14ab
install git
JComins000 Oct 29, 2024
70a34cd
install git on linux builds
JComins000 Oct 29, 2024
333ff1e
playing with syntax
JComins000 Oct 29, 2024
8543868
when unless syntax
JComins000 Oct 29, 2024
af98fab
python image
JComins000 Oct 29, 2024
583648e
when, unless
JComins000 Oct 29, 2024
63ffe35
windows???
JComins000 Oct 29, 2024
bb31565
windows???
JComins000 Oct 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,16 +87,6 @@ jobs:
mv -v tmpfile "<<pipeline.parameters.params_basename>>"
fi

- run:
name: Set det-version parameter.
davidfluck-hpe marked this conversation as resolved.
Show resolved Hide resolved
command: |
VERSION=$(./version.sh)
echo "Version is: [${VERSION}]"
echo "CIRCLE_TAG is: [${CIRCLE_TAG}]"
jq --arg version "${VERSION}" '. += {"det-version": $version}' < "<<pipeline.parameters.params_basename>>" > tmpfile
mv -v tmpfile "<<pipeline.parameters.params_basename>>"
cat "<<pipeline.parameters.params_basename>>"

# this must be last; persist the file to the workspace
- persist_to_workspace:
root: .
Expand Down
47 changes: 33 additions & 14 deletions .circleci/real_config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,12 @@ orbs:
executors:
python-38:
docker:
- image: python:3.8-slim-bookworm
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these didnt have git installed, so they couldn't run the script

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

- image: cimg/python:3.8
python-39:
docker:
- image: python:3.9-slim-bookworm
- image: cimg/python:3.9

parameters:
det-version:
type: string
default: ""
docker-image:
type: string
default: determinedai/cimg-base:latest
Expand Down Expand Up @@ -391,18 +388,17 @@ commands:

install-wheel:
parameters:
package-name:
type: string
package-location:
type: string
steps:
- run:
name: Install <<parameters.package-name>>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this previously required pipeline.parameters.det-version, so idk why were were parameterizing on package name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dannysauer

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this could install any package the same way instead of only installing Determined. 🤷‍♂️ If it's only used for one package, then I agree that the parameter is unecessary.

(FWIW, the blame probably shows that I wrote all this stuff because I renamed the config file when switching to dynamic config. :))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ohh okay! thanks for looking

name: Install determined
working_directory: <<parameters.package-location>>
command: |
VERSION=$(~/project/version.sh)
make build
pip install --find-links dist <<parameters.package-name>>==<< pipeline.parameters.det-version >>
pip install --no-deps --force-reinstall --find-links dist <<parameters.package-name>>==<< pipeline.parameters.det-version >>
pip install --find-links dist determined==${VERSION}
pip install --no-deps --force-reinstall --find-links dist determined==${VERSION}

setup-python-venv:
description: Set up and create Python venv.
Expand Down Expand Up @@ -507,7 +503,6 @@ commands:
name: Install pypa builder
command: python3 -m pip install build
- install-wheel:
package-name: determined
package-location: ./harness
- run:
name: Install <<parameters.extras-requires>>
Expand Down Expand Up @@ -2700,16 +2695,40 @@ jobs:
executor-name:
type: string
executor: << parameters.executor-name >>
environment:
VERSION: "<< pipeline.parameters.det-version >>"
steps:
- checkout
- when:
condition:
equal: [ win/default, << parameters.executor-name >> ]
steps:
- run:
name: Set VERSION environment variable
command: |
$version = & bash ./version.sh
setx VERSION $version
- unless:
condition:
equal: [ win/default, << parameters.executor-name >> ]
steps:
- run:
name: Set VERSION environment variable
command: |
echo "export VERSION=$(cd ~/project && ./version.sh)" >> $BASH_ENV
- python-report
# Running the pip executable causes an error with the win/default executor for some reason.
- run: python -m pip install --upgrade --user pip
- run: pip install wheel setuptools build
- run: cd harness; python -m build --wheel --outdir ../build
- run: pip install --find-links build determined==<< pipeline.parameters.det-version >>
- when:
condition:
equal: [ win/default, << parameters.executor-name >> ]
steps:
- run: pip install --find-links build determined==$env:VERSION
- unless:
condition:
equal: [ win/default, << parameters.executor-name >> ]
steps:
- run: pip install --find-links build determined==${VERSION}
- run: pip freeze --all
# Allow this to fail, but it is useful for debugging.
- run: sh -c "pip check || true"
Expand Down
Loading