Integration of ROCK images into Seldon Core Operator #133

i-chvets · 2023-05-17T12:13:57Z

Description

Integration of ROCK images.
https://github.com/canonical/seldonio-rocks

The following ROCK images need to be integrated into Seldon Core Operator:
Charm managed:

seldonio-rocks/seldon-core-operator: Rock is done, integration tests are done, integrated into CKF

Workload managed:

seldonio-rocks/sklearnserver: Rock is done, integration tests are done, integrated into CKF
seldonio/mlserver-sklearn
seldonio/mlserver-xgboost
seldonio/mlserver-mlflow
seldonio/mlserver-huggingface
~~seldonio/mlserver-slim~~
seldonio/mlflowserver
~~seldonio-rocks/xgboostserver v1~~
seldonio-rocks/xgboostserver v2
seldonio-rocks/seldon-core-s2i-python3
~~seldonio-rocks/seldon-core-s2i-python36 (could be drop candidate)~~
~~seldonio-rocks/seldon-core-s2i-python37 (could be drop candidate)~~
seldonio-rocks/seldon-core-s2i-python38
~~seldonio-rocks/seldon-core-s2i-python36-gpu (could be drop candidate)~~
~~seldonio-rocks/seldon-core-s2i-python37-gpu (could be drop candidate)~~
seldonio-rocks/seldon-core-s2i-python38-gpu
seldonio-rocks/tensorflow/serving
seldonio-rocks/seldonio/tfserving-proxy

Design

Design of how ROCKs are built, tested and integrated is captured in related specification (KF-044).
Main design points for Seldon ROCKs:

metadata.yaml is updated with references to new ROCK image.
Seldon Core Operator ConfigMap updated with new ROCK images.

Testing

Existing integration tests are to be re-used to test functionality of new ROCK images.
Follow Seldon documenation for testing:
https://docs.seldon.io/projects/seldon-core/en/latest/reference/apis/index.html
https://docs.seldon.io/projects/seldon-core/en/latest/nav/config/servers.html

Integration tests

Implementation

#133 Summary of changes: - Added integration test for Seldon prepackages servers: xgboost v1 and v2, mlflowserver v1. - Remove tests fails intermittently.

#133 Summary of changes: - Bump resources for mlflowserver tests.

#133 Summary of changes: - Use lightkube for deletion and testing. - Verify remove test when juju/python-libjuju#877 is resolved.

#133 Summary of changes: - Updated integration tests with server names to retrieve proper image name for testing. - Added test for removal of workload deployed configmap.

canonical/seldon-core-operator#133 Summary of changes: - Updated rockcraft.yaml with new run-user option to run as non-root. - Updated import procedure. - Tested with integration tests on the branch.

canonical/seldon-core-operator#133 Summary of changes: - Updated rockcraft.yaml with new run-user option to run as non-root. - Updated import procedure.

@kimwnasptd

* feat: sklearnserver rock integration Summary of changes: - Added ROCK integrity tests. - Updated ROCK image definition. - Added tox with unit and integration tests. - Addressed review comments. - Modified to run tests. - Added handling of jinja2 templating - Updated tox.ini to properly update configmap template. - Updated rockcraft.yaml with new run-user option to run as non-root. canonical/seldon-core-operator#133 - Updated import procedure. - Tested with integration tests on the branch. - Reverted to use of shell commands and `yq` instead of CheckRock test class from chisme package. - Remove chisme package NOTE: Use of bash shell commands significantly reduces maintability of tox.ini ACK: @kimwnasptd

* feat: integration tests for seldon rocks #133 Summary of changes: - Added integration test for Seldon prepackages servers: sklearn v1 and v2, xgboost v1 and v2, mlflowserver v1. - Bump resources for mlflowserver tests. - Updated integration tests with server names to retrieve proper image name for testing. - Added test for removal of workload deployed configmap. - Updated workflow to remove pin of Juju agent version and use of 2.9/stable channel. - Reverted back to using remove_application() from model instead of subprocess.

canonical/seldon-core-operator#133 Summary of changes: - Created rockcraft file for mlsever-sklearn rock. - Added unit test. - Added tox.ini

canonical/seldon-core-operator#133 Summary of changes: - mlserver-xgboost rockcraft - Unit tests. - tox.ini ready for integration tests.

canonical/seldon-core-operator#133 Summary of changes: - Created rockcraft file for mlserver-mlflow rock according to best practices spec. - Added unit test. - Added tox.ini

canonical/seldon-core-operator#133 Summary of changes: - Created rockcraft file for mlserver-xgboost rock according to best practices spec. - Added unit test. - Added tox.ini

* feat: mlserver-sklearn rock canonical/seldon-core-operator#133 Summary of changes: - Created rockcraft file for mlsever-sklearn rock. - Added unit test. - Added tox.ini - Use variable ROCK_RUNTIME to simplify development. - Modified rockcraft.yaml to use environment variables in services instead of calling export in command line.

#133 Summary of changes: - Added integration tests files. - Updated tests. - Updated config map to point to published rock.

canonical/seldon-core-operator#133 Summary of changes: - Updated rockcraft.yaml to generate ROCK that can be started in Seldon Deployment.

canonical/seldon-core-operator#133 Summary of changes: - Initial rockcraft file and tox.ini

* feat: mlserver-xgboost rock canonical/seldon-core-operator#133 Summary of changes: - mlserver-xgboost rockcraft - Unit tests. - tox.ini ready for integration tests. --------- Co-authored-by: Andrew Scribner <[email protected]>

* feat: mlserver-sklearn rock canonical/seldon-core-operator#133 Summary of changes: - Created rockcraft file for mlsever-sklearn rock. - Added unit test. - Added tox.ini - Use variable ROCK_RUNTIME to simplify development. - Modified rockcraft.yaml to use environment variables in services instead of calling export in command line. - Updatde rockcraft yaml to ensure proper startup of service.

* feat: mlserver-mlflow rock canonical/seldon-core-operator#133 Summary of changes: - Created rockcraft file for mlserver-mlflow rock according to best practices spec. - Added unit test. - Added tox.ini - Updated command to execute properly in Seldon Deployment environment. --------- Co-authored-by: Andrew Scribner <[email protected]>

#133 Integration tests for mlsever-mlflow. Similar to other mlserver-* tests. Summary of changes: - Added integration tests files. - Updated tests. - Updated config map to point to published rock.

* feat: mlserver-huggingface rock canonical/seldon-core-operator#133 Summary of changes: - Created rockcraft file for mlserver-xgboost rock according to best practices spec. - Added unit test. - Added tox.ini - Updated rockcraft.yaml to generate ROCK that can be started in Seldon Deployment. - Upstream version upgrade due to breaking changes: Optimum 1.7 changed the import name from `SUPPORTED_TASKS` to `ORT_SUPPORTED_TASKS`. - Update mlserver-huggingface/tox.ini --------- Co-authored-by: Andrew Scribner <[email protected]>

* feat: integration of mlserver-xgboost rock Details are in #133 Summary of changes: - Added reference to published mlserver-xgboost rock to configmap. - Updated sklearn test - Updated integration test response data

Adding ROCKs to weekly CVE scan is part of ROCK delivery. Details are in canonical/seldon-core-operator#133 Summary of changes: Added Seldon ROCKs to weekly scan.

i-chvets · 2023-07-14T15:37:02Z

Recording status of s2i.
Updated ROCK:
https://github.com/canonical/seldonio-rocks/blob/feat-s2i-rock/seldon-core-s2i-python3.8/rockcraft.yaml
Use existing example to buid. When using the ROCK container it fails. Evenlocation of those assemble/run scripts is specified with --image-scripts-url, it still refuses to run:

$ git clone https://github.com/seldonio/seldon-core.git
$ cd seldon-core
$ s2i build --image-scripts-url "image:///s2i/bin" wrappers/s2i/python/test/model-template-app  charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1 test-image
WARNING: could not inspect the builder image for labels: reading manifest v1.16.0_22.04_1 in docker.io/charmedkubeflow/seldon-core-s2i: requested access to the resource is denied
warning: Image sha256:9d65f600b06a0a916b70a3d48f16d1b4197549a63e7cc5f5b9014ce26228ad8d does not contain a value for the io.openshift.s2i.scripts-url label
Build failed
ERROR: An error occurred: failed to install [assemble run]
ERROR: Suggested solution: set the scripts URL parameter with the location of the S2I scripts, or check if the image has the "io.openshift.s2i.scripts-url" label set
ERROR: If the problem persists consult the docs at https://github.com/openshift/source-to-image/tree/master/docs. Eventually reach us on freenode #openshift or file an issue at https://github.com/openshift/source-to-image/issues providing us with a log from your build using log output level 3.

And those script do exist in the ROCK container:

$ docker run charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1 exec ls -la /s2i/bin
drwxr-xr-x 2 root root 4096 Jul 14 15:11 .
drwxr-xr-x 3 root root 4096 Jul 14 15:11 ..
-rwxr-xr-x 1 root root 2399 Jul 14 15:10 assemble
-rwxr-xr-x 1 root root 1495 Jul 14 15:10 run
-rwxr-xr-x 1 root root  406 Jul 14 15:10 save-artifacts
-rwxr-xr-x 1 root root 1867 Jul 14 15:10 usage

i-chvets · 2023-07-14T16:25:09Z

Add label to existin container to enable s2i to find scripts:

echo "FROM charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1" | docker build --label io.openshift.s2i.scripts-url="image:///s2i/bin" -t "charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1" -

Problem: openshift/source-to-image#475

i-chvets · 2023-07-14T16:31:06Z

Update ROCK:

echo '''FROM charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1
ENTRYPOINT []''' | docker build --label io.openshift.s2i.scripts-url="image:///s2i/bin" -t "charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1" -

i-chvets · 2023-07-14T18:59:13Z

It is possible to build S2I base container image using rockcraft.yaml as a source, however it will require some adjustmetns and basically a rebuild using Docker.
The process is as follows:

Build ROCK using rockcraft based on rockcaft.yaml
Copy ROCK to Docker daemon using skopeo
Update label and entrypoint in the image and rebuild

This will create a container image suitable to be used in S2I build process describe in Seldon documentation: https://docs.seldon.io/projects/seldon-core/en/latest/python/python_wrapping_s2i.html

This method does not really produce an S2I base ROCK image based on rockcraft.yaml, but rather Dockerfile based container image.

* feat: initial commit canonical/seldon-core-operator#133 Summary of changes: - Initial rockcraft file and tox.ini - Added organize. - Finalized bazel build. - Install python3.8 via packages - Uncommented cleanup - Added tensorflow-serving rock to ci

i-chvets changed the title ~~Integration of ROCK images~~ Integration of ROCK images into Seldon Core Operator May 17, 2023

NohaIhab added the enhancement New feature or request label May 25, 2023

i-chvets mentioned this issue Jun 6, 2023

feat: Seldon rock integration canonical/seldonio-rocks#12

Merged

i-chvets pushed a commit that referenced this issue Jun 13, 2023

feat: integration tests for seldon rocks

fc03c8e

#133 Summary of changes: - Added integration test for Seldon prepackages servers: xgboost v1 and v2, mlflowserver v1. - Remove tests fails intermittently.

i-chvets mentioned this issue Jun 13, 2023

feat: Integration tests for set of Seldon rocks #150

Merged

i-chvets pushed a commit that referenced this issue Jun 14, 2023

feat: integration tests bumping resources

924c7c3

#133 Summary of changes: - Bump resources for mlflowserver tests.

i-chvets pushed a commit that referenced this issue Jun 14, 2023

eat: integration tests bumping resources

0bfbe30

#133 Summary of changes: - Use lightkube for deletion and testing. - Verify remove test when juju/python-libjuju#877 is resolved.

i-chvets pushed a commit that referenced this issue Jun 14, 2023

feat: integration tests bumping resources

cf666d6

#133 Summary of changes: - Updated integration tests with server names to retrieve proper image name for testing. - Added test for removal of workload deployed configmap.

i-chvets mentioned this issue Jun 14, 2023

feat: sklearnserver integration canonical/seldonio-rocks#11

Merged

i-chvets pushed a commit to canonical/seldonio-rocks that referenced this issue Jun 30, 2023

feat: mlserver-sklearn rock

8269660

canonical/seldon-core-operator#133 Summary of changes: - Created rockcraft file for mlsever-sklearn rock. - Added unit test. - Added tox.ini

i-chvets mentioned this issue Jun 30, 2023

feat: mlserver-sklearn rock canonical/seldonio-rocks#18

Closed

i-chvets pushed a commit to canonical/seldonio-rocks that referenced this issue Jun 30, 2023

feat: mlserver-xgboost rock

6e09e9c

canonical/seldon-core-operator#133 Summary of changes: - mlserver-xgboost rockcraft - Unit tests. - tox.ini ready for integration tests.

i-chvets mentioned this issue Jun 30, 2023

feat: mlserver-xgboost rock canonical/seldonio-rocks#19

Closed

This was referenced Jun 30, 2023

feat: mlserver-mlflow rock canonical/seldonio-rocks#20

Merged

feat: mlserver sklearn rock canonical/seldonio-rocks#21

Merged

feat: mlserver-xgboost rock canonical/seldonio-rocks#22

Merged

i-chvets mentioned this issue Jun 30, 2023

feat: mlserver-huggingface rock canonical/seldonio-rocks#23

Merged

This was referenced Jul 8, 2023

feat mlserver sklearn rock canonical/seldonio-rocks#25

Merged

feat: mlserver-sklearn integration #160

Merged

i-chvets pushed a commit that referenced this issue Jul 8, 2023

feat: mlserver-mlflow integration

383c46d

#133 Summary of changes: - Added integration tests files. - Updated tests. - Updated config map to point to published rock.

i-chvets mentioned this issue Jul 8, 2023

feat: mlserver-mlflow integration #161

Merged

i-chvets pushed a commit to canonical/seldonio-rocks that referenced this issue Jul 8, 2023

feat: mlserver-huggingface integration of rock

f26bc46

canonical/seldon-core-operator#133 Summary of changes: - Updated rockcraft.yaml to generate ROCK that can be started in Seldon Deployment.

i-chvets pushed a commit to canonical/seldonio-rocks that referenced this issue Jul 8, 2023

feat: initial commit

1cbd85d

canonical/seldon-core-operator#133 Summary of changes: - Initial rockcraft file and tox.ini

i-chvets mentioned this issue Jul 8, 2023

feat: mlflowserver rock canonical/seldonio-rocks#26

Open

i-chvets mentioned this issue Jul 10, 2023

feat: mlserver-huggingface rock integration #164

Merged

i-chvets mentioned this issue Jul 10, 2023

feat: integration of mlserver-xgboost rock #165

Merged

i-chvets mentioned this issue Jul 11, 2023

feat: added seldon rocks to scan workflow canonical/seldonio-rocks#28

Merged

i-chvets mentioned this issue Jul 26, 2023

Kf 2271 feat tensorflow serving rock canonical/seldonio-rocks#32

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration of ROCK images into Seldon Core Operator #133

Integration of ROCK images into Seldon Core Operator #133

i-chvets commented May 17, 2023 •

edited

Loading

i-chvets commented Jul 14, 2023

i-chvets commented Jul 14, 2023

i-chvets commented Jul 14, 2023

i-chvets commented Jul 14, 2023 •

edited

Loading

Integration of ROCK images into Seldon Core Operator #133

Integration of ROCK images into Seldon Core Operator #133

Comments

i-chvets commented May 17, 2023 • edited Loading

Description

Design

Testing

Integration tests

Implementation

i-chvets commented Jul 14, 2023

i-chvets commented Jul 14, 2023

i-chvets commented Jul 14, 2023

i-chvets commented Jul 14, 2023 • edited Loading

i-chvets commented May 17, 2023 •

edited

Loading

i-chvets commented Jul 14, 2023 •

edited

Loading