Add image substitution feature to either DSL compiler or KFP client

[Ark-kun]

We need a way to properly perform container image substitution.
The substitution can be performed either during the pipeline compilation or before the run submission. (It can also be performed in the backend.)

run = client.run_pipeline(exp.id, 'my-tfx', 'my-tfx.tar.gz', params={}, image_substitution_map={'gcr.io/old-image': 'gcr.io/new-image'})

[gaoning777]

The name is currently substituted during the compilation stage.

[gaoning777]

#879

[Ark-kun]

#879

That is a different issue. This issue is about replacing gcr.io/ml-pipeline/ with gcr.io/ml-pipeline-test/ in the whole pipeline during compilation or testing.

[gaoning777]

I understand the usefulness of the image replacement but I'm leaning towards supporting the image replacement by the user. It is simply one sed bash command. WDYT?

[elikatsis]

PipelineParams may also be used in the image's place. So they can be set that way on run creation.

[gaoning777]

JFYI, The current DSL compiler already supports substituting images with runtime PipelineParam.
I can only see the use case where the pipeline is uploaded to the KFP while users want to submit a run using that pipeline but need image substitution.
In such case, I would recommend uploading another pipeline since this is another pipeline version, which is simply another API call. WDYT?

[gaoning777]

@Ark-kun I'm gonna close this issue now. Feel free to reopen this if you think otherwise.