Make "HyperParameters" a map

[epa095]

/kind feature

Describe the solution you'd like
I would like to extract the value of a single hyper-parameter inside a worker template. All the examples I see are of the type

            {{- with .HyperParameters}}
            {{- range .}}
            - "{{.Name}}={{.Value}}"
            {{- end}}
            {{- end}}

but I think it would make sense to be able to do something like {{ index HyperParameters "lr" }} or {{ $HyperParameters.lr }}. It might already be possible, but I did not manage to figure out how, and it seems like maybe HyperParameters is a list of pairs, not a map?

[gaocegege]

Thanks for the issue!

Could you please explain more about the use case? We use list, then add the lists to the arguments. And it works well, IMO.

[epa095]

Our usecase is that we have a generall model builder which can take in the model-config as a json (e.g. a json representation of a scikit learn pipeline) like this:

{
    "sklearn.pipeline.Pipeline": {
        "steps": [
            "sklearn.preprocessing.data.MinMaxScaler",
            {
                "sklearn.ensemble.RandomForestClassifier": {
                    "n_estimators": 4,
                    "max_depth": 7,
                    "min_samples_split": 0.2,
                    "max_features": 2
                }
            }
        ]
    }
}

And we would like to HP tune over those parameters. I would like to be able to write:

{
    "sklearn.pipeline.Pipeline": {
        "steps": [
            "sklearn.preprocessing.data.MinMaxScaler",
            {
                "sklearn.ensemble.RandomForestClassifier": {
                    "n_estimators": {{ $HyperParameters.n_estimators }},
                    "max_depth": {{ $HyperParameters.max_depth }},
                    "min_samples_split": {{ $HyperParameters.min_samples_split }},
                    "max_features": {{ $HyperParameters.max_features }}
                }
            }
        ]
    }
}

Today we have to have the model-builder read the config as a string, expand it as a jinja template and then load the json. The result is that we have to do this:

                  {{- with .HyperParameters}}
                 {{- range .}}
                 - "--model-parameter"
                 - "{{.Name}},{{.Value}}"
                 {{- end}}
                 {{- end}}
...
{
    "sklearn.pipeline.Pipeline": {
        "steps": [
            "sklearn.preprocessing.data.MinMaxScaler",
            {
                "sklearn.ensemble.RandomForestClassifier": {
                    "n_estimators":  {{"{{"}} n_estimators {{"}}"}},
                    "max_depth": {{"{{"}} max_depth {{"}}"}},
                    "min_samples_split": {{"{{"}} min_samples_split {{"}}"}},
                    "max_features": {{"{{"}} max_features {{"}}"}}
                }
            }
        ]
    }
}

I would also imagine that if you want to use a docker image you don't directly control, or don't want to make changes to it is nice to be able to extract the hyperparameters to craft the execution of the image as you want.

I must admit I don't know go, but in Python a list of tuples provides no advantage over a dictionary, it is unproblematic to loop over the key-value pairs of a dictionary, and if it is the same in go then I see no reason to not have it as a dictionary(?)

[gaocegege]

Make sense.

WDYT @richardsliu @johnugeorge

[johnugeorge]

Shall we delay this to next release?

[richardsliu]

I wonder if we can create a lightweight tool or UI to create the Experiment spec from simple Python code. That way we don't need to wait until a new API release.

[gaocegege]

If we want it, it should be in the next release. As for the simple wrapper for experiment spec, I think UI can help us. Do we have plan to support it in the new UI?

[andreyvelich]

In the UI we can submit Experiment from yaml file or by parameters.
In the future we can add support to create Experiment from Python code.

[andreyvelich]

@epa095 If it is still relevant, please try out new Trial template API: #1208.
This is an example: https://github.com/kubeflow/katib/blob/master/examples/v1beta1/random-example.yaml#L38-L65.

I think it should work well with your case.

[issue-label-bot]

Issue-Label Bot is automatically applying the labels:

Label	Probability
kind/feature	0.89

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.