Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

refactor: modify artifact-id logic; jsonify responses #390

Merged
merged 6 commits into from
May 10, 2022

Conversation

jupyterjazz
Copy link
Member

@jupyterjazz jupyterjazz commented May 9, 2022

Closes https://github.com/jina-ai/finetuner.fit/issues/209 and https://github.com/jina-ai/finetuner.fit/issues/210

  • Support jsonified dicts instead of requests.Response objects
  • Change the logic behind retrieving artifact-id for downloading finetuned model(s)

@jupyterjazz
Copy link
Member Author

Tests aren't passing here because we haven't updated FINETUNER_HOST secret on Github.

Copy link
Member

@bwanglzu bwanglzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some comments

finetuner/client/client.py Outdated Show resolved Hide resolved
finetuner/client/client.py Outdated Show resolved Hide resolved
Comment on lines 238 to 240
artifact_ids = self.get_run(experiment_name=experiment_name, run_name=run_name)[
ARTIFACT_IDS
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this part let me clarify it to you, i think it is wrong.

@@ -19,3 +19,4 @@
EVAL_DATA = 'eval_data'
FINETUNED_MODELS_DIR = 'finetuned_models'
MODEL = 'model'
ARTIFACT_IDS = 'FINETUNER_ARTIFACT_IDS'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as mentioned above, it is the env variable we send to AWS batch, not the field we saved in the database.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but how do I, as a client, retrieve it then?

@jupyterjazz jupyterjazz requested a review from bwanglzu May 9, 2022 14:46
@@ -47,16 +47,16 @@ def _handle_request(
method: str,
params: Optional[dict] = None,
json_data: Optional[dict] = None,
) -> requests.Response:
) -> Union[dict, List[dict]]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

list[dict]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nevermind, I guess we want to support >=3.7


:param experiment_name: The name of the experiment.
:param run_name: The name of the run.
:param path: Directory where the model will be stored.
:returns: A str object indicates the download path on localhost.
:returns: A list of str object(s) that indicate the download path on localhost.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

localhost? You mean local disk?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly just copied that from hubble's download_artifact docstring. But I see your point, it's a bit confusing here.

Copy link
Member

@bwanglzu bwanglzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@bwanglzu
Copy link
Member

test is failing

@jupyterjazz
Copy link
Member Author

@bwanglzu

test is failing

Tests are failing here because we haven't updated FINETUNER_HOST secret on Github.

Copy link
Member

@gmastrapas gmastrapas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@jupyterjazz jupyterjazz merged commit dd6c5ee into dev May 10, 2022
@jupyterjazz jupyterjazz deleted the refactor-artifact-id branch May 10, 2022 08:32
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants