Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Commit

Permalink
Merge branch 'dev' into docs-text-to-image-refactor
Browse files Browse the repository at this point in the history
  • Loading branch information
jupyterjazz authored Jun 17, 2022
2 parents ef55bcc + cdc27d5 commit 6f5495a
Show file tree
Hide file tree
Showing 2 changed files with 42 additions and 35 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- Refactor the guide for image to image search. ([#458](https://github.com/jina-ai/finetuner/pull/458))

- Refactor the guide for text to image search. ([#459](https://github.com/jina-ai/finetuner/pull/459))

### Fixed
Expand Down
75 changes: 40 additions & 35 deletions docs/3_finetuner_in_action/3_2_image_to_image.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
(image-to-image)=
# Image to image search using ResNet
# ResNet for image to image search

This guide will demonstrate how to fine-tune a ResNet model for image to image retrieval.

## Task Overview
## Task
More specifically, we will fine-tune ResNet50 on [Totally Looks Like Dataset](https://sites.google.com/view/totally-looks-like-dataset).
The dataset consists of 6016 pairs of images (12032 in total).

Expand All @@ -17,30 +17,32 @@ The dataset consists of pairs of images, these are the positive pairs. Negative
After fine-tuning, the embeddings of positive pairs are expected to be pulled closer, while the embeddings for negative pairs are expected to be pushed away.


## Preparing data
Training and evaluation data are already prepared and pushed to Hubble following the {ref}`instructions <create-training-data>`.
You can either pull the data:
```python
from docarray import DocumentArray
train_data = DocumentArray.pull('resnet-ttl-train-data')
eval_data = DocumentArray.pull('resnet-ttl-eval-data')
## Data
Our journey starts locally. We have to {ref}`prepare the data and push it to the cloud <create-training-data>` and Finetuner will be able to get the dataset by its name. For this example,
we already prepared the data, and we'll provide the names of training and evaluation data (`resnet-ttl-train-data` and `resnet-ttl-eval-data`) directly to Finetuner.

```{admonition}
:class: tip
We don't require you to push data to the cloud by yourself. Instead of a name, you can provide a `DocumentArray` and Finetuner will do the job for you.
```
Or specify given `DocumentArray` names (`resnet-ttl-train-data` and `resnet-ttl-eval-data`) directly to Finetuner.

## Choosing the model
Now let's see what backbone models we can use. You can see available models either in {ref}`choose backbone <choose-backbone>` section or by calling `finetuner.describe_models()`.

## Backbone model
Now let's see which backbone models we can use. You can see available models either in {ref}`choose backbone <choose-backbone>` section or by calling {meth}`~finetuner.describe_models()`.


For this example, we're gonna go with `resnet50`.

## Creating a fine-tuning job
You can easily start a fine-tuning run with `finetuner.fit`.
## Fine-tuning
From now on, all the action happens in the cloud!

```{admonition} Login to Jina Cloud
:class: tip
Before creating a run, you need to {ref}`login to Jina ecosystem <login-to-jina-ecosystem>` by calling `finetuner.login()`.
First you need to {ref}`login to Jina ecosystem <login-to-jina-ecosystem>`:
```python
import finetuner
finetuner.login()
```

Now, you can easily start a fine-tuning job with {meth}`~finetuner.fit`:
```python
from finetuner.callback import BestModelCheckpoint, EvaluationCallback

Expand All @@ -56,20 +58,23 @@ run = finetuner.fit(
learning_rate=0.001,
)
```
Now, let's understand what this piece of code does.
Let's understand what this piece of code does:
```{admonition} finetuner.fit parameters
:class: tip
The only required arguments are `model` and `train_data`. We provide default values for others. Here is the [full list of the parameters](../../api/finetuner/#finetuner.fit).
```
As you can see, we have to provide the `model` which we picked before. We also set `run_name` and `description`, which are optional,
but recommended in order to retrieve your run easily and have some context about it. Furthermore, we had to provide `train_data` and `eval_data`. As you can see,
we used the names of the `DocumentArray`s that are already on Hubble, but we could also pass a `DocumentArray` object itself, which will be automatically uploaded to Hubble. As stated before, we want to use the `TripletLoss`, and that's what `loss='TripletMarginLoss'` corresponds to.
Additionally, we use `BestModelCheckpoint` to save the best model after each epoch and `EvaluationCallback` for evaluation. Lastly, we set number of `epochs` and provide a `learning_rate`.
* As you can see, we have to provide the `model` which we picked before.
* We also set `run_name` and `description`, which are optional,
but recommended in order to retrieve your run easily and have some context about it.
* Furthermore, we had to provide names of the `train_data` and `eval_data`.
* We set `TripletMarginLoss`.
* Additionally, we use {class}`~finetuner.callback.BestModelCheckpoint` to save the best model after each epoch and {class}`~finetuner.callback.EvaluationCallback` for evaluation.
* Lastly, we set number of `epochs` and provide a `learning_rate`.


## Monitoring your runs
## Monitoring

Let's check the status of our runs.
Let's check the status of the run.
```python
print(run.status())
```
Expand All @@ -87,18 +92,11 @@ finetuner.login()
run = finetuner.get_run('resnet-ttl')
```

You can continue monitoring the runs by checking the status - `run.status()` or the logs - `run.logs()`.

## Saving your model
You can continue monitoring the runs by checking the status - {meth}`~finetuner.run.Run.status()` or the logs - {meth}`~finetuner.run.Run.logs()`.

If your runs have finished successfully, you can save fine-tuned models in the following way:
```python
run.save_model('resnet-model')
```

## Evaluating your model
Currently, we don't have a user-friendly way to get evaluation metrics from the `EvaluationCallback` we initialized previously.
What you can do for now is to call `run.logs()` in the end of the run and see evaluation results:
## Evaluating
Currently, we don't have a user-friendly way to get evaluation metrics from the {class}`~finetuner.callback.EvaluationCallback` we initialized previously.
What you can do for now is to call {meth}`~finetuner.run.Run.logs()` in the end of the run and see evaluation results:

```bash
[10:37:49] DEBUG Metric: 'model_average_precision' Value: 0.30105 __main__.py:217
Expand All @@ -117,3 +115,10 @@ What you can do for now is to call `run.logs()` in the end of the run and see ev
[10:38:14] INFO Pushed model artifact ID: '62a1af491597c219f6a330fe' __main__.py:246
INFO Finished 🚀 __main__.py:248
```

## Saving

After the run has finished successfully, you can download the tuned model on your local machine:
```python
run.save_model('resnet-model')
```

0 comments on commit 6f5495a

Please sign in to comment.