Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FEAT: Add finetune_depth parameter #471

Merged
merged 24 commits into from
Oct 15, 2024
Merged

FEAT: Add finetune_depth parameter #471

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
85a7336
Add finetune_depth param
marcopeix Sep 10, 2024
3f21133
Adjust tutorials and capabilities with finetune_depth
marcopeix Sep 10, 2024
2e171ef
Merge branch 'main' into feature/finetune_depth
marcopeix Sep 13, 2024
b61788b
Remove mentions of layers for finetune_depth
marcopeix Sep 13, 2024
f7ff206
Remove mention of layers
marcopeix Sep 16, 2024
a7b6e73
Merge branch 'main' into feature/finetune_depth
elephaint Sep 24, 2024
7bc1b5d
remove_layer_mentions
elephaint Sep 24, 2024
92c8562
Apply suggestions from code review
elephaint Sep 25, 2024
cfe1a20
fix_linting_add_test
elephaint Sep 25, 2024
4517b7f
clean_nb
elephaint Sep 25, 2024
f296c26
Merge branch 'main' into feature/finetune_depth
elephaint Oct 3, 2024
0cccc0a
remove_test
elephaint Oct 3, 2024
c1e980d
Merge branch 'feature/finetune_depth' of https://github.com/Nixtla/ni…
elephaint Oct 3, 2024
9609faf
clean_nb
elephaint Oct 3, 2024
8cad1a5
rerun_notebook
elephaint Oct 3, 2024
b905f61
fix_example
elephaint Oct 3, 2024
a9aa1a3
fix_example
elephaint Oct 3, 2024
d87c1cf
fix_comments
elephaint Oct 3, 2024
fa1b344
Merge branch 'main' into feature/finetune_depth
marcopeix Oct 8, 2024
c650266
Rephrase according to Cristian comments
marcopeix Oct 8, 2024
e729452
Merge branch 'main' into feature/finetune_depth
elephaint Oct 10, 2024
87110ce
fix_typos
elephaint Oct 10, 2024
b3eb02b
clean_nb
elephaint Oct 10, 2024
3c66060
Merge branch 'main' into feature/finetune_depth
elephaint Oct 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 31 additions & 26 deletions nbs/docs/capabilities/forecast/07_finetuning.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,20 +55,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"[![](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Nixtla/nixtla/blob/main/nbs/docs/capabilities/forecast/07_finetuning.ipynb)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"outputs": [],
"source": [
"#| echo: false\n",
"if not IN_COLAB:\n",
Expand Down Expand Up @@ -124,18 +111,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:nixtla.nixtla_client:Validating inputs...\n",
"INFO:nixtla.nixtla_client:Preprocessing dataframes...\n",
"INFO:nixtla.nixtla_client:Inferred freq: MS\n",
"INFO:nixtla.nixtla_client:Calling Forecast Endpoint...\n"
]
}
],
"outputs": [],
"source": [
"# Read data\n",
"df = pd.read_csv(\"https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/air_passengers.csv\")\n",
Expand Down Expand Up @@ -166,6 +142,35 @@
"> By default, `timegpt-1` is used. Please see [this tutorial](https://docs.nixtla.io/docs/tutorials-long_horizon_forecasting) on how and when to use `timegpt-1-long-horizon`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By default, only the last layer of the model is finetuned. We can finetune more layers using `finetune_depth` parameter. "
marcopeix marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Read data\n",
"df = pd.read_csv(\"https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/air_passengers.csv\")\n",
"\n",
"# Forecast with fine-tuning.\n",
"# Here, we fine-tune for 5 steps\n",
"# and we finetune more than just the last layer\n",
"forecast_df = nixtla_client.forecast(\n",
" df=df,\n",
" h=12,\n",
" finetune_steps=5,\n",
" finetune_depth=2,\n",
" time_col='timestamp',\n",
" target_col=\"value\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down
178 changes: 168 additions & 10 deletions nbs/docs/tutorials/06_finetuning.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

21 changes: 21 additions & 0 deletions nbs/src/nixtla_client.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@
"#| exporti\n",
"_Loss = Literal[\"default\", \"mae\", \"mse\", \"rmse\", \"mape\", \"smape\"]\n",
"_Model = Literal[\"azureai\", \"timegpt-1\", \"timegpt-1-long-horizon\"]\n",
"_Finetune_Depth = Literal[1, 2, 3, 4, 5]\n",
"\n",
"_date_features_by_freq = {\n",
" # Daily frequencies\n",
Expand Down Expand Up @@ -913,6 +914,7 @@
" level: Optional[List[Union[int, float]]] = None,\n",
" quantiles: Optional[List[float]] = None,\n",
" finetune_steps: NonNegativeInt = 0,\n",
" finetune_depth: _Finetune_Depth = 1,\n",
" finetune_loss: _Loss = 'default',\n",
" clean_ex_first: bool = True,\n",
" validate_api_key: bool = False,\n",
Expand Down Expand Up @@ -964,6 +966,9 @@
" finetune_steps : int (default=0)\n",
" Number of steps used to finetune learning TimeGPT in the\n",
" new data.\n",
" finetune_depth: int (default=1)\n",
" Depth of finetuning, on a scale from 1 to 5. 1 means only the last layer is finetuned\n",
" and 5 means the whole model if finetuned.\n",
" finetune_loss : str (default='default')\n",
" Loss function to use for finetuning. Options are: `default`, `mae`, `mse`, `rmse`, `mape`, and `smape`.\n",
" clean_ex_first : bool (default=True)\n",
Expand Down Expand Up @@ -1013,6 +1018,7 @@
" level=level,\n",
" quantiles=quantiles,\n",
" finetune_steps=finetune_steps,\n",
" finetune_depth=finetune_depth,\n",
" finetune_loss=finetune_loss,\n",
" clean_ex_first=clean_ex_first,\n",
" validate_api_key=validate_api_key,\n",
Expand Down Expand Up @@ -1099,6 +1105,7 @@
" 'clean_ex_first': clean_ex_first,\n",
" 'level': level,\n",
" 'finetune_steps': finetune_steps,\n",
" 'finetune_depth': finetune_depth,\n",
" 'finetune_loss': finetune_loss,\n",
" 'feature_contributions': feature_contributions and X is not None,\n",
" }\n",
Expand Down Expand Up @@ -1331,6 +1338,7 @@
" n_windows: PositiveInt = 1,\n",
" step_size: Optional[PositiveInt] = None,\n",
" finetune_steps: NonNegativeInt = 0,\n",
" finetune_depth: _Finetune_Depth = 1,\n",
" finetune_loss: str = 'default',\n",
" clean_ex_first: bool = True,\n",
" date_features: Union[bool, List[str]] = False,\n",
Expand Down Expand Up @@ -1383,6 +1391,9 @@
" finetune_steps : int (default=0)\n",
" Number of steps used to finetune TimeGPT in the\n",
" new data.\n",
" finetune_depth: int (default=1)\n",
" Depth of finetuning, on a scale from 1 to 5. 1 means only the last layer is finetuned\n",
" and 5 means the whole model if finetuned.\n",
" finetune_loss : str (default='default')\n",
" Loss function to use for finetuning. Options are: `default`, `mae`, `mse`, `rmse`, `mape`, and `smape`.\n",
" clean_ex_first : bool (default=True)\n",
Expand Down Expand Up @@ -1426,6 +1437,7 @@
" step_size=step_size,\n",
" validate_api_key=validate_api_key,\n",
" finetune_steps=finetune_steps,\n",
" finetune_depth=finetune_depth,\n",
" finetune_loss=finetune_loss,\n",
" clean_ex_first=clean_ex_first,\n",
" date_features=date_features,\n",
Expand Down Expand Up @@ -1508,6 +1520,7 @@
" 'clean_ex_first': clean_ex_first,\n",
" 'level': level,\n",
" 'finetune_steps': finetune_steps,\n",
" 'finetune_depth': finetune_depth,\n",
" 'finetune_loss': finetune_loss,\n",
" }\n",
" with httpx.Client(**self._client_kwargs) as client:\n",
Expand Down Expand Up @@ -2618,6 +2631,7 @@
" level: Optional[List[Union[int, float]]],\n",
" quantiles: Optional[List[float]],\n",
" finetune_steps: NonNegativeInt,\n",
" finetune_depth: _Finetune_Depth,\n",
" finetune_loss: _Loss,\n",
" clean_ex_first: bool,\n",
" validate_api_key: bool,\n",
Expand Down Expand Up @@ -2645,6 +2659,7 @@
" level=level,\n",
" quantiles=quantiles,\n",
" finetune_steps=finetune_steps,\n",
" finetune_depth=finetune_depth,\n",
" finetune_loss=finetune_loss,\n",
" clean_ex_first=clean_ex_first,\n",
" validate_api_key=validate_api_key,\n",
Expand Down Expand Up @@ -2700,6 +2715,7 @@
" n_windows: PositiveInt,\n",
" step_size: Optional[PositiveInt],\n",
" finetune_steps: NonNegativeInt,\n",
" finetune_depth: _Finetune_Depth,\n",
" finetune_loss: str,\n",
" clean_ex_first: bool,\n",
" date_features: Union[bool, List[str]],\n",
Expand All @@ -2720,6 +2736,7 @@
" n_windows=n_windows,\n",
" step_size=step_size,\n",
" finetune_steps=finetune_steps,\n",
" finetune_depth=finetune_depth,\n",
" finetune_loss=finetune_loss,\n",
" clean_ex_first=clean_ex_first,\n",
" date_features=date_features,\n",
Expand Down Expand Up @@ -2806,6 +2823,7 @@
" level: Optional[List[Union[int, float]]],\n",
" quantiles: Optional[List[float]],\n",
" finetune_steps: NonNegativeInt,\n",
" finetune_depth: _Finetune_Depth,\n",
" finetune_loss: _Loss,\n",
" clean_ex_first: bool,\n",
" validate_api_key: bool,\n",
Expand Down Expand Up @@ -2861,6 +2879,7 @@
" level=level,\n",
" quantiles=quantiles,\n",
" finetune_steps=finetune_steps,\n",
" finetune_depth=finetune_depth,\n",
" finetune_loss=finetune_loss,\n",
" clean_ex_first=clean_ex_first,\n",
" validate_api_key=validate_api_key,\n",
Expand Down Expand Up @@ -2942,6 +2961,7 @@
" n_windows: PositiveInt,\n",
" step_size: Optional[PositiveInt],\n",
" finetune_steps: NonNegativeInt,\n",
" finetune_depth: _Finetune_Depth,\n",
" finetune_loss: _Loss,\n",
" clean_ex_first: bool,\n",
" date_features: Union[bool, List[Union[str, Callable]]],\n",
Expand Down Expand Up @@ -2978,6 +2998,7 @@
" n_windows=n_windows,\n",
" step_size=step_size,\n",
" finetune_steps=finetune_steps,\n",
" finetune_depth=finetune_depth,\n",
" finetune_loss=finetune_loss,\n",
" clean_ex_first=clean_ex_first,\n",
" date_features=date_features,\n",
Expand Down
21 changes: 21 additions & 0 deletions nixtla/nixtla_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@
# %% ../nbs/src/nixtla_client.ipynb 7
_Loss = Literal["default", "mae", "mse", "rmse", "mape", "smape"]
_Model = Literal["azureai", "timegpt-1", "timegpt-1-long-horizon"]
_Finetune_Depth = Literal[1, 2, 3, 4, 5]

_date_features_by_freq = {
# Daily frequencies
Expand Down Expand Up @@ -847,6 +848,7 @@ def forecast(
level: Optional[List[Union[int, float]]] = None,
quantiles: Optional[List[float]] = None,
finetune_steps: NonNegativeInt = 0,
finetune_depth: _Finetune_Depth = 1,
finetune_loss: _Loss = "default",
clean_ex_first: bool = True,
validate_api_key: bool = False,
Expand Down Expand Up @@ -898,6 +900,9 @@ def forecast(
finetune_steps : int (default=0)
Number of steps used to finetune learning TimeGPT in the
new data.
finetune_depth: int (default=1)
Depth of finetuning, on a scale from 1 to 5. 1 means only the last layer is finetuned
marcopeix marked this conversation as resolved.
Show resolved Hide resolved
and 5 means the whole model if finetuned.
finetune_loss : str (default='default')
Loss function to use for finetuning. Options are: `default`, `mae`, `mse`, `rmse`, `mape`, and `smape`.
clean_ex_first : bool (default=True)
Expand Down Expand Up @@ -947,6 +952,7 @@ def forecast(
level=level,
quantiles=quantiles,
finetune_steps=finetune_steps,
finetune_depth=finetune_depth,
finetune_loss=finetune_loss,
clean_ex_first=clean_ex_first,
validate_api_key=validate_api_key,
Expand Down Expand Up @@ -1035,6 +1041,7 @@ def forecast(
"clean_ex_first": clean_ex_first,
"level": level,
"finetune_steps": finetune_steps,
"finetune_depth": finetune_depth,
"finetune_loss": finetune_loss,
"feature_contributions": feature_contributions and X is not None,
}
Expand Down Expand Up @@ -1283,6 +1290,7 @@ def cross_validation(
n_windows: PositiveInt = 1,
step_size: Optional[PositiveInt] = None,
finetune_steps: NonNegativeInt = 0,
finetune_depth: _Finetune_Depth = 1,
finetune_loss: str = "default",
clean_ex_first: bool = True,
date_features: Union[bool, List[str]] = False,
Expand Down Expand Up @@ -1335,6 +1343,9 @@ def cross_validation(
finetune_steps : int (default=0)
Number of steps used to finetune TimeGPT in the
new data.
finetune_depth: int (default=1)
Depth of finetuning, on a scale from 1 to 5. 1 means only the last layer is finetuned
marcopeix marked this conversation as resolved.
Show resolved Hide resolved
and 5 means the whole model if finetuned.
finetune_loss : str (default='default')
Loss function to use for finetuning. Options are: `default`, `mae`, `mse`, `rmse`, `mape`, and `smape`.
clean_ex_first : bool (default=True)
Expand Down Expand Up @@ -1378,6 +1389,7 @@ def cross_validation(
step_size=step_size,
validate_api_key=validate_api_key,
finetune_steps=finetune_steps,
finetune_depth=finetune_depth,
finetune_loss=finetune_loss,
clean_ex_first=clean_ex_first,
date_features=date_features,
Expand Down Expand Up @@ -1460,6 +1472,7 @@ def cross_validation(
"clean_ex_first": clean_ex_first,
"level": level,
"finetune_steps": finetune_steps,
"finetune_depth": finetune_depth,
"finetune_loss": finetune_loss,
}
with httpx.Client(**self._client_kwargs) as client:
Expand Down Expand Up @@ -1618,6 +1631,7 @@ def _forecast_wrapper(
level: Optional[List[Union[int, float]]],
quantiles: Optional[List[float]],
finetune_steps: NonNegativeInt,
finetune_depth: _Finetune_Depth,
finetune_loss: _Loss,
clean_ex_first: bool,
validate_api_key: bool,
Expand Down Expand Up @@ -1645,6 +1659,7 @@ def _forecast_wrapper(
level=level,
quantiles=quantiles,
finetune_steps=finetune_steps,
finetune_depth=finetune_depth,
finetune_loss=finetune_loss,
clean_ex_first=clean_ex_first,
validate_api_key=validate_api_key,
Expand Down Expand Up @@ -1702,6 +1717,7 @@ def _cross_validation_wrapper(
n_windows: PositiveInt,
step_size: Optional[PositiveInt],
finetune_steps: NonNegativeInt,
finetune_depth: _Finetune_Depth,
finetune_loss: str,
clean_ex_first: bool,
date_features: Union[bool, List[str]],
Expand All @@ -1722,6 +1738,7 @@ def _cross_validation_wrapper(
n_windows=n_windows,
step_size=step_size,
finetune_steps=finetune_steps,
finetune_depth=finetune_depth,
finetune_loss=finetune_loss,
clean_ex_first=clean_ex_first,
date_features=date_features,
Expand Down Expand Up @@ -1811,6 +1828,7 @@ def _distributed_forecast(
level: Optional[List[Union[int, float]]],
quantiles: Optional[List[float]],
finetune_steps: NonNegativeInt,
finetune_depth: _Finetune_Depth,
finetune_loss: _Loss,
clean_ex_first: bool,
validate_api_key: bool,
Expand Down Expand Up @@ -1867,6 +1885,7 @@ def format_X_df(
level=level,
quantiles=quantiles,
finetune_steps=finetune_steps,
finetune_depth=finetune_depth,
finetune_loss=finetune_loss,
clean_ex_first=clean_ex_first,
validate_api_key=validate_api_key,
Expand Down Expand Up @@ -1950,6 +1969,7 @@ def _distributed_cross_validation(
n_windows: PositiveInt,
step_size: Optional[PositiveInt],
finetune_steps: NonNegativeInt,
finetune_depth: _Finetune_Depth,
finetune_loss: _Loss,
clean_ex_first: bool,
date_features: Union[bool, List[Union[str, Callable]]],
Expand Down Expand Up @@ -1986,6 +2006,7 @@ def _distributed_cross_validation(
n_windows=n_windows,
step_size=step_size,
finetune_steps=finetune_steps,
finetune_depth=finetune_depth,
finetune_loss=finetune_loss,
clean_ex_first=clean_ex_first,
date_features=date_features,
Expand Down