Fix Grokking Notebook #450
Conversation
| @@ -188,7 +190,8 @@ | |||
| "outputs": [], | |||
| "source": [ | |||
| "# where we save the model\n", | |||
| "PTH_LOCATION = \"/workspace/_scratch/grokking_demo.pth\"" | |||
| "PTH_LOCATION = \"/workspace/_scratch/grokking_demo.pth\"\n", | |||
| "assert os.path.exists(Path(PTH_LOCATION).parent), f\"Please create the folder {Path(PTH_LOCATION).parent} before running this notebook\"" | |||
There was a problem hiding this comment.
probably bettern than before but TBH, the AssertionError is not really that much fun to see after returning to the Colab after letting it install and run for a few minutes ... would os.mkdir(...) be possible instead?
There was a problem hiding this comment.
also, what's the deal with absolute paths? does Google Colab map root folder to something I can create?
FTR relative path (without starting /) seems to work fine in Colab:
| @@ -3497,7 +3515,7 @@ | |||
| } | |||
| ], | |||
| "source": [ | |||
| "print(loss_fn(all_logits, labels))" | |||
| "print(loss_fn(all_logits, labels)) # This bugged on models not fully trained " | |||
There was a problem hiding this comment.
👍 thanks for the comment ... it does indeed 😅
| @@ -3274,7 +3292,7 @@ | |||
| "get_metrics(model, metric_cache, get_cos_sim, \"cos_sim\")\n", | |||
There was a problem hiding this comment.
FTR this bugs out on the free Colab GPU:
OutOfMemoryError: CUDA out of memory. Tried to allocate 76.00 MiB. GPU 0 has a total capacty of 14.75 GiB of which 26.81 MiB is free. Process 2651 has 14.72 GiB memory in use. Of the allocated memory 14.05 GiB is allocated by PyTorch, and 512.72 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
|
Thanks! Test fail seems unrelated so I'm merging |
This used several old imports.
Changed to install TL main and https://github.com/neelnanda-io/neel-plotly