Skip to content

Code for the paper "Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression"

License

Notifications You must be signed in to change notification settings

mansheej/icl-task-diversity

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression

Code for Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression

Research supported with Cloud TPUs from Google's TPU Research Cloud (TRC)

All experiments were run on TPUs using the Google TPU Research Cloud. Apply for TPU access at https://sites.research.google/trc/about/.

After provisioning a TPU VM, create a Python virtual environment using:

conda create -n icl -y python=3.10
conda activate icl

and install dependencies using

pip install -r requirements.txt
pip install -e .

To train a model, modify icl/configs/example.py and then run:

python run.py --config=icl/configs/example.py

About

Code for the paper "Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages