Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add auxiliary_experiments to Experiment #2634

Closed
wants to merge 3 commits into from

Conversation

ItsMrLin
Copy link
Contributor

@ItsMrLin ItsMrLin commented Aug 4, 2024

Summary: Add auxiliary_experiments to Experiment.

Differential Revision: D60542566

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Aug 4, 2024
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D60542566

@codecov-commenter
Copy link

codecov-commenter commented Aug 4, 2024

Codecov Report

Attention: Patch coverage is 90.80460% with 8 lines in your changes missing coverage. Please review.

Project coverage is 95.25%. Comparing base (a61c362) to head (d77a67f).

Files Patch % Lines
ax/utils/testing/preference_stubs.py 76.92% 6 Missing ⚠️
ax/core/auxiliary.py 87.50% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2634      +/-   ##
==========================================
- Coverage   95.26%   95.25%   -0.01%     
==========================================
  Files         493      495       +2     
  Lines       47753    47831      +78     
==========================================
+ Hits        45493    45563      +70     
- Misses       2260     2268       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Summary: Expanding get_pbo_experiment to be able to generate preference exploration experiments with only preferential feedback and nearly unbounded (technically really large) search space.

Differential Revision: D60192601
ItsMrLin added a commit to ItsMrLin/Ax that referenced this pull request Aug 14, 2024
Summary:
Pull Request resolved: facebook#2634

Add `auxiliary_experiments_by_purpose ` to `Experiment` object.

This change allows for `Experiment` to hold a dictionary of list of auxiliary experiments for different purposes. This is helpful for methods such as transfer learning or preference learning where we are using other experiments as auxiliary information (e.g., in modelbridge) to facilitate the optimization of the experiment of interest.

Differential Revision: D60542566
ItsMrLin added a commit to ItsMrLin/Ax that referenced this pull request Aug 14, 2024
Summary:
Pull Request resolved: facebook#2634

Add `auxiliary_experiments_by_purpose ` to `Experiment` object.

This change allows for `Experiment` to hold a dictionary of list of auxiliary experiments for different purposes. This is helpful for methods such as transfer learning or preference learning where we are using other experiments as auxiliary information (e.g., in modelbridge) to facilitate the optimization of the experiment of interest.

Differential Revision: D60542566
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D60542566

ItsMrLin added a commit to ItsMrLin/Ax that referenced this pull request Aug 14, 2024
Summary:
Pull Request resolved: facebook#2634

Add `auxiliary_experiments_by_purpose ` to `Experiment` object.

This change allows for `Experiment` to hold a dictionary of list of auxiliary experiments for different purposes. This is helpful for methods such as transfer learning or preference learning where we are using other experiments as auxiliary information (e.g., in modelbridge) to facilitate the optimization of the experiment of interest.

Reviewed By: saitcakmak

Differential Revision: D60542566
Summary:
Pull Request resolved: facebook#2632

Adding AuxiliaryExperiment

Meta: Design doc https://docs.google.com/document/d/1MYq4WHPDLoWp-RUyk7jXiqswKARtyoSeM68PRayPGUk/edit?usp=sharing

Differential Revision: D60192602

Reviewed By: mgarrard
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D60542566

ItsMrLin added a commit to ItsMrLin/Ax that referenced this pull request Aug 14, 2024
Summary:
Pull Request resolved: facebook#2634

Add `auxiliary_experiments_by_purpose ` to `Experiment` object.

This change allows for `Experiment` to hold a dictionary of list of auxiliary experiments for different purposes. This is helpful for methods such as transfer learning or preference learning where we are using other experiments as auxiliary information (e.g., in modelbridge) to facilitate the optimization of the experiment of interest.

Reviewed By: saitcakmak

Differential Revision: D60542566
ItsMrLin added a commit to ItsMrLin/Ax that referenced this pull request Aug 14, 2024
Summary:
Pull Request resolved: facebook#2634

Add `auxiliary_experiments_by_purpose ` to `Experiment` object.

This change allows for `Experiment` to hold a dictionary of list of auxiliary experiments for different purposes. This is helpful for methods such as transfer learning or preference learning where we are using other experiments as auxiliary information (e.g., in modelbridge) to facilitate the optimization of the experiment of interest.

Differential Revision: D60542566
Summary:
Pull Request resolved: facebook#2634

Add `auxiliary_experiments_by_purpose ` to `Experiment` object.

This change allows for `Experiment` to hold a dictionary of list of auxiliary experiments for different purposes. This is helpful for methods such as transfer learning or preference learning where we are using other experiments as auxiliary information (e.g., in modelbridge) to facilitate the optimization of the experiment of interest.

Reviewed By: saitcakmak

Differential Revision: D60542566
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D60542566

ItsMrLin added a commit to ItsMrLin/Ax that referenced this pull request Aug 15, 2024
Summary:
Pull Request resolved: facebook#2634

Add `auxiliary_experiments_by_purpose ` to `Experiment` object.

This change allows for `Experiment` to hold a dictionary of list of auxiliary experiments for different purposes. This is helpful for methods such as transfer learning or preference learning where we are using other experiments as auxiliary information (e.g., in modelbridge) to facilitate the optimization of the experiment of interest.

Differential Revision: D60542566
ItsMrLin added a commit to ItsMrLin/Ax that referenced this pull request Aug 15, 2024
Summary:
Pull Request resolved: facebook#2634

Add `auxiliary_experiments_by_purpose ` to `Experiment` object.

This change allows for `Experiment` to hold a dictionary of list of auxiliary experiments for different purposes. This is helpful for methods such as transfer learning or preference learning where we are using other experiments as auxiliary information (e.g., in modelbridge) to facilitate the optimization of the experiment of interest.

Differential Revision: D60542566
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 02df121.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants