Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

[Retiarii] Grid search, random and evolution strategy #3377

Merged
merged 13 commits into from
Feb 24, 2021

Conversation

ultmaster
Copy link
Contributor

@ultmaster ultmaster commented Feb 9, 2021

This PR refines and adds new strategies to Retiarii.

  • Grid search
  • Random
  • Evolution
  • tests

@ultmaster ultmaster marked this pull request as draft February 9, 2021 11:18


class Random(BaseStrategy):
def __init__(self, variational=False, dedup=True):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the meaning of variational?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs added.

else:
_logger.info('Random search running in fixed size mode. Dedup: %s.', 'on' if self.dedup else 'off')
search_space = dry_run_for_search_space(base_model, applied_mutators)
for sample in random_generator(search_space, dedup=self.dedup):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will webui show if strategy exits but maxtrialnum and maxduration are not reached?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested. The whole experiment directly exits and dispatcher (strategy) is terminated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dispatcher is not strategy, why dispatcher exits?...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Dispatcher terminated" is printed on the console. Afterwards, no more trials appear on the WebUI. I'm not sure about the details.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, this is a normal behavior

@ultmaster ultmaster marked this pull request as ready for review February 22, 2021 06:56
population_size : int
The number of individuals to keep in the population.
cycles : int
The number of cycles (trials) the algorithm should run for.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cycle is a little misleading, cycle means trial in the paper?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The terminology is used in paper and their open-source implementation.

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@J-shang J-shang mentioned this pull request Feb 24, 2021
94 tasks
@ultmaster ultmaster merged commit fddc8ad into microsoft:master Feb 24, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants