Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate registry registration for tasks #116

Conversation

polm-stability
Copy link
Collaborator

Tasks are managed in a registry. Historically this registry is a manually updated giant dictionary. This PR adds code that automates adding functions to the registry, which reduces the potential for mistakes and removes redundant code.

Currently this only handles Japanese tasks. It's hard to automate this for all existing tasks because the relation of the task name (the dictionary key) to the class or file names is very inconsistent. While updating the code to add manual specifications wouldn't be difficult, there are so many tasks it would take a while, so for now older tasks have just been left as-is.

Rather than manually build the registry, this iterates overall subtasks
of Task to build it. This makes registry entries less manual and error
prone, and removes redundant code.

This is not complete. There are two main issues:

1. Registry manipulation functions (with names like `create_tasks`)
   should be removed.

2. For older English tasks with multiple versions, tasks aren't added to
   the registry properly.

Neither of these are complicated, they will be added in further commits.
This restores the manual creation of the registry for non-Japanese
tasks. While it's possible to register them automatically, the mapping
of the module or class name to the task name varies considerably, so it
would require a lot of special casing to get it right.
@polm-stability polm-stability requested review from mrorii and mkshing and removed request for jon-tow November 21, 2023 04:52
Copy link

@mrorii mrorii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Copy link

@mkshing mkshing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@polm-stability polm-stability merged commit effdbea into Stability-AI:jp-stable Nov 21, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants