Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define useful celery beat task for development #3762

Merged
merged 5 commits into from
Mar 23, 2018

Conversation

humitos
Copy link
Member

@humitos humitos commented Mar 10, 2018

Define the task here and remove it from readthedocs-ops.

Copy link
Member

@ericholscher ericholscher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like having these as part of the core RTD code, since they do make sense outside of the scale of RTD (unlike the cold storage stuff). We need to make sure all the existing prod configs work well with this though.

orphan_domain_path = os.path.join(domain_path, cname)
log.info('Unlinking orphan CNAME: %s', orphan_domain_path)
os.unlink(orphan_domain_path)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if these should go in another file. projects/tasks is already huge, so it might make sense to move this into core/tasks.py or somewhere else specifically for scheduled tasks?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I agree with this.

I created this issue (#3775) to refactor this and include some other tasks also --we can discuss in the issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually dislike putting things in core. Why not split up into:

  • readthedocs.projects.tasks.build
  • readthedocs.projects.tasks.sync
  • etc

Some of the tasks might make more sense in builds/ as well. I find core/ is not a helpful concept, save for only the most generalized of code.

@@ -240,6 +243,19 @@ def USE_PROMOS(self): # noqa
CELERY_CREATE_MISSING_QUEUES = True

CELERY_DEFAULT_QUEUE = 'celery'
CELERYBEAT_SCHEDULE = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to make sure we aren't overridding this elsewhere. We need a good pattern here, but I'm guessing we aren't extending this everywhere else we use it, so this might get killed by a downstream config file overwriting it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found we use this setting in 3 places:

@ericholscher please confirm this, and I will open a PR for corporate ops to modify this file https://github.com/readthedocs/readthedocs-corporate-ops/blob/master/salt/corporate/readthedocsinc/settings/base.py

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need a property override there as well, correct.

@humitos humitos mentioned this pull request Mar 12, 2018
8 tasks
@humitos humitos self-assigned this Mar 22, 2018
Copy link
Contributor

@agjohnson agjohnson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes here look good, I've noted some pieces for our next deploy as well.

Feel free to merge when tests come back

@agjohnson
Copy link
Contributor

Looks like just a linting error. We can address that separately. Merging!

@agjohnson agjohnson merged commit b242b41 into master Mar 23, 2018
@agjohnson agjohnson deleted the humitos/celerybeat/definetask branch March 23, 2018 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants