Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve CloudFlare cache clear logging and GitHub Stars bugfix #1587

Merged
merged 7 commits into from
Jul 19, 2020

Conversation

rogermparent
Copy link
Contributor

@rogermparent rogermparent commented Jul 17, 2020

This PR was made to help diagnose the CloudFlare cache purge issue we've been having, though Review Apps don't actually have CloudFlare credentials.

The changes of this PR move the CloudFlare cache cleaning logic into a regular Node module instead of a CLI-invoked script. This module is then used at the end of the deploy script. Effectively, this moves the CloudFlare cache clean to "the end of the deploy phase" as opposed to "the start of the runtime process".

As I understand it, this effectively doesn't change anything except for logs. The log changes, however, mean that the CloudFlare script outputs in the build logs instead of the runtime logs- this is good because runtime logs expire without add-ons, but build-time logs last forever by default.

You can see in the Deploy Preview for this PR that the cache clear module is being run, as it outputs the line Skipping CloudFlare cache purge because CONTEXT is not set to production.. To actually test it out, I just need to manually add keys to the deploy app's env vars. I'll do another update when I do.

Update: When testing with the keys in prod on the Heroku review app, this error appears:

Error: CloudFlare cache clear failed! {
  "status": 403,
  "errors": [
    {
      "code": 10000,
      "message": "Authentication error"
    }
  ]
}

When testing with a new token on a Review App, the CloudFlare cache is successfully cleared. This could either be because the token is invalid or deploy apps fail an auth check on the old key that prod passes. Either way, I think it's worth trying this new log behavior and a new key on the production Heroku instance.

Update: While working on this issue, I ran into a bug related to GitHub stars that would break our cache re-use occasionally and force a full rebuild- the fix was to turn the static stars GraphQL endpoint into a simple resolver instead of a true Node, solving the issue that necessitated the separate node as well as the cache issue the first iteration of the fix introduced.

@rogermparent rogermparent marked this pull request as draft July 17, 2020 20:03
@shcheklein shcheklein temporarily deployed to dvc-landing-fix-cloudfl-gb7wrv July 17, 2020 20:03 Inactive
@rogermparent rogermparent temporarily deployed to dvc-landing-fix-cloudfl-gb7wrv July 17, 2020 21:04 Inactive
Since Heroku's runtime logs are much trickier to track down than dyno runtime
logs, this change will allow us to better see CloudFlare cache clearing
feedback.

The script in `clear-cloudflare-cache` is changed from a shell-invoked style to
a Node module that's imported into `deploy-with-s3`.
@rogermparent rogermparent had a problem deploying to dvc-landing-fix-cloudfl-gb7wrv July 17, 2020 23:07 Failure
@rogermparent rogermparent temporarily deployed to dvc-landing-fix-cloudfl-gb7wrv July 17, 2020 23:19 Inactive
@rogermparent rogermparent changed the title Fix CloudFlare cache clearing Improve CloudFlare cache logging Jul 17, 2020
@rogermparent rogermparent changed the title Improve CloudFlare cache logging Improve CloudFlare cache clear logging Jul 17, 2020
@rogermparent rogermparent had a problem deploying to dvc-landing-fix-cloudfl-gb7wrv July 18, 2020 03:17 Failure
@rogermparent rogermparent had a problem deploying to dvc-landing-fix-cloudfl-gb7wrv July 18, 2020 03:29 Failure
This replaces the Node with a Resolver, so the in-between layer is thinner and
not subject to stale cache pruning as it is only a middleman.

This also adds `createResolvers` to Gatsby APIs exposed to Models
@rogermparent rogermparent temporarily deployed to dvc-landing-fix-cloudfl-gb7wrv July 18, 2020 04:20 Inactive
@rogermparent rogermparent mentioned this pull request Jul 18, 2020
@shcheklein
Copy link
Member

do we want to merge this?

@rogermparent rogermparent marked this pull request as ready for review July 18, 2020 20:42
@rogermparent rogermparent marked this pull request as draft July 18, 2020 20:54
@rogermparent rogermparent had a problem deploying to dvc-landing-fix-cloudfl-gb7wrv July 18, 2020 21:03 Failure
@rogermparent rogermparent temporarily deployed to dvc-landing-fix-cloudfl-gb7wrv July 18, 2020 21:09 Inactive
@rogermparent rogermparent marked this pull request as ready for review July 18, 2020 21:14
@rogermparent
Copy link
Contributor Author

I believe we do want to merge this, the primary change will mean any future CloudFlare cache cleaner problems will be much more visible (i.e. if the new token I make ever expires for some reason)
On top of that, I managed to pin down and fix a hard-to-replicate bug while working with this PR on Heroku that would occasionally break Gatsby cache and trigger a full rebuild.

@rogermparent rogermparent changed the title Improve CloudFlare cache clear logging Improve CloudFlare cache clear logging and GitHub Stars bugfix Jul 18, 2020
@shcheklein shcheklein merged commit a7f4d15 into master Jul 19, 2020
shcheklein added a commit that referenced this pull request Jul 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants