Stripe integration

[micahflee]

I'm closing the #534 PR and using this one instead. I've also merged #593 into it. This will address #364.

How to test this

Install and configure stripe-cli, and then run this command to forward webhook events. The output will show you a webhook secret:

stripe listen --forward-to localhost:8080/premium/webhook

Copy .env.stripe-sample into .env-stripe and edit it to set STRIPE_SECRET_KEY (you can find this in the Stripe dashboard at https://dashboard.stripe.com/test/apikeys) and STRIPE_WEBHOOK_SECRET (the stripe-cli command showed you that).

Then, run the docker-compose.stripe.yaml containers:

docker compose -f docker-compose.stripe.yaml up --build

Things to do

• Update infra repo to launch a separate worker container, for Stripe webhooks

Done

• When logging in, if user.tier_id is None, prompt the user to upgrade
• Write tests!
• Switch to Stripe Checkout to avoid having any Stripe JS
• When a user downgrades, let them have the rest of their subscription term before downgrading (the subscription object includes fields current_period_end and current_period_start, so it should be simple enough)
• Update create_products_and_prices to intelligently make sure the db and Stripe are synced up, without creating lots of extra dev products
• Add "upgrade" to nav for logged in free users, and add "manage account" to advanced settings for logged in premium users

[github-actions]

Terraform plan in terraform/dev in the hushline-dev-paid-features2 workspace

With variables

branch = "paid-features2"
name   = "dev-paid-features2"

Plan: 3 to add, 0 to change, 0 to destroy. Changes to Outputs.

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+   create

Terraform will perform the following actions:

  # digitalocean_project.hush_line_dev will be created
+   resource "digitalocean_project" "hush_line_dev" {
+       created_at  = (known after apply)
+       description = "Development instance based on the paid-features2 branch"
+       environment = "Development"
+       id          = (known after apply)
+       is_default  = false
+       name        = "dev-paid-features2"
+       owner_id    = (known after apply)
+       owner_uuid  = (known after apply)
+       purpose     = "Web Application"
+       resources   = (known after apply)
+       updated_at  = (known after apply)
    }

  # module.app.digitalocean_app.app will be created
+   resource "digitalocean_app" "app" {
+       active_deployment_id = (known after apply)
+       created_at           = (known after apply)
+       default_ingress      = (known after apply)
+       id                   = (known after apply)
+       live_url             = (known after apply)
+       project_id           = (known after apply)
+       updated_at           = (known after apply)
+       urn                  = (known after apply)

+       dedicated_ips (known after apply)

+       spec {
+           domains  = (known after apply)
+           features = [
+               "buildpack-stack=ubuntu-22",
            ]
+           name     = "dev-paid-features2"
+           region   = "sfo"

+           alert {
+               disabled = false
+               rule     = "DEPLOYMENT_FAILED"
            }

+           domain (known after apply)

+           ingress (known after apply)

+           service {
+               dockerfile_path    = "Dockerfile"
+               http_port          = 8080
+               instance_count     = 1
+               instance_size_slug = "apps-s-1vcpu-0.5gb"
+               internal_ports     = (known after apply)
+               name               = "app"
+               run_command        = (known after apply)

+               github {
+                   branch         = "paid-features2"
+                   deploy_on_push = true
+                   repo           = "scidsg/hushline"
                }

+               health_check {
+                   http_path = "/health.json"
                }

+               routes (known after apply)
            }
+           service {
+               http_port          = (known after apply)
+               instance_count     = 1
+               instance_size_slug = "apps-s-1vcpu-0.5gb"
+               internal_ports     = [
+                   5432,
                ]
+               name               = "db"
+               run_command        = (known after apply)

+               image {
+                   registry      = "library"
+                   registry_type = "DOCKER_HUB"
+                   repository    = "postgres"
+                   tag           = "16.4-alpine3.20"

+                   deploy_on_push (known after apply)
                }

+               routes (known after apply)
            }
        }
    }

  # module.app.random_password.local_db_password will be created
+   resource "random_password" "local_db_password" {
+       bcrypt_hash      = (sensitive value)
+       id               = (known after apply)
+       length           = 16
+       lower            = true
+       min_lower        = 0
+       min_numeric      = 0
+       min_special      = 0
+       min_upper        = 0
+       number           = true
+       numeric          = true
+       override_special = "!#$%&*()-_=+[]{}<>:?"
+       result           = (sensitive value)
+       special          = true
+       upper            = true
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Changes to Outputs:
+   app_live_url = (known after apply)

✅ Plan applied in Deploy/Destroy Branch Dev Environment #282

Outputs

app_live_url = "https://dev-paid-features2-oltb6.ondigitalocean.app"

[github-actions]

Terraform plan in terraform/dev in the hushline-dev-paid-features2 workspace

With variables

branch = "paid-features2"
name   = "dev-paid-features2"

Plan: 3 to add, 0 to change, 0 to destroy. Changes to Outputs.

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+   create

Terraform will perform the following actions:

  # digitalocean_project.hush_line_dev will be created
+   resource "digitalocean_project" "hush_line_dev" {
+       created_at  = (known after apply)
+       description = "Development instance based on the paid-features2 branch"
+       environment = "Development"
+       id          = (known after apply)
+       is_default  = false
+       name        = "dev-paid-features2"
+       owner_id    = (known after apply)
+       owner_uuid  = (known after apply)
+       purpose     = "Web Application"
+       resources   = (known after apply)
+       updated_at  = (known after apply)
    }

  # module.app.digitalocean_app.app will be created
+   resource "digitalocean_app" "app" {
+       active_deployment_id = (known after apply)
+       created_at           = (known after apply)
+       default_ingress      = (known after apply)
+       id                   = (known after apply)
+       live_url             = (known after apply)
+       project_id           = (known after apply)
+       updated_at           = (known after apply)
+       urn                  = (known after apply)

+       dedicated_ips (known after apply)

+       spec {
+           domains  = (known after apply)
+           features = [
+               "buildpack-stack=ubuntu-22",
            ]
+           name     = "dev-paid-features2"
+           region   = "sfo"

+           alert {
+               disabled = false
+               rule     = "DEPLOYMENT_FAILED"
            }

+           domain (known after apply)

+           ingress (known after apply)

+           service {
+               dockerfile_path    = "Dockerfile"
+               http_port          = 8080
+               instance_count     = 1
+               instance_size_slug = "apps-s-1vcpu-0.5gb"
+               internal_ports     = (known after apply)
+               name               = "app"
+               run_command        = (known after apply)

+               github {
+                   branch         = "paid-features2"
+                   deploy_on_push = true
+                   repo           = "scidsg/hushline"
                }

+               health_check {
+                   http_path = "/health.json"
                }

+               routes (known after apply)
            }
+           service {
+               http_port          = (known after apply)
+               instance_count     = 1
+               instance_size_slug = "apps-s-1vcpu-0.5gb"
+               internal_ports     = [
+                   5432,
                ]
+               name               = "db"
+               run_command        = (known after apply)

+               image {
+                   registry      = "library"
+                   registry_type = "DOCKER_HUB"
+                   repository    = "postgres"
+                   tag           = "16.4-alpine3.20"

+                   deploy_on_push (known after apply)
                }

+               routes (known after apply)
            }
        }
    }

  # module.app.random_password.local_db_password will be created
+   resource "random_password" "local_db_password" {
+       bcrypt_hash      = (sensitive value)
+       id               = (known after apply)
+       length           = 16
+       lower            = true
+       min_lower        = 0
+       min_numeric      = 0
+       min_special      = 0
+       min_upper        = 0
+       number           = true
+       numeric          = true
+       override_special = "!#$%&*()-_=+[]{}<>:?"
+       result           = (sensitive value)
+       special          = true
+       upper            = true
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Changes to Outputs:
+   app_live_url = (known after apply)

❌ Error applying plan in Deploy/Destroy Branch Dev Environment #281

[github-actions]

🚀 App successfully deployed to https://dev-paid-features2-oltb6.ondigitalocean.app!

[micahflee]

EDIT: moving the to do list to the PR description

[gitguardian]

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
Once a secret has been leaked into a git repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.

---

^{_{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}}

[micahflee]

This is just about ready for a final review, as well as for some design work! Let me show how it all works.

When new users register accounts, the first time they login they are prompted to select a tier:

This displays the premium-select-tier.html template, which includes subtemplates for the free and business feature lists that are shared with the premium.html template now. Also, in premium/business-features.html, where it says "Monthly Price: $20", it actually pulls the $20 from whatever we set the price to, it's no longer hard-coded. So if we change the price, it will change there too.

Once you select a plan (I selected free) it no longer displays that on login.

If you're logged into the free tier, there's now an "Upgrade" button in the nav bar. You may want to change how it looks:

Clicking it brings you to /premium/. It's the same as the link in the Profile page of Settings:

Here's the premium index page:

I've switched it to use Stripe Checkout. If you click "Upgrade Now", it now creates a new checkout session and redirects to a Stripe-hosted page to collect payment info:

After filling out the payment info, it briefly shows a waiting page (premium-waiting.html, though it went by too quickly to get a screenshot) and then loads the premium page again:

This time, now that the user is signed in and using a business account, it no longer has the Upgrade button in the nav bar. It also displays a table of invoices at the bottom (this part needs design work). And instead of just canceling the subscription, there's a whole workflow for disabling and re-enabling auto-renew. Here's a short video showing it:

Screencast.from.2024-10-01.10-21-14.webm

The buttons need some design work too, but it works!

If you're on a business plan, and you load the profile page of Settings, it should no longer show the Upgrade button there:

And if you go to the Advanced tab, it should now have a link to manage your plan (this definitely needs some design work):

That just links back to the premium page, where you can choose to not renew/cancel your subscription, or access your invoices.

===

All of these premium features should only happen if the STRIPE_SECRET_KEY environment variable is set. It injects is_premium_enabled into the templates in hushline/__init__.py:

@app.context_processor
def inject_is_premium_enabled() -> dict[str, Any]:
    return {"is_premium_enabled": bool(app.config.get("STRIPE_SECRET_KEY", False))}

So an important thing to check for while reviewing is that if premium mode is disabled (no STRIPE_SECRET_KEY), that none of these changes appear.

This PR also adds a Tier model with two tiers, free and business. It adds a bunch of new fields to the User model:

• tier_id
• stripe_customer_id
• stripe_subscription_id
• stripe_subscription_cancel_at_period_end
• stripe_subscription_status
• stripe_subscription_current_period_end
• stripe_subscription_current_period_start

And it adds models for StripeInvoice and StripeEvent.

===

Stripe webhooks work like this:

The webhook URL is /premium/webhook, so in prod we will need to login to the Stripe dashboard and configure a webhook to use the URL https://tips.hushline.app/premium/webhook. This route is configured to verify the signature on the webhook event (to make sure it came from Stripe and not randos making POST requests to that URL), and then insert the event into the database (with a "pending" status) and return 200.

The premium features will require an extra worker container, so before we can deploy this to prod, there needs to be some infra work to spin up an extra container. The worker container is just the app contain but it starts with this command: poetry run flask stripe start-worker

The worker container runs an infinite loop looking for pending Stripe events and processing them. Right now it only handles the following events (which are all that's needed for the functionality we're using):

• customer.subscription.created
• customer.subscription.updated
• customer.subscription.deleted
• invoice.created
• invoice.updated

So when creating the webhook in the Stripe dashboard, we can limit it to these events.

[brassy-endomorph]

Went through the manual test plan. Everything works as advertised. I have some open remarks that might be good to address, but those can be in follows to. Approved.