Run analyze after changing post.url type (ref #4983)

[Nutomic]

No description provided.

[Nutomic]

Renamed the folder so that ci can test the migration. Have to rename it back once it passes.

[dessalines]

Just want to note, that I manually ran changing the column back to varchar(512) in prod, and things still stayed slow.

So even if we do this manually on the DB, we need to take a DB backup before deploying this again.

[dessalines]

It'd also be good to run

vacuum full analyze post;

based on @phiresky 's recomendations

[Nutomic]

Ive changed it to run ANALYZE post (url);, vacuum shouldnt be necessary as far as I understand. Should we reindex the column as well?

cc @phiresky @dullbananas

[dessalines]

I've verified that @Nutomic 's suggestion of running analyze post(url); fixes the issue, nice!

The culprit for some reason wasn't actually post/list, but the GetPost / post?id= endpoint, which was timing out. This means the crosspost query was likely timing out:

// Fetch the cross_posts
let cross_posts = if let Some(url) = &post_view.post.url {
  let mut x_posts = PostQuery {
    url_search: Some(url.inner().as_str().into()),
    local_user: local_user.as_ref(),
    ..Default::default()
  }
  .list(&local_site.site, &mut context.pool())
  .await?;

I did this by:

• Loading a production lemmy.ml DB into my dev machine.
• Checking out 0.19.5
• Running ./api_benchmark, on various endpoints. Everything fine.
• Checkout out release/v0.19.6-no-migrations (which includes the post url column increase)
• Running the benchmark again, and finding a timeout for post. Also just going to /post/ANY_POST in the browser timed out.
• Ran analyze post(url);
• Everything sped up and is accessible.
• The ./api_benchmark gave good results.

We should probably run analyze any time we make a column change like that, especially if its used for a join.

[dessalines]

@Nutomic the index type change probably isn't necessary, only running the analyze.

[phiresky]

nothing physical changes at all due to this migration, it just clears the statistics, so yes just analyzing the column should be enough.

it is interesting how devastating the effect of this missing statistics is in this case. I've never personally seen this effect, even on larger databases. Autovacuum also runs analyze, so this problem solves itself after some time, but I'm not sure how long that is on average:

The autovacuum daemon will automatically issue ANALYZE commands whenever the content of a table has changed sufficiently

So not sure if that starts counting as "everything changed" when stats are gone or "nothing changed". If the latter, it might take ages, otherwise analyze should happen on the first autovacuum trigger.

[dessalines]

Yep I've never seen anything like this before either, and I'm sure we're not the only ones to run into this issue.

I sent an email to postgres' bug tracker and linked this PR.

[Nutomic]

Renamed the migrations back so that we can merge it.