storage: automatic ballast files

[mwang1026]

With a change jackson pushed, we now have automatic ballast files without having to use the CLI flag. PR: cockroachdb/cockroach#66893

We should change our recommendation about how / when / whether to use a ballast file. e.g. in these docs https://www.cockroachlabs.com/docs/v21.1/recommended-production-settings.html#storage

We should also mark the old way of doing ballast files as deprecated (let's talk a bit more about that though) and guide users to use the new way of creating ballast files.

---

Update: below moved to separate issue #11917

Separately, we don't talk a lot about the OOD scenario -- why it's so nefarious (you need to write data to delete data, e.g.) and how to avoid it (read: alert alert alert, vertically scale if you have persistent disk like EBS, otherwise you have to horizontally scale before you run out of space / make space by removing ballast and log files but at that point you might not have enough space)

We have this doc, but we don't talk about the alerting you should set up about disk outage -- how a ballast might not be enough if you want to horizontally scale -- and generally best practices to avoid OOD situations. We could separate this into a separate issue.

https://www.cockroachlabs.com/docs/v21.1/operational-faqs.html#what-happens-when-a-node-runs-out-of-disk-space

[rmloveland]

Hey @mwang1026 it looks to me like this docs work is to cover the Jira epic CRDB-1452 - Is that correct?

[mwang1026]

that's right