Fix common compilation issues by auto-adjusting ninja MAX_JOBS env var

[Qubitium]

Flash-attn takes a lot of ram and cpu cores to compile. Most users do not have 64 cores or over 96GB of ram and hand adjusting MAX_JOBS is trial-and-error and not to mention a waste of dev time.

Problem this PR attempts to solve:

1. Ninja MAX_JOBS is unaware of threads=4 spawned by nvcc.
2. Ninja MAX_JOBS is unaware of actual available memory in environment
3. Both 1 and 2 combined results in oom, heavy swap usage, oversubscribed threads to core count (thread starvation) and general reduced efficiency in compiling since ninja is not resource aware.

Without this PR, python setup install will OOM on an Intel 14700k consumer machine with 96GB of ram (swap disabled). Using readme recommended MAX_JOBS=4 will lead to very slow compilation. Both options are sub-optimal. This led to the reason I created this PR. The problem becomes worse for consumer machines that generally have 32GB of ram especially for laptops. Overall, the chance of you over-waiting for flash-attention to compile due to under-utilization or flat-out oom is extremely high for all environments.

PR tries to solve most of the headaches and auto-adjusts ninja MAX_JOBS based on both cpu core count and available memory so that it will compile near max efficiency under both consumer and server environments. There is no longer a need to manually adjust the MAX_JOBS value.

Base logic is the code calculate max efficient MAX_JOBS based on 2 metrics: 1. cpu cores and 2. available memory and then take the min() value of the two. Cores are real-cores / 2 since threads=4 and Memory is divided by a constant of 9GB since I observe each job uses from 8-9GB of ram during peak.  

Test Env:

Ubuntu 22.04
Torch 2.2
Intel 14700k + 96GB DDR5 6600 + swap disabled (to reproduce the worst possible oom situations). 

[tridao]

This is great, thanks so much @Qubitium !