Enable float8 CI on sm89

[jainapurva]

The current CI/CD pipeline skipped float8 tests as they weren't compatible with the A10G GPUs. This PR adds a new CI job which runs on NVIDIA L4 Tensor Core GPUs for float8 tests.

Fixes Issue: #575

Test Plan : Compare the logs of Regression Test and Float8 Test to check for the skipped tests in Regression Test which are now being run on the new CI job of Float8 Test.

Sample of some test cases which were being skipped in CUDA Nightly in Regression Tests are running in Float8.

• test/float8/test_numerics_integration.py::TestFloat8NumericsIntegrationTest::test_encoder_fw_bw
• test/float8/test_inference_flows.py::TestHPTrainToFP8LinearInference::test_dynamic_fp8_mlp
• test/float8/test_inference_flows.py::TestHPTrainToFP8LinearInference::test_static_fp8_mlp

Screenshot of logs of CUDA Nightly

Screenshot of logs of Float8 test

For full list of passed tests: https://gist.github.com/jainapurva/a5ddd17c809219151485de8d1708d078

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/587

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 01f0ba1 with merge base 013cce3 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[msaroufim]

So question here

1. Are these g6 machines cheaper than g5? Cause it sounds like count goes from 12 to 4. If it is cheaper we cna just use this machine by default instead and you won't need an extra job
2. Also if the plan is launch an extra job for fp8 then the new job should only run fp8 tests, itll make your tests run a lot faster

EDIT: Ok that's a lot of test failures, mind opening up a tracker with them so we can assign them to right owners

1. Sparsity issues will be fixed by @jcaip
2. Affine quantized tensor by @jerryzh168

[msaroufim]

I noticed the fp8 test is not being triggered here so instead what I did was

1. Copy pasted your updated regression test file
2. Deleted all strategies except the one that needs to run the g6 machines
3. Updated the test to only run fp8

So you can go ahead and make the updates here from this PR #603 I won't land 603

For Github Actions because you can't iterate fast because you need to wait on CI, would recommend copy pasting as much so you minimize surface area of missing tpos as possible or running tiny experiments where you in your branch temporarily delete all jobs except the one you're interested in iterating on. It makes iterating far more pleasant

Also regarding pricing if we compare g5.12x prices vs g6.4x prices https://aws.amazon.com/ec2/instance-types/g6/ and https://aws.amazon.com/ec2/instance-types/g5/

• g5.12x: $5.672 per hour
• g6.4x: $1.323 per hour

So you could reduce our CI costs by about 4x just changing the default machine type but we can do this in a future PR

fat finger