Revert "fixes to properly shard FSDP across cpu and meta for cpu_effcient_loading for prequantized 4bit (#32276)"

[matthewdouglas]

What does this PR do?

This reverts commit 62c60a3 (PR #32276)

We uncovered an issue with this change that caused our training runs to hang. It works with the PR reverted. I'll create a separate issue to continue work on addressing the CPU memory usage with FSDP.

See internal discussion

Trainable params: 627,572,736 || all params: 410,708,819,968 || trainable%: 0.1528
[rank0]:[E ProcessGroupNCCL.cpp:1316] [PG 0 Rank 0] Heartbeat monitor timed out! Process will be terminated after dumping debug info. workMetaList_.size()=8
[rank0]:[E ProcessGroupNCCL.cpp:1153] [PG 0 Rank 0] ProcessGroupNCCL preparing to dump debug info.
[rank0]:[F ProcessGroupNCCL.cpp:1169] [PG 0 Rank 0] [PG 0 Rank 0] ProcessGroupNCCL's watchdog got stuck for 600 seconds without making progress in monitoring enqueued collectives. This typically indicates a NCCL/CUDA API hang blocking the watchdog, and could be triggered by another thread holding the GIL inside a CUDA api, or other deadlock-prone behaviors.If you suspect the watchdog is not actually stuck and a longer timeout would help, you can either increase the timeout (TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC) to a larger value or disable the heartbeat monitor (TORCH_NCCL_ENABLE_MONITORING=0).If either of aforementioned helps, feel free to file an issue to PyTorch about the short timeout or false positive abort; otherwise, please attempt to debug the hang. workMetaList_.size() = 8
W0802 08:05:08.490000 140537132394304 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1038311 closing signal SIGTERM
W0802 08:05:08.495000 140537132394304 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1038312 closing signal SIGTERM
W0802 08:05:08.496000 140537132394304 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1038313 closing signal SIGTERM
W0802 08:05:08.496000 140537132394304 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1038314 closing signal SIGTERM
W0802 08:05:08.496000 140537132394304 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1038315 closing signal SIGTERM
W0802 08:05:08.497000 140537132394304 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1038316 closing signal SIGTERM
W0802 08:05:08.498000 140537132394304 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 1038317 closing signal SIGTERM
E0802 08:05:09.741000 140537132394304 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: -6) local_rank: 0 (pid: 1038310)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@ArthurZucker @muellerzr

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[ArthurZucker]

Thanks, cc @winglian I can patch if you find a solution later on!

[muellerzr]

Thanks a bunch!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.