pip install flash-attn always happens ModuleNotFoundError: No module named 'packaging',but actually i have pip install packaging #453

sky2future · 2023-08-16T00:54:41Z

Collecting flash-attn
Using cached flash_attn-2.0.7.tar.gz (2.2 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [20 lines of output]
Traceback (most recent call last):
File "C:\Users\24259\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in
main()
File "C:\Users\24259\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\24259\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\24259\AppData\Local\Temp\pip-build-env-t80nwpmz\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\24259\AppData\Local\Temp\pip-build-env-t80nwpmz\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires
self.run_setup()
File "C:\Users\24259\AppData\Local\Temp\pip-build-env-t80nwpmz\overlay\Lib\site-packages\setuptools\build_meta.py", line 507, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "C:\Users\24259\AppData\Local\Temp\pip-build-env-t80nwpmz\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
exec(code, locals())
File "", line 8, in
ModuleNotFoundError: No module named 'packaging'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
PS C:\d\code_program\python\ai> pip install flash-attn
Collecting flash-attn
Using cached flash_attn-2.0.7.tar.gz (2.2 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [20 lines of output]
Traceback (most recent call last):
File "C:\Users\24259\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in
main()
File "C:\Users\24259\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\24259\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\24259\AppData\Local\Temp\pip-build-env-wdj0xi3r\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\24259\AppData\Local\Temp\pip-build-env-wdj0xi3r\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires
self.run_setup()
File "C:\Users\24259\AppData\Local\Temp\pip-build-env-wdj0xi3r\overlay\Lib\site-packages\setuptools\build_meta.py", line 507, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "C:\Users\24259\AppData\Local\Temp\pip-build-env-wdj0xi3r\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
exec(code, locals())
File "", line 8, in
ModuleNotFoundError: No module named 'packaging'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

tmm1 · 2023-08-17T13:43:25Z

Try this:

pip install -U wheel

tmm1 · 2023-08-17T17:08:23Z

Try this:

pip install -U wheel

someone reported this worked in their environment.

but when we tried in a fresh docker/conda env, its not working. nor is installing packaging.

see https://github.com/OpenAccess-AI-Collective/axolotl/pull/426/files for the dockerfile

Lemaqwq · 2023-08-22T17:36:49Z

Try this:

pip install -U wheel

someone reported this worked in their environment.

but when we tried in a fresh docker/conda env, its not working. nor is installing packaging.

see https://github.com/OpenAccess-AI-Collective/axolotl/pull/426/files for the dockerfile

Same question here. And it is not working:
(plaid) PS D:\Users\12625\PycharmProjects\plaid-main\plaid-main> pip install -U wheel
Requirement already satisfied: wheel in c:\users\12625\anaconda3\envs\plaid\lib\site-packages (0.38.4)
Collecting wheel
Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
Using cached wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
Installing collected packages: wheel
Attempting uninstall: wheel
Found existing installation: wheel 0.38.4
Uninstalling wheel-0.38.4:
Successfully uninstalled wheel-0.38.4
Successfully installed wheel-0.41.2
(plaid) PS D:\Users\12625\PycharmProjects\plaid-main\plaid-main> pip install flash-attn
Collecting flash-attn
Using cached flash_attn-2.0.9.tar.gz (2.2 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [17 lines of output]
Traceback (most recent call last):
File "C:\Users\12625\anaconda3\envs\plaid\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in
main()
File "C:\Users\12625\anaconda3\envs\plaid\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\Users\12625\anaconda3\envs\plaid\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "C:\Users\12625\AppData\Local\Temp\pip-build-env-lwydw453\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "C:\Users\12625\AppData\Local\Temp\pip-build-env-lwydw453\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires
self.run_setup()
File "C:\Users\12625\AppData\Local\Temp\pip-build-env-lwydw453\overlay\Lib\site-packages\setuptools\build_meta.py", line 507, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "C:\Users\12625\AppData\Local\Temp\pip-build-env-lwydw453\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
exec(code, locals())
File "", line 8, in
ModuleNotFoundError: No module named 'packaging'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

yunzhongyan0 · 2023-08-25T02:43:35Z

Try this:

pip install -U wheel

someone reported this worked in their environment.
but when we tried in a fresh docker/conda env, its not working. nor is installing packaging.
see https://github.com/OpenAccess-AI-Collective/axolotl/pull/426/files for the dockerfile

Same question here. And it is not working: (plaid) PS D:\Users\12625\PycharmProjects\plaid-main\plaid-main> pip install -U wheel Requirement already satisfied: wheel in c:\users\12625\anaconda3\envs\plaid\lib\site-packages (0.38.4) Collecting wheel Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata Using cached wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB) Using cached wheel-0.41.2-py3-none-any.whl (64 kB) Installing collected packages: wheel Attempting uninstall: wheel Found existing installation: wheel 0.38.4 Uninstalling wheel-0.38.4: Successfully uninstalled wheel-0.38.4 Successfully installed wheel-0.41.2 (plaid) PS D:\Users\12625\PycharmProjects\plaid-main\plaid-main> pip install flash-attn Collecting flash-attn Using cached flash_attn-2.0.9.tar.gz (2.2 MB) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [17 lines of output] Traceback (most recent call last): File "C:\Users\12625\anaconda3\envs\plaid\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in main() File "C:\Users\12625\anaconda3\envs\plaid\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "C:\Users\12625\anaconda3\envs\plaid\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) File "C:\Users\12625\AppData\Local\Temp\pip-build-env-lwydw453\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=['wheel']) File "C:\Users\12625\AppData\Local\Temp\pip-build-env-lwydw453\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires self.run_setup() File "C:\Users\12625\AppData\Local\Temp\pip-build-env-lwydw453\overlay\Lib\site-packages\setuptools\build_meta.py", line 507, in run_setup super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script) File "C:\Users\12625\AppData\Local\Temp\pip-build-env-lwydw453\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup exec(code, locals()) File "", line 8, in ModuleNotFoundError: No module named 'packaging' [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

have you sovled it ?

tmm1 · 2023-08-25T02:46:15Z

Use:

pip install -U flash-attn --no-build-isolation

kig1929 · 2023-08-25T07:00:38Z

I experienced the same issue and it seems to be the issue with the order of installation.

In my case, I removed flash-attn from requirements.txt and ran pip install -r requirements.txt.
After installation of the other packages, then ran pip install flash-attn --no-build-isolation.

tmm1 · 2023-08-25T15:52:48Z

The issue here is that once you add a pyproject.toml, pip will use that and use build isolation.

To make isolation work, we would need to add to the toml:

[build-system]
requires = [
    "setuptools",
    "packaging",
    "wheel",
    "torch",
]

However that can be annoying too since it will take longer to install torch in an isolated environment, esp when it's just downloading the binary wheels anyway..

I wonder if there's a way to ignore the pyproject.toml's presence when generating the packages, so they don't try to use build isolation.

tmm1 · 2023-08-25T22:22:32Z

fixed by 73bd3f3

Andcircle · 2023-08-28T07:40:20Z

Hey @tmm1, sorry to bother

Still facing the same issue:

MAX_JOBS=4 pip install -U flash-attn --no-build-isolation
Collecting flash-attn
  Using cached flash_attn-2.1.0.tar.gz (2.2 MB)
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 380, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 487, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 8, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.

tmm1 · 2023-08-28T14:04:59Z

The fix is out. Try:

pip install -v flash-attn==2.1.1

Andcircle · 2023-09-02T04:22:47Z

@tmm1 thanks, it works =)

tmm1 · 2023-09-02T13:17:39Z

@tridao this issue can be closed. If you want to give me issue maint privileges, I can help out keeping things tidy.

Bec-k · 2023-09-05T14:49:20Z

Same issue for me in v2.1.2.post3

Bec-k · 2023-09-05T14:49:56Z

pip install -v flash-attn==2.1.1 doesn't help either

Bec-k · 2023-09-05T14:50:37Z

I'm trying pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary and getting this error:

Collecting git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary
  Cloning https://github.com/HazyResearch/flash-attention.git to /tmp/pip-req-build-fmhz3e3e
  Running command git clone --filter=blob:none --quiet https://github.com/HazyResearch/flash-attention.git /tmp/pip-req-build-fmhz3e3e
  Resolved https://github.com/HazyResearch/flash-attention.git to commit 913922cac57efd7c5e05f08155b37e74c427cf32
  Running command git submodule update --init --recursive -q
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/home/beck/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/beck/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/beck/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-jskeyd_r/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-jskeyd_r/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-jskeyd_r/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 507, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-jskeyd_r/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 5, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

tridao · 2023-09-05T17:55:32Z

pip install -v flash-attn==2.1.1 doesn't help either

Did you try --no-build-isolation?

adeepak7 · 2023-11-10T07:21:24Z

pip install -v flash-attn==2.1.1 doesn't help either

Did you try --no-build-isolation?

Not working either

frankxyy · 2023-11-21T12:59:56Z

hi，is there a solution now?

tridao · 2023-11-21T18:43:26Z

pip install packaging
then
pip install flash-attn --no-build-isolation.

blakejrobinson · 2023-11-28T02:04:35Z

pip install packaging then pip install flash-attn --no-build-isolation.

No joy on this. it was missing wheel. Installing wheel package then just errored.

JeffreyWardman · 2024-01-02T01:21:10Z

This still seems to be an ongoing issue when adding to poetry.lock.

  ModuleNotFoundError: No module named 'packaging'
  

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/chef.py:147 in _prepare
      143│ 
      144│                 error = ChefBuildError("\n\n".join(message_parts))
      145│ 
      146│             if error is not None:
    → 147│                 raise error from None
      148│ 
      149│             return path
      150│ 
      151│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with flash-attn (2.4.2) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517 "flash-attn (==2.4.2)"'.

Pip installing doesn't include freezing the dependencies. There's no guarantee of matching dependencies in a production environment with the current solution (that provided by @tridao - [thank you for sharing your solution]).

tridao · 2024-01-02T01:29:51Z

Yeah I don't have much bandwidth to spend on packaging, python packaging is kind of a mess once you involve CUDA and torch.
I just use docker for reproducible environment.

BBerabi · 2024-01-03T17:03:00Z

The issue is still relevant when you want to use a requirement.txt file combined with a build system. Can we add packaging and ninja properly to the dependencies of flash-attn so that pip can manage the dependencies. This would significantly simplify the installation process.

lidingsnyk · 2024-02-01T14:49:24Z

Indeed, --no-build-isolation isn't allowed in requirement.txt in the most recent version of pip (23)

lidingsnyk · 2024-02-07T13:07:04Z

requirement.txt user will have to manually pick a wheel url instead of declaring flash-attn==2.5.2

e.g. In my requirement.txt:

https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.2/flash_attn-2.5.2+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

Despite being manual, this is more flexible since setup.py picks the wheel based on CUDA, torch version of the machine where the build happens. IRL, you might build and packaged in build farm machines that doesn't have the same CUDA version as your worker machines.

michaelze · 2024-04-12T11:13:38Z

I'm definitely not an expert on this topic. But couldn't this problem be solved by publishing the .whl files to pypi.org?
Compare e.g. vllms index (https://pypi.org/simple/vllm/) with the index of flash-attention (https://pypi.org/simple/flash-attn/).
The former contains .whl files, the latter does not...

tridao · 2024-04-12T18:54:06Z

PyPI has a file size limit of 60MB.

fernandorodriguez-aily · 2024-05-17T16:04:10Z

Has the error with poetry.lock been fixed? I am still facing issues when trying to install flash_attn with Poetry:

~/GitHub/test-vllm$ poetry add flash_attn

Using version ^2.5.8 for flash-attn

Updating dependencies
Resolving dependencies... (0.8s)

Package operations: 1 install, 0 updates, 0 removals

 - Installing flash-attn (2.5.8): Failed

 ChefBuildError

 Backend subprocess exited when trying to invoke get_requires_for_build_wheel
  
 Traceback (most recent call last):
  File "/home/ubuntu/.local/share/pipx/venvs/poetry/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
   main()
  File "/home/ubuntu/.local/share/pipx/venvs/poetry/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
   json_out["return_val"] = hook(**hook_input["kwargs"])
  File "/home/ubuntu/.local/share/pipx/venvs/poetry/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 134, in get_requires_for_build_wheel
   return hook(config_settings)
  File "/tmp/tmpdgvzy9wj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
   return self._get_build_requires(config_settings, requirements=['wheel'])
  File "/tmp/tmpdgvzy9wj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
   self.run_setup()
  File "/tmp/tmpdgvzy9wj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
   super().run_setup(setup_script=setup_script)
  File "/tmp/tmpdgvzy9wj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
   exec(code, locals())
  File "<string>", line 9, in <module>
 ModuleNotFoundError: No module named 'packaging'
  

 at ~/.local/share/pipx/venvs/poetry/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare
   160│ 
   161│         error = ChefBuildError("\n\n".join(message_parts))
   162│ 
   163│       if error is not None:
  → 164│         raise error from None
   165│ 
   166│       return path
   167│ 
   168│   def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with flash-attn (2.5.8) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "flash-attn (==2.5.8)"'.

mirekphd · 2024-05-24T15:32:13Z

The solution (required for older torch versions, e.g. 2.0.1) was to correct the obsolete way of importing packaging module from pkg_resources (distributed with setuptools and not to be confused with the packaging Python package) to support newer major versions of setuptools (70.0.0 and above). So upgrading to the latest torch is the preferred solution now.

A quick and dirty workaround allowing us to install flash-attn without risking torch upgrades is to downgrade setuptools:

pip install setuptools"<70.0.0" && pip install flash-attn #  --no-build-isolation is not relevant to this issue

More info (from build logs):

Collecting flash-attn
  Downloading flash_attn-2.5.8.tar.gz (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 101.1 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [9 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-fwyobwo6/flash-attn_86cb3e09b5d749c792dbc302c1822f5f/setup.py", line 20, in <module>
          from torch.utils.cpp_extension import (
        File "/opt/conda/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 25, in <module>
          from pkg_resources import packaging  # type: ignore[attr-defined]
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      ImportError: cannot import name 'packaging' from 'pkg_resources' (/opt/conda/lib/python3.11/site-packages/pkg_resources/__init__.py)
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

…tion

sangshuduo · 2024-10-02T22:32:45Z

pip3 install -U pip
pip3 install packaging

above commands work to me

r0b8 mentioned this issue Aug 25, 2023

ModuleNotFoundError: No module named 'packaging' lm-sys/FastChat#2245

Closed

This was referenced Aug 25, 2023

add build-system.requirements to pyproject.toml #493

Closed

remove pyproject.toml to simplify installation #495

Closed

subercui mentioned this issue Aug 31, 2023

Error during install in HPC bowang-lab/scGPT#74

Closed

tranlm mentioned this issue Jan 10, 2024

Unable to install flash-attn due to SSL error #756

Closed

orxai mentioned this issue Jan 30, 2024

No module named 'packaging' ducdauge/sft-llm#3

Closed

This was referenced Feb 19, 2024

Can't be installed using "uv" due to an issue in the setup script #833

Open

Support "--no-build-isolation" when installing packages astral-sh/uv#1715

Closed

WoosukKwon mentioned this issue Mar 5, 2024

Separate attention backends vllm-project/vllm#3005

Merged

Sid-Data-Universe mentioned this issue Mar 24, 2024

Support for larger models at a future block. macrocosm-os/pretraining#83

Merged

weiji14 mentioned this issue May 5, 2024

Add flash-attn conda-forge/staged-recipes#26239

Merged

10 tasks

dhellmann mentioned this issue May 17, 2024

add pyproject.toml with build dependencies #958

Open

Narsil mentioned this issue Jun 4, 2024

Making make install work better by default. huggingface/text-generation-inference#2004

Merged

5 tasks

nouhadziri mentioned this issue Jun 10, 2024

New beaker image for evaluating OLMo-1.7 on Open-Instruct eval suite allenai/open-instruct#175

Merged

astefanutti added a commit to opendatahub-io/distributed-workloads that referenced this issue Jun 26, 2024

Work-around Dao-AILab/flash-attention#453 when installing Flash Atten…

4a13ddf

…tion

astefanutti added a commit to opendatahub-io/distributed-workloads that referenced this issue Jun 28, 2024

Work-around Dao-AILab/flash-attention#453 when installing Flash Atten…

d406830

…tion

astefanutti added a commit to opendatahub-io/distributed-workloads that referenced this issue Jun 28, 2024

Work-around Dao-AILab/flash-attention#453 when installing Flash Atten…

504b60b

…tion

astefanutti added a commit to opendatahub-io/distributed-workloads that referenced this issue Jul 1, 2024

Work-around Dao-AILab/flash-attention#453 when installing Flash Atten…

da054a1

…tion

astefanutti added a commit to opendatahub-io/distributed-workloads that referenced this issue Jul 2, 2024

Work-around Dao-AILab/flash-attention#453 when installing Flash Atten…

3ccd4c3

…tion

tsailiming pushed a commit to tsailiming/distributed-workloads that referenced this issue Jul 16, 2024

Work-around Dao-AILab/flash-attention#453 when installing Flash Atten…

a7bb0e4

…tion

ncau pushed a commit to ncau/distributed-workloads that referenced this issue Aug 22, 2024

Work-around Dao-AILab/flash-attention#453 when installing Flash Atten…

2bed659

…tion

rentainhe mentioned this issue Aug 26, 2024

ImportError: flash_attn not found when running grounded_sam2_florence2_image_demo.py IDEA-Research/Grounded-SAM-2#35

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pip install flash-attn always happens ModuleNotFoundError: No module named 'packaging',but actually i have pip install packaging #453

pip install flash-attn always happens ModuleNotFoundError: No module named 'packaging',but actually i have pip install packaging #453

sky2future commented Aug 16, 2023

tmm1 commented Aug 17, 2023

tmm1 commented Aug 17, 2023

Lemaqwq commented Aug 22, 2023

yunzhongyan0 commented Aug 25, 2023

tmm1 commented Aug 25, 2023

kig1929 commented Aug 25, 2023 •

edited

Loading

tmm1 commented Aug 25, 2023 •

edited

Loading

tmm1 commented Aug 25, 2023

Andcircle commented Aug 28, 2023 •

edited

Loading

tmm1 commented Aug 28, 2023

Andcircle commented Sep 2, 2023

tmm1 commented Sep 2, 2023

Bec-k commented Sep 5, 2023

Bec-k commented Sep 5, 2023

Bec-k commented Sep 5, 2023

tridao commented Sep 5, 2023

adeepak7 commented Nov 10, 2023

frankxyy commented Nov 21, 2023

tridao commented Nov 21, 2023 •

edited

Loading

blakejrobinson commented Nov 28, 2023

JeffreyWardman commented Jan 2, 2024

tridao commented Jan 2, 2024

BBerabi commented Jan 3, 2024

lidingsnyk commented Feb 1, 2024 •

edited

Loading

lidingsnyk commented Feb 7, 2024

michaelze commented Apr 12, 2024 •

edited

Loading

tridao commented Apr 12, 2024

fernandorodriguez-aily commented May 17, 2024

mirekphd commented May 24, 2024 •

edited

Loading

sangshuduo commented Oct 2, 2024

pip install flash-attn always happens ModuleNotFoundError: No module named 'packaging',but actually i have pip install packaging #453

pip install flash-attn always happens ModuleNotFoundError: No module named 'packaging',but actually i have pip install packaging #453

Comments

sky2future commented Aug 16, 2023

tmm1 commented Aug 17, 2023

tmm1 commented Aug 17, 2023

Lemaqwq commented Aug 22, 2023

yunzhongyan0 commented Aug 25, 2023

tmm1 commented Aug 25, 2023

kig1929 commented Aug 25, 2023 • edited Loading

tmm1 commented Aug 25, 2023 • edited Loading

tmm1 commented Aug 25, 2023

Andcircle commented Aug 28, 2023 • edited Loading

tmm1 commented Aug 28, 2023

Andcircle commented Sep 2, 2023

tmm1 commented Sep 2, 2023

Bec-k commented Sep 5, 2023

Bec-k commented Sep 5, 2023

Bec-k commented Sep 5, 2023

tridao commented Sep 5, 2023

adeepak7 commented Nov 10, 2023

frankxyy commented Nov 21, 2023

tridao commented Nov 21, 2023 • edited Loading

blakejrobinson commented Nov 28, 2023

JeffreyWardman commented Jan 2, 2024

tridao commented Jan 2, 2024

BBerabi commented Jan 3, 2024

lidingsnyk commented Feb 1, 2024 • edited Loading

lidingsnyk commented Feb 7, 2024

michaelze commented Apr 12, 2024 • edited Loading

tridao commented Apr 12, 2024

fernandorodriguez-aily commented May 17, 2024

mirekphd commented May 24, 2024 • edited Loading

sangshuduo commented Oct 2, 2024

kig1929 commented Aug 25, 2023 •

edited

Loading

tmm1 commented Aug 25, 2023 •

edited

Loading

Andcircle commented Aug 28, 2023 •

edited

Loading

tridao commented Nov 21, 2023 •

edited

Loading

lidingsnyk commented Feb 1, 2024 •

edited

Loading

michaelze commented Apr 12, 2024 •

edited

Loading

mirekphd commented May 24, 2024 •

edited

Loading