Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hashcat: fix failed to create metal library #181921

Closed
wants to merge 1 commit into from

Conversation

wen-long
Copy link
Contributor

this patch can precent macos hashcat v6.2.6 run into failed to create metal library and fix CI the existing v6.2.6 bottle works fine, I dont know why, maybe it's caused by different command line tools or mac os version

Where did I find this patch code?

I build hashcat locally without homebrew, confirm the latest code works fine And I do git bisect to find which commit fix the issue(screen shoot will attach)
image

git bisect start --term-old broken --term-new fixed  master v6.2.6
...
git bisect run ./b.sh
...
git bisect log
# fixed: [6716447dfce969ddde42a9abe0681500bee0df48] Add support for zero-length salts in Electrum $4 and $5
# broken: [e5b302363654cf4717dbe5bbe417504fc5d0ca72] hashcat 6.2.6
git bisect start '--term-old' 'broken' '--term-new' 'fixed' 'master' 'v6.2.6'
# fixed: [f3fe379571de944f6a9d457a12c1395572ad9f5d] Fix missing entry for -m 31800 in changes.txt
git bisect fixed f3fe379571de944f6a9d457a12c1395572ad9f5d
# fixed: [d19882ff71ad14cb2c252b47b1329bb39ad65fdd] Set a maximum thread count for -m 30901 to 32 for performance reasons
git bisect fixed d19882ff71ad14cb2c252b47b1329bb39ad65fdd
# broken: [29a3ea2583cfbf0dd0249fbb3916247e411709f1] Merge pull request #3494 from ventaquil/feature/exodus-cleanup
git bisect broken 29a3ea2583cfbf0dd0249fbb3916247e411709f1
# fixed: [d008c5cb11c79af40da9132d3b62dc8f33c43e90] Merge pull request #3522 from rjancewicz/rjancewicz/m07350-rakp-hmac-md5
git bisect fixed d008c5cb11c79af40da9132d3b62dc8f33c43e90
# fixed: [01938c374ce7b6fd4a8dfe3c522093eb42c2edd6] Merge remote-tracking branch 'origin/master' into bcrypt_sha256
git bisect fixed 01938c374ce7b6fd4a8dfe3c522093eb42c2edd6
# fixed: [66b22fa64472b4d809743c35fb05fc3c993a5cd2] Add support for Metal > 300 and reject support for older version
git bisect fixed 66b22fa64472b4d809743c35fb05fc3c993a5cd2
# broken: [6aa3e0882d83a6ee8f0a4decbdbad06b5be2affb] Mark some hash-modes for Apple Metal as unstable
git bisect broken 6aa3e0882d83a6ee8f0a4decbdbad06b5be2affb
# broken: [a15eeac44fb990879457ff9c1f8fa3ed22bd3765] Backport some AMD HIP headers from ROCm 5.3.0
git bisect broken a15eeac44fb990879457ff9c1f8fa3ed22bd3765
# broken: [f5c53a7e7736df04bc1b01e3ce6ffd06160f380e] added mode 30500
git bisect broken f5c53a7e7736df04bc1b01e3ce6ffd06160f380e
# broken: [adcd3e3e87a3448ecf8f3f82f0fdcc37d76eed81] Merge pull request #3508 from piwvvo/mode-30500
git bisect broken adcd3e3e87a3448ecf8f3f82f0fdcc37d76eed81
# first fixed commit: [66b22fa64472b4d809743c35fb05fc3c993a5cd2] Add support for Metal > 300 and reject support for older version

b.sh

make distclean; make -j6
newdir=`mktemp -d`
echo $newdir
cp hashcat $newdir/
cp -r modules $newdir/
cp -r OpenCL $newdir/
cp hashcat.hcstat2 $newdir/
! $newdir/hashcat  --benchmark -m 0 -D 1,2 -w 2 -d 1

Note. move hashcat to a newdir evert time is necessary. as I mentioned in #181691

And I try to find which line in 66b22fa cause the fix, finally, the macro METAL is the key

the process is reproducible, only cost ~10 min

  • Have you followed the guidelines for contributing?
  • Have you ensured that your commits follow the commit style guide?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

Copy link
Member

@carlocab carlocab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks; very nice! One suggestion.

Formula/h/hashcat.rb Outdated Show resolved Hide resolved
@github-actions github-actions bot added the autosquash Automatically squash pull request commits according to Homebrew style. label Aug 21, 2024
@carlocab carlocab added the CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. label Aug 21, 2024
this patch can precent macos hashcat v6.2.6 run into `failed to create metal library` and fix CI
the existing v6.2.6 bottle works fine, I dont know why, maybe it's caused by different command line tools or mac os version

Where did I find this patch code?

I build hashcat locally without homebrew, confirm the latest code works fine
And I do git bisect to find which commit fix the issue(screen shoot will attach)

```
git bisect start --term-old broken --term-new fixed  master v6.2.6
...
git bisect run ./b.sh
...
git bisect log
# fixed: [6716447dfce969ddde42a9abe0681500bee0df48] Add support for zero-length salts in Electrum $4 and $5
# broken: [e5b302363654cf4717dbe5bbe417504fc5d0ca72] hashcat 6.2.6
git bisect start '--term-old' 'broken' '--term-new' 'fixed' 'master' 'v6.2.6'
# fixed: [f3fe379571de944f6a9d457a12c1395572ad9f5d] Fix missing entry for -m 31800 in changes.txt
git bisect fixed f3fe379571de944f6a9d457a12c1395572ad9f5d
# fixed: [d19882ff71ad14cb2c252b47b1329bb39ad65fdd] Set a maximum thread count for -m 30901 to 32 for performance reasons
git bisect fixed d19882ff71ad14cb2c252b47b1329bb39ad65fdd
# broken: [29a3ea2583cfbf0dd0249fbb3916247e411709f1] Merge pull request Homebrew#3494 from ventaquil/feature/exodus-cleanup
git bisect broken 29a3ea2583cfbf0dd0249fbb3916247e411709f1
# fixed: [d008c5cb11c79af40da9132d3b62dc8f33c43e90] Merge pull request Homebrew#3522 from rjancewicz/rjancewicz/m07350-rakp-hmac-md5
git bisect fixed d008c5cb11c79af40da9132d3b62dc8f33c43e90
# fixed: [01938c374ce7b6fd4a8dfe3c522093eb42c2edd6] Merge remote-tracking branch 'origin/master' into bcrypt_sha256
git bisect fixed 01938c374ce7b6fd4a8dfe3c522093eb42c2edd6
# fixed: [66b22fa64472b4d809743c35fb05fc3c993a5cd2] Add support for Metal > 300 and reject support for older version
git bisect fixed 66b22fa64472b4d809743c35fb05fc3c993a5cd2
# broken: [6aa3e0882d83a6ee8f0a4decbdbad06b5be2affb] Mark some hash-modes for Apple Metal as unstable
git bisect broken 6aa3e0882d83a6ee8f0a4decbdbad06b5be2affb
# broken: [a15eeac44fb990879457ff9c1f8fa3ed22bd3765] Backport some AMD HIP headers from ROCm 5.3.0
git bisect broken a15eeac44fb990879457ff9c1f8fa3ed22bd3765
# broken: [f5c53a7e7736df04bc1b01e3ce6ffd06160f380e] added mode 30500
git bisect broken f5c53a7e7736df04bc1b01e3ce6ffd06160f380e
# broken: [adcd3e3e87a3448ecf8f3f82f0fdcc37d76eed81] Merge pull request Homebrew#3508 from piwvvo/mode-30500
git bisect broken adcd3e3e87a3448ecf8f3f82f0fdcc37d76eed81
# first fixed commit: [66b22fa64472b4d809743c35fb05fc3c993a5cd2] Add support for Metal > 300 and reject support for older version
```

b.sh
```
make distclean; make -j6
newdir=`mktemp -d`
echo $newdir
cp hashcat $newdir/
cp -r modules $newdir/
cp -r OpenCL $newdir/
cp hashcat.hcstat2 $newdir/
! $newdir/hashcat  --benchmark -m 0 -D 1,2 -w 2 -d 1
```

Note. move hashcat to a newdir evert time is necessary. as I mentioned in Homebrew#181691

And I try to find which line in 66b22fa cause the fix, finally, the macro __METAL__ is the key

the process is reproducible, only cost ~10 min
@github-actions github-actions bot removed the autosquash Automatically squash pull request commits according to Homebrew style. label Aug 21, 2024
@Bo98
Copy link
Member

Bo98 commented Aug 21, 2024

arm64 VMs don't support OpenCL so adjusting the test to add --backend-ignore-opencl when OS.mac? && Hardware::CPU.arm? && Hardware::CPU.virtualized? may work.

@carlocab
Copy link
Member

arm64 VMs don't support OpenCL so adjusting the test to add --backend-ignore-opencl when OS.mac? && Hardware::CPU.arm? && Hardware::CPU.virtualized? may work.

Weird that it works on 13-arm64 and 14-arm64 tho?

@Bo98
Copy link
Member

Bo98 commented Aug 21, 2024

It doesn't - there's no bottle for them.

https://github.com/Homebrew/homebrew-core/actions/runs/10494147858/job/29070706687?pr=181921#step:3:132

@carlocab carlocab added the CI-no-bottles Merge without publishing bottles label Aug 21, 2024
@wen-long
Copy link
Contributor Author

arm64 VMs don't support OpenCL so adjusting the test to add --backend-ignore-opencl when OS.mac? && Hardware::CPU.arm? && Hardware::CPU.virtualized? may work.

I start action with hashcat binary from homebrew, no command can succeed
do expand to check detail log
https://github.com/wen-long/hashcat_play/actions/runs/10495497522/job/29074054929

And is the test really necessary?

@carlocab
Copy link
Member

Might be worth trying in CI here, since we don't use GitHub Actions runners to build. Could very well end up with the same result though.

And is the test really necessary?

Yes, even if only a very minimal one.

@wen-long
Copy link
Contributor Author

neofetch and system_profiler did't detect GPU
https://github.com/wen-long/hashcat_play/actions/runs/10495637687/job/29074484393
check display info

@wen-long
Copy link
Contributor Author

Might be worth trying in CI here, since we don't use GitHub Actions runners to build. Could very well end up with the same result though.

And is the test really necessary?

Yes, even if only a very minimal one.

hashcat --version is an option?

@carlocab
Copy link
Member

Might be worth trying in CI here, since we don't use GitHub Actions runners to build. Could very well end up with the same result though.

And is the test really necessary?

Yes, even if only a very minimal one.

hashcat --version is an option?

Yes, but only once we've verified the existing tests with the flags suggested above don't work in our CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-no-bottles Merge without publishing bottles CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants