-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Phar problems: crc32 mismatch on file #2592
Comments
I can confirm this behavior on php 7.4, but with a success rate way less than 50% |
Thanks for your report. I never had that problem. Can you please add -vvv to the command and post the output (maybe you need to remove some confidential data), thanks! |
I just realized the error message is slightly different, but it is very similar to coffeeneer's error message: $ dep -V
Deployer 7.0.0-beta.23
$ dep ssh -vvv
In ClassLoader.php line 444:
[ErrorException]
include(phar:///usr/local/bin/dep/vendor/composer/../../src/Component/Ssh/Client.php): failed to open stream: phar error: internal corruption of phar "/usr/local/b
in/dep" (crc32 mismatch on file "src/Component/Ssh/Client.php")
Exception trace:
at phar:///usr/local/bin/dep/vendor/composer/ClassLoader.php:444
{closure}() at phar:///usr/local/bin/dep/vendor/composer/ClassLoader.php:444
include() at phar:///usr/local/bin/dep/vendor/composer/ClassLoader.php:444
Composer\Autoload\includeFile() at phar:///usr/local/bin/dep/vendor/composer/ClassLoader.php:322
Composer\Autoload\ClassLoader->loadClass() at n/a:n/a
spl_autoload_call() at phar:///usr/local/bin/dep/src/Command/SshCommand.php:86
Deployer\Command\SshCommand->execute() at phar:///usr/local/bin/dep/vendor/symfony/console/Command/Command.php:256
Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/dep/vendor/symfony/console/Application.php:971
Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/dep/vendor/symfony/console/Application.php:290
Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/dep/vendor/symfony/console/Application.php:166
Symfony\Component\Console\Application->run() at phar:///usr/local/bin/dep/src/Deployer.php:304
Deployer\Deployer::run() at phar:///usr/local/bin/dep/bin/dep:104
require() at /usr/local/bin/dep:4 |
With verbose output we get the same thing: ➜ php --version
PHP 8.0.7 (cli) (built: Jun 2 2021 04:04:16) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.7, Copyright (c) Zend Technologies
➜ php ./vendor/bin/dep --version
Deployer 7.0.0-beta.23
➜ php ./vendor/bin/dep ssh test -vvv
In ClassLoader.php line 444:
[ErrorException]
include(phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/composer/../../src/Componen
t/Ssh/Client.php): Failed to open stream: phar error: internal corruption of phar "PROJECTPATH/vendor/deployer/dist/dep" (crc32 mismatch on file "src/Component/Ssh/Client.php")
Exception trace:
at phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/composer/ClassLoader.php:444
{closure}() at phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/composer/ClassLoader.php:444
include() at phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/composer/ClassLoader.php:444
Composer\Autoload\includeFile() at phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/composer/ClassLoader.php:322
Composer\Autoload\ClassLoader->loadClass() at phar://PROJECTPATH/vendor/deployer/dist/dep/src/Command/SshCommand.php:86
Deployer\Command\SshCommand->execute() at phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/symfony/console/Command/Command.php:256
Symfony\Component\Console\Command\Command->run() at phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/symfony/console/Application.php:971
Symfony\Component\Console\Application->doRunCommand() at phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/symfony/console/Application.php:290
Symfony\Component\Console\Application->doRun() at phar://PROJECTPATH/vendor/deployer/dist/dep/vendor/symfony/console/Application.php:166
Symfony\Component\Console\Application->run() at phar://PROJECTPATH/vendor/deployer/dist/dep/src/Deployer.php:304
Deployer\Deployer::run() at phar:///PROJECTPATH/vendor/deployer/dist/dep/bin/dep:104
require() at PROJECTPATH/vendor/deployer/dist/dep:4
ssh [<hostname>] |
That's interesting, up until now I hadn't noticed the log in my issue had a different error. I've seen the Ssh/Client.php issue more often on my local machine. |
hmm...
This is a downloaded phar file, from https://deployer.org/releases/v7.0.0-beta.23/deployer.phar |
as @coffeeneer wrote this is an intermittent error - so it works every 4th or 5th call (for me here on wsl2, ubuntu 20.04). |
I had this some time in the past, but can't remember where and how. If I interpret the error correctly it says "Your damn PHAR file is broken and it doesn't match the hash anymore". So the questions is why?
Everything I come up with is broken hardware. Happy for other explanations! |
Yeah I don't get why this is intermittent. I'd expect the file is either okay or broken. But for me it's happening intermittently on multiple systems. My personal system is a really new build with pretty high end hardware. Haven't had any other intermittent issues. To me the hardware has proven itself stable. Only thing I could think of for this system is buggy PHP since Arch is usually pretty bleeding edge. My build server is a bit older, but running really stable with ECC memory. The build is done with Gitlab runner on a VM. Build image is lorisleiva/laravel:latest docker image, which is alpine based. Alpine uses an alternative C/C++ std lib. So both Arch and Alpine could be suspects, but I noticed @bayer has issues with ubuntu on wsl2. From both machines I do not expect any issues regarding stability of the systems themselves. If I find some time I'll do some tests with yet another system and maybe try a different target server. I've tried multiple versions of PHP on my personal rig to no avail. Could try with a live usb to rule out all OS issues. |
@Schrank A colleague suggested it could be a race condition in phar loading. Where the crc is checked while it is still loading or multiple processes are loading the phar at the same time and corrupting the unpacked files. An important piece of information missing on this issue: I'm using the deployer/dist package installed with composer. I've tried with the source build 'deployer/deployer' and lo and behold, everything works fine. So this issue seems to be only with the deployer/dist package. And it seems to be related to composer class loading. I've found something kind of related maybe: |
@antonmedv are the phar files on deployer.org from deployer/dist? |
Other way around. Deployer/dist has files from deployer.org. Phar built on server during release. |
Running into the same issue in gitlab CI server (image: thecodingmachine/php:7.4-v4-cli-node10):
Result:
Any ideas what to do, workaround etc? |
This is really strange. Maybe release phar is broken? |
I can see that someone above has the same issue with release 23 - I was using 24. I'm not really familiar with the internals of PHP, but based on the above, it seems like some sort of a deeper issue? |
Yes, it looks like some deeper issue. Can you test some other releases? Like 17-18? |
I tested on my local WSL2 environment (ubuntu 20.04 / php7.4) like this: r20:
r21:
|
After some additional testing it seems that r20 also breaks, but quite rarely. Rolling dice didn't seem to trigger the error, but |
Managed to reproduce the same issue down to at least r10, until the version differences got big enough that my deploy.php no longer worked. I also attempted building the phar using build script from 6.x branch and code from r24, but still got the same errors (I don't really know much about building phar archives, so perhaps I did something wrong / there's some internal cache somewhere / etc - so please take this with a grain of salt). |
Same problems here! PHP 8.0.7 + Deployer v7.0.0-beta.24 |
And what with older php version? |
https://twitter.com/shyim97/status/1418271370421719041?s=21 |
same on php7.4, using Deployer v7.0.0-beta.23 and -beta.24 |
Same with php 8.0.7 and last beta. I Will try add phar to repository root, but with no luck. Crash happens randomly on Bitbucket pipeline. |
Maybe try using unique names to save phar? Current datetime for example. I think its a problem with CI/CD container file structure. |
I am trying to do this:
|
The first attempt was successful. If someone can, check with yourself. |
the error is repeated ((( |
Sorry to hear. Will try to look more into this issue , but I can't reproduce so it's difficult for me to debug it. |
Error is still there as of today. Tried both PHP 7.4 and PHP 8.0.7 on MacOS 11.5.2 Big Sur — throwths crc errors randomly, using Deployer 7.0.0-beta.24. |
Can you try to build phar yourself? Will it help? Maybe it’s related to my build machine? |
Just chiming in here because I've encountered and addressed similar problems for some client project with https://github.com/clue/phar-composer in the past and it looks like this could be related to inherited file descriptors in forked PHP processes. See reactphp/child-process#51 and in particular the linked issues for all the gory details. (Yep, messing with these subtle, low-level details in PHP is what I happen to do for a living and actually spend a lot of time on.) The gist: Forking is easy – except it's not. Whenever you spawn a child process or fork a process on Unix-based platforms, all child processes will inherit all open file descriptors by default. On Unix-based platforms, this can be controlled through the In itself, this isn't a big deal for many applications as child processes would usually disregard any inherited file descriptors and simply open new ones themselves. This is however a major problem when using This can be avoided by making sure each child process has a unique reference to the Phar archive, i.e. by using I think https://stackoverflow.com/questions/29413013/phar-internal-corruption-crc32-mismatch-during-process-fork does a pretty good job at explaining what's going on here. I hope this helps 👍 |
Yes, thanks a lot. I already know how to fix it. Will write pr today and release it. |
Will be happy to support with testing in various configs I run, please notify me when PR is there. |
Released 7.0.0-beta.25. Please test it out! Should be fixed now! |
🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 |
Tested using PHP 8.0 & PHP 7.4 on Github Actions (ubuntu-latest) and MacOS Big Sur (latest build): everything works perfectly. Tests on Windows build agents will follow next week, but I will keep silence unless any issue found. Thank you @antonmedv for quick resolution of this nasty bug! Great tool! |
Wow!!! Significant!!! |
I'm having some issues with deployer connections. When I try to
php ./vendor/bin/dep ssh test
it fails 50% of the time. This is both on my local machine running Arch as well on my Gitlab server in CI/CD. Php version is 8.0.7 on Arch, 8.0.5 on the alpine based docker image used for CI/CD in Gitlab.In testing I have reduced my deploy.yaml to a minimal setup like this:
This is what I get when the CD fails with this issue:
The application server is running OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017. I could try to have that updated to a newer build as it seems a bit old to me.
The text was updated successfully, but these errors were encountered: