CI failures: Cannot run program "./cs": error=2, No such file or directory

[nicolasstucki]

We have a few spurious failures of CI involving CoursierScalaTests. Usually rerunning the tests fixes the issue.

• https://github.com/lampepfl/dotty/actions/runs/4427028539/jobs/7768013294
• https://github.com/lampepfl/dotty/actions/runs/4430193399/jobs/7771618203
• https://github.com/lampepfl/dotty/actions/runs/4414553285/jobs/7736411129
• ...

[info] Test dotty.tools.coursier.CoursierScalaTests.allTests started
Error:  Test dotty.tools.coursier.CoursierScalaTests.allTests failed: java.io.IOException: Cannot run program "./cs": error=2, No such file or directory, took 0.088 sec
Error:      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1142)
Error:      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
Error:      at scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:85)
Error:      at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.run(ProcessBuilderImpl.scala:119)
Error:      at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$anonfun$runBuffered$1(ProcessBuilderImpl.scala:205)
Error:      at scala.runtime.java8.JFunction0$mcI$sp.apply(JFunction0$mcI$sp.scala:17)
Error:      at scala.sys.process.ProcessLogger$$anon$1.buffer(ProcessLogger.scala:103)
Error:      at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.runBuffered(ProcessBuilderImpl.scala:205)
Error:      at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.!(ProcessBuilderImpl.scala:146)
Error:      at dotty.tools.coursier.CoursierScalaTests$.execCmd(CoursierScalaTests.scala:154)
Error:      at dotty.tools.coursier.CoursierScalaTests$.csCmd(CoursierScalaTests.scala:169)
Error:      at dotty.tools.coursier.CoursierScalaTests$.csScalaCmd(CoursierScalaTests.scala:158)
Error:      at dotty.tools.coursier.CoursierScalaTests.scriptArgs$1(CoursierScalaTests.scala:42)
Error:      at dotty.tools.coursier.CoursierScalaTests.allTests(CoursierScalaTests.scala:55)
Error:      at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Error:      at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
Error:      at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Error:      at java.lang.reflect.Method.invoke(Method.java:567)
Error:      ...
Error:  Caused by: java.io.IOException: error=2, No such file or directory
Error:      at java.lang.ProcessImpl.forkAndExec(Native Method)
Error:      at java.lang.ProcessImpl.this(ProcessImpl.java:313)
Error:      at java.lang.ProcessImpl.start(ProcessImpl.java:244)
Error:      at java.lang.ProcessBuilder.start(ProcessBuilder.java:1109)
Error:      ... 51 more

[nicolasstucki]

I wonder if it is a problematic CI worker.

[anatoliykmetyuk]

The workers appear to be all online. Why do you think this has to do with a worker?

[nicolasstucki]

I was just a theory based on how often the issue happened. I also noticed that this issue happens more often when the CI is at full capacity (i.e. in the CET afternoons).

[nicolasstucki]

Here is anotherone from today https://github.com/lampepfl/dotty/actions/runs/4430193399/jobs/7771618203?pr=17114

[anatoliykmetyuk]

I suspect this could be because the test fails to download coursier at some times. The way it is currently written, there is no safeguard against a failure in download.

https://github.com/lampepfl/dotty/blob/2dec6825d42beb39004f8aec9bba3248803f2e1f/compiler/test-coursier/dotty/tools/coursier/CoursierScalaTests.scala#L174

execCmd returns a List[String], hence #&& doesn't do anything here.

I've added a PR #17127 that safeguards against failed downloads and mimics the intent of #&&. Let's see the next time the error manifests if that is the failed download indeed.

[nicolasstucki]

Now we got a different error

Error:  Test dotty.tools.coursier.CoursierScalaTests failed: java.lang.AssertionError: Failed to run curl -fLo cs https://git.io/coursier-cli-linux, exit code: 22, output:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Error:                                   Dload  Upload   Total   Spent    Left  Speed
Error:  
Error:    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Error:    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Error:  
Error:    0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
Error:  
Error:    0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
Error:  
Error:    0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
Error:  curl: (22) The requested URL returned error: 500 , took 0.0 sec
Error:      at dotty.tools.coursier.CoursierScalaTests$.runAndCheckCmd$1(CoursierScalaTests.scala:178)
Error:      at dotty.tools.coursier.CoursierScalaTests$.setup(CoursierScalaTests.scala:180)
Error:      at dotty.tools.coursier.CoursierScalaTests.setup(CoursierScalaTests.scala)
Error:      at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Error:      at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
Error:      at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Error:      at java.lang.reflect.Method.invoke(Method.java:567)
Error:      ...

https://github.com/lampepfl/dotty/actions/runs/4487258054/jobs/7890504165?pr=17059

[dwijnand]

Yes, I suspect this is a problem with git.io reliability. Can we switch it using github.com instead?

$ curl -I https://git.io/coursier-cli-linux
HTTP/2 303
location: https://github.com/coursier/launchers/raw/master/cs-x86_64-pc-linux
content-type: text/html;charset=utf-8
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
content-length: 0
x-github-backend: Kubernetes
x-github-request-id: E2A9:5011:3A34B4:13F1A39:641AC78B