Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project configuration fails after migrating to 0.11.2+ #2844

Closed
MaciejG604 opened this issue Oct 20, 2023 · 14 comments · Fixed by #2883
Closed

Project configuration fails after migrating to 0.11.2+ #2844

MaciejG604 opened this issue Oct 20, 2023 · 14 comments · Fixed by #2883
Labels
bug The issue represents an bug workaround-available
Milestone

Comments

@MaciejG604
Copy link

After migrating from 0.11.1 to any higher version, a project, that was working fine, fails on certain commands with an exception.

It happens in this repo: https://github.com/VirtusLab/scala-js-cli e.g. after doing ./mill -i 'tests.test'

Stack trace after setting the version to 0.11.2:

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at mill.main.client.IsolatedMillMainLoader.runMain(IsolatedMillMainLoader.java:58)
        at mill.main.client.MillClientMain.main(MillClientMain.java:64)
Caused by: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:662)
        at scala.collection.immutable.Nil$.head(List.scala:661)
        at mill.eval.GroupEvaluator$$anonfun$1.applyOrElse(GroupEvaluator.scala:103)
        at mill.eval.GroupEvaluator$$anonfun$1.applyOrElse(GroupEvaluator.scala:88)
        at scala.collection.Iterator$$anon$7.hasNext(Iterator.scala:525)
        at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:601)
        at scala.collection.IterableOnceOps.foldLeft(IterableOnce.scala:675)
        at scala.collection.IterableOnceOps.foldLeft$(IterableOnce.scala:670)
        at scala.collection.AbstractIterator.foldLeft(Iterator.scala:1300)
        at scala.collection.IterableOnceOps.sum(IterableOnce.scala:945)
        at scala.collection.IterableOnceOps.sum$(IterableOnce.scala:943)
        at scala.collection.AbstractIterator.sum(Iterator.scala:1300)
        at mill.eval.GroupEvaluator.evaluateGroupCached(GroupEvaluator.scala:136)
        at mill.eval.GroupEvaluator.evaluateGroupCached$(GroupEvaluator.scala:43)
        at mill.eval.EvaluatorImpl.evaluateGroupCached(EvaluatorImpl.scala:15)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$2(EvaluatorCore.scala:116)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at mill.eval.ExecutionContexts$RunNow$.execute(ExecutionContexts.scala:13)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$1(EvaluatorCore.scala:92)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$1$adapted(EvaluatorCore.scala:90)
        at scala.collection.immutable.VectorStatics$.foreachRec(Vector.scala:2122)
        at scala.collection.immutable.Vector.foreach(Vector.scala:2128)
        at mill.eval.EvaluatorCore.evaluate0(EvaluatorCore.scala:90)
        at mill.eval.EvaluatorCore.$anonfun$evaluate$1(EvaluatorCore.scala:43)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at mill.eval.EvaluatorCore.evaluate(EvaluatorCore.scala:34)
        at mill.eval.EvaluatorCore.evaluate$(EvaluatorCore.scala:26)
        at mill.eval.EvaluatorImpl.evaluate(EvaluatorImpl.scala:15)
        at mill.main.RunScript$.evaluateNamed(RunScript.scala:38)
        at mill.main.RunScript$.$anonfun$evaluateTasksNamed$2(RunScript.scala:26)
        at scala.util.Either.map(Either.scala:382)
        at mill.main.RunScript$.evaluateTasksNamed(RunScript.scala:26)
        at mill.runner.MillBuildBootstrap$.evaluateWithWatches(MillBuildBootstrap.scala:358)
        at mill.runner.MillBuildBootstrap.$anonfun$processFinalTargets$2(MillBuildBootstrap.scala:268)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at mill.runner.MillBuildBootstrap.processFinalTargets(MillBuildBootstrap.scala:268)
        at mill.runner.MillBuildBootstrap.evaluateRec(MillBuildBootstrap.scala:159)
        at mill.runner.MillBuildBootstrap.evaluate(MillBuildBootstrap.scala:47)
        at mill.runner.MillMain$.$anonfun$main0$5(MillMain.scala:207)
        at mill.runner.Watching$.watchLoop(Watching.scala:27)
        at mill.runner.MillMain$.$anonfun$main0$1(MillMain.scala:193)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withErr(Console.scala:193)
        at mill.api.SystemStreams$.$anonfun$withStreams$2(SystemStreams.scala:56)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withOut(Console.scala:164)
        at mill.api.SystemStreams$.$anonfun$withStreams$1(SystemStreams.scala:55)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withIn(Console.scala:227)
        at mill.api.SystemStreams$.withStreams(SystemStreams.scala:54)
        at mill.runner.MillMain$.main0(MillMain.scala:80)
        at mill.runner.MillMain$.main(MillMain.scala:58)
        at mill.runner.MillMain.main(MillMain.scala)
        ... 6 more

It points to main/eval/src/mill/eval/GroupEvaluator.scala line 103, where a head call has been introduced in #2417.

@lefou
Copy link
Member

lefou commented Oct 20, 2023

Thanks for this report. While I had a first glance, I recognized you're using some external plugins, and at least one (mill-native-image) has a problematic transitive dependency setup. This may result in an unclean classpath, as multiple versions of Mill artifacts can end up in it. This does not necessarily mean, this is the cause for the reported issue, but since Mill 0.11.2 analyzes API signatures of the whole classpath, an unclean classpath has a high potential to break this logic.

We have a new CLI option --disable-callgraph-invalidation to disable that logic, and indeed, once I use it, the reported issue goes away.

MILL_VERSION="0.11.2" mill -i --disable-callgraph-invalidation  "tests.test"
Mill version 0.11.2 is different than configured for this directory!
Configured version is 0.11.1 (/home/lefou/work/tmp/scala-js-cli/.mill-version)
[138/138] tests.test.test 
org.scalajs.cli.tests.Tests:
  + tests 7.845s

I opened alexarchambault/mill-native-image#69 for it.

@lefou
Copy link
Member

lefou commented Oct 30, 2023

mill-native-image version 0.1.26 was just released. @MaciejG604 Can you update to it and check, if that already fixes your issue? Thanks

@lefou
Copy link
Member

lefou commented Nov 2, 2023

Bumping mill-native-image does not fix the issue.

diff --git a/build.sc b/build.sc
index 54f7eda..57610af 100644
--- a/build.sc
+++ b/build.sc
@@ -1,6 +1,6 @@
 import $ivy.`de.tototec::de.tobiasroeser.mill.vcs.version::0.4.0`
-import $ivy.`io.github.alexarchambault.mill::mill-native-image::0.1.25`
-import $ivy.`io.github.alexarchambault.mill::mill-native-image-upload:0.1.19`
+import $ivy.`io.github.alexarchambault.mill::mill-native-image::0.1.26`
+import $ivy.`io.github.alexarchambault.mill::mill-native-image-upload:0.1.26`
 import $ivy.`io.get-coursier::coursier-launcher:2.1.0-M2`
 
 import de.tobiasroeser.mill.vcs.version._
@@ -112,7 +112,7 @@ trait ScalaJsCliNativeImage extends ScalaModule with NativeImage {
   def graalVmVersion = "22.3.3"
   def nativeImageGraalVmJvmId = s"graalvm-java17:$graalVmVersion"
   def nativeImageName = "scala-js-ld"
-  def moduleDeps() = Seq(
+  def moduleDeps = Seq(
     cli
   )
   def compileIvyDeps = super.compileIvyDeps() ++ Seq(
@@ -353,7 +353,7 @@ object ci extends Module {
 
     val path = os.Path(directory, os.pwd)
     val launchers = os.list(path).filter(os.isFile(_)).map { path =>
-      path.toNIO -> path.last
+      path -> path.last
     }
     val ghToken = Option(System.getenv("UPLOAD_GH_TOKEN")).getOrElse {
       sys.error("UPLOAD_GH_TOKEN not set")

Output:

$ ± MILL_VERSION="0.11.5" mill -i "tests.test"
Mill version 0.11.5 is different than configured for this directory!
Configured version is 0.11.1 (/home/lefou/work/tmp/scala-js-cli/.mill-version)
[130/138] tests.test.forkEnv 
An unexpected error occurred
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at mill.main.client.IsolatedMillMainLoader.runMain(IsolatedMillMainLoader.java:58)
        at mill.main.client.MillClientMain.main(MillClientMain.java:64)
Caused by: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:662)
        at scala.collection.immutable.Nil$.head(List.scala:661)
        at mill.eval.GroupEvaluator$$anonfun$1.applyOrElse(GroupEvaluator.scala:104)
        at mill.eval.GroupEvaluator$$anonfun$1.applyOrElse(GroupEvaluator.scala:89)
        at scala.collection.Iterator$$anon$7.hasNext(Iterator.scala:525)
        at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:601)
        at scala.collection.IterableOnceOps.foldLeft(IterableOnce.scala:675)
        at scala.collection.IterableOnceOps.foldLeft$(IterableOnce.scala:670)
        at scala.collection.AbstractIterator.foldLeft(Iterator.scala:1300)
        at scala.collection.IterableOnceOps.sum(IterableOnce.scala:945)
        at scala.collection.IterableOnceOps.sum$(IterableOnce.scala:943)
        at scala.collection.AbstractIterator.sum(Iterator.scala:1300)
        at mill.eval.GroupEvaluator.evaluateGroupCached(GroupEvaluator.scala:137)
        at mill.eval.GroupEvaluator.evaluateGroupCached$(GroupEvaluator.scala:44)
        at mill.eval.EvaluatorImpl.evaluateGroupCached(EvaluatorImpl.scala:15)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$2(EvaluatorCore.scala:116)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at mill.eval.ExecutionContexts$RunNow$.execute(ExecutionContexts.scala:13)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$1(EvaluatorCore.scala:92)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$1$adapted(EvaluatorCore.scala:90)
        at scala.collection.immutable.VectorStatics$.foreachRec(Vector.scala:2124)
        at scala.collection.immutable.Vector.foreach(Vector.scala:2130)
        at mill.eval.EvaluatorCore.evaluate0(EvaluatorCore.scala:90)
        at mill.eval.EvaluatorCore.$anonfun$evaluate$1(EvaluatorCore.scala:43)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at mill.eval.EvaluatorCore.evaluate(EvaluatorCore.scala:34)
        at mill.eval.EvaluatorCore.evaluate$(EvaluatorCore.scala:26)
        at mill.eval.EvaluatorImpl.evaluate(EvaluatorImpl.scala:15)
        at mill.main.RunScript$.evaluateNamed(RunScript.scala:38)
        at mill.main.RunScript$.$anonfun$evaluateTasksNamed$2(RunScript.scala:26)
        at scala.util.Either.map(Either.scala:382)
        at mill.main.RunScript$.evaluateTasksNamed(RunScript.scala:26)
        at mill.runner.MillBuildBootstrap$.evaluateWithWatches(MillBuildBootstrap.scala:399)
        at mill.runner.MillBuildBootstrap.$anonfun$processFinalTargets$3(MillBuildBootstrap.scala:308)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at mill.runner.MillBuildBootstrap.processFinalTargets(MillBuildBootstrap.scala:308)
        at mill.runner.MillBuildBootstrap.evaluateRec(MillBuildBootstrap.scala:196)
        at mill.runner.MillBuildBootstrap.evaluate(MillBuildBootstrap.scala:48)
        at mill.runner.MillMain$.$anonfun$main0$6(MillMain.scala:234)
        at mill.runner.Watching$.watchLoop(Watching.scala:27)
        at mill.runner.MillMain$.$anonfun$main0$1(MillMain.scala:219)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withErr(Console.scala:193)
        at mill.api.SystemStreams$.$anonfun$withStreams$2(SystemStreams.scala:62)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withOut(Console.scala:164)
        at mill.api.SystemStreams$.$anonfun$withStreams$1(SystemStreams.scala:61)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withIn(Console.scala:227)
        at mill.api.SystemStreams$.withStreams(SystemStreams.scala:60)
        at mill.runner.MillMain$.main0(MillMain.scala:101)
        at mill.runner.MillMain$.liftedTree1$1(MillMain.scala:78)
        at mill.runner.MillMain$.main(MillMain.scala:69)
        at mill.runner.MillMain.main(MillMain.scala)
        ... 6 more

@lefou
Copy link
Member

lefou commented Nov 2, 2023

@lihaoyi Could you have a look?

@Gedochao
Copy link

Gedochao commented Nov 10, 2023

@lefou although --disable-callgraph-invalidation seems to solve the core issue, we ended up reverting to Mill 0.11.1, as the workaround seems to have broken our CI.

I did try to work around it in VirtusLab/scala-cli#2528, but eventually gave up with yet another failure (unclear if because of 0.11.5, --disable-callgraph-invalidation or some other problem with the Scala CLI build configuration). There was definitely a big chunk of headaches tied to #2869 sneakily breaking our publishing.

Looking forward to this issue getting fixed, regardless, we'll stay on 0.11.1 until then.

@lefou
Copy link
Member

lefou commented Nov 18, 2023

I had some spare time and invested in narrowing down this issue. First, I made sure this is not related to external dependencies. I found an packaging issue in mill-native-image which is fixed in the meantime. So I opened PR VirtusLab/scala-js-cli#38.

I also found some other issues which I addressed with PR VirtusLab/scala-js-cli#39.

So, based on the branch/PR VirtusLab/scala-js-cli#39, I'm able to reproduce or mute the reported issue with a change of a single line:

The failure with Mill 0.11.5

diff --git a/.mill-version b/.mill-version
index 027934e..ce3191f 100644
--- a/.mill-version
+++ b/.mill-version
@@ -1 +1 @@
-0.11.1
\ No newline at end of file
+0.11.5-29-bc84d5
> mill -i -w tests.test
[134/141] tests.test.forkWorkingDir 
An unexpected error occurred
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at mill.main.client.IsolatedMillMainLoader.runMain(IsolatedMillMainLoader.java:58)
        at mill.main.client.MillClientMain.main(MillClientMain.java:64)
Caused by: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:662)
        at scala.collection.immutable.Nil$.head(List.scala:661)
        at mill.eval.GroupEvaluator$$anonfun$1.applyOrElse(GroupEvaluator.scala:104)
        at mill.eval.GroupEvaluator$$anonfun$1.applyOrElse(GroupEvaluator.scala:89)
        at scala.collection.Iterator$$anon$7.hasNext(Iterator.scala:525)
        at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:601)
        at scala.collection.IterableOnceOps.foldLeft(IterableOnce.scala:675)
        at scala.collection.IterableOnceOps.foldLeft$(IterableOnce.scala:670)
        at scala.collection.AbstractIterator.foldLeft(Iterator.scala:1300)
        at scala.collection.IterableOnceOps.sum(IterableOnce.scala:945)
        at scala.collection.IterableOnceOps.sum$(IterableOnce.scala:943)
        at scala.collection.AbstractIterator.sum(Iterator.scala:1300)
        at mill.eval.GroupEvaluator.evaluateGroupCached(GroupEvaluator.scala:137)
        at mill.eval.GroupEvaluator.evaluateGroupCached$(GroupEvaluator.scala:44)
        at mill.eval.EvaluatorImpl.evaluateGroupCached(EvaluatorImpl.scala:15)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$2(EvaluatorCore.scala:116)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at mill.eval.ExecutionContexts$RunNow$.execute(ExecutionContexts.scala:13)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$1(EvaluatorCore.scala:92)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$1$adapted(EvaluatorCore.scala:90)
        at scala.collection.immutable.Vector.foreach(Vector.scala:2124)
        at mill.eval.EvaluatorCore.evaluate0(EvaluatorCore.scala:90)
        at mill.eval.EvaluatorCore.$anonfun$evaluate$1(EvaluatorCore.scala:43)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at mill.eval.EvaluatorCore.evaluate(EvaluatorCore.scala:34)
        at mill.eval.EvaluatorCore.evaluate$(EvaluatorCore.scala:26)
        at mill.eval.EvaluatorImpl.evaluate(EvaluatorImpl.scala:15)
        at mill.main.RunScript$.evaluateNamed(RunScript.scala:38)
        at mill.main.RunScript$.$anonfun$evaluateTasksNamed$2(RunScript.scala:26)
        at scala.util.Either.map(Either.scala:382)
        at mill.main.RunScript$.evaluateTasksNamed(RunScript.scala:26)
        at mill.runner.MillBuildBootstrap$.evaluateWithWatches(MillBuildBootstrap.scala:399)
        at mill.runner.MillBuildBootstrap.$anonfun$processFinalTargets$3(MillBuildBootstrap.scala:308)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at mill.runner.MillBuildBootstrap.processFinalTargets(MillBuildBootstrap.scala:308)
        at mill.runner.MillBuildBootstrap.evaluateRec(MillBuildBootstrap.scala:196)
        at mill.runner.MillBuildBootstrap.evaluate(MillBuildBootstrap.scala:48)
        at mill.runner.MillMain$.$anonfun$main0$6(MillMain.scala:234)
        at mill.runner.Watching$.watchLoop(Watching.scala:27)
        at mill.runner.MillMain$.$anonfun$main0$1(MillMain.scala:219)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withErr(Console.scala:193)
        at mill.api.SystemStreams$.$anonfun$withStreams$2(SystemStreams.scala:62)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withOut(Console.scala:164)
        at mill.api.SystemStreams$.$anonfun$withStreams$1(SystemStreams.scala:61)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withIn(Console.scala:227)
        at mill.api.SystemStreams$.withStreams(SystemStreams.scala:60)
        at mill.runner.MillMain$.main0(MillMain.scala:101)
        at mill.runner.MillMain$.liftedTree1$1(MillMain.scala:78)
        at mill.runner.MillMain$.main(MillMain.scala:69)
        at mill.runner.MillMain.main(MillMain.scala)
        ... 6 more

The single-line change to remove the reported error

diff --git a/.mill-version b/.mill-version
index 027934e..ce3191f 100644
--- a/.mill-version
+++ b/.mill-version
@@ -1 +1 @@
-0.11.1
\ No newline at end of file
+0.11.5-29-bc84d5
diff --git a/build.sc b/build.sc
index 9df587f..1c7c722 100644
--- a/build.sc
+++ b/build.sc
@@ -55,7 +55,7 @@ trait Cli extends ScalaModule with ScalaJsCliPublishModule {
       Parameters,
       Preamble
     }
-    val cp = jarClassPath().map(_.path)
+    val cp = Seq.empty[os.Path]
     val mainClass0 = mainClass().getOrElse(sys.error("No main class"))
 
     val dest = T.ctx().dest / (if (isWin) "launcher.bat" else "launcher")
> mill -i -w tests.test
[build.sc] [49/53] compile 
[info] compiling 1 Scala source to /home/lefou/work/tmp/scala-js-cli/out/mill-build/compile.dest/classes ...
[warn] /home/lefou/work/tmp/scala-js-cli/build.sc:315:21: constructor SonatypePublisher in class SonatypePublisher is deprecated (since mill 0.10.8): Use other constructor instead
[warn]     val publisher = new scalalib.publish.SonatypePublisher(
[warn]                     ^
[warn] one warning found
[info] done compiling
[88/88] tests.test.jvm 
org.scalajs.cli.tests.Tests:
==> X org.scalajs.cli.tests.Tests.tests  3.129s os.SubprocessException: Result of /home/lefou/work/tmp/scala-js-cli/out/cli/standaloneLauncher.dest/launcher…: 255
Error: class org.scalajs.cli.Scalajsld not found

    at os.proc.call(ProcessOps.scala:89)
    at org.scalajs.cli.tests.Tests.$anonfun$new$1(Tests.scala:48)
1 targets failed
tests.test.jvm 1 tests failed: 
  org.scalajs.cli.tests.Tests.tests org.scalajs.cli.tests.Tests.tests

Obviously the test fails, as I have just emptied the jarClasspath, but the issue in Mill is gone. I hope this helps @lihaoyi to further analyze the classpath analyzer?

@lefou
Copy link
Member

lefou commented Nov 18, 2023

Here is a minimal reproducer:

// /tmp/mill-2844/build.sc
import mill._

object foo extends Module {
  def bar = T {
    baz()
  }
}

private def baz = T { "baz" }
> mill -i dev.run /tmp/mill-2844 -i foo.bar
[1691/1691] dev.run 
[info] compiling 1 Scala source to /tmp/mill-2844/out/mill-build/compile.dest/classes ...
[info] done compiling
An unexpected error occurred
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at mill.main.client.IsolatedMillMainLoader.runMain(IsolatedMillMainLoader.java:58)
        at mill.main.client.MillClientMain.main(MillClientMain.java:64)
Caused by: java.util.NoSuchElementException: head of empty list
        at scala.collection.immutable.Nil$.head(List.scala:662)
        at scala.collection.immutable.Nil$.head(List.scala:661)
        at mill.eval.GroupEvaluator$$anonfun$1.applyOrElse(GroupEvaluator.scala:104)
        at mill.eval.GroupEvaluator$$anonfun$1.applyOrElse(GroupEvaluator.scala:89)
        at scala.collection.Iterator$$anon$7.hasNext(Iterator.scala:525)
        at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:601)
        at scala.collection.IterableOnceOps.foldLeft(IterableOnce.scala:675)
        at scala.collection.IterableOnceOps.foldLeft$(IterableOnce.scala:670)
        at scala.collection.AbstractIterator.foldLeft(Iterator.scala:1300)
        at scala.collection.IterableOnceOps.sum(IterableOnce.scala:945)
        at scala.collection.IterableOnceOps.sum$(IterableOnce.scala:943)
        at scala.collection.AbstractIterator.sum(Iterator.scala:1300)
        at mill.eval.GroupEvaluator.evaluateGroupCached(GroupEvaluator.scala:137)
        at mill.eval.GroupEvaluator.evaluateGroupCached$(GroupEvaluator.scala:44)
        at mill.eval.EvaluatorImpl.evaluateGroupCached(EvaluatorImpl.scala:15)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$2(EvaluatorCore.scala:116)
        at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
        at mill.eval.ExecutionContexts$RunNow$.execute(ExecutionContexts.scala:13)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$1(EvaluatorCore.scala:92)
        at mill.eval.EvaluatorCore.$anonfun$evaluate0$1$adapted(EvaluatorCore.scala:90)
        at scala.collection.immutable.Vector.foreach(Vector.scala:2124)
        at mill.eval.EvaluatorCore.evaluate0(EvaluatorCore.scala:90)
        at mill.eval.EvaluatorCore.$anonfun$evaluate$1(EvaluatorCore.scala:43)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at mill.eval.EvaluatorCore.evaluate(EvaluatorCore.scala:34)
        at mill.eval.EvaluatorCore.evaluate$(EvaluatorCore.scala:26)
        at mill.eval.EvaluatorImpl.evaluate(EvaluatorImpl.scala:15)
        at mill.main.RunScript$.evaluateNamed(RunScript.scala:38)
        at mill.main.RunScript$.$anonfun$evaluateTasksNamed$2(RunScript.scala:26)
        at scala.util.Either.map(Either.scala:382)
        at mill.main.RunScript$.evaluateTasksNamed(RunScript.scala:26)
        at mill.runner.MillBuildBootstrap$.evaluateWithWatches(MillBuildBootstrap.scala:399)
        at mill.runner.MillBuildBootstrap.$anonfun$processFinalTargets$3(MillBuildBootstrap.scala:308)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at mill.runner.MillBuildBootstrap.processFinalTargets(MillBuildBootstrap.scala:308)
        at mill.runner.MillBuildBootstrap.evaluateRec(MillBuildBootstrap.scala:196)
        at mill.runner.MillBuildBootstrap.evaluate(MillBuildBootstrap.scala:48)
        at mill.runner.MillMain$.$anonfun$main0$6(MillMain.scala:234)
        at mill.runner.Watching$.watchLoop(Watching.scala:27)
        at mill.runner.MillMain$.$anonfun$main0$1(MillMain.scala:219)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withErr(Console.scala:193)
        at mill.api.SystemStreams$.$anonfun$withStreams$2(SystemStreams.scala:62)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withOut(Console.scala:164)
        at mill.api.SystemStreams$.$anonfun$withStreams$1(SystemStreams.scala:61)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
        at scala.Console$.withIn(Console.scala:227)
        at mill.api.SystemStreams$.withStreams(SystemStreams.scala:60)
        at mill.runner.MillMain$.main0(MillMain.scala:101)
        at mill.runner.MillMain$.liftedTree1$1(MillMain.scala:78)
        at mill.runner.MillMain$.main(MillMain.scala:69)
        at mill.runner.MillMain.main(MillMain.scala)
        ... 6 more

Removing private from baz fixes it.

> mill -i dev.run /tmp/mill-2844 -i show foo.bar
[1691/1691] dev.run 
[info] compiling 1 Scala source to /tmp/mill-2844/out/mill-build/compile.dest/classes ...
[info] done compiling
"baz"

@lefou lefou added the bug The issue represents an bug label Nov 18, 2023
lefou added a commit to lefou/scala-js-cli that referenced this issue Nov 18, 2023
This not only makes them accessible from the CLI, it also fixes an issue with Mill 0.11.{2-5}.

See com-lihaoyi/mill#2844
@lihaoyi
Copy link
Member

lihaoyi commented Nov 19, 2023

Thanks for the minimization @lefou ! looks like it should be something to do with how private methods are showing up in getDeclaredMethods

@lihaoyi
Copy link
Member

lihaoyi commented Nov 19, 2023

Seems like when I look at the class build, I see baz given the name millbuild$build$$baz rather than just baz

GroupEvaluator.scala:98 namedTask: baz
GroupEvaluator.scala:99 namedTask.ctx.enclosingCls: class millbuild.build$
GroupEvaluator.scala:102 c: class millbuild.build$
GroupEvaluator.scala:103 c.getDeclaredMethods.map(_.getName): Array()
GroupEvaluator.scala:102 c: class millbuild.build
GroupEvaluator.scala:103 c.getDeclaredMethods.map(_.getName): Array(
  "$deserializeLambda$",
  "foo",
  "foo$lzycompute$1",
  "millbuild$build$$baz",
  "$anonfun$baz$1",
  "$anonfun$baz$3",
  "$anonfun$baz$2"
)

@lihaoyi
Copy link
Member

lihaoyi commented Nov 19, 2023

A more minimal repro of this behavior appears to be the following Ammonite session, without Mill, where we demonstrate that the JVM name for private def bar is different depending on whether or not we have an object qux{ bar } beside it

@  trait Foo { private def bar = () } ; object Foo extends Foo 
defined trait Foo
defined object Foo

@ classOf[Foo].getDeclaredMethods.map(_.getName) 
res11: Array[String] = Array("$init$", "bar")

@  trait Foo { object qux{ bar }; private def bar = () } ; object Foo extends Foo 
defined trait Foo
defined object Foo

@ classOf[Foo].getDeclaredMethods.map(_.getName) 
res13: Array[String] = Array("$init$", "ammonite$$sess$cmd12$Foo$$bar$", "ammonite$$sess$cmd12$Foo$$bar", "qux")

@lihaoyi
Copy link
Member

lihaoyi commented Nov 19, 2023

seems like removing the private clause would be a workaround for now. We can probably reverse-engineer this alternative encoding of JVM method names without too much difficulty. I asked in the contributors discord to see if there's any context around this distinction in generated method names

@lihaoyi
Copy link
Member

lihaoyi commented Nov 19, 2023

Should hopefully be fixed by https://github.com/com-lihaoyi/mill/pull/2883/files

@lefou
Copy link
Member

lefou commented Nov 19, 2023

Additional to the proper fix, we should improve the error handling here. Now, that we know that head call on the detected parents can fail, we should promote the exception to a MillException, which will get more nicer handling / reporting by Mill. The error message should tell which target is the issue (this took me some time) and how to work around it (using --disable-callgraph-invalidation).

@lefou
Copy link
Member

lefou commented Nov 19, 2023

Additional to the proper fix, we should improve the error handling here. Now, that we know that head call on the detected parents can fail, we should promote the exception to a MillException, which will get more nicer handling / reporting by Mill. The error message should tell which target is the issue (this took me some time) and how to work around it (using --disable-callgraph-invalidation).

Fixed in #2885

lihaoyi added a commit that referenced this issue Nov 19, 2023
…private nature (#2883)

Fixes #2844

Adds an integration test to cover this edge case, a few other
permutations of it that I could come up with, and other
`private`-related behavior. I'm not sure if I managed to catch all
different ways these methods can be mangled, but if I missed any we can
discover/add them later. This new test fails if I remove any part of the
code change in `GroupEvaluator.scala`

I'm actually not sure if this should be an integration test or a unit
test.

* Technically this code path should be exercised in unit tests as well I
think, and I think this failure mode should be triggerable.
* But it might be affected by exactly how the `def`s are wrapped in
nested `object`s/`trait`s/`class`es, which is something that unit tests
do not do realistically.
* Also, a unit test cannot exercise the failure code path when you try
and specify a `private` target from the command, since it will simply
not compile (being `private` and all), and AFAICT this is somewhere
we're missing coverage (unrelated to this specific failure mode)

So I went with integration test.

Pull request: #2883
@lefou lefou added this to the 0.11.6 milestone Nov 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug The issue represents an bug workaround-available
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants