Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix undercompilation upon ctor change #19911

Merged
merged 2 commits into from
Mar 17, 2024
Merged

Conversation

eed3si9n
Copy link
Member

Fixes #19910
Fixes sbt/zinc#1334

Problem

Scala 3 compiler registers special zincMangledName for constructors for the purpose of incremental compilation. Currently the zincMangledName contains the package name, which does not match the use site tracking,
thereby causing undercompilation during incremental compilation after a ctor change, like adding a parameter.

There is an existing scripted test, but coincidentally the test class does NOT include packages, so the test ends up passing.

Solution

  1. This PR reproduces the issue by adding package name to the test.
  2. This also fixes the problem by changing the zincMangedName to sym.owner.name ++ ";init;".

Note about zincMangedName

zincMangedName in general is a good idea, which adds the class name into otherwise common name <init>.
By making the name more unique it prevents overcompilation when one of the ctors changes in a file.

**Problem**
Scala 3 compiler registers special `zincMangledName` for
constructors for the purpose of incremental compilation.
Currently the `zincMangledName` contains the package name,
which does not match the use site tracking,
thereby causing undercompilation during incremental compilation
after a ctor change, like adding a parameter.

There is an existing scripted test, but coincidentally
the test class does NOT include packages, so the test ends up passing.

**Solution**
This PR reproduces the issue by adding package name to the test.
This also fixes the problem by changing the `zincMangedName`
to `sym.owner.name ++ ";init;"`.
@smarter
Copy link
Member

smarter commented Mar 10, 2024

Currently the zincMangledName contains the package name, which does not match the use site tracking

Hmm, but shouldn't the use-site tracking also go through zincMangledName? If it isn't right now, then can we avoid this discrepancy? sbt/zinc#288 also takes the package name into account so it seems like it should be possible in theory.

@eed3si9n
Copy link
Member Author

Yes, you're right. As far as I can tell, the undercompilation bug does not exist in Scala 2.x, and adding a new parameter correctly fails to compile using the https://github.com/sake92/metals-mill-scala3-class-params-repro example on Scala 2.12.19:

[info] compiling 1 Scala source to /private/tmp/metals-mill-scala3-class-params-repro/scalaHelloWorld/target/scala-2.12/classes ...
[error] /private/tmp/metals-mill-scala3-class-params-repro/scalaHelloWorld/src/main/scala/example/Hello.scala:5:17: not enough arguments for constructor MyService: (str: String, x: Int)example.MyService.
[error] Unspecified value parameter x.
[error]   val service = new MyService("fssdfs")
[error]                 ^
[error] one error found
[error] (hello / Compile / compileIncremental) Compilation failed

Also Scala 3 addUsedName(...) uses zincMangledName so in theory they should match up:

def addUsedName(sym: Symbol, includeSealedChildren: Boolean = false)(using Context): Unit =
addUsedRawName(sym.zincMangledName, includeSealedChildren)

so something about Scala 3 bridge is causing this.

@bishabosha
Copy link
Member

bishabosha commented Mar 13, 2024

I did some extra testing and it seems the actual problem is the . separator in the fullName, if I replace it with e.g. :: then it's fine, so I'll suggest to do that to keep the fine-grained-ness

@eed3si9n
Copy link
Member Author

eed3si9n commented Mar 13, 2024

@bishabosha Feel free to take whatever from this PR and send a new one, or add commits to your liking on this PR.

@bishabosha bishabosha requested review from sjrd and removed request for bishabosha March 13, 2024 15:20
@bishabosha bishabosha assigned sjrd and unassigned bishabosha Mar 13, 2024
@bishabosha
Copy link
Member

bishabosha commented Mar 13, 2024

@sjrd I added a new name kind, kept it private to sbt package to avoid unintentional leaking to TASTy. The other option would be creating a new SimpleName by replacing . in the mangledString but I think that would unnecessarily add extra entries to the name cache

@bishabosha bishabosha force-pushed the wip/parameter branch 2 times, most recently from e44c89a to e12bc65 Compare March 13, 2024 15:26
@sjrd
Copy link
Member

sjrd commented Mar 13, 2024

If the problem is the . in string form, consider changing how you serialize the Names as Strings for the purposes of giving them to zinc. This would avoid the extra NameKind.

@@ -306,7 +306,7 @@ class ExtractUsedNamesSpecification {
// All classes extend Object
"Object",
// All classes have a default constructor called <init>
"java.lang.Object;init;",
"java::lang::Object;init;",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be useful to use ; as a separator like in sbt/zinc#288 so that builds involving both scala2 and scala3 projects have incremental compilation working correctly? (Which makes me wonder if sbt/zinc#288 is missing a change in the java api extractor)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh that's a good point seeing as dependency tracking is across modules

@bishabosha
Copy link
Member

@smarter @sjrd now I avoid the new name kind, and we use ; in the mangled string. In the ideal case we would avoid creating new termName entries, but that can be improved later.

The main problem is that ExtractDependencies uses the name as a key in the cache, which is more efficient than converting names to string, so we would maybe need a new structure that can accommodate strings and names

@bishabosha bishabosha requested a review from smarter March 16, 2024 08:49
compiler/src/dotty/tools/dotc/sbt/ExtractAPI.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/sbt/package.scala Outdated Show resolved Hide resolved
@smarter smarter added the backport:nominated If we agree to backport this PR, replace this tag with "backport:accepted", otherwise delete it. label Mar 16, 2024
Copy link
Member

@smarter smarter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks everyone!

Instead of avoiding fully qualified names,
use a different separator in zincMangledName.
@bishabosha bishabosha merged commit b782cbf into scala:main Mar 17, 2024
17 checks passed
@eed3si9n eed3si9n deleted the wip/parameter branch March 17, 2024 15:10
@eed3si9n
Copy link
Member Author

Thanks @bishabosha for getting this across the finish line!

@Kordyjan Kordyjan added this to the 3.4.2 milestone Mar 28, 2024
WojciechMazur added a commit that referenced this pull request Jul 4, 2024
Backports #19911 to the LTS branch.

PR submitted by the release tooling.
[skip ci]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:nominated If we agree to backport this PR, replace this tag with "backport:accepted", otherwise delete it.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Class constructor parameter undercompilation Changing class parameters doesnt recompile dependent code
5 participants