Remove language fragmentation -Y flags. Only -Xsource allowed henceforth.

[adriaanm]

I've counted close to 20 flags that change language semantics in various ways. 2^20 variants of the language is unacceptable complexity both for users and maintainers. As of 2.13, -Y flags will no longer be allowed to change language semantics. We will retain -Xsource as a migration/preview mechanism. We will clearly list which features can be turned on by specifying a future source version, and as soon as the first milestone of that version is released, the feature will be turned on by default.

A feature submitted to our codebase is only accepted if we expect it to become officially supported. -Xsource is meant to allow previewing and migrating to the new version that offers this feature by default. If the feature is not of high enough quality to be turned on along with the other -Xsource:N features, it is not ready for submission to scala/scala.

The official Scala compiler is not a vehicle for research/experimentation. There are plenty of alternatives for that. Having many rarely exercised code paths lead to bitrot and undertested code. They also set the wrong expectations for users.

The mechanism is also used by the presentation compiler to simulate older versions of the compiler while running the latest version to provide the IDE experience. We will retain this functionality where appropriate.

[Jasper-M]

Does that mean popular features like -Ypartial-unification will become part of flagless Scala 2.13, or that you'll need Typelevel Scala to use them?

[xeno-by]

What about -Yrangepos?

[adriaanm]

Partial unification, assuming it has proven itself (I’d say it has), will be on by default. Ditto for rangepos.

[jvican]

I really like this.

> $ scalac -Y    
Usage: scalac <options> <source files>

-- Notes on option parsing --
Boolean settings are always false unless set.
Where multiple values are accepted, they should be comma-separated.
  example: -Xplugin:option1,option2
<phases> means one or a comma-separated list of:
  (partial) phase names, phase ids, phase id ranges, or the string "all".
  example: -Xprint:all prints all phases.
  example: -Xprint:expl,24-26 prints phases explicitouter, closelim, dce, jvm.
  example: -Xprint:-4 prints only the phases up to typer.

Possible private options include:
  -Ybreak-cycles                           Attempt to break cycles encountered during typing
  -Ybrowse:<phases>                        Browse the abstract syntax tree after <phases>
  -Ycheck:<phases>                         Check the tree at the end of <phases>
  -Ycompact-trees                          Use compact tree printer when displaying trees.
  -Ydebug                                  Increase the quantity of debugging output.
  -Ydelambdafy:<strategy>                  Strategy used for translating lambdas into JVM code. Choices: (inline,method), default: method.
  -YdisableFlatCpCaching                   Do not cache flat classpath representation of classpath elements from jars across compiler instances.
  -Ydump-classes <dir>                     Dump the generated bytecode to .class files (useful for reflective compilation that utilizes in-memory classloaders).
  -Ygen-asmp <dir>                         Generate a parallel output directory of .asmp files (ie ASM Textifier output).
  -Yinfer-argument-types                   Infer types for arguments of overridden methods.
  -Ylog:<phases>                           Log operations during <phases>
  -Ylog-classpath                          Output information about what classpath is being applied.
  -Ymacro-debug-lite                       Trace essential macro-related activities.
  -Ymacro-debug-verbose                    Trace all macro-related activities: compilation, generation of synthetics, classloading, expansion, exceptions.
  -Ymacro-expand:<policy>                  Control expansion of macros, useful for scaladoc and presentation compiler. Choices: (normal,none,discard), default: normal.
  -Yno-adapted-args                        Do not adapt an argument list (either by inserting () or creating a tuple) to match the receiver.
  -Yno-completion                          Disable tab-completion in the REPL.
  -Yno-generic-signatures                  Suppress generation of generic signatures for Java.
  -Yno-imports                             Compile without importing scala.*, java.lang.*, or Predef.
  -Yno-predef                              Compile without importing Predef.
  -Yopt-inline-heuristics:<strategy>       Set the heuristics for inlining decisions. Choices: (at-inline-annotated,everything,default), default: default.
  -Yopt-log-inline <package/Class.method>  Print a summary of inliner activity; `_` to print all, prefix match to select.
  -Yopt-trace <package/Class.method>       Trace the optimizer progress for methods; `_` to print all, prefix match to select.
  -Yoverride-objects                       Allow member objects to be overridden.
  -Yoverride-vars                          Allow vars to be overridden.
  -Ypartial-unification                    Enable partial unification in type constructor inference
  -Ypatmat-exhaust-depth <n>               off
  -Ypresentation-any-thread                Allow use of the presentation compiler from any thread
  -Ypresentation-delay <n>                 Wait number of ms after typing before starting typechecking
  -Ypresentation-log <file>                Log presentation compiler events into file
  -Ypresentation-replay <file>             Replay presentation compiler events from file
  -Ypresentation-strict                    Do not report type errors in sources with syntax errors.
  -Ypresentation-verbose                   Print information about presentation compiler tasks.
  -Yprofile-destination <file>             where to send profiling output - specify a file, default is to the console.
  -Yprofile-enabled                        Enable profiling.
  -Yprofile-external-tool[:phases]         Enable profiling for a phase using an external tool hook. Generally only useful for a single phase <phases> (default: typer)
  -Yprofile-run-gc[:phases]                Run a GC between phases - this allows heap size to be accurate at the expense of more time. Specify a list of phases, or * <phases> (default: _)
  -Yrangepos                               Use range positions for syntax trees.
  -Yrecursion <n>                          Set recursion depth used when locking symbols.
  -Yreify-copypaste                        Dump the reified trees in copypasteable representation.
  -Yrepl-class-based                       Use classes to wrap REPL snippets instead of objects
  -Yrepl-outdir <path>                     Write repl-generated classfiles to given output directory (use "" to generate a temporary dir)
  -Yrepl-sync                              Do not use asynchronous code for repl startup
  -Yresolve-term-conflict:<strategy>       Resolve term conflicts. Choices: (package,object,error), default: error.
  -Yshow:<phases>                          (Requires -Xshow-class or -Xshow-object) Show after <phases>
  -Yshow-member-pos <output style>         Show start and end positions of members
  -Yshow-symkinds                          Print abbreviated symbol kinds next to symbol names.
  -Yshow-symowners                         Print owner identifiers next to symbol names.
  -Yshow-syms                              Print the AST symbol hierarchy after each phase.
  -Yshow-trees                             (Requires -Xprint:) Print detailed ASTs in formatted form.
  -Yshow-trees-compact                     (Requires -Xprint:) Print detailed ASTs in compact form.
  -Yshow-trees-stringified                 (Requires -Xprint:) Print stringifications along with detailed ASTs.
  -Yskip:<phases>                          Skip <phases>
  -Ystatistics:<_,phase,-phase>            Print compiler statistics for specific phases: `_' for all, `-Ystatistics:help' to list choices.
  -Ystop-after:<phases>                    Stop after <phases>
  -Ystop-before:<phases>                   Stop before <phases>
  -Yvirtpatmat                             Enable pattern matcher virtualization
  -Ywarn-adapted-args                      Warn if an argument list is modified to match the receiver.
  -Ywarn-dead-code                         Warn when dead code is identified.
  -Ywarn-extra-implicit                    Warn when more than one implicit parameter section is defined.
  -Ywarn-inaccessible                      Warn about inaccessible types in method signatures.
  -Ywarn-infer-any                         Warn when a type argument is inferred to be `Any`.
  -Ywarn-nullary-override                  Warn when non-nullary `def f()' overrides nullary `def f'.
  -Ywarn-nullary-unit                      Warn when nullary methods return Unit.
  -Ywarn-numeric-widen                     Warn when numerics are widened.
  -Ywarn-unused:<_,warning,-warning>       Enable or disable specific `unused' warnings: `_' for all, `-Ywarn-unused:help' to list choices.
  -Ywarn-unused-import                     Warn when imports are unused.
  -Ywarn-value-discard                     Warn when non-Unit expression results are unused.

Additional debug settings:
  -Ydoc-debug                              Trace all scaladoc activity.
  -Yide-debug                              Generate, validate and output trees using the interactive compiler.
  -Yissue-debug                            Print stack traces when a context issues an error.
  -Ypatmat-debug                           Trace pattern matching translation.
  -Ypos-debug                              Trace position validation.
  -Ypresentation-debug                     Enable debugging output for the presentation compiler.
  -Yquasiquote-debug                       Trace quasiquote-related activities.
  -Yreify-debug                            Trace reification.
  -Ytyper-debug                            Trace all type assignments.

Deprecated settings:
  -Ymacro-no-expand                        Don't expand macros. Might be useful for scaladoc and presentation compiler, but will crash anything which uses macros and gets past typer.
                                             deprecated: Use -Ymacro-expand:none

I guess those -Y flags related to debugging will stay or renamed? And perhaps things like -Yprofile-enabled should be moved to -X?

[adriaanm]

I only intend to change/remove -Y flags that affect language semantics. Debugging flags can stay since they don't fragment the language.

[jvican]

  -Yno-imports                             Compile without importing scala.*, java.lang.*, or Predef.
  -Yno-predef                              Compile without importing Predef.

These affect language semantics, though I guess they will stay too.

[adriaanm]

Yes, exceptions will be needed, just not 19 ;-) I hope the predef/imports options will be consolidated -- there's a PR for that.

[adriaanm]

-Y should be reserved for compiler development. No guarantees. The help text should not mention them, and they should never be used in a build (so that we can drop/change them without fear of breaking the community build and by extension the eco-system).

-X should be for the more adventurous user, and we should do our best to keep them stable acros compiler versoins. I expect -Ximports:... to replace the -Y flags that will disappear.

[som-snytt]

I guess no more override-objects. The repl options could be moved as part of the refactor. There's GenericRunnerSettings for -i, where the repl fragmentation flags could live.

[smarter]

-Y should be reserved for compiler development. No guarantees.

Sounds good to me, you may want to figure out what should happen to the billion flags recommended in https://tpolecat.github.io/2017/04/25/scalac-flags.html

[smarter]

I suggest replacing -Ywarn-xxx by -Wxxx and -Wno-xxx like gcc/clang

[som-snytt]

Flags to increase verbose output like -Ytyper-debug could become -Vtyper.

[milessabin]

I'm very sympathetic to your desire to minimize unnecessary language fragmentation. Nevertheless, there is a need to ship compilers with more fine-grained control over new or experimental features so that people can get experience with them which can in turn influence the design, or indeed acceptance of the feature.

Concretely, Typelevel Scala needs a mechanism of this sort. We could continue to use -Y flags, in which case we'd appreciate some flexibility when it comes to PRs which include new -Y flags ... we would be quite happy to commit to removing them prior to them being merged here. Or we could introduce a Typelevel-specific flag along the lines of -Xsource:typelevel(,feature)*.

We also need to think about Dotty. It's making language changes which it would be beneficial to backport to Scala (eg. scala/scala#6037) but which would break existing reasonable code if unconditionally enabled. Should there be an -Xsource:dotty for example?

I would also be fairly strongly in favour of deprecating -Xexperimental and -Xfuture in favour of a more uniform mechanism.

[SethTisue]

I think Typelevel Scala users might actually like and prefer a -Xsource:typelevel style flag that makes it lexically obvious which features are TLS features

[dwijnand]

I suggest replacing -Ywarn-xxx by -Wxxx and -Wno-xxx like gcc/clang

I think warnings like that should be moved out of the compiler to Scalafix linters (or even better rewrites), where (a) there are already many more possible warnings, opt-in/opt-out, and (b) they can become rewrites.

The one I'm concerned about is -Yno-adapted-args.

[som-snytt]

For typelevel and dotty options, -Z awaits its day in the sun.

If plugins could contribute prefixed options, avoiding ugly -P:name:options, and if some compiler-internal info were exposed as api, then it becomes appealing to farm out warnings. The compiler could do extra work under any -W without caring if there is a particular linter installed that consumes the data.

[schmitch]

I think warnings like that should be moved out of the compiler to Scalafix linters (or even better
rewrites), where (a) there are already many more possible warnings, opt-in/opt-out, and (b)
they can become rewrites.

I'm not sure if I like that. Additional tooling always comes with a cost.
And something like "-Xfatal-warnings", "-Ywarn-unused:imports", "-Ywarn-dead-code", "-Yno-adapted-args" is pretty nice, especially for the younger staff of a company who work at a new codebase.

[SethTisue]

heads up, -Yuse-stupid-types. your days are numbered. better get out of town now

[SethTisue]

I split the -Xfuture part of this out as #471 in the hopes of attracting a volunteer.

[SethTisue]

scala/scala#6505 is merged for 2.13.0-M4

[SethTisue]

I'm confused why -Yno-adapted-args got the axe. I've opened a separate ticket on it; let's discuss there: scala/bug#11110

[Krever]

I must say I strongly disagree with removal of linter-like flags. user shouldn't need to know ecosystem to write safe code. Linting has to be done by the compiler or in some other way that doesn't require knowing about Scalafix (e.g. by integrating it with sbt by default or something like that). When I start learning new language it's a matter of months before I get familiar with the ecosystem and I want get as much help from tooling as possible in this rough period.

It would be nice to have tpolecat's flags enabled by default btw.

[adriaanm]

Linting doesn't change the language. I'm fine with linter flags staying, and I'd like them unified under the -Xlint flag.

[som-snytt]

The linked PR proposes moving warnings to -W for discoverability. Probably no one discovers them except tpolecat. Note that his list goes stale, so a worthwhile curated list deserves a compiler or sbt plugin. Otherwise, just use -Xlint. Of course not all warnings are lint. For folks worried about migrating from wartremover, maybe there should be an option -Werror-wart.

I need as much help as possible from tooling in the rough period when I'm doing any coding at all.

[adriaanm]

Here's another candidate for removal as pointed out by @SethTisue in scala/bug#1472: -Yrecursion

[SethTisue]

retargeted for 2.13.1 since as far as I know, there is nothing remaining here that is essential for 2.13.0

[SethTisue]

It would be nice to have tpolecat's flags enabled by default btw.

@Krever this could be considered on a case-by-case basis; PRs welcome. (probably better as separate PRs unless there are no-brainers that can be batched.) note that "enabled by default" is one thing and "enabled by default as part of -Xlint" is another, the latter is an easier sell.

if you want to discuss linter flags further, let's do it on a new ticket, and keep this ticket restricted to language fragmentation flags

[SethTisue]

closing since the major work already happened. if there are remaining pieces we can have individual tickets for them.

[SethTisue]

note that the reduced set of available options got cleaned up and reorganized in scala/scala#7908