diff --git a/NEWS.md b/NEWS.md index 723819f15cbd4..c219828398cca 100644 --- a/NEWS.md +++ b/NEWS.md @@ -8,6 +8,19 @@ New language features Language changes ---------------- +* The interactive REPL now uses "soft scope" for top-level expressions: an assignment inside a + scope block such as a `for` loop automatically assigns to a global variable if one has been + defined already. This matches the behavior of Julia versions 0.6 and prior, as well as + [IJulia](https://github.com/JuliaLang/IJulia.jl). + Note that this only affects expressions interactively typed or pasted directly into the + default REPL ([#28789], [#33864]). + +* Outside of the REPL (e.g. in a file), assigning to a variable within a top-level scope + block is considered ambiguous if a global variable with the same name exists. + A warning is given if that happens, to alert you that the code will work differently + than in the REPL. + A new command line option `--warn-scope` controls this warning ([#33864]). + * Converting arbitrary tuples to `NTuple`, e.g. `convert(NTuple, (1, ""))` now gives an error, where it used to be incorrectly allowed. This is because `NTuple` refers only to homogeneous tuples (this meaning has not changed) ([#34272]). @@ -19,7 +32,6 @@ Language changes (in addition to the one that enters the help mode) to see the full docstring. ([#25930]) - Multi-threading changes ----------------------- diff --git a/base/options.jl b/base/options.jl index f7dec64f9b41d..9716d77a9dc57 100644 --- a/base/options.jl +++ b/base/options.jl @@ -41,6 +41,7 @@ struct JLOptions outputji::Ptr{UInt8} output_code_coverage::Ptr{UInt8} incremental::Int8 + warn_scope::Int8 end # This runs early in the sysimage != is not defined yet diff --git a/doc/src/manual/variables-and-scoping.md b/doc/src/manual/variables-and-scoping.md index c8b62f19a59cd..e6dfc6075b877 100644 --- a/doc/src/manual/variables-and-scoping.md +++ b/doc/src/manual/variables-and-scoping.md @@ -1,33 +1,36 @@ # [Scope of Variables](@id scope-of-variables) The *scope* of a variable is the region of code within which a variable is visible. Variable scoping -helps avoid variable naming conflicts. The concept is intuitive: two functions can both have arguments -called `x` without the two `x`'s referring to the same thing. Similarly, there are many other cases -where different blocks of code can use the same name without referring to the same thing. The -rules for when the same variable name does or doesn't refer to the same thing are called scope -rules; this section spells them out in detail. +helps avoid variable naming conflicts. The concept is intuitive: two functions can both have +arguments called `x` without the two `x`'s referring to the same thing. Similarly, there are many +other cases where different blocks of code can use the same name without referring to the same +thing. The rules for when the same variable name does or doesn't refer to the same thing are called +scope rules; this section spells them out in detail. Certain constructs in the language introduce *scope blocks*, which are regions of code that are eligible to be the scope of some set of variables. The scope of a variable cannot be an arbitrary -set of source lines; instead, it will always line up with one of these blocks. There are two -main types of scopes in Julia, *global scope* and *local scope*. The latter can be nested. The -constructs introducing scope blocks are: +set of source lines; instead, it will always line up with one of these blocks. There are two main +types of scopes in Julia, *global scope* and *local scope*. The latter can be nested. There is also +a distinction in Julia between constructs which introduce a "hard scope" and those which only +introduce a "soft scope", which affects whether shadowing a global variable by the same name is +allowed or not. ### [Scope constructs](@id man-scope-table) -Construct | Scope type | Scope blocks it may be nested in ------------- | ------------- |--------------------------- -[`module`](@ref), [`baremodule`](@ref) | global | global -interactive prompt (REPL) | global | global -(mutable) [`struct`](@ref), [`macro`](@ref) | local | global -[`for`](@ref), [`while`](@ref), [`try-catch-finally`](@ref try), [`let`](@ref) |local | global or local -functions (either syntax, anonymous & do-blocks) | local | global or local -comprehensions, broadcast-fusing | local | global or local +The constructs introducing scope blocks are: + +Construct | Scope type | Allowed within +----------|------------|--------------- +[`module`](@ref), [`baremodule`](@ref) | global | global +[`struct`](@ref) | local (soft) | global +[`for`](@ref), [`while`](@ref), [`try`](@ref try) | local (soft) | global or local +[`macro`](@ref) | local (hard) | global +[`let`](@ref), functions, comprehensions, generators | local (hard) | global or local Notably missing from this table are [begin blocks](@ref man-compound-expressions) and [if blocks](@ref man-conditional-evaluation) which do *not* introduce new scopes. -Both types of scopes follow somewhat different rules which will be explained below. +The three types of scopes follow somewhat different rules which will be explained below. Julia uses [lexical scoping](https://en.wikipedia.org/wiki/Scope_%28computer_science%29#Lexical_scoping_vs._dynamic_scoping), meaning that a function's scope does not inherit from its caller's scope, but from the scope in @@ -52,15 +55,21 @@ julia> Bar.foo() 1 ``` -Thus *lexical scope* means that the scope of variables can be inferred from the source code alone. +Thus *lexical scope* means that what a variable in a particular piece of code refers to can be +deduced from the code in which it appears alone and does not depend on how the program executes. A +scope nested inside another scope can "see" variables in all the outer scopes in which it is +contained. Outer scopes, on the other hand, cannot see variables in inner scopes. ## Global Scope -Each module introduces a new global scope, separate from the global scope of all other modules; -there is no all-encompassing global scope. Modules can introduce variables of other modules into -their scope through the [using or import](@ref modules) statements or through qualified access using the -dot-notation, i.e. each module is a so-called *namespace*. Note that variable bindings can only -be changed within their global scope and not from an outside module. +Each module introduces a new global scope, separate from the global scope of all other modules—there +is no all-encompassing global scope. Modules can introduce variables of other modules into their +scope through the [using or import](@ref modules) statements or through qualified access using the +dot-notation, i.e. each module is a so-called *namespace* as well as a first-class data structure +associating names with values. Note that while variable bindings can be read externally, they can only +be changed within the module to which they belong. As an escape hatch, you can always evaluate code +inside that module to modify a variable; this guarantees, in particular, that module bindings cannot +be modified externally by code that never calls `eval`. ```jldoctest julia> module A @@ -93,244 +102,359 @@ Note that the interactive prompt (aka REPL) is in the global scope of the module ## Local Scope -A new local scope is introduced by most code blocks (see above -[table](@ref man-scope-table) for a complete list). -A local scope inherits all the variables from a parent local scope, -both for reading and writing. -Unlike global scopes, local scopes are not namespaces, -thus variables in an inner scope cannot be retrieved from the parent scope through some sort of -qualified access. - -The following rules and examples pertain to local scopes. -A newly introduced variable in a local scope cannot be referenced by a parent scope. -For example, here the ``z`` is not introduced into the top-level scope: +A new local scope is introduced by most code blocks (see above [table](@ref man-scope-table) for a +complete list). Some programming languages require explicitly declaring new variables before using +them. Explicit declaration works in Julia too: in any local scope, writing `local x` declares a new +local variable in that scope, regardless of whether there is already a variable named `x` in an +outer scope or not. Declaring each new local like this is somewhat verbose and tedious, however, so +Julia, like many other languages, considers assignment to a new variable in a local scope to +implicitly declare that variable as a new local. Mostly this is pretty intuitive, but as with many +things that behave intuitively, the details are more subtle than one might naïvely imagine. + +When `x = ` occurs in a local scope, Julia applies the following rules to decide what the +expression means based on where the assignment expression occurs and what `x` already refers to at +that location: + +1. **Existing local:** If `x` is *already a local variable*, then the existing local `x` is +assigned; + +2. **Hard scope:** If `x` is *not already a local variable* and assignment occurs inside of any hard +scope construct (i.e. within a let block, function or macro body, comprehension, or generator), a new +local named `x` is created in the scope of the assignment; + +3. **Soft scope:** If `x` is *not already a local variable* and all of the scope constructs +containing the assignment are soft scopes (loops, `try`/`catch` blocks, or `struct` blocks), the +behavior depends on whether the global variable `x` is defined: + * if global `x` is **undefined**, a new local named `x` is created in the scope of the assignment; + * if global `x` is **defined**, the assignment is considered ambiguous: + * in non-interactive contexts (files, eval), an ambiguity warning is printed and a new local + is created; + * in interactive contexts (REPL, notebooks), the global variable `x` is assigned. + +You may note that in non-interactive contexts the hard and soft scope behaviors are identical except +that a warning is printed when an implicitly local variable (i.e. not declared with `local x`) +shadows a global. In interactive contexts, the rules follow a more complex heuristic for the sake of +convenience. This is covered in depth in examples that follow. + +Now that you know the rules, let's look at some examples. Each example is assumed to be evaluated in +a fresh REPL session so that the only globals in each snippet are the ones that are assigned in that +block of code. + +We'll begin with a nice and clear-cut situation—assignment inside of a hard scope, in this case a +function body, when no local variable by that name already exists: ```jldoctest -julia> for i = 1:10 - z = i +julia> function greet() + x = "hello" # new local + println(x) end +greet (generic function with 1 method) -julia> z -ERROR: UndefVarError: z not defined -``` - -!!! note - In this and all following examples it is assumed that their top-level is a global scope - with a clean workspace, for instance a newly started REPL. +julia> greet() +hello -Inner local scopes can, however, update variables in their parent scopes: - -```jldoctest -julia> for i = 1:1 - z = i - for j = 1:1 - z = 0 - end - println(z) - end -0 +julia> x # global +ERROR: UndefVarError: x not defined ``` -Inside a local scope a variable can be forced to be a new local variable using the [`local`](@ref) keyword: +Inside of the `greet` function, the assignment `x = "hello"` causes `x` to be a new local variable +in the function's scope. There are two relevant facts: the assignment occurs in local scope and +there is no existing local `x` variable. Since `x` is local, it doesn't matter if there is a global +named `x` or not. Here for example we define `x = 123` before defining and calling `greet`: ```jldoctest -julia> for i = 1:1 - x = i + 1 - for j = 1:1 - local x = 0 - end +julia> x = 123 # global +123 + +julia> function greet() + x = "hello" # new local println(x) end -2 -``` +greet (generic function with 1 method) -Inside a local scope a global variable can be assigned to by using the keyword [`global`](@ref): +julia> greet() +hello -```jldoctest -julia> for i = 1:10 - global z - z = i - end +julia> x # global +123 +``` -julia> z -10 +Since the `x` in `greet` is local, the value (or lack thereof) of the global `x` is unaffected by +calling `greet`. The hard scope rule doesn't care whether a global named `x` exists or not: +assignment to `x` in a hard scope is local (unless `x` is declared global). + +The next clear cut situation we'll consider is when there is already a local variable named `x`, in +which case `x = ` always assigns to this existing local `x`. The function `sum_to` computes +the sum of the numbers from one up to `n`: + +```julia +function sum_to(n) + s = 0 # new local + for i = 1:n + s = s + i # assign existing local + end + return s # same local +end ``` -The location of both the `local` and `global` keywords within the scope block is irrelevant. -The following is equivalent to the last example (although stylistically worse): +As in the previous example, the first assignment to `s` at the top of `sum_to` causes `s` to be a +new local variable in the body of the function. The `for` loop has its own inner local scope within +the function scope. At the point where `s = s + i` occurs, `s` is already a local variable, so the +assignment updates the existing `s` instead of creating a new local. We can test this out by calling +`sum_to` in the REPL: ```jldoctest -julia> for i = 1:10 - z = i - global z +julia> function sum_to(n) + s = 0 # new local + for i = 1:n + s = s + i # assign existing local + end + return s # same local end +sum_to (generic function with 1 method) -julia> z -10 -``` +julia> sum_to(10) +55 -The `local` and `global` keywords can also be applied to destructuring assignments, e.g. -`local x, y = 1, 2`. In this case the keyword affects all listed variables. - -In a local scope, all variables are inherited from its parent -global scope block unless: +julia> s # global +ERROR: UndefVarError: s not defined +``` - * an assignment would result in a modified *global* variable, or - * a variable is specifically marked with the keyword `local`. +Since `s` is local to the function `sum_to`, calling the function has no effect on the global +variable `s`. We can also see that the update `s = s + i` in the `for` loop must have updated the same +`s` created by the initialization `s = 0` since we get the correct sum of 55 for the integers 1 +through 10. -Thus global variables are only inherited for reading, not for writing: +Let's dig into the fact that the `for` loop body has its own scope for a second by writing a slightly +more verbose variation which we'll call `sum_to′`, in which we save the sum `s + i` in a variable `t` +before updating `s`: ```jldoctest -julia> x, y = 1, 2; - -julia> function foo() - x = 2 # assignment introduces a new local - return x + y # y refers to the global - end; - -julia> foo() -4 +julia> function sum_to′(n) + s = 0 # new local + for i = 1:n + t = s + i # new local `t` + s = t # assign existing local `s` + end + return s, @isdefined(t) + end +sum_to′ (generic function with 1 method) -julia> x -1 +julia> sum_to′(10) +(55, false) ``` -An explicit `global` is needed to assign to a global variable: +This version returns `s` as before but it also uses the `@isdefined` macro to return a boolean +indicating whether there is a local variable named `t` defined in the function's outermost local +scope. As you can see, there is no `t` defined outside of the `for` loop body. This is because of the +hard scope rule again: since the assignment to `t` occurs inside of a function, which +introduces a hard scope, the assignment causes `t` to become a new local variable in the local scope +where it appears, i.e. inside of the loop body. Even if there were a global named `t`, it would make +no difference—the hard scope rule isn't affected by anything in global scope. -!!! sidebar "Avoiding globals" - Avoiding changing the value of global variables is considered by many - to be a programming best-practice. - Changing the value of a global variable can cause "action at a distance", - making the behavior of a program harder to reason about. - This is why the scope blocks that introduce local scope require the `global` - keyword to declare the intent to modify a global variable. +Let's move onto some more ambiguous cases covered by the soft scope rule. We'll explore this by +extracting the bodies of the `greet` and `sum_to′` functions into soft scope contexts. First, let's put the +body of `greet` in a `for` loop—which is soft, rather than hard—and evaluate it in the REPL: ```jldoctest -julia> x = 1; - -julia> function foobar() - global x = 2 - end; - -julia> foobar(); +julia> for i = 1:3 + x = "hello" # new local + println(x) + end +hello +hello +hello julia> x -2 +ERROR: UndefVarError: x not defined ``` -Note that *nested functions* can modify their parent scope's *local* variables: +Since the global `x` is not defined when the `for` loop is evaluated, the first clause of the soft +scope rule applies and `x` is created as local to the `for` loop and therefore global `x` remains +undefined after the loop executes. Next, let's consider the body of `sum_to′` extracted into global +scope, fixing its argument to `n = 10` -```jldoctest -julia> x, y = 1, 2; - -julia> function baz() - x = 2 # introduces a new local - function bar() - x = 10 # modifies the parent's x - return x + y # y is global - end - return bar() + x # 12 + 10 (x is modified in call of bar()) - end; - -julia> baz() -22 - -julia> x, y # verify that global x and y are unchanged -(1, 2) ``` - -The reason to allow modifying local variables of parent scopes in -nested functions is to allow constructing [`closures`](https://en.wikipedia.org/wiki/Closure_%28computer_programming%29) -which have private state, for instance the `state` variable in the -following example: +s = 0 +for i = 1:10 + t = s + i + s = t +end +s +@isdefined(t) + ``` + +What does this code do? Hint: it's a trick question. The answer is "it depends." If this code is +entered interactively, it behaves the same way it does in a function body. But if the code appears +in a file, it prints an ambiguity warning and throws an undefined variable error. Let's see it +working in the REPL first: ```jldoctest -julia> let state = 0 - global counter() = (state += 1) - end; +julia> s = 0 # global +0 -julia> counter() -1 +julia> for i = 1:10 + t = s + i # new local `t` + s = t # assign global `s` + end -julia> counter() -2 +julia> s # global +55 + +julia> @isdefined(t) # global +false ``` -See also the closures in the examples in the next two sections. A variable, -such as `x` in the first example and `state` in the second, that is inherited -from the enclosing scope by the inner function is sometimes called a -*captured* variable. Captured variables can present [performance challenges -discussed in performance tips](@ref man-performance-captured). +The REPL approximates being in the body of a function by deciding whether assignment inside the loop +assigns to a global or creates new local based on whether a global variable by that name is defined +or not. If a global by the name exists, then the assignment updates it. If no global exists, then +the assignment creates a new local variable. In this example we see both cases in action: -The distinction between inheriting global scope and nesting local scope -can lead to some slight differences between functions -defined in local versus global scopes for variable assignments. -Consider the modification of the last example by moving `bar` to the global scope: +* There is no global named `t`, so `t = s + i` creates a new `t` that is local to the `for` loop; +* There is a global named `s`, so `s = t` assigns to it. + +The second fact is why execution of the loop changes the global value of `s` and the first fact is +why `t` is still undefined after the loop executes. Now, let's try evaluating this same code as +though it were in a file instead: ```jldoctest -julia> x, y = 1, 2; +julia> code = """ + s = 0 # global + for i = 1:10 + t = s + i # new local `t` + s = t # new local `s` with warning + end + s, # global + @isdefined(t) # global + """; + +julia> include_string(Main, code) +┌ Warning: Assignment to `s` in top-level block is ambiguous because an outer global binding by the same name already exists. Use `global s` to assign to the outer global `s` variable or use `local s` to force a new local by the same name. +└ @ string:4 +ERROR: LoadError: UndefVarError: s not defined +``` -julia> function bar() - x = 10 # local, no longer a closure variable - return x + y - end; +Here we use [`include_string`](@ref), to evaluate `code` as though it were the contents of a file. +We could also save `code` to a file and then call `include` on that file—the result would be the +same. As you can see, this behaves quite different from evaluating the same code in the REPL. Let's +break down what's happening here: + +* global `s` is defined with the value `0` before the loop is evaluated +* the assignment `s = t` occurs in a soft scope—a `for` loop outside of any function body or other hard + scope construct +* therefore the second clause of the soft scope rule applies, and the assignment is ambiguous so a + warning is emitted +* execution continues, making `s` local to the `for` loop body +* since `s` is local to the `for` loop, it is undefined when `t = s + i` is evaluated, causing an error +* evaluation stops there, but if it got to `s` and `@isdefined(t)`, it would return `0` and `false`. + +This demonstrates some important aspects of scope: in a scope, each variable can only have one +meaning, and that meaning is determined regardless of the order of expressions. The presence of the +expression `s = t` in the loop causes `s` to be local to the loop, which means that it is also local +when it appears on the right hand side of `t = s + i`, even though that expression appears first and is +evaluated first. One might imagine that the `s` on the first line of the loop could be global while +the `s` on the second line of the loop is local, but that's not possible since the two lines are in +the same scope block and each variable can only mean one thing in a given scope. + +#### On Soft Scope + +We have now covered all the local scope rules, but before wrapping up this section, perhaps a few +words should be said about why the ambiguous soft scope case is handled differently in interactive +and non-interactive contexts. There are two obvious questions one could ask: + +1. Why doesn't it just work like the REPL everywhere? +2. Why doesn't it just work like in files everywhere? And maybe skip the warning? + +In Julia ≤ 0.6, all global scopes did work like the current REPL: when `x = ` occurred in a +loop (or `try`/`catch`, or `struct` body) but outside of a function body (or `let` block or comprehension), +it was decided based on whether a global named `x` was defined or not whether `x` should be local to +the loop. This behavior has the advantage of being intuitive and convenient since it approximates +the behavior inside of a function body as closely as possible. In particular, it makes it easy to +move code back and forth between a function body and the REPL when trying to debug the behavior of a +function. However, it has some downsides. First, it's quite a complex behavior: many people over the +years were confused about this behavior and complained that it was complicated and hard both to +explain and understand. Fair point. Second, and arguably worse, is that it's bad for programming "at +scale." When you see a small piece of code in one place like this, it's quite clear what's going on: -julia> function quz() - x = 2 # local - return bar() + x # 12 + 2 (x is not modified) - end; +``` +s = 0 +for i = 1:10 + s += i +end +``` -julia> quz() -14 +Obviously the intention is to modify the existing global variable `s`. What else could it mean? +However, not all real world code is so short or so clear. We found that code like the following +often occurs in the wild: -julia> x, y # verify that global x and y are unchanged -(1, 2) ``` +x = 123 -Note that the above nesting rules do not pertain to type and macro definitions as they can only appear -at the global scope. There are special scoping rules concerning the evaluation of default and -keyword function arguments which are described in the [Function section](@ref man-functions). +# much later +# maybe in a different file -An assignment introducing a variable used inside a function, type or macro definition need not -come before its inner usage: +for i = 1:10 + x = "hello" + println(x) +end -```jldoctest -julia> f = y -> y + a; +# much later +# maybe in yet another file +# or maybe back in the first one where `x = 123` -julia> f(3) -ERROR: UndefVarError: a not defined -Stacktrace: -[...] +y = x + 234 +``` -julia> a = 1 -1 +It's far less clear what should happen here. Since `x + "hello"` is a method error, it seems +probable that the intention is for `x` to be local to the `for` loop. But runtime values and what +methods happen to exist cannot be used to determine the scopes of variables. With the Julia ≤ 0.6 +behavior, it's especially concerning that someone might have written the `for` loop first, had it +working just fine, but later when someone else adds a new global far away—possibly in a different +file—the code suddenly changes meaning and either breaks noisily or, worse still, silently does the +wrong thing. This kind of "spooky action at a distance" is something that good programming language +designs should prevent. + +So in Julia 1.0, we simplified the rules for scope: in any local scope, assignment to a name that +wasn't already a local variable created a new local variable. This eliminated the notion of soft +scope entirely as well as removing the potential for spooky action. We uncovered and fixed a significant number of bugs due to the removal of soft scope, vindicating the choice to get rid of it. +And there was much rejoicing! Well, no, not really. Because some people were +angry that they now had to write: -julia> f(3) -4 +``` +s = 0 +for i = 1:10 + global s += i +end ``` -This behavior may seem slightly odd for a normal variable, but allows for named functions -- which -are just normal variables holding function objects -- to be used before they are defined. This -allows functions to be defined in whatever order is intuitive and convenient, rather than forcing -bottom up ordering or requiring forward declarations, as long as they are defined by the time -they are actually called. As an example, here is an inefficient, mutually recursive way to test -if positive integers are even or odd: +Do you see that `global` annotation in there? Hideous. Obviously this situation could not be +tolerated. But seriously, there are two main issues with requiring `global` for this kind of +top-level code: -```jldoctest -julia> even(n) = (n == 0) ? true : odd(n - 1); +1. It's no longer convenient to copy and paste the code from inside a function body into the REPL + to debug it—you have to add `global` annotations and then remove them again to go back; -julia> odd(n) = (n == 0) ? false : even(n - 1); +2. Beginners will write this kind of code without the `global` and have no idea why their code + doesn't work—the error that they get is that `s` is undefined, which does not seem to enlighten + anyone who happens to make this mistake. -julia> even(3) -false +As of Julia 1.5, this code works without the `global` annotation in interactive contexts like the +REPL or Jupyter notebooks (just like Julia 0.6) and in files and other non-interactive contexts, it +prints this very direct warning: -julia> odd(3) -true -``` +> Assignment to `s` in top-level block is ambiguous because an outer global binding by the same name +already exists. Use `global s` to assign to the outer global `s` variable or use `local s` to force +a new local by the same name. -Julia provides built-in, efficient functions to test for oddness and evenness called [`iseven`](@ref) -and [`isodd`](@ref) so the above definitions should only be considered to be examples of scope, -not efficient design. +This addresses both issues while preserving the "programming at scale" benefits of the 1.0 behavior: +global variables have no spooky effect on the meaning of code that may be far away; in the REPL +copy-and-paste debugging works and beginners don't have any issues; any time someone either forgets +a `global` annotation or accidentally shadows an existing global with a local in a soft scope, +which would be confusing anyway, they get a nice clear warning. + +An important property of this design is that any code that executes in a file without a warning will +behave the same way in a fresh REPL. And on the flip side, if you take a REPL session and save it to +file, if it behaves differently than it did in the REPL, then you will get a warning. ### Let Blocks @@ -409,11 +533,11 @@ julia> let Since `let` introduces a new scope block, the inner local `x` is a different variable than the outer local `x`. -### For Loops and Comprehensions +### Loops and Comprehensions -`for` loops, `while` loops, and [comprehensions](@ref man-comprehensions) have the following behavior: any new variables +In loops and [comprehensions](@ref man-comprehensions), new variables introduced in their body scopes are freshly allocated for each loop iteration, as if the loop body -were surrounded by a `let` block: +were surrounded by a `let` block, as demonstrated by this example: ```jldoctest julia> Fs = Vector{Any}(undef, 2); @@ -435,6 +559,7 @@ A `for` loop or comprehension iteration variable is always a new variable: julia> function f() i = 0 for i = 1:3 + # empty end return i end; @@ -450,6 +575,7 @@ This can be done conveniently by adding the keyword `outer`: julia> function f() i = 0 for outer i = 1:3 + # empty end return i end; @@ -549,12 +675,11 @@ WARNING: redefining constant a 1 ``` -Note that although sometimes possible, changing the value of a `const` variable -is strongly discouraged, and is intended only for convenience during -interactive use. -Changing constants can cause various problems or unexpected behaviors. -For instance, if a method references a constant and is already -compiled before the constant is changed then it might keep using the old value: +Note that although sometimes possible, changing the value of a `const` variable is strongly +discouraged, and is intended only for convenience during interactive use. Changing constants can +cause various problems or unexpected behaviors. For instance, if a method references a constant and +is already compiled before the constant is changed, then it might keep using the old value: + ```jldoctest julia> const x = 1 1 diff --git a/src/ast.c b/src/ast.c index 352760775096f..783cb3996fef4 100644 --- a/src/ast.c +++ b/src/ast.c @@ -131,6 +131,17 @@ static jl_value_t *scm_to_julia(fl_context_t *fl_ctx, value_t e, jl_module_t *mo static value_t julia_to_scm(fl_context_t *fl_ctx, jl_value_t *v); static jl_value_t *jl_expand_macros(jl_value_t *expr, jl_module_t *inmodule, struct macroctx_stack *macroctx, int onelevel); +value_t fl_defined_julia_global(fl_context_t *fl_ctx, value_t *args, uint32_t nargs) +{ + // tells whether a var is defined in and *by* the current module + argcount(fl_ctx, "defined-julia-global", nargs, 1); + (void)tosymbol(fl_ctx, args[0], "defined-julia-global"); + jl_ast_context_t *ctx = jl_ast_ctx(fl_ctx); + jl_sym_t *var = jl_symbol(symbol_name(fl_ctx, args[0])); + jl_binding_t *b = jl_get_module_binding(ctx->module, var); + return (b != NULL && b->owner == ctx->module) ? fl_ctx->T : fl_ctx->F; +} + value_t fl_current_module_counter(fl_context_t *fl_ctx, value_t *args, uint32_t nargs) { jl_ast_context_t *ctx = jl_ast_ctx(fl_ctx); @@ -138,6 +149,16 @@ value_t fl_current_module_counter(fl_context_t *fl_ctx, value_t *args, uint32_t return fixnum(jl_module_next_counter(ctx->module)); } +value_t fl_julia_current_file(fl_context_t *fl_ctx, value_t *args, uint32_t nargs) +{ + return symbol(fl_ctx, jl_filename); +} + +value_t fl_julia_current_line(fl_context_t *fl_ctx, value_t *args, uint32_t nargs) +{ + return fixnum(jl_lineno); +} + // Check whether v is a scalar for purposes of inlining fused-broadcast // arguments when lowering; should agree with broadcast.jl on what is a // scalar. When in doubt, return false, since this is only an optimization. @@ -197,9 +218,12 @@ value_t fl_julia_logmsg(fl_context_t *fl_ctx, value_t *args, uint32_t nargs) } static const builtinspec_t julia_flisp_ast_ext[] = { + { "defined-julia-global", fl_defined_julia_global }, { "current-julia-module-counter", fl_current_module_counter }, { "julia-scalar?", fl_julia_scalar }, { "julia-logmsg", fl_julia_logmsg }, + { "julia-current-file", fl_julia_current_file }, + { "julia-current-line", fl_julia_current_line }, { NULL, NULL } }; @@ -227,6 +251,7 @@ static void jl_init_ast_ctx(jl_ast_context_t *ast_ctx) JL_NOTSAFEPOINT ctx->task = NULL; ctx->module = NULL; set(symbol(fl_ctx, "*depwarn-opt*"), fixnum(jl_options.depwarn)); + set(symbol(fl_ctx, "*scopewarn-opt*"), fixnum(jl_options.warn_scope)); } // There should be no GC allocation while holding this lock @@ -859,9 +884,10 @@ jl_value_t *jl_parse_eval_all(const char *fname, } // expand non-final expressions in statement position (value unused) expression = - fl_applyn(fl_ctx, 3, - symbol_value(symbol(fl_ctx, iscons(cdr_(ast)) ? "jl-expand-to-thunk-stmt" : "jl-expand-to-thunk")), - expression, symbol(fl_ctx, jl_filename), fixnum(jl_lineno)); + fl_applyn(fl_ctx, 4, + symbol_value(symbol(fl_ctx, "jl-expand-to-thunk-warn")), + expression, symbol(fl_ctx, jl_filename), fixnum(jl_lineno), + iscons(cdr_(ast)) ? fl_ctx->T : fl_ctx->F); } jl_get_ptls_states()->world_age = jl_world_counter; form = scm_to_julia(fl_ctx, expression, inmodule); @@ -1171,6 +1197,13 @@ JL_DLLEXPORT jl_value_t *jl_macroexpand1(jl_value_t *expr, jl_module_t *inmodule return expr; } +// Lower an expression tree into Julia's intermediate-representation. +JL_DLLEXPORT jl_value_t *jl_expand(jl_value_t *expr, jl_module_t *inmodule) +{ + return jl_expand_with_loc(expr, inmodule, "none", 0); +} + +// Lowering, with starting program location specified JL_DLLEXPORT jl_value_t *jl_expand_with_loc(jl_value_t *expr, jl_module_t *inmodule, const char *file, int line) { @@ -1183,10 +1216,25 @@ JL_DLLEXPORT jl_value_t *jl_expand_with_loc(jl_value_t *expr, jl_module_t *inmod return expr; } -// Lower an expression tree into Julia's intermediate-representation. -JL_DLLEXPORT jl_value_t *jl_expand(jl_value_t *expr, jl_module_t *inmodule) +// Same as the above, but printing warnings when applicable +JL_DLLEXPORT jl_value_t *jl_expand_with_loc_warn(jl_value_t *expr, jl_module_t *inmodule, + const char *file, int line) { - return jl_expand_with_loc(expr, inmodule, "none", 0); + JL_TIMING(LOWERING); + JL_GC_PUSH1(&expr); + expr = jl_copy_ast(expr); + expr = jl_expand_macros(expr, inmodule, NULL, 0); + jl_ast_context_t *ctx = jl_ast_ctx_enter(); + fl_context_t *fl_ctx = &ctx->fl; + JL_AST_PRESERVE_PUSH(ctx, old_roots, inmodule); + value_t arg = julia_to_scm(fl_ctx, expr); + value_t e = fl_applyn(fl_ctx, 4, symbol_value(symbol(fl_ctx, "jl-expand-to-thunk-warn")), arg, + symbol(fl_ctx, file), fixnum(line), fl_ctx->F); + expr = scm_to_julia(fl_ctx, e, inmodule); + JL_AST_PRESERVE_POP(ctx, old_roots); + jl_ast_ctx_leave(ctx); + JL_GC_POP(); + return expr; } // expand in a context where the expression value is unused diff --git a/src/jlfrontend.scm b/src/jlfrontend.scm index 994ee150bfe5b..5ab4d91944a31 100644 --- a/src/jlfrontend.scm +++ b/src/jlfrontend.scm @@ -29,6 +29,11 @@ '(error "malformed expression")))) thk)) +;; this is overwritten when we run in actual julia +(define (defined-julia-global v) #f) +(define (julia-current-file) 'none) +(define (julia-current-line) 0) + ;; parser entry points ;; parse one expression (if greedy) or atom, returning end position @@ -128,16 +133,38 @@ (begin0 (expand-toplevel-expr-- e file line) (set! *in-expand* last)))))) -; expand a piece of raw surface syntax to an executable thunk -(define (jl-expand-to-thunk expr file line) +;; used to collect warnings during lowering, which are usually discarded +;; unless logging is requested +(define lowering-warning (lambda lst (void))) + +;; expand a piece of raw surface syntax to an executable thunk + +(define (expand-to-thunk- expr file line) (error-wrap (lambda () (expand-toplevel-expr expr file line)))) +(define (expand-to-thunk-stmt- expr file line) + (expand-to-thunk- (if (toplevel-only-expr? expr) + expr + `(block ,expr (null))) + file line)) + +(define (jl-expand-to-thunk-warn expr file line stmt) + (let ((warnings '())) + (with-bindings + ((lowering-warning (lambda lst (set! warnings (cons lst warnings))))) + (begin0 + (if stmt + (expand-to-thunk-stmt- expr file line) + (expand-to-thunk- expr file line)) + (for-each (lambda (args) (apply julia-logmsg args)) + (reverse warnings)))))) + +(define (jl-expand-to-thunk expr file line) + (expand-to-thunk- expr file line)) + (define (jl-expand-to-thunk-stmt expr file line) - (jl-expand-to-thunk (if (toplevel-only-expr? expr) - expr - `(block ,expr (null))) - file line)) + (expand-to-thunk-stmt- expr file line)) (define (jl-expand-macroscope expr) (error-wrap (lambda () @@ -215,6 +242,8 @@ (if (equal? instead "") "" (string #\newline "Use `" instead "` instead.")))) +(define *scopewarn-opt* 1) + ; Corresponds to --depwarn 0="no", 1="yes", 2="error" (define *depwarn-opt* 1) diff --git a/src/jloptions.c b/src/jloptions.c index bed7c60074f0d..858940b8e14a7 100644 --- a/src/jloptions.c +++ b/src/jloptions.c @@ -71,7 +71,8 @@ jl_options_t jl_options = { 0, // quiet NULL, // output-ji NULL, // output-code_coverage 0, // incremental - 0 // image_file_specified + 0, // image_file_specified + JL_OPTIONS_WARN_SCOPE_ON // ambiguous scope warning }; static const char usage[] = "julia [switches] -- [programfile] [args...]\n"; @@ -109,7 +110,8 @@ static const char opts[] = // error and warning options " --depwarn={yes|no|error} Enable or disable syntax and method deprecation warnings (\"error\" turns warnings into errors)\n" - " --warn-overwrite={yes|no} Enable or disable method overwrite warnings\n\n" + " --warn-overwrite={yes|no} Enable or disable method overwrite warnings\n" + " --warn-scope={yes|no} Enable or disable warning for ambiguous top-level scope\n\n" // code generation options " -C, --cpu-target Limit usage of CPU features up to ; set to \"help\" to see the available options\n" @@ -169,6 +171,7 @@ JL_DLLEXPORT void jl_parse_opts(int *argcp, char ***argvp) opt_output_bc, opt_depwarn, opt_warn_overwrite, + opt_warn_scope, opt_inline, opt_polly, opt_trace_compile, @@ -225,6 +228,7 @@ JL_DLLEXPORT void jl_parse_opts(int *argcp, char ***argvp) { "output-incremental",required_argument, 0, opt_incremental }, { "depwarn", required_argument, 0, opt_depwarn }, { "warn-overwrite", required_argument, 0, opt_warn_overwrite }, + { "warn-scope", required_argument, 0, opt_warn_scope }, { "inline", required_argument, 0, opt_inline }, { "polly", required_argument, 0, opt_polly }, { "trace-compile", required_argument, 0, opt_trace_compile }, @@ -554,7 +558,15 @@ JL_DLLEXPORT void jl_parse_opts(int *argcp, char ***argvp) else if (!strcmp(optarg,"no")) jl_options.warn_overwrite = JL_OPTIONS_WARN_OVERWRITE_OFF; else - jl_errorf("julia: invalid argument to --warn-overwrite={yes|no|} (%s)", optarg); + jl_errorf("julia: invalid argument to --warn-overwrite={yes|no} (%s)", optarg); + break; + case opt_warn_scope: + if (!strcmp(optarg,"yes")) + jl_options.warn_scope = JL_OPTIONS_WARN_SCOPE_ON; + else if (!strcmp(optarg,"no")) + jl_options.warn_scope = JL_OPTIONS_WARN_SCOPE_OFF; + else + jl_errorf("julia: invalid argument to --warn-scope={yes|no} (%s)", optarg); break; case opt_inline: if (!strcmp(optarg,"yes")) diff --git a/src/julia-syntax.scm b/src/julia-syntax.scm index 5c936908afb06..db7da9ff726f8 100644 --- a/src/julia-syntax.scm +++ b/src/julia-syntax.scm @@ -1102,13 +1102,14 @@ (cdr (cadr e)) (list (cadr e)))) -(define (expand-let e) +(define (expand-let e (hard? #t)) (let ((ex (caddr e)) - (binds (let-binds e))) + (binds (let-binds e)) + (hs (if hard? '((hardscope)) '()))) (expand-forms (if (null? binds) - `(scope-block (block ,ex)) + `(scope-block (block ,@hs ,ex)) (let loop ((binds (reverse binds)) (blk ex)) (if (null? binds) @@ -1118,7 +1119,7 @@ ;; just symbol -> add local (loop (cdr binds) `(scope-block - (block + (block ,@hs (local ,(car binds)) ,blk)))) ((and (length= (car binds) 3) @@ -1132,7 +1133,7 @@ (error "invalid let syntax")) (loop (cdr binds) `(scope-block - (block + (block ,@hs ,(if (expr-contains-eq name (caddar binds)) `(local ,name) ;; might need a Box for recursive functions `(local-def ,name)) @@ -1145,14 +1146,15 @@ (if (expr-contains-eq vname (caddar binds)) (let ((tmp (make-ssavalue))) `(scope-block - (block (= ,tmp ,(caddar binds)) + (block ,@hs + (= ,tmp ,(caddar binds)) (scope-block (block (local-def ,(cadar binds)) (= ,vname ,tmp) ,blk))))) `(scope-block - (block + (block ,@hs (local-def ,(cadar binds)) (= ,vname ,(caddar binds)) ,blk)))))) @@ -1167,12 +1169,12 @@ `(block (= ,temp ,(caddr (car binds))) (scope-block - (block + (block ,@hs ,@(map (lambda (v) `(local-def ,v)) vars) (= ,(cadr (car binds)) ,temp) ,blk)))) `(scope-block - (block + (block ,@hs ,@(map (lambda (v) `(local-def ,v)) vars) ,(car binds) ,blk)))))) @@ -1584,7 +1586,7 @@ (if (null? (cdr lhss)) `(break-block loop-cont - (let (block ,@(map (lambda (v) `(= ,v ,v)) copied-vars)) + (soft-let (block ,@(map (lambda (v) `(= ,v ,v)) copied-vars)) ,body)) `(scope-block ,body)))) `(block (= ,coll ,(car itrs)) @@ -1849,6 +1851,7 @@ 'function expand-function-def '-> expand-arrow 'let expand-let + 'soft-let (lambda (e) (expand-let e #f)) 'macro expand-macro-def 'struct expand-struct-def 'try expand-try @@ -2447,12 +2450,20 @@ (define (find-local-def-decls e) (find-decls 'local-def e)) (define (find-global-decls e) (find-decls 'global e)) +(define (find-scope-decl e kind) + (expr-contains-p + (lambda (x) (and (pair? x) (eq? (car x) kind) x)) + e + (lambda (x) (not (and (pair? x) + (memq (car x) '(lambda scope-block module toplevel))))))) + (define (check-valid-name e) (or (valid-name? e) (error (string "invalid identifier name \"" e "\"")))) -(define (make-scope (lam #f) (args '()) (locals '()) (globals '()) (sp '()) (renames '()) (prev #f)) - (vector lam args locals globals sp renames prev)) +(define (make-scope (lam #f) (args '()) (locals '()) (globals '()) (sp '()) (renames '()) (prev #f) + (soft? #f) (hard? #f) (implicit-globals '()) (warn-vars #f)) + (vector lam args locals globals sp renames prev soft? hard? implicit-globals warn-vars)) (define (scope:lam s) (aref s 0)) (define (scope:args s) (aref s 1)) (define (scope:locals s) (aref s 2)) @@ -2460,6 +2471,10 @@ (define (scope:sp s) (aref s 4)) (define (scope:renames s) (aref s 5)) (define (scope:prev s) (aref s 6)) +(define (scope:soft? s) (aref s 7)) +(define (scope:hard? s) (aref s 8)) +(define (scope:implicit-globals s) (aref s 9)) +(define (scope:warn-vars s) (aref s 10)) (define (var-kind var scope) (if scope @@ -2472,6 +2487,14 @@ (define (in-scope? var scope) (not (eq? (var-kind var scope) 'none))) +(define (warn-var?! v scope) + (and scope + (let ((w (scope:warn-vars scope))) + (if (and w (has? w v)) + (begin (del! w v) + #t) + (warn-var?! v (scope:prev scope)))))) + (define (all-local-names scope) (define (all-lists s) (if s @@ -2480,7 +2503,7 @@ (apply append (all-lists scope))) ;; returns lambdas in the form (lambda (args...) (locals...) body) -(define (resolve-scopes- e scope (sp '())) +(define (resolve-scopes- e scope (sp '()) (loc #f)) (cond ((symbol? e) (let lookup ((scope scope)) (if scope @@ -2509,6 +2532,8 @@ (if (not (in-scope? (cadr e) scope)) (error "no outer local variable declaration exists for \"for outer\"")) '(null)) + ((or (eq? (car e) 'softscope) (eq? (car e) 'hardscope)) + '(null)) ((eq? (car e) 'locals) (let* ((names (filter (lambda (v) (and (not (gensym? v)) @@ -2537,20 +2562,34 @@ (let* ((blok (cadr e)) ;; body of scope-block expression (lam (scope:lam scope)) (argnames (lam:vars lam)) + (toplevel? (and (null? argnames) (eq? e (lam:body lam)))) (current-locals (caddr lam)) ;; locals created so far in our lambda (globals (find-global-decls blok)) + (assigned (find-assigned-vars blok)) (locals-def (find-local-def-decls blok)) (local-decls (find-local-decls blok)) - (toplevel? (and (null? argnames) (eq? e (lam:body lam)))) + (hard? (and (null? argnames) (or (scope:hard? scope) + (find-scope-decl blok 'hardscope)))) + (soft? (and (null? argnames) (not hard?) + (let ((ss (find-scope-decl blok 'softscope))) + (cond ((not ss) (scope:soft? scope)) + ((equal? (cadr ss) '(true)) #t) + ((equal? (cadr ss) '(false)) #f) + (else (scope:soft? scope)))))) + (nonloc-assigned (filter (lambda (v) (and (not (memq v locals-def)) + (not (memq v local-decls)))) + assigned)) + (implicit-globals (if toplevel? nonloc-assigned '())) (implicit-locals (filter (if toplevel? ;; make only assigned gensyms implicitly local at top level some-gensym? (lambda (v) (and (memq (var-kind v scope) '(none static-parameter)) - (not (memq v locals-def)) - (not (memq v local-decls)) + (not (and soft? + (or (memq v (scope:implicit-globals scope)) + (defined-julia-global v)))) (not (memq v globals))))) - (find-assigned-vars blok))) + nonloc-assigned)) (locals-nondef (delete-duplicates (append local-decls implicit-locals))) (need-rename? (lambda (vars) (filter (lambda (v) (or (memq v current-locals) (in-scope? v scope))) @@ -2561,7 +2600,22 @@ (renamed (map named-gensy need-rename)) (renamed-def (map named-gensy need-rename-def)) (newnames (append (diff locals-nondef need-rename) renamed)) - (newnames-def (append (diff locals-def need-rename-def) renamed-def))) + (newnames-def (append (diff locals-def need-rename-def) renamed-def)) + (warn-vars + (and (not toplevel?) (null? argnames) (not soft?) (not hard?) + (let ((vars (filter (lambda (v) + (and (or (memq v (scope:implicit-globals scope)) + (defined-julia-global v)) + (eq? (var-kind v scope) 'none) + (not (memq v globals)))) + nonloc-assigned))) + (if (pair? vars) + (let ((t (table))) + (for-each (lambda (v) (put! t v #t)) + vars) + t) + #f))))) + (for-each (lambda (v) (if (or (memq v locals-def) (memq v local-decls)) (error (string "variable \"" v "\" declared both local and global")))) @@ -2591,7 +2645,15 @@ '() (append (map cons need-rename renamed) (map cons need-rename-def renamed-def)) - scope))) + scope + (and soft? (null? argnames)) + hard? + (if toplevel? + implicit-globals + (scope:implicit-globals scope)) + warn-vars) + '() + loc)) (append! (map (lambda (v) `(local ,v)) newnames) (map (lambda (v) `(local-def ,v)) newnames-def))) )) @@ -2599,10 +2661,10 @@ (error "\"module\" expression not at top level")) ((eq? (car e) 'break-block) `(break-block ,(cadr e) ;; ignore type symbol of break-block expression - ,(resolve-scopes- (caddr e) scope))) ;; body of break-block expression + ,(resolve-scopes- (caddr e) scope '() loc))) ;; body of break-block expression ((eq? (car e) 'with-static-parameters) `(with-static-parameters - ,(resolve-scopes- (cadr e) scope (cddr e)) + ,(resolve-scopes- (cadr e) scope (cddr e) loc) ,@(cddr e))) ((and (eq? (car e) 'method) (length> e 2)) `(method @@ -2610,8 +2672,26 @@ ,(resolve-scopes- (caddr e) scope) ,(resolve-scopes- (cadddr e) scope (method-expr-static-parameters e)))) (else + (if (and (eq? (car e) '=) (symbol? (cadr e)) + scope (null? (lam:vars (scope:lam scope))) + (warn-var?! (cadr e) scope) + (= *scopewarn-opt* 1)) + (let* ((v (cadr e)) + (loc (extract-line-file loc)) + (line (if (= (car loc) 0) (julia-current-line) (car loc))) + (file (if (eq? (cadr loc) 'none) (julia-current-file) (cadr loc)))) + (lowering-warning + 1000 'warn (symbol (string file line)) file line + (string "Assignment to `" v "` in top-level block is ambiguous " + "because an outer global binding by the same name already exists." + " Use `global " v "` to assign to the outer global `" v + "` variable or use `local " v "` to force a new " + "local by the same name.")))) (cons (car e) - (map (lambda (x) (resolve-scopes- x scope)) + (map (lambda (x) + (if (linenum? x) + (set! loc x)) + (resolve-scopes- x scope '() loc)) (cdr e)))))) (define (resolve-scopes e) (resolve-scopes- e #f)) diff --git a/src/julia.h b/src/julia.h index e34f98d2093e7..96d315e53d402 100644 --- a/src/julia.h +++ b/src/julia.h @@ -1615,6 +1615,8 @@ JL_DLLEXPORT jl_value_t *jl_load_file_string(const char *text, size_t len, JL_DLLEXPORT jl_value_t *jl_expand(jl_value_t *expr, jl_module_t *inmodule); JL_DLLEXPORT jl_value_t *jl_expand_with_loc(jl_value_t *expr, jl_module_t *inmodule, const char *file, int line); +JL_DLLEXPORT jl_value_t *jl_expand_with_loc_warn(jl_value_t *expr, jl_module_t *inmodule, + const char *file, int line); JL_DLLEXPORT jl_value_t *jl_expand_stmt(jl_value_t *expr, jl_module_t *inmodule); JL_DLLEXPORT jl_value_t *jl_expand_stmt_with_loc(jl_value_t *expr, jl_module_t *inmodule, const char *file, int line); @@ -1934,6 +1936,7 @@ typedef struct { const char *output_code_coverage; int8_t incremental; int8_t image_file_specified; + int8_t warn_scope; } jl_options_t; extern JL_DLLEXPORT jl_options_t jl_options; @@ -1990,6 +1993,9 @@ JL_DLLEXPORT int jl_generating_output(void) JL_NOTSAFEPOINT; #define JL_OPTIONS_WARN_OVERWRITE_OFF 0 #define JL_OPTIONS_WARN_OVERWRITE_ON 1 +#define JL_OPTIONS_WARN_SCOPE_OFF 0 +#define JL_OPTIONS_WARN_SCOPE_ON 1 + #define JL_OPTIONS_POLLY_ON 1 #define JL_OPTIONS_POLLY_OFF 0 diff --git a/src/toplevel.c b/src/toplevel.c index b4ccf1bfb80a4..0c2f3a660c413 100644 --- a/src/toplevel.c +++ b/src/toplevel.c @@ -631,7 +631,7 @@ jl_value_t *jl_toplevel_eval_flex(jl_module_t *JL_NONNULL m, jl_value_t *e, int size_t last_age = ptls->world_age; if (!expanded && jl_needs_lowering(e)) { ptls->world_age = jl_world_counter; - ex = (jl_expr_t*)jl_expand_with_loc(e, m, jl_filename, jl_lineno); + ex = (jl_expr_t*)jl_expand_with_loc_warn(e, m, jl_filename, jl_lineno); ptls->world_age = last_age; } jl_sym_t *head = jl_is_expr(ex) ? ex->head : NULL; diff --git a/stdlib/REPL/src/REPL.jl b/stdlib/REPL/src/REPL.jl index 514ce37c14400..241f7dd0e2e4f 100644 --- a/stdlib/REPL/src/REPL.jl +++ b/stdlib/REPL/src/REPL.jl @@ -73,6 +73,24 @@ mutable struct REPLBackend new(repl_channel, response_channel, in_eval) end +function softscope!(ex) + if ex isa Expr + h = ex.head + if h === :toplevel + for i = 1:length(ex.args) + ex.args[i] = softscope!(ex.args[i]) + end + elseif h in (:meta, :import, :using, :export, :module, :error, :incomplete, :thunk) + return ex + else + return Expr(:block, Expr(:softscope, true), ex) + end + end + return ex +end + +const repl_ast_transforms = Any[softscope!] + function eval_user_input(@nospecialize(ast), backend::REPLBackend) lasterr = nothing Base.sigatomic_begin() @@ -83,6 +101,9 @@ function eval_user_input(@nospecialize(ast), backend::REPLBackend) put!(backend.response_channel, (lasterr,true)) else backend.in_eval = true + for xf in repl_ast_transforms + ast = xf(ast) + end value = Core.eval(Main, ast) backend.in_eval = false # note: use jl_set_global to make sure value isn't passed through `expand` diff --git a/stdlib/Test/src/Test.jl b/stdlib/Test/src/Test.jl index 3877b5dd68203..0600dbff0bb0b 100644 --- a/stdlib/Test/src/Test.jl +++ b/stdlib/Test/src/Test.jl @@ -1110,7 +1110,9 @@ function testset_beginend(args, tests, source) try # RNG is re-seeded with its own seed to ease reproduce a failed test Random.seed!(RNG.seed) - $(esc(tests)) + let + $(esc(tests)) + end catch err err isa InterruptException && rethrow() # something in the test block threw an error. Count that as an @@ -1200,7 +1202,9 @@ function testset_forloop(args, testloop, source) Random.seed!(RNG.seed) local tmprng = copy(RNG) try - $(Expr(:for, Expr(:block, [esc(v) for v in loopvars]...), blk)) + let + $(Expr(:for, Expr(:block, [esc(v) for v in loopvars]...), blk)) + end finally # Handle `return` in test body if !first_iteration diff --git a/test/syntax.jl b/test/syntax.jl index 7b1e7dbe22e5e..f807415ce5cbd 100644 --- a/test/syntax.jl +++ b/test/syntax.jl @@ -2000,3 +2000,93 @@ end == 1 @test Meta.parse("'\U0061'") == 'a' test_parseerror("''", "invalid empty character literal") test_parseerror("'abc'", "character literal contains multiple characters") + +# optional soft scope: #28789, #33864 + +@test @eval begin + $(Expr(:softscope, true)) + x28789 = 0 # new global included in same expression + for i = 1:2 + x28789 += i + end + x28789 +end == 3 + +y28789 = 1 # new global defined in separate top-level input +@eval begin + $(Expr(:softscope, true)) + for i = 1:10 + y28789 += i + end +end +@test y28789 == 56 + +@eval begin + $(Expr(:softscope, true)) + for i = 10:10 + z28789 = i + end + @test z28789 == 10 + z28789 = 0 # new global assigned after loop but in same soft scope +end + +@eval begin + $(Expr(:softscope, true)) + let y28789 = 0 # shadowing with let + y28789 = 1 + end +end +@test y28789 == 56 + +@eval begin + $(Expr(:softscope, true)) + let + y28789 = -8 # let is always a hard scope + end +end +@test y28789 == 56 + +@eval begin + $(Expr(:softscope, true)) + for y28789 in 0:0 + for x in 2:2 + for y in 3:3 + z28789 = 42 # assign to global despite several loops + end + end + end +end +@test z28789 == 42 + +@eval begin + $(Expr(:softscope, true)) + let x = 0 + ww28789 = 88 # not global + let y = 3 + ww28789 = 89 + end + @test ww28789 == 89 + end +end +@test !@isdefined(ww28789) + +@eval begin + $(Expr(:softscope, true)) + for x = 0 + ww28789 = 88 # not global + for y = 3 + ww28789 = 89 + end + @test ww28789 == 89 + end +end +@test !@isdefined(ww28789) + +@eval begin + $(Expr(:softscope, true)) + function f28789() + z28789 = 43 + end + f28789() +end +@test z28789 == 42