internal/eval: strange let declaration behaviour

[myitcv]

What version of CUE are you using (cue version)?

$ cue version
cue version +a4a38ed8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What did you do?

Consider the following:

-- broken.cue --
package p

import "strings"

#f: {
	#in: string
	let List = strings.Split(#in, "/")
	List[len(List)-1]
}

test: #f & {_, #in: "hello/world/you"}
-- working.cue --
package p

import "strings"

#f: {
	#in: string
	let List = strings.Split(#in, "/")
	List
}

test: #f & {_, #in: "hello/world/you"}

cue export working.cue behaves as expected:

$ cue export !$
cue export working.cue
{
    "test": [
        "hello",
        "world",
        "you"
    ]
}

However, cue eval working.cue looks incorrect; notice the let declaration has shifted scope. This could, however, be a presentational issue:

$ cue eval working.cue
import "strings"

let List = strings.Split(#in, "/")
#f: {
    List
    #in: string
}
test: {
    #in: "hello/world/you"
    ["hello", "world", "you"]
}

Now consider cue export broken.cue:

$ cue export broken.cue
test: undefined field: #in:
    ./broken.cue:7:27

The trigger for this appears to be the fact that List is referenced from the index expression (noting however that List + List also fails with the same error). Contrast working.cue where the plain use of List works just fine.

cue eval broken.cue also looks "broken" in the same way as working.cue:

$ cue eval broken.cue
import "strings"

let List = strings.Split(#in, "/")
#f: {
    List[len(List)-1]
    #in: string
}
test: #f & {
    _
    #in: "hello/world/you"
}

What did you expect to see?

• cue eval showing the let expression in the correct scope
• cue export to not error in either case.

What did you see instead?

As above.

[myitcv]

I forgot to note in the initial description that this came to light as a result of the discussion in #1096 - thanks @PierreR

[PierreR]

@myitcv Is this related to the fact that let needs to be enclosed in an embedded ?

// KO
test:
    let Name = "hello"
    Name

// OK
test: {
    let Name = "hello"
    Name
}

Or is this working as intended ?

[myitcv]

That is working as intended AFAICT. From the spec:

Within a struct, a let clause binds an identifier to the given expression.

Within the scope of the identifier, the identifier refers to the locally declared expression. The expression is evaluated in the scope it was declared.

[mpvl]

I think

test:
    let Name = "hello"
    Name

Is a separate issue. It should probably not result in a valid parse.

[mpvl]

Minimal reproducer of the above:

#a: ["a", "b"]
let List = #a
List[1]

[mpvl]

Analysis: let gets evaluated before the all arcs have been added to the struct. Normally this is fine and evaluation is postponed. But for lets the evaluations are cached and thus it never recovers.

A solution would be to not store incomplete errors. But this may lead to exponential runtime.

A better solution would be treat let expressions as field, but just handle their scope differently. The same treatment intended for optional fields, which is necessary for structure sharing.

[mpvl]

Will submit a fix for the original problem. Note that the unrelated issue of a let in a non-scoped field value position is still open.

[mpvl]

See also:

Name: 1
test:
    let Name = "hello"
    Name

[myitcv]

Further reproducer of the let moving out of scope (running with 9982526):

exec cue def x.cue
stdin stdout
exec cue def -

-- x.cue --
#TypeBool: {
	_args: required: bool
	let Args = _args
	if !Args.required {
		default: bool
	}
}
#TypePrimitive: {
	_args: {
		required: bool
	}
	let Args = _args
	{"bool": #TypeBool & {_args: required: Args.required}}
}

Which gives the output:

> exec cue def x.cue
[stdout]
#TypeBool: {
        _args: {
                required: bool
        }
        if !Args.required {
                default: bool
        }
}
let Args_1 = _args
#TypePrimitive: {
        _args: {
                required: bool
        }
        bool: #TypeBool & {
                _args: {
                        required: Args_1.required
                }
        }
}
> stdin stdout
> exec cue def -
[stderr]
#TypeBool: reference "Args" not found:
    -:5:6
let[]: reference "_args" not found:
    -:9:14
[exit status 1]

Notice the failure for the second cue def which fails as a result of one let being dropped from #TypeBool but also the let previously within #TypePrimitive moving up a scope. It's almost like the two similarly named let's are being incorrectly collapsed into one.

[mpvl]

Regarding the latter, the issue seems to occur when the reference is nested, so

Foo: {
	_args: bool
	let Args = _args
	bool:  Args.required
}

gives the right input, but using {bool: Args.required}, bool: [Args.required], or bool: {args: required} does not. More nesting pushes it up higher, if possible.