put/filter: Error when comment on last line of DSL expression with no final newline

[archetyped]

Miller throws an error when the last line of a DSL expression file used with put and filter verbs contains a comment.

Example:

echo 'test\n(123)\n134' | mlr --c2p cat \
then put -f testing.mlr

Contents of testing.mlr:

$test = 0;
# End file with comment

Error message:

mlr: cannot parse DSL expression.
Parse error on token "# End file with comment" at line 2 column 1.
Please check for missing semicolon.
Expected one of:
  ␚ ; { unset filter print printn eprint eprintn dump edump tee emitf emit1
  emit ( emitp field_name $[ braced_field_name $[[ $[[[ full_srec oosvar_name
  @[ braced_oosvar_name full_oosvar all non_sigil_name arr bool float int
  map num str var funct + - .+ .- ! ~ string_literal regex_case_insensitive
  int_literal float_literal boolean_literal null_literal inf_literal nan_literal
  const_M_PI const_M_E panic [ ctx_IPS ctx_IFS ctx_IRS ctx_OPS ctx_OFS ctx_ORS
  ctx_FLATSEP ctx_NF ctx_NR ctx_FNR ctx_FILENAME ctx_FILENUM env call begin
  end if while do for break continue func subr return

This is the same error that occurs when an inline expression ends with a comment and is closed on the same line:

echo 'test\n(123)\n134' | mlr --c2p cat \
then put '
$test = 1;
# End inline expression with comment'

Of course, this is because everything to the end of the line is part of the comment (including the closing single-quote). Adding a newline after the comment resolves the issue:

echo 'test\n(123)\n134' | mlr --c2p cat \
then put '
$test = 1;
# End inline expression with comment
'

As files are separate entities of sorts, perhaps a trailing newline could be added automatically by Miller when pulling in expressions from external files?

Thanks.

[johnkerl]

@archetyped wow this is embarrassing 😬

mlr -n put '$x = 1 #' 

is a parse error. I'm truly surprised I hadn't caught this before ...

[johnkerl]

The lexer rule is right here:
https://github.com/johnkerl/miller/blob/main/internal/pkg/parsing/mlr.bnf#L65

!comment : '#'  {.} '\n' ;

so I don't have a rule to match #...EOF

I'll fix this.

[archetyped]

Thanks!