Only run/interpret/compile individual files

[nomeata]

this fixes #404 (issue not found).

The command line still accepts multiple files, but just handles then
independently and in sequence. In pipeline.ml, there is always only
ever one input file.

No change to generated WebAssembly output.

[nomeata]

Oh, right, this doesn't work yet with the test suite of the produce exchange, due to #400. So this needs to be shelved until we have a solution there.

[rossberg]

Hm, we should continue to allow multiple files with both -r and -i, simply as sequential execution. That's usually possible in other interpreters and REPLs and it's pretty convenient when you want to preload some libs.

[nomeata]

it's pretty convenient when you want to preload some libs.

if libraries are supposed to be imported, then shouldn’t that be the way to get them in the repl as well?

[rossberg]

Moving forward, imports should apply to compiled modules, while this allows using them as source programs. Sometimes that's useful. Why forbid it?

[nomeata]

Moving forward, imports should apply to compiled modules

Clearly influence by Haskell, I don't agree. Whether a file has been compiled already or not should not affect import, but rather determine whether asc recompiles lib.as to lib.o or something.

But this boils down our diverging vision for how imports and libraries and modules should work and interact.

while this allows using them as source programs. Sometimes that's useful. Why forbid it?

Simplicity and consistency.

If you can’t run asc lib.as prog.as -o out.wasm (with the files somehow combined) then why should asc -i lib.as prog.as work?

[rossberg]

If you can’t run asc lib.as prog.as -o out.wasm

But I should be able to (eventually)! That would be invoking static linking after compilation, the same way cc and other standard batch compilers work.

[nomeata]

But I should be able to (eventually)! That would be invoking static linking after compilation, the same way cc and other standard batch compilers work.

Since when is C a valid inspiration for us? :-)

If main.as depends on lib.as, then running asc on main.as should be enough (as it is in Haskell, Ocaml, and surely many other modern languages).

[nomeata]

Rebased this on #448, which makes the tests pass again.

[rossberg]

It's not just C, it's pretty much every batch compiler following standard Unix tool chain conventions. That includes ocamlc (not to be confused with ocamlbuild, which is a separate tool on top). Dependency analysis and such is best left to separate tools, because it may involve more than one processor.

[nomeata]

So if asc is no longer a developer-facing tool, but rather hidden behind something else (but what? And who is going to write that other thing? Or do you want our poor developers to write Makefiles?), then why even have convenience features like processing multiple files in one command line?

Also, wouldn't it then be the task of that other tool to make sure dependencies are compiled and up-to-date, even when using the interpreter?

How would the interpreter even work with this kind of strongly decoupled separate compilation? Would it interpret the .wasm files for the libraries? Because if does look at the source, then we need the dependency analysis code anyways.

[rossberg]

Why would it no longer be a dev-facing tool?

I just want to follow standard conventions. If anything, that helps devs to combine their tool chain in whatever way they see fit. I've used various different options on different occasions.

How would the interpreter even work with this kind of strongly decoupled separate compilation? Would it interpret the .wasm files for the libraries?

Ah, you are right. The interpreter will probably need to load imports as source, because, well, it's an interpreter. :) But for some tasks it's still useful to be able to just dump a couple of files on the command line. (In fact, I got complaints when one of my interpreters did not allow that at first.)

[nomeata]

Oh, maybe there is a misunderstanding:

Hm, we should continue to allow multiple files with both -r and -i, simply as sequential execution

It still does for -r! For some reason I thought you wanted the concatenation semantics, where asc -r a.as b.as puts the definitions of a.as into scope in b.as. But you didn’t actually say that; sorry.

I only disabled it for -i, because it was unclear to me what you want to have in scope at the end, and because I wanted get rid of the concatenation semantics. (Maybe -i should actually not allow any files besides, essentially, stdin?)

[nomeata]

The interpreter will probably need to load imports as source, because, well, it's an interpreter.

But if the interpreter does it anyways, why not have the compiler do the same thing (possibly using cached .o files and .asi files, once we have interface files), like GHC does?

[rossberg]

For some reason I thought you wanted the concatenation semantics, where asc -r a.as b.as puts the definitions of a.as into scope in b.as.

That is indeed what I want, otherwise it's a bit useless. Essentially, the semantic should be that of pasting each file as a single input into the REPL.

The interpreter will probably need to load imports as source, because, well, it's an interpreter.

But if the interpreter does it anyways, why not have the compiler do the same thing (possibly using cached .o files and .asi files, once we have interface files), like GHC does?

That would require the interpreter to embed a Wasm VM. How would their heaps interoperate?

[nomeata]

Essentially, the semantic should be that of pasting each file as a single input into the REPL.

I find it odd if that’s not the semantics anywhere else. Doesn’t that mean that b.as is likely useless in all other contexts, e.g. when compiled on its own, or import’ed, because has unbound variable from a.as? How would an SDK make sense of this file? How would an editor know how to type-check it?

This is a mode of operation that I don’t see as relevant for the language and target audience that we are developing here.

For some reason I thought you wanted the concatenation semantics, where asc -r a.as b.as puts the definitions of a.as into scope in b.as.

The interpreter will probably need to load imports as source, because, well, it's an interpreter.

But if the interpreter does it anyways, why not have the compiler do the same thing (possibly using cached .o files and .asi files, once we have interface files), like GHC does?

That would require the interpreter to embed a Wasm VM. How would their heaps interoperate?

No, I meant it the other way: If the interpreter already requires source files to be present, and imports need to point to source files (not compiled artifacts), and the dependency analysis code needs to be there, then why not have the compiler do the same thing, and make incremental compilation (i.e. producing .o and .asi files) something that happens under the hood that the developer normally doesn’t need to directly deal with.

[rossberg]

I find it odd if that’s not the semantics anywhere else. Doesn’t that mean that b.as is likely useless in all other contexts

It's not useless if it contains e.g. a module or actor definition, which can be named. It's also useful for scripting sorts of applications, although I agree that is less of a use case for AS (despite the name :) ).

why not have the compiler do the same thing

Well, it's part of execution, not the pipeline.

One big difference is that the interpreter does not have to produce out files, which complicates matters for the compiler. For example, where put implicitly generated out files? Source dir, working dir, configurable dir, computed dir? What if there already is a wasm file but the AS file is newer? Should the compiler check, ignore, etc? Do we need mode flags for that? Globally or for individual files?

[nomeata]

It's not useless if it contains e.g. a module or actor definition, which can be named.

I am not talking about what it exports, but rather what it imports: If you concatenate a.as and b.as, then b can refer to names defined in a, but this is completely implicit.

Or do you mean that asc -i a.as b.as opens a prompt with the definitions from a.as and b.as in scope, but without the definitions from a.as in scope in b.as? (In other words, asc -i a.as is equivalent to asc -i followed by open import "a.as", if we had open imports)

One big difference is that the interpreter does not have to produce out files, which complicates matters for the compiler. For example, where put implicitly generated out files? Source dir, working dir, configurable dir, computed dir?

Sure, these questions need to be answered. But there is enough precedence for inspiration.

And even if we defer them to some asc-build tool, we have to answer them, so we don't gain anything here.

What if there already is a wasm file but the AS file is newer? Should the compiler check, ignore, etc?

Recompile of course! Not having to worry about these things as a developer is a big plus of this model. Otherwise you keep editing your lib.as, but you test still fail, until you notice an hour later that you forgot to regenerate lib.wasm.

Do we need mode flags for that? Globally or for individual files?

Not sure if we need a flag. Maybe there is a -c flag that really only looks at one file, for integration into existing build systems, but I don’t see that as a priority for 0.5 or 1.0.

A normal developer should just run asc foo.as and get foo.wasm, without any worry about whether libraries have been compiled or how incremental compilation works under the hood.

[rossberg]

If you concatenate a.as and b.as, then b can refer to names defined in a, but this is completely implicit.

True, that wouldn't be necessary in the purely library use cases, but it may still be convenient in others. Also, I think it's nasty to introduce yet another scoping semantics that you cannot even emulate syntactically. The nested scopes semantics of the REPL seems good enough to reuse, is simple, and is standard.

Not having to worry about these things as a developer is a big plus of this model.

The minus is that this is implicit magic, which I'm not a big fan of, at least not for the basic blocks of the tool chain. But I wouldn't oppose, as long as we get the basics right.

[nomeata]

Also, I think it's nasty to introduce yet another scoping semantics that you cannot even emulate syntactically.

Here I agree. (although I am not fond of the REPL model for anything but the repl).

I still don’t see how this is an important use case. Who will want to sequence multiple files in the REPL that way?