Enough Debugging data for tools to switch to via-IR

[NunoFilipeSantos]

What

The via-IR pipeline, especially with enabled optimizer, is hard to process for tools, since it doesn't preserve as many fixed patterns in the bytecode as the legacy pipeline. We want to improve this situation in a forward compatible manner by providing tooling with adequate and sufficient debugging data to reconstruct the relations between Solidity code, Yul code and bytecode.

Why

We consider good tooling support a prerequisite for switching to via-IR by default, which we want to do as soon as possible to reduce our maintenance burden.

How

We initiated a debugging data working group (currently spearheaded by @gnidan and @jtohman), with the goal to get to a consensus among tooling about what debugging information precisely and in which format would be most helpful for tooling. We intend to support this process as much as possible, which will hopefully result in a specification of debugging data, which we will then generate from the compiler.

Tasks

• Support the debugging data working group in defining the required debugging information and format.
• Implement support for the resulting debugging data in the compiler, i.e. emit the debugging data.
• IR-based compilation w/o optimization uses way too many stack slots, easily hits stack too deep errors #12533
• Output Yul ASTs #9590

Older issues for consideration

• Compiling in Yul mode doesn't produce sources output #11043
• Unmapped code doesn't end up unmapped when called from a constructor #10935

[cameel]

We could add #9590 as one of the subtasks here. Yul ASTs were requested at the debug data meeting at Devcon Bogota and I think it's pretty clear we need them. The format is also clear I think (same as the Solidity ASTs, which we already output). This is a bit we could work on even without waiting for other decisions about debug data.

[ekpyron]

Linking this to #12533

[NunoFilipeSantos]

We could add #9590 as one of the subtasks here. Yul ASTs were requested at the debug data meeting at Devcon Bogota and I think it's pretty clear we need them. The format is also clear I think (same as the Solidity ASTs, which we already output). This is a bit we could work on even without waiting for other decisions about debug data.

@ekpyron WDYT? Is this small enough, or are we expanding the scope too much?

[ekpyron]

We'll only actually define the scope here properly after the debugging data working group comes back to us at the end of this quarter depending on the outcome of that.

[ekpyron]

But yes, #9590 likely falls under this.

[NunoFilipeSantos]

Added to the tasklist.

[ekpyron]

Linking #11043 as well for consideration.