Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add solcOptimizerDetails option #683

Merged
merged 1 commit into from
Jan 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 4 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,30 +55,6 @@ npx hardhat coverage [command-options]

(Additional Hardhat-specific info can be found [here][37])

### Buidler [Deprecated]

**Add** the plugin in `buidler.config.js` ([Buidler docs][26])
```javascript
usePlugin('solidity-coverage')

module.exports = {
networks: {
coverage: {
url: 'http://localhost:8555'
}
},
}
```
**Run**
```
npx buidler coverage --network coverage [command-options]
```

**Buidler Project Examples:**

+ Simple: [buidler-metacoin][32]
+ More complex: [MolochDao/moloch][33]

### @openzeppelin/test-environment

OpenZeppelin have written their own coverage generation scripts for `test-environment` using the solidity-coverage API.
Expand Down Expand Up @@ -132,6 +108,7 @@ module.exports = {
| onTestsComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the tests complete, *before* Istanbul reports are generated. [More...][23]|
| onIstanbulComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the Istanbul reports are generated, *before* the ganache server is shut down. Useful if you need to clean resources up. [More...][23]|
| configureYulOptimizer | *Boolean* | false | (Experimental) Setting to `true` should resolve "stack too deep" compiler errors in large projects using ABIEncoderV2 |
| solcOptimizerDetails | *Object* | (Experimental) Must be used in combination with `configureYulOptimizer`Allows you define the [solc optimizer details][1001]. Useful if the default remedy for stack-too-deep errors doesn't work in your case (See FAQ below). |

[<sup>*</sup> Advanced use][14]

Expand All @@ -152,9 +129,8 @@ Common problems & questions:
+ [Running in CI][7]
+ [Running out of gas][13]
+ [Running out of time][6]
+ [Running out of stack] [1002] (Stack too deep)
+ [Running out of memory][5]
+ [Why are `require` statements highlighted as branch points?][8]


## Example reports
+ [metacoin][9] (Istanbul HTML)
Expand Down Expand Up @@ -239,4 +215,6 @@ $ yarn
[36]: https://hardhat.org/
[37]: https://github.com/sc-forks/solidity-coverage/blob/master/HARDHAT_README.md
[38]: https://github.com/sindresorhus/globby#globbing-patterns
[1001]: https://docs.soliditylang.org/en/v0.8.0/using-the-compiler.html#input-description
[1002]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-stack

41 changes: 35 additions & 6 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
* [Continuous Integration](#continuous-integration)
* [Running out of memory](#running-out-of-memory)
* [Running out of time](#running-out-of-time)
* [Running out of stack](#running-out-of-stack)
* [Notes on gas distortion](#notes-on-gas-distortion)
* [Notes on branch coverage](#notes-on-branch-coverage)

Expand Down Expand Up @@ -68,12 +69,12 @@ We use [Codecov.io][2] here as a coverage provider for our JS tests - they're gr

If your target contains dozens (and dozens) of large contracts, you may run up against Node's memory cap during the
contract compilation step. This can be addressed by setting the size of the memory space allocated to the command
when you run it.
when you run it.
```
// Truffle
$ node --max-old-space-size=4096 ./node_modules/.bin/truffle run coverage [options]

// Buidler
// Buidler
$ node --max-old-space-size=4096 ./node_modules/.bin/buidler coverage [options]
```

Expand All @@ -85,7 +86,7 @@ RuntimeError: memory access out of bounds
at wasm-function[833]:1152
at wasm-function[147]:18
at wasm-function[21880]:5

// solc 0.5.x
Downloading compiler version 0.5.16
* Line 1, Column 1
Expand All @@ -94,7 +95,7 @@ Downloading compiler version 0.5.16
Extra non-whitespace after JSON value.
```

...try setting the `measureStatementCoverage` option to `false` in `.solcoverjs`. This will reduce the footprint of
...try setting the `measureStatementCoverage` option to `false` in `.solcoverjs`. This will reduce the footprint of
the instrumentation solidity-coverage adds to your files. You'll still get line, branch and function coverage but the data Istanbul collects
for statements will be omitted.

Expand All @@ -121,13 +122,41 @@ module.exports = {
}
```

## Running out of stack

If your project is large, complex and uses ABI encoder V2 or Solidity >= V8, you may see "stack too deep" compiler errors when using solidity-coverage. This happens because:

+ solidity-coverage turns the solc optimizer off in order trace code execution correctly
+ some projects cannot compile unless the optimizer is turned on.

Work-arounds for this problem are tracked below. (These are only available in hardhat. If you're using hardhat and none of them work for you, please open an issue.)

**Work-around #1**
+ Set the `.solcoverjs` option `configureYulOptimizer` to `true`.

**Work-around #2**
+ Set the `.solcoverjs` option: `configureYulOptimizer` to `true`.
+ Set the `.solcoverjs` option: `solcOptimizerDetails` to:
+ ```js
{
peephole: false,
inliner: false,
jumpdestRemover: false,
orderLiterals: true, // <-- TRUE! Stack too deep when false
deduplicate: false,
cse: false,
constantOptimizer: false,
yul: false
}
```

## Notes on gas distortion

Solidity-coverage instruments by injecting statements into your code, increasing its execution costs.

+ If you are running gas usage simulations, they will **not be accurate**.
+ If you have hardcoded gas costs into your tests, some of them may **error**.
+ If your solidity logic constrains gas usage within narrow bounds, it may **fail**.
+ If your solidity logic constrains gas usage within narrow bounds, it may **fail**.
+ Solidity's `.send` and `.transfer` methods usually work fine though.

Using `estimateGas` to calculate your gas costs or allowing your transactions to use the default gas
Expand All @@ -153,6 +182,6 @@ Clearly, the coverage should be the same in these situations, as the code is (fu
If an `assert` or `require` is marked with an `I` in the coverage report, then during your tests the conditional is never true. If it is marked with an `E`, then it is never false.

[1]: https://coveralls.io/builds/25886294
[2]: https://codecov.io/
[2]: https://codecov.io/
[3]: https://user-images.githubusercontent.com/7332026/28502310-6851f79c-6fa4-11e7-8c80-c8fd80808092.png
[4]: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-170.md
2 changes: 2 additions & 0 deletions lib/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ class API {
this.istanbulFolder = config.istanbulFolder || false;
this.istanbulReporter = config.istanbulReporter || ['html', 'lcov', 'text', 'json'];

this.solcOptimizerDetails = config.solcOptimizerDetails;

this.setLoggingLevel(config.silent);
this.ui = new AppUI(this.log);
}
Expand Down
20 changes: 14 additions & 6 deletions plugins/hardhat.plugin.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ const {
// Toggled true for `coverage` task only.
let measureCoverage = false;
let configureYulOptimizer = false;
let instrumentedSources
let instrumentedSources;
let optimizerDetails;

// UI for the task flags...
const ui = new PluginUI();
Expand Down Expand Up @@ -63,11 +64,16 @@ subtask(TASK_COMPILE_SOLIDITY_GET_COMPILATION_JOB_FOR_FILE).setAction(async (_,
// This is fixes a stack too deep bug in ABIEncoderV2
// Experimental because not sure this works as expected across versions....
if (configureYulOptimizer) {
settings.optimizer.details = {
yul: true,
yulDetails: {
stackAllocation: true,
},
if (optimizerDetails === undefined) {
settings.optimizer.details = {
yul: true,
yulDetails: {
stackAllocation: true,
},
}
// Other configurations may work as well. This loads custom details from .solcoverjs
} else {
settings.optimizer.details = optimizerDetails;
}
}
}
Expand Down Expand Up @@ -101,6 +107,8 @@ task("coverage", "Generates a code coverage report for tests")
ui = new PluginUI(config.logger.log);
api = new API(utils.loadSolcoverJS(config));

optimizerDetails = api.solcOptimizerDetails;

// Catch interrupt signals
process.on("SIGINT", nomiclabsUtils.finish.bind(null, config, api, true));

Expand Down
12 changes: 11 additions & 1 deletion test/integration/projects/solc-8/.solcover.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,15 @@ module.exports = {
silent: process.env.SILENT ? true : false,
skipFiles: ['skipped-folder'],
istanbulReporter: ['json-summary', 'text'],
configureYulOptimizer: true
configureYulOptimizer: true,
solcOptimizerDetails: {
peephole: false,
inliner: false,
jumpdestRemover: false,
orderLiterals: true, // <-- TRUE! Stack too deep when false
deduplicate: false,
cse: false,
constantOptimizer: false,
yul: false
}
}