feat: Add hardhat deployment infrastructure #2950

nicholaspai · 2021-05-06T15:25:13Z

Motivation

Add example deployment, etherscan-verification, and post-deployment setup sprints, all using the native hardhat tools as much as possible!

Summary

Follow up features:

Figure out best way to export deployed addresses, bytecode, and ABI similar to core/networks/X.json.
Take full advantage of hardhat-deploy tags to make deploying different systems (test, prod, sink-oracle, etc.) possible in one line CLI commands.
Make verification work on Matic, might require some custom logic because Matic has an Etherscan substitute. We can hopefully use sourcify to verify programatically on Matic.
Merge core/hardhat.config.js with common/HardhatConfig.js which generally should serve the same role. It would be great if we could share HardhatConfig.js similar to how we use TruffleConfig.js across packages.
Use deployments in tests.

Details

NOTE FOR REVIEWERS:

The entrypoint for this PR is in README.md and package.json to see the new CLI scripts added.

Testing

Check a box to describe how you tested these changes and list the steps for reviewers to test.

Ran end-to-end test, running the code as in production
New unit tests created
Existing tests adequate, no new tests required
All existing tests pass
Untested

Issue(s)

Fixes #XXXX

Signed-off-by: Nick Pai <[email protected]>

Co-authored-by: Matt Rice <[email protected]>

Signed-off-by: Nick Pai <[email protected]>

packages/core/deployments/goerli/solcInputs/87af3bddb96fa5cfcfc99d0e6fd39498.json

Signed-off-by: Nick Pai <[email protected]>

mrice32 · 2021-05-07T21:38:26Z

packages/common/src/Ethers.js

@@ -0,0 +1,20 @@
+const { hexlify, toUtf8Bytes } = require("ethers/utils");
+
+function stringToBytes32(text) {


Just curious, is this copied from somewhere? I feel like I recognize the code

Got it from this ethers issue

Ah yep! I think @Tulun used this some snippet in the voter dapp. May be useful (later) to pull this out into a separate lib.

Yeah thats why I started this Ethers.js file to start collecting common ethers utils

mrice32 · 2021-05-07T21:40:25Z

packages/core/deploy/001_deploy_finder.js

@@ -0,0 +1,14 @@
+const func = async function(hre) {


i'm guessing we'll add to these when we add contracts to the setup (like financial contracts, collateral whitelists, etc)?

yup! For each contract we add extra functionality to run if the contract gets newly deployed.

mrice32 · 2021-05-07T21:42:03Z

packages/core/deploy/008_mock_oracle.js

@@ -0,0 +1,33 @@
+const { stringToBytes32, interfaceName, ZERO_ADDRESS } = require("@uma/common");


Is the mock oracle only expected to be deployed on a test L1? Do you think we should have a prod tag and a test tag or something so we can deploy the entire test setup or the entire prod setup? Also, are you allowed to provide multiple tags on deployment? If so, how do they mix (union or intersection of the tagged scripts?)?

Ah yes we should totally use prod and test tags. I'm thinking we give each contract two tags at least, its name (for use as a dependency in other deployment scripts) and a prod tag. This way we can leave out the prod tag from the MockOracle so we can just run yarn hardhat deploy --tags prod to deploy all prod contracts

mrice32 · 2021-05-07T21:46:39Z

packages/core/README.md

+The following commands are implemented as [hardhat tasks](https://hardhat.org/guides/create-task.html) that make it easy to interact with deployed contracts via the CLI:
+
+`yarn hardhat register-deployer --network <NETWORK-NAME>`
+
+Registers the `deployer` account (as defined in the `namedAccounts` param in `hardhat.config.js`) with the deployed Registry for the network.
+
+`yarn hardhat setup-finder --registry --bridge --generichandler --network <NETWORK-NAME>`
+
+Sets specified contracts in the deployed Finder. More contracts available to be set can be found in the `core/scripts/hardhat/tasks/finder.js` script.


Is the idea that most/all scripts that we currently have in scripts/ should be implemented as tasks in hardhat?

Yeah that's what my crazy thought is long term. They could be ported pretty easily as scripts, which would make the CLI invocation a lot less verbose. The main benefit it seems of using tasks is you get access to the hardhat runtime environment which has a lot of nifty objects: https://hardhat.org/advanced/hardhat-runtime-environment.html

I think you could also get the HRE in hardhat scripts too right? https://hardhat.org/guides/scripts.html

Note: nothing to change now, but worth a discussion as we start to port things over.

Yep you can use HRE in js scripts

packages/core/networks/hardhat/deployed.json

mrice32 · 2021-05-07T21:50:35Z

packages/core/scripts/hardhat/deployContracts.sh

+#!/usr/bin/env bash
+
+# The type of oracle system to set up, for example "beacon-l1" or "beacon-l2". Default is "dvm".
+ORACLE_TYPE=$1


Could we just make higher-level tags (in addition to the existing ones), like beacon-l1, beacon-l2, dvm to avoid having to write this script? Or is that a misuse of tags?

Yeah I just created production, sink-oracle and source-oracle tags for now. Will remove this script

Awesome! Very cool. What does the production tag run? Both oracle deployments or just the source?

production runs everything, but I'm thinking now we should change the name to "dvm" and not deploy either sink or source. "dvm" would be Finder, Registry, IdWhitelist, etc.

packages/core/scripts/hardhat/tasks/registry.js

mrice32 · 2021-05-07T21:52:30Z

packages/core/scripts/hardhat/tasks/registry.js

@@ -0,0 +1,33 @@
+const { task } = require("hardhat/config");


I generally find the task syntax to be more pleasant than the truffle syntax. Do you prefer this to a normal node script? What do you think the tradeoffs are between this and a normal node script for this sort of script?

I agree! you also get CLI input args and params super easily along, and tasks can call each other easily as well: https://hardhat.org/guides/create-task.html

I think we should switch all of our scripts over to hardhat tasks because we can remove all of the logic that sets up parts of web3.

I think the big downside is coupling ourselves (again) to a specific framework. I can see arguments either way. But definitely agreed. Might be nice to avoid web3/provider setup, although, we could always write a utility function to do all that work for us (or just use getWeb3 :)).

mrice32 · 2021-05-07T21:54:08Z

packages/core/scripts/hardhat/verifyDeployedContracts.sh

+fi
+
+# The network to verify Etherscan contracts for
+yarn hardhat etherscan-verify --license AGPL-3.0 --force-license --network $NETWORK_NAME


We need this actually because our license tag SPDX-License-Identifier: AGPL-3.0-only is "not supported" by Etherscan?

Weird, I thought it was listed on etherscan when doing manual verification....

Signed-off-by: Nick Pai <[email protected]>

mrice32 · 2021-05-07T22:36:15Z

packages/core/hardhat.config.js

  }
 };

+// Tasks: These tasks are conveniently available via the hardhat CLI: `yarn hardhat <TASK>`
+// Documentation on creating tasks here: https://hardhat.org/guides/create-task.html
+require("./scripts/hardhat/tasks/finder");


(Nothing to change now)

One advantage to hardhat (or node) scripts rather than tasks is that we don't need to import them into the config. That matters for two reasons:

Every time you add a "script" you have to add it to this long list of imports. If you forget, it doesn't work.

Every time the config is loaded, node will need to parse (and execute the global-level stuff, like adding the task to hardhat's global list) these tasks, even when not used, which just means added performance overhead to all hardhat commands.

Hm... I see your point. We could move to using scripts in the next PR but i wanted to demonstrate the power of tasks.

That's fair. It's just something I think we should consider. I think tasks are meant for high-level custom commands (similar to hardhat compile or deploy, but not built in) that are more general. I don't think it was meant as a place to put you contract-specific scripts, which is where hardhat run comes in.

mrice32

LGTM! Nothing blocking, but I left a few responses and comments that we can talk about more as we port more of core to hardhat!

chrismaree

This is pretty awesome! I really like the tasks. I look forward to us being able to move everything over to using this cleaner, more useable syantx!

good work!

One additional question: did you get running node scripts working correctly with hardhat? i.e the equivalent of yarn truffle run using the yarn hardhat run.

chrismaree · 2021-05-10T07:04:16Z

packages/core/README.md

+
+`yarn void:deploy`
+
+This will deploy your contracts on the in-memory hardhat network and exit, leaving no trace. Quickest way to ensure that deployments work as intended without consequences.


do you want to put the description above the command? makes it a bit easier to read.

chrismaree · 2021-05-10T07:04:35Z

packages/core/README.md

+
+`yarn hardhat deploy --network <NETWORK-NAME> --tags <TAGS>`
+
+Deploy all contracts to specified network. Requires a `CUSTOM_NODE_URL` HTTP(s) endpoint and a `MNEMONIC` to be set in environment. Available contract tags can be found in `/deploy` scripts, and available networks are found in the `networks` object within `hardhat.config.js`. Tags can be powerful, for example running `yarn hardhat deploy --tags Bridge` will only deploy the Bridge contract its dependencies (such as the Finder).


chrismaree · 2021-05-10T07:07:03Z

packages/core/deploy/007_identifier_whitelist.js

+      stringToBytes32(interfaceName.IdentifierWhitelist),
+      deployResult.address
+    );
+    log(


some scripts have logs and others dont. should we make that more consistent?

So my rule has been to only log if there is additional work being done besides the deployment itself. For example, this script adds the IDWhitelist to the Finder.

chrismaree · 2021-05-10T07:08:16Z

packages/core/scripts/hardhat/tasks/finder.js

+const { stringToBytes32, interfaceName } = require("@uma/common");
+
+task("setup-finder", "Points Finder to DVM system contracts")
+  .addFlag("registry", "Use if you want to set Registry")


this is cool. will be very powerful for us to define all logic and scripts using these lags.

Agreed, but see Matt's comment about coupling ourselves to another framework. I tend to agree with him, but maybe we should (1) make sure all current scripts are compatible with yarn hardhat run and (2) rather than replace all scripts with tasks, add some commonly used tasks that we might want to run via CLI!

chrismaree · 2021-05-10T07:09:55Z

packages/core/scripts/hardhat/verifyDeployedContracts.sh

+fi
+
+# The network to verify Etherscan contracts for
+yarn hardhat etherscan-verify --license AGPL-3.0 --force-license --network $NETWORK_NAME


you don't need to tell this function what contracts you are verifying? does it just try every contract? how does this work exactly.

it just tries to verify every single contract stored in the local deployments folder. This is a description of the etherscan-verify task that the hardhat-deploy plugin adds to hardhat:

This plugin adds the etherscan-verify task to Hardhat. This task will submit the contract source and other info of all deployed contracts to allow etherscan to verify and record the sources.

nice. this is pretty nifty

nicholaspai and others added 29 commits April 26, 2021 11:00

WIP

71a9559

Updates

a0911cf

Merge branch 'master' into npai/beacon-oracle

4de73e2

reorg contracts

736908d

refactored Source and Sink oracle to be much simpler, updated tests

18e8668

Signed-off-by: Nick Pai <[email protected]>

Set resource ID inside contract

c44fb8c

Add destinationChainId to sinkOracle constructor

c6c5eea

mvp works; need to refactor and comment

f85a2f5

Signed-off-by: Nick Pai <[email protected]>

clean up tests

921fd28

Signed-off-by: Nick Pai <[email protected]>

Sink and Source should use same resource ID, link by source chain ID

f99ea36

Update packages/core/contracts/chainbridge/BeaconOracle.sol

0217baa

Co-authored-by: Matt Rice <[email protected]>

respond to matt's comments

6143633

Signed-off-by: Nick Pai <[email protected]>

WIP

1e579f9

Add chainID to price request hash key

b12cf3a

refactor has/getPrice

5a84fb7

Merge branch 'npai/beacon-oracle' into npai/beacon-oracle-unit-tests

14a8f85

finish tests

6128cd5

Signed-off-by: Nick Pai <[email protected]>

Update sourceOracle.js

19d7d51

remove OracleAncillaryInterface from SourceOracle

64826ac

Signed-off-by: Nick Pai <[email protected]>

Merge branch 'npai/beacon-oracle' into npai/beacon-oracle-unit-tests

e1630a3

change tests

660d379

Signed-off-by: Nick Pai <[email protected]>

Merge branch 'master' into npai/beacon-oracle-unit-tests

534b0b3

bump BeaconMock to 0.8

2760cf0

Signed-off-by: Nick Pai <[email protected]>

WIP

96efcd7

Merge branch 'master' into npai/beacon-deployment

148b879

Update package.json

0e3681a

add demo tasks

53d8b73

Signed-off-by: Nick Pai <[email protected]>

remove ts-compile

5484f95

Signed-off-by: Nick Pai <[email protected]>

Merge branch 'master' into npai/beacon-deployment

4b8c9dd

nicholaspai changed the title ~~WIP: Add hardhat deployment infrastructure~~ feat: Add hardhat deployment infrastructure May 7, 2021

nicholaspai requested review from chrismaree and mrice32 May 7, 2021 16:07

nicholaspai added 4 commits May 7, 2021 12:08

Lint

c52f65b

Update PublicNetworks.js

81fd8e7

Update finder.js

98e351c

Update README.md

a3205f5

mrice32 reviewed May 7, 2021

View reviewed changes

packages/core/deployments/goerli/solcInputs/87af3bddb96fa5cfcfc99d0e6fd39498.json Outdated Show resolved Hide resolved

nicholaspai added 2 commits May 7, 2021 16:10

WIP

f1af286

Update scripts and README

d3b7cb4

nicholaspai requested a review from mrice32 May 7, 2021 21:16

lint-fix

264c216

Signed-off-by: Nick Pai <[email protected]>

mrice32 reviewed May 7, 2021

View reviewed changes

nicholaspai added 4 commits May 7, 2021 18:06

add tags

3fe8857

Signed-off-by: Nick Pai <[email protected]>

Add tags to distinguish between production, sink-oracle, source-oracle

84faedf

Remove deployed.json

9fe9977

use tags instead of deploy.sh script

8ce9c89

mrice32 reviewed May 7, 2021

View reviewed changes

mrice32 self-requested a review May 7, 2021 22:43

mrice32 approved these changes May 7, 2021

View reviewed changes

nicholaspai added 2 commits May 7, 2021 22:01

Update 008_mock_oracle.js

9447363

Change "production" tag to "dvm"

b607c9a

chrismaree approved these changes May 10, 2021

View reviewed changes

nicholaspai requested a review from chrismaree May 10, 2021 12:41

nicholaspai added 5 commits May 10, 2021 08:42

Update README.md

e9215c4

Add identifiers.js task

62a0cc1

Lint

c6907a5

Add Mumbai net ID

60f992b

lint

9375e0c

nicholaspai merged commit 45c8851 into master May 10, 2021

nicholaspai mentioned this pull request May 11, 2021

feat(common): Refactor HardhatConfig exported module #2960

Merged

5 tasks

		@@ -0,0 +1,20 @@
		const { hexlify, toUtf8Bytes } = require("ethers/utils");

		function stringToBytes32(text) {

		@@ -0,0 +1,33 @@
		const { stringToBytes32, interfaceName, ZERO_ADDRESS } = require("@uma/common");


		`yarn void:deploy`

		This will deploy your contracts on the in-memory hardhat network and exit, leaving no trace. Quickest way to ensure that deployments work as intended without consequences.


		`yarn hardhat deploy --network <NETWORK-NAME> --tags <TAGS>`

		Deploy all contracts to specified network. Requires a `CUSTOM_NODE_URL` HTTP(s) endpoint and a `MNEMONIC` to be set in environment. Available contract tags can be found in `/deploy` scripts, and available networks are found in the `networks` object within `hardhat.config.js`. Tags can be powerful, for example running `yarn hardhat deploy --tags Bridge` will only deploy the Bridge contract its dependencies (such as the Finder).

feat: Add hardhat deployment infrastructure #2950

feat: Add hardhat deployment infrastructure #2950

Conversation

nicholaspai commented May 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrice32 May 7, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrice32 May 7, 2021 • edited Loading

Choose a reason for hiding this comment

mrice32 May 7, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrice32 May 10, 2021 • edited Loading

Choose a reason for hiding this comment

mrice32 left a comment

Choose a reason for hiding this comment

chrismaree left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicholaspai May 10, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicholaspai commented May 6, 2021 •

edited

Loading

mrice32 May 7, 2021 •

edited

Loading

mrice32 May 7, 2021 •

edited

Loading

mrice32 May 7, 2021 •

edited

Loading

mrice32 May 10, 2021 •

edited

Loading

nicholaspai May 10, 2021 •

edited

Loading