Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: standardize node directory structure #1944

Merged
merged 14 commits into from
Apr 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 24 additions & 10 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,36 @@ misc/
docker-compose.yml
tests/docker-integration/

# gitignore
*.sw[pon]
cmd/foo.go
unsorted.*
# Copied from .gitignore

# Editor Leftovers
.DS_Store
.vscode
.idea

# Default node data
gnoland-data
genesis.json

# Build leftovers
build

# Legacy .gitignore
data/*
proto/*
testdir/
pkgs/sdk/vm/_testdata
build/*
testdir

# Log + generation leftovers
*.tx
*.log.*
*.log
*.gno.gen.go
*.gno.gen_test.go
.vscode
.idea
*.pb.go
pbbindings.go
*~
.#*
*#

# Test coverage leftovers
cover.out
coverage.out
23 changes: 16 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,22 +1,31 @@
*.sw[pon]
cmd/foo.go
unsorted.*
# Editor Leftovers
.DS_Store
.vscode
.idea

# Default node data
gnoland-data
genesis.json

# Build leftovers
build

# Legacy .gitignore
data/*
testdir
pkgs/sdk/vm/_testdata
thehowl marked this conversation as resolved.
Show resolved Hide resolved
build

# Log + generation leftovers
*.tx
*.log.*
*.log
*.gno.gen.go
*.gno.gen_test.go
.vscode
.idea
*.pb.go
pbbindings.go
*~
.#*
*#

# Test coverage leftovers
cover.out
coverage.out
1 change: 1 addition & 0 deletions contribs/gnokeykc/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ require (
)

require (
dario.cat/mergo v1.0.0 // indirect
github.com/alessio/shellescape v1.4.1 // indirect
github.com/btcsuite/btcd/btcec/v2 v2.3.3 // indirect
github.com/btcsuite/btcd/btcutil v1.1.5 // indirect
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/local-setup/premining-balances.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ In order for us to premine funds on a fresh chain, we need to make sure we do no
from previous chain runs.

The blockchain node, when it runs, works with an embedded DB locally on disk to store execution data (such as
configuration files, or the state DB). For Gno blockchain nodes, this working directory is labeled as `testdir` by
configuration files, or the state DB). For Gno blockchain nodes, this working directory is labeled as `gnoland-data` by
default.

To clean out old blockchain data, navigate to the `gno.land` folder and run the appropriate make command:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,11 @@ Let's break down the most important default settings:
- `genesis-balances-file` - the initial premine balances file, which contains initial native currency allocations for
the chain. By default, the genesis balances file is located in `gno.land/genesis/genesis_balances.txt`, this is also the
reason why we need to navigate to the `gno.land` sub-folder to run the command with default settings
- `root-dir` - the working directory for the node configuration and node data (state DB)
- `data-dir` - the working directory for the node configuration and node data (state DB)

:::info Resetting the chain

As mentioned, the working directory for the node is located in `root-dir`. To reset the chain, you need
As mentioned, the working directory for the node is located in `data-dir`. To reset the chain, you need
to delete this directory and start the node up again. If you are using the default node configuration, you can run
`make fclean` from the `gno.land` sub-folder to delete the `tempdir` working directory.

Expand Down
4 changes: 2 additions & 2 deletions docs/gno-tooling/cli/gnoland.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ gnoland
### **Options**

| Name | Type | Description |
| -------------------------- | ------- | --------------------------------------------------------------------------------------- |
|----------------------------| ------- | --------------------------------------------------------------------------------------- |
| `chainid` | String | The id of the chain (default: `dev`). |
| `genesis-balances-file` | String | The initial GNOT distribution file (default: `./gnoland/genesis/genesis_balances.txt`). |
| `genesis-remote` | String | Replacement '%%REMOTE%%' in genesis (default: `"localhost:26657"`). |
| `genesis-txs-file` | String | Initial txs to be executed (default: `"./gnoland/genesis/genesis_txs.jsonl"`). |
| `root-dir` | String | directory for config and data (default: `testdir`). |
| `data-dir` | String | directory for config and data (default: `gnoland-data`). |
| `skip-failing-genesis-txs` | Boolean | Skips transactions that fail from the `genesis-txs-file` |
| `skip-start` | Boolean | Quits after initialization without starting the node. |
2 changes: 1 addition & 1 deletion gno.land/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ install.gnokey:; go install ./cmd/gnokey

.PHONY: fclean
fclean: clean
rm -rf testdir
rm -rf gnoland-data genesis.json

.PHONY: clean
clean:
Expand Down
29 changes: 13 additions & 16 deletions gno.land/cmd/gnoland/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -120,12 +120,11 @@
"the root directory of the gno repository",
)

// XXX: Use home directory for this
fs.StringVar(
&c.dataDir,
"data-dir",
"testdir",
"directory for config and data",
"gnoland-data",
"the path to the node's data directory",
)

fs.StringVar(
Expand Down Expand Up @@ -192,18 +191,14 @@
log.ConsoleFormat.String(),
"log format for the gnoland node",
)

// XXX(deprecated): use data-dir instead
fs.StringVar(
&c.dataDir,
"root-dir",
"testdir",
"deprecated: use data-dir instead - directory for config and data",
)
}

func execStart(c *startCfg, io commands.IO) error {
dataDir := c.dataDir
// Get the absolute path to the node's data directory
nodeDir, err := filepath.Abs(c.dataDir)
if err != nil {
return fmt.Errorf("unable to get absolute path for data directory, %w", err)

Check warning on line 200 in gno.land/cmd/gnoland/start.go

View check run for this annotation

Codecov / codecov/patch

gno.land/cmd/gnoland/start.go#L200

Added line #L200 was not covered by tests
}

var (
cfg *config.Config
Expand All @@ -223,7 +218,7 @@
cfg, loadCfgErr = config.LoadConfigFile(c.nodeConfigPath)
} else {
// Load the default node configuration
cfg, loadCfgErr = config.LoadOrMakeConfigWithOptions(dataDir)
cfg, loadCfgErr = config.LoadOrMakeConfigWithOptions(nodeDir)
}

if loadCfgErr != nil {
Expand All @@ -246,7 +241,9 @@
logger := log.ZapLoggerToSlog(zapLogger)

// Write genesis file if missing.
genesisFilePath := filepath.Join(dataDir, cfg.Genesis)
// NOTE: this will be dropped in a PR that resolves issue #1883:
// https://github.com/gnolang/gno/issues/1883
genesisFilePath := filepath.Join(nodeDir, "../", "genesis.json")

if !osm.FileExists(genesisFilePath) {
// Create priv validator first.
Expand All @@ -270,7 +267,7 @@
cfg.TxEventStore = txEventStoreCfg

// Create application and node.
gnoApp, err := gnoland.NewApp(dataDir, c.skipFailingGenesisTxs, logger, c.genesisMaxVMCycles)
gnoApp, err := gnoland.NewApp(nodeDir, c.skipFailingGenesisTxs, logger, c.genesisMaxVMCycles)
if err != nil {
return fmt.Errorf("error in creating new app: %w", err)
}
Expand All @@ -280,7 +277,7 @@
io.Println(startGraphic)
}

gnoNode, err := node.DefaultNewNode(cfg, logger)
gnoNode, err := node.DefaultNewNode(cfg, genesisFilePath, logger)
if err != nil {
return fmt.Errorf("error in creating node: %w", err)
}
Expand Down
3 changes: 2 additions & 1 deletion gno.land/pkg/gnoland/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
"github.com/gnolang/gno/gno.land/pkg/sdk/vm"
"github.com/gnolang/gno/gnovm/pkg/gnoenv"
abci "github.com/gnolang/gno/tm2/pkg/bft/abci/types"
"github.com/gnolang/gno/tm2/pkg/bft/config"
dbm "github.com/gnolang/gno/tm2/pkg/db"
"github.com/gnolang/gno/tm2/pkg/log"
"github.com/gnolang/gno/tm2/pkg/sdk"
Expand Down Expand Up @@ -132,7 +133,7 @@
}

// Get main DB.
cfg.DB, err = dbm.NewDB("gnolang", dbm.GoLevelDBBackend, filepath.Join(dataRootDir, "data"))
cfg.DB, err = dbm.NewDB("gnolang", dbm.GoLevelDBBackend, filepath.Join(dataRootDir, config.DefaultDBDir))

Check warning on line 136 in gno.land/pkg/gnoland/app.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/gnoland/app.go#L136

Added line #L136 was not covered by tests
if err != nil {
return nil, fmt.Errorf("error initializing database %q using path %q: %w", dbm.GoLevelDBBackend, dataRootDir, err)
}
Expand Down
28 changes: 0 additions & 28 deletions gno.land/pkg/gnoland/node_inmemory.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,34 +59,6 @@ func NewDefaultTMConfig(rootdir string) *tmcfg.Config {
return tmcfg.TestConfig().SetRootDir(rootdir)
}

// NewInMemoryNodeConfig creates a default configuration for an in-memory node.
func NewDefaultInMemoryNodeConfig(rootdir string) *InMemoryNodeConfig {
tm := NewDefaultTMConfig(rootdir)

// Create Mocked Identity
pv := NewMockedPrivValidator()
genesis := NewDefaultGenesisConfig(pv.GetPubKey(), tm.ChainID())

// Add self as validator
self := pv.GetPubKey()
genesis.Validators = []bft.GenesisValidator{
{
Address: self.Address(),
PubKey: self,
Power: 10,
Name: "self",
},
}

return &InMemoryNodeConfig{
PrivValidator: pv,
TMConfig: tm,
Genesis: genesis,
GenesisTxHandler: PanicOnFailingTxHandler,
GenesisMaxVMCycles: 10_000_000,
}
}

func (cfg *InMemoryNodeConfig) validate() error {
if cfg.PrivValidator == nil {
return fmt.Errorf("`PrivValidator` is required but not provided")
Expand Down
2 changes: 1 addition & 1 deletion gnovm/pkg/doc/dirs_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ func TestDirs_findPackage(t *testing.T) {
}},
{"alpha", []bfsDir{
{importPath: "dirs.mod/dep/alpha", dir: filepath.Join(td, "dirsdep/pkg/mod/dirs.mod/dep/alpha")},
// no testdir/module/alpha as it is inside a module
// no gnoland-data/module/alpha as it is inside a module
}},
{"math", []bfsDir{
{importPath: "math", dir: filepath.Join(td, "dirs/math")},
Expand Down
2 changes: 1 addition & 1 deletion misc/deployments/staging.gno.land/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ logs:
down:
docker compose down
docker volume rm -f staginggnoland_gnonode
docker compose run gnoland rm -rf /opt/gno/src/gno.land/testdir/data /opt/gno/src/gno.land/testdir/config
docker compose run gnoland rm -rf /opt/gno/src/gno.land/gnoland-data/data /opt/gno/src/gno.land/gnoland-data/config

pull:
git pull
Expand Down
2 changes: 1 addition & 1 deletion misc/deployments/staging.gno.land/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ services:
- --chainid=staging
- --genesis-remote=staging.gno.land:36657
volumes:
- "./data/gnoland:/opt/gno/src/gno.land/testdir"
- "./data/gnoland:/opt/gno/src/gno.land/gnoland-data"
ports:
- 36656:36656
- 36657:36657
Expand Down
2 changes: 1 addition & 1 deletion misc/docker-compose/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ services:
- LOG_LEVEL=4
command: [ "gnoland", "start" ]
volumes:
- "gnonode:/opt/gno/src/testdir"
- "gnonode:/opt/gno/src/gnoland-data"
networks:
- gnonode
restart: on-failure
Expand Down
2 changes: 1 addition & 1 deletion misc/loop/cmd/snapshotter.go
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ func (s snapshotter) startPortalLoopContainer(ctx context.Context) (*types.Conta
Binds: []string{
fmt.Sprintf("%s/scripts:/scripts", s.cfg.hostPWD),
fmt.Sprintf("%s/backups:/backups", s.cfg.hostPWD),
fmt.Sprintf("%s:/opt/gno/src/testdir", s.containerName),
fmt.Sprintf("%s:/opt/gno/src/gnoland-data", s.containerName),
},
}, nil, nil, s.containerName)
if err != nil {
Expand Down
10 changes: 5 additions & 5 deletions misc/loop/scripts/start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ gnoland start \
--skip-start=true \
--skip-failing-genesis-txs

sed -i "s#^moniker = \".*\"#moniker = \"${MONIKER}\"#" ./testdir/config/config.toml
sed -i "s#laddr = \".*:26656\"#laddr = \"${P2P_LADDR}\"#" ./testdir/config/config.toml
sed -i "s#laddr = \".*:26657\"#laddr = \"${RPC_LADDR}\"#" ./testdir/config/config.toml
sed -i "s#^moniker = \".*\"#moniker = \"${MONIKER}\"#" ./gnoland-data/config/config.toml
sed -i "s#laddr = \".*:26656\"#laddr = \"${P2P_LADDR}\"#" ./gnoland-data/config/config.toml
sed -i "s#laddr = \".*:26657\"#laddr = \"${RPC_LADDR}\"#" ./gnoland-data/config/config.toml

sed -i "s#seeds = \".*\"#seeds = \"${SEEDS}\"#" ./testdir/config/config.toml
sed -i "s#persistent_peers = \".*\"#persistent_peers = \"${PERSISTENT_PEERS}\"#" ./testdir/config/config.toml
sed -i "s#seeds = \".*\"#seeds = \"${SEEDS}\"#" ./gnoland-data/config/config.toml
sed -i "s#persistent_peers = \".*\"#persistent_peers = \"${PERSISTENT_PEERS}\"#" ./gnoland-data/config/config.toml

exec gnoland start --skip-failing-genesis-txs
4 changes: 2 additions & 2 deletions tm2/pkg/bft/blockchain/reactor_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ func newBlockchainReactor(logger *slog.Logger, genDoc *types.GenesisDoc, privVal
func TestNoBlockResponse(t *testing.T) {
t.Parallel()

config = cfg.ResetTestRoot("blockchain_reactor_test")
config, _ = cfg.ResetTestRoot("blockchain_reactor_test")
defer os.RemoveAll(config.RootDir)
genDoc, privVals := randGenesisDoc(1, false, 30)

Expand Down Expand Up @@ -182,7 +182,7 @@ func TestFlappyBadBlockStopsPeer(t *testing.T) {

testutils.FilterStability(t, testutils.Flappy)

config = cfg.ResetTestRoot("blockchain_reactor_test")
config, _ = cfg.ResetTestRoot("blockchain_reactor_test")
defer os.RemoveAll(config.RootDir)
genDoc, privVals := randGenesisDoc(1, false, 30)

Expand Down
Loading
Loading