Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

upgrade lotus-soup testplans and reduce deals concurrency to a single miner #6122

Merged
merged 2 commits into from
Apr 28, 2021
Merged

upgrade lotus-soup testplans and reduce deals concurrency to a single miner #6122

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a9e4a0d
upgrade docker images
nonsense Apr 27, 2021
0304beb
remove deals concurrency
nonsense Apr 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions testplans/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,18 @@ download-proofs:
go run github.com/filecoin-project/go-paramfetch/paramfetch 2048 ./docker-images/proof-parameters.json

build-images:
docker build -t "iptestground/oni-buildbase:v13-lotus" -f "docker-images/Dockerfile.oni-buildbase" "docker-images"
docker build -t "iptestground/oni-runtime:v7" -f "docker-images/Dockerfile.oni-runtime" "docker-images"
docker build -t "iptestground/oni-runtime:v8-debug" -f "docker-images/Dockerfile.oni-runtime-debug" "docker-images"
docker build -t "iptestground/oni-buildbase:v14-lotus" -f "docker-images/Dockerfile.oni-buildbase" "docker-images"
docker build -t "iptestground/oni-runtime:v9" -f "docker-images/Dockerfile.oni-runtime" "docker-images"
docker build -t "iptestground/oni-runtime:v9-debug" -f "docker-images/Dockerfile.oni-runtime-debug" "docker-images"

push-images:
docker push iptestground/oni-buildbase:v13-lotus
docker push iptestground/oni-runtime:v7
docker push iptestground/oni-runtime:v8-debug
docker push iptestground/oni-buildbase:v14-lotus
docker push iptestground/oni-runtime:v9
docker push iptestground/oni-runtime:v9-debug

pull-images:
docker pull iptestground/oni-buildbase:v13-lotus
docker pull iptestground/oni-runtime:v7
docker pull iptestground/oni-runtime:v8-debug
docker pull iptestground/oni-buildbase:v14-lotus
docker pull iptestground/oni-runtime:v9
docker pull iptestground/oni-runtime:v9-debug

.PHONY: download-proofs build-images push-images pull-images
4 changes: 2 additions & 2 deletions testplans/docker-images/Dockerfile.oni-buildbase
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
ARG GO_VERSION=1.15.6
ARG GO_VERSION=1.16.3

FROM golang:${GO_VERSION}-buster

RUN apt-get update && apt-get install -y ca-certificates llvm clang mesa-opencl-icd ocl-icd-opencl-dev jq gcc git pkg-config bzr libhwloc-dev

ARG FILECOIN_FFI_COMMIT=62f89f108a6a8fe9ad6ed52fb7ffbf8594d7ae5c
ARG FILECOIN_FFI_COMMIT=d82899449741ce190e950a3582ebe33806f018a9
ARG FFI_DIR=/extern/filecoin-ffi

RUN mkdir -p ${FFI_DIR} \
Expand Down
4 changes: 2 additions & 2 deletions testplans/docker-images/Dockerfile.oni-runtime
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG GO_VERSION=1.15.6
ARG GO_VERSION=1.16.3

FROM golang:${GO_VERSION}-buster as downloader

Expand All @@ -8,7 +8,7 @@ FROM golang:${GO_VERSION}-buster as downloader
## 3. Trigger the download.
## Output will be in /var/tmp/filecoin-proof-parameters.

RUN go get github.com/filecoin-project/go-paramfetch/paramfetch
RUN go get github.com/filecoin-project/go-paramfetch/paramfetch@master
COPY /proof-parameters.json /
RUN paramfetch 8388608 /proof-parameters.json

Expand Down
6 changes: 3 additions & 3 deletions testplans/docker-images/Dockerfile.oni-runtime-debug
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG GO_VERSION=1.15.6
ARG GO_VERSION=1.16.3

FROM golang:${GO_VERSION}-buster as downloader

Expand All @@ -8,11 +8,11 @@ FROM golang:${GO_VERSION}-buster as downloader
## 3. Trigger the download.
## Output will be in /var/tmp/filecoin-proof-parameters.

RUN go get github.com/filecoin-project/go-paramfetch/paramfetch
RUN go get github.com/filecoin-project/go-paramfetch/paramfetch@master
COPY /proof-parameters.json /
RUN paramfetch 8388608 /proof-parameters.json

ARG LOTUS_COMMIT=b4ad2e5e93dc710d985eb9cf3ee04142efb47bf0
ARG LOTUS_COMMIT=7e25a811c3d80ea3e007a54aa1da089985110c2c

## for debug purposes
RUN apt update && apt install -y mesa-opencl-icd ocl-icd-opencl-dev gcc git bzr jq pkg-config libhwloc-dev curl && git clone https://github.com/filecoin-project/lotus.git && cd lotus/ && git checkout ${LOTUS_COMMIT} && make clean && make all && make install
Expand Down
2 changes: 1 addition & 1 deletion testplans/lotus-soup/_compositions/baseline-k8s-3-1.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
[[groups]]
id = "miners"
[groups.resources]
memory = "4096Mi"
memory = "8192Mi"
cpu = "1000m"
[groups.instances]
count = 1
Expand Down
4 changes: 3 additions & 1 deletion testplans/lotus-soup/deals_e2e.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,9 @@ func dealsE2E(t *testkit.TestEnvironment) error {
// give some time to the miner, otherwise, we get errors like:
// deal errored deal failed: (State=26) error calling node: publishing deal: GasEstimateMessageGas
// error: estimating gas used: message execution failed: exit 19, reason: failed to lock balance: failed to lock client funds: not enough balance to lock for addr t0102: escrow balance 0 < locked 0 + required 640297000 (RetCode=19)
time.Sleep(50 * time.Second)
time.Sleep(40 * time.Second)

time.Sleep(time.Duration(t.GlobalSeq) * 5 * time.Second)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea here is that we give 5sec. in-between of each deal from various clients, which is certainly more than 1 block time.


// generate 1600 bytes of random data
data := make([]byte, 5000000)
Expand Down
6 changes: 6 additions & 0 deletions testplans/lotus-soup/init.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ func init() {
_ = log.SetLogLevel("stats", "WARN")
_ = log.SetLogLevel("dht/RtRefreshManager", "ERROR") // noisy
_ = log.SetLogLevel("bitswap", "ERROR") // noisy
_ = log.SetLogLevel("badgerbs", "ERROR") // noisy
_ = log.SetLogLevel("sub", "ERROR") // noisy
_ = log.SetLogLevel("pubsub", "ERROR") // noisy
_ = log.SetLogLevel("chain", "ERROR") // noisy
_ = log.SetLogLevel("chainstore", "ERROR") // noisy
_ = log.SetLogLevel("basichost", "ERROR") // noisy

_ = os.Setenv("BELLMAN_NO_GPU", "1")

Expand Down
4 changes: 2 additions & 2 deletions testplans/lotus-soup/manifest.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ enabled = true

[builders."docker:go"]
enabled = true
build_base_image = "iptestground/oni-buildbase:v13-lotus"
runtime_image = "iptestground/oni-runtime:v8-debug"
build_base_image = "iptestground/oni-buildbase:v14-lotus"
runtime_image = "iptestground/oni-runtime:v9-debug"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've upgraded docker images with latest lotus and lotus-miner binaries (aliased to lll and lm within the images), so that we can debug deadlocked devnets, when a deal ends up in the StorageDealAwaitingPreCommit state.


[runners."local:exec"]
enabled = true
Expand Down