Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/stackerdb rpc #3848

Merged
merged 74 commits into from
Sep 2, 2023
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
f36cc8d
refactor: move streaming logic out of StacksChainState and into net, …
jcnelson Aug 15, 2023
626587a
chore: refactor stream move
jcnelson Aug 15, 2023
b024638
chore: update copyright
jcnelson Aug 15, 2023
8aa654c
chore: update copyright
jcnelson Aug 15, 2023
4e66c42
chore: update copyright
jcnelson Aug 15, 2023
78e618a
chore: update copyright
jcnelson Aug 15, 2023
b4876e8
chore: update copyright
jcnelson Aug 15, 2023
dd3545d
chore: update copyright
jcnelson Aug 15, 2023
04f7470
chore: update copyright
jcnelson Aug 15, 2023
53a2fe7
feat: transport-layer implementation for getting chunks, storing chun…
jcnelson Aug 15, 2023
be593f3
chore: update copyright
jcnelson Aug 15, 2023
2721efb
feat: message structs for getting, loading, and querying metadada in …
jcnelson Aug 15, 2023
24984c3
feat: process stackerdb reconfiguration every Bitcoin block
jcnelson Aug 15, 2023
2f7d934
chore: update copyright
jcnelson Aug 15, 2023
3fe02b9
chore: update copyright
jcnelson Aug 15, 2023
068c8d0
chore: update copyright
jcnelson Aug 15, 2023
d79f412
feat: add handlers for getting chunks, storing chunks, and listing ch…
jcnelson Aug 15, 2023
d28715e
chore: update copyright
jcnelson Aug 15, 2023
4b20a59
chore: constructor for unsigned chunk metadata
jcnelson Aug 15, 2023
961e900
feat: propagate the reason for an error in querying the stackerdb sma…
jcnelson Aug 15, 2023
6ecb4c7
feat: add DB method for querying the whole DB's metadata
jcnelson Aug 15, 2023
c8ec972
chore: remove unnecessary write timestamp check on chunk receipt; ins…
jcnelson Aug 15, 2023
8c21dd2
chore: if pushing a chunk leads us to discover newer chunks, then pro…
jcnelson Aug 15, 2023
837c402
chore: expose stream module
jcnelson Aug 15, 2023
45e840d
feat: post-stackerdb helper
jcnelson Aug 15, 2023
bb67412
chore: API sync
jcnelson Aug 15, 2023
6b795f1
fix: send chunks as "text" (no SIP-003 length prefix)
jcnelson Aug 15, 2023
977fef2
fix: API sync
jcnelson Aug 15, 2023
49fb7a9
fix: API sync
jcnelson Aug 15, 2023
a28812b
fix: return stacker DBs as a string
jcnelson Aug 15, 2023
09e955d
fix: don't enforce write rate-limiting in the DB
jcnelson Aug 15, 2023
ed19283
fix: use bitcoind v0.25 descriptors for importing an address
jcnelson Aug 15, 2023
62e0436
feat: configurable stackerdbs
jcnelson Aug 15, 2023
c85083b
feat: load up stackerdbs from config file when instantiating peerdb a…
jcnelson Aug 15, 2023
2a6db51
fix: API sync
jcnelson Aug 15, 2023
022c60c
feat: test for load/store chunks in stackerdb
jcnelson Aug 15, 2023
39ac9c6
Merge branch 'feat/stackerdb-config' into feat/stackerdb-rpc
jcnelson Aug 15, 2023
9b85505
fix: compile-time merge errors
jcnelson Aug 15, 2023
4c10f7d
fix: stacker_dbs is optional
jcnelson Aug 15, 2023
08d9adf
feat: factor out StackerDB client logic into its own crate
jcnelson Aug 16, 2023
79b7092
chore: add libstackerdb to cargo.lock and cargo.toml
jcnelson Aug 16, 2023
c85b0d7
chore: depend on libstackerdb
jcnelson Aug 16, 2023
4eab26b
chore: move stackerdb RPC structs into their own crate
jcnelson Aug 16, 2023
bf60466
chore: use libstackerdb instead of stackslib for stackerdb RPC
jcnelson Aug 16, 2023
7547a60
chore: remove dead files
jcnelson Aug 16, 2023
b42d312
chore: remove ContractId and use QualifiedContractIdentifier pervasively
jcnelson Aug 16, 2023
ee7cc50
chore: address all PR feedback, and address a bug in which we acciden…
jcnelson Aug 18, 2023
5b14fce
chore: cargo fmt and fix compile error
jcnelson Aug 18, 2023
11a6f03
fix: include `stackerdb` in RPC request paths
jcnelson Aug 18, 2023
a791248
chore: fix failing test case
jcnelson Aug 21, 2023
d18ef3a
feat: track uploaded stackerdb chunks in the network result
jcnelson Aug 22, 2023
6a87551
feat: unify MemPoolEventDispatcher and StackerDBEventDispatcher behin…
jcnelson Aug 22, 2023
395effe
feat: POST /v2/stackerdb/... will now report the chunk to the HTTP se…
jcnelson Aug 22, 2023
57919d9
feat: StackerDBEventObserver trait
jcnelson Aug 22, 2023
6640032
feat: "stackerdb" event key
jcnelson Aug 22, 2023
d5ea39e
feat: push stackerdb events to observers with the "stackerdb" or "any…
jcnelson Aug 22, 2023
13884dc
feat: add event handler for stackerdb events to test observer
jcnelson Aug 22, 2023
f890c6e
feat: integration test for stackerdb event observer
jcnelson Aug 22, 2023
1276dcb
docs: describe /stackerdb_chunks endpoint for event observer
jcnelson Aug 22, 2023
765dc61
fix: use Bitcoin 0.25 for integration tests
jcnelson Aug 23, 2023
2a3ce4f
fix: use Bitcoin 0.25 for integration tests
jcnelson Aug 23, 2023
7b7b032
Merge branch 'feat/stackerdb-rpc' into feat/stackerdb-event-observer
jcnelson Aug 23, 2023
739c085
fix: API sync with unit tests
jcnelson Aug 23, 2023
54d66d8
chore: cargo fmt
jcnelson Aug 23, 2023
1f3bd2e
fix: use Bitcoin 25 for all CI tests
jcnelson Aug 23, 2023
4ee5928
fix: use Bitcoin 25 for all CI tests
jcnelson Aug 23, 2023
9c4e540
fix: the stackerdb event observer now sends chunks over the event obs…
jcnelson Aug 25, 2023
8d9ef90
fix: API sync with unit tests
jcnelson Aug 25, 2023
a45cfdf
chore: run stackerdb tests in CI
jcnelson Aug 29, 2023
bdcb0e8
chore: address PR feedback -- make stackerdb events consumptive, and …
jcnelson Aug 29, 2023
20eec10
Merge branch 'feat/stackerdb-config' into feat/stackerdb-rpc
jcnelson Aug 29, 2023
9480c71
Merge branch 'feat/stackerdb-rpc' into feat/stackerdb-event-observer
jcnelson Aug 29, 2023
a5cd26e
Merge branch 'develop' into feat/stackerdb-rpc
jcnelson Sep 1, 2023
ef6c5a4
Merge pull request #3870 from stacks-network/feat/stackerdb-event-obs…
jcnelson Sep 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/actions/bitcoin-int-tests/Dockerfile.atlas-test
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ COPY . .

RUN cargo test --no-run --workspace

RUN cd / && wget https://bitcoin.org/bin/bitcoin-core-0.20.0/bitcoin-0.20.0-x86_64-linux-gnu.tar.gz
RUN cd / && tar -xvzf bitcoin-0.20.0-x86_64-linux-gnu.tar.gz
RUN cd / && wget https://bitcoin.org/bin/bitcoin-core-25.0/bitcoin-25.0-x86_64-linux-gnu.tar.gz
RUN cd / && tar -xvzf bitcoin-25.0-x86_64-linux-gnu.tar.gz

RUN ln -s /bitcoin-0.20.0/bin/bitcoind /bin/
RUN ln -s /bitcoin-25.0/bin/bitcoind /bin/

ENV BITCOIND_TEST 1
WORKDIR /src/testnet/stacks-node
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ ENV RUSTFLAGS="-Cinstrument-coverage" \
RUN cargo test --no-run && \
cargo build

RUN cd / && wget https://bitcoin.org/bin/bitcoin-core-0.20.0/bitcoin-0.20.0-x86_64-linux-gnu.tar.gz
RUN cd / && tar -xvzf bitcoin-0.20.0-x86_64-linux-gnu.tar.gz
RUN cd / && wget https://bitcoin.org/bin/bitcoin-core-25.0/bitcoin-25.0-x86_64-linux-gnu.tar.gz
RUN cd / && tar -xvzf bitcoin-25.0-x86_64-linux-gnu.tar.gz

RUN ln -s /bitcoin-0.20.0/bin/bitcoind /bin/
RUN ln -s /bitcoin-25.0/bin/bitcoind /bin/

ENV BITCOIND_TEST 1
6 changes: 3 additions & 3 deletions .github/actions/bitcoin-int-tests/Dockerfile.large-genesis
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ WORKDIR /src

COPY . .

RUN cd / && wget https://bitcoin.org/bin/bitcoin-core-0.20.0/bitcoin-0.20.0-x86_64-linux-gnu.tar.gz
RUN cd / && tar -xvzf bitcoin-0.20.0-x86_64-linux-gnu.tar.gz
RUN cd / && wget https://bitcoin.org/bin/bitcoin-core-25.0/bitcoin-25.0-x86_64-linux-gnu.tar.gz
RUN cd / && tar -xvzf bitcoin-25.0-x86_64-linux-gnu.tar.gz

RUN ln -s /bitcoin-0.20.0/bin/bitcoind /bin/
RUN ln -s /bitcoin-25.0/bin/bitcoind /bin/

RUN rustup component add llvm-tools-preview && \
cargo install grcov
Expand Down
8 changes: 4 additions & 4 deletions .github/actions/bitcoin-int-tests/Dockerfile.net-tests
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ WORKDIR /src

COPY . .

RUN cd / && wget https://bitcoin.org/bin/bitcoin-core-0.20.0/bitcoin-0.20.0-x86_64-linux-gnu.tar.gz
RUN cd / && tar -xvzf bitcoin-0.20.0-x86_64-linux-gnu.tar.gz
RUN cd / && wget https://bitcoin.org/bin/bitcoin-core-25.0/bitcoin-25.0-x86_64-linux-gnu.tar.gz
RUN cd / && tar -xvzf bitcoin-25.0-x86_64-linux-gnu.tar.gz

RUN ln -s /bitcoin-0.20.0/bin/bitcoind /bin/
RUN ln -s /bitcoin-0.20.0/bin/bitcoin-cli /bin/
RUN ln -s /bitcoin-25.0/bin/bitcoind /bin/
RUN ln -s /bitcoin-25.0/bin/bitcoin-cli /bin/

RUN apt-get update
RUN apt-get install -y jq screen net-tools ncat sqlite3 xxd openssl curl
Expand Down
14 changes: 14 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@ members = [
"pox-locking",
"clarity",
"stx-genesis",
"libstackerdb",
"testnet/stacks-node"]
33 changes: 33 additions & 0 deletions libstackerdb/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
[package]
name = "libstackerdb"
version = "0.0.1"
authors = [ "Jude Nelson <[email protected]>" ]
license = "GPLv3"
homepage = "https://github.com/blockstack/stacks-blockchain"
repository = "https://github.com/blockstack/stacks-blockchain"
description = "Client library for the StackerDB subsystem"
keywords = [ "stacks", "stx", "bitcoin", "crypto", "blockstack", "decentralized", "dapps", "blockchain" ]
readme = "README.md"
resolver = "2"
edition = "2021"

[lib]
name = "libstackerdb"
path = "./src/libstackerdb.rs"

[dependencies]
serde = "1"
serde_derive = "1"
serde_stacker = "0.1"
stacks-common = { path = "../stacks-common" }
clarity = { path = "../clarity" }

[dependencies.secp256k1]
version = "0.24.3"
features = ["serde", "recovery"]

[target.'cfg(all(any(target_arch = "x86_64", target_arch = "x86", target_arch = "aarch64"), not(target_env = "msvc")))'.dependencies]
sha2 = { version = "0.10", features = ["asm"] }

[target.'cfg(any(not(any(target_arch = "x86_64", target_arch = "x86", target_arch = "aarch64")), target_env = "msvc"))'.dependencies]
sha2 = { version = "0.10" }
Loading