Script against ICF archive databases

[Nostradamus411]

I've copied down the Hub1 daemon / cli / database from the ICF archives @sunnya97 shared on the Ion telegram.

My script calling the cli to get vote & deposit data is here.

I'm currently running the script for the full Ions.json but here is a sample using the first 10 or so rows from the json plus the top few Ion drop recipient addresses. This is showing a correlation of early voting, contrary to my borked script against the figment.io endpoint.

I look to add the other two hubs, but due to disk space going to have to do them one at a time. I'll be adding to my repo, which started as a script against the Osmosis daemon to find "unclaimed" ions.

[Nostradamus411]

Full hub1 extract json is now added to my repo.

[jaekwon]

thank you, awesome.

[jaekwon]

I'll replicate some myself, and this work seems to be on the mark for a large portion of the bounty. Please share your osmosis address here or privately.

[Nostradamus411]

Well in working on running hub2 scrape I realized an issue when my method of checking votes always returned:
{"error":"'' is not a valid vote option"}

So I've still got some correcting to do. It looks like I'm not taking into account the need to query for votes/deposits at a block height where they are available for the already passed/rejected proposals based on this.

Going to need to find some time to figure out how to determine what the proper block height to query for each past proposal and see about updating the script.

I've direct messaged you an OSMO address on Telegram (my ID is also Nostradamus411 on TG).

[RowanUtrecht]

@Nostradamus411 @jaekwon
I've checked the file submitted by Nostradamus https://github.com/Nostradamus411/unclaimed-ions/blob/main/ions_hub1_gov.json

I checked the biggest receiver of ION (27 - cosmos1wtv0kp6ydt03edd8kyr5arr4f3yc52vp5g7na0)
According to the file in the repo, that address did not vote on prop 1,2,3
According to mintscan, it did vote on prop 1,2,3 (I think it even voted on all 26 proposals that are stored on Mintscan. I can check again if needed)
https://www.mintscan.io/cosmos/txs/B7C010FF08E433E7208E4E6FA182BB5C3CBE04C6FFA7EC23EC642E80AA8AA843
https://www.mintscan.io/cosmos/txs/37571DCB71ACF90B4AC14B003F5B7C13C9B7599DA7745298419BD03226DA05DA
https://www.mintscan.io/cosmos/txs/5A754E03B53484CBA5B66DC6F81E596FE467A96F55D2A7BA131ABEE8C75FD880

So it still seems so hard to get the right data. Could it be that sometimes the name of the validator is used for votes? And sometimes the wallet address?

[RowanUtrecht]

Also according to the figment page, Kytzu (the name of that validator) voted
https://hubble.figment.io/cosmos/chains/cosmoshub-1/governance/proposals/1

[Nostradamus411]

According to the file in the repo, that address did not vote on prop 1,2,3
According to mintscan, it did vote on prop 1,2,3 (I think it even voted on all 26 proposals that are stored on Mintscan. I can check again if needed)

Yep, this would be due to how data on proposal sis removed from state after they've ended their voting period from what I have gathered.

--height flag does nothing?

It also doesn't seem to me that the --height flag is working with gaiacli or I'm doing something wrong. Below show how the cli is returning the same deposit information regardless of the height flag trying to set the block to either one or one billion.

$ ~/cosmos/hub1/gaiacli query gov deposits 4 **--height=1000000000** --trust-node
Deposits for Proposal 4:
  cosmos1vjqault583hwktfj63k2hh7r02sj2a66v0chv3: 162000000uatom
  cosmos1sxx9mszve0gaedz5ld7qdkjkfv8z992arw3rr5: 250000000uatom
  cosmos1nqeu6cz8rc0a9sk7ett9ews32pmhcejv0nwlae: 100000000uatom
  cosmos1hzzkqn4kpqcvzauhdzlnkkkmr4ryaf8rj6rhkj: 1uatom

$ ~/cosmos/hub1/gaiacli query gov deposits 4 **--height=1** --trust-node
Deposits for Proposal 4:
  cosmos1vjqault583hwktfj63k2hh7r02sj2a66v0chv3: 162000000uatom
  cosmos1sxx9mszve0gaedz5ld7qdkjkfv8z992arw3rr5: 250000000uatom
  cosmos1nqeu6cz8rc0a9sk7ett9ews32pmhcejv0nwlae: 100000000uatom
  cosmos1hzzkqn4kpqcvzauhdzlnkkkmr4ryaf8rj6rhkj: 1uatom

Query Gov Votes -- broken?

Additionally when trying to make a gaiacli query gov votes call for a proposal I get errors:

$ ~/cosmos/hub1/gaiacli query gov votes 1 --height=1 --trust-node
panic: runtime error: index out of range
goroutine 1 [running]:
github.com/cosmos/cosmos-sdk/x/gov.Votes.String(0x0, 0x0, 0x0, 0x197, 0x0)
        /home/parallels/go/src/github.com/cosmos/cosmos-sdk/x/gov/depositsvotes.go:25 +0x263
github.com/cosmos/cosmos-sdk/client/context.CLIContext.PrintOutput(0xc000140af0, 0x0, 0x11ff9a0, 0xc000c6ac60, 0x0, 0x0, 0x11d0860, 0xc000010018, 0xf33ec5, 0x4, ...)
        /home/parallels/go/src/github.com/cosmos/cosmos-sdk/client/context/context.go:258 +0x24c
github.com/cosmos/cosmos-sdk/x/gov/client/cli.GetCmdQueryVotes.func1(0xc0001a3680, 0xc000c57380, 0x1, 0x3, 0x0, 0x0)
        /home/parallels/go/src/github.com/cosmos/cosmos-sdk/x/gov/client/cli/query.go:242 +0x79f
github.com/cosmos/cosmos-sdk/vendor/github.com/spf13/cobra.(*Command).execute(0xc0001a3680, 0xc000c572c0, 0x3, 0x3, 0xc0001a3680, 0xc000c572c0)
        /home/parallels/go/src/github.com/cosmos/cosmos-sdk/vendor/github.com/spf13/cobra/command.go:762 +0x465
github.com/cosmos/cosmos-sdk/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc000152000, 0x878c3f, 0xc000152000, 0xf32f03)
        /home/parallels/go/src/github.com/cosmos/cosmos-sdk/vendor/github.com/spf13/cobra/command.go:852 +0x2c0
github.com/cosmos/cosmos-sdk/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /home/parallels/go/src/github.com/cosmos/cosmos-sdk/vendor/github.com/spf13/cobra/command.go:800
github.com/cosmos/cosmos-sdk/vendor/github.com/tendermint/tendermint/libs/cli.Executor.Execute(0xc000152000, 0x10a9e10, 0x2, 0xc000b583c0)
        /home/parallels/go/src/github.com/cosmos/cosmos-sdk/vendor/github.com/tendermint/tendermint/libs/cli/setup.go:89 +0x3c
main.main()
        /home/parallels/go/src/github.com/cosmos/cosmos-sdk/cmd/gaia/cmd/gaiacli/main.go:103 +0x76a

[jaekwon]

I believe ABCI queries don't work with all --height options due to pruning; and probably would be difficult to make it work without pruning due to it not being designed for it. Or maybe that's what the unpruned database provides -- which did you use?

[jaekwon]

Maybe instead of ABCI state, we can iterate over the blockchain instead of state, and look for ABCI response codes to see which governance vote transactions succeeded. The results are always included in the following block (not the current block), and are probably stored by tendermint under block info or block meta data.

[Nostradamus411]

which did you use?

I downloaded the non-pruned 59GB hub1 database.

The results are always included in the following block (not the current block), and are probably stored by tendermint under block info or block meta data.

So I'm looking at the tendermint rpc block get request, that what you're referring to? Not sure what to be looking at to identify ABCI response codes.

I see there is a block_meta portion of the result

"block_meta": {
"block_id": {
"hash": "954B81C860ABF508F272C2A402AC81BFBBDA8BA4B2A627A60730FA36915C53AF",
"parts": {
"total": "1",
"hash": "842ADFA93717FDBE551BDAB96BED2E50D67FCA216421C97DE1A47BC61FF50E58"
}
},
"header": {
"version": {
"block": "10",
"app": "0"
},
"chain_id": "cosmoshub-1",
"height": "7",
"time": "2019-03-13T23:08:19.589587064Z",
"num_txs": "1",
"total_txs": "1",
"last_block_id": {
"hash": "A5873B845F3CD93203D8E1306F756F3BD93E1E3828A2343FD9B898D4DD66BD55",
"parts": {
"total": "1",
"hash": "3E5A9C4ACC1C217DAEBD68E29B99B013603D2902BF974F9695FCA08E1E4E8867"
}
},
"last_commit_hash": "D164CF0C566EEB93829EF72FE8C1921FA6EB2CFBFC578433F684B95FDEDAEFC9",
"data_hash": "E9734E910DFA6A6B806DC96E76E9D8D260E938E1E4B4B9D215C38422BF22AAB3",
"validators_hash": "50006FF803B9ED42EB8A3BAD658917FAD1472AD35C0560F05913F1144F07FEEB",
"next_validators_hash": "50006FF803B9ED42EB8A3BAD658917FAD1472AD35C0560F05913F1144F07FEEB",
"consensus_hash": "29C5629148426FB74676BE07F40F2ED79674A67F5833E4C9CCBF759C9372E99C",
"app_hash": "B48429AC69D3B1878D50F80D51EFD7085E9998B2570DF535A59E62FF1EE8F3DC",
"last_results_hash": "",
"evidence_hash": "",
"proposer_address": "9C17C94F7313BB4D6E064287BEEDE5D3888E8855"
}
},

and a data dict with txs in it.

{'txs': ['zgHwYl3uCkaSHS5OChR7rwr+AtElgr5ihpdR5pJmvyoUAxIUhL/4TH3a0Ry4wHOG6RkoxWdcpLwaFAoFdWF0b20SCzUwMDAwMDAwMDAwEhQKDgoFdWF0b20SBTY5NzUyEMrWBRpqCibrWumHIQLCsZ2sHeLoT+KmkAxNBsnAZrqS2QuF61pUvZ/FTYVQyhJAsJaWDT3mC2I03PGW9xyTtIgsxrB6Jgvi/3yRThRM6AF9apk3DcNY3dyUwpmANBpMR+EuHoPkuDiqs0paEGW0tQ==']}

Are either of these the right area? Is this even the correct RPC get request to be making to check?

[jaekwon]

hopefully the old versions have this: /block_results

// BlockResults gets ABCIResults at a given height.
// If no height is provided, it will fetch results for the latest block.
//
// Results are for the height of the block containing the txs.
// Thus response.results.deliver_tx[5] is the results of executing
// getBlock(h).Txs[5]
//
// shell // curl 'localhost:26657/block_results?height=10' //

please try that one.

[Nostradamus411]

@jaekwon we have the source code, no need to reverse engineer it any longer.

Can find it all here: https://github.com/Nostradamus411/ion-airdrop-code

[jaekwon]

Thank you @Nostradamus411. The main bounty was for cosmoshub 1,2,3 data. I will definitely reward people who have worked on the Osmosis airdrop analysis, but primarily I'm looking for a script for general usage. Thank you! Also, I am adding to the bounty, yet unspecified amount of GNO tokens. Cheers!

[jaekwon]

Thank you, please check this summary: https://github.com/gnolang/bounties/blob/main/bounties/001_cosmoshub_blockchain_data/README.md