RawRange query

[chipshort]

We currently have a way to read from and range through our own storage, but for external storage we can only read a single key using WasmQuery::Raw.

We should add a way to query a range of keys from an external contract.
This should be paginated to be able to avoid gas / memory exhaustion if the external contract is not fully trusted.

Something like this?

RawRange {
    /// Inclusive bound
    start: Option<Vec<u8>>,
    /// Maximum number of elements to return
    limit: u16,
    order: Order,
}

I'm a little conflicted here because on the one side it would be nice to stay close to the way Storage::range works (start, end, order), but with that it's not as easy to control the amount of items you get. Using the end key for that is impractical because it leads to very short responses in sparsely populated maps. We could add an additional limit to that, but then it becomes a bit complicated to reason about.

[webmaster128]

I'm a little conflicted here because on the one side it would be nice to stay close to the way Storage::range works (start, end, order), but with that it's not as easy to control the amount of items you get. Using the end key for that is impractical because it leads to very short responses in sparsely populated maps. We could add an additional limit to that, but then it becomes a bit complicated to reason about.

The big difference between the two cases is that Storage::range is an iterator interface you can always stop in the middle of the operation with something like .take(my_limit).

---

I think in this case it is a bit hard decide which fields belong in the main query and which fields are pagination-related because it's somewhat all the same.

The analogous query in wasmd is this: AllContractState (https://github.com/CosmWasm/wasmd/blob/v0.52.0/proto/cosmwasm/wasm/v1/query.proto#L154-L161). What this is missing is:

• start key
• end key
  But we see a clear separation between the pagination rest of the query.

That being said, at some place in the query request we need to add:

• contract address
• start key/end key for cases other than exports. E.g. get the contents of a single Map in a different contract
• order
• Max number of elements

Then the response needs to indicate

• data
• next_key as in AllDenomMetadataResponse

[chipshort]

Alright, so we have our own PageRequest type already, but I don't think it makes sense in this case, since we want a mix of start + end and limit.
So, we are looking at something like this:

enum QueryRequest {
    // ...
    RawRange {
        contract_address: String,
        /// Inclusive bound
        start: Option<Vec<u8>>,
        /// Exclusive bound
        end: Option<Vec<u8>>,
        /// Maximum number of elements to return
        limit: u16,
        /// The order in which you want to receive the key-value pairs
        order: Order,
    }
}

struct RawRangeResponse {
    /// The key-value pairs
    data: Vec<Record>,
    /// `None` if there are no more key-value pairs
    next_key: Option<Vec<u8>>,
}

Some points:

• if start > end, we return an empty response, even if order is descending (same behaviour as storage range)
• the more restrictive one out of limit and end wins
• if limit wins against end, replacing start with the next_key you get back allows you to continue ranging through.
• if end wins, next_key is None