feat: w3 aggregation #51
Conversation
|
I was reading through this and noted down a slightly simplier version (IMO! haha) https://hackmd.io/ZjfqlXaPRtO39rVzpOqy_A?view I don't really like using "filecoin" everywhere, this is actually a service for submitting aggregates to another service that makes deals with Filecoin storage providers. It's pretty far removed. |
|
@alanshaw using aggregateCid for initial request is a good approach. We need to define the schema inside only. I am not sure aggregate is a good naming though. My main concerns at the moment, are the pull based logic Spade is requiring. We would need to keep state in the proxy which I would avoid to the bare minimum and not have state there. This is, w3filecoin should be pulled if Spade will always work like this |
It is only the list of aggregates that spade hasn't pulled yet...🤷♂️ I'm kinda ok with that...
...but, why? 🙈 |
vasco-santos
force-pushed
the
feat/filecoin-spec
branch
2 times, most recently
from
f9a673e
to
635884e
Compare
vasco-santos
force-pushed
the
feat/filecoin-spec
branch
from
635884e
to
898f2eb
Compare
There was a problem hiding this comment.
Submitted a feedback inline, tldr;
- I think we should frame it as a following steps
- Storefront offers broker an aggregate
- Broker queues the offer and returns effect that when performed will review the offer
- Broker reviews an offer and issues receipt for either accepting (success) or denying (failure) the offer.
- Once offer is accepted broker takes care of arranging & renewing deals
- Storefront can query state of the aggregate deals using
aggregate/get
Co-authored-by: Irakli Gozalishvili <[email protected]>
Co-authored-by: Irakli Gozalishvili <[email protected]>
w3-aggregation.md
Outdated
| ] | ||
| ``` | ||
|
|
||
| Each entry of the decoded offers block, has all the necessary information for a Storage Provider to fetch and store a CAR file. The `link` field has the CAR CID, while the `commitmentProof` field has the required `proof` bytes by Storage Providers (for example, `commP`). The `src` field of each piece MUST be set to a (alphabetically sorted) list of URLs from which it can be fetched. The `size` field MUST be set to the byte size of the CAR file. |
There was a problem hiding this comment.
Do we want to specify that the src field is optional (allowing it to be provided when the deal is signed)?
Co-authored-by: Alan Shaw <[email protected]>
| @@ -179,7 +179,7 @@ Invoking `aggregate/offer` capability submits an aggregate to a broker service f | |||
| ] | |||
| ``` | |||
|
|
|||
| Each entry of the decoded offers block, has all the necessary information for a Storage Provider to fetch and store a CAR file. The `link` field has the CAR CID, while the `commitmentProof` field has the required `proof` bytes by Storage Providers (for example, `commP`). The `src` field of each piece MUST be set to a (alphabetically sorted) list of URLs from which it can be fetched. The `size` field MUST be set to the byte size of the CAR file. | |||
| Each entry of the decoded offers block, has all the necessary information for a Storage Provider to fetch and store a CAR file. The `link` field has the CAR CID, while the `commitmentProof` field has the required `proof` bytes by Storage Providers (for example, `commP`). The `size` field MUST be set to the byte size of the CAR file. The `src` field of each piece MUST be set to a (alphabetically sorted) list of URLs from which it can be fetched. Note that `src` field is optional and can be provided in a different part of the flow such as when deal is signed or through a previously agreed API. | |||
There was a problem hiding this comment.
Hey folks, this is concerning. The byte/payload size of the CAR file is not something payload size mentions from the spec 😅
There was a problem hiding this comment.
@ribasushi good callout. It is true that Spade does not care about this and won't validate.
We can validate this in our side if we think that is important (perform HEAD requests to the src URLs). Main goal of having the size in an invocation is that it allows us to track metrics for data that we are requesting to Spade over time. All our metrics are being built around the information within invocations.
With that in mind, is there any strong opinion against keeping size?
There was a problem hiding this comment.
You can keep it on your end, but there is very strong desire on the side of
So having something as a MUST when on my end I have MUST DISCARD is... odd ;)
There was a problem hiding this comment.
yeah, spade side won't need to do anything at it, even at the spade-proxy side. Just our UCAN Stream log processor will inspect these UCAN invocations and compute metrics over time from them.
Co-authored-by: Alan Shaw <[email protected]>
Preview