Modification: DataCap Removal Bot on Stale Applications

[simonkim0515]

Issue Description

Building upon insights from recent discussions from community members, the notary governance call on 8/22/23, and iterating from #967, a new proposal was made to address the situation of clients holding onto unused DataCap. The aim is to remove DataCap from all closed/stale applications that are showing no signs of activity and to push for clients to provide more engagement on their applications.

Impact

The focus is to support notaries by allowing them to push their efforts toward applications that are actively progressing and not stalled. Clients who are not ready and fully committed should come back when they are. Applications showing no signs of activity (28 days in total) should undergo DataCap reevaluation and that removing DataCap should not be seen as a punishment because clients will have the opportunity to reapply.

This underscores a broader notion that the client may not be fully prepared to accommodate the requested volume or the weekly onboarding rate that was stated in the application. It is important for clients to return when they are fully ready and are aligned with the information that was provided in their application. It is essential for clients to have all of the information on their application ready to go if they would like to apply or reapply for DataCap.

Clients are strongly advised to concentrate their efforts on their applications that are currently active and under development.

While we acknowledge that the rate of DataCap utilization should not be accelerated, transparency plays a pivotal role, so open communication and the use of DataCap is essential. When delays in DataCap utilization occur, proactive and transparent communication becomes imperative. Clients should provide status updates to ensure community members and notaries remain informed about the progress of the project and any obstacles that may affect the intended onboarding rate.

As always, in pursuit of a more comprehensive approach, this proposal is open to suggestions on the process and timing adjustments of DataCap being removed from stale applications.

Proposed Solution(s)

Reasons to remove DataCap from stale applications and unused DataCap:

• Applications inactive for 14 days after being closed/stale (totaling 28 days) face automatic DataCap removal.

• Client requested 'self-close'

• If a DataCap allocation is received, and an applicant continues to keep the application open by commenting, but DataCap has not been used after another 4 weeks, then the bot automatically closes the application and removes DataCap.

Process for reopening stale applications:

• Provide a detailed status updates to community members and notaries about where you currently stand in your project

• SPs you are working with (SP IDs, Entity/Organization Name, Location)

• Timeline of onboarding project

• Reasoning on delay of response and providing updates

Reopening applications require comprehensive updates on project status and delays. If unused DataCap persists after 4 weeks, automatic removal of DataCap ensues.

Timeline

Technical dependencies

End of POC checkpoint (if applicable)

Risks and mitigations

Related Issues

#967

[15012700225]

Do not tolerate DC abuse and oppose unhelpful operations. It is obvious that we have misunderstandings about the portrayal of abusers. Real customers need to consider the impact of various factors. Generally speaking, the pace is not very fast, but abusers and DC quota buyers and sellers, their market is broad, but their pace is fast. This measure will greatly harm normal customers, and oppose this proposal.

[alchemypunk]

Has the proposal already been implemented?

[zcfil]

Is this suggestion to inform everyone that it has been implemented? Or send it out for discussion?

[spaceT9]

Maybe we should vote for this?

[Destore2023]

Currently it's very controversial on these four active versions. Suggesting a vote to reach consensus.

A、2 weeks（closing）+2weeks（Removal）
B、2 weeks（closing）+8weeks （Removal）
C、2weeks（closing）+ keep available
D、Allowing Unrestricted Minting of Datacap

[WisseHeyne]

Do not tolerate DC abuse and oppose unhelpful operations. It is obvious that we have misunderstandings about the portrayal of abusers. Real customers need to consider the impact of various factors. Generally speaking, the pace is not very fast, but abusers and DC quota buyers and sellers, their market is broad, but their pace is fast. This measure will greatly harm normal customers, and oppose this proposal.

Agreed, we're also against this proposal.

[Aaronn85]

Currently it's very controversial on these four active versions. Suggesting a vote to reach consensus.

A、2 weeks（closing）+2weeks（Removal）
B、2 weeks（closing）+8weeks （Removal）
C、2weeks（closing）+ keep available
D、Allowing Unrestricted Minting of Datacap

We prefer D.

[MegaFil]

Currently it's very controversial on these four active versions. Suggesting a vote to reach consensus.

A、2 weeks（closing）+2weeks（Removal）
B、2 weeks（closing）+8weeks （Removal）
C、2weeks（closing）+ keep available
D、Allowing Unrestricted Minting of Datacap

Seems like a vote is necessary. We'll take C.

[Dave-lgtm]

We prefer C.

[simonkim0515]

This proposal has not been implemented. This is a new and refined proposal. Would be great to get feedback before a decision is finalized.

[Wengeding]

Prefer C: 2weeks（closing）+ keep available

DataCap Removal Bot would create more friction.

[alchemypunk]

Currently it's very controversial on these four active versions. Suggesting a vote to reach consensus.

A、2 weeks（closing）+2weeks（Removal）
B、2 weeks（closing）+8weeks （Removal）
C、2weeks（closing）+ keep available
D、Allowing Unrestricted Minting of Datacap

We prefer C, too.

[Chris00618]

Currently it's very controversial on these four active versions. Suggesting a vote to reach consensus.

A、2 weeks（closing）+2weeks（Removal）
B、2 weeks（closing）+8weeks （Removal）
C、2weeks（closing）+ keep available
D、Allowing Unrestricted Minting of Datacap

i prefer B.

[maxDi84]

We prefer C , thanks!

[1ane-1]

Looking forward the results！

[galen-mcandrew]

There seems to be some confusion here about what is being proposed, who is making the decision, and how the Fil+ community makes these iterative improvements.

This proposal is for automatic client DataCap removal proposals to RKH in 3 conditions:

• Inactive LDN, 28 days total (14 days inactive open + 14 days inactive closed)
• Client requested removal via comment in LDN
• Open LDN but no deal-making, client has not utilized any granted DataCap for 28 days (4 weeks)

We have heard opinions from community members, but this decision will be made by current active notaries. From checking the comments, there is only one active notary against this proposal to remove stale and client requested DataCap.

In checking https://datacapstats.io/, it looks like notaries have granted ~1.576EiB to clients, and ~1.5EiB has been used in deals with SPs. This means that less than 5% of the created DataCap is currently held by clients, and only a percentage of that would be considered stale under the described conditions.

Specifically, I would like to hear from more notaries:

• why they want to keep stale DataCap at unresponsive & non-deal-making addresses
• what their preferred timelines would be (more or less than 28 days)
• or whether they support the proposed conditions

[Chris00618]

Why aren't the rules announced in advance?
Why will this decision be made ONLY by current active notaries?
Did you guys start with the idea that Fil+ isn't about clients, so any rules block them out by default?

@galen-mcandrew

[nicelove666]

There seems to be some confusion here about what is being proposed, who is making the decision, and how the Fil+ community makes these iterative improvements.

This proposal is for automatic client DataCap removal proposals to RKH in 3 conditions:

• Inactive LDN, 28 days total (14 days inactive open + 14 days inactive closed)
• Client requested removal via comment in LDN
• Open LDN but no deal-making, client has not utilized any granted DataCap for 28 days (4 weeks)

We have heard opinions from community members, but this decision will be made by current active notaries. From checking the comments, there is only one active notary against this proposal to remove stale and client requested DataCap.

In checking https://datacapstats.io/, it looks like notaries have granted ~1.576EiB to clients, and ~1.5EiB has been used in deals with SPs. This means that less than 5% of the created DataCap is currently held by clients, and only a percentage of that would be considered stale under the described conditions.

Specifically, I would like to hear from more notaries:

• why they want to keep stale DataCap at unresponsive & non-deal-making addresses
• what their preferred timelines would be (more or less than 28 days)
• or whether they support the proposed conditions

I agree with @galen-mcandrew,It can helps to better regulate DC and make FIL+ healthier

[joshua-ne]

From checking the comments, there is only one active notary against this proposal to remove stale and client requested DataCap.

I think voting should be done more formally with notary signing, instead of just commenting. And we should also ensure most, if not all of the notaries get the voting notice. Last, but not the least, there should be a threshold of voting for the proposal to pass.

[joshua-ne]

Personally, I totally agree with the idea of removing stale DataCap. However, I feel not 100% comfortable with this procedure. I believe it might be better to announce to rules first (after it pass notary voting) and ONLY be applied to LDNs started after the formal announcement.

[simonkim0515]

Hey @Chris00618, so this proposal was actually built off of the discussion from the notary governance call on 8/22/23, which was recorded, and from #967.

Community member input is very important, however the insights of active community members/notaries, owing to their regular engagement and contributions, often hold a pivotal role to the program. However, we value and appreciate input from all members, recognizing the diverse perspectives they bring. A lot of comments that do not have reasoning often do not carry much influence.

[simonkim0515]

@junyaoren, let's try to keep the discussion about the proposal itself. Nonetheless, I agree with your perspective regarding a more structured notary signing process. If executed effectively, this could offer significant value in the future. It might be beneficial to start a discussion on this topic and collaborate on ideas with the community.

[AthSmith]

Personally, I totally agree with the idea of removing stale DataCap. However, I feel not 100% comfortable with this procedure. I believe it might be better to announce to rules first (after it pass notary voting) and ONLY be applied to LDNs started after the formal announcement.

In support of this point, we should clarify the rules and process before discussing the proposal itself.
Voting is the fairest and most effective way to address such issues.

[Chris00618]

A lot of comments that do not have reasoning often do not carry much influence.

Hey @simonkim0515
Perhaps I should have to remind you that this statement of yours is extremely misleading. We need to respect and tolerate opinions we can't understand, that's the spirit of web3.

You need to believe in the power of the community vote!

[Wengeding]

Our comments are as follows:

1、Inactive LDN, 28 days total (14 days inactive open + 14 days inactive closed)
Not Support
Reason：This is lazy government behavior. Inactivity does not equal abuse. The community is not really efficient, and reapplying will only make the experience worse for qualified participants.

2、Client requested removal via comment in LDN
Not Support
Reason：Just don't use it by DPs$Clients. No need to waste energy modifying code

3、Open LDN but no deal-making, client has not utilized any granted DataCap for 28 days (4 weeks)
Not Support
Reason：This is lazy government behavior. Inactivity does not equal abuse. This is a serious breach of the spirit of contract and we have no right to do this to our clients, and reapplying will only make the experience worse for qualified participants.

[15012700225]

What are the criteria for an active notary?
@galen-mcandrew

[joshua-ne]

let's try to keep the discussion about the proposal itself

You are trying to avoid the main question here. My main point is not about how to vote, but instead the formal procedure, sufficient notification, and proper record and counting of the votes. Statements like "From checking the comments, there is only one active notary against this proposal to remove stale and client requested DataCap" can definitely not be counted as effective voting.

[simonkim0515]

@Wengeding @0xXPunkX @maxDi84 @WisseHeyne @HiFil

I want to understand a little more here. Is there an alternative solution that you would suggest if you don't agree with this proposal? The main idea is to remove DataCap from those who aren't using it. For the people who disagree with this proposal to keep stale DataCap at unresponsive & non-deal-making addresses, can you explain why. If there are no applications showing signs of life, why should they hold onto it when there's the opportunity to always reapply.

[simonkim0515]

@Chris00618 I agree with this that community input is important, maybe we can ideate on a tool around a voting process. As of now this does not exist. I'm happy to discuss more about this further if you would like.

[kernelogic]

Since there is only less than 5% DC stale as mentioned by Galen, I don't think it's necessary to target 100% DC recycle. I appreciate the effort to give enough time for client to reopen / reapply etc.

The only thing I am not in support is the following point. I think as long as client keep commenting, it can be considered the application is still active. There are many scenarios that onboarding is slowed down or paused and they can take months, I don't think we want to discourage "engaging clients" by forcing them to go through the time consuming re-apply process again. (BTW this is really easy to get around too, just onboard one deal every 20 some days, so it's kinda pointless to enforce this rule won't you say?)

If a DataCap allocation is received, and an applicant continues to keep the application open by commenting, but DataCap has not been used after another 4 weeks, then the bot automatically closes the application and removes DataCap.

[herrehesse]

I greatly appreciate your well-thought-out proposal. It is apparent to me that the potential negative repercussions associated with this initiative are practically negligible. It seems the only resistance is coming from those who might be exploiting the current system for their benefit. Unfortunately, they appear to be willing to ignore the evident misuse in the "keep open" and "closed by applicant" processes, focusing instead on minor potential pitfalls that could affect an insignificant number of individuals.

In my opinion, your proposal stands as an excellent move towards steering FIL+ towards a brighter and more sustainable future. I wholeheartedly recommend proceeding with the implementation at the earliest convenience.

[Chris00618]

@Chris00618 I agree with this that community input is important, maybe we can ideate on a tool around a voting process. As of now this does not exist. I'm happy to discuss more about this further if you would like.

Could you tell me why you think as of now this does not exist? @simonkim0515

[jamerduhgamer]

The only thing I am not in support is the following point. I think as long as client keep commenting, it can be considered the application is still active. There are many scenarios that onboarding is slowed down or paused and they can take months, I don't think we want to discourage "engaging clients" by forcing them to go through the time consuming re-apply process again. (BTW this is really easy to get around too, just onboard one deal every 20 some days, so it's kinda pointless to enforce this rule won't you say?)

If a DataCap allocation is received, and an applicant continues to keep the application open by commenting, but DataCap has not been used after another 4 weeks, then the bot automatically closes the application and removes DataCap.

I agree with Fei Yan's point here. We have run into similar issues where the onboarding has slowed down or stopped due to various reasons and I am actively commenting on the applications just to keep them open.

However, I am okay with using the work around and sealing 1 deal every 20 days is okay if we need to push this proposal through.

[1475Notary]

Based on the above comments and feedback from the active notaries. Seeing no absolute consensus, seems this proposal did not pass the community review and requires further iterations.

[herrehesse]

@jamerduhgamer I'm completely on board with your perspective. It's important to note, though, that your viewpoint represents a very small fraction of genuine actors; the majority are far from being as credible.

Eager to see this being put into action. Let those who misuse the platform voice their thoughts, but only on their own behalf, not representing the entire community.

[SmallMiner]

Removal is supported, but voting from the active clients and applicants is also required.

Let's not use the wrong method to do the right thing.

[simonkim0515]

Applications that are closed or deleted by the client, or when the client indicates they would no longer want DataCap or wish to continue with the onboarding process, should have their DataCap removed from the associated wallet. This indicates that the client does have interest with the associated application.

If the client ever wants to come back, they may always reapply.

[simonkim0515]

@kernelogic @jamerduhgamer

Let's consider a time-limited experiment, for example it can either be 6 months, 3 months, 2 months, etc. What do you think is a reasonable duration here? It is understandable that onboarding can face delays, but the client should inform to the community members about any application pauses. But would like your feedback and anyone who feels that 28 days (1 month) is insufficient.

[Kevin-FF-USA]

Will be discussing on the #978

[Chris00618]

@Chris00618 I agree with this that community input is important, maybe we can ideate on a tool around a voting process. As of now this does not exist. I'm happy to discuss more about this further if you would like.

Could you tell me why you think as of now this does not exist? @simonkim0515

any response?

[jamerduhgamer]

@kernelogic @jamerduhgamer

Let's consider a time-limited experiment, for example it can either be 6 months, 3 months, 2 months, etc. What do you think is a reasonable duration here? It is understandable that onboarding can face delays, but the client should inform to the community members about any application pauses. But would like your feedback and anyone who feels that 28 days (1 month) is insufficient.

I think 28 days is a sufficient duration. That is plenty of time for clients to inform the community of any application pauses.

[joshua-ne]

Since there is only less than 5% DC stale as mentioned by Galen, I don't think it's necessary to target 100% DC recycle. I appreciate the effort to give enough time for client to reopen / reapply etc.

Agree with this comment.

[joshua-ne]

Currently it's very controversial on these four active versions. Suggesting a vote to reach consensus.

A、2 weeks（closing）+2weeks（Removal） B、2 weeks（closing）+8weeks （Removal） C、2weeks（closing）+ keep available D、Allowing Unrestricted Minting of Datacap

I would support option D, and put real-deal incentivization outside core consensus.

[herrehesse]

Unrestricted minting of datacap yeah sure 😂

[Neal-fil]

Currently it's very controversial on these four active versions. Suggesting a vote to reach consensus.
A、2 weeks（closing）+2weeks（Removal） B、2 weeks（closing）+8weeks （Removal） C、2weeks（closing）+ keep available D、Allowing Unrestricted Minting of Datacap

I would support option D, and put real-deal incentivization outside core consensus.

Voting is the best way to choose, I'll go with option C.

[joshua-ne]

I agree a notary-address-signing-based-voting would fit best, and it should be quite easy to implement one.

[laurarenpanda]

Since there is only less than 5% DC stale as mentioned by Galen, I don't think it's necessary to target 100% DC recycle. I appreciate the effort to give enough time for client to reopen / reapply etc.

Agree with @kernelogic 's point.
FIL+ is an incentive mechanism to attract more data clients and encourage more valuable data onboard the Filecoin Network. We want more fresh blood to come into this ecosystem, which should be treated as beginners of Filecoin, instead of a master. There are a few steps, both technical and non-technical, that data clients would face when storing data on Filecoin.
I think it could be better to not set a strict limitation on time and give clients more time to do data processing, SPs matching, etc.

[spaceT9]

Is there an easy way to repen those applications? And when will the proposal be finalized?

[TobyShea]

C is the best choice.

[Chris00618]

Currently it's very controversial on these four active versions. Suggesting a vote to reach consensus.
A、2 weeks（closing）+2weeks（Removal）
B、2 weeks（closing）+8weeks （Removal）
C、2weeks（closing）+ keep available
D、Allowing Unrestricted Minting of Datacap

Looks like some people have officially initiated a FIP discussion to support “D”. Let's all go voice our opinions.
filecoin-project/FIPs#844