-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Majority without gaps #1544
Majority without gaps #1544
Conversation
src/main/scala/org/constellation/concurrency/cuckoo/CuckooFilter.scala
Outdated
Show resolved
Hide resolved
src/main/scala/org/constellation/concurrency/cuckoo/MutableCuckooFilter.scala
Show resolved
Hide resolved
src/main/scala/org/constellation/concurrency/cuckoo/MutableCuckooFilter.scala
Show resolved
Hide resolved
src/main/scala/org/constellation/concurrency/cuckoo/CuckooFilter.scala
Outdated
Show resolved
Hide resolved
src/main/scala/org/constellation/concurrency/cuckoo/MutableCuckooFilter.scala
Outdated
Show resolved
Hide resolved
src/main/scala/org/constellation/concurrency/cuckoo/MutableCuckooFilter.scala
Outdated
Show resolved
Hide resolved
src/main/scala/org/constellation/concurrency/cuckoo/MutableCuckooFilter.scala
Show resolved
Hide resolved
src/main/scala/org/constellation/infrastructure/endpoints/SnapshotEndpoints.scala
Outdated
Show resolved
Hide resolved
|
||
def persistPeerProposal(peer: Id, proposal: Signed[SnapshotProposal]): F[Unit] = | ||
def removePeerProposalsBelowHeight(height: Long): F[Unit] = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if we should just remove proposals based on our calculated majority. In case someone else gets stuck because of a missing proposal, we may not have that proposal anymore at the time when the node will be asking for it. I would take into account the filter and check if all other nodes have that proposal present/majority picked at certain height and only then remove it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We store at least 40 heights of proposals, a node that is 30 heights below or more would start redownloading itself already before we would remove any proposals he may need. These parameters could change, but implementing this would introduce just more complexity and with the current config, that complexity would never be used. Also, another case is what if someone doesn't have proposals at all, should we keep all of them indefinitely? If so memory leak, so another case to handle.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get your point. If you think it's too much hassle and no gain then there is no point in implementing that indeed. Especially if the logic right now is sealed e.i. all the cases are handled and we won't have nodes continuing ahead indefinitely - in consequence the redownloads will work as intended.
schema/src/main/scala/org/constellation/schema/serialization/SchemaKryoRegistrar.scala
Show resolved
Hide resolved
src/main/scala/org/constellation/infrastructure/endpoints/SnapshotEndpoints.scala
Outdated
Show resolved
Hide resolved
src/main/scala/org/constellation/infrastructure/endpoints/SnapshotEndpoints.scala
Show resolved
Hide resolved
c2beb36
to
e6de9a3
Compare
schema/src/main/scala/org/constellation/schema/snapshot/HeightRange.scala
Show resolved
Hide resolved
src/main/scala/org/constellation/concurrency/cuckoo/MutableCuckooFilter.scala
Outdated
Show resolved
Hide resolved
e6de9a3
to
6a18345
Compare
6a18345
to
a153785
Compare
a153785
to
5f06c24
Compare
5f06c24
to
8d21d4b
Compare
8d21d4b
to
36d122a
Compare
446b65f
to
5d6e608
Compare
Already merged to |
Resolves #1497
MajorityInfo
withCuckooFilter
POST /snapshot/proposal/_query