-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve collections count #260
Improve collections count #260
Conversation
# Conflicts: # rxmongo/src/main/scala/akka/contrib/persistence/mongodb/RxMongoJournaller.scala # rxmongo/src/main/scala/akka/contrib/persistence/mongodb/RxMongoSnapshotter.scala
I'm a bit concerned this is unsupported on some of the mongo versions that this driver supports. Looks like a 4.x feature? https://docs.mongodb.com/manual/reference/method/db.collection.estimatedDocumentCount/#behavior |
Hi @scullxbones I missed that point, sorry. So, after digging into the code and performing mongo tests on some big data set, I can submit a new solution:
In both cases, this first count may be inaccurate in cluster environment. So, if it gives us zero, lets perform another count, using However, we can optimize this method, using appropriate index as a |
# Conflicts: # scala/src/main/scala/akka/contrib/persistence/mongodb/ScalaDriverPersistenceSnapshotter.scala
Can you implement this for the other drivers as well? At minimum the It's important to me to keep parity. |
Done. There is no simple way to call |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding the RXM implementation, LGTM
Hi,
This PR is about the ability to drop suffixed collections if empty.
For now, we used the
countDocuments
method that performs an aggregate operation in order to count all documents of the collection, which may be resource consuming if the collection contains a huge amount of data.However, as we only need to know if the collection is actually empty and not how many documents it contains, a better way could be to use the 'estimatedDocumentCount' method that sends us an immediate answer from metadata, performing no counting at all (as I understand from MongoDB documentation)
Happy reviewing