-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question: Slow loading of Explorer pagination #532
Comments
Hey! This might be a slightly misguided conclusion since I don't believe the log messages you refer to are directly tied to individual query requests. When navigating to a table in the Surrealist Explorer, only two queries are executed in parallel. The first query will fetch the records for the current page SELECT * FROM [table name] LIMIT [page max] START [page start]; The second, and likely slowest query, will count the total amount of records within a table in order to know how many pages you can retrieve SELECT count() AS count FROM [table name]; The first query will likely finish first, while the second query needs a longer time to complete, which is what the loading indicators in the UI represent. I believe this second query is also triggering these log messages. For context, which version of SurrealDB are you connecting to? |
Ok, my guess could very well have been wrong! But it was the only thing that I could think of to explain why it takes so long to count and populate the data. Why would a simple count query take dozens of seconds for 30-60000 rows (and MUCH longer for millions)? Moreover, it should probably not lock up the UI. I have the latest version of surrealdb and surrealist (and it has been happening since the very first v2/v3 releases, I just haven't reported it til now. It might have happened before that, but that's when I started working with surrealdb). |
Interesting, that's indeed unexpected. I have some follow up questions
|
Sorry, I wasn't clear enough. I'm on my phone right now, but I'm pretty sure that you can navigate the ui, but can't perform other operations while it is locked up with the spinner. I'll test again later to confirm. No schema, no indexes. I'll try the count query via the query view later. Though I did do a count query on a large table yesterday and it wasn't too slow (though, oddly, it was slower than simply returning fields for each result - I'll provide details on that later too) |
Hi @nickchomey there are a few things going on here:
Could you check what the performance is like when running with |
Trace was the issue! You know your product well. It doesn't happen with any other log level (other than, obviously, full). I dont recall why I turned on trace, but it might just be because it is in most of the commands in the docs. eg https://surrealdb.com/docs/surrealdb/installation/running/memory I'll use |
Thanks @nickchomey. We'll update the documentation so that this isn't as confusing. Thanks for your suggestion 🚀 🎉 ! |
Describe the bug
When I bulk insert many records via an sdk, and then try to access the table in Surrealist, I can see in the surrealdb server logs that it is just rapidly sending requests, seemingly to fetch each record one by one. See the attached screen recording where I added 34k and 67k records into two tables (which really isn't very much). It takes a very long time to load those tables. I've done it on other data that has millions of rows and it takes like 10 minutes.
Code_JbdI6c0V22.mp4
(note, ignore that it shows how many records and pages there are right away for the 1st table - I think that info was cached in surrealist already. You can see how it normally behaves when I switch to the 2nd table)
Steps to reproduce
Expected behaviour
I expect it to be fast, or at least tolerable.
Surrealist Environment
Version: 3.0.8
Flags: feature_flags: false, query_view: true, explorer_view: true, graphql_view: true, designer_view: true, auth_view: true, functions_view: true, models_view: true, apidocs_view: true, cloud_view: true, themes: true, newsfeed: true, database_version_check: true, highlight_tool: false, legacy_serve: false, cloud_endpoints: production, cloud_access: false, cloud_killswitch: true, changelog: auto
Contact Details
No response
Is there an existing issue for this?
Code of Conduct
The text was updated successfully, but these errors were encountered: