-
Notifications
You must be signed in to change notification settings - Fork 252
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Virtualize WAL methods #53
Commits on Dec 13, 2022
-
pager, wal: virtualize WAL methods
This preparatory commit moves all WAL routines to a virtual table. Also, a helper libsql_open() function is provided. It allows passing a new parameter - name of the custom WAL methods implementation.
Configuration menu - View commit details
-
Copy full SHA for 519a33b - Browse repository at this point
Copy the full SHA 519a33bView commit details -
wal: add bUsesShm flag to declare relying on shared memory
Before enabling WAL journaling, libSQL detects whether WAL is fully supported. Historically it meant either operating in exclusive mode or supporting shared memory locks in the underlying VFS, but it may not be the case with custom WAL implementations. Thus, custom WAL methods can declare whether they rely on shared memory - if not, it will also not get verified when checking for WAL support.
Configuration menu - View commit details
-
Copy full SHA for c927cac - Browse repository at this point
Copy the full SHA c927cacView commit details -
wal: Virtualize pathname handling
It makes little sense to store the WAL file name if the virtual WAL implementation is not based on a single file. Therefore, WAL pathname handling is also virtualized, with the original implementation still producing a <dbname>-wal file.
Configuration menu - View commit details
-
Copy full SHA for 423512f - Browse repository at this point
Copy the full SHA 423512fView commit details -
ext: add a stub for custom WAL methods
This commit adds a stub implementation of custom WAL methods in ext/vwal subdirectory. It can be used as a starting point for implementing custom WAL routines.
Configuration menu - View commit details
-
Copy full SHA for 284c370 - Browse repository at this point
Copy the full SHA 284c370View commit details -
test: add a Rust test for virtual WAL
The test case registers a new naive virtual WAL coded on top of Rust's std::collections::HashMap. The WAL implementation only covers reading and writing pages, without checkpointing or savepoints, but it's enough to validate that the virtual method table works. After registering custom WAL, a few simple operations are performed on the database to validate that pages were indeed stored and retrieved properly. For extra logs, run the test with -- --nocapture.
Configuration menu - View commit details
-
Copy full SHA for ec756d9 - Browse repository at this point
Copy the full SHA ec756d9View commit details -
The command is heavily inspired by .vfslist and lists all the registered custom WAL methods.
Configuration menu - View commit details
-
Copy full SHA for 7e99eda - Browse repository at this point
Copy the full SHA 7e99edaView commit details -
wal: add autoinitialization to libsql_wal_methods* methods
These are customarily run early, before a call to libsql_open, so it makes sense to auto-initialize.
Configuration menu - View commit details
-
Copy full SHA for 8ca58ac - Browse repository at this point
Copy the full SHA 8ca58acView commit details -
ext: allow registering WAL methods via extension API
With this patch applied, WAL methods can be registered from a loadable extension module dynamically.
Configuration menu - View commit details
-
Copy full SHA for 5170e4e - Browse repository at this point
Copy the full SHA 5170e4eView commit details -
Expose wal_methods_unregister in the extension API as well
Not critical, as WAL methods would customarily live as long as the program that runs it, but it's good practice to be able to unregister during a cleanup.
Configuration menu - View commit details
-
Copy full SHA for 47e59b7 - Browse repository at this point
Copy the full SHA 47e59b7View commit details -
Add a pre-open-main-db-file hook
WAL is open lazily on first access, while it is useful to be able to set up ground for it even before the main db file is open. This optional hook allows it.
Configuration menu - View commit details
-
Copy full SHA for 798539e - Browse repository at this point
Copy the full SHA 798539eView commit details -
vwal: introduce iVersion for being future-proof
Similarly to how other interfaces work, the version number in WAL methods lets the user know which functions are supported, and which aren't yet.
Configuration menu - View commit details
-
Copy full SHA for 7c59954 - Browse repository at this point
Copy the full SHA 7c59954View commit details -
test: update the virtual WAL Rust test
The test now properly sets the iVersion and pre-main-db-open hook.
Configuration menu - View commit details
-
Copy full SHA for 8419336 - Browse repository at this point
Copy the full SHA 8419336View commit details
Commits on Dec 14, 2022
-
wal remove ifdefs from libsql_wal_methods
ABI should be consistent regardless of the compilation options, so the optional functions are in there anyway - they can be simply set to nulls if the user did not compile support for them in libSQL.
Configuration menu - View commit details
-
Copy full SHA for 929f9ba - Browse repository at this point
Copy the full SHA 929f9baView commit details -
wal: add optional pointer to private methods data in struct Wal
It will be useful for any state that custom WAL methods could potentially have.
Configuration menu - View commit details
-
Copy full SHA for 486dd38 - Browse repository at this point
Copy the full SHA 486dd38View commit details
Commits on Dec 20, 2022
-
In amalgamation mode it compiled just fine, but otherwise it relies on the wal.h header now.
Configuration menu - View commit details
-
Copy full SHA for c150084 - Browse repository at this point
Copy the full SHA c150084View commit details -
doc: add a paragraph on virtual WAL
The paragraph briefly explains the newly introduced mechanism.
Configuration menu - View commit details
-
Copy full SHA for 045225e - Browse repository at this point
Copy the full SHA 045225eView commit details