-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sqlite Migrations: Table rebuilds #329
Comments
Yes, great RDBMS |
Re-opening. I've "regressed" this. (not sure if it ever actually worked since there weren't any tests) |
I've discovered a limitation that makes me wonder if the complication of rebuilds will exceed their value. The limitation Consider the following setup: Table "Parent" - no FK's
Scenario 1 and 2 could be resolved by just executing the proper
https://www.sqlite.org/pragma.html#pragma_foreign_keys This means we cannot execute an entire migration within one transaction, but need to break SQLite migrations into multiple transactions. Reverting to the beginning of a migration becomes impossible. |
You are entering a World of Pain 😃 |
… validation and better testing. See #329
@natemcmaster Did you try the approach recommended in the SQLite documentation: Making Other Kinds Of Table Schema Changes? Triggers will be lost. Views may be broken. Indexes (like columns) would be re-created from the model snapshot. But otherwise it looks like it'll work. Note, we won't be able to use the simpler and faster procedure since SQLite doesn't support variables and Migrations is designed in a way that lets you generate SQL scripts. |
Making this a pri1 issue given the new information. |
Nothing really new here, just jotting down some notes since I started thinking about this again today: // UNDONE: Not supported by SQLite
//migrationBuilder.AlterColumn<string>(
// name: "Title",
// table: "Posts",
// nullable: false,
// defaultValue: "",
// oldClrType: typeof(string),
// oldType: "TEXT",
// oldNullable: true);
// Create a new table with the desired schema
migrationBuilder.CreateTable(
name: "ef_temp_Posts",
columns: table => new
{
Id = table.Column<int>(nullable: false)
.Annotation("Sqlite:Autoincrement", true),
Title = table.Column<string>(nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_Posts", x => x.Id);
});
// Create temporary UNIQUE indexes to enforce during INSERT
migrationBuilder.CreateIndex(
name: "ef_temp_IX_Posts_Title",
table: "ef_temp_Posts",
column: "Title",
unique: true);
// Copy data from the old table. Use IFNULL to specify a default value for newly
// required columns
migrationBuilder.Sql(@"
INSERT INTO ef_temp_Posts (Id, Title)
SELECT Id, IFNULL(Title, '')
FROM Posts;
");
// Suspend foreign key enforcement during the swap
// NB: This commits the current transaction. We can't rollback the migration if
// anything after this fails. We can mitigate it by doing rebuilds as late as
// possible
migrationBuilder.Sql("PRAGMA foreign_keys = 0;", suppressTransaction: true);
// Swap in the new table
migrationBuilder.DropTable(
name: "Posts");
migrationBuilder.RenameTable(
name: "ef_temp_Posts",
newName: "Posts");
// NB: This will cause an integrity check which may fail if foreign keys were
// not previously enforced and inconsistent data has crept in
migrationBuilder.Sql("PRAGMA foreign_keys = 1;", suppressTransaction: true);
// Drop temporary UNIQUE indexes
migrationBuilder.DropIndex(
name: "ef_temp_IX_Posts_Title",
table: "Posts",
column: "Title");
// Rebuild any indexes
migrationBuilder.CreateIndex(
name: "IX_Posts_Title",
table: "Posts",
column: "Title",
unique: true); |
I looked into using We discussed the consequences of We also discussed blindly turning foreign keys back on (when we don't know if they were enabled in the first place) and decided that it's OK. If you turn them off for performance reasons, your data should still be consistent otherwise additional things in EF will break. The default foreign key behavior will just be reset when the connection is closed and re-opened after migrating anyway. Other points that came up in the discussion:
|
From the SQLiteStudio there is a short sintax to clone the original table CREATE TABLE sqlitestudio_temp_table AS SELECT *
FROM Table; this way we can reduce the complexity. |
@danielmeza you would lose primary key and foreign keys
|
Yes, in that single statement data is lost. But it's a valuable technique for cloning the schema. One could then clear out the data and repopulate, bringing all keys along this time. |
@ir2000 yes but we not need that info in the temp table. |
@bricelam we should cover in this version escenarios with an existing db? Example: code first from db escenarios. |
@danielmeza I would say we need to create every constraint in the "temp table". Because what you call "temp table" is not really temporary. It is in fact the new table that will be kept once renamed. So I'm not sure CREATE TABLE AS is the best way to create the new table. |
@Damus765 I was missing the rename process, I thought that the data will be copy to the new table so the process goes like this. Create the temp table -> delete the old table -> create the new table with the new schema -> copy data back from the temp table -> delete the temp table. |
@danielmeza According to the SQLite documentation, the recommended steps are as follows:
Only one data copy is required and you don't need a temp table. |
@danielmeza Our strategy for using migrations with existing databases is tracked by #2167 |
But seriously, my six-year-old wasn't even born when I filed this issue. Crazy. May all your rebuilding dreams come true! Check out our daily builds if you're eager to give it a try. Let me know if you find any bugs. (I still have a bit more testing to do myself.) |
In SQLite, the following operations will require rebuilding the table - https://docs.microsoft.com/en-us/ef/core/providers/sqlite/limitations.
Other databases also have a smaller set of limitations that require a table rebuild (i.e. changing Identity on a column in SQL Server).
The text was updated successfully, but these errors were encountered: