Update Replication Offset at shutdown #1223
Comments
readysetbot
pushed a commit
that referenced
this issue
Apr 22, 2024
The MySQL min and max positions can be far apart. This happens on unbalance workloads, where for example one of the tables do not receive updates for a long period of time, like a config table that might be static. This causes the MySQL min position to fall behind the max position, in case of a restart, the replicator has to catch up starting from the min position. This causes a lot on unnecessary data to be re-streamed. Currently we only update the min position when MySQL rotates the binary log and we receive a EventType::ROTATE_EVENT. Adjusting the position on all tables might be costly if the installation has a lot of tables. This commit adjust the replicator to adjust table position on a fixed interval. This interval is hardcoded to 60 seconds. The event we act on is either the EventType::QueryEvent when we receive a "COMMIT" query, or the EventType::XidEvent. They are virtually the same thing (a commit), but depending on the storage engine MySQL reports a query COMMIT or an XID event. We also report the position once we have finished the initial catch up phase. Ref: REA-4326 Closes: #1223 Change-Id: I6dfaf523b8851597a6a0fd97f4d4627ca2f4ea80
readysetbot
pushed a commit
that referenced
this issue
Apr 24, 2024
The MySQL min and max positions can be far apart. This happens on unbalance workloads, where for example one of the tables do not receive updates for a long period of time, like a config table that might be static. This causes the MySQL min position to fall behind the max position, in case of a restart, the replicator has to catch up starting from the min position. This causes a lot on unnecessary data to be re-streamed. Currently we only update the min position when MySQL rotates the binary log and we receive a EventType::ROTATE_EVENT. Adjusting the position on all tables might be costly if the installation has a lot of tables. This commit adjust the replicator to adjust table position on a fixed interval. This interval is hardcoded to 60 seconds. The event we act on is either the EventType::QueryEvent when we receive a "COMMIT" query, or the EventType::XidEvent. They are virtually the same thing (a commit), but depending on the storage engine MySQL reports a query COMMIT or an XID event. We also report the position once we have finished the initial catch up phase. Ref: REA-4326 Closes: #1223 Change-Id: I6dfaf523b8851597a6a0fd97f4d4627ca2f4ea80
readysetbot
pushed a commit
that referenced
this issue
May 7, 2024
The MySQL min and max positions can be far apart. This happens on unbalance workloads, where for example one of the tables do not receive updates for a long period of time, like a config table that might be static. This causes the MySQL min position to fall behind the max position, in case of a restart, the replicator has to catch up starting from the min position. This causes a lot on unnecessary data to be re-streamed. Currently we only update the min position when MySQL rotates the binary log and we receive a EventType::ROTATE_EVENT. Adjusting the position on all tables might be costly if the installation has a lot of tables. This commit adjust the replicator to adjust table position on a fixed interval. This interval is hardcoded to 60 seconds. The event we act on is either the EventType::QueryEvent when we receive a "COMMIT" query, or the EventType::XidEvent. They are virtually the same thing (a commit), but depending on the storage engine MySQL reports a query COMMIT or an XID event. We also report the position once we have finished the initial catch up phase. Ref: REA-4326 Closes: #1223 Change-Id: I6dfaf523b8851597a6a0fd97f4d4627ca2f4ea80
readysetbot
pushed a commit
that referenced
this issue
May 7, 2024
The MySQL min and max positions can be far apart. This happens on unbalance workloads, where for example one of the tables do not receive updates for a long period of time, like a config table that might be static. This causes the MySQL min position to fall behind the max position, in case of a restart, the replicator has to catch up starting from the min position. This causes a lot on unnecessary data to be re-streamed. Currently we only update the min position when MySQL rotates the binary log and we receive a EventType::ROTATE_EVENT. Adjusting the position on all tables might be costly if the installation has a lot of tables. This commit adjust the replicator to adjust table position on a fixed interval. This interval is hardcoded to 60 seconds. The event we act on is either the EventType::QueryEvent when we receive a "COMMIT" query, or the EventType::XidEvent. They are virtually the same thing (a commit), but depending on the storage engine MySQL reports a query COMMIT or an XID event. We also report the position once we have finished the initial catch up phase. Ref: REA-4326 Closes: #1223 Change-Id: I6dfaf523b8851597a6a0fd97f4d4627ca2f4ea80
readysetbot
pushed a commit
that referenced
this issue
May 13, 2024
The MySQL min and max positions can be far apart. This happens on unbalance workloads, where for example one of the tables do not receive updates for a long period of time, like a config table that might be static. This causes the MySQL min position to fall behind the max position, in case of a restart, the replicator has to catch up starting from the min position. This causes a lot on unnecessary data to be re-streamed. Currently we only update the min position when MySQL rotates the binary log and we receive a EventType::ROTATE_EVENT. Adjusting the position on all tables might be costly if the installation has a lot of tables. This commit adjust the replicator to adjust table position on a fixed interval. This interval is hardcoded to 10 seconds. The event we act on is either the EventType::QueryEvent when we receive a "COMMIT" query, or the EventType::XidEvent. They are virtually the same thing (a commit), but depending on the storage engine MySQL reports a query COMMIT or an XID event. We also report the position once we have finished the initial catch up phase. Ref: REA-4326 Closes: #1223 Change-Id: I6dfaf523b8851597a6a0fd97f4d4627ca2f4ea80
readysetbot
pushed a commit
that referenced
this issue
May 15, 2024
The MySQL min and max positions can be far apart. This happens on unbalance workloads, where for example one of the tables do not receive updates for a long period of time, like a config table that might be static. This causes the MySQL min position to fall behind the max position, in case of a restart, the replicator has to catch up starting from the min position. This causes a lot on unnecessary data to be re-streamed. Currently we only update the min position when MySQL rotates the binary log and we receive a EventType::ROTATE_EVENT. Adjusting the position on all tables might be costly if the installation has a lot of tables. This commit adjust the replicator to adjust table position on a fixed interval. This interval is hardcoded to 10 seconds. The event we act on is either the EventType::QueryEvent when we receive a "COMMIT" query, or the EventType::XidEvent. They are virtually the same thing (a commit), but depending on the storage engine MySQL reports a query COMMIT or an XID event. We also report the position once we have finished the initial catch up phase. Ref: REA-4326 Closes: #1223 Release-Note-Core: Adjusted MySQL replicator to report table position on a fixed interval(10 seconds). This makes the replicator to keep distance short between min and max positions. This is useful when Readyset restart, ensuring that we do not have to re-stream a lot of binary logs to catch up. Change-Id: I6dfaf523b8851597a6a0fd97f4d4627ca2f4ea80
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Currently we record the replication offset on a per table basis. We have the notion of min offset and max offset.
During start-up we run the catch-up, which is re-stream replication from the minimum table offset readyset has until the max offset.
We update the offsets when:
If an user has an unbalanced workload, like for example a config table, which is static, we will always be re-streaming unnecessary replication events.
As one example, MySQL binlog can have up to 1GB, which will be a lot of unnecessary work to be performed at each restart.
Readyset replicator should emit a
ReplicationAction::LogPositionevent during normal shutdown.Change in user-visible behavior
Requires documentation change
The text was updated successfully, but these errors were encountered: