-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix slave's binlog offset greater than master bug #1682
fix slave's binlog offset greater than master bug #1682
Conversation
codis/pkg/proxy/router.go
Outdated
@@ -138,6 +138,7 @@ func (s *Router) isOnline() bool { | |||
func (s *Router) dispatch(r *Request) error { | |||
hkey := getHashKey(r.Multi, r.OpStr) | |||
var id = Hash(hkey) % uint32(models.GetMaxSlotNum()) | |||
r.HashID = id | |||
slot := &s.slots[id] | |||
return slot.forward(r, hkey) | |||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The provided code patch seems to be a small change in the dispatch
method of a Router
struct. The change adds a line that assigns the value of id
to r.HashID
. Here's a brief code review:
-
Variable Naming: The variable
id
used for storing the result ofHash(hkey) % uint32(models.GetMaxSlotNum())
could benefit from a more descriptive name to improve code readability. Consider using a name that reflects the purpose or meaning of this value. -
Error Handling: The code snippet does not contain any error handling logic. It's important to handle any potential errors from operations like
Hash()
andmodels.GetMaxSlotNum()
. Depending on your requirements, you may need appropriate error checks and error propagation mechanisms. -
Contextual Information: Without additional context, it's difficult to determine if this code patch is part of a larger program or system. Ensure that this change aligns with the overall design and architecture of the software.
Apart from these points, the code patch appears to be straightforward and efficient.
codis/pkg/proxy/request.go
Outdated
@@ -4,9 +4,6 @@ | |||
package proxy | |||
|
|||
import ( | |||
"sync" | |||
"unsafe" | |||
|
|||
"pika/codis/v2/pkg/proxy/redis" | |||
"pika/codis/v2/pkg/utils/sync2/atomic2" | |||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the provided code patch, it seems that some import statements have been removed: "sync" and "unsafe". It's difficult to perform a comprehensive code review without the full context of the codebase, but here are a few general suggestions:
-
Check if removing the "sync" and "unsafe" packages was intentional. These packages provide synchronization primitives and low-level memory access, respectively. Make sure their removal does not introduce any issues or break functionality.
-
Review the changes made to the "import" block. Verify that the remaining imports are still necessary and relevant.
-
Consider adding comments or documentation to explain the purpose and functionality of the package and individual functions, especially if they are non-trivial.
-
Pay attention to error handling. Ensure that appropriate error checks and handling are implemented throughout the codebase.
-
It's good practice to follow consistent formatting and coding conventions, such as indentation, naming conventions, and code organization, to improve code readability and maintainability.
-
Review any logic or algorithm changes introduced in the patch. Verify that they are correct and handle all possible edge cases.
Remember, without more information about the surrounding code, it's challenging to provide specific recommendations. It's always useful to test the modified code thoroughly to identify any potential bugs or performance issues.
(keys[0].compare(0, KSlotKeyPrefixLen, SlotKeyPrefix) == 0 || keys[0].compare(0, KSlotKeyPrefixLen, SlotTagPrefix) == 0)) { | ||
return Status::OK(); | ||
} | ||
|
||
LogOffset log_offset; | ||
|
||
stable_logger_->Logger()->Lock(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, the code patch you provided seems fine. Here are a few observations and suggestions:
-
The added code segment checks if the command name is "SAdd" and if the first key in the command starts with either "SlotKeyPrefix" or "SlotTagPrefix". If these conditions are met, it returns
Status::OK()
without further processing. This logic assumes thatkCmdNameSAdd
,KSlotKeyPrefixLen
, andSlotKeyPrefix
/SlotTagPrefix
are correctly defined and initialized elsewhere in the code. -
One improvement suggestion for the added code segment is to use
std::string::starts_with
function (introduced in C++20) instead of thecompare()
method. It provides a cleaner and more expressive way to check if a string starts with a specific prefix.
Example:
if (cmd_ptr->name() == kCmdNameSAdd && !keys.empty() &&
(keys[0].starts_with(SlotKeyPrefix) || keys[0].starts_with(SlotTagPrefix))) {
return Status::OK();
}
-
Make sure that the
LogOffset log_offset
variable is being used correctly after this code block. It seems it might be used later, so double-check its usage in the surrounding code for correctness. -
Consider adding comments to describe the purpose and rationale of the added code segment. Comments can improve code clarity and make it easier for future maintainers to understand the code's intent.
Remember to thoroughly test the code with various scenarios to ensure it functions as intended and doesn't introduce any unintended side effects.
src/pika_consensus.cc
Outdated
std::vector<std::string> keys = cmd_ptr->current_key(); | ||
// slotkey shouldn't add binlog | ||
if (cmd_ptr->name() == kCmdNameSAdd && !keys.empty() && | ||
(keys[0].compare(0, KSlotKeyPrefixLen, SlotKeyPrefix) == 0 || keys[0].compare(0, KSlotKeyPrefixLen, SlotTagPrefix) == 0)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
key_type == "s" && ((*iter).find(SlotKeyPrefix) != std::string::npos || (*iter).find(SlotTagPrefix) != std::string::npos)
别用上面那样写
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个没问题,代码中大量这种写法
include/pika_slot_command.h
Outdated
@@ -11,6 +11,8 @@ | |||
|
|||
const std::string SlotKeyPrefix = "_internal:slotkey:4migrate:"; | |||
const std::string SlotTagPrefix = "_internal:slottag:4migrate:"; | |||
const size_t KSlotKeyPrefixLen = 27; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
别这么写
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
金鸽的意思是,把 size_t 改为int,大家评估下
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
不是,27怎么来的?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
不是,27怎么来的?
SlotTagPrefix 字符串的长度
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
不是,27怎么来的?
优化了下,直接使用 length
codis/pkg/proxy/request.go
Outdated
@@ -4,7 +4,7 @@ | |||
package proxy | |||
|
|||
import ( | |||
"sync" | |||
"sync" | |||
"unsafe" | |||
|
|||
"pika/codis/v2/pkg/proxy/redis" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Below is a code review of the provided code patch:
- The import statement for the
"sync"
package has an indentation inconsistency. It should be indented with a tab or spaces to match the other imports. - The comment lines at the beginning of the file look incorrect. They should start with
//
instead of@@
. - There is an unused import for
"pika/codis/v2/pkg/proxy/redis"
. If it is not required, it can be safely removed.
Improvement suggestions:
- It's important to provide more context and details about the code being reviewed in order to identify potential bug risks or improvements accurately.
- Include the entire code snippet or function if possible, so that I can analyze it more comprehensively.
Please provide more specific information or code if you'd like a more detailed review or have any particular concerns.
@@ -32,7 +34,6 @@ std::string GetSlotKey(int slot); | |||
std::string GetSlotsTagKey(uint32_t crc); | |||
int GetSlotsID(const std::string &str, uint32_t *pcrc, int *phastag); | |||
void RemSlotKeyByType(const std::string &type, const std::string &key, const std::shared_ptr<Slot>& slot); | |||
void WriteSAddToBinlog(const std::string &key, const std::string &value, const std::shared_ptr<Slot>& slot); | |||
|
|||
class PikaMigrate { | |||
public: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the code patch you provided, here's a brief code review:
-
It seems like
kSlotKeyPrefixLen
is introduced as a constant to store the length ofSlotKeyPrefix
. -
WriteSAddToBinlog()
function has been removed or commented out. Ensure that this change doesn't affect the intended functionality. Verify if this function is no longer needed or if it should be replaced with an alternative implementation.
Apart from these specific changes, without the context of the entire codebase and its purpose, it's challenging to provide further bug risks or improvement suggestions. A comprehensive code review typically involves examining the logic, error handling, performance optimizations, coding conventions, and adherence to best practices. Consider providing more details or sharing the rest of the code if you require a more thorough review.
@@ -31,7 +33,6 @@ std::string GetSlotKey(int slot); | |||
std::string GetSlotsTagKey(uint32_t crc); | |||
int GetSlotsID(const std::string &str, uint32_t *pcrc, int *phastag); | |||
void RemSlotKeyByType(const std::string &type, const std::string &key, const std::shared_ptr<Slot>& slot); | |||
void WriteSAddToBinlog(const std::string &key, const std::string &value, const std::shared_ptr<Slot>& slot); | |||
|
|||
class PikaMigrate { | |||
public: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here are some observations and suggestions for code review:
-
Inconsistent formatting: The indentation of the code seems to be inconsistent. It's important to maintain a consistent coding style for better readability.
-
Magic numbers: The value "27" appears in the code as the length of
SlotKeyPrefix
. Consider defining it as a constant, similar tokSlotKeyPrefixLen
, to improve code clarity. -
Unused function: The function
WriteSAddToBinlog
is declared but not defined or used in the provided code patch. Make sure it is necessary and remove it if it's not needed. -
Missing function details: It would be helpful to see the implementation and usage of the functions
GetSlotKey()
,GetSlotsTagKey()
,GetSlotsID()
,RemSlotKeyByType()
, and the classPikaMigrate
to perform a more comprehensive code review. -
Commented out code: There doesn't seem to be any commented-out code in the provided patch, but it is good practice to remove any commented-out code from the final codebase.
Note: Without the complete code context and implementation details, it may be challenging to identify potential bugs or provide further improvement suggestions.
g_pika_rm->GetSyncMasterSlotByName(SlotInfo(slot->GetDBName(), slot->GetSlotID())); | ||
Status s = sync_slot->ConsensusProposeLog(cmd_ptr); | ||
if (!s.ok()) { | ||
LOG(ERROR) << "write sadd key to binlog failed, key: " << key; | ||
} | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the provided code patch, here are some observations and suggestions:
-
It appears that the function
WriteSAddToBinlog
has been removed from the code. If this is intentional and no longer required, then the removal is fine. Otherwise, double-check if it should be kept or if there's an alternative implementation planned. -
There doesn't seem to be any bug risk in the modified code section since you have removed the
WriteSAddToBinlog
function call along with its log output. However, without the full context of the codebase, it's challenging to assess the overall impact of these changes. -
Consider refactoring the code to adhere to consistent coding conventions for better readability. For example, use meaningful parameter names instead of single-letter variable names like
type
,key
, etc., to make the code more understandable. -
Ensure that error handling is appropriate in the calling function. Since the call to
WriteSAddToBinlog
has been removed, it's essential to handle errors or perform any alternative actions if necessary. -
Review the surrounding code and unit tests to verify that these changes do not introduce any unwanted side effects or break existing functionality.
-
It's generally a good practice to include comments explaining the purpose and intention behind each code change to provide clear documentation for future developers.
Remember to thoroughly test the modified code to ensure its correctness and robustness after making any changes.
@@ -31,7 +32,6 @@ std::string GetSlotKey(int slot); | |||
std::string GetSlotsTagKey(uint32_t crc); | |||
int GetSlotsID(const std::string &str, uint32_t *pcrc, int *phastag); | |||
void RemSlotKeyByType(const std::string &type, const std::string &key, const std::shared_ptr<Slot>& slot); | |||
void WriteSAddToBinlog(const std::string &key, const std::string &value, const std::shared_ptr<Slot>& slot); | |||
|
|||
class PikaMigrate { | |||
public: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without the complete context and understanding of the codebase, it is challenging to provide a thorough code review. However, based on the provided code patch, here are some observations:
-
Unused Function: The function
WriteSAddToBinlog
seems to be declared but not used in the code patch. If it is not necessary, consider removing it or verifying if it serves any purpose. -
Magic Numbers: The value
10 * 1024
appears asMaxKeySendSize
. Consider using named constants or variables with descriptive names instead of "magic numbers" to improve code readability and maintainability. -
Style Consistency: Ensure consistent code formatting and indentation throughout the codebase for better readability and maintainability.
Apart from these specific observations, perform a thorough testing of the modified code to check for bug risks or potential improvements. Additionally, consider including comments that explain the purpose and functionality of key functions or blocks of code to aid future maintenance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
没意见。文档说明一下特殊key的处理。能测的话加一下测试用例
cp ../conf/pika.conf ./pika_master.conf | ||
cp ../conf/pika.conf ./pika_slave.conf | ||
cp ../tests/conf/pika.conf ./pika_master.conf | ||
cp ../tests/conf/pika.conf ./pika_slave.conf | ||
mkdir slave_data | ||
sed -i -e 's|databases : 1|databases : 2|' -e 's|#daemonize : yes|daemonize : yes|' ./pika_master.conf | ||
sed -i -e 's|databases : 1|databases : 2|' -e 's|port : 9221|port : 9231|' -e 's|log-path : ./log/|log-path : ./slave_data/log/|' -e 's|db-path : ./db/|db-path : ./slave_data/db/|' -e 's|dump-path : ./dump/|dump-path : ./slave_data/dump/|' -e 's|pidfile : ./pika.pid|pidfile : ./slave_data/pika.pid|' -e 's|db-sync-path : ./dbsync/|db-sync-path : ./slave_data/dbsync/|' -e 's|#daemonize : yes|daemonize : yes|' ./pika_slave.conf |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the code patch you provided, here are some observations and recommendations for improvement:
-
File Paths:
- The script is copying the
pika.conf
file from either../conf/
or../tests/conf/
to the current directory. Make sure these paths exist and point to the correct location. - It's generally a good practice to use absolute paths when dealing with file operations to avoid any ambiguity.
- The script is copying the
-
Script Purpose:
- It would be helpful to add comments or documentation explaining the purpose of the script, what it does, and how it should be used. This can make it easier for others (and yourself) to understand and modify the script in the future.
-
Code Duplication:
- The lines
cp ../tests/conf/pika.conf ./pika_master.conf
andcp ../tests/conf/pika.conf ./pika_slave.conf
are duplicated. Consider assigning the path to a variable once and then using that variable for both copy operations.
- The lines
-
Error Handling:
- Currently, there is no error handling in the script. It might be beneficial to check if the copy operations (
cp
) are successful before proceeding further. You can consider adding conditional checks or error handling mechanisms to handle such scenarios gracefully.
- Currently, there is no error handling in the script. It might be beneficial to check if the copy operations (
-
Consistency:
- Review the changes made by the sed commands to ensure they are aligned with your requirements and configuration.
- For consistency, consider using the same quoting style consistently throughout the script (either single quotes or double quotes).
Remember that a thorough code review requires a deeper understanding of the project and its specific requirements. These suggestions are based solely on the provided patch.
…#1682) * fix slave's binlog offset greater than master bug
…#1682) * fix slave's binlog offset greater than master bug
fix #1681