-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Refactor pstd/env with std::filesystem #1470
Conversation
6df4992
to
8cf9c16
Compare
@4kangjc pls fix the file conflict and continue to finish this pr. |
src/pika_stable_log.cc
Outdated
@@ -209,8 +208,7 @@ Status StableLog::PurgeFileAfter(uint32_t filenum) { | |||
for (auto& it : binlogs) { | |||
if (it.first > filenum) { | |||
// Do delete | |||
Status s = pstd::DeleteFile(log_path_ + it.second); | |||
if (!s.ok()) { | |||
if (pstd::DeleteFile(log_path_ + it.second)) { | |||
return s; | |||
} | |||
LOG(WARNING) << "Delete file " << log_path_ + it.second; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code patch seems to be trying to delete some log files, and there are no obvious bugs in the code. However, there are some suggestions for improvement:
-
Error Handling: It might be useful to add an error handling mechanism to handle cases where
pstd::DeleteFile
return an error status. -
Logging: The logging statement inside the loop might cause a lot of unnecessary repetition if there are a large number of files being deleted. Consider moving the message outside the loop or adding a counter to reduce the frequency of logs.
-
Magic Number: The values 111 and 209 used in the code patch are magic numbers, and it's not clear from the code snippet what they represent. Consider using named constants instead.
-
Naming: The functions and variables used in the patch are not named consistently in terms of capitalization. Consider following a consistent naming convention.
-
Code Clarity: The code patch could be made more readable by adding comments explaining what each section does.
Overall, the code patch seems safe, but I recommend reviewing it in the context of the overall codebase to ensure that it integrates properly with other parts of the code.
c++fs | ||
) | ||
endif() | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code patch seems to be adding support for linking the C++ filesystem library for GNU and Clang compilers based on their IDs. Here are some suggestions:
-
It is a good practice to add comments in code that explain why certain changes are made. This can help other developers understand the reasoning behind the changes easily.
-
The code patch only checks if the compiler is GNU or Clang, but it does not verify if the compiler supports the C++ filesystem library. You may want to consider adding an additional check for this using the
check_cxx_compiler_flag
function and setting a flag variable accordingly. -
When linking libraries, it is recommended to use target-based linkage instead of specifying libraries directly. This way, dependencies between targets can be automatically resolved by CMake. You can create a target with the
add_library
command and specify its dependencies usingtarget_link_libraries
. Then, link the target to your main target usingtarget_link_libraries
. -
It is better to use a
switch
statement instead of multipleif-elseif
blocks when checking for different compiler IDs. This can make the code more concise and easier to read. -
Finally, you may want to consider using consistent casing for variable names and formatting the code consistently throughout the project.
int GetChildren(const std::string& dir, std::vector<std::string>& result); | ||
bool GetDescendant(const std::string& dir, std::vector<std::string>& result); | ||
void GetDescendant(const std::string& dir, std::vector<std::string>& result); | ||
|
||
uint64_t NowMicros(); | ||
void SleepForMicroseconds(int micros); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the provided diff, here are a few observations:
- The return type of
DeleteFile
is changed fromStatus
tobool
. This may introduce some semantic changes and can affect error handling depending on how the API is used. LockFile
andUnlockFile
functions are removed from the header file, so they are no longer part of the public API. This may be intentional or accidental, but it's worth checking if any downstream code depends on these functions.- The return type of
GetDescendant
is changed frombool
tovoid
, implying that the function no longer returns success/failure status. It's unclear if this was intentional. - Overall, it's hard to say anything more definitive without knowing what the implementation of these functions looks like, their purpose, and how they're used in the code.
} | ||
LOG(WARNING) << "Delete file " << log_path_ + it.second; | ||
LOG(WARNING) << "Delete file " << filename; | ||
} | ||
} | ||
return Status::OK(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code patch looks fine.
There is an improvement suggestion that can be made, where instead of concatenating strings in the DeleteFile
function call, it will be better to pass it as a separate parameter.
So instead of:
pstd::DeleteFile(log_path_ + it->second)
It should be:
pstd::DeleteFile(log_path_, it->second)
This can help make the code more understandable and avoid any string concatenation issues.
endif() | ||
endif() | ||
|
||
target_link_libraries(pstd PUBLIC ${PSTD_DEP_LIBS}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code patch modifies the CMake project file for a target named "pstd". The patch adds a condition to link the standard C++ file system library "stdc++fs
" on GNU compilers versions less than 9, and "c++fs
" on Clang compilers versions less than 9. This is done by appending the required library to a list of dependent libraries called PSTD_DEP_LIBS
. The patch also replaces the direct linking of dependent libraries with the target_link_libraries
command, making it more concise.
A possible improvement to this patch is to use the CMAKE_CXX_STANDARD_LIBRARY_LIB
variable instead of manually appending the required library for each compiler and version. The CMAKE_CXX_STANDARD_LIBRARY_LIB
variable contains the name of the C++ standard library implementation (e.g., "stdc++", "c++") and can be used to link the corresponding file system library.
Another potential improvement is to add a comment explaining the reason for linking the file system library only on specific compiler versions. Without proper documentation, future maintainers might not understand why the library is linked conditionally, leading to confusion and potential bugs.
int GetChildren(const std::string& dir, std::vector<std::string>& result); | ||
bool GetDescendant(const std::string& dir, std::vector<std::string>& result); | ||
void GetDescendant(const std::string& dir, std::vector<std::string>& result); | ||
|
||
uint64_t NowMicros(); | ||
void SleepForMicroseconds(int micros); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code patch seems to change the return type of the DeleteFile
function from Status
to bool
, and remove the declarations for LockFile
and UnlockFile
. It also changes the return type of GetDescendant
from bool
to void
.
Without seeing the implementation of these functions or the context in which they are used, it is hard to determine whether these changes introduce any bugs or potential improvements. However, changing the return types could break compatibility with existing code that uses these functions, so it is important to make sure any other code using these functions is updated accordingly.
} | ||
LOG(WARNING) << "Delete file " << log_path_ + it.second; | ||
LOG(WARNING) << "Delete file " << filename; | ||
} | ||
} | ||
return Status::OK(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code patch seems to be a review of a piece of code that deals with purging files. Here are a few points about the code:
- There are no obvious bug risks in the code patch.
- The improvement that could be suggested in this code patch is to log the actual reason for failure when deleting a file fails instead of using a generic error message. This can give more information on what went wrong during the delete operation and help debug the issue.
- In the function
PurgeFileAfter
, it would be better to concatenate the filename outside of thepstd::DeleteFile
call and store it in a variable to avoid code duplication, as done in thePurgeFiles
function. - It may be good to consider adding more checks before deleting files, such as verifying file ownership (if applicable) and checking whether the file being deleted is actually the one intended to be deleted.
endif() | ||
endif() | ||
|
||
target_link_libraries(pstd PUBLIC ${PSTD_DEP_LIBS}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code patch seems to be modifying the CMake build script for the pstd
target, specifically the linking of dependent libraries.
The changes appear to add conditional linker flags/arguments to link against the stdc++fs
or c++fs
library depending on the version of the GCC or Clang compiler.
There are no obvious bug risks in this code patch, but it is always good to test the changes and ensure they work as intended. As for improvement suggestions, it is not clear from this code patch if there are any other areas that could be improved without more context and knowledge about the project.
int GetChildren(const std::string& dir, std::vector<std::string>& result); | ||
bool GetDescendant(const std::string& dir, std::vector<std::string>& result); | ||
void GetDescendant(const std::string& dir, std::vector<std::string>& result); | ||
|
||
uint64_t NowMicros(); | ||
void SleepForMicroseconds(int micros); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code patch looks like a simple header file with function signatures and declarations, so it's hard to provide any concrete feedback without seeing the implementation details. However, some general suggestions include:
- Use consistent naming conventions for functions, such as using either camelCase or snake_case, and stick to them throughout the codebase.
- Use descriptive function names that make clear what the function does, especially for public-facing functions.
- Consider using
std::filesystem
instead of rolling your own file manipulation functions, as it provides a more robust and standardized API for dealing with files and directories. - The return type of the
DeleteFile()
function has been changed fromStatus
tobool
. Make sure this is intentional and doesn't introduce bugs or unexpected behavior elsewhere in the code. - Consider adding documentation comments (
///
) to the function signatures, describing their purpose and expected behavior.
} | ||
LOG(WARNING) << "Delete file " << log_path_ + it.second; | ||
LOG(WARNING) << "Delete file " << filename; | ||
} | ||
} | ||
return Status::OK(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, the code patch looks fine. However, there are a few improvements that could be made:
-
In both functions, it would be better to use a range-based for loop instead of an iterator.
-
In the
PurgeFiles
function, theif (s.ok())
check is unnecessary. The return value ofpstd::DeleteFile
is already a boolean indicating success or failure. -
In both functions, it would be better to store the full path of the file to be deleted in a separate variable before calling
pstd::DeleteFile
. This makes it easier to log in case of errors. -
In the error message of the
PurgeFiles
function, it would be better to provide more information about why the delete failed instead of just saying "delete file failed". -
In the
PurgeFileAfter
function, the misspelling of "failed" in the error message should be corrected. -
In both functions, it might be a good idea to add a check that the file exists before attempting to delete it, to avoid deleting non-existent files.
-
Finally, it would be nice if there were comments explaining what the functions do and how they work.
endif() | ||
endif() | ||
|
||
target_link_libraries(pstd PUBLIC ${PSTD_DEP_LIBS}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code patch seems to update the target_link_libraries
command for the pstd
target in a CMake project.
The changes look reasonable, as it defines an intermediate PSTD_DEP_LIBS
variable to hold the dependent libraries and then links them with the pstd
target using target_link_libraries
.
The additional code checks for the compiler type and version. If the compiler is GCC or Clang and its version is less than 9, it appends the stdc++fs
or c++fs
library (depending on the compiler) to the PSTD_DEP_LIBS
variable. This is because these libraries were introduced in GCC and Clang 9 respectively.
One potential improvement could be to use target_compile_features
instead of checking the compiler versions manually. For example, you could use target_compile_features(pstd PUBLIC cxx_std_17)
to enforce that the code is compiled with C++17 or later features, which should automatically include the filesystem library for GCC 9+ and Clang 9+.
int GetChildren(const std::string& dir, std::vector<std::string>& result); | ||
bool GetDescendant(const std::string& dir, std::vector<std::string>& result); | ||
void GetDescendant(const std::string& dir, std::vector<std::string>& result); | ||
|
||
uint64_t NowMicros(); | ||
void SleepForMicroseconds(int micros); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code patch seems to be a header file defining various file-related functions and classes. Here are some observations:
-
It would be better to return
Status
instead ofbool
in theDeleteFile
function, as it allows conveying specific error messages to the caller. -
Similarly,
LockFile
andUnlockFile
should also returnStatus
instead ofbool
. -
The visibility of the
GetDescendant
function is unclear - it is not clear whether it is supposed to be used outside the header file or just within its implementation. If it is meant to be used outside, then it should have a status return value to indicate any errors. -
It would be good to add documentation (comments or doxygen-style) to provide context and usage examples for all these functions.
-
It's hard to give specific improvement suggestions without knowing the intended use of these functions. For example, it might make sense to add overloads for
DeleteFile
andLockFile
that takestd::filesystem::path
objects instead of a string.
* feat: supported 'ptype' command * Simplified command `type` and `ptype` code * Remove the capacity judgment * fix slavaof serialize response bug (#1583) Co-authored-by: liuyuecai <[email protected]> * fix (#1587) (#1588) * fix unit test:type/set (#1577) * Fix memory leaks in HandleBGWorkerWriteDB and HandleMetaSyncResponse (#1590) * fix memory leaks in PikaReplBgWorker::HandleBGWorkerWriteDB(void* arg) and PikaReplClientConn::HandleMetaSyncResponse(void* arg). * add address/thread sanitizer to CMakeLists --------- Co-authored-by: cjh <[email protected]> * refactor: replace pstd/env with std::filesystem (#1470) * fix bug issue 1554: using unique_ptr.reset to fix the SIGABRT error (#1595) using unique_ptr.reset to fix the SIGABRT error && using time-wait lock to notify quickly * fix: incorrect manner of terminating the process (#1596) * fix issue#1597: add rocksdb dependency to pstd (#1599) * fix g++15 compile failure * add rocksdb dependency to pstd --------- Co-authored-by: J1senn <[email protected]> * fix_info_command * fix: gcc13 compile failed (#1601) * ci: add unit test to github action (#1604) * ci: add unit test in github workflow * chore:change `PLATFORM` field logic (#1615) * fix issue 1517: scan 命令不支持 type 参数 (#1582) * fix: the unit test of type/set (#1617) * test: optimize the return results of srandmember to avoid approximate results * fix: use last_seed for random engine * [fix issue1621] fix deadlock (#1620) * [fix] fix deadlock * [fix] fix * command `type` and `ptype` add unit test * fix member variable initialization * Update issue templates * feat: supported 'ptype' command * fix unit test:type/set (#1577) * fix issue#1597: add rocksdb dependency to pstd (#1599) * fix g++15 compile failure * add rocksdb dependency to pstd --------- Co-authored-by: J1senn <[email protected]> * fix: gcc13 compile failed (#1601) * fix: the unit test of type/set (#1617) * test: optimize the return results of srandmember to avoid approximate results * fix: use last_seed for random engine * Modify other modules to use the new `type` function * fix function repeat --------- Co-authored-by: Yuecai Liu <[email protected]> Co-authored-by: liuyuecai <[email protected]> Co-authored-by: Peter Chan <[email protected]> Co-authored-by: chenbt <[email protected]> Co-authored-by: Junhua Chen <[email protected]> Co-authored-by: cjh <[email protected]> Co-authored-by: kang jinci <[email protected]> Co-authored-by: Xin.Zh <[email protected]> Co-authored-by: machinly <[email protected]> Co-authored-by: A2ureStone <[email protected]> Co-authored-by: J1senn <[email protected]> Co-authored-by: chejinge <[email protected]> Co-authored-by: baerwang <[email protected]> Co-authored-by: ptbxzrt <[email protected]> Co-authored-by: 你不要过来啊 <[email protected]>
} | ||
result.emplace_back(entry->d_name); | ||
for (auto& de : filesystem::directory_iterator(dir)) { | ||
result.emplace_back(de.path()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
result.emplace_back(de.path()); | |
result.emplace_back(de.path().filename()); |
* feat: supported 'ptype' command * Simplified command `type` and `ptype` code * Remove the capacity judgment * fix slavaof serialize response bug (OpenAtomFoundation#1583) Co-authored-by: liuyuecai <[email protected]> * fix (OpenAtomFoundation#1587) (OpenAtomFoundation#1588) * fix unit test:type/set (OpenAtomFoundation#1577) * Fix memory leaks in HandleBGWorkerWriteDB and HandleMetaSyncResponse (OpenAtomFoundation#1590) * fix memory leaks in PikaReplBgWorker::HandleBGWorkerWriteDB(void* arg) and PikaReplClientConn::HandleMetaSyncResponse(void* arg). * add address/thread sanitizer to CMakeLists --------- Co-authored-by: cjh <[email protected]> * refactor: replace pstd/env with std::filesystem (OpenAtomFoundation#1470) * fix bug issue 1554: using unique_ptr.reset to fix the SIGABRT error (OpenAtomFoundation#1595) using unique_ptr.reset to fix the SIGABRT error && using time-wait lock to notify quickly * fix: incorrect manner of terminating the process (OpenAtomFoundation#1596) * fix issue#1597: add rocksdb dependency to pstd (OpenAtomFoundation#1599) * fix g++15 compile failure * add rocksdb dependency to pstd --------- Co-authored-by: J1senn <[email protected]> * fix_info_command * fix: gcc13 compile failed (OpenAtomFoundation#1601) * ci: add unit test to github action (OpenAtomFoundation#1604) * ci: add unit test in github workflow * chore:change `PLATFORM` field logic (OpenAtomFoundation#1615) * fix issue 1517: scan 命令不支持 type 参数 (OpenAtomFoundation#1582) * fix: the unit test of type/set (OpenAtomFoundation#1617) * test: optimize the return results of srandmember to avoid approximate results * fix: use last_seed for random engine * [fix issue1621] fix deadlock (OpenAtomFoundation#1620) * [fix] fix deadlock * [fix] fix * command `type` and `ptype` add unit test * fix member variable initialization * Update issue templates * feat: supported 'ptype' command * fix unit test:type/set (OpenAtomFoundation#1577) * fix issue#1597: add rocksdb dependency to pstd (OpenAtomFoundation#1599) * fix g++15 compile failure * add rocksdb dependency to pstd --------- Co-authored-by: J1senn <[email protected]> * fix: gcc13 compile failed (OpenAtomFoundation#1601) * fix: the unit test of type/set (OpenAtomFoundation#1617) * test: optimize the return results of srandmember to avoid approximate results * fix: use last_seed for random engine * Modify other modules to use the new `type` function * fix function repeat --------- Co-authored-by: Yuecai Liu <[email protected]> Co-authored-by: liuyuecai <[email protected]> Co-authored-by: Peter Chan <[email protected]> Co-authored-by: chenbt <[email protected]> Co-authored-by: Junhua Chen <[email protected]> Co-authored-by: cjh <[email protected]> Co-authored-by: kang jinci <[email protected]> Co-authored-by: Xin.Zh <[email protected]> Co-authored-by: machinly <[email protected]> Co-authored-by: A2ureStone <[email protected]> Co-authored-by: J1senn <[email protected]> Co-authored-by: chejinge <[email protected]> Co-authored-by: baerwang <[email protected]> Co-authored-by: ptbxzrt <[email protected]> Co-authored-by: 你不要过来啊 <[email protected]>
* feat: supported 'ptype' command * Simplified command `type` and `ptype` code * Remove the capacity judgment * fix slavaof serialize response bug (OpenAtomFoundation#1583) Co-authored-by: liuyuecai <[email protected]> * fix (OpenAtomFoundation#1587) (OpenAtomFoundation#1588) * fix unit test:type/set (OpenAtomFoundation#1577) * Fix memory leaks in HandleBGWorkerWriteDB and HandleMetaSyncResponse (OpenAtomFoundation#1590) * fix memory leaks in PikaReplBgWorker::HandleBGWorkerWriteDB(void* arg) and PikaReplClientConn::HandleMetaSyncResponse(void* arg). * add address/thread sanitizer to CMakeLists --------- Co-authored-by: cjh <[email protected]> * refactor: replace pstd/env with std::filesystem (OpenAtomFoundation#1470) * fix bug issue 1554: using unique_ptr.reset to fix the SIGABRT error (OpenAtomFoundation#1595) using unique_ptr.reset to fix the SIGABRT error && using time-wait lock to notify quickly * fix: incorrect manner of terminating the process (OpenAtomFoundation#1596) * fix issue#1597: add rocksdb dependency to pstd (OpenAtomFoundation#1599) * fix g++15 compile failure * add rocksdb dependency to pstd --------- Co-authored-by: J1senn <[email protected]> * fix_info_command * fix: gcc13 compile failed (OpenAtomFoundation#1601) * ci: add unit test to github action (OpenAtomFoundation#1604) * ci: add unit test in github workflow * chore:change `PLATFORM` field logic (OpenAtomFoundation#1615) * fix issue 1517: scan 命令不支持 type 参数 (OpenAtomFoundation#1582) * fix: the unit test of type/set (OpenAtomFoundation#1617) * test: optimize the return results of srandmember to avoid approximate results * fix: use last_seed for random engine * [fix issue1621] fix deadlock (OpenAtomFoundation#1620) * [fix] fix deadlock * [fix] fix * command `type` and `ptype` add unit test * fix member variable initialization * Update issue templates * feat: supported 'ptype' command * fix unit test:type/set (OpenAtomFoundation#1577) * fix issue#1597: add rocksdb dependency to pstd (OpenAtomFoundation#1599) * fix g++15 compile failure * add rocksdb dependency to pstd --------- Co-authored-by: J1senn <[email protected]> * fix: gcc13 compile failed (OpenAtomFoundation#1601) * fix: the unit test of type/set (OpenAtomFoundation#1617) * test: optimize the return results of srandmember to avoid approximate results * fix: use last_seed for random engine * Modify other modules to use the new `type` function * fix function repeat --------- Co-authored-by: Yuecai Liu <[email protected]> Co-authored-by: liuyuecai <[email protected]> Co-authored-by: Peter Chan <[email protected]> Co-authored-by: chenbt <[email protected]> Co-authored-by: Junhua Chen <[email protected]> Co-authored-by: cjh <[email protected]> Co-authored-by: kang jinci <[email protected]> Co-authored-by: Xin.Zh <[email protected]> Co-authored-by: machinly <[email protected]> Co-authored-by: A2ureStone <[email protected]> Co-authored-by: J1senn <[email protected]> Co-authored-by: chejinge <[email protected]> Co-authored-by: baerwang <[email protected]> Co-authored-by: ptbxzrt <[email protected]> Co-authored-by: 你不要过来啊 <[email protected]>
No description provided.