Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fast_transfer_file API 分发文件，如果源文件中的文件名包含空格，会报错 #812 #1649

Open
wants to merge 5 commits into
base: 3.5.x
Choose a base branch
from
Open

fast_transfer_file API 分发文件，如果源文件中的文件名包含空格，会报错 #812 #1649

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
83fd4ac
fast_transfer_file API 分发文件，如果源文件中的文件名包含空格，会报错 #812
Dec 30, 2022
f843a2a
bugfix: fast_transfer_file API 分发文件，如果源文件中的文件名包含空格，会报错 #812
Jan 5, 2023
658a77a
Merge remote-tracking branch 'origin/3.5.x_issue812' into 3.5.x_issue812
Jan 5, 2023
16fec58
bugfix: fast_transfer_file API 分发文件，如果源文件中的文件名包含空格，会报错 #812
Jan 6, 2023
3e64ca3
bugfix: fast_transfer_file API 分发文件，如果源文件中的文件名包含空格，会报错 #812
liuliaozhong Jan 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 35 additions & 30 deletions ...mmons/common-utils/src/main/java/com/tencent/bk/job/common/util/FilePathValidateUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,24 @@
* 文件路径合法性校验工具类
*/
public class FilePathValidateUtil {
// 传统DOS正则表达式
private static final String CONVENTIONAL_DOS_PATH_REGEX = "(^[A-Za-z]:\\\\[^\\\\].*)|(^[A-Za-z]:[\\\\])";
// DOS设备路径正则表达式
private static final String DOS_DEVICE_PATH_REGEX = "^\\\\\\\\.\\\\.+|\\\\\\\\\\?\\\\.+";
// UNC路径正则表达式
private static final String UNC_PATH_REGEX = "^\\\\\\\\[^\\\\]+\\\\[^\\\\]+\\\\.*";
// Linux路径正则表达式
private static final String LINUX_PATH_REGEX = "(^/([^/].*/{0,1})+)|(/)";

// 传统DOS Pattern
private static final Pattern CONVENTIONAL_DOS_PATH_PATTERN = Pattern.compile(CONVENTIONAL_DOS_PATH_REGEX);
// DOS设备Pattern
private static final Pattern DOS_DEVICE_PATH_PATTERN = Pattern.compile(DOS_DEVICE_PATH_REGEX);
// UNC路径Pattern
private static final Pattern UNC_PATH_PATTERN = Pattern.compile(UNC_PATH_REGEX);
// UNC路径Pattern
private static final Pattern LINUX_PATH_PATTERN = Pattern.compile(LINUX_PATH_REGEX);

/**
* 验证文件系统绝对路径的合法性
* @param path 绝对路径
Expand Down Expand Up @@ -41,40 +59,30 @@ private static boolean isLinuxAbsolutePath(String path) {
}

/**
* 1 DOS设备路径：
* 设备路径说明符（\\.\ 或 \\?\），它将路径标识为DOS设备路径
* 2 UNC路径
* 以\\开头的服务器名或主机名,路径必须始终是完全限定的
* 组成：\\服务器名\共享名\可选目录名\可选文件名
* 3 传统DOS路径
* 标准的DOS路径可由以下三部分组成：
* 1)卷号或驱动器号，后跟卷分隔符(:)。
* 2)目录名称。目录分隔符用来分隔嵌套目录层次结构中的子目录。
* 3)文件名。目录分隔符用来分隔文件路径和文件名。
*
* 1 传统DOS路径
* 标准的DOS路径可由以下三部分组成：
* 1)卷号或驱动器号，后跟卷分隔符(:)。
* 2)目录名称。目录分隔符用来分隔嵌套目录层次结构中的子目录。
* 3)文件名。目录分隔符用来分隔文件路径和文件名。
* 2 DOS设备路径：
* 设备路径说明符（\\.\ 或 \\?\），它将路径标识为DOS设备路径
* 3 UNC路径
* 以\\开头的服务器名或主机名,路径必须始终是完全限定的
* 组成：\\服务器名\共享名\可选目录名\可选文件名
* @param path
* @return boolean
*/
private static boolean validateWindowsFileSystemAbsolutePath(String path) {
// DOS设备
String pattern = "^\\\\\\\\.\\\\.+|\\\\\\\\\\?\\\\.+";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GSE 目前只支持传统dos路径，Job需要保持一致；可以去掉DOS设备和UNC路径的判断

Pattern r = Pattern.compile(pattern);
Matcher m = r.matcher(path);
if (m.matches()) {
// 传统DOS
if (CONVENTIONAL_DOS_PATH_PATTERN.matcher(path).matches()) {
return true;
}
// UNC
pattern = "^\\\\\\\\[^\\\\]+\\\\[^\\\\]+\\\\.*";
r = Pattern.compile(pattern);
m = r.matcher(path);
if (m.matches()) {
// DOS设备
if (DOS_DEVICE_PATH_PATTERN.matcher(path).matches()) {
return true;
}
// 传统DOS
pattern = "^[A-Za-z]:\\\\[^\\\\].*";
r = Pattern.compile(pattern);
m = r.matcher(path);
if (m.matches()) {
// UNC
if (UNC_PATH_PATTERN.matcher(path).matches()) {
return true;
}
return false;
Expand All @@ -87,10 +95,7 @@ private static boolean validateWindowsFileSystemAbsolutePath(String path) {
* @return boolean
*/
private static boolean validateLinuxFileSystemAbsolutePath(String path) {
String pattern = "^/([^/].*/{0,1})+";
Pattern r = Pattern.compile(pattern);
Matcher m = r.matcher(path);
if (m.matches()) {
if (LINUX_PATH_PATTERN.matcher(path).matches()) {
return true;
}
return false;
Expand Down
22 changes: 16 additions & 6 deletions ...s/common-utils/src/test/java/com/tencent/bk/job/common/util/FilePathValidateUtilTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,17 @@
public class FilePathValidateUtilTest {
@Test
void testFileSystemAbsolutePath(){
// 传统DOS路径
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:\\Documents\\abc.txt")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("c:\\Documents\\abc.txt")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:\\Documents\\嘉 abc.txt")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath(":\\abc.txt")).isFalse();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:")).isFalse();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:\\\\")).isFalse();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:\\")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:\\logs\\..\\access.log")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:\\.config\\conf")).isTrue();

// DOS设备路径
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("\\\\.\\C:\\Test\\Foo.txt")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("\\\\?\\C:\\Test\\Foo.txt")).isTrue();
Expand All @@ -19,16 +30,15 @@ void testFileSystemAbsolutePath(){
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("\\\\Server201\\C$\\Test1\\Fo o.txt")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("\\\\system07")).isFalse();

// 传统DOS路径
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:\\Documents\\abc.txt")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("c:\\Documents\\abc.txt")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("C:\\Documents\\嘉 abc.txt")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath(":\\abc.txt")).isFalse();

// linux路径
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("/data/test_2022-04-12.apk")).isTrue();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

需要补充更多的测试用例，如一些比较常见的:

  1. 根目录 /
  2. /tmp/
  3. /tmp/test/../test.log

assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("/data/test_2022 04 12.apk")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("/")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("/tmp/")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("/tmp/.conf/abc")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("/tmp/test/../test.log")).isTrue();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("data/test_2022-04-12.apk")).isFalse();
assertThat(FilePathValidateUtil.validateFileSystemAbsolutePath("///")).isFalse();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. /// 是合法linux路径
  2. 多个/的场景需要加入测试用例，比如/tmp////, /tmp//test/

}

}