Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(115): add offline download tool (close #6888) #6954

Merged
merged 1 commit into from
Aug 10, 2024

Conversation

SheltonZhu
Copy link
Member

  • 增加115的离线下载工具,仿照pikpak的写法实现
  • 由于115自己维护一个离线下载列表,可能会发生不同步的问题
  • 目前当下载成功后,删除离线列表中完成的任务
  • 已知问题: 115已经在离线列表中的任务url不能再次添加

@xhofe xhofe merged commit 29165d8 into alist-org:main Aug 10, 2024
3 checks passed
@anwen-anyi
Copy link
Member

anwen-anyi commented Aug 11, 2024

添加了这个115离线下载的PR后115似乎无法使用了

  • Request failed with status code 500
    image
错误日志 DEBU[2024-08-11 15:56:23]/source/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /115_自己的 [GIN] 2024/08/11 - 15:56:23 | 200 | 19.2255ms | 192.168.31.15 | POST "/api/fs/get" DEBU[2024-08-11 15:56:23]/source/server/middlewares/auth.go:72 github.com/alist-org/alist/v3/server/middlewares.Auth() use login token: &{ID:1 Username:admin PwdHash:49c18cd84f01185615e7ec035fedfc5c7292ce37c8d84761064b034d9fe47e1f PwdTS:1703574958 Salt:tcifiCqAM55BpsHq Password: BasePath:/ Role:2 Disabled:false Permission:0 OtpSecret: SsoID: Authn:[]} DEBU[2024-08-11 15:56:23]/source/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /115_自己的 DEBU[2024-08-11 15:56:23]/source/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List / DEBU[2024-08-11 15:56:23]/source/internal/op/fs.go:167 github.com/alist-org/alist/v3/internal/op.Get() op.Get / DEBU[2024-08-11 15:56:23]/source/internal/op/fs.go:120 github.com/alist-org/alist/v3/internal/op.List() list dir: &{ID:0 Path: Name:root Size:0 Modified:2024-08-11 15:33:36.8111866 +0800 +0800 Ctime:0001-01-01 00:00:00 +0000 UTC IsFolder:true HashInfo:null}

2024/08/11 15:56:23 [Recovery] 2024/08/11 - 15:56:23 panic recovered:
POST /api/fs/list HTTP/1.1
Host: 192.168.31.15:5244
Accept: application/json, text/plain, /
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Authorization: *
Connection: keep-alive
Content-Length: 77
Content-Type: application/json;charset=UTF-8
Cookie: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Origin: http://192.168.31.15:5244
Referer: http://192.168.31.15:5244/115_%E8%87%AA%E5%B7%B1%E7%9A%84
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36

time: missing Location in call to Date

runtime/debug.Stack()
/go/pkg/mod/golang.org/[email protected]/src/runtime/debug/stack.go:24 +0x5e
github.com/alist-org/alist/v3/pkg/singleflight.newPanicError({0x2da1160, 0x4a2a200})
/source/pkg/singleflight/singleflight.go:35 +0x25
github.com/alist-org/alist/v3/pkg/singleflight.(*Group[...]).doCall.func2.1()
/source/pkg/singleflight/singleflight.go:188 +0x34
panic({0x2da1160?, 0x4a2a200?})
/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:770 +0x132
time.Date(0xc000f93c5e?, 0xc000f93c50?, 0x334e113?, 0xc000846540?, 0xc000e6c788?, 0x0?, 0x0?, 0x4a2f220?)
/go/pkg/mod/golang.org/[email protected]/src/time/time.go:1490 +0x4c5
time.parse({0x334e106, 0x10}, {0xc000f93c50, 0x10}, 0x0, 0x0)
/go/pkg/mod/golang.org/[email protected]/src/time/format.go:1398 +0x2079
time.ParseInLocation({0x334e106, 0x10}, {0xc000f93c50, 0x10}, 0x0)
/go/pkg/mod/golang.org/[email protected]/src/time/format.go:1029 +0xe6
github.com/SheltonZhu/115driver/pkg/driver.(*File).from(0xc000ecab58, 0xc000eca9d0)
/go/pkg/mod/github.com/!shelton!zhu/[email protected]/pkg/driver/file.go:46 +0xb5
github.com/SheltonZhu/115driver/pkg/driver.(*Pan115Client).ListWithLimit(0xc000889200, {0x295d540, 0x1}, 0x38)
/go/pkg/mod/github.com/!shelton!zhu/[email protected]/pkg/driver/dir.go:44 +0x428
github.com/alist-org/alist/v3/drivers/115.(*Pan115).getFiles(0xc00088f810?, {0x295d540?, 0xc000248800?})
/source/drivers/115/util.go:66 +0x50
github.com/alist-org/alist/v3/drivers/115.(*Pan115).List(0xc000c76160, {0x4a47fb0?, 0xc000248800?}, {0x4a51c18?, 0xc000245300?}, {{0xc0008e4e68?, 0xc0008e4ee0?}, 0xe2?, 0xec?})
/source/drivers/115/driver.go:53 +0x94
github.com/alist-org/alist/v3/internal/op.List.func1()
/source/internal/op/fs.go:125 +0x96
github.com/alist-org/alist/v3/pkg/singleflight.(*Group[...]).doCall.func2(0xc000f09860, 0xd0, 0x30de720?)
/source/pkg/singleflight/singleflight.go:193 +0x66
github.com/alist-org/alist/v3/pkg/singleflight.(*Group[...]).doCall(0xc000c8cdb0?, 0xc000b2a610?, {0xc000b2a610?, 0xc0008e51d8?}, 0x0?)
/source/pkg/singleflight/singleflight.go:195 +0xa5
github.com/alist-org/alist/v3/pkg/singleflight.(*Group[...]).Do(0x4a6d600, {0xc000b2a610, 0xe}, 0xc0008e5208)
/source/pkg/singleflight/singleflight.go:108 +0x16f
github.com/alist-org/alist/v3/internal/op.List({0x4a47fb0, 0xc000248800}, {0x4a52630, 0xc000c76160}, {0x4a21630, 0x1}, {{0xc000b2a5b0, 0xe}, 0x0, 0x0})
/source/internal/op/fs.go:124 +0x455
github.com/alist-org/alist/v3/internal/fs.list({0x4a47fb0, 0xc000248800}, {0xc000b2a5b0, 0xe}, 0xc000ecb495)
/source/internal/fs/list.go:25 +0x1c9
github.com/alist-org/alist/v3/internal/fs.List({0x4a47fb0?, 0xc000248800?}, {0xc000b2a5b0, 0xe}, 0xc000ecb495)
/source/internal/fs/fs.go:22 +0x3c
github.com/alist-org/alist/v3/server/handles.FsList(0xc000248800)
/source/server/handles/fsread.go:86 +0x2ee
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185
github.com/alist-org/alist/v3/server/middlewares.Auth(0xc000248800)
/source/server/middlewares/auth.go:73 +0x4d6
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185
github.com/alist-org/alist/v3/server/middlewares.StoragesLoaded(0xc000248800)
/source/server/middlewares/check.go:14 +0xbb
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc000248800)
/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 +0x7a
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185
github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc000248800)
/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:249 +0xe5
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc000bc2000, 0xc000248800)
/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:633 +0x892
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc000bc2000, {0x4a41cc0, 0xc0001f75e0}, 0xc0005a07e0)
/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:589 +0x1b2
net/http.serverHandler.ServeHTTP({0x4a390d0?}, {0x4a41cc0?, 0xc0001f75e0?}, 0x6?)
/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3137 +0x8e
net/http.(*conn).serve(0xc000b28000, {0x4a47b50, 0xc000b26c30})
/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2039 +0x5e8
created by net/http.(*Server).Serve in goroutine 126
/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3285 +0x4b4

/source/pkg/singleflight/singleflight.go:165 (0x147f931)
/source/pkg/singleflight/singleflight.go:200 (0x1486ac1)
/source/pkg/singleflight/singleflight.go:108 (0x148700e)
/source/internal/op/fs.go:124 (0x14748b4)
/source/internal/fs/list.go:25 (0x19200c8)
/source/internal/fs/fs.go:22 (0x191ea5b)
/source/server/handles/fsread.go:86 (0x274ab4d)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x27126f5)
/source/server/middlewares/auth.go:73 (0x27124dd)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x2712f1a)
/source/server/middlewares/check.go:14 (0x2712f08)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x190cb19)
/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 (0x190cb07)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x190bc44)
/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:249 (0x190bc2b)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x190b071)
/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:633 (0x190aae0)
/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:589 (0x190a611)
/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3137 (0x1290b4d)
/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2039 (0x128be27)
/go/pkg/mod/golang.org/[email protected]/src/runtime/asm_amd64.s:1695 (0xf9b660)

[GIN] 2024/08/11 - 15:56:23 | 500 | 347.7519ms | 192.168.31.15 | POST "/api/fs/list"


然后根据main分支合并记录在Actions分别下载了最新的(由于此并未自动编译就找了个最新的PR包含了此次提交的代码)和比此PR合并前的
image

包含了此PR的更新的代码 ❌

https://github.com/alist-org/alist/actions/runs/10336768743
fix(deps): update module github.com/meilisearch/meilisearch-go to v0.…
build #3201: Commit 8f3c5b1 pushed by xhofe

本PR未编译无法下载,下载了包含此代码更新的 ❕

https://github.com/alist-org/alist/actions/runs/10331787948
feat(115): add offline download tool (close #6888 in #6954)
build #3186: Commit 29165d8 pushed by xhofe

此PR合并前的 ✅

https://github.com/alist-org/alist/actions/runs/10278026036
feat: persistant Task (#6925 close #5313)
build #3180: Commit 74f8295 pushed by xhofe

此PR合并前的可以使用,此PR合并后的无法使用

@SheltonZhu
Copy link
Member Author

SheltonZhu commented Aug 11, 2024

添加了这个115离线下载的PR后115似乎无法使用了

  • Request failed with status code 500
    image

错误日志
然后根据main分支合并记录在Actions分别下载了最新的(由于此并未自动编译就找了个最新的PR包含了此次提交的代码)和比此PR合并前的 image

包含了此PR的更新的代码 ❌

https://github.com/alist-org/alist/actions/runs/10336768743 fix(deps): update module github.com/meilisearch/meilisearch-go to v0.… build #3201: Commit 8f3c5b1 pushed by xhofe

本PR未编译无法下载,下载了包含此代码更新的 ❕

https://github.com/alist-org/alist/actions/runs/10331787948
feat(115): add offline download tool (close #6888 in #6954)
build #3186: Commit 29165d8 pushed by xhofe

此PR合并前的 ✅

https://github.com/alist-org/alist/actions/runs/10278026036
feat: persistant Task (#6925 close #5313)
build #3180: Commit 74f8295 pushed by xhofe

此PR合并前的可以使用,此PR合并后的无法使用

看日志是时间解析部分报错了,是115依赖包那部分的问题,是修复时区部分的代码 #6920 导致的和离线下载功能本身应该没关系

@anwen-anyi
Copy link
Member

看日志是时间解析部分报错了,是115依赖包那部分的问题,是修复时区部分的代码 #6704 导致的和离线下载功能本身应该没关系

好的,看修复时区也是大佬PR的 😹,刚准备试试115离线下载。

  • 115可以在AList提交链接的和在115官网提交的格式都一样吧

@SheltonZhu
Copy link
Member Author

SheltonZhu commented Aug 11, 2024

看日志是时间解析部分报错了,是 115 依赖包那部分的问题,是修复时区部分的代码 #6704 导致的和离线下载功能本身应该没关系

好的,看修复时区也是大佬 PR 的 😹,刚准备试试 115 离线下载。

  • 115 可以在 AList 提交链接的和在 115 官网提交的格式都一样吧

应该一样的,我只测试了三种:

@SheltonZhu
Copy link
Member Author

添加了这个115离线下载的PR后115似乎无法使用了

  • Request failed with status code 500
    image

错误日志
然后根据main分支合并记录在Actions分别下载了最新的(由于此并未自动编译就找了个最新的PR包含了此次提交的代码)和比此PR合并前的 image

包含了此PR的更新的代码 ❌

https://github.com/alist-org/alist/actions/runs/10336768743 fix(deps): update module github.com/meilisearch/meilisearch-go to v0.… build #3201: Commit 8f3c5b1 pushed by xhofe

本PR未编译无法下载,下载了包含此代码更新的 ❕

https://github.com/alist-org/alist/actions/runs/10331787948
feat(115): add offline download tool (close #6888 in #6954)
build #3186: Commit 29165d8 pushed by xhofe

此PR合并前的 ✅

https://github.com/alist-org/alist/actions/runs/10278026036
feat: persistant Task (#6925 close #5313)
build #3180: Commit 74f8295 pushed by xhofe

此PR合并前的可以使用,此PR合并后的无法使用

你的部署环境是什么,能发一下吗
目前看下来好像是找不到时区文件导致的 @anwen-anyi

@anwen-anyi
Copy link
Member

你的部署环境是什么,能发一下吗 目前看下来好像是找不到时区文件导致的 @anwen-anyi

  • 本地笔记本 Windows 10 还需要什么吗?

@SheltonZhu
Copy link
Member Author

SheltonZhu commented Aug 11, 2024

你的部署环境是什么,能发一下吗 目前看下来好像是找不到时区文件导致的 @anwen-anyi

  • 本地笔记本 Windows 10 还需要什么吗?

我本地是mac,所以没有测出你这个问题,win10推测应该就是这个问题,我修复吧,当找不到Asia/Shanghai的时候,直接用UTC+8了,虽然这个在夏令时的时候可能会有问题

已修复,等下次115依赖自动更新到v1.0.17的时候应该就可以了 @anwen-anyi

@anwen-anyi
Copy link
Member

我本地是mac,所以没有测出你这个问题,win10推测应该就是这个问题,我修复吧,当找不到Asia/Shanghai的时候,直接用UTC+8了,虽然这个在夏令时的时候可能会有问题

已修复,等下次115依赖自动更新到v1.0.17的时候应该就可以了 @anwen-anyi

已测试,可用木的问题

@(PXC4IP~FTWNJ@4D285L9W

@anwen-anyi
Copy link
Member

顺便 问一下大佬 115 这个 QrCode 是怎么使用的? ❤️

我现在使用的是填写 cookie选项,工具使用的是https://gist.github.com/ChenyangGao/d26a592a0aeb13465511c885d5c7ad61 获取的

@SheltonZhu
Copy link
Member Author

SheltonZhu commented Aug 12, 2024

顺便 问一下大佬 115 这个 QrCode 是怎么使用的? ❤️

我现在使用的是填写 cookie选项,工具使用的是https://gist.github.com/ChenyangGao/d26a592a0aeb13465511c885d5c7ad61 获取的

这个是仿照阿里云写的,参考文档 https://alist.nn.ci/zh/guide/drivers/115.html

  • 先点获取二维码, 等二维码生成
  • 手机扫码,授权后点击按钮
  • 将token填入 alist 115 的配置中(注意要保证cookie那一栏是空的, 原理是用token换取cookie,所以你配置完成后,token就清空了,cookie那栏会自动填入)

@anwen-anyi
Copy link
Member

顺便 问一下大佬 115 这个 QrCode 是怎么使用的? ❤️

我现在使用的是填写 cookie选项,工具使用的是https://gist.github.com/ChenyangGao/d26a592a0aeb13465511c885d5c7ad61 获取的

这个是仿照阿里云写的,参考文档 https://alist.nn.ci/zh/guide/drivers/115.html

  • 先点获取二维码, 等二维码生成
  • 手机扫码,授权后点击按钮
  • 将token填入 alist 115 的配置中(注意要保证cookie那一栏是空的, 原理是用token换取cookie,所以你配置完成后,token就清空了,cookie那栏会自动填入)

QRCode那一栏有好多不同设备的,手机APP扫码后可以随意指定吗?

@SheltonZhu
Copy link
Member Author

顺便 问一下大佬 115 这个 QrCode 是怎么使用的? ❤️

我现在使用的是填写 cookie选项,工具使用的是https://gist.github.com/ChenyangGao/d26a592a0aeb13465511c885d5c7ad61 获取的

这个是仿照阿里云写的,参考文档 https://alist.nn.ci/zh/guide/drivers/115.html

  • 先点获取二维码, 等二维码生成
  • 手机扫码,授权后点击按钮
  • 将token填入 alist 115 的配置中(注意要保证cookie那一栏是空的, 原理是用token换取cookie,所以你配置完成后,token就清空了,cookie那栏会自动填入)

QRCode那一栏有好多不同设备的,手机APP扫码后可以随意指定吗?

可以的,他这token获取cookie的原理是根据参数来区分是哪个设备登陆的,你选择哪个设备,获取cookie就传哪个参数,其实就是从你那个py脚本里抄的

@anwen-anyi
Copy link
Member

顺便 问一下大佬 115 这个 QrCode 是怎么使用的? ❤️

我现在使用的是填写 cookie选项,工具使用的是https://gist.github.com/ChenyangGao/d26a592a0aeb13465511c885d5c7ad61 获取的

这个是仿照阿里云写的,参考文档 https://alist.nn.ci/zh/guide/drivers/115.html

  • 先点获取二维码, 等二维码生成
  • 手机扫码,授权后点击按钮
  • 将token填入 alist 115 的配置中(注意要保证cookie那一栏是空的, 原理是用token换取cookie,所以你配置完成后,token就清空了,cookie那栏会自动填入)

QRCode那一栏有好多不同设备的,手机APP扫码后可以随意指定吗?

可以的,他这token获取cookie的原理是根据参数来区分是哪个设备登陆的,你选择哪个设备,获取cookie就传哪个参数,其实就是从你那个py脚本里抄的

明白了~ 不过现在 Windows已经下架了(客户端),Mac客户端不清楚,新增了 阿里和微信小程序的 嘿嘿~ ❤️

@SheltonZhu
Copy link
Member Author

SheltonZhu commented Aug 12, 2024

顺便 问一下大佬 115 这个 QrCode 是怎么使用的? ❤️

我现在使用的是填写 cookie选项,工具使用的是https://gist.github.com/ChenyangGao/d26a592a0aeb13465511c885d5c7ad61 获取的

这个是仿照阿里云写的,参考文档 https://alist.nn.ci/zh/guide/drivers/115.html

  • 先点获取二维码, 等二维码生成
  • 手机扫码,授权后点击按钮
  • 将token填入 alist 115 的配置中(注意要保证cookie那一栏是空的, 原理是用token换取cookie,所以你配置完成后,token就清空了,cookie那栏会自动填入)

QRCode那一栏有好多不同设备的,手机APP扫码后可以随意指定吗?

可以的,他这token获取cookie的原理是根据参数来区分是哪个设备登陆的,你选择哪个设备,获取cookie就传哪个参数,其实就是从你那个py脚本里抄的

明白了~ 不过现在 Windows已经下架了(客户端),Mac客户端不清楚,新增了 阿里和微信小程序的 嘿嘿~ ❤️

👌我之后更新一下

测试了一下,win, mac, linux 三端都下架了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants