Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: make file api Portable #510

Closed
wenxuwan opened this issue Apr 29, 2022 · 5 comments · Fixed by #556
Closed

proposal: make file api Portable #510

wenxuwan opened this issue Apr 29, 2022 · 5 comments · Fixed by #556
Assignees

Comments

@wenxuwan
Copy link
Member

What would you like to be added:

背景

当前的File接口很难做到可移植性,对于OSS来说,本身就有S3协议这种标准协议,因此本ISSUE提出通过Layotto的拓展API的能力,直接支持S3协议。

官方的阿里云OSS给出了接口的兼容列表,详情见AWS S3兼容性,因此S3协议的设计可以先根据这个兼容列表来做,一些不兼容的接口能力也允许用户调用,但会根据具体的服务提供商来实现。官方列出的接口经测试下来,也并不完整,有接口支持并未列出。本人已对增删改查Object接口做过测试。

设计

在进行接口设计前要理清楚一个问题,S3既然天然支持了可移植性,那么用户如果用S3的SDK编程,就天然具有了可移植性。Layotto要想抽象该套接口,最多也就是将S3接口给抄录下来,请求结构体中的数据基本都是动态参数,每次请求都可能不同,也没法将参数存放到配置文件中去。Layotto唯一能带来的优势能想到的就以下两点:
● AK,SK、EndPoint等信息的托管。
● 用户无需面向SDK开发,client初始化的逻辑可以封装到Layotto中。

基于上述的两个条件,我们需要做的事情就是将S3的结构体搬到pb里面去:

以GetObject为例,此处只是抽出来了部分参数,具体的参数可以全部拷贝过来,也可以看哪些常用的拷贝过来:

image

image

对于该接口因为已经测试过兼容性,因此直接使用aws的能力进行请求(此处先直接桥接到了现有的aws实现上):

image

然后通过下面的方式将该S3Server注册到Layotto上。

image

@seeflood seeflood changed the title Make file api Portable proposal: make file api Portable Apr 30, 2022
@seeflood
Copy link
Member

I propose we call it "AWS S3 API" instead of "File API" :)

@github-actions
Copy link

This issue has been automatically marked as stale because it has not had recent activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue or help wanted) or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label May 31, 2022
@github-actions
Copy link

github-actions bot commented Jun 8, 2022

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue or help wanted. Thank you for your contributions.

@github-actions github-actions bot closed this as completed Jun 8, 2022
@seeflood seeflood reopened this Jun 16, 2022
@github-actions
Copy link

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue or help wanted. Thank you for your contributions.

@seeflood seeflood reopened this Jun 29, 2022
@seeflood
Copy link
Member

seeflood commented Jun 29, 2022

进展:

  • 接口适配了AWS跟阿里云
  • UT覆盖率差0.7%,正在补充
  • 社区代码review
  • 跟生产用户沟通,把OSS底层实现迁移到新版接口上,达到真正的跨平台部署
  • api-spec 代码review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants