Storage Driver for Aliyun OSS

[denverdino]

This driver implements the storagedriver.StorageDriver interface and uses Aliyun OSS Open Storage Service as storage backend.

The documentation for this driver is in https://github.com/denverdino/distribution/blob/master/docs/storage-drivers/oss.md

To run the unit test case for this driver, you need set the following environment variables before

export ALIYUN_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>
export ALIYUN_ACCESS_KEY_SECRET=<YOUR_ACCESS_SECRET>
export OSS_REGION=<OSS_REGION_ID>   E.g. oss-us-west-1
export OSS_BUCKET=<OSS_BUCKET_NAME>
export OSS_ENCRYPT=false

NOTE: This driver reuses the code logic from the S3 driver.

Signed-off-by: Li Yi ([email protected]) Boshi Lian([email protected])

[stevvooe]

@denverdino This looks like a duplicate of #513. Which one would you like to keep?

[denverdino]

@stevvooe, Steve

This is looking like a straight copy paste from the s3 driver? Any way we can modify the s3 driver to support Aliyun OSS?

If we can abstract S3 driver with the interface, we can have the Aliyun OSS implementation. And currently, I provide the Aliyun OSS GO API in https://github.com/denverdino/aliyungo which refers the API of https://github.com/AdRoll/goamz. So, it should be easy to provide the OSS impl with that.

[stevvooe]

@denverdino Why can't the s3 driver be used directly? What is missing? What is different?

[denverdino]

@stevvooe ,
The current S3 driver is using the impl of github.com/AdRoll/goamz. OSS API is similar to S3, but there are still some differences, e.g. Authentication, Region definition, host for Bucket to access, etc. That is why, we cannot use the S3 driver directly.

So if we can have one abstract S3 driver, I can make the OSS Go API provide the compatibility layer to hide the implementation differences.

Thanks

[GordonTheTurtle]

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:

$ git clone -b "master" [email protected]:denverdino/distribution.git somewhere
$ cd somewhere
$ git rebase -i HEAD~5
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

[stevvooe]

@denverdino Let us know if you'll continue work on this PR. We are closing for now.

[denverdino]

@stevvooe
We will continue maintain the OSS driver. and we have already used it in several internal projects in Aliyun of Alibaba inc. It is pretty solid now.

Can you reopen this PR?

Thanks

[stevvooe]

@denverdino Reopened. Please rebase or merge, since a large number of commits seem to have diverged.

[denverdino]

@Stevooe

Thank you. I will update that.

[aaronlehmann]

This note about consistency looks like it's copied from the S3 driver. Is it really valid for OSS as well?

[denverdino]

Yes, OSS provide the similar capability to S3. Thanks

[aaronlehmann]

AWS has different consistency guarantees in different regions. I was wondering if OSS also has differences between the regions. If not (or if we're not sure), we should remove the part that says "(although in most regions a new object put has guaranteed read after write)". It's a very minor thing - I just want to make sure the comment is accurate.

[dmp42]

@denverdino we fixed the build on master - can you rebase and push -f ?

Thanks!

[denverdino]

I have rebased the PR. Thanks!

[denverdino]

Update the comments of the consistence model per Aaron's comment.
Thanks

[aaronlehmann]

@denverdino: Sorry to request one more change... could you please add build tags to the new files so the driver can be enabled or disabled at compile time? See the rados driver for an example.

Once this is done, we should be able to merge this PR.

Thanks for the contribution!

[tg123]

Hi build tag has been added

should I change circle.yml and add include_oss into DOCKER_BUILDTAGS

[tg123]

@stevvooe
thanks

1. fix with goimports
2. link added
3. todo formatted
4. comment codes removed
5. rebase with latest master

[stevvooe]

@tg123 Is there a link to documentation?

[tg123]

@stevvooe
do you mean add a link to storage-drivers/oss.md in docs/storagedrivers.md

here is the diff 3eaab7d

[stevvooe]

@tg123 Imagine I am a user who has never heard of aliyun oss. Where can I go to get information about how it works?

[tg123]

hi @stevvooe
product link was added in 3eaab7d

http://www.aliyun.com/product/oss is the link to the product

detail
+- [oss](storage-drivers/oss.md): A driver storing objects in [Aliyun OSS](http://www.aliyun.com/product/oss).

or anywhere else I need to add the link to ?

[stevvooe]

@tg123 I requested a link in docs/storage-drivers/oss.md, per #514 (comment).

This ensures the user doesn't have to click back to get further information.

[tg123]

@stevvooe
Hi
link added

but tests did not pass, any way to trigger them again?

[aaronlehmann]

I reran the tests.

[stevvooe]

LGTM

[aaronlehmann]

Merged. Thanks for your contribution!

[denverdino]

@aaronlehmann @stevvooe Thanks for your support

[longwuyuan]

Hi, please advise if this is wrong place to ask this. I intend to use Docker Containers running on Aliyun ECS Servers. Aliyun only has stock CentOS with a 2.6 kernel. So I intend to update kernel and run containers with docker-compose. Does the driver here mean that I can use a specific storage-driver with docker on the servers I described above and relocate /var/lib/docker on OSS ?

[aaronlehmann]

No, this storage driver is for the Docker Registry, which is a server that provides container images for docker pull. It's not involved in running containers.

[denverdino]

@longwuyuan, you can contact to me for this question with email [email protected].

And you can try the Docker Machine with ECS driver for using Docker on Aliyun, the simple tutorial is enclosed

https://github.com/denverdino/aliyungo/wiki/%08%E9%98%BF%E9%87%8C%E4%BA%91Docker-Machine%E9%A9%B1%E5%8A%A8