Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use chunked upload for all put object. #595

Merged
merged 1 commit into from
Aug 18, 2017

Conversation

balamurugana
Copy link
Member

@balamurugana
Copy link
Member Author

encrypted putObject() fails with unknown reason. Debugging it.

@deekoder deekoder requested review from krisis and nitisht July 28, 2017 17:18
@balamurugana balamurugana force-pushed the fix-issue-568 branch 2 times, most recently from b4c3f46 to 6ea9652 Compare July 29, 2017 07:09
@balamurugana
Copy link
Member Author

All issues are fixed.
All functional tests run successfully on play and Amazon S3.

@nitisht
Copy link
Contributor

nitisht commented Aug 1, 2017

Could you please fix the conflict in FunctionalTest.java @balamurugana

@balamurugana balamurugana force-pushed the fix-issue-568 branch 2 times, most recently from 102f4cc to 88e1df3 Compare August 1, 2017 07:59
@balamurugana
Copy link
Member Author

travis-ci fails due to its internal problem.

@harshavardhana
Copy link
Member

travis-ci fails due to its internal problem.

oraclejdk8 failed for functional tests.

The command "./gradlew check" exited with 0.
104.52s$ if java -version 2>&1 | grep -q 'java version "1.8.'; then ./gradlew runFunctionalTest; else true; fi
:api:compileJava UP-TO-DATE
:api:processResources UP-TO-DATE
:api:classes UP-TO-DATE
:api:jar UP-TO-DATE
:functional:compileJava
:functional:processResources UP-TO-DATE
:functional:classes
:functional:runFunctionalTest
Test: makeBucket(String bucketName)
Test: listBuckets()
Test: bucketExists(String bucketName)
Test: removeBucket(String bucketName)
Test: putObject(String bucketName, String objectName, String filename)
Test: multipart: putObject(String bucketName, String objectName, String filename)
Test: putObject(String bucketName, String objectName, String filename, String contentType)
Test: putObject(String bucketName, String objectName, InputStream body, long size, String contentType)
Test: multipart resume: putObject(String bucketName, String objectName, InputStream body, long size, String contentType)
Test: objectName with path segments: putObject(String bucketName, String objectName, InputStream body, long size, String contentType)
Test: putObject(String bucketName, String objectName, InputStream body, String contentType)
Test: multipart: putObject(String bucketName, String objectName, InputStream body, String contentType)
Test: putObject(String bucketName, String objectName, InputStream stream, long size, String contentType, SecretKey key).
Test: putObject(String bucketName, String objectName, InputStream stream, long size, String contentType, KeyPair keyPair).
Test: putObject(String bucketName, String objectName, InputStream stream, long size, Map<String, String> headerMap).
Test: statObject(String bucketName, String objectName)
Test: getObject(String bucketName, String objectName)
Test: getObject(String bucketName, String objectName, long offset)
Test: getObject(String bucketName, String objectName, long offset, Long length)
Test: getObject(String bucketName, String objectName, String filename)
Test: objectName with multiple path segments: getObject(String bucketName, String objectName, String filename)
Test: getObject(String bucketName, String objectName, SecretKey key).
:functional:runFunctionalTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':functional:runFunctionalTest'.
> Process 'command '/usr/lib/jvm/java-8-oracle/bin/java'' finished with non-zero exit value 137
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 1 mins 43.336 secs
The command "if java -version 2>&1 | grep -q 'java version "1.8.'; then ./gradlew runFunctionalTest; else true; fi" exited with 1.

@balamurugana
Copy link
Member Author

@harshavardhana you would need to check other runs as well.

@harshavardhana
Copy link
Member

@harshavardhana you would need to check other runs as well.

Travis has updated their image to trusty which doesn't ship oraclejdk7 anymore - travis-ci/travis-ci#7884 - Fix is to remove oraclejdk7

http://www.webupd8.org/2017/06/why-oracle-java-7-and-6-installers-no.html - We need to rework our .travis.yml

@nitisht
Copy link
Contributor

nitisht commented Aug 1, 2017

Tested locally against AWS S3, tests pass.

nitisht
nitisht previously approved these changes Aug 1, 2017
Copy link
Contributor

@nitisht nitisht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


this.bytesRead++;
// Return value must be between 0 to 255.
return this.chunkBody[this.chunkPos++] & 0xFF;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this.chunkBody[this.chunkPos++] evaluate? Does it return the value at index this.chunkPos before the increment? Is it non-idiomatic to do the increment and indexing in separate statements?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it return the value at index this.chunkPos before the increment?

Yes. As ++ operator placed after variable, the variable is incremented after reading the value. I am breaking the statement into two for better readability

@balamurugana balamurugana force-pushed the fix-issue-568 branch 2 times, most recently from a81252d to 2b0fa3c Compare August 3, 2017 08:44
@balamurugana
Copy link
Member Author

Depends on #596 for travis-ci failure.

@balamurugana
Copy link
Member Author

I see oraclejdk8 fails in travis-ci with below error

Test: getObject(String bucketName, String objectName, SecretKey key).

:functional:runFunctionalTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:

Execution failed for task ':functional:runFunctionalTest'.

> Process 'command '/usr/lib/jvm/java-8-oracle/bin/java'' finished with non-zero exit value 137

here, exit code denotes signal 9 killed the process (formula: 128 + signal = 137). It means OOM in linux. Not sure why OOM only on oraclejdk8.

@harshavardhana
Copy link
Member

I see oraclejdk8 fails in travis-ci with below error

Test: getObject(String bucketName, String objectName, SecretKey key).

:functional:runFunctionalTest FAILED

FAILURE: Build failed with an exception.

  • What went wrong:

Execution failed for task ':functional:runFunctionalTest'.

Process 'command '/usr/lib/jvm/java-8-oracle/bin/java'' finished

We might have to test it with oraclejdk8 . I don't have oraclejdk8 locally will install and check.

@balamurugana balamurugana force-pushed the fix-issue-568 branch 4 times, most recently from 7ebeb9d to 0a9f542 Compare August 14, 2017 19:53
@balamurugana
Copy link
Member Author

Travis CI succeeded without any change in the PR

krisis
krisis previously approved these changes Aug 15, 2017
@balamurugana
Copy link
Member Author

@krisis @nitisht please add +1 which disappeared by my refresh

nitisht
nitisht previously approved these changes Aug 17, 2017
krisis
krisis previously approved these changes Aug 17, 2017
@deekoder deekoder merged commit e1ea8c6 into minio:master Aug 18, 2017
@balamurugana balamurugana deleted the fix-issue-568 branch August 29, 2017 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants