cleanup all shell scripts #128

balamurugana · 2017-09-06T18:17:37Z

No description provided.

nitisht · 2017-09-07T02:21:57Z

looks like ShellCheck is failing @balamurugana can you please fix.

poornas · 2017-09-07T19:02:32Z

build/minio-go/install.sh

+#  limitations under the License.
+#
+
+MINIO_GO_VERSION="v3.0.1"


latest go version is 3.0.2

poornas · 2017-09-07T19:20:39Z

run/core/aws-sdk-php/README.md

+This directory serves as the location for Mint tests using `aws-sdk-php`.  Top level `tests.sh` calls `run.sh` to execute tests.
+
+## Adding new tests
+New tests is added into `quick-tests.php` like as new functions.


New tests is added into quick-tests.php ~~like~~ as new functions.

poornas · 2017-09-07T19:22:26Z

preinstall.sh

+$APT install $(cat install-packages.list)
+
+# set python 3.5 as default
+update-alternatives --install /usr/bin/python python /usr/bin/python3.5 1


python 3.6.2 is latest stable version - should we change version here?

no. python is used whatever is provided by default by ubuntu 16.04

poornas · 2017-09-07T19:25:10Z

mint.sh

+        mkdir -p "$BASE_LOG_DIR/$sdk_name"
+
+        (cd "$BASE_LOG_DIR/$sdk_name"; ./run.sh "$BASE_LOG_DIR/$sdk_name/$OUTPUT_LOG_FILE" "$BASE_LOG_DIR/$sdk_name/$ERROR_LOG_FILE")
+        rv=$?


s/rv/rc/ - more idiomatic.

poornas · 2017-09-07T19:26:48Z

mint.sh

+function main()
+{
+    ## $MINT_MODE is used inside every sdks.
+    echo "To get intermittent logs, 'sudo docker cp ${CONTAINER_ID}:/mint/log /tmp/mint-logs'"


not sure what intermittent means in this context?

you could fetch logs when tests are in progress/running

poornas · 2017-09-07T19:30:07Z

mint.sh

+
+CONTAINER_ID=$(awk -F / '{ print substr($NF, 1, 12) }' /proc/1/cpuset)
+MINT_DATA_DIR=${MINT_DATA_DIR:-/mint/data}
+MINT_MODE=${MINT_MODE:-core}


we need MINT_MODE defined here -minio-go functional tests use MODE=quick, dotnet defines MINT_MODE=quick in latest PR. We need to standardize on the env variable so that SDK's can distinguish between quick tests and verbose tests. Each SDK's Readme.md in the mint run/core/... dir also needs a update.

Will fix in a separate PR dealing with mint modes

balamurugana · 2017-09-07T23:12:25Z

@nitisht fixed shellcheck errors. However shellcheck errors shown now are not errors.

nitisht

Currently, if a specific SDK tests fails, say minio-go fails, mint stops execution of minio-go tests, moves to the next SDK and starts the tests.

Wanted to clarify if that is the expected behaviour or mint should stop the execution completely on first failure and exit?

nitisht · 2017-09-08T06:47:42Z

run/core/aws-sdk-php/README.md

@@ -0,0 +1,17 @@
+## `aws-sdk-php` tests
+This directory serves as the location for Mint tests using `aws-sdk-php`.  Top level `tests.sh` calls `run.sh` to execute tests.


Top level run.sh calls tests.sh to execute tests.

nitisht · 2017-09-08T06:47:50Z

run/core/aws-sdk-ruby/README.md

@@ -0,0 +1,17 @@
+## `aws-sdk-ruby` tests
+This directory serves as the location for Mint tests using `aws-sdk-ruby`.  Top level `tests.sh` calls `run.sh` to execute tests.


Top level run.sh calls aws-stub-tests.rb to execute tests.

nitisht · 2017-09-08T06:48:00Z

run/core/awscli/README.md

-## `awscli` tests.
-This directory serves as the location for Mint tests using `awscli`. To add new test case to Mint `awscli` app, just add new method in the `test.sh` file.
+## `awscli` tests
+This directory serves as the location for Mint tests using `awscli`.  Top level `tests.sh` calls `run.sh` to execute tests.


Top level run.sh calls tests.sh to execute tests.

nitisht · 2017-09-08T06:48:09Z

run/core/mc/README.md

-## `mc` tests.
-This directory serves as the location for Mint tests using `mc`. To add new test case to Mint `mc` app, just add new method in the `test.sh` file.
+## `mc` tests
+This directory serves as the location for Mint tests using `mc`.  Top level `tests.sh` calls `run.sh` to execute tests.


Top level run.sh calls tests.sh to execute tests.

nitisht · 2017-09-08T06:49:52Z

run/core/minio-java/README.md

@@ -0,0 +1,17 @@
+## `minio-java` tests
+This directory serves as the location for Mint tests using `minio-java`.  Top level `tests.sh` calls `run.sh` to execute tests.


Top level run.sh executes Java tests.

nitisht · 2017-09-08T06:50:06Z

run/core/minio-js/README.md

@@ -0,0 +1,17 @@
+## `minio-js` tests
+This directory serves as the location for Mint tests using `minio-js`.  Top level `tests.sh` calls `run.sh` to execute tests.


Top level run.sh executes minio-js tests.

nitisht · 2017-09-08T06:50:23Z

run/core/minio-py/README.md

@@ -0,0 +1,17 @@
+## `minio-py` tests
+This directory serves as the location for Mint tests using `minio-py`.  Top level `tests.sh` calls `run.sh` to execute tests.


Top level run.sh executes minio-py tests.

nitisht · 2017-09-08T06:50:51Z

run/core/minio-go/README.md

@@ -0,0 +1,17 @@
+## `minio-go` tests
+This directory serves as the location for Mint tests using `minio-go`.  Top level `tests.sh` calls `run.sh` to execute tests.


Top level run.sh executes minio-go tests.

nitisht · 2017-09-08T07:01:55Z

However shellcheck errors shown now are not errors.

Agreed, but we need to follow one style and the linter shellcheck enforces that. We can look at other better linter options if available. Travis still fails.

harshavardhana · 2017-09-08T07:07:28Z

@nitisht fixed shellcheck errors. However shellcheck errors shown now are not errors.

@balamurugana there is a reasoning behind such a requested change - https://github.com/koalaman/shellcheck/wiki/SC2046

poornas · 2017-09-08T17:32:06Z

@balamurugana , ran the Dockerfile and Dockerfile.dev - a few more issues:

pressing Ctrl-C would cancel the run earlier but no longer works. Needs to be resuscitated.
minio-js fails with Dockerfile.dev but not Dockerfile

1) functional tests tests for putObject copyObject getObject getPartialObject statObject removeObject should download 100KB string upload and match content:
     Error: content mismatch
      at IncomingMessage.stream.on (test/functional-tests.js:258:16)
      at endReadableNT (_stream_readable.js:974:12)
      at _combinedTickCallback (internal/process/next_tick.js:80:11)
      at process._tickCallback (internal/process/next_tick.js:104:9)

  2) functional tests tests for putObject copyObject getObject getPartialObject statObject removeObject should download partial data (100kb of the 6mb file) and match content:
     Error: content mismatch
      at IncomingMessage.stream.on (test/functional-tests.js:313:16)
      at endReadableNT (_stream_readable.js:974:12)
      at _combinedTickCallback (internal/process/next_tick.js:80:11)
      at process._tickCallback (internal/process/next_tick.js:104:9)

poornas · 2017-09-08T17:40:45Z

might be a good idea to log the sdk version as well in the output/error log.

balamurugana · 2017-09-08T19:22:44Z

All shellcheck errors and comments are addressed

balamurugana · 2017-09-08T19:29:46Z

@nitisht

Currently, if a specific SDK tests fails, say minio-go fails, mint stops execution of minio-go tests, moves to the next SDK and starts the tests.

Wanted to clarify if that is the expected behaviour or mint should stop the execution completely on first failure and exit?

This is the current (and correct) behavior.

@poornas

pressing Ctrl-C would cancel the run earlier but no longer works. Needs to be resuscitated.

done

minio-js fails with Dockerfile.dev but not Dockerfile

Test failures need to be addressed separately in respective SDKs.
Dockerfile should fail as it clones mint and runs release.sh where release.sh is not available (until this PR merges)

balamurugana · 2017-09-08T19:33:32Z

travis failure SC2064: Use single quotes, otherwise this expands now rather than when signalled. is not an error. expansion should not occur later.

nitisht

Looks good other than one typo

nitisht · 2017-09-09T10:24:10Z

run/core/awscli/README.md

- A normal response should be redirected to `$AWSCLI_LOG_FILE`, and error response should be redirected to `$AWSCLI_ERROR_LOG_FILE`.
- Do not proceed to next testcase in case of an error.
+## Adding new tests
+New tests is added into `mint.sh` as new functions.


New tests is added into test.sh as new functions.

This patch does below * cleans up all shell scripts * provides package installation separately * provides way to run tests in non-docker enviroment

balamurugana requested a review from nitisht September 6, 2017 18:18

nitisht requested a review from poornas September 7, 2017 02:22

poornas requested changes Sep 7, 2017

View reviewed changes

balamurugana changed the title ~~Simply all run scripts~~ cleanup all shell scripts Sep 7, 2017

nitisht suggested changes Sep 8, 2017

View reviewed changes

nitisht reviewed Sep 9, 2017

View reviewed changes

cleanup all shell scripts

23026ce

This patch does below * cleans up all shell scripts * provides package installation separately * provides way to run tests in non-docker enviroment

nitisht approved these changes Sep 9, 2017

View reviewed changes

poornas approved these changes Sep 12, 2017

View reviewed changes

nitisht merged commit 120400b into minio:master Sep 12, 2017

balamurugana deleted the cleanup branch September 19, 2017 01:02

kannappanr mentioned this pull request Sep 19, 2017

Modifying the Java build file to handle newly added MintLogger.java #124

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cleanup all shell scripts #128

cleanup all shell scripts #128

balamurugana commented Sep 6, 2017

nitisht commented Sep 7, 2017

poornas Sep 7, 2017

balamurugana Sep 7, 2017

poornas Sep 7, 2017

balamurugana Sep 7, 2017

poornas Sep 7, 2017

balamurugana Sep 7, 2017

poornas Sep 7, 2017

poornas Sep 7, 2017

balamurugana Sep 7, 2017

poornas Sep 7, 2017

balamurugana Sep 7, 2017

balamurugana commented Sep 7, 2017

nitisht left a comment

nitisht Sep 8, 2017

nitisht Sep 8, 2017

nitisht Sep 8, 2017

nitisht Sep 8, 2017

nitisht Sep 8, 2017

nitisht Sep 8, 2017

nitisht Sep 8, 2017

nitisht Sep 8, 2017

nitisht commented Sep 8, 2017 •

edited

Loading

harshavardhana commented Sep 8, 2017

poornas commented Sep 8, 2017

poornas commented Sep 8, 2017

balamurugana commented Sep 8, 2017

balamurugana commented Sep 8, 2017

balamurugana commented Sep 8, 2017

nitisht left a comment

nitisht Sep 9, 2017

balamurugana Sep 9, 2017

		@@ -0,0 +1,17 @@
		## `aws-sdk-php` tests
		This directory serves as the location for Mint tests using `aws-sdk-php`. Top level `tests.sh` calls `run.sh` to execute tests.

		@@ -0,0 +1,17 @@
		## `aws-sdk-ruby` tests
		This directory serves as the location for Mint tests using `aws-sdk-ruby`. Top level `tests.sh` calls `run.sh` to execute tests.

		@@ -0,0 +1,17 @@
		## `minio-java` tests
		This directory serves as the location for Mint tests using `minio-java`. Top level `tests.sh` calls `run.sh` to execute tests.

		@@ -0,0 +1,17 @@
		## `minio-js` tests
		This directory serves as the location for Mint tests using `minio-js`. Top level `tests.sh` calls `run.sh` to execute tests.

		@@ -0,0 +1,17 @@
		## `minio-py` tests
		This directory serves as the location for Mint tests using `minio-py`. Top level `tests.sh` calls `run.sh` to execute tests.

		@@ -0,0 +1,17 @@
		## `minio-go` tests
		This directory serves as the location for Mint tests using `minio-go`. Top level `tests.sh` calls `run.sh` to execute tests.

cleanup all shell scripts #128

cleanup all shell scripts #128

Conversation

balamurugana commented Sep 6, 2017

nitisht commented Sep 7, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

balamurugana commented Sep 7, 2017

nitisht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nitisht commented Sep 8, 2017 • edited Loading

harshavardhana commented Sep 8, 2017

poornas commented Sep 8, 2017

poornas commented Sep 8, 2017

balamurugana commented Sep 8, 2017

balamurugana commented Sep 8, 2017

balamurugana commented Sep 8, 2017

nitisht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nitisht commented Sep 8, 2017 •

edited

Loading