Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix!: drop node 10 support, update TypeScript, remove deprecated fields, remove configstore, align resumable upload behavior, utilize internalized CRC32C utilities #1876

Merged
merged 16 commits into from
May 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/sync-repo-settings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ branchProtectionRules:
- "ci/kokoro: System test"
- docs
- lint
- test (10)
- test (12)
- test (14)
- test (16)
- cla/google
- windows
- OwlBot Post Processor
2 changes: 1 addition & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node: [10, 12, 14]
node: [12, 14, 16]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
Expand Down
34 changes: 0 additions & 34 deletions .kokoro/continuous/node10/common.cfg

This file was deleted.

4 changes: 0 additions & 4 deletions .kokoro/continuous/node10/docs.cfg

This file was deleted.

9 changes: 0 additions & 9 deletions .kokoro/continuous/node10/test.cfg

This file was deleted.

24 changes: 0 additions & 24 deletions .kokoro/continuous/node8/common.cfg

This file was deleted.

Empty file.
34 changes: 0 additions & 34 deletions .kokoro/presubmit/node10/common.cfg

This file was deleted.

4 changes: 0 additions & 4 deletions .kokoro/presubmit/node10/docs.cfg

This file was deleted.

4 changes: 0 additions & 4 deletions .kokoro/presubmit/node10/lint.cfg

This file was deleted.

Empty file.
24 changes: 0 additions & 24 deletions .kokoro/presubmit/node8/common.cfg

This file was deleted.

Empty file removed .kokoro/presubmit/node8/test.cfg
Empty file.
43 changes: 43 additions & 0 deletions internal-tooling/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# nodejs-storage benchmarking

**This is not a supported Google product**

This benchmarking script intended for use by Storage client library maintainers to benchmark various workloads and collect metrics in order to improve performance of the library. Currently the benchmarking runs a Write-1-Read-3 workload and measures throughput.

## Run example:
This runs 10K iterations of Write-1-Read-3 on 5KiB to 2GiB files, and generates output to a CSV file:
```bash
$ cd nodejs-storage
$ npm install
$ cd build/internal-tooling
$ node performanceTest.js --iterations 10000
```

## CLI parameters

| Parameter | Description | Possible values | Default |
| --------- | ----------- | --------------- |:-------:|
| --iterations | number of iterations to run | any positive integer | `100` |
| --numthreads | number of threads to run | any positive integer | `1` |
| --bucket | bucket to upload/download to/from | any string bucket name | `nodejs-perf-metrics` |
| --small | number of bytes for lower bound file size | any positive integer | `5120` |
| --large | number of bytes for upper bound file size | any positive integer | `2.147e9` |
| --projectid | project ID to use | any string project ID | `undefined` |


## Workload definition and CSV headers

For each invocation of the benchmark, write a new object of random size between `small` and `large` . After the successful write, download the object in full three times. For each of the 4 operations record the following fields:

| Field | Description |
| ----- | ----------- |
| Op | the name of the operations (WRITE, READ[{0,1,2}]) |
| ObjectSize | the number of bytes of the object |
| LibBufferSize | configured to use the library default of 100 MiB |
| Crc32cEnabled | whether crc32c was computed for the operation |
| MD5Enabled | whether MD5 was computed for the operation |
| ApiName | default to JSON|
| ElapsedTimeUs | the elapsed time in microseconds the operation took |
| Status | completion state of the operation [OK, FAIL] |
| AppBufferSize | N/A |
| CpuTimeUs | N/A |
Loading