Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-35224][SQL][TESTS][3.1][3.0] Fix buffer overflow in MutableProjectionSuite #32347

Conversation

MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Apr 26, 2021

What changes were proposed in this pull request?

In the test "unsafe buffer with NO_CODEGEN" of MutableProjectionSuite, fix unsafe buffer size calculation to be able to place all input fields without buffer overflow + meta-data.

Why are the changes needed?

To make the test suite MutableProjectionSuite more stable.

Does this PR introduce any user-facing change?

No

How was this patch tested?

By running the affected test suite:

$ build/sbt "test:testOnly *MutableProjectionSuite"

Authored-by: Max Gekk [email protected]
Signed-off-by: Max Gekk [email protected]
(cherry picked from commit d572a85)

…ite`

In the test `"unsafe buffer with NO_CODEGEN"` of `MutableProjectionSuite`, fix unsafe buffer size calculation to be able to place all input fields without buffer overflow + meta-data.

To make the test suite `MutableProjectionSuite` more stable.

No

By running the affected test suite:
```
$ build/sbt "test:testOnly *MutableProjectionSuite"
```

Closes apache#32339 from MaxGekk/fix-buffer-overflow-MutableProjectionSuite.

Authored-by: Max Gekk <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
(cherry picked from commit d572a85)
Signed-off-by: Max Gekk <[email protected]>
@github-actions github-actions bot added the SQL label Apr 26, 2021
@MaxGekk MaxGekk changed the title [SPARK-35224][SQL][TESTS][3.1] Fix buffer overflow in MutableProjectionSuite [SPARK-35224][SQL][TESTS][3.1][3.0] Fix buffer overflow in MutableProjectionSuite Apr 26, 2021
@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42460/

@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42460/

@MaxGekk
Copy link
Member Author

MaxGekk commented Apr 26, 2021

GA passed. Merging to 3.1 and 3.0 (I ran tests for 3.0 locally).
Thank you @maropu and @cloud-fan for reviews.

MaxGekk added a commit that referenced this pull request Apr 26, 2021
…ojectionSuite`

### What changes were proposed in this pull request?
In the test `"unsafe buffer with NO_CODEGEN"` of `MutableProjectionSuite`, fix unsafe buffer size calculation to be able to place all input fields without buffer overflow + meta-data.

### Why are the changes needed?
To make the test suite `MutableProjectionSuite` more stable.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
By running the affected test suite:
```
$ build/sbt "test:testOnly *MutableProjectionSuite"
```

Authored-by: Max Gekk <max.gekkgmail.com>
Signed-off-by: Max Gekk <max.gekkgmail.com>
(cherry picked from commit d572a85)

Closes #32347 from MaxGekk/fix-buffer-overflow-MutableProjectionSuite-3.1.

Authored-by: Max Gekk <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
MaxGekk added a commit that referenced this pull request Apr 26, 2021
…ojectionSuite`

### What changes were proposed in this pull request?
In the test `"unsafe buffer with NO_CODEGEN"` of `MutableProjectionSuite`, fix unsafe buffer size calculation to be able to place all input fields without buffer overflow + meta-data.

### Why are the changes needed?
To make the test suite `MutableProjectionSuite` more stable.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
By running the affected test suite:
```
$ build/sbt "test:testOnly *MutableProjectionSuite"
```

Authored-by: Max Gekk <max.gekkgmail.com>
Signed-off-by: Max Gekk <max.gekkgmail.com>
(cherry picked from commit d572a85)

Closes #32347 from MaxGekk/fix-buffer-overflow-MutableProjectionSuite-3.1.

Authored-by: Max Gekk <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
(cherry picked from commit c59db3d)
Signed-off-by: Max Gekk <[email protected]>
@cloud-fan cloud-fan closed this Apr 26, 2021
@SparkQA
Copy link

SparkQA commented Apr 26, 2021

Test build #137938 has finished for PR 32347 at commit 1349c4f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

flyrain pushed a commit to flyrain/spark that referenced this pull request Sep 21, 2021
…ojectionSuite`

### What changes were proposed in this pull request?
In the test `"unsafe buffer with NO_CODEGEN"` of `MutableProjectionSuite`, fix unsafe buffer size calculation to be able to place all input fields without buffer overflow + meta-data.

### Why are the changes needed?
To make the test suite `MutableProjectionSuite` more stable.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
By running the affected test suite:
```
$ build/sbt "test:testOnly *MutableProjectionSuite"
```

Authored-by: Max Gekk <max.gekkgmail.com>
Signed-off-by: Max Gekk <max.gekkgmail.com>
(cherry picked from commit d572a85)

Closes apache#32347 from MaxGekk/fix-buffer-overflow-MutableProjectionSuite-3.1.

Authored-by: Max Gekk <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants