Fix null values indexed as "null" strings in flat_object field

[kkewwei]

Signed-off-by: kkewwei [email protected]

Description

The pr soloves three cases:

• fix null value case, I have test too many cases about null value, all of the tests are passed.
• deduplication the value of fieldName, valueList, valueAndPathList, there is no need to store the duplicate values.
• fix infinite loop with null value (comment)

Related Issues

Resolves #13880

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 53b8006: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 3a814c8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 687dbfb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 3d59204: SUCCESS

[github-actions]

❌ Gradle check result for 92c8ddd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[kkewwei]

FAILURE

❌ Gradle check result for 92c8ddd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ReindexBasicTests.testMultipleSources #13913
MasterServiceTests.testClusterStateUpdateLoggingWithDebugEnabled #15117

[github-actions]

❌ Gradle check result for 7023bd1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[kkewwei]

❌ Gradle check result for 7023bd1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

MasterServiceTests.testClusterStateUpdateLoggingWithDebugEnabled #15117

[github-actions]

✅ Gradle check result for 53306f7: SUCCESS

[msfroh]

Hey @kkewwei -- thank you so much for really diving into this fix!

My earlier fix over at #15375 wasn't nearly so complete. With this fix, I believe we'll finally be null-safe.

I left a minor suggestion that should reduce the diff on JsonToStringXContentParserTests.

[msfroh]

Actually, while I still agree with my prior review comments, they don't affect the usefulness of this fix. I'm going to approve and merge, and I can take care of the cosmetic changes.

[kkewwei]

@msfroh, very thank you for your follow-up cosmetic changes.

[cameronattard]

I think I'm encountering the infinite loop. Any chance of getting this backported and released into 2.15 any time soon?

[dblock]

I think I'm encountering the infinite loop. Any chance of getting this backported and released into 2.15 any time soon?

@cameronattard only high sev security fixes get released in patch releases, and only on the latest (2.17), so 2.15 is long gone

[cameronattard]

I think I'm encountering the infinite loop. Any chance of getting this backported and released into 2.15 any time soon?

@cameronattard only high sev security fixes get released in patch releases, and only on the latest (2.17), so 2.15 is long gone

Surely being able to DoS an entire cluster by indexing a single document is a high severity security issue. And 2.15 support was only just added to Amazon OpenSearch Service a week ago. Hardly long gone.