[FEATURE] Add retry for BWC tests

[chishui]

Is your feature request related to a problem?

BWC tests are quite flaky, here is one failed test example:

• https://github.com/opensearch-project/neural-search/actions/runs/9377650430/job/25871682884

We can introduce retry for BWC tests to reduce the chance of random failures.

What solution would you like?

OpenSearch core repo uses test-retry plugin to retry flaky tests: https://github.com/opensearch-project/OpenSearch/blob/main/build.gradle#L469

What alternatives have you considered?

N/A

Do you have any additional context?

N/A

[dblock]

[Catch All Triage - Attendees 1, 2, 3, 4, 5]

[dblock]

Double check that the errors are truly transient. Retrying often just hides eventual consistency errors and is not recommended.

[Catch All Triage - Attendees 1, 2, 3, 4, 5]