Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable CI for kvrocks2redis #2175

Merged
merged 16 commits into from
Apr 21, 2024
Merged

Conversation

Zakelly
Copy link
Contributor

@Zakelly Zakelly commented Mar 16, 2024

This fixes #1976 and fixes #1338 . Add CI for kvrocks2redis.

Limitation:

  1. Segment fault in ubuntu with Tsan.
  2. Tests failed with Asan or MacOS. Seems the kvrocks2redis does not work properly.

So CI disabled for these env for now.

Copy link
Member

@jihuayu jihuayu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!
Can you create the issues to trace the failed test?
Redis-server compile failed in m1 mac(macos-14), Maybe we can disable install it in macos-14 runner

.github/workflows/kvrocks.yaml Show resolved Hide resolved
.github/workflows/kvrocks.yaml Show resolved Hide resolved
Copy link
Member

@PragmaTwice PragmaTwice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution.

.github/workflows/kvrocks.yaml Outdated Show resolved Hide resolved
utils/kvrocks2redis/tests/ci/kvrocks.conf Outdated Show resolved Hide resolved
@PragmaTwice
Copy link
Member

Tests failed with Asan or MacOS. Seems the kvrocks2redis does not work properly.

It could be very helpful if you can open an issue for the test failure, with asan log.

@Zakelly
Copy link
Contributor Author

Zakelly commented Mar 19, 2024

I enable the CI for macos and with tsan/asan, we can check the failure first.

@Zakelly Zakelly force-pushed the kvrocks2redis_ci branch 2 times, most recently from 20726e6 to aa8e555 Compare March 19, 2024 15:28
@jihuayu
Copy link
Member

jihuayu commented Mar 21, 2024

Hi @Zakelly, you don't need to use force push to make sure commit only one.
We will squash your PR to one commit on merge.

@Zakelly
Copy link
Contributor Author

Zakelly commented Mar 22, 2024

Here's one example result for enabling CI in all envs. https://github.com/Zakelly/kvrocks/actions/runs/8344973371/job/22838786340
FYI. I'm not sure if I can retrieve an asan log from this.

@PragmaTwice
Copy link
Member

PragmaTwice commented Mar 22, 2024

No worry. You can firstly work on this PR and make it merged, with ASAN/TSAN/macos.. disabled.

And then feel free to investigate the memory issue in kvrocks2redis, if you are interested.

@Zakelly
Copy link
Contributor Author

Zakelly commented Mar 23, 2024

Created #2195 to track CI problem with tsan/asan and macos. Disabled for them....

Comment on lines 322 to 324
sleep 15s
python3 utils/kvrocks2redis/tests/populate-kvrocks.py --password="" --flushdb=true
sleep 15s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why should it be 15s?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We wait here to make sure the daemonized processes are initialized, or the background process finishes its work. 10s also works.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change it to 10s to save some CI time : )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, changed to 10s. : )

@Zakelly
Copy link
Contributor Author

Zakelly commented Mar 24, 2024

And I found the redis-server compiles failed in macos14, seems related with redis/redis#12585. Could we upgrade the redis from 6.2.7 to 6.2.14 in CI?

@jihuayu
Copy link
Member

jihuayu commented Mar 26, 2024

And I found the redis-server compiles failed in macos14, seems related with redis/redis#12585. Could we upgrade the redis from 6.2.7 to 6.2.14 in CI?

I think it is OK! @Zakelly

@Zakelly
Copy link
Contributor Author

Zakelly commented Apr 8, 2024

@jihuayu @PragmaTwice I think this PR is in good shape, and all the CI has passed for commit 2246fac: https://github.com/Zakelly/kvrocks/actions/runs/8589761419 .

Please let me know if you have any other suggestions or comments, thanks!

jihuayu
jihuayu previously approved these changes Apr 9, 2024
Copy link
Member

@jihuayu jihuayu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for your persistence and efforts.

@PragmaTwice PragmaTwice merged commit 0f2de7d into apache:unstable Apr 21, 2024
30 checks passed
Copy link

sonarcloud bot commented Apr 21, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add kvrocks2redis test case runnig to CI workflow Add basic tests for kvrocks2redis
4 participants