Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky Cassandra test: testIndexUpdatesWithReindexAndRemove #1498

Open
FlorianHockmann opened this issue Mar 18, 2019 · 5 comments
Open

Flaky Cassandra test: testIndexUpdatesWithReindexAndRemove #1498

FlorianHockmann opened this issue Mar 18, 2019 · 5 comments
Labels
storage/cassandra testing/flaky

Comments

@FlorianHockmann
Copy link
Member 

[ERROR] Failures: 
[ERROR]   AstyanaxGraphTest>JanusGraphTest.testIndexUpdatesWithReindexAndRemove:1500 expected: not equal but was: <0>

Job was:

MODULE='cassandra' ARGS='-Dtest=**/graphdb/astyanax/*'
@kptfh
Copy link
Contributor

kptfh commented Mar 22, 2019

Have the same failure for Aerospike storage backend
In my case it's the only failing test but it fails constantly.
It fails on this assert
As I see because this query doesn't filter on indexRelationTypeName

@kptfh
Copy link
Contributor

kptfh commented Apr 5, 2019

@FlorianHockmann here is the root cause of this issue

@FlorianHockmann
Copy link
Member Author

@kptfh Thanks for digging into this! Do you plan to submit a PR to fix this?

@kptfh
Copy link
Contributor

kptfh commented Apr 5, 2019

Not sure that I know how to fix it correctly

@FlorianHockmann
Copy link
Member Author

The test is still flaky:

Error:  Errors: 
Error:    CQLGraphCacheTest>JanusGraphTest.testIndexUpdatesWithReindexAndRemove:2010 » JanusGraph Index did not stabilize within the given amount of time. For sufficiently long wait periods this is most likely caused by a failed/incorrectly shut down JanusGraph instance or a lingering transaction.

FlorianHockmann added a commit to GDATASoftwareAG/janusgraph that referenced this issue Sep 7, 2023
@FlorianHockmann
Mark more tests as flaky
80ca920 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#1498
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959
* JanusGraph#3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: JanusGraph#3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

Signed-off-by: Florian Hockmann <[email protected]>
@FlorianHockmann FlorianHockmann mentioned this issue Sep 7, 2023
Merged
FlorianHockmann added a commit to GDATASoftwareAG/janusgraph that referenced this issue Sep 13, 2023
@FlorianHockmann
Mark more tests as flaky
87911fd 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#1498
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959
* JanusGraph#3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: JanusGraph#3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

I also added a link to the relevant issue to all flaky tests.

Signed-off-by: Florian Hockmann <[email protected]>
FlorianHockmann added a commit that referenced this issue Sep 14, 2023
@FlorianHockmann
Mark more tests as flaky
2ba933c 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* #1457
* #1459
* #1462
* #1464
* #1465
* #1497
* #1498
* #2272
* #3142
* #3356
* #3392
* #3393
* #3651
* #3931
* #3959
* #3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: #3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

I also added a link to the relevant issue to all flaky tests.

Signed-off-by: Florian Hockmann <[email protected]>
FlorianHockmann added a commit to FlorianHockmann/janusgraph that referenced this issue Sep 14, 2023
@FlorianHockmann
Mark more tests as flaky
b7386bc 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#1498
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959
* JanusGraph#3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: JanusGraph#3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

I also added a link to the relevant issue to all flaky tests.

Signed-off-by: Florian Hockmann <[email protected]>
(cherry picked from commit 2ba933c)

# Conflicts:
#	janusgraph-cql/src/test/java/org/janusgraph/graphdb/cql/CQLGraphTest.java
#	janusgraph-solr/src/test/java/org/janusgraph/diskstorage/solr/CQLSolrTest.java
#	janusgraph-test/src/test/java/org/janusgraph/diskstorage/cache/ExpirationCacheTest.java
FlorianHockmann added a commit that referenced this issue Sep 14, 2023
@FlorianHockmann
Mark more tests as flaky
9553819 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* #1457
* #1459
* #1462
* #1464
* #1465
* #1497
* #1498
* #2272
* #3142
* #3356
* #3392
* #3393
* #3651
* #3931
* #3959
* #3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: #3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

I also added a link to the relevant issue to all flaky tests.

Signed-off-by: Florian Hockmann <[email protected]>
(cherry picked from commit 2ba933c)

# Conflicts:
#	janusgraph-cql/src/test/java/org/janusgraph/graphdb/cql/CQLGraphTest.java
#	janusgraph-solr/src/test/java/org/janusgraph/diskstorage/solr/CQLSolrTest.java
#	janusgraph-test/src/test/java/org/janusgraph/diskstorage/cache/ExpirationCacheTest.java
FlorianHockmann added a commit to FlorianHockmann/janusgraph that referenced this issue Mar 11, 2024
@FlorianHockmann
Mark test as flaky for all backends
b6ee391 
We only annotated the test for CQL as flaky, but
it's also flaky for other backends. contributor
@kptfh already commented on the issue that it's
also flaky for the Aerospike backend:
JanusGraph#1498 (comment)

and Natalia recently commented on Discord that the test is flaky for
in-memory.

So, we should just move the annotation to the test itself to be not
specific for any backend.
Since the test is copied to `JanusGraphCustomIdTest`, we also need the
annotation there (and this is also the place where Natalia hit the flaky
test).

Issue: JanusGraph#1498 (not fixed by this)

Signed-off-by: Florian Hockmann <[email protected]>
@FlorianHockmann FlorianHockmann mentioned this issue Mar 11, 2024
Merged
9 tasks
FlorianHockmann added a commit that referenced this issue Mar 12, 2024
@FlorianHockmann
Mark test as flaky for all backends
e955783 
We only annotated the test for CQL as flaky, but
it's also flaky for other backends. contributor
@kptfh already commented on the issue that it's
also flaky for the Aerospike backend:
#1498 (comment)

and Natalia recently commented on Discord that the test is flaky for
in-memory.

So, we should just move the annotation to the test itself to be not
specific for any backend.
Since the test is copied to `JanusGraphCustomIdTest`, we also need the
annotation there (and this is also the place where Natalia hit the flaky
test).

Issue: #1498 (not fixed by this)

Signed-off-by: Florian Hockmann <[email protected]>
janusgraph-automations pushed a commit that referenced this issue Mar 12, 2024
@FlorianHockmann
Mark test as flaky for all backends
ab87971 
We only annotated the test for CQL as flaky, but
it's also flaky for other backends. contributor
@kptfh already commented on the issue that it's
also flaky for the Aerospike backend:
#1498 (comment)

and Natalia recently commented on Discord that the test is flaky for
in-memory.

So, we should just move the annotation to the test itself to be not
specific for any backend.
Since the test is copied to `JanusGraphCustomIdTest`, we also need the
annotation there (and this is also the place where Natalia hit the flaky
test).

Issue: #1498 (not fixed by this)

Signed-off-by: Florian Hockmann <[email protected]>
(cherry picked from commit e955783)
FlorianHockmann added a commit that referenced this issue Mar 13, 2024
@FlorianHockmann
Mark test as flaky for all backends
f0a08f9 
We only annotated the test for CQL as flaky, but
it's also flaky for other backends. contributor
@kptfh already commented on the issue that it's
also flaky for the Aerospike backend:
#1498 (comment)

and Natalia recently commented on Discord that the test is flaky for
in-memory.

So, we should just move the annotation to the test itself to be not
specific for any backend.
Since the test is copied to `JanusGraphCustomIdTest`, we also need the
annotation there (and this is also the place where Natalia hit the flaky
test).

Issue: #1498 (not fixed by this)

Signed-off-by: Florian Hockmann <[email protected]>
(cherry picked from commit e955783)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
storage/cassandra testing/flaky
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kptfh @FlorianHockmann