Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testcase for data being deleted using compaction while having 90% storage utilization #9166

Open
Tracked by #9129
pehala opened this issue Nov 11, 2024 · 1 comment
Assignees

Comments

@pehala
Copy link
Contributor

pehala commented Nov 11, 2024

  • Create 3 node cluster with rf=3.
  • Reaching 90% disk usage.
  • Keep it running for an hour, in-between we should clean up some data by dropping tables and keyspace.
  • Wait for compaction
@cezarmoise
Copy link
Contributor

cezarmoise commented Nov 14, 2024

reached 91% disk usage and started waiting for 30mins, no write or read.

16:59:51  < t:2024-11-14 14:59:50,996 f:full_storage_utilization_test.py l:166  c:FullStorageUtilizationTest2 p:INFO  > Current max disk usage after writing to keyspace14: 91% (395 GB / 392.40000000000003 GB)
16:59:53  < t:2024-11-14 14:59:53,007 f:full_storage_utilization_test.py l:134  c:FullStorageUtilizationTest2 p:INFO  > Wait for 1800 seconds

After 30min idle time, started dropping some tables.

17:31:02  < t:2024-11-14 15:31:02,190 f:full_storage_utilization_test_2.py l:25   c:FullStorageUtilizationTest2 p:INFO  > Executing DROP TABLE keyspace_large3.standard1
17:41:05  < t:2024-11-14 15:41:03,922 f:full_storage_utilization_test_2.py l:25   c:FullStorageUtilizationTest2 p:INFO  > Executing DROP TABLE keyspace_large4.standard1
17:51:08  < t:2024-11-14 15:51:05,916 f:full_storage_utilization_test_2.py l:25   c:FullStorageUtilizationTest2 p:INFO  > Executing DROP TABLE keyspace_small7.standard1
18:01:11  < t:2024-11-14 16:01:08,301 f:full_storage_utilization_test_2.py l:25   c:FullStorageUtilizationTest2 p:INFO  > Executing DROP TABLE keyspace_small5.standard1

Disk usage drops to about 56%

Image

Argus link https://argus.scylladb.com/tests/scylla-cluster-tests/00e90f4b-46ae-489d-b3a4-12b42e540a5e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants