HDDS-11367. Improve ozone balancing status command output #7139

juncevich · 2024-08-30T13:52:50Z

What changes were proposed in this pull request?

Minor improvements to command output:

job task time consumption
iteration time consumption
unfinished iteration shouldn't be in iteration history section
show data units, even it less than gb show it in mb

What is the link to the Apache JIRA

HDDS-11367

How was this patch tested?

Tested by robot test

siddhantsangwan

@juncevich can you please elaborate on what these points mean?

job task time consumption
iteration time consumption

It will help reviewers understand what you're proposing.

juncevich · 2024-09-11T13:33:07Z

@juncevich can you please elaborate on what these points mean?

job task time consumption
iteration time consumption

It will help reviewers understand what you're proposing.

Hi, @siddhantsangwan. Sure, i can explain. Balancing job constists from some iterations. Job task time - the whole balancing time, iteration time - particular iteration time. Job time = iteration time + iteration time ...

...ds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java

juncevich · 2024-09-23T19:31:09Z

Hi siddhantsangwan. Can you review my PR?

siddhantsangwan

@juncevich thanks for working on this. Other than my comments below, we definitely need more testing here. I'd like to at least see:

Unit tests for balancer that verify the iteration duration and other properties are correct.
Tests for the balancer status command that verify the code is correctly picking up the mocked values (mocked response from SCM).

We can also add verifications to the robot tests.

...hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java

...cm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java

...ds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java

siddhantsangwan · 2024-10-01T11:43:14Z

...ds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java

            "Scheduled to move containers", containerMovesScheduled,
            "Already moved containers", containerMovesCompleted,
            "Failed to move containers", containerMovesFailed,
            "Failed to move containers by timeout", containerMovesTimeout,
            "Entered data to nodes", enteringDataNodeList,
            "Exited data from nodes", leavingDataNodeList);
  }
+
+  private String getPrettyIterationStatusInfo(long duration) {


There are probably existing utility methods to do this as well. See if you can find any? It's better to use them since they will be well tested.

I didn't find utility methods. I will extract this method to separate util class and write test for it.

...r-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java

...src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerStatusInfo.java

...cm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java

.../org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTaskIterationStatusInfo.java

...ds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java

ivanzlenko · 2024-10-10T12:33:52Z

...ds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java

                    "%-50s %n%s" +
                    "%-50s %n%s",
            "Key", "Value",
-            "Iteration number", iterationNumber,
+            "Iteration number", iterationNumber == 0 ? "-" : iterationNumber,


For what reason are we inserting string constants using String.format? It doesn't feel right quite to be honest. Can we refactor this?

I don't understand why it's bad and how I should refactor it.

hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/util/DurationUtil.java

...cm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java

...src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerStatusInfo.java

...r-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java

.../org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTaskIterationStatusInfo.java

...cm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java

...r-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java

.../org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTaskIterationStatusInfo.java

adoroszlai · 2024-10-17T15:30:57Z

@juncevich can you please check test timeouts / failures?

org.apache.hadoop.hdds.scm.container.balancer.TestContainerBalancerDatanodeNodeLimit
org.apache.hadoop.hdds.scm.container.balancer.TestContainerBalancerStatusInfo
org.apache.hadoop.hdds.scm.container.balancer.TestContainerBalancerTask

https://github.com/juncevich/ozone/actions/runs/11380961580/artifacts/2067990610

juncevich · 2024-10-17T19:55:17Z

@juncevich can you please check test timeouts / failures?
org.apache.hadoop.hdds.scm.container.balancer.TestContainerBalancerDatanodeNodeLimit
org.apache.hadoop.hdds.scm.container.balancer.TestContainerBalancerStatusInfo
org.apache.hadoop.hdds.scm.container.balancer.TestContainerBalancerTask
https://github.com/juncevich/ozone/actions/runs/11380961580/artifacts/2067990610

Thanks for the comment. Tests fixed.

adoroszlai · 2024-10-18T05:17:21Z

@siddhantsangwan can you please take another look?

# Conflicts: # hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot

siddhantsangwan · 2024-10-22T08:43:55Z

@juncevich Is this ready for another round of reviews?

juncevich · 2024-10-22T09:06:15Z

@juncevich Is this ready for another round of reviews?

Yes, I try to fix the review notices ASAP because they are blocking another fix. I didn’t bring it up because it’s a different task

...cm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java

...r-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java

ivanzlenko · 2024-10-23T13:58:33Z

...r-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java

@@ -118,7 +121,7 @@ public class ContainerBalancerTask implements Runnable {
  private int nextIterationIndex;
  private boolean delayStart;
  private List<ContainerBalancerTaskIterationStatusInfo> iterationsStatistic;
-
+  private OffsetDateTime currentIterationStarted;


Whilst it maybe updated in one thread read access can be done from multiple other threads. This variable should be Atomic.

...r-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java

...s/src/test/java/org/apache/hadoop/hdds/scm/cli/datanode/TestContainerBalancerSubCommand.java

juncevich · 2024-10-23T21:14:20Z

@ivanzlenko No need to use Atomic for OffsetDateTime, because it thread-safe by design.

ivanzlenko · 2024-10-24T08:44:01Z

.../org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTaskIterationStatusInfo.java

-  private final Map<UUID, Long> sizeEnteringNodesGB;
-  private final Map<UUID, Long> sizeLeavingNodesGB;
+  private final Map<UUID, Long> sizeEnteringNodes;
+  private final Map<UUID, Long> sizeLeavingNodes;

  @SuppressWarnings("checkstyle:ParameterNumber")
  public ContainerBalancerTaskIterationStatusInfo(


It is NOT okay to follow bad practices. Please fix it.

HDDS-11367. Improve ozone balancing status command output

774fbbc

juncevich force-pushed the HDDS-11367 branch from f6f34c0 to 9f928d1 Compare September 1, 2024 18:29

ivandika3 requested a review from siddhantsangwan September 3, 2024 02:41

HDDS-11367. Improve ozone balancing status command output

00ad471

juncevich force-pushed the HDDS-11367 branch from 9f928d1 to 00ad471 Compare September 3, 2024 19:00

kerneltime requested review from sodonnel and kerneltime September 9, 2024 16:50

siddhantsangwan reviewed Sep 11, 2024

View reviewed changes

afilpp reviewed Sep 19, 2024

View reviewed changes

...ds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerBalancerStatusSubcommand.java Outdated Show resolved Hide resolved

HDDS-11367. Add space between Gb and Mb in getPrettySize

bac5794

juncevich requested review from siddhantsangwan and afilpp September 19, 2024 07:51

afilpp approved these changes Sep 19, 2024

View reviewed changes

siddhantsangwan requested changes Oct 1, 2024

View reviewed changes

juncevich added 4 commits October 3, 2024 11:01

HDDS-11367. Fixe review notices + add unit tests

9dbd39c

Merge branch 'master' into HDDS-11367

4f6af05

HDDS-11367. Small fixes + improve robot test

570025f

HDDS-11367. Add licences to files

0c04a2a

ivanzlenko suggested changes Oct 8, 2024

View reviewed changes

...src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerStatusInfo.java Outdated Show resolved Hide resolved

juncevich added 2 commits October 8, 2024 10:47

HDDS-11367. Fix review notice

f1f11ec

HDDS-11367. Improve balancer robot test

68766f9

juncevich requested review from ivanzlenko and siddhantsangwan October 8, 2024 08:11

ivanzlenko suggested changes Oct 10, 2024

View reviewed changes

HDDS-11367. Fix review notices

4138ad7

juncevich requested a review from ivanzlenko October 10, 2024 16:18

ivanzlenko suggested changes Oct 15, 2024

View reviewed changes

HDDS-11367. Fix review notices

4d916a5

juncevich requested a review from ivanzlenko October 15, 2024 09:19

ivanzlenko suggested changes Oct 16, 2024

View reviewed changes

juncevich added 2 commits October 16, 2024 23:23

HDDS-11367. Fix review notices

75a103c

HDDS-11367. Improve javadocs

33af9b1

juncevich requested a review from ivanzlenko October 17, 2024 08:16

HDDS-11367. fix timeouts in balancer test

8ff20ee

juncevich added 7 commits October 18, 2024 10:14

Merge branch 'master' into HDDS-11367

4a7d8cd

# Conflicts: # hadoop-ozone/dist/src/main/smoketest/balancer/testBalancer.robot

HDDS-11367. fix robo test

5284e6b

HDDS-11367. fix robo test

051ea1d

HDDS-11367. fix robo test

49b6e14

HDDS-11367. fix robo test

1513b9c

HDDS-11367. fix robo test

16683a2

HDDS-11367. fix robo test

899f314

ivanzlenko suggested changes Oct 23, 2024

View reviewed changes

HDDS-11367. fix review notices

c8e21d9

ivanzlenko suggested changes Oct 24, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDDS-11367. Improve ozone balancing status command output #7139

HDDS-11367. Improve ozone balancing status command output #7139

juncevich commented Aug 30, 2024

siddhantsangwan left a comment

juncevich commented Sep 11, 2024

juncevich commented Sep 23, 2024

siddhantsangwan left a comment

siddhantsangwan Oct 1, 2024

juncevich Oct 2, 2024

ivanzlenko Oct 10, 2024

juncevich Oct 23, 2024

adoroszlai commented Oct 17, 2024

juncevich commented Oct 17, 2024

adoroszlai commented Oct 18, 2024

siddhantsangwan commented Oct 22, 2024

juncevich commented Oct 22, 2024

ivanzlenko Oct 23, 2024

juncevich commented Oct 23, 2024

ivanzlenko Oct 24, 2024

HDDS-11367. Improve ozone balancing status command output #7139

Are you sure you want to change the base?

HDDS-11367. Improve ozone balancing status command output #7139

Conversation

juncevich commented Aug 30, 2024

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

siddhantsangwan left a comment

Choose a reason for hiding this comment

juncevich commented Sep 11, 2024

juncevich commented Sep 23, 2024

siddhantsangwan left a comment

Choose a reason for hiding this comment

siddhantsangwan Oct 1, 2024

Choose a reason for hiding this comment

juncevich Oct 2, 2024

Choose a reason for hiding this comment

ivanzlenko Oct 10, 2024

Choose a reason for hiding this comment

juncevich Oct 23, 2024

Choose a reason for hiding this comment

adoroszlai commented Oct 17, 2024

juncevich commented Oct 17, 2024

adoroszlai commented Oct 18, 2024

siddhantsangwan commented Oct 22, 2024

juncevich commented Oct 22, 2024

ivanzlenko Oct 23, 2024

Choose a reason for hiding this comment

juncevich commented Oct 23, 2024

ivanzlenko Oct 24, 2024

Choose a reason for hiding this comment