[fix][broker] Fix ModularLoadManagerImpl always delete active bundle-data. sec ver.

[lifepuzzlefun]

Motivation

fix #20544

this problem will cause LoadManager's loadSheding and BundleSplitTask not functioning. because of the lost of real cluster-wide bundle-data in LoadManager.

master code logic in ModularLoadManagerImpl will try to clean up bundle-data if the namespace bundle is splitted.

but the current logic will cause amost all bundle-datas in the cluster to be deleted, each time the updateBundleData method triggered.

Modifications

move bundle-data cleanup logic after iterator all broker to get cluster wide activeBundles in updateBundleData method.

Verifying this change

add unittest to check if the broker number is more than one.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository:

[lifepuzzlefun]

@heesung-sn I think this version is more easily to understand. please take a look

[heesung-sn]

Can you explain why this reordering can resolve the issue?

[lifepuzzlefun]

Sure.
If we need to figure out which bundle is not active, we should get cluster wide all bundle data, right ? To get all cluster wide bundle data, we need iterate all brokerData to collect this.
Current we only iterate the first brokerData and check active bundles. so the all the bundleData which is not owned by the first broker will be cleaned at the first broker check.

[heesung-sn]

Sorry, maybe I missed some logic here.

But, In this func, I don't see bundleData and activeBundles are updated between these lines. https://github.com/apache/pulsar/pull/20620/files#diff-642d3e26ddad51db8fb736b0f5519f4c929814a4f69d85e16829d19b885be0ebR562-R594 .
So, I am curious to know how this reordering can be effective.

To get all cluster wide bundle data, we need iterate all brokerData to collect this.
Current we only iterate the first brokerData and check active bundles. so the all the bundleData which is not owned by the first broker will be cleaned at the first broker check.

Can you provide the code lines which iterate all brokerData and fill activeBundles and bundleData, other than here?
https://github.com/apache/pulsar/pull/20620/files#diff-642d3e26ddad51db8fb736b0f5519f4c929814a4f69d85e16829d19b885be0ebR549-R558

[lifepuzzlefun]

before:
after:

@heesung-sn if you fold the code lines you can see the logic clearly.

[heesung-sn]

Oh. I mis-read the change. I got it now. This removal has been moved out of the loop, for (Map.Entry<String, BrokerData> brokerEntry : loadData.getBrokerData().entrySet())

            //Remove not active bundle from loadData
            for (String bundle : bundleData.keySet()) {
                if (!activeBundles.contains(bundle)){
                    bundleData.remove(bundle);
                    if (pulsar.getLeaderElectionService().isLeader()){
                        deleteBundleDataFromMetadataStore(bundle);
                    }
                }
            }

LGTM.

[lifepuzzlefun]

@Demogorgon314 could you help review this pr?

[Demogorgon314]

LGTM.

[aloyszhang]

Nice cath, LGTM.

[aloyszhang]

@lifepuzzlefun plz resolve the conflict

[lifepuzzlefun]

@lifepuzzlefun plz resolve the conflict

Thanks for review. Conflict has been resolved. : -)