[fix][broker] Fix incomplete NAR file extraction which prevents broker from starting

[nikhilerigila09]

Fixes 23273

Motivation

This PR addresses an issue where the Pulsar broker fails to restart due to incomplete NAR file extraction. The broker attempts to reuse an incomplete directory in the /tmp folder, leading to a NoSuchFileException. This occurs when the broker is stopped during the extraction process, causing the NAR files to remain in an inconsistent state, which affects future restarts.

Modifications

• extract to a temporary directory which is renamed to the target directory in the last step. This ensures the consistency of the extracted files.

Verifying this change

• Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• Adding new tests that simulate an incomplete NAR extraction, followed by a broker restart to ensure it handles incomplete extractions.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: cognitree#15

[lhotari]

Please follow a different approach where a temp directory is used which is then renamed instead of using a "success file". Example: lhotari@07b2151

[lhotari]

@nikhilerigila09 Are there any challenges in following the advice to use file renaming instead of using a "success file"?

[lhotari]

LGTM, good work @nikhilerigila09

[lhotari]

This could also impact Pulsar Functions or localrun. I've seen that once in the past where localrun would fail due to a incomplete NAR file extraction and the way to resolve the issue was to manually delete the /tmp/pulsar-nar directory.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 38.46154% with 8 lines in your changes missing coverage. Please review.

Project coverage is 73.53%. Comparing base (bbc6224) to head (7868762).
Report is 589 commits behind head on master.

Files with missing lines	Patch %	Lines
...java/org/apache/pulsar/common/nar/NarUnpacker.java	38.46%	6 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23274      +/-   ##
============================================
- Coverage     73.57%   73.53%   -0.05%     
- Complexity    32624    34556    +1932     
============================================
  Files          1877     1929      +52     
  Lines        139502   154539   +15037     
  Branches      15299    18024    +2725     
============================================
+ Hits         102638   113634   +10996     
- Misses        28908    32222    +3314     
- Partials       7956     8683     +727     

Flag	Coverage Δ
inttests	29.39% <0.00%> (+4.80%)	⬆️
systests	25.87% <38.46%> (+1.55%)	⬆️
unittests	72.70% <38.46%> (-0.14%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...java/org/apache/pulsar/common/nar/NarUnpacker.java	72.85% <38.46%> (-5.84%)	⬇️

... and 589 files with indirect coverage changes