enable jobs retry & multi-arch builds of succeeding docker image builds

[AyodeAwe]

Approach:

1. Enable the construction of multi-architecture images even if not all image build matrices are successful. This allows for the publishing of manifests for the images that have been successfully built without waiting for all builds to complete.
   • In this change, we also modify the multi-arch job to only generate a manifest when the necessary images are available; otherwise, we make the job fail.
2. Modify the deletion job to execute only after the publishing and tests jobs succeed. This prevents the need to restart all jobs when a single image build job fails.

Here are the new behaviors:

• The build-multiarch-manifest no longer depends on the build jobs to all pass before publishing multi-arch images to dockerhub. Once the build job completes (either succeeds or fails), the multiarch job builds and pushes manifests for the succeeding images.
• The test job still uses the images from build to run tests.
• The delete-temp-images job will only run AFTER build-multiarch-manifest and test jobs succeed.
• If any jobs fail during the workflow run, we can simply re-run all failed jobs now. Once all jobs pass, the delete job will trigger.

[AyodeAwe]

Will revert this build-multiarch-manifest job to run only on branches after this PR is reviewed/approved.

[AyodeAwe]

PR is continued here (re-targeted for 24.10).