Skip to content

Commit

Permalink
[SPARK-49134][INFRA] Support retry for deploying artifacts to Nexus s…
Browse files Browse the repository at this point in the history 
…taging repository

### What changes were proposed in this pull request?

This PR improves `dev/create-release/release-build.sh` by enabling 3-times retry for deploying artifacts to the Nexus staging repository

When I was setting up 3.5.2-rc5 on my AWS EC2 instance, I encountered an issue with closing the `orgapachespark-1461` due to a timeout while uploading a sha1 file.

```xml
  Uploading spark-streaming-kafka-0-10_2.13/3.5.2/spark-streaming-kafka-0-10_2.13-3.5.2-test-sources.jar.sha1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   262  100   221  100    41     15      2  0:00:20  0:00:13  0:00:07    58
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>408 Request Timeout</title>
</head><body>
<h1>Request Timeout</h1>
<p>Server timeout waiting for the HTTP request from the client.</p>
</body></html>

```

I might choose to upload it manually but I didn't, because I was afraid of making some unpredictable errors.

So I regenerated and uploaded `orgapachespark-1462`.

### Why are the changes needed?

To avoid temporary network errors when performing the publish step for release managers.
### Does this PR introduce _any_ user-facing change?

no

### How was this patch tested?

existing tests

### Was this patch authored or co-authored using generative AI tooling?

no

Closes apache#47643 from yaooqinn/SPARK-49134.

Authored-by: Kent Yao <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
  • Loading branch information
@yaooqinn @HyukjinKwon
yaooqinn authored and HyukjinKwon committed Aug 7, 2024
1 parent 490f0bc commit ce4f185
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion dev/create-release/release-build.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ if [[ "$1" == "publish-release" ]]; then
file_short=$(echo $file | sed -e "s/\.\///")
dest_url="$nexus_upload/org/apache/spark/$file_short"
echo " Uploading $file_short"
curl -u $ASF_USERNAME:$ASF_PASSWORD --upload-file $file_short $dest_url
curl --retry 3 --retry-all-errors -u $ASF_USERNAME:$ASF_PASSWORD --upload-file $file_short $dest_url
done

echo "Closing nexus staging repository"
Expand Down

0 comments on commit ce4f185

Please sign in to comment.