Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-31231][BUILD] Explicitly setuptools version as 46.0.0 in pip p…
…ackage test ### What changes were proposed in this pull request? For a bit of background, PIP packaging test started to fail (see [this logs](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120218/testReport/)) as of setuptools 46.1.0 release. In pypa/setuptools#1424, they decided to don't keep the modes in `package_data`. In PySpark pip installation, we keep the executable scripts in `package_data` https://github.com/apache/spark/blob/fc4e56a54c15e20baf085e6061d3d83f5ce1185d/python/setup.py#L199-L200, and expose their symbolic links as executable scripts. So, the symbolic links (or copied scripts) executes the scripts copied from `package_data`, which doesn't have the executable permission in its mode: ``` /tmp/tmp.UmkEGNFdKF/3.6/bin/spark-submit: line 27: /tmp/tmp.UmkEGNFdKF/3.6/lib/python3.6/site-packages/pyspark/bin/spark-class: Permission denied /tmp/tmp.UmkEGNFdKF/3.6/bin/spark-submit: line 27: exec: /tmp/tmp.UmkEGNFdKF/3.6/lib/python3.6/site-packages/pyspark/bin/spark-class: cannot execute: Permission denied ``` The current issue is being tracked at pypa/setuptools#2041 </br> For what this PR proposes: It sets the upper bound in PR builder for now to unblock other PRs. _This PR does not solve the issue yet. I will make a fix after monitoring https://github.com/pypa/setuptools/issues/2041_ ### Why are the changes needed? It currently affects users who uses the latest setuptools. So, _users seem unable to use PySpark with the latest setuptools._ See also pypa/setuptools#2041 (comment) ### Does this PR introduce any user-facing change? It makes CI pass for now. No user-facing change yet. ### How was this patch tested? Jenkins will test. Closes #27995 from HyukjinKwon/investigate-pip-packaging. Authored-by: HyukjinKwon <[email protected]> Signed-off-by: HyukjinKwon <[email protected]> (cherry picked from commit c181c45) Signed-off-by: HyukjinKwon <[email protected]>
- Loading branch information