[CI] Speed up wheel commit validation check by 100x (#33434)

Speed up wheel commit validation check by 100x. Also hopefully will alleviate if not eliminate the 'Observed wheel commit () is not expected' issue (#32156) that has been creeping through many of ci/cd builds in our pipeline. The existing code uses pipe to read from a rather large file (>50MB). Pipe however has buffer limit which by default in term of kb (https://man7.org/linux/man-pages/man7/pipe.7.html) so what we look for might not exist. We can fix this by tell unzip the exact file we are looking for. That file is pretty small so we should not hit buffer limit. Signed-off-by: Cuong Nguyen <[email protected]>
ray-project · Mar 20, 2023 · 52f0ee8 · 52f0ee8
1 parent 305dc2a
commit 52f0ee8
Showing 1 changed file with 2 additions and 2 deletions.
diff --git a/ci/ci.sh b/ci/ci.sh
@@ -415,10 +415,10 @@ validate_wheels_commit_str() {
       continue
     fi
 
-    WHL_COMMIT=$(unzip -p "$whl" | grep "^__commit__" | awk -F'"' '{print $2}')
+    WHL_COMMIT=$(unzip -p "$whl" "*ray/__init__.py" | grep "^__commit__" | awk -F'"' '{print $2}')
 
     if [ "${WHL_COMMIT}" != "${EXPECTED_COMMIT}" ]; then
-      echo "Error: Observed wheel commit (${WHL_COMMIT}) is not expected commit (${EXPECTED_COMMIT}). Aborting."
+      echo "Wheel ${basename} has incorrect commit: (${WHL_COMMIT}) is not expected commit (${EXPECTED_COMMIT}). Aborting."
       exit 1
     fi