Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache PROJ library build when building wheels #1345

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

Cache PROJ library build when building wheels #1345

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
7e8e0c7
Cache PROJ library build when building wheels
djhoese Sep 22, 2023
4198d96
Fix bash if statement
djhoese Sep 22, 2023
fd5f9c0
Fix proj cache condition
djhoese Sep 22, 2023
bfb5e14
Fix return instead of exit in bash function
djhoese Sep 22, 2023
01bc016
Forgot to copy built PROJ to the cache directory
djhoese Sep 22, 2023
27dc3f9
Use a recursive copy for PROJ cache
djhoese Sep 22, 2023
7dda894
Cleanup typos and cp -R usage
djhoese Sep 22, 2023
3823395
Don't cache PROJ build on Windows
djhoese Sep 22, 2023
716f13b
Add verbose flag to cp to debug what's going on
djhoese Sep 22, 2023
ded8d65
Create PROJ_DIR when copying from cache
djhoese Sep 22, 2023
dd62f2a
Try installing PROJ dependencies directly in cache
djhoese Sep 22, 2023
49600d0
Debug linux repair wheel not finding libtiff
djhoese Sep 22, 2023
709cf73
Test if LD_LIBRARY_PATH fixes libtiff auditwheel issue
djhoese Sep 23, 2023
dcba859
Maybe fix quoting in repair wheel command
djhoese Sep 23, 2023
d7b7b7a
Try with no bash
djhoese Sep 23, 2023
4629979
Try lib for library path
djhoese Sep 23, 2023
b3e0a70
Try lib64 for library path
djhoese Sep 23, 2023
32045d2
Add missing /host prefix
djhoese Sep 24, 2023
55316f9
Try using LD_RUN_PATH instead of LD_LIBRARY_PATH
djhoese Sep 25, 2023
a4aba7e
Use LDFLAGS rpath in wheel build
djhoese Sep 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/release.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,13 @@ jobs:
mkdir -p ${GITHUB_WORKSPACE}/pyproj/proj_dir/share/proj
cp "$VCPKG_INSTALLATION_ROOT/installed/${{ matrix.triplet }}/share/proj/"* ${GITHUB_WORKSPACE}/pyproj/proj_dir/share/proj/

- name: Cache PROJ build
if: ${{ !contains(matrix.os, 'windows') }}
uses: actions/cache@v3
with:
path: ${{ runner.temp }}/proj-${{ env.PROJ_VERSION }}
key: ${{ matrix.os }}-${{ matrix.arch }}-${{ env.PROJ_VERSION }}-${{ hashFiles('ci/proj-compile-wheels.sh') }}-cache0

- name: Build wheels
uses: pypa/[email protected]
env:
Expand All @@ -125,11 +132,16 @@ jobs:
PROJ_WHEEL=true
PROJ_NETWORK=ON
PROJ_VERSION=${{ env.PROJ_VERSION }}
PROJ_CACHE=/host${{ runner.temp }}/proj-${{ env.PROJ_VERSION }}/proj
BUILD_PREFIX=/host${{ runner.temp }}/proj-${{ env.PROJ_VERSION }}/proj-deps
PROJ_DIR=/project/pyproj/proj_dir
LDFLAGS="${LDFLAGS} -Wl,-rpath,/host${{ runner.temp }}/proj-${{ env.PROJ_VERSION }}/proj-deps/lib"
CIBW_ENVIRONMENT_MACOS:
PROJ_WHEEL=true
PROJ_NETWORK=ON
PROJ_VERSION=${{ env.PROJ_VERSION }}
PROJ_CACHE=${{ runner.temp }}/proj-${{ env.PROJ_VERSION }}/proj
BUILD_PREFIX=${{ runner.temp }}/proj-${{ env.PROJ_VERSION }}/proj-deps
PROJ_DIR=${GITHUB_WORKSPACE}/pyproj/proj_dir
MACOSX_DEPLOYMENT_TARGET=10.9
CMAKE_OSX_ARCHITECTURES='${{ matrix.cmake_osx_architectures }}'
Expand Down
24 changes: 23 additions & 1 deletion ci/proj-compile-wheels.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,9 +289,31 @@ function build_proj {
touch proj-stamp
}

# Run installation process
function copy_cached_proj_to_build {
if [ ! -d $PROJ_CACHE ]; then
return 1
fi

mkdir -p ${PROJ_DIR}/
cp -Rv ${PROJ_CACHE}/* ${PROJ_DIR}/
}

function copy_build_proj_to_cache {
mkdir -p ${PROJ_CACHE}
cp -Rv ${PROJ_DIR}/* ${PROJ_CACHE}/
}


update_env_for_build_prefix

if copy_cached_proj_to_build ; then
echo "Using cached PROJ build"
exit 0
fi

# Run installation process
suppress build_zlib
suppress build_sqlite
suppress build_libtiff
build_proj
copy_build_proj_to_cache
Loading