Configure cross-compilation for Apple Silicon #6360

SiarheiFedartsou · 2022-09-09T14:40:40Z

Issue

Adds support for Apple Silicon binaries for our NodeJs bindings.

Tasklist

CHANGELOG.md entry (How to write a changelog entry)
update relevant Wiki pages
add tests (see testing documentation)
review
adjust for comments
cherry pick to release branch

Requirements / Relations

Link any requirements here. Other pull requests this PR is based on?

SiarheiFedartsou · 2022-09-14T20:20:46Z

CMakeLists.txt

@@ -185,7 +195,7 @@ if(ENABLE_GOLD_LINKER)
    execute_process(COMMAND ${CMAKE_C_COMPILER} -fuse-ld=gold -Wl,--version ERROR_QUIET OUTPUT_VARIABLE LD_VERSION)
    if("${LD_VERSION}" MATCHES "GNU gold")
        set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=gold -Wl,--disable-new-dtags")
-        set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=gold -Wl,--disable-new-dtags")
+        set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -fuse-ld=gold -Wl,--disable-new-dtags")


It looks as just "copy-paste" mistake(and a reason why I had CMAKE_SHARED_LINKER_FLAGS overridden here :) )

SiarheiFedartsou · 2022-09-14T20:21:08Z

CMakeLists.txt

@@ -489,11 +499,15 @@ if(ENABLE_CONAN)
    NO_OUTPUT_DIRS
    OPTIONS boost:filesystem_version=3 # https://stackoverflow.com/questions/73392648/error-with-boost-filesystem-version-in-cmake
            onetbb:shared=${TBB_SHARED}
+            boost:without_stacktrace=True # Apple Silicon cross-compilation fails without it


We do not use it anyway.

SiarheiFedartsou · 2022-09-14T20:21:55Z

scripts/ci/node_package.sh

-    if [[ ${BUILD_TYPE} == "Debug" ]]; then
-        NPM_FLAGS='--debug'
-    fi
+NPM_FLAGS=''


I don't see why don't we run it on every CI. We have a couple of useful checks here and it turned out that one of them was red, so worth running it always probably.

one of them was red

I mean this check on Linux that we have proper ORIGIN flags a couple of lines below

Yep, makes sense.

SiarheiFedartsou · 2022-09-14T20:23:40Z

CMakeLists.txt

@@ -61,6 +61,16 @@ if (POLICY CMP0074)
 endif()
 project(OSRM C CXX)

+# add @loader_path/$ORIGIN to rpath to support relocatable binaries
+set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)


When I was testing this new package this check was red for me

osrm-backend/scripts/ci/node_package.sh

Line 22 in 96f5780

if [[ $(uname -s) == 'Linux' ]]; then

Tbh I don't know how it worked before, but it is how I fixed that.

SiarheiFedartsou · 2022-09-14T20:25:26Z

.github/workflows/osrm-backend.yml

@@ -234,7 +234,7 @@ jobs:
            CCOMPILER: gcc-7


How I have tested this: since we do not have Apple Silicon based CI we cannot run tests there, so I just downloaded binaries from CI artefacts and ran tests locally putting these binaries to ./lib/binding folder before it.

SiarheiFedartsou · 2022-09-14T20:27:40Z

.github/workflows/osrm-backend.yml

@@ -273,6 +273,20 @@ jobs:
            ENABLE_ASSERTIONS: ON
            ENABLE_CONAN: ON

+          - name: conan-macos-arm64-release-node-16


Added it for only Node 16 & Node-latest(i.e. Node 18), not sure if we actually need it for older versions, but it is easy to add if needed.

SiarheiFedartsou · 2022-09-14T20:32:42Z

scripts/ci/node_package.sh

+    NPM_FLAGS='--debug'
+fi
+
+# append --target_arch in order to handle cross-compilation for Apple Silicon


The only actual change in this file.

SiarheiFedartsou · 2022-09-15T14:47:51Z

Hi @mjjbell, please take a look when you have a chance :)

mjjbell

👍

mjjbell · 2022-09-17T11:59:35Z

scripts/ci/node_package.sh

-    if [[ ${BUILD_TYPE} == "Debug" ]]; then
-        NPM_FLAGS='--debug'
-    fi
+NPM_FLAGS=''


Yep, makes sense.

v5.27.0 - Changes from 5.26.0 - API: - ADDED: Add Flatbuffers support to NodeJS bindings. [Project-OSRM#6338](Project-OSRM#6338) - CHANGED: Add `data_version` field to responses of all services. [Project-OSRM#5387](Project-OSRM#5387) - FIXED: Use Boost.Beast to parse HTTP request. [Project-OSRM#6294](Project-OSRM#6294) - FIXED: Fix inefficient osrm-routed connection handling [Project-OSRM#6113](https://gihub.com/Project-OSRM/osrm-backend/pull/6113) - FIXED: Fix HTTP compression precedence [Project-OSRM#6113](Project-OSRM#6113) - NodeJS: - FIXED: Support `skip_waypoints` in Node bindings [Project-OSRM#6060](Project-OSRM#6060) - Misc: - ADDED: conanbuildinfo.json for easy reading of dependencies [Project-OSRM#6388](Project-OSRM#6388) - CHANGED: Improve performance of JSON rendering. Fix undefined behaviour in JSON numbers formatting. [Project-OSRM#6380](Project-OSRM#6380) - ADDED: Add timestamps for logs. [Project-OSRM#6375](Project-OSRM#6375) - CHANGED: Improve performance of map matching via getPathDistance optimization. [Project-OSRM#6378](Project-OSRM#6378) - CHANGED: Optimize RestrictionParser performance. [Project-OSRM#6344](Project-OSRM#6344) - ADDED: Support floats for speed value in traffic updates CSV. [Project-OSRM#6327](Project-OSRM#6327) - CHANGED: Use Lua 5.4 in Docker image. [Project-OSRM#6346](Project-OSRM#6346) - CHANGED: Remove redundant nullptr check. [Project-OSRM#6326](Project-OSRM#6326) - CHANGED: missing files list is included in exception message. [Project-OSRM#5360](Project-OSRM#5360) - CHANGED: Do not use deprecated Callback::Call overload in Node bindings. [Project-OSRM#6318](Project-OSRM#6318) - FIXED: Fix distance calculation consistency. [Project-OSRM#6315](Project-OSRM#6315) - FIXED: Fix performance issue after migration to sol2 3.3.0. [Project-OSRM#6304](Project-OSRM#6304) - CHANGED: Pass osm_node_ids by reference in osrm::updater::Updater class. [Project-OSRM#6298](Project-OSRM#6298) - FIXED: Fix bug with reading Set values from Lua scripts. [Project-OSRM#6285](Project-OSRM#6285) - FIXED: Bug in bicycle profile that caused exceptions if there is a highway=bicycle in the data. [Project-OSRM#6296](Project-OSRM#6296) - FIXED: Internal refactoring of identifier types used in data facade [Project-OSRM#6044](Project-OSRM#6044) - CHANGED: Update docs to reflect recent build and dependency changes [Project-OSRM#6383](Project-OSRM#6383) - Build: - REMOVED: Get rid of Mason. [Project-OSRM#6387](Project-OSRM#6387) - CHANGED: Use clang-format from CI base image. [Project-OSRM#6391](Project-OSRM#6391) - ADDED: Build Node bindings on Windows. [Project-OSRM#6334](Project-OSRM#6334) - ADDED: Configure cross-compilation for Apple Silicon. [Project-OSRM#6360](Project-OSRM#6360) - CHANGED: Use apt-get to install Clang on CI. [Project-OSRM#6345](Project-OSRM#6345) - CHANGED: Fix TBB in case of Conan + NodeJS build. [Project-OSRM#6333](Project-OSRM#6333) - CHANGED: Migrate to modern TBB version. [Project-OSRM#6300](Project-OSRM#6300) - CHANGED: Enable performance-move-const-arg clang-tidy check. [Project-OSRM#6319](Project-OSRM#6319) - CHANGED: Use the latest node on CI. [Project-OSRM#6317](Project-OSRM#6317) - CHANGED: Migrate Windows CI to GitHub Actions. [Project-OSRM#6312](Project-OSRM#6312) - ADDED: Add smoke test for Docker image. [Project-OSRM#6313](Project-OSRM#6313) - CHANGED: Update libosmium to version 2.18.0. [Project-OSRM#6303](Project-OSRM#6303) - CHANGED: Remove EXACT from find_package if using Conan. [Project-OSRM#6299](Project-OSRM#6299) - CHANGED: Configure Undefined Behaviour Sanitizer. [Project-OSRM#6290](Project-OSRM#6290) - CHANGED: Use Conan instead of Mason to install code dependencies. [Project-OSRM#6284](Project-OSRM#6284) - CHANGED: Migrate to C++17. Update sol2 to 3.3.0. [Project-OSRM#6279](Project-OSRM#6279) - CHANGED: Update macOS CI image to macos-11. [Project-OSRM#6286](Project-OSRM#6286) - CHANGED: Enable even more clang-tidy checks. [Project-OSRM#6273](Project-OSRM#6273) - CHANGED: Configure CMake to not build flatbuffers tests and samples. [Project-OSRM#6274](Project-OSRM#6274) - CHANGED: Enable more clang-tidy checks. [Project-OSRM#6270](Project-OSRM#6270) - CHANGED: Configure clang-tidy job on CI. [Project-OSRM#6261](Project-OSRM#6261) - CHANGED: Use Github Actions for building container images [Project-OSRM#6138](Project-OSRM#6138) - CHANGED: Upgrade Boost dependency to 1.70 [Project-OSRM#6113](Project-OSRM#6113) - CHANGED: Upgrade Ubuntu CI builds to 20.04 [Project-OSRM#6119](Project-OSRM#6119) - CHANGED: Make building osrm-routed optional [Project-OSRM#6144](Project-OSRM#6144) - FIXED: Run all unit tests in CI [Project-OSRM#5248](Project-OSRM#5248) - FIXED: Fix installation of Mason CMake and 32 bit CI build [Project-OSRM#6170](Project-OSRM#6170) - FIXED: Fixed Node docs generation check in CI. [Project-OSRM#6058](Project-OSRM#6058) - CHANGED: Docker build, enabled arm64 build layer [Project-OSRM#6172](Project-OSRM#6172) - CHANGED: Docker build, enabled apt-get update/install caching in separate layer for build phase [Project-OSRM#6175](Project-OSRM#6175) - FIXED: Bump CI complete meta job to ubuntu-20.04 [Project-OSRM#6323](Project-OSRM#6323) - CHANGED: Node packages are now scoped by @Project-OSRM [Project-OSRM#6386](Project-OSRM#6386) - Routing: - CHANGED: Lazily generate optional route path data [Project-OSRM#6045](Project-OSRM#6045) - FIXED: Completed support for no_entry and no_exit turn restrictions. [Project-OSRM#5988](Project-OSRM#5988) - ADDED: Add support for non-round-trips with a single fixed endpoint. [Project-OSRM#6050](Project-OSRM#6050) - FIXED: Improvements to maneuver override processing [Project-OSRM#6125](Project-OSRM#6125) - ADDED: Support snapping to multiple ways at an input location. [Project-OSRM#5953](Project-OSRM#5953) - FIXED: Fix snapping target locations to ways used in turn restrictions. [Project-OSRM#6339](Project-OSRM#6339) - ADDED: Support OSM traffic signal directions. [Project-OSRM#6153](Project-OSRM#6153) - FIXED: Ensure u-turn exists in intersection view. [Project-OSRM#6376](Project-OSRM#6376) - FIXED: Gracefully handle no-turn intersections in guidance processing. [Project-OSRM#6382](Project-OSRM#6382) - Profile: - CHANGED: Bicycle surface speeds [Project-OSRM#6212](Project-OSRM#6212) - Tools: - CHANGED: Do not generate intermediate .osrm file in osrm-extract. [Project-OSRM#6354](Project-OSRM#6354)

Adds support for Apple Silicon binaries for NodeJS bindings.

SiarheiFedartsou changed the title ~~Sf m1~~ Configure cross-compilation for Apple Silicon Sep 9, 2022

SiarheiFedartsou mentioned this pull request Sep 10, 2022

5.27 Release? #6325

Closed

7 tasks

SiarheiFedartsou force-pushed the sf-m1 branch from 0d6f828 to ecf77e5 Compare September 14, 2022 20:19

SiarheiFedartsou commented Sep 14, 2022

View reviewed changes

SiarheiFedartsou force-pushed the sf-m1 branch 2 times, most recently from 46b9a10 to dcaa44d Compare September 15, 2022 13:20

Configure cross-compilation for Apple Silicon

6ed0c11

SiarheiFedartsou force-pushed the sf-m1 branch from dcaa44d to 6ed0c11 Compare September 15, 2022 13:20

SiarheiFedartsou marked this pull request as ready for review September 15, 2022 14:47

SiarheiFedartsou mentioned this pull request Sep 16, 2022

Build Node bindings on Windows #6334

Merged

6 tasks

mjjbell approved these changes Sep 17, 2022

View reviewed changes

scripts/ci/node_package.sh

if [[ ${BUILD_TYPE} == "Debug" ]]; then

NPM_FLAGS='--debug'

fi

NPM_FLAGS=''

Copy link

Member

mjjbell Sep 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, makes sense.

mjjbell merged commit 1c65ea2 into master Sep 17, 2022

mattwigway pushed a commit to mattwigway/osrm-backend that referenced this pull request Jul 20, 2023

Configure cross-compilation for Apple Silicon (Project-OSRM#6360)

3250f09

Adds support for Apple Silicon binaries for NodeJS bindings.

DennisOSRM deleted the sf-m1 branch May 3, 2024 12:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Configure cross-compilation for Apple Silicon #6360

Configure cross-compilation for Apple Silicon #6360

SiarheiFedartsou commented Sep 9, 2022 •

edited

Loading

SiarheiFedartsou Sep 14, 2022

SiarheiFedartsou Sep 14, 2022

SiarheiFedartsou Sep 14, 2022 •

edited

Loading

SiarheiFedartsou Sep 15, 2022 •

edited

Loading

mjjbell Sep 17, 2022

SiarheiFedartsou Sep 14, 2022

SiarheiFedartsou Sep 14, 2022

SiarheiFedartsou Sep 14, 2022 •

edited

Loading

SiarheiFedartsou Sep 14, 2022

SiarheiFedartsou commented Sep 15, 2022

mjjbell left a comment

mjjbell Sep 17, 2022

Configure cross-compilation for Apple Silicon #6360

Configure cross-compilation for Apple Silicon #6360

Conversation

SiarheiFedartsou commented Sep 9, 2022 • edited Loading

Issue

Tasklist

Requirements / Relations

SiarheiFedartsou Sep 14, 2022

Choose a reason for hiding this comment

SiarheiFedartsou Sep 14, 2022

Choose a reason for hiding this comment

SiarheiFedartsou Sep 14, 2022 • edited Loading

Choose a reason for hiding this comment

SiarheiFedartsou Sep 15, 2022 • edited Loading

Choose a reason for hiding this comment

mjjbell Sep 17, 2022

Choose a reason for hiding this comment

SiarheiFedartsou Sep 14, 2022

Choose a reason for hiding this comment

SiarheiFedartsou Sep 14, 2022

Choose a reason for hiding this comment

SiarheiFedartsou Sep 14, 2022 • edited Loading

Choose a reason for hiding this comment

SiarheiFedartsou Sep 14, 2022

Choose a reason for hiding this comment

SiarheiFedartsou commented Sep 15, 2022

mjjbell left a comment

Choose a reason for hiding this comment

mjjbell Sep 17, 2022

Choose a reason for hiding this comment

SiarheiFedartsou commented Sep 9, 2022 •

edited

Loading

SiarheiFedartsou Sep 14, 2022 •

edited

Loading

SiarheiFedartsou Sep 15, 2022 •

edited

Loading

SiarheiFedartsou Sep 14, 2022 •

edited

Loading