grpc 1.40.0 #84990

bevanjkay · 2021-09-10T13:42:19Z

Created with brew bump-formula-pr.

carlocab · 2021-09-11T16:57:24Z

sysdig fails to build on Linux with

/home/linuxbrew/.linuxbrew/bin/ld: ../libsinsp/libsinsp.a(cri.grpc.pb.cc.o): undefined reference to symbol '_ZN4absl12lts_202103245MutexD1Ev'
/home/linuxbrew/.linuxbrew/bin/ld: /home/linuxbrew/.linuxbrew/lib/libabsl_synchronization.so.2103.0.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
userspace/sysdig/CMakeFiles/csysdig.dir/build.make:136: recipe for target 'userspace/sysdig/csysdig' failed
make[2]: *** [userspace/sysdig/csysdig] Error 1

CC @Homebrew/linux

github-actions · 2021-09-14T00:10:28Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. To keep this pull request open, add a help wanted or in progress label.

iMichka · 2021-09-16T05:19:06Z

Might have to do with the linker flags. Maybe the order? The missing symbol comes from abseil:

[ 99%] Linking CXX executable sysdig
cd /tmp/sysdig-20210915-64838-1rj5rj8/sysdig-0.27.1/build/userspace/sysdig && /home/linuxbrew/.linuxbrew/Cellar/cmake/3.21.2/bin/cmake -E cmake_link_script CMakeFiles/sysdig.dir/link.txt --verbose=1
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/shims/linux/super/g++-5  -Wall -ggdb   -std=c++0x -O3 -fno-strict-aliasing -DNDEBUG -rdynamic CMakeFiles/sysdig.dir/fields_info.cpp.o CMakeFiles/sysdig.dir/sysdig.cpp.o -o sysdig  ../libsinsp/libsinsp.a ../libscap/libscap.a -lelf -lz -lcurl -ljsoncpp -ltbb -lcares -lgrpc++_unsecure -lgrpc_unsecure -lprotobuf -lcares -lgrpc++_unsecure -lgrpc_unsecure -lprotobuf -ljq -Wl,-Bstatic -lb64 -Wl,-Bdynamic -lrt -lanl -lssl -lcrypto -Wl,-Bstatic -lluajit -Wl,-Bdynamic -ldl -lpthread
[100%] Linking CXX executable csysdig
cd /tmp/sysdig-20210915-64838-1rj5rj8/sysdig-0.27.1/build/userspace/sysdig && /home/linuxbrew/.linuxbrew/Cellar/cmake/3.21.2/bin/cmake -E cmake_link_script CMakeFiles/csysdig.dir/link.txt --verbose=1
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/shims/linux/super/g++-5  -Wall -ggdb   -std=c++0x -O3 -fno-strict-aliasing -DNDEBUG -rdynamic CMakeFiles/csysdig.dir/fields_info.cpp.o CMakeFiles/csysdig.dir/csysdig.cpp.o -o csysdig  ../libsinsp/libsinsp.a -lncurses -lform ../libscap/libscap.a -lelf -lz -lcurl -ljsoncpp -ltbb -lcares -lgrpc++_unsecure -lgrpc_unsecure -lprotobuf -lcares -lgrpc++_unsecure -lgrpc_unsecure -lprotobuf -ljq -Wl,-Bstatic -lb64 -Wl,-Bdynamic -lrt -lanl -lssl -lcrypto -Wl,-Bstatic -lluajit -Wl,-Bdynamic -ldl -lpthread
/home/linuxbrew/.linuxbrew/bin/ld: ../libsinsp/libsinsp.a(cri.grpc.pb.cc.o): undefined reference to symbol '_ZN4absl12lts_202103245MutexD1Ev'
/home/linuxbrew/.linuxbrew/bin/ld: /home/linuxbrew/.linuxbrew/lib/libabsl_synchronization.so.2103.0.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

carlocab · 2021-10-13T15:33:07Z

Let's see if building with gcc helps.

carlocab · 2021-10-13T16:31:33Z

bear is an issue in nlohmann-json: nlohmann/json#3070, fix in progress at nlohmann/json#3073

apache-arrow probably needs to be built with LLVM 12.

carlocab · 2021-10-13T16:36:16Z

Building sysdig with GCC-11 did not help:

/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/shims/linux/super/g++-11  -Wall -ggdb   -std=c++0x -O3 -fno-strict-aliasing -DNDEBUG -rdynamic CMakeFiles/csysdig.dir/fields_info.cpp.o CMakeFiles/csysdig.dir/csysdig.cpp.o -o csysdig  ../libsinsp/libsinsp.a -lncurses -lform ../libscap/libscap.a -lelf -lz -lcurl -ljsoncpp -ltbb -lcares -lgrpc++_unsecure -lgrpc_unsecure -lprotobuf -lcares -lgrpc++_unsecure -lgrpc_unsecure -lprotobuf -ljq -Wl,-Bstatic -lb64 -Wl,-Bdynamic -lrt -lanl -lssl -lcrypto -Wl,-Bstatic -lluajit -Wl,-Bdynamic -ldl -lpthread 
/home/linuxbrew/.linuxbrew/bin/ld: ../libsinsp/libsinsp.a(cri.grpc.pb.cc.o): undefined reference to symbol '_ZN4absl12lts_202103245MutexD1Ev'
/home/linuxbrew/.linuxbrew/bin/ld: /home/linuxbrew/.linuxbrew/lib/libabsl_synchronization.so.2103.0.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
userspace/sysdig/CMakeFiles/csysdig.dir/build.make:136: recipe for target 'userspace/sysdig/csysdig' failed
make[2]: *** [userspace/sysdig/csysdig] Error 1

danielnachun · 2021-10-13T18:56:32Z

It looks like this is really a problem with abseil (and one we've seen before) where it can't find its own libraries. It's possibly related to the fact that the abseil libraries use $ORIGIN/../lib:/home/linuxbrew/.linuxbrew/Cellar/abseil/20210324.2/lib:/home/linuxbrew/.linuxbrew/lib as their RPATH, and probably need /home/linuxbrew/.linuxbrew/opt/abseil/lib as well.

carlocab · 2021-10-14T02:07:35Z

It's possibly related to the fact that the abseil libraries use $ORIGIN/../lib:/home/linuxbrew/.linuxbrew/Cellar/abseil/20210324.2/lib:/home/linuxbrew/.linuxbrew/lib as their RPATH, and probably need /home/linuxbrew/.linuxbrew/opt/abseil/lib as well.

Why does it need the opt path too? Don't

/home/linuxbrew/.linuxbrew/opt/abseil/lib

and

/home/linuxbrew/.linuxbrew/Cellar/abseil/20210324.2/lib

point to the same place?

carlocab · 2021-10-16T13:22:05Z

bear build failure should be fixed in #87353.

carlocab · 2021-10-16T13:25:32Z

It looks like this is really a problem with abseil (and one we've seen before) where it can't find its own libraries. It's possibly related to the fact that the abseil libraries use $ORIGIN/../lib:/home/linuxbrew/.linuxbrew/Cellar/abseil/20210324.2/lib:/home/linuxbrew/.linuxbrew/lib as their RPATH, and probably need /home/linuxbrew/.linuxbrew/opt/abseil/lib as well.

Also, sysdig doesn't have an abseil dependency... Does it?

Edit: Oh, wait, missed the Linux-only deps.

Also make `llvm` a test dependency on Mojave because of a `brew` bug.

Also, switch to LLVM 12, as this fails to build with LLVM 13.

Also, deparallelise the build on Linux and make dependencies explicit, and compile using C++17 in order to use `abseil`.

carlocab · 2021-10-18T10:25:42Z

sysdig has had 15 downloads on Linux in the past year. grpc has ~8,000 monthly installs on macOS. I don't think we can justify further delaying shipping an update to that many users to avoid breaking so few users. I'm going to merge this, possibly after making sysdig depends_on :macos.

Objections, @Homebrew/core?

MikeMcQuaid · 2021-10-18T15:06:42Z

@carlocab

danielnachun · 2021-10-18T16:39:55Z

Can we just make sure to make a comment as to why it's disabled on Linux? Disabling a formula on Linux because it doesn't build is different from disabling it because it is fundamentally incompatible.

As an aside perhaps we will eventually need some separate DSL for "skip Linux because it doesn't build" versus "skip Linux because it is incompatible".

cho-m · 2021-10-18T16:41:07Z

gRPC 1.41.0 has been available if we want to jump to that, or we can finish this and then open new PR: https://github.com/grpc/grpc/releases/tag/v1.41.0

Sysdig upstream issue: draios/sysdig#1778.

EDIT:

e.g. Patch from that issue https://github.com/hhoffstaette/portage/blob/master/dev-util/sysdig/files/0.27.1-grpc-absl-sync.patch
Patch from Arch https://github.com/archlinux/svntogit-community/blob/packages/sysdig/trunk/0.27.1-grpc-absl-sync.patch

danielnachun · 2021-10-18T16:45:11Z

Maybe try ENV.append "LDFLAGS", "-labsl_synchronization"? Sorry I should have suggested this earlier.

cho-m · 2021-10-18T16:51:14Z

@danielnachun ArchLinux does something similar by patching CMake build scripts. Since they successfully updated to grpc 1.41.0 without breaking sysdig, we can borrow their fix and add comment for upstream issue (draios/sysdig#1778):

https://github.com/archlinux/svntogit-community/blob/packages/sysdig/trunk/0.27.1-grpc-absl-sync.patch

diff -rup sysdig-0.27.1/CMakeLists.txt sysdig-0.27.1-grpc-1.41/CMakeLists.txt
--- sysdig-0.27.1/CMakeLists.txt	2020-09-30 16:21:52.000000000 +0200
+++ sysdig-0.27.1-grpc-1.41/CMakeLists.txt	2021-10-08 23:46:51.058659662 +0200
@@ -552,6 +552,7 @@ if(NOT WIN32 AND NOT APPLE)
 			else()
 				message(FATAL_ERROR "Couldn't find system grpc")
 			endif()
+			find_library(ABSL_SYNC_LIB NAMES absl_synchronization)
 			find_program(GRPC_CPP_PLUGIN grpc_cpp_plugin)
 			if(NOT GRPC_CPP_PLUGIN)
 				message(FATAL_ERROR "System grpc_cpp_plugin not found")
diff -rup sysdig-0.27.1/userspace/libsinsp/CMakeLists.txt sysdig-0.27.1-grpc-1.41/userspace/libsinsp/CMakeLists.txt
--- sysdig-0.27.1/userspace/libsinsp/CMakeLists.txt	2020-09-30 16:21:52.000000000 +0200
+++ sysdig-0.27.1-grpc-1.41/userspace/libsinsp/CMakeLists.txt	2021-10-08 23:46:35.785353019 +0200
@@ -209,6 +209,7 @@ if(NOT WIN32)
 			target_link_libraries(sinsp
 				"${GRPCPP_LIB}"
 				"${GRPC_LIB}"
+				"${ABSL_SYNC_LIB}"
 				"${PROTOBUF_LIB}"
 				"${CARES_LIB}"
 				"${JQ_LIB}"

EDIT: It is also fine to merge this PR, and do follow-up PRs for fixing sysdig and updating to grpc 1.41.0

carlocab

Merging this so that users can brew extract --version=1.40.0 grpc if they need to, plus there might be additional issues with 1.41.0.

BrewTestBot · 2021-10-18T17:06:25Z

@carlocab has triggered a merge.

BrewTestBot added the bump-formula-pr PR was created using `brew bump-formula-pr` label Sep 10, 2021

cho-m added the revision bumps needed Reverse dependencies need to have their revision incremented in the same PR label Sep 10, 2021

carlocab added CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. CI-long-timeout [DEPRECATED] Use longer GitHub Actions CI timeout. and removed revision bumps needed Reverse dependencies need to have their revision incremented in the same PR labels Sep 11, 2021

carlocab added build failure CI fails while building the software linux Linux is specifically affected labels Sep 11, 2021

github-actions bot added the stale No recent activity label Sep 14, 2021

chenrui333 added in progress Stale bot should stay away and removed stale No recent activity labels Sep 15, 2021

carlocab force-pushed the bump-grpc-1.40.0 branch from 6331344 to cf7d6a3 Compare October 13, 2021 15:32

carlocab force-pushed the bump-grpc-1.40.0 branch 2 times, most recently from 14d650c to 28944bf Compare October 16, 2021 17:27

bevanjkay and others added 5 commits October 17, 2021 17:31

grpc 1.40.0

64df43b

Also make `llvm` a test dependency on Mojave because of a `brew` bug.

apache-arrow-glib: revision bump (grpc 1.40.0)

912a61d

apache-arrow: revision bump (grpc 1.40.0)

3a40c27

Also, switch to LLVM 12, as this fails to build with LLVM 13.

bear: revision bump (grpc 1.40.0)

3887925

mavsdk: revision bump (grpc 1.40.0)

63aab48

carlocab force-pushed the bump-grpc-1.40.0 branch 2 times, most recently from ba2799d to b0a2630 Compare October 17, 2021 09:36

carlocab force-pushed the bump-grpc-1.40.0 branch 2 times, most recently from a9453b9 to f591c72 Compare October 17, 2021 13:57

sysdig: revision bump (grpc 1.40.0)

bcf74f8

Also, deparallelise the build on Linux and make dependencies explicit, and compile using C++17 in order to use `abseil`.

carlocab force-pushed the bump-grpc-1.40.0 branch from f591c72 to bcf74f8 Compare October 18, 2021 07:15

carlocab approved these changes Oct 18, 2021

View reviewed changes

BrewTestBot closed this in 5c4eb24 Oct 18, 2021

carlocab mentioned this pull request Oct 20, 2021

sysdig: build on Linux #87581

Merged

6 tasks

github-actions bot added the outdated PR was locked due to age label Nov 18, 2021

github-actions bot locked as resolved and limited conversation to collaborators Nov 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grpc 1.40.0 #84990

grpc 1.40.0 #84990

bevanjkay commented Sep 10, 2021

carlocab commented Sep 11, 2021

github-actions bot commented Sep 14, 2021

iMichka commented Sep 16, 2021

carlocab commented Oct 13, 2021

carlocab commented Oct 13, 2021 •

edited

Loading

carlocab commented Oct 13, 2021

danielnachun commented Oct 13, 2021

carlocab commented Oct 14, 2021

carlocab commented Oct 16, 2021

carlocab commented Oct 16, 2021 •

edited

Loading

carlocab commented Oct 18, 2021

MikeMcQuaid commented Oct 18, 2021

danielnachun commented Oct 18, 2021

cho-m commented Oct 18, 2021 •

edited

Loading

danielnachun commented Oct 18, 2021

cho-m commented Oct 18, 2021 •

edited

Loading

carlocab left a comment

BrewTestBot commented Oct 18, 2021

grpc 1.40.0 #84990

grpc 1.40.0 #84990

Conversation

bevanjkay commented Sep 10, 2021

carlocab commented Sep 11, 2021

github-actions bot commented Sep 14, 2021

iMichka commented Sep 16, 2021

carlocab commented Oct 13, 2021

carlocab commented Oct 13, 2021 • edited Loading

carlocab commented Oct 13, 2021

danielnachun commented Oct 13, 2021

carlocab commented Oct 14, 2021

carlocab commented Oct 16, 2021

carlocab commented Oct 16, 2021 • edited Loading

carlocab commented Oct 18, 2021

MikeMcQuaid commented Oct 18, 2021

danielnachun commented Oct 18, 2021

cho-m commented Oct 18, 2021 • edited Loading

danielnachun commented Oct 18, 2021

cho-m commented Oct 18, 2021 • edited Loading

carlocab left a comment

Choose a reason for hiding this comment

BrewTestBot commented Oct 18, 2021

carlocab commented Oct 13, 2021 •

edited

Loading

carlocab commented Oct 16, 2021 •

edited

Loading

cho-m commented Oct 18, 2021 •

edited

Loading

cho-m commented Oct 18, 2021 •

edited

Loading