User build failing with "error: ‘__fxstat’ undeclared here (not in a function); did you mean ‘fstat’"

[marascio]

I am trying to build the user space tools on Arch linux, with GLIBC 2.33. I am getting the following errors when building:

nice cc -I. -I./../../../include -I/home/lrm/c/other/onload/src/include -I/home/lrm/c/other/onload/src/lib/transport/ip    -fPIC  -m64  -Werror -Wall -Wundef -Wpointer-arith -Wstrict-prototypes -Wnested-externs -Wno-stringop-truncation -Wno-format-truncation -Wimplicit-fallthrough=2 -Wno-array-bounds -Wno-stringop-overflow -Wno-deprecated-declarations -DTRANSPORT_CONFIG_OPT_HDR='<ci/internal/transport_config_opt_extra.h>' -include /home/lrm/c/other/onload/build/gnu_x86_64/cp_intf_ver.h -O2 -g $cflags $cppflags -c /home/lrm/c/other/onload/src/lib/transport/ip/syscall.c -o ci_ip_syscall.o
/home/lrm/c/other/onload/src/include/onload/declare_syscalls.h.tmpl:115:28: error: ‘__fxstat’ undeclared here (not in a function); did you mean ‘fstat’?
  115 | CI_MK_DECL(int           , __fxstat   , (int, int, struct stat *));
      |                            ^~~~~~~~
/home/lrm/c/other/onload/src/lib/transport/ip/syscall.c:26:62: note: in definition of macro ‘CI_MK_DECL’
   26 | #define CI_MK_DECL(ret, fn, args)  ret (*ci_sys_##fn) args = fn
      |                                                              ^~
/home/lrm/c/other/onload/src/include/onload/declare_syscalls.h.tmpl:120:28: error: ‘__fxstat64’ undeclared here (not in a function); did you mean ‘fstat64’?
  120 | CI_MK_DECL(int           , __fxstat64 , (int, int, struct stat64 *));
      |                            ^~~~~~~~~~
/home/lrm/c/other/onload/src/lib/transport/ip/syscall.c:26:62: note: in definition of macro ‘CI_MK_DECL’
   26 | #define CI_MK_DECL(ret, fn, args)  ret (*ci_sys_##fn) args = fn
      |                                                              ^~
make[3]: *** [/home/lrm/c/other/onload/mk/after.mk:151: ci_ip_syscall.o] Error 1

I wonder if this is related to this commit in GLIBC: bminor/glibc@8ed005d

[marascio]

In an effort to just get this to compile I made the following changes:

diff --git a/src/include/onload/syscall_unix.h b/src/include/onload/syscall_unix.h
index d9a2324..94b1922 100644
--- a/src/include/onload/syscall_unix.h
+++ b/src/include/onload/syscall_unix.h
@@ -43,6 +43,20 @@ extern int __ppoll_chk (struct pollfd *__fds, nfds_t __nfds,
 extern __sighandler_t bsd_signal(int signum, __sighandler_t handler);
 extern __sighandler_t sysv_signal(int signum, __sighandler_t handler);
 
+#ifndef _STAT_VER
+ #if defined (__aarch64__)
+  #define _STAT_VER 0
+ #elif defined (__x86_64__)
+  #define _STAT_VER 1
+ #else
+  #define _STAT_VER 3
+ #endif
+#endif
+
+struct stat64;
+extern int __fxstat (int vers, int fd, struct stat *);
+extern int __fxstat64 (int vers, int fd, struct stat64 *);
+
 
 /*! Generate declarations of pointers to the system calls */
 #define CI_MK_DECL(ret,fn,args)  extern ret (*ci_sys_##fn) args CI_HV

I've no idea if these "correct", but it does now build and the unit tests pass.

Running tests
make: Entering directory '/home/lrm/c/other/onload/build/gnu_x86_64/tests/onload/oof'
/usr/bin/timeout 240 prove --exec ' ' \
 "./oof_test sanity" "./oof_test multicast_sanity" "./oof_test namespace_sanity" "./oof_test namespace_macvlan_move" 
./oof_test sanity .................. ok     
./oof_test multicast_sanity ........ ok   
./oof_test namespace_sanity ........ ok     
./oof_test namespace_macvlan_move .. ok     
All tests successful.
Files=4, Tests=158,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.00 cusr  0.01 csys =  0.04 CPU)
Result: PASS
make: Leaving directory '/home/lrm/c/other/onload/build/gnu_x86_64/tests/onload/oof'
make: Entering directory '/home/lrm/c/other/onload/build/gnu_x86_64/tests/onload/cplane_unit'
/usr/bin/timeout 600 prove -j2 --merge --exec '' \
 ./test_route ./test_route_expire ./test_arp_expire ./test_route_stress ./test_teambond ./test_namespace ./test_service_dnat 
./test_route ......... ok                                               
./test_route_expire .. ok                                               
./test_arp_expire .... ok                                               
./test_route_stress .. ok                                               
./test_namespace ..... ok                                               
./test_service_dnat .. ok                                               
./test_teambond ...... ok   
All tests successful.

I would imagine the changes in the diff would need to be conditioned on GLIBC version or something else so as not to always be in effect. I'd be happy to prepare a PR if what I'm doing here isn't totally insane, just let me know.

[ol-alexandra]

Louis, thank you very much for your report. I've created an internal Xilinx issue for this: ON-13062. I can reproduce it with Ubuntu 21.04 preview, and I'll fix it soon.

[ol-alexandra]

Fixed by 5e5201b. And thank you again! Your patch was not the right way to handle it, but it explained me everything about this libc change.

[haelix888]

@ol-alexandra you said you tried building against Ubuntu 21.04, which contains Linux kernel 5.11. Does this mean this kernel version is "known to work" all the way? What is the way to track which is the latest supported linux version for the package? Thanks.

[ol-alexandra]

Yes, the master branch of this repository it properly tested with Ubuntu 21.04, so it works with Linux kernel 5.11.
One of Onload developers also mentioned that it works with linux-5.12, but I did not try it myself.

I'm afraid there is no reasonable way to track the latest supported kernel. Usually I add such support as soon as the kernel is available from one of the major distros: Fedora, Debian (via Debian testing), Ubuntu. Unfortunately sometimes it takes a lot of effort and time to fix Onload for a new kernel.

Probably I should update README.md when I add new kernel support...

[ol-alexandra]

@haelix888 I've updated README.md with the linux kernel version. I'll try to remember doing this in future.