Failed to build //test/extensions/quic_listeners/quiche/* tests

[zihuaweng]

Hi team, I have been working on building a release version of envoy 1.17.0 with --define=boringssl=fips and running all the tests using a oracle linux 7 docker image. After solving many other issues, I have the last type of errors to fix which are related to the extension quiche. I am not sure which direction to go, could you please help me with these errors?

ERROR: /source/test/server/BUILD:270:14: Couldn't build file test/server/_objs/listener_manager_impl_quic_only_test/listener_manager_impl_quic_only_test.o: C++ compilation of rule '//test/server:listener_manager_impl_quic_only_test' failed (Exit 1): linux-sandbox failed: error executing command 

...

test/server/listener_manager_impl_quic_only_test.cc:4:10: fatal error: 'extensions/quic_listeners/quiche/quic_transport_socket_factory.h' file not found
#include "extensions/quic_listeners/quiche/quic_transport_socket_factory.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

...

ERROR: /bazel/b570b5ccd0454dc9af9f65ab1833764d/external/com_googlesource_quiche/BUILD.bazel:2091:17: Couldn't build file external/com_googlesource_quiche/_objs/quic_core_crypto_tls_handshake_lib/tls_connection.o: C++ compilation of rule '@com_googlesource_quiche//:quic_core_crypto_tls_handshake_lib' failed (Exit 1): linux-sandbox failed: error executing command 

bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/tls_connection.cc:121:5: error: cannot initialize a member subobject of type 'int (*)(SSL *, enum ssl_encryption_level_t, const uint8_t *, const uint8_t *, size_t)' (aka 'int (*)(ssl_st *, ssl_encryption_level_t, const unsigned char *, const unsigned char *, unsigned long)') with an lvalue of type 'int (SSL *, enum ssl_encryption_level_t, const SSL_CIPHER *, const uint8_t *, size_t)' (aka 'int (ssl_st *, ssl_encryption_level_t, const ssl_cipher_st *, const unsigned char *, unsigned long)'): type mismatch at 3rd parameter ('const uint8_t *' (aka 'const unsigned char *') vs 'const SSL_CIPHER *' (aka 'const ssl_cipher_st *'))
    TlsConnection::SetReadSecretCallback, TlsConnection::SetWriteSecretCallback,
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/tls_connection.cc:121:43: error: cannot initialize a member subobject of type 'int (*)(SSL *, enum ssl_encryption_level_t, const uint8_t *, size_t)' (aka 'int (*)(ssl_st *, ssl_encryption_level_t, const unsigned char *, unsigned long)') with an lvalue of type 'int (SSL *, enum ssl_encryption_level_t, const SSL_CIPHER *, const uint8_t *, size_t)' (aka 'int (ssl_st *, ssl_encryption_level_t, const ssl_cipher_st *, const unsigned char *, unsigned long)'): different number of parameters (4 vs 5)
    TlsConnection::SetReadSecretCallback, TlsConnection::SetWriteSecretCallback,
                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/tls_connection.cc:122:5: error: cannot initialize a member subobject of type 'int (*)(SSL *)' (aka 'int (*)(ssl_st *)') with an lvalue of type 'int (SSL *, enum ssl_encryption_level_t, const uint8_t *, size_t)' (aka 'int (ssl_st *, ssl_encryption_level_t, const unsigned char *, unsigned long)'): different number of parameters (1 vs 4)
    TlsConnection::WriteMessageCallback, TlsConnection::FlushFlightCallback,
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/tls_connection.cc:122:42: error: cannot initialize a member subobject of type 'int (*)(SSL *, enum ssl_encryption_level_t, uint8_t)' (aka 'int (*)(ssl_st *, ssl_encryption_level_t, unsigned char)') with an lvalue of type 'int (SSL *)' (aka 'int (ssl_st *)'): different number of parameters (3 vs 1)
    TlsConnection::WriteMessageCallback, TlsConnection::FlushFlightCallback,
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4 errors generated
...

bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/crypto_utils.cc:763:27: error: use of undeclared identifier 'ssl_early_data_quic_parameter_mismatch'; did you mean 'ssl_early_data_alpn_mismatch'?
    RETURN_STRING_LITERAL(ssl_early_data_quic_parameter_mismatch);
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                          ssl_early_data_alpn_mismatch
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/crypto_utils.cc:695:8: note: expanded from macro 'RETURN_STRING_LITERAL'
  case x:                        \
       ^
external/boringssl_fips/boringssl/include/openssl/ssl.h:3356:3: note: 'ssl_early_data_alpn_mismatch' declared here
  ssl_early_data_alpn_mismatch,
  ^
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/crypto_utils.cc:763:27: error: duplicate case value 'ssl_early_data_alpn_mismatch'
    RETURN_STRING_LITERAL(ssl_early_data_quic_parameter_mismatch);
                          ^
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/crypto_utils.cc:759:27: note: previous case defined here
    RETURN_STRING_LITERAL(ssl_early_data_alpn_mismatch);
                          ^
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/crypto/crypto_utils.cc:766:38: error: use of undeclared identifier 'ssl_early_data_reason_max_value'
  QUIC_BUG_IF(reason < 0 || reason > ssl_early_data_reason_max_value)
                                     ^
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/platform/api/quic_bug_tracker.h:11:21: note: expanded from macro 'QUIC_BUG_IF'
#define QUIC_BUG_IF QUIC_BUG_IF_IMPL
                    ^
3 errors generated

...

bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/tls_chlo_extractor.cc:321:9: error: cannot initialize a member subobject of type 'int (*)(SSL *, enum ssl_encryption_level_t, const uint8_t *, const uint8_t *, size_t)' (aka 'int (*)(ssl_st *, ssl_encryption_level_t, const unsigned char *, const unsigned char *, unsigned long)') with an lvalue of type 'int (SSL *, enum ssl_encryption_level_t, const SSL_CIPHER *, const uint8_t *, size_t)' (aka 'int (ssl_st *, ssl_encryption_level_t, const ssl_cipher_st *, const unsigned char *, unsigned long)'): type mismatch at 3rd parameter ('const uint8_t *' (aka 'const unsigned char *') vs 'const SSL_CIPHER *' (aka 'const ssl_cipher_st *'))
        TlsChloExtractor::SetReadSecretCallback,
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/tls_chlo_extractor.cc:322:9: error: cannot initialize a member subobject of type 'int (*)(SSL *, enum ssl_encryption_level_t, const uint8_t *, size_t)' (aka 'int (*)(ssl_st *, ssl_encryption_level_t, const unsigned char *, unsigned long)') with an lvalue of type 'int (SSL *, enum ssl_encryption_level_t, const SSL_CIPHER *, const uint8_t *, size_t)' (aka 'int (ssl_st *, ssl_encryption_level_t, const ssl_cipher_st *, const unsigned char *, unsigned long)'): different number of parameters (4 vs 5)
        TlsChloExtractor::SetWriteSecretCallback,
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/tls_chlo_extractor.cc:323:9: error: cannot initialize a member subobject of type 'int (*)(SSL *)' (aka 'int (*)(ssl_st *)') with an lvalue of type 'int (SSL *, enum ssl_encryption_level_t, const uint8_t *, size_t)' (aka 'int (ssl_st *, ssl_encryption_level_t, const unsigned char *, unsigned long)'): different number of parameters (1 vs 4)
        TlsChloExtractor::WriteMessageCallback,
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/tls_chlo_extractor.cc:324:9: error: cannot initialize a member subobject of type 'int (*)(SSL *, enum ssl_encryption_level_t, uint8_t)' (aka 'int (*)(ssl_st *, ssl_encryption_level_t, unsigned char)') with an lvalue of type 'int (SSL *)' (aka 'int (ssl_st *)'): different number of parameters (3 vs 1)
        TlsChloExtractor::FlushFlightCallback,
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4 errors generated
...

bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/tls_client_handshaker.cc:405:47: error: use of undeclared identifier 'SSL_CIPHER_get_value'; did you mean 'SSL_CIPHER_get_name'?
    crypto_negotiated_params_->cipher_suite = SSL_CIPHER_get_value(cipher);
                                              ^~~~~~~~~~~~~~~~~~~~
                                              SSL_CIPHER_get_name
external/boringssl_fips/boringssl/include/openssl/ssl.h:1301:28: note: 'SSL_CIPHER_get_name' declared here
OPENSSL_EXPORT const char *SSL_CIPHER_get_name(const SSL_CIPHER *cipher);
                           ^
bazel-out/k8-opt/bin/external/com_googlesource_quiche/quiche/quic/core/tls_client_handshaker.cc:405:47: error: assigning to 'uint16_t' (aka 'unsigned short') from incompatible type 'const char *'
    crypto_negotiated_params_->cipher_suite = SSL_CIPHER_get_value(cipher);
                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated
...



FAILED: Build did NOT complete successfully
//test/exe:build_id_test                                              NO STATUS
//test/exe:envoy_static_test                                          NO STATUS
//test/exe:pie_test                                                   NO STATUS
//test/exe:version_out_test                                           NO STATUS
//test/integration:hotrestart_test                                    NO STATUS
//test/integration:run_envoy_test                                     NO STATUS
//test/extensions/quic_listeners/quiche:active_quic_listener_config_test FAILED TO BUILD
//test/extensions/quic_listeners/quiche:active_quic_listener_test FAILED TO BUILD
//test/extensions/quic_listeners/quiche:envoy_quic_client_session_test FAILED TO BUILD
//test/extensions/quic_listeners/quiche:envoy_quic_client_stream_test FAILED TO BUILD
//test/extensions/quic_listeners/quiche:envoy_quic_dispatcher_test FAILED TO BUIL


Executed 782 out of 800 tests: 781 tests pass, 12 fail to build, 1 fails locally and 6 were skipped.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
FAILED: Build did NOT complete successfully
/source/bazel-testlogs /source
cp: cannot stat '': No such file or directory

The full log is here
https://gist.github.com/zihuaweng/377dea27a5f786a9bb9cd1ce330127e2

[dio]

cc. @lizan

[zihuaweng]

I found out QUICHE can't build against FIPS BoringSSL and would be skipped by adding tag filters referring to #7433

export BAZEL_BUILD_EXTRA_OPTIONS="--build_tag_filters=-nofips"
export BAZEL_EXTRA_TEST_OPTIONS="--test_tag_filters=-nofips"

But are there any performance differences if we diable QUICHE in envoy?

[alyssawilk]

There shouldn't be perf differences, one simply can't use HTTP/3

[zihuaweng]

Thanks for your response!