Building from source failed on Python 3.10

[KumaTea]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): CentOS 7
  • manylinux2014 docker image from pypa / manylinux
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Raspberry Pi 4 B
• TensorFlow installed from (source or binary): source
• TensorFlow version: 2.6.0
• Python version: 3.10.0rc1
• Installed using virtualenv? pip? conda?:
• Bazel version (if compiling from source): 3.7.2
• GCC/Compiler version (if compiling from source): 8.3.1
• CUDA/cuDNN version:
• GPU model and memory:

Describe the problem

distutils is deprecated

The distutils is deprecated in Python 3.10. In python_configure.bzl, the deprecation message will be printed prior to the include path, causing error on return.

ABC is removed

(detailed logs below)

According to the collections package:

Deprecated since version 3.3, will be removed in version 3.10: Moved Collections Abstract Base Classes to the collections.abc module. For backwards compatibility, they continue to be visible in this module through Python 3.9.

_message.so from protobuf seems to cause this problem.

    from google.protobuf.pyext import _message
AttributeError: module 'collections' has no attribute 'MutableSequence'

Any other info / logs

[root@129a8b5b98ad tensorflow]# BAZEL_LINKLIBS=-l%:libstdc++.a bazel build --config=noaws --config=nogcp --config=nohdfs --config=nonccl //tensorflow/tools/pip_package:build_pip_package
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=91
INFO: Reading rc options for 'build' from /opt/tf/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /opt/tf/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --java_toolchain=@tf_toolchains//toolchains/java:tf_java_toolchain --host_java_toolchain=@tf_toolchains//toolchains/java:tf_java_toolchain --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true
INFO: Reading rc options for 'build' from /opt/tf/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/opt/python/cp310-cp310/bin/python3.10 --action_env PYTHON_LIB_PATH=/opt/python/cp310-cp310/lib/python3.10/site-packages --python_path=/opt/python/cp310-cp310/bin/python3.10
INFO: Found applicable config definition build:short_logs in file /opt/tf/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /opt/tf/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:noaws in file /opt/tf/tensorflow/.bazelrc: --define=no_aws_support=true
INFO: Found applicable config definition build:nogcp in file /opt/tf/tensorflow/.bazelrc: --define=no_gcp_support=true
INFO: Found applicable config definition build:nohdfs in file /opt/tf/tensorflow/.bazelrc: --define=no_hdfs_support=true
INFO: Found applicable config definition build:nonccl in file /opt/tf/tensorflow/.bazelrc: --define=no_nccl_support=true
INFO: Found applicable config definition build:linux in file /opt/tf/tensorflow/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=dynamic_kernels --distinct_host_configuration=false
INFO: Found applicable config definition build:dynamic_kernels in file /opt/tf/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
DEBUG: /root/.cache/bazel/_bazel_root/91a00cca6722ee1c07e7cb452ed32c40/external/tf_runtime/third_party/cuda/dependencies.bzl:51:10: The following command will download NVIDIA proprietary software. By using the software you agree to comply with the terms of the license agreement that accompanies the software. If you do not agree to the terms of the license agreement, do not use the software.
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /opt/tf/tensorflow/tensorflow/python/keras/api/BUILD:133:19: Executing genrule //tensorflow/python/keras/api:keras_python_api_gen_compat_v1 failed (Exit 1): bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/91a00cca6722ee1c07e7cb452ed32c40/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/tensorflow/python/keras/api/create_tensorflow.python_api_keras_python_api_gen_compat_v1.runfiles/org_tensorflow/tensorflow/python/tools/api/generator/create_python_api.py", line 26, in <module>
    from tensorflow.python.tools.api.generator import doc_srcs
  File "/root/.cache/bazel/_bazel_root/91a00cca6722ee1c07e7cb452ed32c40/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/tensorflow/python/keras/api/create_tensorflow.python_api_keras_python_api_gen_compat_v1.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 40, in <module>
    from tensorflow.python.eager import context
  File "/root/.cache/bazel/_bazel_root/91a00cca6722ee1c07e7cb452ed32c40/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/tensorflow/python/keras/api/create_tensorflow.python_api_keras_python_api_gen_compat_v1.runfiles/org_tensorflow/tensorflow/python/eager/context.py", line 32, in <module>
    from tensorflow.core.framework import function_pb2
  File "/root/.cache/bazel/_bazel_root/91a00cca6722ee1c07e7cb452ed32c40/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/tensorflow/python/keras/api/create_tensorflow.python_api_keras_python_api_gen_compat_v1.runfiles/org_tensorflow/tensorflow/core/framework/function_pb2.py", line 7, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/root/.cache/bazel/_bazel_root/91a00cca6722ee1c07e7cb452ed32c40/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/tensorflow/python/keras/api/create_tensorflow.python_api_keras_python_api_gen_compat_v1.runfiles/com_google_protobuf/python/google/protobuf/descriptor.py", line 47, in <module>
    from google.protobuf.pyext import _message
AttributeError: module 'collections' has no attribute 'MutableSequence'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /opt/tf/tensorflow/tensorflow/python/tools/BUILD:81:10 Executing genrule //tensorflow/python/keras/api:keras_python_api_gen_compat_v1 failed (Exit 1): bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)
INFO: Elapsed time: 4.857s, Critical Path: 2.92s
INFO: 5 processes: 5 internal.
FAILED: Build did NOT complete successfully
[root@129a8b5b98ad tensorflow]#

[sushreebarsa]

@KumaTea Could you please have a look at the similar issue and link.Please let us know if it helps?Thank you!

[KumaTea]

@KumaTea Could you please have a look at the similar issue and link.Please let us know if it helps?Thank you!

Hi, thank you for your reply!

I've checked the links, but it seems irrelevant to typed_ast, setuptools or mypy. The Python environment should also be okay, because it was pulled from pypa / manylinux's manylinux2014 docker image.

I still believe the errors were caused by deprecated imports.

Thank you!

[KumaTea]

For the first error I've created a pull request (#51782) with a minor fix, and Python 3.6 - 3.10 compabilities have been tested.

[yongtang]

Added a PR #51865 to resolve another distutils related issue.

[mihaimaruseac]

We don't yet support py3.10 but now given that it has a final release we can start on the roadmap to support it by next release.

Depending on our dependencies, plan is to start supporting python 3.10 in tf-nightly first and then in the TF 2.8 release.

[sushreebarsa]

@KumaTea Please see the discussion on TF Forum .Thanks!

[KumaTea]

@KumaTea Please see the discussion on TF Forum .Thanks!

Hi, I've check the thread and will wait for official support. Thank you!

[KumaTea]

With this commit, the second error which might be only caused by protobuf was fixed.

I've successfully built TensorFlow 2.7.0 for Python 3.10, so I think it's completely compatible now.

(Except for these dependencies: h5py and tensorflow-io)

[sushreebarsa]

@KumaTea Thank you for the update!
Could you please see TF Forum for more details and move this issue to closed status if it is resolved ?Thank you!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No