[Python] Remove no longer used serialize/deserialize PyArrow C++ code

[jorisvandenbossche]

We deprecated and removed the pyarrow.serialization functionality (#29705 / #34926). But that PR only removed the cython bindings, but not the underlying C++ code in python/pyarrow/src/arrow/python/(de)serialize.h/cc.

Those C++ APIs were exposed using ARROW_PYTHON_EXPORT, so should they also first be deprecated before we remove them?

https://github.com/apache/arrow/blob/dea09ec9c0dd0d6f68b67101dc520ac737eac277/python/pyarrow/src/arrow/python/serialize.h
https://github.com/apache/arrow/blob/dea09ec9c0dd0d6f68b67101dc520ac737eac277/python/pyarrow/src/arrow/python/deserialize.h

• [Python] Deprecate the no longer used serialize/deserialize Pyarrow C++ functions #44063

[pitrou]

Perhaps a Github search would allow finding out whether those APIs are being used by third-party projects? ARROW_PYTHON_EXPORT mostly means the given symbols are exposed in libarrow_python.dll which allows using them from Cython.

[jorisvandenbossche]

The problem for a GitHub search is that a name like SerializeObject is quite generic. Although when limiting the search to cython, I think I only see occurrences that are copies from the pyarrow source code. And the same for the other exported names.

So based on that limited github search, those APIs are seemingly not used (publicly).

It's not that the code gives any maintenance overhead (since it is not tested anymore, and if we leave it alone like in PRs that triggered this issue), so I am also fine with just keeping it a few months longer with a ARROW_DEPRECATED decoration for at least one release. But directly removing is also fine with me.

[pitrou]

We can definitely annotate those functions with ARROW_DEPRECATED and remove them in one or two versions.