build: Explicit protobuf build version; consistent build/setup deps (#…

…4472)

build: explicit protobuf build version; consistent build/setup deps

Right now if one downloads `feast-0.40.1-py2.py3-none-any.whl` from
PyPi it contains:
```
$ grep 'Protobuf Python Version' feast/protos/feast/registry/RegistryServer_pb2.py
```
Which is outside
```
$ grep 'protobuf<' feast-0.40.1.dist-info/METADATA
Requires-Dist: protobuf<5.0.0,>=4.24.0
```
Leading to runtime errors (#4437).  This was mitigated by #4438.  This
change tightens this up further by:
 * Deleting the Makefile command that was trying to do this
 unsuccessfully.
 * Aligns the setup/build requirements
 * Sets the version of protobuf to match the *minimum* of the range.
 There is no guarantee that protos generated by `4.X` will work with `4.(X-1)`.

Signed-off-by: Chris Burroughs <[email protected]>

.github/workflows/build_wheels.yml

@@ -97,7 +97,6 @@ jobs:
         # There's a `git restore` in here because `make install-go-ci-dependencies` is actually messing up go.mod & go.sum.
         run: |
           pip install -U pip setuptools wheel twine
-          make install-protoc-dependencies
           make build-ui
           git status
           git restore go.mod go.sum

Makefile

@@ -395,9 +395,6 @@ test-trino-plugin-locally:
 kill-trino-locally:
 	cd ${ROOT_DIR}; docker stop trino
 
-install-protoc-dependencies:
-	pip install --ignore-installed protobuf==4.24.0 "grpcio-tools>=1.56.2,<2" mypy-protobuf==3.1.0
-
 # Docker
 
 build-docker: build-feature-server-python-aws-docker build-feature-transformation-server-docker build-feature-server-java-docker

environment-setup.md

@@ -13,11 +13,10 @@ pip install cryptography -U
 conda install protobuf
 conda install pymssql
 pip install -e ".[dev]"
-make install-protoc-dependencies PYTHON=3.9
 make install-python-ci-dependencies PYTHON=3.9
 ```
 4. start the docker daemon
 5. run unit tests:
 ```bash
 make test-python-unit
-```
+```

pyproject.toml

@@ -1,13 +1,14 @@
 [build-system]
 requires = [
+  "grpcio-tools>=1.56.2,<2",
+  "grpcio>=1.56.2,<2",
+  "mypy-protobuf==3.1",
+  "protobuf==4.24.0",
+  "pybindgen==0.22.0",
   "setuptools>=60",
-  "wheel",
   "setuptools_scm>=6.2",
-  "grpcio",
-  "grpcio-tools>=1.47.0",
-  "mypy-protobuf==3.1",
-  "protobuf>=4.24.0,<5.0.0",
   "sphinx!=4.0.0",
+  "wheel",
 ]
 build-backend = "setuptools.build_meta"
 

setup.py

@@ -403,11 +403,12 @@ def run(self):
     entry_points={"console_scripts": ["feast=feast.cli:cli"]},
     use_scm_version=use_scm_version,
     setup_requires=[
-        "setuptools_scm",
-        "grpcio>=1.56.2,<2",
         "grpcio-tools>=1.56.2,<2",
-        "mypy-protobuf>=3.1",
+        "grpcio>=1.56.2,<2",
+        "mypy-protobuf==3.1",
+        "protobuf==4.24.0",
         "pybindgen==0.22.0",
+        "setuptools_scm>=6.2",
     ],
     cmdclass={
         "build_python_protos": BuildPythonProtosCommand,