Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL Registry does not populate cache correctly #3430

Closed
cbarcroft opened this issue Jan 4, 2023 · 5 comments · Fixed by #3482
Closed

SQL Registry does not populate cache correctly #3430

cbarcroft opened this issue Jan 4, 2023 · 5 comments · Fixed by #3482

Comments

@cbarcroft
Copy link
Contributor

Expected Behavior

Using SQL registry, should be able to retrieve online features for an existing feature view using the Python SDK.

Current Behavior

FeatureStore.get_online_features() currently returns a FeatureviewNotFoundException, despite the feature view existing as evidenced by the output of FeatureStore.list_feature_views().

Steps to reproduce

  1. Configure feature store using an SQL registry
  2. Create a feature view
  3. Materialize data to online store for the feature view
  4. Attempt to retrieve the online features using FeatureStore.get_online_features()

Specifications

  • Version: 0.27.1
  • Platform: macOS Monterey 12.4
  • Python 3.9.16
  • SQL Registry, using postgres on an AWS AuroraDB instance

Possible Solution

This error seems to be specific to the SQL registry, as I have followed the exact same steps using a File registry and there is no issue. I can also circumvent the error by disabling the cache for the SQL registry, so it seems localized to the SQL registry cache itself. The feature views are successfully retrieved, but are rejected by this line because their spec does not have a project.

It appears that normally, the project is attached to the proto spec right before committing registry changes (like so, but the SQL registry does not do this and instead relies on each class's to_proto() output, which does not contain the project. This issue appears to extend to other types of objects as well (entities, etc.)

@cbarcroft cbarcroft changed the title SQL Registry fails to retrieve online features SQL Registry does not populate cache correctly Jan 5, 2023
@cbarcroft
Copy link
Contributor Author

Confirmed issue still exists in 0.28.0

@cbarcroft
Copy link
Contributor Author

Confirmed issue still exists in 0.29.0

@EXPEbdodla
Copy link
Contributor

We have the same issue and my debugging leads to the same solution as mentioned above.

@cbarcroft
Copy link
Contributor Author

Incase it's helpful for anyone, I've been using the following fix in my project for a couple of versions now (works in at least 0.27.1 through 0.29.0).

Replace this line with the following:

                    protos = []
                    for obj in objs:
                        proto = obj.to_proto()
                        if hasattr(proto, "spec"):
                            proto.spec.project = project
                        protos.append(proto)

                    registry_proto_field.extend(protos)

I can't guarantee it's totally correct, but I haven't had any problems with it thus far.

@adchia adchia linked a pull request Feb 5, 2023 that will close this issue
@EXPEbdodla
Copy link
Contributor

Thanks for the fix. Looking forward for the new release. Is there any timeline for the new release?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants