Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure TopKEncoder has correct outputs when model is saved #1225

Merged

Conversation

oliverholworthy
Copy link
Member

@oliverholworthy oliverholworthy commented Nov 8, 2023

Fixes #1220

Goals ⚽

Ensure top-k encoder model outputs are correct when model is saved.

Implementation Details 🚧

Removing output_names in BaseModel.compile when we have a TopKOutput. This enables keras to figure out the output names correctly.

We currently set the output name based on the name of the model output. However, the BruteForce TopKLayer outputs a 2-tuple of scores and ids. Setting model.output_names to ["top_k_output"] keras writes out the model with only one output with this name. The relevant part in the keras code (2.12) is here

Before

The given SavedModel SignatureDef contains the following input(s):
  inputs['category-list__offsets'] tensor_info:
      dtype: DT_INT32
      shape: (-1)
      name: serving_default_category-list__offsets:0
  inputs['category-list__values'] tensor_info:
      dtype: DT_INT64
      shape: (-1)
      name: serving_default_category-list__values:0
  inputs['dayofweek-first'] tensor_info:
      dtype: DT_INT64
      shape: (-1)
      name: serving_default_dayofweek-first:0
  inputs['item_id-list__offsets'] tensor_info:
      dtype: DT_INT32
      shape: (-1)
      name: serving_default_item_id-list__offsets:0
  inputs['item_id-list__values'] tensor_info:
      dtype: DT_INT64
      shape: (-1)
      name: serving_default_item_id-list__values:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['item_id-list/top_k_output'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 100)
      name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict

After

The given SavedModel SignatureDef contains the following input(s):
  inputs['category-list__offsets'] tensor_info:
      dtype: DT_INT32
      shape: (-1)
      name: serving_default_category-list__offsets:0
  inputs['category-list__values'] tensor_info:
      dtype: DT_INT64
      shape: (-1)
      name: serving_default_category-list__values:0
  inputs['dayofweek-first'] tensor_info:
      dtype: DT_INT64
      shape: (-1)
      name: serving_default_dayofweek-first:0
  inputs['item_id-list__offsets'] tensor_info:
      dtype: DT_INT32
      shape: (-1)
      name: serving_default_item_id-list__offsets:0
  inputs['item_id-list__values'] tensor_info:
      dtype: DT_INT64
      shape: (-1)
      name: serving_default_item_id-list__values:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['identifiers'] tensor_info:
      dtype: DT_INT32
      shape: (-1, 100)
      name: StatefulPartitionedCall:0
  outputs['scores'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 100)
      name: StatefulPartitionedCall:1
Method name is: tensorflow/serving/predict

Testing Details 🔍

Adds assertion for expected output signature in test of topk encoder

@oliverholworthy oliverholworthy self-assigned this Nov 8, 2023
Copy link

github-actions bot commented Nov 8, 2023

Documentation preview

https://nvidia-merlin.github.io/models/review/pr-1225

@oliverholworthy oliverholworthy added the bug Something isn't working label Nov 8, 2023
@oliverholworthy oliverholworthy merged commit 80b086f into NVIDIA-Merlin:main Nov 10, 2023
40 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] when we serve topK model on Triton it only returns scores
2 participants