Keras model exported as TensorFlow model doesn't work with TensorFlow serving

[ankushagarwal]

I am using the keras model as defined in this tutorial: https://github.com/hamelsmu/Seq2Seq_Tutorial/blob/master/notebooks/Tutorial.ipynb

I exported the encoder model using extract_encoder_model and exported it as a Tensorflow model. When used with TensorFlow serving, I get the following error

AbortionError: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="Expected multiples argument to be a vector of length 3 but got length 2
[[Node: Encoder-Last-GRU_1/Tile = Tile[T=DT_FLOAT, Tmultiples=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Encoder-Last-GRU_1/ExpandDims, Encoder-Last-GRU_1/Tile/multiples)]]")

[jlewi]

/cc @lakshmanok @nfiedel @rhaertel80

[jlewi]

Could this be an issue with different versions of TF used for training and serving?

[ankushagarwal]

It's unlikely. I ensured that both training and serving were done using tensorflow version 1.5.x

I did have a different error when I trained using TF 1.4 and served using TF 1.5.

[bkungfoo]

The TF serving images in Kubeflow are all 1.4 currently. I haven't tried creating a saved model with Keras in TF 1.5, but from my experience, creating an Estimator with TF 1.5 caused failures in TF serving 1.4. Not sure if Keras suffers from a similar issue with compatibility, although your case is the reverse. Did you try 1.4 for both training and serving?

I was able to run this one successfully with TF 1.4 training and serving.
https://github.com/google-aai/tf-serving-k8s-tutorial/blob/master/keras_resnet_serving_solution.ipynb

Here's the client to call the server:
https://github.com/google-aai/tf-serving-k8s-tutorial/blob/master/client/resnet_client.py

[bkungfoo]

The notebook link above is broken after a github update. I have an updated notebook here which includes not only creating servable models, but also some guidance on unit testing the input-output API: https://github.com/google-aai/tf-serving-k8s-tutorial/blob/master/keras_training_to_serving_solution.ipynb

[bkungfoo]

Created an issue: #48

[texasmichelle]

I don't think we ever solved this, did we? Have there been any updates that would make this straightforward?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.