Pip package for tensorflow serving does not support GPU

[dakshvar22]

i installed tensorflow serving pip package which comes with tensorflow 1.4 as a dependency. But it seems like the tensorflow package inside does not support GPU. Is there an alternative pip package?
I even installed tensorflow(GPU version) separately and then installed tensorflow serving pip package, but that seems to overwrite the GPU version. Any way by which I can stop tensorflow serving pip package from reinstalling tensorflow?

Thanks

[kirilg]

That's interesting, so 'tensorflow-gpu' overwrites the 'tensorflow' PIP package?
The TensorFlow Serving pip package (tensorflow-serving-api) is a very thin API library that just includes the protos of the PredictionService, so it theory it shouldn't depend on GPU. What you can try doing is installing 'tensorflow-serving-api' (which will require tensorflow), and then install tensorflow-gpu which will overwrite tensorflow. I believe 'tensorflow-serving-api' should continue working, it will just get your newly installed GPU tensorflow.

[jorgemf]

@dakshvar22 as far as I know the open source of tensorflow serving does not compile with GPU support, there are issues.

[ps48]

@dakshvar22 Even I am facing the same issue. Still didn't find any to use serving pip installable with GPU. Any updates on this?

[ps48]

The error is not in Serving pip API as it is only used in client side. Rather the issue is on the Apt-get Install for tensorflow-model-server. Issue #549 comment by @kirilg mentions apt-get doesn't support GPU as of now. I think we can close this issue.

[jorgemf]

@ps48 the tensorflow serving pip package doesn't support GPU because before it couldn't be compiled easily with GPU support before. I think recent versions can be compile with GPU support with the command you saw if the issue #549, but the package hasn't been compiled with this options yet. So either you use a package without GPU or you compile it yourself. It is not related with apt-get.

[ps48]

@jorgemf thanks for the pormpt reply. As far as i see this means serving-api as well as tensorflow-model-server have GPU support and just need to be compiled locally. Please correct me if I'm wrong.

[jorgemf]

@ps48 The serving-api doesn't need GPU support as it is only to make requests to the server. Tensorflow-model-server does need to be compiled with GPU support as the other issue said. This might help: https://gist.github.com/jorgemf/c791841f769bff96718fd54bbdecfd4e