You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Customer request: We use multiple languages for clients and cannot implement detokenization in each one. Need to have server-side detokenization support.
The text was updated successfully, but these errors were encountered:
Just chiming in here that the customer quoted is us :). The main challenger is that we have clients in multiple languages that don’t always have tokenizer implementations readily available. Every other prominent model server does detokenization, hence the request.
Doesn’t hurt that there are so many CPU cores on the TPU VMs that are mostly idle during inference anyway.
The text was updated successfully, but these errors were encountered: