Removing proto functionality from REST /predict endpoint [#803]

[axsaucedo]

This is the implementation to fix #803.

Once this is landed there will still be work to replicate across remaining endpoints (as per #607)

[lennon310]

The profiler result shows parsing time reduces for REST but not GRPC, is it expected?

[axsaucedo]

@lennon310 that is correct, I've replied in #803, we should be able to merge this soon

[lennon310]

Also I was wondering whether this change applies to the prepackaged server as well?

[axsaucedo]

@lennon310 yes, although the model servers will also be bumped a version when they are recompiled.

[axsaucedo]

Added a couple more tests. PR should be ready for review. @lennon310 could you provide a review as well given that you have been currently going through these pieces?

[lennon310]

This PR looks good to me.
Probably not related to this PR, but I was wondering why gRPC path should not go this way.
From my profiler commented in issue it shows in gRPC the input was first converted to datadef in SeldonMessage:

100    0.240    0.002    1.281    0.013 utils.py:234(array_to_grpc_datadef)
20100/100    0.038    0.000    1.041    0.010 utils.py:280(array_to_list_value)

then in predict function it is converted back to Numpy array:

100    0.000    0.000    2.781    0.028 utils.py:505(extract_request_parts)
100    0.001    0.000    2.778    0.028 utils.py:120(get_data_from_proto)
100    0.001    0.000    2.778    0.028 utils.py:147(grpc_datadef_to_array)

The round-trip conversion seems double the latency I guess?

[axsaucedo]

@lennon310 the conversion is necessary as in GRPC the request is received as Proto, and needs to be converted into a numpy array before it's passed to the python wrapper. You can access the raw proto through the predict_raw function, but if you want to access the data in a readable format you will eventually have to convert it. Similarly when sending the response, it's necessary to convert the output (which is often a numpy array) into a Proto - which in turn requires that conversion.

We're happy to take suggestions but if you have a look at the functions, it's basically converting the Proto directly to numpy array. If you run some alternative implementations of methods to convert numpy arrays to/from proto objects that are more efficient, we would certainly be keen to introduce them.