-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mobile: Remove Executor from EnvoyHTTPCallbacks API #32776
Changes from all commits
b012128
8f1a250
0c430d8
d7cad58
305ac95
5efe35f
a2e6eb0
18b9a44
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -66,11 +66,8 @@ public Object onRequestHeaders(long headerCount, boolean endStream, long[] strea | |
* @return Object[], pair of HTTP filter status and optional modified data. | ||
*/ | ||
public Object onRequestData(ByteBuffer data, boolean endStream, long[] streamIntel) { | ||
// Create a copy of the `data` because the `data` uses direct `ByteBuffer` and the `data` will | ||
// be destroyed after calling this callback. | ||
ByteBuffer copiedData = ByteBuffers.copy(data); | ||
return toJniFilterDataStatus( | ||
filter.onRequestData(copiedData, endStream, new EnvoyStreamIntelImpl(streamIntel))); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. question: we weren't running this in a separate thread from the looks of it, so why did we need There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah. I did this initially to be consistent with the callbacks API, but now that we have documented what the behavior is going to be with regards to the |
||
filter.onRequestData(data, endStream, new EnvoyStreamIntelImpl(streamIntel))); | ||
} | ||
|
||
/** | ||
|
@@ -111,11 +108,8 @@ public Object onResponseHeaders(long headerCount, boolean endStream, long[] stre | |
* @return Object[], pair of HTTP filter status and optional modified data. | ||
*/ | ||
public Object onResponseData(ByteBuffer data, boolean endStream, long[] streamIntel) { | ||
// Create a copy of the `data` because the `data` uses direct `ByteBuffer` and the `data` will | ||
// be destroyed after calling this callback. | ||
ByteBuffer copiedData = ByteBuffers.copy(data); | ||
return toJniFilterDataStatus( | ||
filter.onResponseData(copiedData, endStream, new EnvoyStreamIntelImpl(streamIntel))); | ||
filter.onResponseData(data, endStream, new EnvoyStreamIntelImpl(streamIntel))); | ||
} | ||
|
||
/** | ||
|
@@ -144,9 +138,6 @@ public Object onResponseTrailers(long trailerCount, long[] streamIntel) { | |
*/ | ||
public Object onResumeRequest(long headerCount, ByteBuffer data, long trailerCount, | ||
boolean endStream, long[] streamIntel) { | ||
// Create a copy of the `data` because the `data` uses direct `ByteBuffer` and the `data` will | ||
// be destroyed after calling this callback. | ||
ByteBuffer copiedData = ByteBuffers.copy(data); | ||
// Headers are optional in this call, and a negative length indicates omission. | ||
Map<String, List<String>> headers = null; | ||
if (headerCount >= 0) { | ||
|
@@ -159,7 +150,7 @@ public Object onResumeRequest(long headerCount, ByteBuffer data, long trailerCou | |
assert trailerUtility.validateCount(trailerCount); | ||
trailers = trailerUtility.retrieveHeaders(); | ||
} | ||
return toJniFilterResumeStatus(filter.onResumeRequest(headers, copiedData, trailers, endStream, | ||
return toJniFilterResumeStatus(filter.onResumeRequest(headers, data, trailers, endStream, | ||
new EnvoyStreamIntelImpl(streamIntel))); | ||
} | ||
|
||
|
@@ -175,9 +166,6 @@ public Object onResumeRequest(long headerCount, ByteBuffer data, long trailerCou | |
*/ | ||
public Object onResumeResponse(long headerCount, ByteBuffer data, long trailerCount, | ||
boolean endStream, long[] streamIntel) { | ||
// Create a copy of the `data` because the `data` uses direct `ByteBuffer` and the `data` will | ||
// be destroyed after calling this callback. | ||
ByteBuffer copiedData = ByteBuffers.copy(data); | ||
// Headers are optional in this call, and a negative length indicates omission. | ||
Map<String, List<String>> headers = null; | ||
if (headerCount >= 0) { | ||
|
@@ -190,7 +178,7 @@ public Object onResumeResponse(long headerCount, ByteBuffer data, long trailerCo | |
assert trailerUtility.validateCount(trailerCount); | ||
trailers = trailerUtility.retrieveHeaders(); | ||
} | ||
return toJniFilterResumeStatus(filter.onResumeResponse(headers, copiedData, trailers, endStream, | ||
return toJniFilterResumeStatus(filter.onResumeResponse(headers, data, trailers, endStream, | ||
new EnvoyStreamIntelImpl(streamIntel))); | ||
} | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
which thread is onResponseHeaders called from? is it the Envoy worker thread?
same question for other functions below like onResponseData
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's whichever thread that calls the callback, which is probably the worker thread? The idea for the original design is to not block the Envoy main thread, which makes sense. However, this design imposes the requirement that every callback must be executed in a separate thread from within the Envoy Mobile layer, which may not always be desirable depending on the needs. This change moves the responsibility to the
EnvoyHTTPCallbacks
implementer. For example, with Cronvoy as an implementer of theEnvoyHTTPCallbacks
, it can decide when to execute the callback in a separate thread. If you noticed, in the Cronvoy implementation, it hadDirectExecutor
(it is removed in this PR), which is basically a no-opExecutor
because Cronvoy does not want every callback to be executed in a separate thread.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So the Runnable task created in CronvoyUrlRequest.onData (https://github.com/envoyproxy/envoy/pull/32776/files#diff-8f04efffbc03f667d43c68783a3480f3b0b31cc61e136a07c778e5c177aca969R876) gets executed in the thread supplied by the Executor?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before this PR, there were two
Executor
s used inCronvoyUrlRequest
:The method below came from
EnvoyHTTPCallbacks
and the DirectExecutor implementation basically does nothing.The code in
CronvoyUrlRequest.onData
usesexecute
, which runs on mUserExecutor.This PR removes
EnvoyHTTPCallbacks::getExecutor
. It won't matter for Cronvoy because returns a no-opExecutor
anyway forEnvoyHTTPCallbacks::getExecutor
, but Cronvoy still usesmUserExecutor
which is a user-suppliedExecutor
. By not havingEnvoyHTTPCallbacks::getExecutor()
, Envoy Mobile no longer needs to perform a copy of theByteBuffer
before passing it to a user-definedonResponseData
callback in theJvmCallbackContext
.From Envoy Mobile standpoint, we now only have
Envoy::Buffer::Instance
-->envoy_data
and the data passed intoonResponseData
callback will be backed byenvoy_data
directly without any copy. The implementer ofEnvoyHTTPCallbacks::onData
can still make a copy if needed (just like the case with Cronvoy).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so if we want to remove the enforcement that all up calls be run on the non-network thread, I'd think we'd want to ensure all current calls (unless there's exceptions) be run on the non-network thread. I don't see any changes to the non-cronvoy up calls. Are we running those all on other threads already, or would this be a major regression for all non-cronvoy E-M?
Largely I think having an invariant to not run upcalls on the network thread is a good call. Is there any other way to avoid the copy?
/wait-any
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, for Cronvoy, it uses its own
Executor
to run on the non-network thread:envoy/mobile/library/java/org/chromium/net/impl/CronvoyUrlRequest.java
Lines 883 to 901 in bd0130e
The idea is to let the users control that behavior whether or not they want to run the callbacks on the non-network thread. IOW, we don't want the library (the Java/Kotlin library in this case) to enforce that. If we look at the current implementation of onResponseData, it currently does a copy of the
ByteBuffer
before passing it to theExecutor
even though theExecutor
implementation maybe a direct executor (no-op), which is the case the case with the Cronvoy implementation. From the Java/Kotlin library standpoint, there's no way to know if it uses a no-opExecutor
, so the safest way is to always create a copy.Before this PR, Envoy Mobile would create 2 copies:
After this PR, Cronvoy will only do one copy.
I agree that this change may inconvenience the users a little bit since now they have to decide whether or not to run certain callbacks on non-network threads, but at the same time the library will no longer need to always pay the penalty of creating a copy.
Another option that I can think of is changing the
EnvoyHTTPCallbacks::getExecutor
to something likeWith this change, if the
Executor
is empty, we don't need to create a copy. For Cronvoy, we will set it to empty. Let me know what you think.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"The idea is to let the users control that behavior whether or not they want to run the callbacks on the non-network thread." Hm, I'm not sure about this. While I can imagine that for some trivial applications this might be fine, I'm very concerned that this would be a massive footgun. I think Cronet runs callbacks on a different thread, IIRC, so it would seem like we should somehow be able to get the performance we need while preserving this invariant.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changing it to
Optional<Executor> getExecutor
is probably the best of both worlds? We can still default it toOptional.of(Executors.newSingleThreadExecutor())
, but for Cronvoy, we will set it toOptional.empty()
. When theExecutor
is empty, we will not need to do a copy (or the copy can be deferred to the implementer), but when theExecutor
is not empty, we will do a copy. What do you think?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm usually weary of general APIs that take optional values, because most users of an API will truly treat them as optional and not look as closely at them, but this is a major decision that is being left on the user, so I think it's better not to make it optional.
Why does Cronvoy need to create it's own thread in CronvoyUrlRequest if it's already passing in an Executor to have the onResponseData function run in that thread?
To me, from the user's point of view, the ideal API would be:
onResponseData(lambda_to_run, executor_to_run_on)
That way, the invoker doesn't need to create it's own threads to run on, it just passes the executor and the lambda, and the implementation handles running the lambda on the executor with the response data.
I might be misunderstanding something.
As a side note: I agree the copying sucks so it would be nice to get rid of it, but I think the main repercussion of the copies would be memory pressure on the app. I think the effect on CPU usage and latency would be minimal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the Cronvoy API, it wants to have control of the
Executor
per-request. This is not something that the current API supports. That's why Cronvoy implements a no-opExecutor
in theEnvoyHTTPCallbacks
API.This can simply be implemented with below without having to clutter the callback API with passing the
Executor
in each callback.By introducing an
Optional<Executor>
, the library does not always need to create a copy because it can know whether it needs to create a copy or not.FTR, the default
Executor
in the existing code is a primitive Executor.newSingleThreadExecutor, which is good for a toy project, but might not be ideal when running on a real workload.