Experiment sending the complete_request message on the control channel.

[jasongrout]

References

Code changes

With ipykernel 6.0 or later, this enables the complete request to be processed in parallel with execution, so we can still get completions while the kernel is busy.

This is an experiment to see if we can improve autocomplete responsiveness. Luckily, it seems that ipykernel accepts any shell message on the control channel (see ipython/ipykernel@2894558#diff-afe8c3a6fde7c69241bf107a35a82a41da345bcbfa62bb23347ca30f7d25acd8R164 for when this was introduced in 2012), so this should work with any ipykernel version 6 or later. The messaging spec implies that sending completion requests to the control channel is not officially supported.

User-facing changes

In one cell, execute a sleep, like: import time; time.sleep(20)

Immediately after that, while it is still busy, try doing autocomplete (like pri<tab>). The autocomplete should work.

Backwards-incompatible changes

[jupyterlab-probot]

Thanks for making a pull request to jupyterlab!
To try out this branch on binder, follow this link:

[jasongrout]

@krassowski mentions in the dev meeting that jedi, when executed in threads, can cause huge issues. @krassowski, can you elaborate?

[jasongrout]

The Jedi docs include this warning:

By default jedi.settings.fast_parser is enabled, which means that parso reuses modules (i.e. they are not immutable). With this setting Jedi is not thread safe and it is also not safe to use multiple Script instances and its definitions at the same time.
If you are a normal plugin developer this should not be an issue. It is an issue for people that do more complex stuff with Jedi.
This is purely a performance optimization and works pretty well for all typical usages, however consider to turn the setting off if it causes you problems. See also this discussion.

[krassowski]

In addition to the above, davidhalter/jedi#1807 (comment) is another example. pylsp has a lock on each operation to circumvent some issues, but it prevents it from handling multiple requests at once really.

It might not be an issue for completions on control channel though. The problem would be only if this change would allow to send two completion requests simultaneously for concurrent execution.

[jasongrout-db]

The problem would be only if this change would allow to send two completion requests simultaneously for concurrent execution.

This could happen if a client sent a completion request on the control channel while another client sent a completion request on the shell channel.

[jasongrout]

This could happen if a client sent a completion request on the control channel while another client sent a completion request on the shell channel.

Of course, the easy thing here is to introduce a coordination mechanism in ipykernel between the control and main thread to not process completion messages concurrently.