Formally describe command args and results

[bollwyvl]

Commands are critical to the interaction between many parts of a lumino app, yet are very loosely-typed, and generally opaque to implementers and users. This limits the discovery, effective user interface, and reusability of commands. While these problems are likely unsolveable at compile time, runtime validation and discovery would improve all of these for the user of a lumino application, with minimal effort to implementers.

Individual issues and PRs

• Add describedBy to command #322
• Allow describedBy to be an async function #331

Current Challenges

Discovery

At present, commands are difficult to discover. Yes, one can listCommands to get their ids, and then iterate over all of their option values, and try calling them.

When commands are further abstracted, it becomes basically impossible to determine whether a command execution will be well-formed, and what is just nonsense that will fail in a fully-spun-up application.

User Interface

From the application perspective, commands are fire-and-forget, delegating the responsibility to individual implementations. Many commands would benefit from even minimal UI to modify their behavior, and could be, by default, presented in a single, consistent way vs some commands "just working", some opening dialogs, and other unpredictable/inconsistently accessible things that can occur.

Payoff

With well-typed and documented commands, it becomes possible to generate machine-readable representations of possible command states (to-be-executed, was-executed, etc). A command palette, thus enhanced, could provide significant value.

Additionally, with discoverable commands, hybrid gui/cli applications, a la clui become possible.

Finally, the ability to understand each command's arguments and result make it possible to serialize longer chains of commands that reference each other's output: DockPanel's layout serialization represents this in one, particular way, but more generalized approaches could truly make lumino an engine for lisp machine/smalltalk-style self-extensibility and introspection.

Out of scope

Selecting the validation description itself is probably neither viable nor useful: there still exists no standards-body adopted description-of-JSON-in-JSON, much less one implemented in browsers. While JSON Schema is the de facto representation, with ajv the de facto browser validator, it's worth exploring other options... once we have something that can accept any of it, with the usual caveats of performance, security, stability, and cross-language compatibility.

• jddf: https://tools.ietf.org/html/draft-ucarion-jddf-05
• json-e: https://github.com/taskcluster/json-e

Proposal

I don't think any of these would require breaking changes, but it may be worth targeting/tabling this work until such a time as it would make sense. At any rate, I would be able to commit to implementing these feature, just not on what timeline.

• @lumino/coreutils
  • add JSONValidationExt
    • add IValidator which defines a minimal validation scheme, e.g. validate(schema, instance) => Promise<any[]> (suitably genericized)
• @lumino/commands
  • add and accept CommandRegistry.IOptions
    • an optional validator to a
    • add an optional validate: {args: true, result: false}
  • add an describedby to ICommandOptions which returns (a promise to) a schema for args
    • ...and result (if it even makes sense to validate outputs, as many are "fat" objects, e.g. Widget)
• @lumino/application
  • add an optional validator to Application.IOptions (new)
• @lumino/commandpalette
  • add an optional renderArgs to IRenderer
    • no implementation by default, for reasons above.
• examples
  • example application using:
    • JSON Schema Draft 7
    • ajv
    • potentially a bespoke form implementation, a la rjsf or just use it.

Follow-On

• A longer-term convention could introduce a package.json field, e.g. {"lumino": {"commands": {}} which would make available commands truly discoverable.
• a metadata field on describedby could allow for application/implementation specific options, such as rjsf's uiSchema.

References

Downstreams:

• Schema for Workspaces and Commands? jupyterlab#6013
• Start adding some docs on workspace format jupyterlab#7945
• https://github.com/deathbeds/jupyterlab-starters

[meeseeksmachine]

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/is-there-an-api-for-commands-that-are-added-to-the-command-registry/7814/2

[vidartf]

I think this proposal could benefit from some use cases as examples. It would help clarify the benefit, and would make it easier to reason about the pros and cons of the proposal.

[fcollonval]

The description of command results can be tricky as some commands are expected to output a JavaScript object; most notably a new widget.