Declaration files from Rhai engine metadata

[cn-kali-team]

We can use engine.gen_fn_metadata_to_json, generate the function and data type structure, and import lsp as the function and structure type registered by the extension developer.

#[export_module]
mod json_module {
    use rhai::{Dynamic, ImmutableString, Map};

    #[rhai_fn(name = "dump", global)]
    pub fn json_dump(json_map: Map) -> String {
        serde_json::json!(json_map).to_string()
    }

    #[rhai_fn(name = "load", global)]
    pub fn json_load(json_string: &str) -> Dynamic {
        let map: Dynamic = serde_json::from_str(json_string).unwrap_or_default();
        return map;
    }
}

• gen_fn_metadata_to_json

{
    "modules": {
        "json": {
            "functions": [
                {
                    "baseHash": 1772367929775027429,
                    "fullHash": 119576746319902270,
                    "namespace": "global",
                    "access": "public",
                    "name": "dump",
                    "type": "native",
                    "numParams": 1,
                    "params": [
                        {
                            "type": "Map"
                        }
                    ],
                    "returnTypeName": "String",
                    "signature": "dump(json_map: Map) -> String"
                },
                {
                    "baseHash": 10649942213899484619,
                    "fullHash": 1440055287418587681,
                    "namespace": "global",
                    "access": "public",
                    "name": "load",
                    "type": "native",
                    "numParams": 1,
                    "params": [
                        {
                            "type": "&str"
                        }
                    ],
                    "returnTypeName": "Dynamic",
                    "signature": "load(json_string: &str) -> Dynamic"
                }
            ]
        }
    }
}

• When I type json::,automatically complete load or dump.

[schungx]

That would require serializing to JSON and back internally.

I can probably just make a function that outputs FnMetadata types directly so the JSON step is no longer necessary...

[cn-kali-team]

However, many developers will register different function and structure types in their own engines, which requires recompiling rhai-lsp.
It is more convenient for individual developers to export JSON as an automatic completion extension

[schungx]

True. The LSP can take such JSON export and get list of all registered functions.

[tamasfe]

This sounds perfect, much better than my original idea of embedding the LSP.

As far as I see, it should be trivial to generate declaration files from the given metadata, which the LSP already understands.

I'm not yet familiar with the internals, how much effort would it be to include doc comments in this metadata as well?

Also initially I didn't want to touch the engine itself, but it will be inevitable if we want to get the best experience out of the LSP.
Would it be possible (from time and maintainability standpoint) to extend the Rhai parser and macros to parse doc comments and other metadata in the future for the LSP, probably gated by a feature or debug builds so it doesn't interfere with performance?

[schungx]

I'm not yet familiar with the internals, how much effort would it be to include doc comments in this metadata as well?

It would require the metadata feature which requires serde. Doc comments, I think, is already included in the JSON.

[cn-kali-team]

Yes, we don't need to parse the document as the data source, because Metadata already has enough information to provide to the LSP.

[schungx]

Come to think of it... Rhai automatically extracts doc-comments that are put on top of plugin functions into the JSON metadata.

This means that it is possible for the LSP to pick up the comments and potentially display it to the user as popup info.

Right now there are no doc-comments for the Rhai standard library, but maybe I should start adding them!

[tamasfe]

Come to think of it... Rhai automatically extracts doc-comments that are put on top of plugin functions into the JSON metadata.

Great, that's very convenient then. To sum this up a bit, we might need or have:

• functions registered in Rust via macros:
  • function name
  • signature (+ type information)
  • doc comments
• functions registered in Rust at runtime
  • function name
  • signature (without parameter names)
  • doc comments
• types registered in Rust
  • type name
  • doc comments
• functions that are defined in Rhai scripts:
  • function name
  • signature (parameter count but no type information I assume)
  • doc comments

Right now there are no doc-comments for the Rhai standard library, but maybe I should start adding them!

No need to hurry. I still have exams, and a project deadline on 24th, I can only continue work on the LSP after.

[schungx]

signature (parameter count but no type information I assume)

Script-defined function parameters are always Dynamic, and returns Dynamic, so the type is deterministic!

functions registered in Rust at runtime

You still get a signature with parameter types, but not the actual name of the parameters. So the signature will be something like fn foo(_: i64, _: bool) -> String.

[schungx]

Dear all, if you'd pull from the latest dev repo, you'll find that gen_fn_metadata_to_json includes doc-comments for all standard library functions including examples!

[schungx]

When do we expect the feature to load metadata files? With this, all the errors will go away and the LSP is actually usable!

BTW, there are a number of "standard" functions that should be automatically recognized because they are built-in. They are counted as reserved keywords:

• print
• debug
• type_of
• Fn
• call
• curry
• is_def_fn
• is_def_var
• is_shared
• eval

[tamasfe]

When do we expect the feature to load metadata files?

I'll work on this after I fixed the current bugs :)

there are a number of "standard" functions

Are these included in any of the generated metadata? If not, definition files will have to be hand-written.

[schungx]

Are these included in any of the generated metadata? If not, definition files will have to be hand-written.

Unfortunately no... But I can write them for you.

It is better to have free-form docs for these, as they can be special. For example, the call and carry functions take a variable number of parameters.