[v3] Binding generator improvements

[fbbdev]

Description

This is a large PR that brings many fixes and improvements to the binding generator. While the changes should be strictly non-breaking to the average user of the framework (with a very small exception, listed below), the edits will almost surely generate massive conflicts for people who might be concurrently working on the binding generator.

For that reason, after discussing this with @leaanthony, I am opening the PR in draft mode to wait for comments from the community. I am open to any suggestion and (reasonable) request and can collaborate if needed with people who might find it difficult to reconcile their work with the changes submitted here.

I think its best to keep the following description as short as possible, everybody please feel free to ask any question and I will expand on the details. Note that some commits contain additional explanations in the commit message.

Summary

• Support for custom struct parameters in bound methods (parameter unmarshaling was previously wrong)
• Support for multiple error/non-error return values (previously at most 1 non-error and 1 error were supported)
• Support for variadic parameters
• Improved support for slice, pointer and map types (as many people have noticed, JS types were often generated incorrectly when using slices)
• The binding generator has been refactored and is now based on Go templates (like the model generator)
• Compliance with encoding/json's behaviour (field names, field visibility, json tags)
• Support for embedded struct fields
• Lots of smaller bug fixes
• General refactoring and dead code cleanup

The changes stack up on each other so it would be somewhat difficult to split them into separate PRs, but I tried to ensure every commit has consistent and working code, so it should be possible to merge up to some commit and delay later ones (apart from test updates, which are bundled in two final commits and will need to be redone). This should roughly correspond to keeping only an initial segment of the features listed above.

Support for complex types is still quite limited (for example there is no support for slices of slices, pointers to slices, and many other possible combinations). I just tried to fix the implementation for the most common cases. One way to work around this, for the time being, is to create a hierarchy of structs with simpler field types.

There are also many problems in the code that resolves imports and selector expression, but I'm leaving those out of this patch.

Assessing the patch

The best way to assess the final user-facing effects of the patch might be to inspect the diffs on testdata for the parser package and on the binding example with a good diff viewer (the VSCode one is fine, the github one doesn't look so bad either). The relevant commits are the last two.

Breaking changes

The only directly breaking change I am aware of is the following one:

• when sanitizing identifiers or generating them ex-novo, the generator now prepends a '$' sign instead of an underscore '_' or nothing at all; the names of anonymous structs become therefore '$anon' instead of just 'anon'. The reason for this change is that Go identifiers may not contain the '$' sign, thus there may be no conflicts with user-provided identifiers.

Of course, people who came to depend on buggy behaviour may experience the fixes as breakage.

Main changes

The largest single change is probably be the move to templates for methods. All previous code has been thrown out and its behaviour reimplemented with Go templates. The code in binding.go is now minimal and quite similar to the code that was previously in models.go, modulo some refactorings.

The benefit of this change is that now there is a single authoritative source that defines the shape and content of generated JS code, so maintenance and improvement should become easier. The only other code that still generates some javascript is the JS() method on the ParameterType struct (subsuming the many duplicated instances that were lying around previously), and the DefaultValue() method on the Field struct.

The other important change is that now model generation is now driven by the same algorithm used by the encoding/json package for determining field names, visibility and other options. Json tags are now taken fully into account.

The benefit of this change is that people who respect generated types should see everything work as expected when passing structs into and out of bound methods.

Main fixes

The most important problem fixed by this patch is that the binding system used to Unmarshal all input arguments to the default types (i.e. maps) instead of the types expected by the methods, thus leading to runtime error. This is fixed by the first commit.

A connected problem was that the generated models relied upon a very simple heuristic for determining field names and visibility which did not match the one used by the encoding/json package, leading to errors and/or broken expectations. This motivated the second change described in the previous section.

A secondary problem was that the binding generator used to pass around package paths during generation and sometimes the wrong ones were passed (or package aliases in place of paths). This has been fixed by directly passing around pointers to package descriptors.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update <-- NOT SURE

How Has This Been Tested?

All binding generator tests, as well as the binding example, have been updated and verified to pass locally. Additional tests have been created for new functionality. All generated files have been carefully inspected by hand.

• Windows
• macOS
• Linux

Test Configuration

Go Version     | go1.22.0                                                                      
Revision       | 12d12a33dec1d31f487b70f87bc31e80252f327c                                      
Modified       | true                                                                          
-buildmode     | exe                                                                           
-compiler      | gc                                                                            
CGO_CFLAGS     |                                                                               
CGO_CPPFLAGS   |                                                                               
CGO_CXXFLAGS   |                                                                               
CGO_ENABLED    | 1                                                                             
CGO_LDFLAGS    |                                                                               
DefaultGODEBUG | httplaxcontentlength=1,httpmuxgo121=1,tls10server=1,tlsrsakex=1,tlsunsafeekm=1
GOAMD64        | v1                                                                            
GOARCH         | amd64                                                                         
GOOS           | darwin                                                                        
vcs            | git                                                                           
vcs.modified   | true                                                                          
vcs.revision   | 12d12a33dec1d31f487b70f87bc31e80252f327c                                      
vcs.time       | 2024-02-21T12:40:32Z                                                          

# System
Name            | MacOS                                   
Version         | 12.6.6                                  
ID              | 21G646                                  
Branding        | Monterey                                
Platform        | darwin                                  
Architecture    | amd64                                   
Apple Silicon   | unknown                                 
CPU             | Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
Xcode cli tools | 2395                                    
CPU             | Unknown                                 
GPU             | Unknown                                 
Memory          | Unknown                                 

# Diagnosis
 SUCCESS  Your system is ready for Wails development!

Checklist:

• I have updated website/src/pages/changelog.mdx with details of this PR
• My code follows the general coding style of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[coderabbitai]

Important

Auto Review Skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository.

To trigger a single review, invoke the @coderabbitai review command.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

---

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• The JSON schema for the configuration file is available here.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

[leaanthony]

Thanks for doing this @fbbdev - Huge! I'll make my way through it in the coming days.

[leaanthony]

@coderabbitai review

[leaanthony]

@fbbdev - Initial passthrough looks good 👍
Because this is such a big change, I don't see there being many more "features" for this, so I was thinking that it might be a good idea to document the usage and behaviour of this feature now? I can set up a page ready for the markdown contents. Or we could do it later, but I was thinking that being part of this PR would be good. Thoughts?

[tmclane]

Testing this out against my test project on Manjaro now.

[abichinger]

I found a test case where no bindings are generated.
9c2dd58

[fbbdev]

I found a test case where no bindings are generated. 9c2dd58

Thank you @abichinger for finding this out.
Unfortunately this is due to a problem with the deprecated identifier resolution system included with the Go parser.

I pulled in your test and committed a temporary workaround that makes it pass, but a full solution would require a switch to the new type checking system. I am currently experimenting with that, I'll make a post as soon as I have something working.

[fbbdev]

@leaanthony Thanks for having a look at it so soon!

In general, I think it might be better to have a dedicated PR for documentation. We could start working on it right now if you wish. However, I think the current implementation has still quite some problems and we might want to consider changing something if things go well with the type checker. What do you think?

[abichinger]

Nice! Switching to the go/types package sounds promising.

[leaanthony]

I can start that today. Just wanted to keep momentum 😅

[abichinger]

I encountered two more bugs.

1. Models with fields of type bool are not generated correctly 8458d03
2. If error is nil, BoundMethod.Call should return nonErrorOutputs 70f36bb

[fbbdev]

I encountered two more bugs.

Nice catch!
Merged, thanks!

[leaanthony]

Linked new docs PR

[abichinger]

Hi guys, I picked up the feat/bindgen_v2 branch from @fbbdev and tried to reimplement the binding and model generator with go/types. It's not done yet, but I thought I should let you know that I'm working on it.
You can track my progress at https://github.com/abichinger/wails/tree/feat/bindgen_v2

[fairking]

I guess this PR also solves #3056. Great job guys. 👍

[theplatformer]

I've been testing this with my project this week (macOS) and it's been great. It has resolved a couple of issues I've had with slice return types and returning both non-error and error values, both working as expected now. It also is now consistently working across the main package as well as importing from other packages, and both struct literals and constructors.

Awesome! 😃

The only thing that has come up is the models.ts files throwing a 'parsedSource' is never reassigned. Use 'const' instead. linter error from ESLint. I notice this is the same behaviour in the current v3-alpha branch so it may just need a .eslintignore entry seeing as this is codegen.

[leaanthony]

As the request of @fbbdev, I'm closing this in favour of #3397, which is based on this work. All further efforts/feedback should be directed there. Thanks all 🙏