Support a React/next.js-like framework

[matthewmueller]

Hey folks! Now that I've managed to get the installation process under control, I'm starting to focus on getting a really minimal, but productive frontend framework working that either uses or is based on React/Preact & next.js/Vue.

This has been more difficult than I thought because some of React's paradigms don't really translate 1:1 to Go. Specifically how they do setState and some of the lifecycle methods like componentWillReceiveProps are tough to model in the same way.

I'll be filling in this issue with more information and thoughts as I go, but I'm also looking for ideas. We have tons of variables we can tweak, so it's just about figuring out what feels right. For example, some of the options we have are: tweaking React's API a bit to more like idiomatic Go, generating additional Go code via go generate, updating Joy's JS output to better support React.

Please help make Go great for web development!

[tryy3]

I am curious of how you are gonna do this, will you do this with mirror functions or just through functions that allows you to access javascript functions/variables?
Like will you do mirror functions similar to the stdlib or what is the approach you are gonna go?

[matthewmueller]

Hmm, not 100% sure what you mean by mirror functions. Do you mind sharing an example?

I keep oscillating between having more of a Go way of doing things and a React way of doing things.

The farther I go towards React, the more familiar the API is, but the more Go AST rewriting I need to get Go there. I think a pure Go implementation will require rethinking how React works. @tj has experimented with how this would look a bit.

The parts I'm struggling the most with are the lifecycle hooks and the local state. Knowing if they're really needed or if there are other ways to do the same thing.

I think for the first release, I'd think like to make it as compatible with the existing frameworks as I can, and will take advantage of the fact that the actual data flows once compiled are in JS and untyped. We'll leave it up to the frameworks (React + Preact) to give values back that are consistent with the types in our structs.

This is what I have in mind for right now for the API:

package basic

import "time"

////
// React stuff (incomplete)
////

// Component interface
type Component interface {
	SetState(state interface{})
}

// Node interface
type Node interface {
	String() string
}

// Text fn
type Text string

func (t Text) String() string {
	return string(t)
}

////
// Clock component
////

// Props struct
type Props struct {
	Name string // property
}

type state struct {
	Count int
}

// Clock struct
type Clock struct {
	Component

	props Props
	state state
}

// DidMount fn
func (c *Clock) DidMount() {
	go func() {
		time.Sleep(1 * time.Second)
		c.SetState(&state{
			Count: c.state.Count + 1,
		})
	}()
}

func (c *Clock) WillReceiveProps(nextProps Props, nextState state) {
}

// Render fn
func (c *Clock) Render() Node {
	return Text(c.state.Count)
}

PROS

• Very similar API to React
• Utilizes battle-tested virtual DOM frameworks
• Typed props and state (even in lifecycle hooks)

CONS

• Go AST rewrites needed (not a huge deal, just slows down the build performance)

We need to match up the props and state fields exactly with React if we don't want any AST rewrite.

• 2-3 structs per component (ideally it's just 1 with capitalized fields as the props)

This is to support the types on things like: WillReceiveProps(nextProps Props, nextState state). and ComponentShouldUpdate(...).

• Not very idiomatic (lifecycle hooks gets called magically)

This is problematic because by default Joy will eliminate the lifecycle hooks because no part of the Go code is calling them, so not only are we invoking magic here, we also need to back out of the dead-code elimination for these methods.

This is still very experimental though. I'm really hoping you guys have some ideas here too!

[shaban]

@matthewmueller Actually this looks very neat and immediately familiar for everyone who already worked with React.

But there is a few thoughts i had when thinking about how this all plays along Go's existing technologies.

So sorry to be the odd guy, but i thought about an interesting way to use joy in regards to how we use go today to deliver websites.

The main goal is to use Go's server-side web prowess to have a fast first render and the isomorphic nature to have just one Data Type and it's methods to serve both the needs of server and client side.

It uses existing Go technology like the template package.
It would need a non cli version of joy, that is implemented as template helper.
This way plugging into the rich web toolbox of GO.

For Illustration how this might look like i made up some dummy design suggestion.
It is far from being thought to the end, but i hope people more knowledgeable than me can say something about the viability and the merits of such a system.

So let's take a look at a very minimal Go server side example. it's not actual code like the Component interface and WebComponent Data type is stuff that should be realized in a library that powers the whole thing.

type Component interface{
	OnUpdate()
	OnMount()
	OnUnmount()
	State()
	SetState()
	RenderAs()
	HtmlName()
	// + whatever needed
}
type WebComponent struct{
	uniqueID string
	// general helper fields needed for managing the component
	// like the internal unique String ID 
	// for javascript bookkeeping like references Dom traversal and whatnot
}

type NamedComponent struct{
	*WebComponent// 
	//here we use struct tags for kebab case and rendering
	// the component asDiv instead of "named-component"
	// let's say for old browsers or personal preference
	ID int
	Label string
	Slug string
}
func (nc *NamedComponent)State(){
	// here we set the data since it's go we have many many options like
	// setting the whole Instance at once or composite it from different sources,
	// with calculations and what not
	// and all with the rich typesystem
}
func (nc *NamedComponent)RenderAs(){
	// implement this function to make it render as Div
	return "div"
}
func (nc *NamedComponent)HtmlName(){
	// implement this function to make the component name kebap-case
	return "named-component"
}
func(nc *NamedComponent)SetState(){
	// change some Data
	// call internal webcomponents methods like manually calling update
	// and have that cascade through all subcomponents
}
func (nc *NamedComponent)ComputedValue()string
	// do somethign with the Data to create a computedValue
}

Then we can hook functions into the template parser to inject our javascript side of the we component to take over once go has done the first loading of the component via standard html/template.

<!-- lives for example in the file components/named-component.html -->
<named-component>
  <div id={{.ID}}>
    <a href={{.Slug}}>{{.Label}}</a>
    <!-- equivalent of calling sub-component -->
    {{template "sub-component.html" .ComputedValue}}
  </div>
  <style>
    /* 
    for styles that have to do with functionality like hiding translating, etc.
    some very basic styles for making it a very sober/bland looking thing that can 
    be styled into any direction you want.
    Goal is to have a reusable component
    Pure Desing should go to the global style sheet 
    */
  </style>
  <script>
    // Since we are using Go on both ends, we now have a lot of cool options
    // we can have an ajax helper that gives us the same data we had serversude
    // we can have a go map to store function receivers, event names
    // and trigger them with payload to deliver to the receivers
    // or stop listen to messages by asking the map to delete the entry
    // some template Helpers for common tasks
    // interfaces one can implement for more fine grained control
    // effectively taking over a task
    {{inject "jsRepresentationOfNamedComponent"}}
  </script>
</named-component>

In a nutshell the beauty lies in tapping into existing Go Web techniques for a blazing first render.
And then have joy take over for managing state, databinding and such.

There is a lot of open questions like synthetic Events, Two-Way Databinding, yes or no, Shadow DOM (i think it's a bit hyped and not a must have). Efficient rerendering techniques and so on.
Also it would mean to hook into the parseTree of the template to take some metrics needed later on, so that joy's rerendering and go's template initial run are doing the same.

Once again sorry for slightly side tracking this topic, but at this early stage i thought it would be interesting to take a look on where the journey could go and to make sure to not idiomatically think too slavishly alongside existing js solutions but to harness the full power of go with its beautiful design principles.

Before i forget, this way you also have perfect editor support for html css and js.
It is even possible to use wellingtons excellent speedy libsass implementation to inject scss via a template helper function. Although it wasn't my intention when starting to write it, it looks also remotely similar to vue.js.

Let me know what you think.

[matthewmueller]

So sorry to be the odd guy, but i thought about an interesting way to use joy in regards to how we use go today to deliver websites.

Thanks for your thoughts on this! I'll need to review a bit more, but this is definitely the kinds of things I'd like to see and try. I think this is more closer to what Vue does, but able to take advantage of all of Go's existing tooling, which is rad.

I was thinking Vue would actually probably be better final target, but given the plethora of tooling already available in Go, I think it makes more sense to start with a Next.js-like framework and just have all the code be Go.

I think your approach, once refined, would build towards a longer-term goal which is to have the react-like framework written in Go and using more of Go's idioms. I'm hesitant to start on this work at this stage because React is so battle-hardened, but I'm hopeful there will be others that take on this challenge.

[shaban]

Yeah i agree getting something out now is better than trying to rethink reactive framework from a go perspective right from the start.

Having something workable shorter term will ignite the sparks anyways.

About the limitations you mentioned:

Not very idiomatic (lifecycle hooks gets called magically)

As long as the architecture is built with these things in mind and a logical place is reserved where missing features can be implemented in future iterations you should be good.

2-3 structs per component (ideally it's just 1 with capitalized fields as the props)

All the things you get for free from Go will make up for slightly less convenience than usual. I wouldn't sacrifice proper distinction between props and other fields by abusing the Capitaliztion for a different paradigm.
It will only look confusing if someone needs exported struct fields for something non state related.

we also need to back out of the dead-code elimination

Since joy doesn't work like gopherjs and will probably have very slim libraries in the near future, DCE can be worked on when it actually becomes a topic.

On a sidenote i read like 3 articles covering joy in my google news site, so gratz on that :-)

[matthewmueller]

For those interested, I'm writing down a spec of how this is going to look: https://docs.google.com/document/d/12KWk9wnQMyy8fnOu7UV8eJorGHcUN710aIYJF2ppJRE/edit?usp=sharing

It's very much a WIP and I would love some help carving it into shape.

I'm finding it's a lot easier to think through the design separately from the implementation. If you have time, read through and add comments 😊

Happy holidays! 🎄

Update: I linked to a private project called elmo in there, if you'd like to take a look at that, let me know and I'll invite you.

[shaban]

It will also primarily target Preact since I have yet to find an issue where Preact fails and Preact is much smaller and easier to understand.

Preact is well structured and small with only a handful of short Typescript files which makes it the best candidate for porting/adapting. Wise choice. The only downside is that it is kind of the second slowest react clone, but this should be a non issue for the scope of this first iteration.

OPEN ISSUE: What happens if the props field doesn’t exist? In that case we’ll probably want to error out since I’m betting that most of the time you just forgot it rather than you intentionally don’t need it. Also even if you don’t need props now, that doesn’t mean you won’t need props in the future

One thing that people might want to make a "dumb" component for might be repetitive large strings, like inline svg e.g close button etc.

OPEN ISSUE: Do folks hate functional HTML like this? It’s actually less keystrokes than HTML, but perhaps it’s a bit odd and you can’t just copy and paste from somewhere like JSBin.

The question is, will doing it like this make things WAY easier for you or not. If it does, don't worry since non enthusiast Developers who just want to be productive here and now are bound to hate the first iteration.

It's the enthusiasts who see the potential in it and will help you to make this thing being production ready later on who count.

If you think it won't make a big difference for you doing it the HTML way i'd bet on devs loving their documentation bits of their UI Framework of choice working without too much hassle.

OPEN ISSUE: we could probably combine pathname and query as a url.URL in Go.

If you wish to be able to use something like URI parsing and manipulation (maybe abusing the fragment part for interesting things) https://medialize.github.io/URI.js/ might be interesting to you. I tried it out on a project where i had to work with documents loaded from a server editing them, renaming etc. and it worked like a charm.

https://github.com/medialize/URI.js/blob/gh-pages/src/URI.min.js is 46kb minified , so probably only 7kb or so gzipped.

OPEN ISSUE: can we merge GetInitialProps with ComponentWillMount?

Only for the root components if i got that right from next.js documentation.

Other only remotely related problem is:
Sometimes there is a good reason to conceptually differentiate, like even though two methods might be in your case completely in order with nothing happening in between them they still give the dev the opportunity to write code that has a different scope.
This is especially relevant from a go perspective where paradigms don't translate 1by1.

Since we don't have constructors in go we need to offload that part to one of the lifecycle hooks.
The question is, by doing so, will we have nice distinct places for following two tasks?

• create meaningful defaults when let's say you have to depend on the timing of external 3rd part libraries, frameworks etc.

• and do tasks that build on these sane defaults, like doing things that are necessary because they can't be handled by automatic propagation down the hierarchy.

and not run into a situation where you have too much code that does two different things in 1 hook.

OPEN ISSUE: would supporting netlify’s _redirects file require breaking changes in the future to support S3 + Cloudfront? Particularly wondering about the /:author.rss redirect and the status codes.

Asked the other way around would having a generic implementation that only makes sure that all of the pieces of data needed are structured and accessible for jolly -export=netlify, export=whatever not be more desirable?

Reading up on the different formats and the ingredients of the recipes should give you an idea what data needs to be aggregated and you can delegate that to a different task enabling you to focus more on the general goals.

Adding Command Line Flags later on to transform generic data would not cause any portability issues, but baking in some sort of default export format might.

[matthewmueller]

Thanks for taking the time to read though this!

One thing that people might want to make a "dumb" component for might be repetitive large strings, like inline svg e.g close button etc.

Great point, I think there should be something where you can just write raw HTML. It just gets a bit tricky when templating starts to get involved. I've definitely run into your exact example before though, bringing in SVGs. You wouldn't want to convert those by hand, though it's worth noting that you still need to do some conversion with react, changing hyphen case to camel case. I think we wouldn't even need to do this though.

The question is, will doing it like this make things WAY easier for you or not. If it does, don't worry since non enthusiast Developers who just want to be productive here and now are bound to hate the first iteration.

I'm sort of betting that having the type system and syntax highlighting will make this worth it, but I'm not 100% sure and I'm open to alternative solutions here. This could also be solved at the editor level too with pre-compiling HTML into these functional elements.

If you wish to be able to use something like URI parsing and manipulation (maybe abusing the fragment part for interesting things) https://medialize.github.io/URI.js/

Ahh yep, I've used that one. Filling in the possibility space a bit, I was also looking at https://github.com/component/url, though this doesn't actually work in web workers, so that might be an issue. There's also this one: https://github.com/unshiftio/url-parse. There's also a browser native version now, but I'm not sure about browser compatibility.

Sometimes there is a good reason to conceptually differentiate, like even though two methods might be in your case completely in order with nothing happening in between them they still give the dev the opportunity to write code that has a different scope.

Good question, the more I think about this, the more I think you're right on. No reason to conflate the hooks for a few less keystrokes.

Asked the other way around would having a generic implementation that only makes sure that all of the pieces of data needed are structured and accessible for jolly -export=netlify, export=whatever not be more desirable?

Oh yah, that could work too. Eventually, it'd probably be good to have a jolly deploy netlify or perhaps like git with remotes. At this point we could have a script that will consume a _redirect file and make the appropriate API calls on AWS.

This is mostly an issue of... do we have the ability to do this via API calls on AWS?

[shaban]

Great point, I think there should be something where you can just write raw HTML

that would do the job.

This could also be solved at the editor level too with pre-compiling HTML into these functional elements.

That's true. With all the parsers and tokenizers around now for go shouldn't be hard to get some intermediate representation going for syntactical sugar later on.

There's also a browser native version now, but I'm not sure about browser compatibility.

Haven't heard of that one, but i know there is some nice hack by creating an anchor dom element and abuse it for parsing it's href. An Example that looks a bit more robust is here:

https://j11y.io/javascript/parsing-urls-with-the-dom/

This is mostly an issue of... do we have the ability to do this via API calls on AWS?

TBH i don't know enough about this stuff. I am blessed with an abundance of dedicated servers for my work :-)