As of kotlin-react version pre.282,
the Kotlin wrappers now offer a new DSL to integrate with React.
Basically, the new version uses the new "ChildrenBuilder" instead of RBuilder as receiver to build the DOM, and
really puts the emphasis on functional components (with the FC/VFC helper functions).
Migrating was really not easy because there is no documentation apart from
a tiny snippet in the CHANGELOG.
It turned out I also needed to switch my styling approach to use the Emotion library, because kotlin-styled-next
was not compatible with the new DSL (at least as far as I remember).
Of course, I also needed to migrate my BlueprintJS Kotlin wrapper to use the new DSL before I could really use it in the Seven Wonders project.
Here are the changes that I ended up doing for the migration. When I have time, maybe I should document my learnings about this, maybe even contribute an actual documentation for them.
The kotlin-wrappers repository now provides a BOM to align dependency versions. Switching to this BOM will make the upgrades of the Kotlin Wrapper dependencies way more convenient!
During this period, the work was mostly about maintaining dependency versions and streamlining the process.
This is when I improved or cleaned up my GitHub Actions workflows, switched to Gradle's Dependency Catalog to manage dependencies centrally, etc.
Using Kotlin/React for the frontend implies writing Kotlin bindings for JavaScript libraries. So far, I had written those bindings right in the Seven Wonders project, but it would be rude to not share this effort with the world.
This is why I decided to extract the BlueprintJS Kotlin wrappers as an external OSS library on GitHub.
Heroku's free plan is nice because it's free, but the server is shutdown after 30min of inactivity. Technically, the Seven Wonders server takes only about 4s to start, but the Heroku workers can take much more (~30s-1min), which makes this pretty annoying when people click on the link for the first time.
To provide a better experience to users, and also to learn a bit about Kubernetes and cloud providers, I set up a Kubernetes cluster on DigitalOcean and started deploying the public instance of Seven Wonders Online there.
I also got lots of help from @ArchangelX360 for the Kubernetes setup, and also when setting up proper monitoring with Prometheus, Loki, and Grafana Cloud.
With a target goal to later deploy "more easily" a Seven Wonders server, publishing a Docker image seems to be a rather good first step. It also has the benefit of allowing people to run their own server locally more easily.
Synchronizing backend and frontend when making changes to the API is a pain. Model classes need to be written and modified twice to be kept in sync. Moreover, a Java client also needs to be maintained to run integration tests for the server. This is why I decided to leverage Kotlin multiplatform capabilities to define a reusable common model and client. It took some time because Kotlin itself changed a lot during the past year.
Also, there was no existing solution for a multiplatform STOMP client, so I ended up building Krossbow. This took a lot of time too.
Once a common client and model was defined, there is the problem of the local project dependencies. Using a TypeScript/React frontend built with npm, it wasn't possible to depend on a local gradle subprojects for the model and the client. That's why it was time to try out Kotlin React.
Kotlin React is easy to use in itself, but using React component libraries from Kotlin is quite tedious at the time. Dukat, the TypeScript to Kotlin externals conversion tool, doesn't work reliably and doesn't work at all for React stuff, so the declarations have to be written manually.
Flow is nice, but doesn't give me the safety I expect. In its nature, Flow is a type checker aside from the build of the project. This means that it is possible to build the project successfully even with type errors in it, depending on how it is configured.
I also wasn't too happy about the performance of the IDE integration of Flow. Maybe I didn't put much effort into configuring things properly, but I did try multiple settings with more or less safety/performance. The overall experience I had was not that great in this respect.
TypeScript came and saved the day. During the migration, it spotted several places that Flow didn't point out and where the types were incorrect. Also, I could clean up a bit the types of the redux actions and their creators.
Kotlin really improves on Java on multiple aspects:
- nullability is encoded in the type system, which means no more NPEs and unnecessary null checks
- no unnecessary verbosity: Kotlin is much more condensed than Java for declaring classes and all their members, implementing delegation, transforming collections, etc.
- the stdlib has 2 different interfaces for read only and mutable collections
- extension functions allow to add methods to an existing type without extending it
- and many more...
Kotlin is completely interoperable with Java and can be adopted incrementally, so I decided to give it a try. I migrated the game engine and backend server of Seven Wonders to see how it goes and if I could do this kind of change at work. For now, this has been quite a success.
Not much has been done on the Seven Wonders project for a while, because I stopped to build Livedoc. I now have a decent and usable documentation generation tool, which requires almost no configuration at all.
It still lacks a couple features:
- Specialized STOMP endpoints presentation (they currently are presented as HTTP endpoints)
- Client generation (especially Flow/TS type defs, bonus for the WebSocket client)
I feel like we're not far from having something very useful now.
Javascript can easily become a mess when the code base grows, and static typing helps wrapping one's head around what's going on, especially when multiple people are involved in writing code in the same project.
That's why we considered TypeScript and Flow for Seven wonders. Since I use TypeScript at work already, I had a preference for Flow in order to learn something new, and @victorchabbert had actually this in mind already.
Victor introduced Flow in the project and started adding some types already, but this didn't work well with
seamless-immutable. After reconsideration, it seems that I had missed the Record type of ImmutableJS, which allows
for direct property access like player.displayName instead of the player.get('displayName') I hated so much. This
removes one of the biggest downsides I felt about ImmutableJS.
Of course, there is still the problem of debugging immutable structures in the console, but I guess we can deal with that for now. Here we are, back to Immutable JS.
Unit tests are great, but only check individual components. To build a more robust test suite, we needed to add tests that validate:
- the Spring configuration
- STOMP destinations, subscriptions and exchanges
- the serialized data in the messages
- all of the above in the context of a real-life scenario
Using @SpringBootTest made it fairly easy to have a server running during unit tests. On top of that, I had to
configure a web socket client to properly exchange Jackson-serialized objects over STOMP.
Since the configuration of such a client was not as smooth as I expected, I created Jackstomp client, which makes it quite straightforward to get a client running with sensible defaults.
Using Immutable JS has proved to be a pain, especially because the cumbersome API is not contained in the reducers but
leaks out in the React components. As far as accessing the data is concerned, I dislike not being able to do it the
native way (myObj.prop), well supported by IDEs. What's more, using strings in such accesses (myObj.get('prop'))
is not refactoring-friendly and it obscures the errors when we make typos.
Using .toJS() in each selector seemed to be a solution to avoid such accesses in React components, but it in fact
destroys performance as the new props are never considered the same as the old ones, and therefore the DOM is always
completely re-rendered.
After reading Alex Faunt's experience and
Richard Feldman's article,
I decided to move to Seamless Immutable, which was our life savior:
it provides immutability with an equivalent mutation API, while keeping native read accesses. Of course migrating to
Seamless Immutable required using redux-seamless-immutable for
seamless-immutable-compatible combineReducers() and routerReducer() functions.
As frontend development was starting, we felt the need for a better API doc than a plain shared Google Sheet. The best doc is an up-to-date doc that stays so. The easiest way I found to keep it up-to-date is to generate it.
Now multiple tools were available, but none really for websocket APIs documentation. Most of the tools out there are specialized in REST APIs. A very popular one is Swagger, but it can't be used yet for Websocket communications.
My attention then went to JsonDoc, which looked promising but with the same drawbacks: no
support for publish/subscribe mechanisms like websockets. Contributing to the project seems easier than contributing
to Swagger, and I already could tweak a little bit the source code of JsonDoc to support @MessageMapping methods.
That being said, Fabio Mafioletti does not seem to have a lot of time available for collaboration in implementing this support, so I might have to release from my own fork of the project, or create a new project based on JsonDoc.
@victorchabbert based the frontend architecture on mxstbr's react-boilerplate (thanks Max!). The recommended structure for his boilerplate is to group files by features. As such, in a feature folder, you would find reducers, actions types and creators, selectors, sagas as well as containers.
At first, this choice seemed suitable for our project but with time, we found out that it was causing us headaches because of the amount of shared data and logic we have between containers and pages. This is when Victor remembered Matteo's blog post on a maintainable project structure from last october. He highly recommends reading it.
He refactored our code to use the Ducks principle (the word comes from reDUX) and can already see the ease to
import all redux specific files.
Some nice plugins allowed for bundling, minification etc. directly from Gradle, but I wanted to make use of the local
package.json scripts to be consistent with the frontend development workflow.
That way, the global build would not bring any surprise compared to the standalone frontend build.
In order to take advantage of Create React App's genericity, I could not temper with the webpack config in order to
customize the source path to src/main/js, like in the example of
Spring React Boilerplate. It eventually made more sense to
separate the react app sources in a subproject, following the example of
Geowarin Boot React.
Using the Gradle Node Plugin and its Yarn tasks allowed for an easy setup
of the react frontend build. It already solved most of the frontend build problems: download node/npm/yarn, install
dependencies, bundle the JS using the frontend tools... We just had to include the result of the frontend build into the
static folder of the backend jar, by customizing the backend's build.gradle.
First problem with Heroku: it now tried to run gradle stage as it did not recognize a Spring Boot application anymore
in the root folder. I had to manually override the Gradle command in the app settings on Heroku to run a gradle build.
Second issue, the gradle build command actually runs the tests, which in the frontend subproject starts in watch mode,
because not on a CI server (checking the environment variable CI=true). Also, the tests were already run by Travis CI,
so we didn't in fact need them there. The final command for Heroku was gradle assemble, which does the job.
Third problem, the default web process for Heroku's dyno looked for jars in the {projectRoot}/build/lib folder. I had
to add a Procfile to manually specify the backend subproject in the process command:
web: java -Dserver.port=$PORT $JAVA_OPTS -jar backend/build/libs/*.jar
We decided to put the react stuff into src/main/js as it matches maven/gradle project structure conventions. As
src/main/java contains the Java sources, why not put the JavaScript sources in src/main/js?
The technical choices were mostly guided by @victorchabbert, as I had little experience with React, and the frontend stuff in general.
Technical decisions:
- Create React App: because the front-end world evolves too fast to keep up with the tools!
- Yarn : for its performance and reliability compared to npm
- Redux: because it seems to solve the state management problem really well, and sagas look promising
- Immutable.js: because Redux likes pure functions, and Immutable.js does a great job at enforcing it
So far, no special build including the produced static inside the webapp's Jar. Frontend and backend can be developed independently and both the frontend dev server and the spring boot server can run locally and communicate.
Technical decisions:
-
Websockets: as we needed server pushes to reach all clients when a player plays a card, we chose WebSockets as main transport. HTTP/2 didn't seem to have sufficient browser support at the moment, and long polling is not a very clean solution to this problem. Now, since we have a permanent TCP connection because of the WebSocket, we may as well use it for all actions (even those who would naturally follow a request/response mechanism) for simplicity and to avoid HTTP's overhead.
-
Spring Boot Websocket: because of my personal preference for (and ease with) the Java language, and because it is really quick to setup, Spring Boot was our choice as backend. It has the advantage to run without a container, with a simple
main()method which also pretty cool. This gave us the quickest start as far as the backend was concerned. -
STOMP: coming with Spring official support, it was quite natural to use STOMP over WebSockets as a subprotocol when setting up Spring Boot WebSocket. It allows for an easy management of the payload and provides some sort of standard for the publish/subscribe mechanism.