FEATURE: Upgrade yarn to 3.2, TS to 4.6 and replace lerna

[Sebobo]

Resolves: #3115 & #3114

[mhsdesign]

Hi, what is missing for this one?

[Sebobo]

As I said during our call, testing testing testing ;)

And try whether we can use the new plow-js 3.0.0

[markusguenther]

What a change, thanks for the hard work @Sebobo @JamesAlias and @grebaldi

[kdambekalns]

Is .yarn really supposed to be in the repository?

[Sebobo]

Is .yarn really supposed to be in the repository?

Yes. With Yarn 2 you can optionally check in quite a lot of stuff up to the point, where you don't need to install anything form a remote. Therefore achieving exactly the same build for every dev.

[kdambekalns]

Therefore achieving exactly the same build for every dev.

• I still firmly believe in lock files for that. The prupose of yarn.lock should be exactly this, IMHO.
• And having things named cache in version control gives me shivers, totally.

From https://classic.yarnpkg.com/lang/en/docs/yarn-lock/:

All yarn.lock files should be checked into source control (e.g. git or mercurial). This allows Yarn to install the same exact dependency tree across all machines, whether it be your coworker’s laptop or a CI server.

And following https://classic.yarnpkg.com/en/docs/cli/install#toc-yarn-install it should be stable even for normal use to just use yarn install, but to be honest I'd prefer yarn install --frozen-lockfile over having all those files in the repository…

$ du -hs .yarn/*
155M	.yarn/cache
2.2M	.yarn/plugins
2.1M	.yarn/releases

According to https://yarnpkg.com/getting-started/qa#which-files-should-be-gitignored the cache is to be versioned for Zero-Installs, so I reckon we use that philosophy now? Not sure, since we don't have a .pnp.cjs file , so it seems we are not using "Plug'n'Play to resolve dependencies via the cache folder".

I am still a) unsure about all that and b) not entirely convinced.

---

Rant on.

If "the main source of entropy from the equation: Yarn itself." is what needs to be solved, then I wonder why Composer can do it. It installs the exact set of packages/version requested in the lock file. If that's not possible, it fails. To me that seems to logical and easy, that I just don't get the JS/node/… way of dependency management…

Rant off.

[mhsdesign]

i agreed with the chachy thing, because it makes stuff faster no?

[Sebobo]

@kdambekalns during the sprint in Dresden we decided to test yarn 2 (with its new features) and pnpm in parallel and choose the one that works best for us.
Yarn 2 won due to speed, compatibility and simplicity and we also decided to test the pnp feature for faster installs and less errors which both were a problem in the past when someone wanted to work on the ui.
Yes, the lock file in yarn classic was already a big improvement in comparison to npm but it wasn't perfect.

We didn't fully switch to pnp with zero-install (therefore no pnp.cjs) yet as there are still some small issues with peer dependencies. There is also a comment in the .yarnrc.yml. We fixed most of that but not everything. But even so when you download the ui and build it, yarn will not resolve & download anything and be done very quickly (works without internet). This also makes building stages for testing much faster. pnp will speed that up further.

The 160mb are bad but we added the necessary configurations to the composer.json and .gitattributes so the cache and other yarn stuff is not included in tagged ui releases and doesn't slow down normal composer installs. I tested and verified this in another repo with the same settings.

So far none of us who worked on this branch encountered any issue with this config as to my knowledge. If anyone finds a valid reason why not to use I'm open for discussion.

Package management in the JS world is a bit of a different beast than in the happy PHP world.
And getting rid of the install step in many cases for people who want to work in the UI sounds great to me.

I'm now checking if we can finally fully switch to pnp and check the finding to give better feedback on that topic.

[kdambekalns]

Ok, thanks for the explanation. If that's the best one can get in JS-land, then I'm not going to block it… 🤷‍♂️

Not blocking this…

[markusguenther]

So cool that we now have an up-to-date stack and get rid of four vulnerabilities. It looks so much better than a red flag with the four ;)

Now we can start to upgrade the rest of the dependencies :D

[mhsdesign]

since this pr, the unittest wont work on ci ... see #3256