[quilt] Build Quilt using sewing-kit-next #1538

b-ryu · 2020-06-24T23:07:45Z

Description

Related to #1522

This PR integrates sewing-kit-next into Quilt's builds.

yarn build
yarn test
yarn lint

Some notes

I've noticed some intermittent issues with Watchman when trying to run tests, especially right after a fresh build. Usually running watchman shutdown-server and dev up after restarting the terminal after a while fixes things. I suspect it may have something to do with the volume of built files (note: look into configuring Watchman)
This PR bumps the maximum memory allowed to VS Code's builtin TS plugin. Adding a tsconfig.json to the workspace root causes the plugin to traverse the entire workspace when you open a .ts file not included in one of the referenced projects/packages (opening a .ts file in a project/package loads the TS for that package only, as before). This behaviour also occurs in other similarly structured workspaces, the difference being that Shopify/quilt is relatively large so the plugin appears to run out of memory halfway, causing it to hang and crash repeatedly. Increasing the memory limit seems to fix the problem, though it might be something to look into further

cejaekl

I've only looked at the i18n files (on "behalf" of intl-globalization) but see no cause for concern there. OK by me, for what that's worth. :)

ismail-syed · 2020-07-22T18:24:02Z

.eslintrc.js

-    'plugin:@shopify/react',
-    'plugin:@shopify/jest',
-    'plugin:@shopify/prettier',
+    'plugin:@sewing-kit/typescript',


Where do these configs live?

Here, I believe.

Found it, https://github.com/lemonmade/sewing-kit/tree/main/packages/eslint-plugin/src/config

Not related to quilt. It's interesting that there's a@sewing-kit/eslint-plugin vs @sewing-kit/plugin-eslint. One is a plugin for sewing-kit and one is an eslint plugin.

Sorry, github is being wonky with this big PR. Comment thread didn't' refresh and the page is freezing.

You should keep the Shopify ESLint plugins, I only added a custom ESLint plugin in sewing-kit to get rid of a few rules I hate, and show how it could be authored with TypeScript (unlike our actual plugin, that's all in JS)

ismail-syed · 2020-07-22T18:49:26Z

.vscode/settings.json

@@ -23,6 +23,7 @@
    "build": true,
    "**/*.js": true
  },
+  "typescript.tsserver.maxTsServerMemory": 8000,


Can you share some re-producible steps for the issues you had with VSCode, please? I can try to take a deeper look.

Sure thing:

Pull this branch down and open it in VS Code

Get rid of the typescript.tsserver.maxTsServerMemory setting fix

Close all tabs in and close VS Code

Open Quilt in VS Code

Open sewing-kit.config.ts in the workspace root

On my editor at least, the builtin TS/JS plugin hangs and never stops initializing, as noted in the bottom toolbar. The plugin eventually restarts itself a couple times and then crashes, leaving a warning popup

In VS Code you can use the command >Open TS Server log to see the plugin logs. For me, the logs just stop in the middle of loading a package around halfway through.

I was able to repro this on first attempt. I tried a few hacks to see what would happen (eg. move the root tsconfig.json to packages/tsconfig.json). That seemed to help. I reverted my changes and cleaned out my git env and I'm not able to reproduce this anymore :/

It might have something to do with my own local setup then 🤔 @lemonmade @TheMallen is this an issue for you guys as well?

I'll try it out after our 1:1

ismail-syed · 2020-07-22T19:12:54Z

config/sewing-kit/index.ts

+  return createComposedProjectPlugin<Package>('Quilt.Package', [
+    javascript(),
+    typescript(),
+    react(),


We have some quilt packages that don't use any react (eg. address-consts). Do they still need the react plugin?

Probably not, I can make that configurable via arguments instead 👍

ismail-syed · 2020-07-22T19:22:02Z

config/sewing-kit/index.ts

+  createProjectTestPlugin,
+  createProjectBuildPlugin,
+} from '@sewing-kit/plugins';
+import {react} from '@sewing-kit/plugin-react';


Something to keep in mind for the future. It would be nice to have some of these common plugins re-exported from @sewing-kit/plugins so devs can do something like:

import { // other stuff react, javascript, typescript, } from '@sewing-kit/plugins';

instead of

import {react} from '@sewing-kit/plugin-react'; import {javascript} from '@sewing-kit/plugin-javascript'; import {typescript} from '@sewing-kit/plugin-typescript';

@lemonmade do you foresee any issues with this?

I agree the core plugins are annoying to remember what all the package names are

This is what composed plugins are for (e.g., https://github.com/lemonmade/quilt/blob/main/packages/sewing-kit-plugins/src/index.ts#L29), so I don't think it's needed.

ismail-syed · 2020-07-22T19:30:54Z

config/sewing-kit/index.ts

+  };
+}
+
+export function quiltPackage({binaryOnly = true, jestEnv = 'jsdom'} = {}) {


What's this binaryOnly flag for? It's defaulted to true and I don't see it being used anywhere else?

I think I just copied that from Chris' Quilt config. Wasn't sure if we wanted .mjs and .esnext files in our builds. I can get rid of it and hardcode that to be true instead for now.

Let's leave them out for now. Adding support for .mjs and .esnext feels out of scope in validating sewing-kit-next in quilt.

disagree, one of the major benefits of using sewing-kit-next is that it outputs files that are actually tree-shakeable.

I stand corrected. @b-ryu, sorry for the confusion. We'll need this.

ismail-syed · 2020-07-22T20:15:22Z

packages/address-consts/package.json

@@ -24,7 +24,7 @@
  },
  "homepage": "https://github.com/Shopify/quilt/blob/master/packages/address-consts/README.md",
  "files": [
-    "dist/*",
+    "build/*",


Is there a reason why these are all changing to build? Anyway we can keep it as dist?

Looking through the plugin code I don't believe it's a hard requirement, but I think it's the preferred default. I'll see if keeping it as dist changes anything 👍

Something to verify. Do to generated root level entry points respect the output folder in the tsconfig.

Update the output folder in packages/web-worker/tsconfig.json to dist

Run a build

Verify that all the root level entry points generated for the following include a ./dist

pkg.entry({root: './src/index'}); pkg.entry({name: 'babel', root: './src/babel-plugin'}); pkg.entry({name: 'webpack', root: './src/webpack-parts'}); pkg.entry({name: 'worker', root: './src/worker'});

Did some digging. Apparently setting outDir in tsconfig.json only affects TS output. It seems that other outputs are hardcoded to go into the build folder (the generated entrypoints point to the proper respective folder).

Thanks for digging into this. I'm ok with standardizing on our tooling to use build as the default output dir. We'll just need to make sure we update all our cases where we hard-coded dist. @shopify/polyfills comes to mind. It's hurt us in the past.

We'll probably want an escape hatch out of this in the future though, unless it's already possible today (cc/ @lemonmade)

ismail-syed · 2020-07-22T20:18:13Z

packages/address-consts/package.json

@@ -3,8 +3,8 @@
  "version": "2.0.1",
  "license": "MIT",
  "description": "Constants and types relating to `@shopify/address`",
-  "main": "dist/src/index.js",
-  "types": "dist/src/index.d.ts",
+  "main": "index.js",


Is outputting an index.js in the root a defaults from sewing-kit next? I personally like when all the build output is in one folder that way it's easier to undo/clear caches. Probably worth a discussion with the team though.

I can look into this; I think it's the default, not sure if it's configureable (based on PackageEntryOptions in SK-next). FWIW, I added a yarn clean script to package.json that clears away any built files/entrypoints.

After seeing the use case for multiple entries into a package (eg. web-workers), this sounds like a reasonable tradeoff.

pkg.entry({root: './src/index'}); pkg.entry({name: 'babel', root: './src/babel-plugin'}); pkg.entry({name: 'webpack', root: './src/webpack-parts'}); pkg.entry({name: 'worker', root: './src/worker'});

This whole automation of a package entry is really neat!

Yeah, I don't love littering the root of a package with all its entry files, but unfortunately there's not really any other way to do multi-entry packages consistently.

ismail-syed · 2020-07-22T20:31:46Z

tsconfig.json

@@ -0,0 +1,76 @@
+{


Does sewing-kit-next expect the tsconfig.json to live in the root dir?

Not certain, but I believe so? I remember getting some errors about not finding a quilt/tsconfig.json when playing around with with different tsconfig.json setups.

ismail-syed · 2020-07-22T20:49:05Z

package.json

+    "build": "sk build",
+    "check": "lerna run check",
+    "ci:lint-docs": "yarn generate docs && test -z \"$(git status --porcelain)\" || echo 'The root README has not been updated. Run `yarn generate docs` in the root of your quilt directory and try again.'",
+    "clean": "rimraf './packages/*/build' './packages/*/*.d.ts' './packages/*/*.js' './packages/*/*.esnext' './packages/*/*.mjs' '.sewing-kit'",


Clearing all these files manually feels like a guessing game to me. We should rely on sewing-kit to handle this.

There are a number of files that match the following './packages/*/*.d.ts' './packages/*/*.js' that are removed in this PR. Did any of these get accidentally picked up by a yarn run clean or are all of them covered by sewing-kit's pkg.entry automation?

I don't expect an answer for this 😅 Just more things to consider during our validation of sewing-kit-next.

I think I deliberately got rid of some packages/*/*.d.ts and packages/*/*.js files (also added them to .gitignore) since we didn't need them anymore with our new generated entrypoints, but it's probably worth it for me to go back and double-check that they were all actually just entrypoints 👍

ismail-syed

Haven't looked into the VSCode issues yet. This is looking awesome so far! Love the automated entry points with pkg.entry. I'm afraid we're hitting a performance tripwire.

	Before (`c3baa5f`)	After (`3505c07`)
Cold	402.35s	634.54s
Warm	1.56s	148.41s

lemonmade · 2020-07-23T18:59:44Z

.eslintrc.js

-    'plugin:@shopify/react',
-    'plugin:@shopify/jest',
-    'plugin:@shopify/prettier',
+    'plugin:@sewing-kit/typescript',


You should keep the Shopify ESLint plugins, I only added a custom ESLint plugin in sewing-kit to get rid of a few rules I hate, and show how it could be authored with TypeScript (unlike our actual plugin, that's all in JS)

lemonmade · 2020-07-23T19:00:47Z

.gitignore

+packages/*/*.node
+packages/*/*.mjs
+packages/*/*.d.ts
+packages/*/*.js


The thing to watch out for here is that we sometimes nest .eslintrc.js or similar in subdirectories, and we do generally want to include those files.

lemonmade · 2020-07-23T19:01:32Z

.travis.yml

-        - yarn build --verbose
-        - yarn test:ci --testPathPattern react-server address
+        - yarn build
+        - yarn type-check


build effectively includes a type-check, because part of the build is generating the TypeScript outputs

lemonmade · 2020-07-23T19:03:09Z

config/sewing-kit/index.ts

+      hooks.configure.hook(hooks => {
+        hooks.jestEnvironment?.hook(() => jestEnv);
+
+        hooks.jestConfig?.hook(config => ({


There are more specific hooks for jestTransforms and the watch ignore patterns

lemonmade · 2020-07-23T19:05:10Z

package.json

+    "release": "yarn lint:changelogs && lerna publish",
+    "release-beta": "lerna publish --dist-tag beta",
+    "pretest": "yarn build",
+    "test": "yarn _test",


I think this may be changing the default test command to be in watch mode, which may not be desirable

lemonmade · 2020-07-23T19:05:53Z

packages/address-consts/package.json

@@ -3,8 +3,8 @@
  "version": "2.0.1",
  "license": "MIT",
  "description": "Constants and types relating to `@shopify/address`",
-  "main": "dist/src/index.js",
-  "types": "dist/src/index.d.ts",
+  "main": "index.js",


Yeah, I don't love littering the root of a package with all its entry files, but unfortunately there's not really any other way to do multi-entry packages consistently.

lemonmade · 2020-07-23T19:10:21Z

packages/react-cookie/package.json

-  "main": "dist/src/index.js",
-  "types": "dist/src/index.d.ts",
+  "main": "index.js",
+  "types": "index.d.ts",


For packages to get the full benefits of the different build outputs, you'll eventually need to also add esnext/ module entries to package.json (and, if you're interested, the future-looking exports map). Example: https://github.com/lemonmade/quilt/blob/main/packages/quilt/package.json#L10-L32

Get rid of extra TS version

We're no longer using the TS builds

Maybe discuss this approach

b-ryu · 2020-08-11T15:18:57Z

packages/polyfills/src/index.ts

@@ -32,7 +32,7 @@ export const polyfills: {[polyfill: string]: PolyfillDescriptor} = {
 export function mappedPolyfillsForEnv(
  env: 'node' | 'jest' | string[],
 ): {[key: string]: string} {
-  const prefix = `@shopify/polyfills/build/cjs`;
+  const prefix = `@shopify/polyfills`;


I'm wondering if it would be a terrible thing to just have the CommonJS builds for @shopify/polyfills and just have this map to build/cjs instead. Not sure what the impact on treeshaking would be, since this package has sideEffects set to true anyway.

b-ryu · 2020-08-13T21:41:45Z

Since the SK-next project is still in the exploration phase, we've decided to hold off on merging/releasing this until the build phase. Closing this PR for now.

b-ryu force-pushed the 1522-sk-next branch 3 times, most recently from d3b5765 to e25b8e7 Compare June 30, 2020 21:02

b-ryu force-pushed the 1522-sk-next branch 2 times, most recently from 8df6187 to 29c9333 Compare July 9, 2020 16:51

b-ryu force-pushed the 1522-sk-next branch 3 times, most recently from 5fbbc8f to 9957891 Compare July 15, 2020 00:34

b-ryu changed the title ~~[WIP][quilt] Build Quilt using sewing-kit-next~~ [quilt] Build Quilt using sewing-kit-next Jul 15, 2020

b-ryu marked this pull request as ready for review July 15, 2020 01:38

b-ryu force-pushed the 1522-sk-next branch from d871fcd to 8585f0e Compare July 20, 2020 14:01

b-ryu requested review from ismail-syed, marutypes, michenly and lemonmade July 21, 2020 14:22

shopify-admins requested a review from a team July 21, 2020 14:22

cejaekl approved these changes Jul 21, 2020

View reviewed changes

b-ryu force-pushed the 1522-sk-next branch 2 times, most recently from c382968 to 3505c07 Compare July 22, 2020 17:52

ismail-syed reviewed Jul 22, 2020

View reviewed changes

lemonmade reviewed Jul 23, 2020

View reviewed changes

Brian Ryu added 17 commits August 10, 2020 16:14

Resolve Babel plugin

09e6ac8

Clean up

b9ec759

Get rid of unused config file

a90b61a

Get rid of unused package

3800d9b

Update "dist"

476bf9a

Use type export

e91936a

Replace index field with dot

90e4791

Get rid of extra TS version

Polish

c369a9b

Add changelog entries

af635bb

Change polyfill root to be CJS build, not TS

512eef3

Fix test

3446887

Exclude non-suite test files

2c3068c

Fix test

b810499

Enforce isolatedModules

c9b69a0

Get rid of tslib

07876a2

We're no longer using the TS builds

b-ryu force-pushed the 1522-sk-next branch from 94fa81b to 1a6fdfc Compare August 10, 2020 20:15

Brian Ryu added 2 commits August 10, 2020 17:42

Fix entrypoints for polyfills package

cfb151b

Maybe discuss this approach

Revert beta publish commits

d68b3f0

b-ryu commented Aug 11, 2020

View reviewed changes

Brian Ryu added 2 commits August 12, 2020 20:24

Clean up

bdbfa7c

Fix script

52125bd

b-ryu force-pushed the 1522-sk-next branch from f5a4cfa to 52125bd Compare August 13, 2020 01:15

Fix changelogs

06d55bc

b-ryu closed this Aug 13, 2020

ismail-syed mentioned this pull request Aug 13, 2020

Release esnext supported quilt packages with @sewing-kit toolchain #1592

Closed

ismail-syed mentioned this pull request Dec 14, 2020

Multi-build support (cjs, esm, esnext) for all quilt packages #1698

Merged

3 tasks

BPScott deleted the 1522-sk-next branch May 22, 2021 00:32

[quilt] Build Quilt using sewing-kit-next #1538

[quilt] Build Quilt using sewing-kit-next #1538

Conversation

b-ryu commented Jun 24, 2020 • edited Loading

Description

Some notes

cejaekl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

b-ryu Jul 22, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marutypes Jul 22, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ismail-syed left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

b-ryu Aug 11, 2020 • edited Loading

Choose a reason for hiding this comment

b-ryu commented Aug 13, 2020

b-ryu commented Jun 24, 2020 •

edited

Loading

b-ryu Jul 22, 2020 •

edited

Loading

marutypes Jul 22, 2020 •

edited

Loading

b-ryu Aug 11, 2020 •

edited

Loading