Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf: increase performance of nx console #1038

Merged
merged 4 commits into from
Mar 12, 2021
Merged

perf: increase performance of nx console #1038

merged 4 commits into from
Mar 12, 2021

Conversation

Cammisuli
Copy link
Member

@Cammisuli Cammisuli commented Mar 10, 2021 •
edited
Loading

This removes JSON5 parsing to use regular JSON.parse. There might be some json files that allow comments, so we have stripJsonComments included as well.

This is the same behaviour that's in core Nx, so if there is any configuration file that has JSON5 specs in it, Nx will fail anyway ☹️

Here's a gif with these changes on a workspace that has 1426 projects with the following specs:
image

nx-console-perf-fix 2021-03-12 13_50_42

@Cammisuli Cammisuli changed the title performance fix perf: use json.parse instead of json5 Mar 10, 2021
@Cammisuli Cammisuli marked this pull request as draft March 10, 2021 22:11
@Cammisuli Cammisuli marked this pull request as ready for review March 10, 2021 22:15
@Cammisuli Cammisuli marked this pull request as draft March 10, 2021 22:24
@Cammisuli Cammisuli marked this pull request as ready for review March 10, 2021 23:34
@Cammisuli Cammisuli changed the title perf: use json.parse instead of json5 perf: use json.parse instead of json5.parse Mar 10, 2021
@Cammisuli Cammisuli changed the title perf: use json.parse instead of json5.parse perf: use json.parse instead of json5 Mar 10, 2021
sandikbarr
sandikbarr reviewed Mar 11, 2021
View reviewed changes
libs/server/src/lib/utils/utils.ts
@@ -131,7 +131,7 @@ export function fileExistsSync(filePath: string): boolean {
}

export function readAndParseJson(fullFilePath: string): any {
return JSON5.parse(readFileSync(fullFilePath).toString());
return JSON.parse(stripJsonComments(readFileSync(fullFilePath).toString()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioned this to Isaac and he wondered if it should also try catch and fallback to JSON5 if the regular parse fails

Copy link
Member Author

@Cammisuli Cammisuli Mar 11, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same "technique" used in Nx. If the workspace.json file includes anything related to json5 (trailing commas, etc) then Nx itself won't work anyway ☹️

I still want to work on this though, I think making this asynchronous will help with other things.

Edit: Nope, caching makes things better 😅

@isaacplmann
Copy link
Contributor

isaacplmann commented Mar 11, 2021 via email

@Cammisuli Cammisuli changed the title perf: use json.parse instead of json5 perf: increase performance of nx console Mar 12, 2021
sandikbarr
sandikbarr approved these changes Mar 12, 2021
View reviewed changes
Copy link
Contributor

@sandikbarr sandikbarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@nx-cloud
Copy link

nx-cloud bot commented Mar 12, 2021 •
edited
Loading

Nx Cloud Report

CI ran the following commands for commit 4f7769c. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch

Status Command
#000000 nx run-many --all --parallel --target=build
#000000 nx run-many --all --parallel --target=lint
#000000 nx run-many --all --parallel --target=test

Sent with 💌 from NxCloud.

@Cammisuli Cammisuli merged commit e8f3258 into nrwl:master Mar 12, 2021
@Cammisuli Cammisuli deleted the performance-fix branch March 12, 2021 21:08
Cammisuli pushed a commit that referenced this pull request Aug 9, 2022
@semantic-release-bot
chore(release): 1.0.0 [skip ci]
48c84ec 
# 1.0.0 (2022-08-09)

### Bug Fixes

* add authentication boolean to settings ([a2a1a3f](a2a1a3f))
* add dotenv as a dep ([691cc7a](691cc7a))
* artifact name on windows ([eeb5404](eeb5404))
* await workspace reset ([2e9d880](2e9d880))
* build ([#754](#754)) ([fc87f6b](fc87f6b))
* **build:** add EV certificate and condition to clean script ([0d8bac5](0d8bac5))
* **build:** add missing deps ([61d73c3](61d73c3))
* builders should show the values from angular.json ([8aa7c0a](8aa7c0a))
* **build:** fix linux artifact name ([16b9660](16b9660))
* **build:** fix windows build ([a1df664](a1df664))
* **build:** Update graphql types ([#872](#872)) ([91fbe86](91fbe86))
* bump up electron version to fix security vulnerability ([a1c3f6f](a1c3f6f))
* Bundle visualization accounts for differential loading ([2bc8ae0](2bc8ae0))
* check if dependency path type is directory with bit flag ([#1267](#1267)) ([e75daac](e75daac))
* command status should return an empty array ([b3b0211](b3b0211))
* **deps:** update dependency @angular/cli to v7.0.6 ([2f62cae](2f62cae))
* **deps:** update dependency apollo-server-express to v2.2.2 ([b680263](b680263))
* **deps:** update dependency drivelist to v6.4.3 ([87cead5](87cead5))
* **deps:** update dependency electron-updater to v3.2.3 ([5373827](5373827))
* **deps:** update dependency electron-updater to v4 ([#383](#383)) ([2d8b045](2d8b045))
* **deps:** update dependency express-graphql to v0.7.1 ([e04044c](e04044c))
* **deps:** update dependency get-port to v4 ([#381](#381)) ([e8acf5c](e8acf5c))
* **deps:** update dependency node-pty-prebuilt to v0.7.6 ([e4ae6eb](e4ae6eb))
* **deps:** update dependency universal-analytics to v0.4.20 ([b2e52c7](b2e52c7))
* **deps:** update nrwl monorepo to v0.0.17 ([573d687](573d687))
* Do not prompt for analytics ([#757](#757)) ([aca0c4a](aca0c4a))
* **electron:** icon on a mac ([feb99b1](feb99b1))
* **electron:** Use correct cwd for ng-new ([#500](#500)) ([3897b71](3897b71))
* **error-handling:** Give more feedback for common errors ([#881](#881)) ([e343fbd](e343fbd))
* fix auto updater ([5c52096](5c52096))
* Fixed error output of build task. ([#563](#563)) ([827b016](827b016))
* Fixed scripts so that `yarn startdev.prepare` work ([425f8fa](425f8fa))
* handle hyphenation when looking for generator defaults ([#1315](#1315)) ([60bdadf](60bdadf)), closes [#1313](#1313)
* **icons:** Fix icons on linux ([#430](#430)) ([4b1ba6a](4b1ba6a))
* improve performance for retrieving workspace details ([#1295](#1295)) ([f8d32ca](f8d32ca))
* inline copy-to-clipboard icon ([#1220](#1220)) ([5e113df](5e113df))
* **many:** Various small bug fixes ([#700](#700)) ([d0eb2e9](d0eb2e9))
* misc fixes ([b577ca9](b577ca9))
* misc fixes ([bc6cb95](bc6cb95))
* **new-workspace:** Create a sandbox to create new workpaces ([be0722d](be0722d))
* node pty version for vscode ([#524](#524)) ([868b96c](868b96c))
* not all schematic collections get picked up and displayed ([1b98dd9](1b98dd9))
* only reads and normalizes selected schematic options ([#1074](#1074)) ([d2c4bd4](d2c4bd4))
* project creation works with angular 8 ([b7e9818](b7e9818))
* **projects:** Disable generate/run buttons when applicable ([#763](#763)) ([9b7ea68](9b7ea68))
* properly check positional argument for generating with UI ([#1268](#1268)) ([764b95f](764b95f))
* Reduce logging noise and handle schematics without a description ([#750](#750)) ([3d15f03](3d15f03))
* revert electron-builder version to work with electron 2 ([cc2f336](cc2f336))
* run executors that are local to the workspace (ie local executors) ([#1230](#1230)) ([a7f2362](a7f2362))
* run pnp.setup to patch require ([#1240](#1240)) ([919e05d](919e05d))
* scripts section gets removed from package.json ([ea65ec7](ea65ec7))
* **settings:** Set initial value of disabled animations ([#657](#657)) ([f11f2d5](f11f2d5))
* tweak polling constants to improve perf ([#459](#459)) ([93a62ed](93a62ed))
* validation on workspace name to prevent conflicts ([35765c8](35765c8))
* **vscode-dev:** Copy server assets as part of ng build ([#802](#802)) ([fa208e9](fa208e9))
* **vscode:** Make sure that karma.conf.js is in assets folder ([#482](#482)) ([e601c7a](e601c7a))
* **vscode:** More lenient JSON parsing (allow for trailing commas) ([#902](#902)) ([ecbd0ee](ecbd0ee))
* **vscode:** Remove all logging ([#758](#758)) ([3fd4ffc](3fd4ffc))
* **windows:** strip out node-pty-prebuilt for vscode and intellij ([#745](#745)) ([cb104ce](cb104ce))

### Features

* Add commands to add applications and libraries directly ([#1128](#1128)) ([b908c11](b908c11))
* add configuration support for the import plugin ([#1183](#1183)) ([53b09eb](53b09eb))
* add detailed status for build and serve ([765e06e](765e06e))
* add detailed status for commands ([3a0e90c](3a0e90c))
* add different release channels ([21159d8](21159d8))
* add google analytics ([72dbaea](72dbaea))
* add openInBrowser functionality to be used by plugins ([2f2ad75](2f2ad75))
* add prompts to add nx to angular cli ([#1254](#1254)) ([a669307](a669307))
* add support for docs ([0bfae75](0bfae75))
* add support for notifications ([0d5a78f](0d5a78f))
* add support for workspace schematics ([dcaf8a2](dcaf8a2))
* add support for yarn pnp ([#1213](#1213)) ([b6cfdbf](b6cfdbf))
* adds settings page and fixes tests output ([#313](#313)) ([58c7387](58c7387))
* **build:** Add Renovate to automatically create PRs for updating packages ([62de102](62de102))
* **build:** Bring vscode and electron builds more inline ([#481](#481)) ([7cd4fcc](7cd4fcc))
* **build:** Build for linux distros ([3c3abea](3c3abea))
* **build:** Build images for linux ([b5ce360](b5ce360))
* **build:** New GraphQL Code Generator ([#413](#413)) ([6677531](6677531))
* **build:** Reduce external dependencies ([#622](#622)) ([c5a2a52](c5a2a52))
* **build:** Smaller bundles using esnext and minimal assets ([#747](#747)) ([c1fb0e1](c1fb0e1))
* **build:** Update electron / render linux icons ([53c1448](53c1448))
* **build:** Use GraphQL Code Generator and Apollo-Angular template ([6e83dcb](6e83dcb))
* bump up version of extensions ([22243ea](22243ea))
* bump up version of extensions ([b943ebe](b943ebe))
* **ci:** Cache yarn dependencies and default to use yarn for ng commands. ([28402e2](28402e2))
* **ci:** No longer persist workspace ([#548](#548)) ([8a7d33d](8a7d33d))
* **ci:** Run CI on windows too ([#769](#769)) ([dc4a3ca](dc4a3ca))
* dep-graph integration ([4e4d26d](4e4d26d))
* **e2e:** Speed up prepare.e2e ([#544](#544)) ([eb6a51f](eb6a51f))
* **electron:** Build windows portable installer ([#805](#805)) ([619b1a8](619b1a8))
* enable connect plugin ([8341d30](8341d30))
* **formatting:** Update prettier to 1.15 ([#322](#322)) ([c36a5e1](c36a5e1))
* **IDE:** Initial push for webstorm plugin ([#607](#607)) ([503f573](503f573))
* implement concurrent commands ([ab9630f](ab9630f))
* include typescript plugin for imports ([#1177](#1177)) ([9c330ce](9c330ce))
* **lint:** Add style lint for scss linting ([#631](#631)) ([82ceef9](82ceef9))
* list local plugins ([#1274](#1274)) ([2e6fe76](2e6fe76))
* **ng-new:** Allow users to set ng new schematic flags. ([#328](#328)) ([cf1bb36](cf1bb36))
* **node:** Add api endpoints to detect/install node ([756341e](756341e))
* **nvm:** Add support for .nvmrc files ([#583](#583)) ([6c3e975](6c3e975))
* **nvm:** Add toggle to opt-in/out of nvm ([#601](#601)) ([b05e53e](b05e53e))
* nx lsp ([#1316](#1316)) ([ab8031c](ab8031c))
* **output:** Adds visualization for serve and build output. ([#557](#557)) ([672b5ef](672b5ef))
* **output:** Adds webpack analyzer ([#573](#573)) ([786585a](786585a))
* pinned projects ([#478](#478)) ([a47c438](a47c438))
* project graph integration ([#1305](#1305)) ([ebcd217](ebcd217))
* provide json schema for workspace.json ([#1077](#1077)) ([ae35ec5](ae35ec5))
* provide telemetry to console plugins ([a5049b4](a5049b4))
* Rework analytics ([#801](#801)) ([795bd63](795bd63))
* **schema:** Remove divergent schemas in favor of common ones ([#537](#537)) ([caf4ebf](caf4ebf))
* separate the history of commands from "recent" ([7a5524a](7a5524a))
* **server:** Use angular-cli's built in parser ([#913](#913)) ([c43c7a1](c43c7a1))
* **settings:** Toggle to disable animations completely ([#611](#611)) ([56afb95](56afb95))
* Show all actions from project view ([#490](#490)) ([5e09c0e](5e09c0e))
* show generate ui from explorer context menu ([8b45841](8b45841))
* shows argument list and highlights changed and invalid fields in menu ([#1059](#1059)) ([7d8e8e1](7d8e8e1))
* support nx.json extends property ([#1124](#1124)) ([507ce24](507ce24))
* **support:** Initial commit of angular-console-nrwl-extensions integration ([6b62165](6b62165))
* **tasks:** Add UI for background tasks ([b4c3293](b4c3293))
* **tasks:** Kill tasks via ctrl-c ([#327](#327)) ([1d41c69](1d41c69))
* update angular-apollo ([588274d](588274d))
* update extensions ([27b5e1d](27b5e1d))
* update nx to 6.2.1 ([f3e29a8](f3e29a8))
* update nx to 7.3.0 ([9e72d5c](9e72d5c))
* update the depedency diagram ([a0f8106](a0f8106))
* update to cli7 and nx7 ([#312](#312)) ([d6b16e1](d6b16e1))
* update to nx 7.0.1 ([a7bce06](a7bce06))
* update to nx8 and cli8 ([280ea39](280ea39))
* use x-prompt details in nx-console-field ([7537387](7537387))
* Uses source map to display graph data and fixes Windows path issue. ([#585](#585)) ([d80aca4](d80aca4))
* **vscode:** Angular JSON tree provider ([#810](#810)) ([31ceb9d](31ceb9d))
* **vscode:** Expose tree view ([0484fa1](0484fa1))
* **vscode:** Initial release of VS Code plugin ([a250473](a250473))
* **vscode:** Integrate with vscode-ui ([#839](#839)) ([6c02464](6c02464))
* **vscode:** Reuse terminals for other tasks ([#783](#783)) ([9861fe6](9861fe6))
* **vscode:** Support angular workspaces in subdirectories ([#687](#687)) ([2a64c28](2a64c28))
* **vscode:** Support workspace.json for non Angular workspaces ([#926](#926)) ([b3954c0](b3954c0))
* **vscode:** Use vscode tasks ([#792](#792)) ([2245b10](2245b10))
* **vscode:** vscode-ui form inputs ([#840](#840)) ([9c9e75e](9c9e75e))

### Performance Improvements

* asynchronously retrieve all collection info (schematics, generators, builders and executors) ([#1160](#1160)) ([e193eb8](e193eb8))
* decrease activation time ([#1131](#1131)) ([6a23cea](6a23cea))
* increase performance of nx console ([#1038](#1038)) ([e8f3258](e8f3258))
* No longer poll for updates ([#690](#690)) ([198c51e](198c51e))

### Reverts

* Revert "chore(deps): update dependency @angular-devkit/core to v7 (#372)" (#389) ([569cf2b](569cf2b)), closes [#372](#372) [#389](#389)
* Revert "fix: add validation when creating new workspace with conflicting name" ([34e9ac5](34e9ac5))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sandikbarr sandikbarr sandikbarr approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Cammisuli @isaacplmann @sandikbarr