Reduce bundle size by removing unnecessary files from node_modules

[matuscongrady]

This is a Feature Proposal

Description

• What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.

node_modules include a lot of unnecessary files (license files, test files, CI/CD config files, build config files etc.).
These files are absolutely useless and are not needed at runtime.
They increase the bundle size, the time it takes to zip bundles and upload them to S3.
We can save a significant amount of time (and even money to some degree) by removing them.
Yarn has a functionality called "autoclean". It removes files specified by a glob pattern from node_modules https://yarnpkg.com/en/docs/cli/autoclean
We could use yarn's autoclean funcionality or remove the unnecessary files ourselves.

I think this should be a configurable option.
Maybe we could allow executing an "autoclean" script after packager install it's depencencies? Maybe here?

serverless-webpack/lib/packExternalModules.js

Lines 329 to 332 in 2139384

	return packager
	.install(compositeModulePath, this.configuration.packagerOptions)
	.then(() => this.options.verbose && this.serverless.cli.log(`Package took [${_.now() - start} ms]`))
	.return(stats.stats);

• If there is additional config how would it loo

Maybe something like this?

webpack:
  packagerOptions:
    postInstallScripts:
      - yarn autoclean --force

[HyperBrain]

Hi @mcongy ,

did you try

custom:
  webpack:
    packagerOptions:
      scripts:
        - yarn autoclean --force

The scripts option is made especially to execute vustom scripts after installation of the node modules

[matuscongrady]

Hi @HyperBrain

Sorry for reacting so late.

I didn't realize I can specify post-installation script. It could work very well.
The problem is, in order to be able to "yarn autoclean", the .yarnclean file has to be in current working directory.
In case of packaging individually, I'd have to copy the .yarnclean file into all the folders I want to autoclean.

Do you have an idea, what would be the best solution to doing this?
Or maybe the support for this should be implemented as a separate plugin?

Thanks!

[jabas06]

This worked for me:

  webpack:
    webpackConfig: "webpack.config.js"
    packager: "yarn"
    packagerOptions:
      scripts:
        - yarn autoclean --init
        - yarn autoclean --force
    includeModules: true

Hi @mcongy ,

did you try

custom:
  webpack:
    packagerOptions:
      scripts:
        - yarn autoclean --force

The scripts option is made especially to execute vustom scripts after installation of the node modules

[khaledosman]

if you still have problems with your package size try this out:

I was able to reduce my packaged zip files sizes almost by more than half by using yarn clean for the included node modules and having a more strict .yarnclean config.

This config is a combination of the default .yarnclean file created via yarn autoclean --init and node-prune's config. In my root project I setup my .yarnclean file as follows: https://gist.github.com/khaledosman/e145fffc2de691e780c7da246e2b3da4

then in serverless.yml I use the root .yarnclean file for the included node_modules

  webpack:
    webpackConfig: ./webpack.config.js
    includeModules: true
    forceExclude:
      - aws-sdk
    packager: "yarn"
    packagerOptions:
      scripts:
        - yarn autoclean --init
        - rm .yarnclean && cat ../../.yarnclean >> .yarnclean && echo '\n*.ts' >> .yarnclean
        - rm -rf node_modules/aws-sdk
        - yarn autoclean --force

Then I forcefully remove aws-sdk from the package since it was still being included indirectly via aws-xray-sdk which states it as a dependency in package.json increasing my package size dramatically.

[vsheyanov]

Any solutions for npm that someone has in mind?