[Enhancement] Webpack and friends as alternative for underlying build/bundling system.

[TheLarkInn]

🐘 Alright its time to address the elephant in the room. 🐘

Lets talk about Webpack integration

CLI Is incrediballz, but SystemJS is a less-than-desirable and less-than-configuration-friendly experience.

Use Webpack to have an awesome experience for all environments

CLI was built to make rapid development easy and awesome with Angular and I don't think it should stop there. Out of the box, CLI users should have a bundling system in place that automagically handles giving them a production ready bundle of their Angular code that they spent the tireless 20 command lines to boilerplate.

Things should be easy to configure

Additionally, handling external libraries, build configuration, async module loading, source mapping, code splitting, code sharing, beautifying, etc. should be EASY to reconfigure if a user wants to. Currently it isn't.

Shouldn't replace SystemJS

I do not think that we should rid of SystemJS entirely, however I do believe that it is important that we provide an alternative for people to use. (SystemJS has provided very successful for plunks and apps no larger than 300kb)

ng init --use-webpack

Adding a simple flag like ng init --use-webpack might be a tolerable alternative that people would be more than happy to use.

There are great repo's out there for a starting point

I think @gdi2290 @Foxandxss 's starters are community wide accepted examples of multi environment webpack setups that would be perfect as a starting point. Plus when Weback 2 ships from beta, we'll be able to leverage tree-shaking, async module loading, etc.

I can probably help too

I will add to this issue a more concrete proposal on how Webpack integration might be setup at a high level. (Although really the only thing the user should have to do is ad the --use-webpack flag and everything else should feel or work the same.

@Brocco @hansl we can communicate about this more at length but I'd not only like to hear your thoughts, but community thoughts and opinions as well. Doing this integration the right way should pave the way for giving people the option to have other integrations the right way too.

UPDATE: to add requirements.

Requirements

• existing preprocessor support
  • less
  • sass
  • compass (?)
  • stylus
  • postcss (with plugins?)
  • style sourcemaps
• tree-shaking and dead code elimination
  • TS2.0? es6 => es5
  • Babel?
  • tsickle+gcc@advanced?
• cacheable vendors chunk w/ cache busted main app
• fast build and serve
  • incremental build and in memory fs cache
  • multicore/cluster/forked worker support
• support for existing projects
  • migration path (if needed)
  • test against test-app
  • test against material 2
• 3rd party module support
  • should be as simple as npm install w/ import in app.
  • CommonJS, CommonJS2, AMD, ES6, UMD
• testing support
  • karma integration
    • configurable karma.conf yet bundle 'protected'
    • w/ Istanbul coverage
  • e2e w/ protractor
• JS sourcemaps support
  • pre/post trans?
• dynamic on demand bundles
  • System.import support which creates async dynamic bundle
• Routing
  • Async Route Handling automagically?
• ngc support
  • if not ts.LSH support, build step
  • should not be slow
• asset support
  • assets copied over in distribution bundle
  • automagically resolve images
    • if under x size, base64 and inline image
    • if over x size, serve asset inside of assets folder
    • support for fonts
    • inline SVGs?

[SethDavenport]

+a million. Jokes aside I'm down to help

This is what we've been using for a few months now: https://github.com/rangle/angular2-redux-starter, https://github.com/rangle/angular2-starter

[PatrickJS]

[nickroberts]

Love. Definitely would like to have the option to use Webpack. @TheLarkInn's talk at ng-conf really got my juices flowing. I'm just starting to learn Webpack in more depth, and it seems like a great option.

So far, the SystemJS module loader works great, but seems a bit large (and a little slow) for development purposes.

That being said, I'm really excited about the offline template compiler using Rollup, based on what @robwormald demoed at ng-conf.

Exciting times!

[nickroberts]

Also, I was thinking that this functionality could probably be separated out to a plugin (or addon, as Ember calls it). This plugin could house all of the blueprints that would be needed to get things running.

You can already generate an addon via ng addon <addon-name>. I'm not entirely sure how useful it is, at least for angular-cli yet, though.

Perhaps the ng new myApp --webpack option could automatically check for this new addon, and either use it, or fail and provide instructions on how to install it.

Just some thoughts...

[mattdistefano]

FWIW I don't care about webpack integration per se, but parity with webpack's feature set would be nice to see. I can't claim much expertise here, but my understanding is SystemJS is actually superior in a couple aspects, namely that it is using a standards(ish)-based module format and loader runtime, and provides an easy path forward to "unbundle" for http2. Webpack 2 may address some of that, but fundamentally it seems like SystemJS is a module loader first, and bundler second, whereas Webpack is a bundler first, and loader second.

That said - I'm coming from a webpack background, and, trying out the CLI a week or two ago, I found myself missing the non-JS loading/dependency tracing that webpack does (i.e. being able to import a SCSS file and knowing that webpack will compile it and add any referenced assets - images, fonts, etc - to my dist output), along with the production build file name hashing. I think SystemJS itself actually supports both features, so this may just be a documentation issue?

Likewise, I logged #893 about this already, but I think HMR is going to be very important (as in, important enough that if webpack supports it and CLI doesn't, lots of devs are going to stick with webpack).

[SethDavenport]

My point of view is that I want a bundler, not a module loader. A module loader might makes sense once HTTP2 is mainstream but I'll believe in HTTP2 when I actually see it. In HTTP1 module loaders are a solution looking for a problem IMO.

The non-JS bundling is actually my favourite feature of webpack. Suddenly with chainable loaders and PostCSS we have a really nice CSS workflow - on par with Babel.

[mattdistefano]

Right, but if you could have dynamic module loading and bundling today + painless unbundling for HTTP2 tomorrow, that would be the ideal scenario, yeah? And from my understanding of it, that's basically what SystemJS does, albeit with maybe some more configuration overhead than webpack. IIRC the CLI is already using the bundling features for prod builds too.

More broadly, I think the point of the CLI, to at least some extent, is to abstract the details of the build tooling, test runners, etc. So that's why I say I don't care about webpack vs. SystemJS vs. whatever - if the CLI can incorporate the useful features of webpack without incorporating webpack itself, that's fine with me.

[Foxandxss]

The problem I see @mattdistefano is that even if the CLI abstract you from System.js / webpack / foo. You still need to configure extra tools for your own needs. There you can't avoid to touch System.js / gulp / webpack.

[mattdistefano]

@Foxandxss Yeah, I'd still expect to have to tweak it to some extent. I guess my point is just that "webpack has some features the CLI lacks today" doesn't necessarily lead to "we must support webpack".

[TheLarkInn]

@mattdistefano Mind you I did say "Webpack and friends". I just believe the most popular bundler and build tool would be a great one to start with.

[SanderElias]

Point is, that in the current situation (read http2 not widely available) bundling is not an option, but rather mandatory. Even in development it's hurting. Compare an systemJS hello-world with an webpack one. The waterfall in the systemJs case does really hurt. Even more if you need to work on a networked connection!
(systemJS 293 resources vs 4 to 5 for webpack)
So, a webpack option for the CLI is something we really need.

Then there is the point that, with so many developers, opinions differ. Are we really in a position that we want to exclude a lot of them, simply because they don't like systemJS?

And while we are on this subject of integrating another loader/bundler/packer/whatever, perhaps it's an idea to cater for future expansions in this area. For example RollupJS is also approaching. And I highly doubt that will be the last invention.

[mattdistefano]

Yeah but, again, there may be ways to solve those problems within the existing CLI tooling. That's what I would like to understand better - how many of these issues can be attributed to fundamental limitations of the CLI's current tech, and how many are really just features that haven't been implemented yet, or bugs that can be fixed?

[TheLarkInn]

If we implement this the right way we should be able to add system bundler as an option also in the future.

[mattdistefano]

I guess I'm still hung up on why there needs to be an option at all. I'm not against webpack. I use it in production today and enjoy it and it's my team's choice for angular 1 and 2 apps in the near term.

But, for me, a big appeal of the CLI (and angular2 in general) is that it is opinionated. About project structure, architecture, tooling, etc. So when we start saying that, you know, this and that aspect of the CLI's internals need to be configurable so you can use whatever tooling you want, that's a big red flag for me. Because it means that I can't just sit down and use the CLI - I first have to learn about and choose between various bits of build system minutiae. It means that, when I have an issue, I can't just go find a solution - I'll have to wade through dozens of blog posts and SO threads that are almost but not quite useful because they address toolchain x while I opted for toolchain y. And I can only assume it complicates development of the CLI itself quite a bit.

So, again, for me, if these issues can be addressed without introducing that sort of fragmentation into the CLI, that would be the better option.

[kylecordes]

I am very excited about the possibility of CLI reaching a level of completion and polish that it "just works" for the great majority of all cases. Where even someone without much knowledge of these things (knowledge I have spent many months absorbing, and would prefer to forget) can get both an efficient development cycle and efficient production build. Where this issue tracker starts to get boring because it all "just works".

I think it will be most wise to achieve that before digging into swapping out the underlying toolset (always or optionally) with webpack. Webpack a great tool, just in terms of enabling mass, fast A2 adoption, we need one path that works great, more than N paths in various states of completion.

[SanderElias]

@kylecordes, While I agree with a large part of your sentiment, I stil believe supporting webpack asap is important to the CLI project.

While SystemJS is a great tool, it works very poorly if you have a slow connection to your storage(hdd). There are a lot of developers that need to work over an networked connection. In this situation, a save&reload cycle can easily take up over 10 seconds. Webpack, in the same situation does the same in under 2 seconds.
This alone is enough reason to support webpack, aside from all the other benefits.

For the end user of NG-CLI it's (or at least should be) a one-time setting. Well, until the time they are ready to deploy that is.

Bottom line is, if the CLI is not supporting webpack, it is in it's current state almost unusable for a large group of developers. I don't think that's the idea for the official angular toolkit....

[kylecordes]

@SanderElias I agree completely that performance is a big deal. I will think about this some more and try to suggest some ideas on how to improve it. I have found in our work here that webpack is also not the most performant way to work during development, but I have more to say about it then can reasonably go in this somewhat related issue discussion.

[mattdistefano]

Maybe I'm missing something, but given that SystemJS can bundle, would it not be simpler to expose an option to use bundled output for development too? For example, I just commented out the bundling's production environment check and got bundled output with ng serve. I'm not saying that's necessarily the fix (perhaps bundling isn't performant at scale, perhaps sourcemaps get messed up, obviously we're still missing non-JS bundling support, etc) - point is just that there's potentially a solution to be found within the existing tooling.

[mattdistefano]

I should add, HMR (which IIRC SystemJS, or at least JSPM, already supports) would also solve a lot of these issues.

[kylecordes]

@SanderElias and others - here is the essence of what I was hinting at above. I think you will find it the currently most performant way to compile a lot of typescript code and deliver it to the browser with minimum filesystem activity: let TSC perform the application code bundling, and don't re-bundle libraries except when they really change. This results in far fewer roundtrips to the disk than (for example) the current CLI (broccoli). I implemented this as a fork to the quickstart for a quick demonstration:

https://github.com/kylecordes/quickstart/commit/3cf3041dd80ee8701a3cffb0cbe6264784017a2b

I know of others using this technique at scale in addition to us. However, I have no idea if something like this is suitable for the CLI offhand.

[SanderElias]

@kylecordes, @mattdistefano, I'm not arguing against SystemJS. SystemJS is great. That is not the point. Oh, and I'm pretty convinced that all problems eventually can be solved with it. Still not the point.

I highly agree with @TheLarkInn that when implemented the right way, a bundler/loader should be swappable. We talked about webpack. But it's not just webpack, there are others. An each one has it's own primary purpose. Heck, it should be even possible to use a different one of production. Production is a whole other discussion, but I would like to have the option to use for example tsickle and closure for that.

Assuming that systemjS is the right tool for all projects that will be build with Angular 2 is not the way to go. As far as I'm aware angular-cli is as inclusive as angular itself. That also brings a responsibility. And this project is aimed to be the CLI for all angular things right? So the responsible thing to do is to cater for that. Not the easiest thing to do, but to me, it is the right thing to do.

[mattdistefano]

@SanderElias Well, I'd say being able to solve these problems within SystemJS is the point, at least to the extent that webpack has been advanced in this thread as the solution to these problems. If you think, philosophically, the project should support different tooling, that's fine. But that's a different argument than "we need webpack so we can have bundling".

In any case, on a philosophical level, I'm not wholly opposed to the CLI supporting other tooling, but I am concerned that a) we're going to get a bunch of half-baked integrations with [JS build tool of the week] rather than one really good integration with SystemJS and b) all these new integrations will make getting started and finding help far more difficult. To me, the CLI's first responsibility is to produce a cohesive, functional, and well-documented development experience that makes it trivial to get started using advanced angular concepts while enforcing best practices and providing a degree of consistency between projects. And I suspect that, the more user-configurable the build tooling becomes, the less achievable that goal is.

[hansl]

Here's the official stance of the CLI;

1. Saying "the elephant in the room [...] is webpack" is naive at best. You're talking about a technology, not a feature. The point of the CLI is that the user shouldn't have to worry about technological choices, only the features it (the CLI) provides. Whether it uses webpack, broccoli, or custom made technologies is irrelevant to the user.
2. A list of features as requirements for the build system (see below) has been fleshed out. These are considered blockers for the stable RC0.
3. There's a Proof Of Concept that's being built right now. Once and if we decide to go down the webpack road, your involvement will be welcome. If we go with SystemJS, your involvement is also welcome, of course.

The features we are looking for in a build system (before considering ourselves stable):

1. Support for sourcemaps. Shouldn't be hard, really. SystemJS already provides that.
2. Vendor should be bundled. Don’t download and/or rebuild vendor files, unless necessary.
3. Project Code can be unbundled. Not a requirement, but makes it easy to debug when SourceMaps are a problem.
4. Better third party support:
   1. Node Package Awareness. Using package.json, figure out the vendorNpmFiles and the path mapping automatically, without developer intervention.
   2. Have a better story where npm install is the only thing needed to use some third party libraries.
5. Offline Template Compiler (might be optional for local dev, required for production)
6. Tree shaking.
7. Assets Moved Smartly (css/html copied). Everything outside of public/ should be copied if needed (templates/styles), compiled if needed (sass/less/... and jade/html/...), inlined if needed. Everything inside of public/ should be copied to dist/ folder, compiled if needed.

[SanderElias]

Thank you for setting us straight @hansl!

I do agree with all what you said. What triggered this discussion is not actually the missing of webpack, but rather missing some of the points in your feature list.

However, there is something missing in this feature list. And it's not something easily achieved.
namely developer performance. To be more precise, the time the tooling needs to display a saved change to a browser. And as a close second, the time it takes to run the tests.

Let me explain, while everything may be hunky-dory on your local SSD *nix Intel I7 driven system, there are large number of developers that not have that luxury.
There are a lot of places, where developers need(are forced...) to use a networked drive, or can (not yet) afford to run on the latest hardware.
There are plenty of reasons why not everything is on a local SSD. All of them are valid, and more important: out of our control.

[kylecordes]

@SanderElias I think your observation about performance is very important. I hope performance makes it onto the list that @hansl wrote above, I think his seven items should include your eighth item. This will be important for adoption.

But I'm not sure your focus on webpack vs System really gets to the heart of the problem. System is not the source of the slowness you are seeing. Rather, what you are experiencing I believe has to do with broccoli. This is a much discussed issue with broccoli, broccoli does extensive local disk temp file manipulation, creating thousands of files (mostly symlinks) when used in the typical manner on a small project, and many more a large project. I don't know if this is a solvable problem with broccoli. I don't know how wedded the angular-cli project is to broccoli. I do know that broccoli is conceptually brilliant (shout out to @joliss), in spite of this performance struggle on Windows. (It's really not about SSD, is about filesystem optimization and performance. Mac and OSX can manipulate thousands of small local files very quickly on a local file system. Windows, not so much, it is optimized in other directions. Across a remote file system, manipulating thousands of files is always slow regardless of platform or hardware.)

Going back to webpack, it is not a panacea. I have found that it also becomes awkwardly slow as project size grows. Even on a fast local SSD on a MacBook Pro.

Experimentally I found that the very fastest development cycle occurs when you bundle dependencies into a small number of bundles, only when they change (and not touch your library dependencies at all after each application code edit). Then it use the ability of typescript to emit a system bundle, And simply forgo all of the other bundling tools for application code. TypeScript can do this very quickly if you just let it, and produce a file immediately ready to consume with SystemJS. If instead you have typescript emit numerous JavaScript files which then must be processed by a downstream tool, things are never as fast. I have confirmed with other development teams who have independently reached the same conclusion, that this is the fastest way to set up development tooling on a large Angular 2 project at the moment. I wrote about it here:

http://kylecordes.com/2016/fast-angular-2-compile-cycle-with-tsc

I have no idea whether this approach might be within scope of contemplation for angular-cli, and I am very excited about CLI (because I think it will help considerably with Angular 2 adoption) even if it does not aim for this best available performance. But I do think it would be good if people on the CLI team would at least be aware of this fastest available path, and have it out there is a conceptual target for somewhere down the road.

[intellix]

Currently without bundling, HMR or http/2, every file change makes a request for 650+ files here and it takes about 7 seconds before seeing changes, which is extremely painful during development. I don't experience that with the webpack starter cause of the bundling.

[kushwahashiv]

@intellix exactly the one of reasons we pick webpack.

[un33k]

Not a frontend guy yet, but I am very interested thanks to NG2. Lot's of back & forth between webpack and ng-cli and can't decide which way to go. Found this article which explained everything in simple terms and made it easy for me to lean towards webpack. NG-CLI is a great concept, but with little exposure so far I found my hands tied behind my back everytime I wanted to do something a bit different. The experienced frontend folks are very opinionated when it comes to their tools & environments and rightfully so. They are also the one that can bend the tools to their wills. On the other hand, inexperienced folks like myself are at the mercy of the tools and (IMHO) ng-cli at this stage of development with the current mindset trajectory, might miss all of us.

[ghost]

+1

[mmrath]

I started with cli and now when the project has grown a bit, I started to realise the slowness during development. I now make change and save got to the browser after a few seconds. I changed the project to use webpack starter it is much better. Apart from development performance another thing that is a pain with the current cli is windows support. I think the cli will not be usable by mass corporate users if it does not have good windows support.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}