AngularTransformerGroup memory leak

[heisaf]

Since the upgrade to AngularDart 1.0 we have been experiencing some severe issues with our project build. We needed to allocate about 20 GB heap space for pub and one single build takes about 1h 30m. To simulate a large scale project, i changed the sample application in this branch https://github.com/heisaf/angular.dart/tree/todo_generator. The TodoGenerator is a transformer that copies the Todo App as many times as you want (defaults to 20). You can config the number of Todo Apps by setting the generate property in the pubspec.yaml.

transformers:
- angular_dart_example:
    generate: '5'

To run pub build successfully with 20 Todo Apps, you need to set the heap space to 4GB. If you increase the Todo Apps you also need to increase the heap space (until you run out of physical memory). Otherwise pub build stops with "Exhausted heap space" messages. The size of the allocated heap space should not depend on the number of views that need to be compiled to javascipt.

[heisaf]

In this branch i did an experiment. Instead of copying the Todo App the generator generates components, that are placed into the lib folder. While creating a Todo application with 1000 Todo components the main isolate only consumes about 700 mb. So by minimizing the number of entry point in the web folder, you can reduce the heap space to compile the application to javascript (probably because the resolver create for every entry point an AST). If it's not already done this should be documented somewhere. I think issue #1683 can be closed with this pull request.

[rkirov]

dba6727 explicitly removed sharing the resolved to avoid a deadlock. I guess the deadlock is fixed so we can switch back to sharing resolvers.

@kegluneq is this expected

[kegluneq]

@rkirov, could you clarify what situation the question refers to, that is, what is the "this" in your question?

Also, I may not be the right person to ask - I'm familiar with the Angular 2 transformer but do not have context on this one.

[rkirov]

Is it expected for the memory usage to grow with the number of entry points in the web directory? Would this be the same in Angular 2?

As someone who has worked with transformers more recently than me, do you recommend switching back to a shared resolver between transformation steps as proposed in #1684? Do angular2 transformer steps share resolvers?

[kegluneq]

Thank you for the clarification.

I would recommend sharing a single resolver among the transformation steps. My understanding is that having multiple Resolvers objects will slow things down and consume additional memory. Passing additional entryPoints to Resolvers#get will increase required memory & time, but just adding files to the web/ directory should not. This depends on the import graph though - if you add files to the web/ directory and reference them from your entry point(s), those will be resolved requiring additional resources.

That said, one of the Angular 2 transformer's design goals is to avoid resolution, so I have used Resolvers pretty minimally.