Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate product and sum into own modules #591

Merged
merged 1 commit into from
Aug 3, 2020

Conversation

rossabaker
Copy link
Collaborator

@rossabaker rossabaker commented Jul 31, 2020

Large modules, such as Network.AWS.ECS.Types, take too long to compile. This generates each generated sum and product into their own module, which are then re-exported in Types. There are two advantages:

  1. Smaller modules promote parallel compilation, so wall time drops.
  2. Compilation of large modules is superlinear, so total CPU time drops.

This change should be compatible.

Copy link
Owner

@brendanhay brendanhay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have various branches and code lying around that benchmarked a couple of different type module layout algorithms specifically designed to optimise compile time. Somewhat hilariously, I never tried one type per module! 😆

From what I vaguely remember of the experiment results, the optimal solution was to use Data.Graph.stronglyConnComp in conjunction with some form of bin packing to create 'fairly smol modules, but not too smol' according to an adjustable threshold.

Love this change, straight forward and simple. :shipit:

@rossabaker
Copy link
Collaborator Author

I wouldn't be surprised if bin packing improved it further. Also wouldn't be suprised if the optimal bins depended on the cores. The important thing is that this splits up the EC2.Types monstrosity. 😄

@rossabaker rossabaker merged commit 726bfaf into brendanhay:develop Aug 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants