In 6.x files are transformed/copied multiple times per build

[brendon]

• Operating System: MacOS 10.15.5 (19F101)
• Node Version: v14.4.0
• NPM Version: N/A (Yarn 1.22.4)
• webpack Version: [email protected]
• copy-webpack-plugin Version: 6.x

Expected Behavior

Files should be transformed/copied only once per build

Actual Behavior

In 5.x the files are transformed once per build (verified by logging when transform is called). In 6.x the transform function is called a large number of times (perhaps 6-10 times per file.

I noticed that removing new VueLoaderPlugin() from the plugins list reduced the multiple calls down to two per file. My suspicion is that in 6.x you're hooking into an event that runs even on child builds rather than on the entry point build. new VueLoaderPlugin() seems to kick off a lot of child builds depending on the number of Vue components in the application.

Reverting to 5.x fixed the issue.

Code

// webpack.config.js

...
transform(content, filePath) {
  console.log(filePath)
}
...

new VueLoaderPlugin()

How Do We Reproduce?

The only hint I have is to do with new VueLoaderPlugin(). Hopefully this is enough of a pointer, but if not, let me know and I'll look into creating a minimal reproducible.

[brendon]

This seems to be the likely cause:

https://github.com/webpack-contrib/copy-webpack-plugin/pull/459/files#diff-1fdf421c05c1140f6d71444ea2b27638L25-L27

Switching from tapping into emit that one assumes only gets triggered once, to tapping into compilation that presumably runs for each master and child build.

[alexander-akait]

In 5.x the files are transformed once per build (verified by logging when transform is called). In 6.x the transform function is called a large number of times (perhaps 6-10 times per file.

No, emit and addtionalAssets hooks called on each recompilation

Changing hook was written in CHANGELOG, without it, the plugin does not work correctly

[alexander-akait]

In webpack@5, using 5 version of copy-webpack-plugin was output invalid result and will be broken

[brendon]

Please checkout this repo: https://github.com/brendon/broken-copy

run yarn install then yarn build and you'll see that the files are transformed/copied multiple times per build (I'm logging to the console when transform is executed). The reason for this is that MiniCssExtractPlugin creates a compilation for each component's styles.

I'd instinctively expect CopyWebpackPlugin to copy just once per compile. With your update it looks like you're telling it to copy once per compilation of which there can be many per compile.

What would be the way forward on this one?

[alexander-akait]

I'd instinctively expect CopyWebpackPlugin to copy just once per compile. With your update it looks like you're telling it to copy once per compilation of which there can be many per compile.

Because each compilation requires all assets

What is a problem? You may need an asset inside MiniCssExtractPlugin (not only for this plugin), there are a lot of use cases. You can enable cache for transform.

[brendon]

Ok, turning on caching fixed the problem in this test repo, but I didn't experience the same result in my actual application. I returned a promise from transform for each file (processing JPEGs and PNGs with tinify).

I tried to simulate a delay with a setTimeout but it operates correctly (the first transforms of each file are waited before checking the cache for subsequent transforms):

    new CopyWebpackPlugin({
      patterns: [{
        from: `public`,
        transform(content, filePath) {
          console.log(filePath)
          return new Promise((resolve, reject) => {
            setTimeout(() => {
              resolve(content)
            }, 5000)
          })
        },
        cacheTransform: true
      }]
    })

I must have been doing something wrong. I'll revisit this in my own code now that I understand how it works a bit more deeply.

Thank you for your time :)

[alexander-akait]

Oh, my mistake, I think it is a problem, anyway cache is good idea here too

[brendon]

Definitely. I'll have another go with 6.x now to see if the problem persists.

What do you think the problem is? :)

[brendon]

I've had another go and it doesn't wait for the promises to resolve before checking the cache for the next iteration. I could be misunderstanding the method I'm using as I don't understand TypeScript in this scenario:

https://github.com/tinify/tinify-nodejs/blob/master/src/tinify/Result.ts#L37

But I assume it means that it'll return a promise because that's in the method definition and also this._data in that code is a promise.

[alexander-akait]

@brendon yes, it is return promise

[brendon]

Thanks @evilebottnawi, very much appreciated! :)