Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Significant degradation in performance using Docker #1573

Open
ngr05 opened this issue Jun 13, 2024 · 0 comments
Open

Significant degradation in performance using Docker #1573

ngr05 opened this issue Jun 13, 2024 · 0 comments

Comments

@ngr05
Copy link

ngr05 commented Jun 13, 2024

Has anyone else struggled with a significant degradation in performance when running through Docker? I am looking to run a large number of scenarios (320) against a local running instance of an application. I am using three viewport sizes. When I run on my M1 Mac, the run takes approximatley 3 minutes...

> test
> backstop test --config="backstop.config.js"
BackstopJS v6.3.23
Loading config:  [...]/packages/visual-testing/backstop.config.js
COMMAND | Executing core for "test"
createBitmaps | Selected 320 of 320 scenarios.
Creating Browser
No Playwright browser specified, assuming Chromium.
Browser created
Browser Console Log 0: BackstopTools have been installed.
[...]
report | Test completed...
report | 960 Passed
report | 0 Failed
report | Writing json report
report | Writing browser report
report | Resources copied
report | Wrote Json report to: [...]/packages/visual-testing/data/reports/json_report/jsonReport.json
report | Resources copied
report | Copied json report to: [...]/packages/visual-testing/data/evidence/20240613-084244/report.json
report | Copied jsonp report to: [...]/packages/visual-testing/data/reports/html_report/config.js
COMMAND | Executing core for "openReport"
openReport | Attempting to ping
openReport | Remote not found. Opening data/reports/html_report/index.html
COMMAND | Command "report" successfully executed in [13.696s]
COMMAND | Command "test" successfully executed in [161.644s]

To run my tests I am running backstop test --config="backstop.config.js".

Because I need the tests to run on my local development environment and in GitHub Actions I have implementing running in docker following similar to this.

When I run using Docker (backstop test --config="backstop.config.js" --docker) and it gives me an outcome like this...

> test
> backstop test --config="backstop.config.js" --docker
BackstopJS v6.3.23
Loading config:  [...]/packages/visual-testing/backstop.config.js
COMMAND | Executing core for "test"
Delegating command to Docker... docker run --rm [...] -w /app/packages/visual-testing --name tests  -v [...]/apps:/app/apps:ro -v [...]/packages/visual-testing:/app/packages/visual-testing:ro backstopjs/backstopjs:6.3.23 test "--config=backstop.config.js" "--moby"
BackstopJS v6.3.23
Loading config:  /app/packages/visual-testing/backstop.config.js
COMMAND | Executing core for "test"
createBitmaps | Selected 320 of 320 scenarios.
Creating Browser
No Playwright browser specified, assuming Chromium.
Browser created
Browser Console Log 0: BackstopTools have been installed.
[...]
report | Test completed...
report | 954 Passed
report | 6 Failed
report | Writing json report
report | Writing browser report
report | Resources copied
report | Wrote Json report to: /app/packages/visual-testing/data/reports/json_report/jsonReport.json
report | Resources copied
report | Copied json report to: /app/packages/visual-testing/data/evidence/20240613-074636/report.json
report | Copied jsonp report to: /app/packages/visual-testing/data/reports/html_report/config.js
COMMAND | Executing core for "openReport"
openReport | Attempting to ping
openReport | Remote not found. Opening data/reports/html_report/index.html
[...]
COMMAND | Executing core for "openReport"
openReport | Attempting to ping
COMMAND | Command "test" ended with an error after [4540.916s]
COMMAND | Error: docker run --rm [...] -w /app/packages/visual-testing --name tests  -v [...]/apps:/app/apps:ro -v [...]/packages/visual-testing:/app/packages/visual-testing:ro backstopjs/backstopjs:6.3.23 test "--config=backstop.config.js" "--moby" returned 1
at ChildProcess.<anonymous> ([...]/node_modules/backstopjs/core/util/runDocker.js:68:18)
at ChildProcess.emit (node:events:527:28)
at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
openReport | Remote not found. Opening data/reports/html_report/index.html
npm
ERR! Lifecycle script `test` failed with error:
npm ERR! Error: command failed
npm ERR!   in workspace: visual-testing
npm ERR!   at location: [...]/packages/visual-testing
Error in script: 1

(the failures are minor discrepancies which show why I need to get my project running in Docker)

My config is dynamically built from JSON files but looks something like this...

module.exports = {
  asyncCaptureLimit: 15,
  asyncCompareLimit: 50,
  ci: {
    format: "junit",
    testSuiteName: "backstopJS",
  },
  debug: process.env.DEBUG === "true",
  debugWindow: process.env.DEBUG === "true",
  dockerCommandTemplate: getDockerRunCommand(),
  engine: "playwright",
  engineOptions: {
    args: ["--no-sandbox", "--disable-setuid-sandbox"],
    browsersTypes: ["chromium"],
    gotoParameters: {
      waitUntil: "networkidle0",
    },
    headless: "new",
    ignoreHTzTPSErrors: false,
  },
  fileNameTemplate: "{scenarioLabel}_{viewportLabel}",
  id: "gfe-storybook",
  misMatchThreshold: 20,
  onBeforeScript: "playwright/onBefore.js",
  onReadyScript: "playwright/onReady.js",
  paths,
  report,
  requireSameDimensions: false,
  scenarioDefaults: {
    delay: 0,
    selectorExpansion: true,
  },
  scenarios: getScenarios(),
  viewports: [
    {
      label: "desktop",
      width: 1920,
      height: 1080,
    },
    {
      label: "tablet",
      width: 1024,
      height: 768,
    },
    {
      label: "phone",
      width: 320,
      height: 480,
    },
  ],
};

My project is part of a mono repo so the mounts look a bit strange and I have redacted the full file paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant