Lighthouse stalls when testing bfcache in old headless mode

[abhinav-saini]

FAQ

• Yes, my issue is not about variability or throttling.
• Yes, my issue is not about a specific accessibility audit (file with axe-core instead).

URL

http://www.lumme.info/

What happened?

Such domain is not responding with a scan report in Lighthouse 10.0.1 (with Chrome latest stable version 112) even after waiting for like 10+ minutes - using node module programmatic way. Whereas this works fine with Lighthouse 9.6.8 with Chrome 110.

I am afraid there is some major bug in Lighthouse 10 which is affecting this for many other domains as well.

What did you expect?

Such domain is working fine with Lighthouse 9.6.8 with Chrome 110 but not able to provide any response with Lighthouse 10.

What have you tried?

No response

How were you running Lighthouse?

node

Lighthouse Version

10.0.1

Chrome Version

112.0.5615.49-1

Node Version

18.2

OS

Linux

Relevant log output

No response

[adamraine]

I am unable to reproduce this on Linux with your exact Node, Chrome, and Lighthouse versions. Are you using a node script to run Lighthouse, and if so would you be able to provide the script?

[abhinav-saini]

@adamraine sure, please run the below snippet multiple times, it might pass intermittently but check for the response time of each iteration. For me few iterations are not responding before 10 minutes. For me, it was recently stuck at "Running 3 scan for ..."

const lighthouse = require('lighthouse/core/index.cjs'); //using commonjs
const chromeLauncher = require('chrome-launcher');

const target = "http://www.lumme.info/";

const lhrs = [];

(async () => {
  const Chrome = await chromeLauncher.launch({
    chromeFlags: [
      '--no-first-run',
      '--headless',
      // '--single-process',
      '--no-zygote',
      '--disable-gpu',
      '--no-sandbox',
      '--disable-dev-shm-usage',
    ],
  });
  console.log('--beginning--');
  for (let i = 0; i < 5; i++) {
    console.log(`Running ${i} scan for ${target}`);
    const options = {
      logLevel: 'error',
      output: 'html',
      maxWaitForLoad: 60000,
      onlyCategories: [
        'accessibility',
        'best-practices',
        'performance',
        'pwa',
        'seo',
      ],
      port: Chrome.port,
      hostname: '127.0.0.1',
    }
    const runnerResult = await lighthouse(target, options);

    lhrs.push(runnerResult.lhr);
  }
  console.log('-done-', lhrs.length)
  process.exit(0);
})();

[abhinav-saini]

again to clarify - thousands of domains are working fine, but a few are having this no-response issue.

[adamraine]

Thanks, I can reproduce with your script! It looks like Lighthouse is hanging when it attempts to test bfcache in the old headless mode. There are two workarounds to this issue:

• Use --headless=new flag instead of --headless when launching Chrome. This is my recommended approach as it fixes your issue and provides accurate bfcache testing.
• Add skipAudits: ['bf-cache'] to your Lighthouse options. Even for pages that don't hang forever, this audit will always return erroneous results when using the old headless Chrome bf-cache always fails in headless mode #14784.

Regardless, we should still investigate why the old headless Chrome is stalling on the page you provided.

[abhinav-saini]

@adamraine ah yes, thanks, using skipAudits: ['bf-cache'] option helped in getting the response. Though I am not aware of the effects of skipping this bf-cache audit on the overall Performace metrics scores, but yes this eventually seems to help for the time being.