Cypress can hang when pageLoadTimeout event fired when redirecting back to our app after logging in using Microsoft AAD

[blackgrouse]

Current behavior

Cypress can hang when a pageLoadTimeout event is received when redirecting back to our app after logging in using Microsoft AAD which is causing issues in our CI/CD environment (Azure DevOps (ADO) ubuntu-22.04).

Our app uses MS AAD to authenticate users and we are automating this using v13.8.1 cy.session() and cy.origin() commands with code based on this blog. I have had to tweak the code slightly as our login page is hosted by Microsoft.

Here is my version of the login code via the custom command cy.loginWithMicrosoftAccountAndSelectOrg():

declare global {
    namespace Cypress {
        interface Chainable {
            loginWithMicrosoftAccountAndSelectOrg(options: {
                username: string;
                password: string;
                clickAadTile?: boolean;
                clickDontRemainSignedIn?: boolean;
                orgButtonText?: string;
                postLoginSelector?: string;
                sessionName?: string;
                visit?: { url: string; failOnStatusCode?: boolean };
            }): Chainable<any>;
        }
    }
}

const msLogin = (options: {
    username: string;
    password: string;
    clickAadTile?: boolean;
    clickDontRemainSignedIn?: boolean;
    orgButtonText?: string;
    postLoginSelector?: string;
    useCachedSession?: boolean;
    visit?: { url: string; failOnStatusCode?: boolean };
}) => {
    const sentArgs = {
        username: options.username,
        password: options.password,
        clickAadTile: options.clickAadTile,
        clickDontRemainSignedIn: options.clickDontRemainSignedIn
    };

    const urlToVisit = options.visit?.url ? options.visit?.url : '/';
    const failOnStatusCode = options.visit?.failOnStatusCode ? options.visit?.failOnStatusCode : false;
    cy.visit(urlToVisit, { failOnStatusCode: failOnStatusCode });

    cy.origin(`${getAuthHostName()}.<ourwebsite>.com`, () => {
        cy.contains('Microsoft').click();
    });

    // Enter credentials on MS login pages
    cy.origin(
        'login.microsoftonline.com',
        { args: sentArgs },
        ({
            username,
            password,
            clickAadTile,
            clickDontRemainSignedIn
        }: {
            username: string;
            password: string;
            clickAadTile?: boolean;
            clickDontRemainSignedIn?: boolean;
        }) => {
            cy.get('input[type="email"]').should('be.visible').type(username);
            cy.get('input[type=submit]').should('be.visible').click();

            if (clickAadTile) {
                cy.get('#aadTile').should('be.visible').click();
            }

            cy.get('input[type="password"]').should('be.visible').type(password, { log: false });
            cy.get('input[type=submit]').should('be.visible').click();

            if (clickDontRemainSignedIn) {
                cy.get('input[value="No"]').should('be.visible').click();
            }
        }
    );

    if (options.orgButtonText) {
        const orgButtonText = options.orgButtonText;
        waitForElementToBeVisible({
            selector: '[data-testid="selectTenantPage"]'
        }).then((elementVisible) => {
            if (elementVisible) {
                cy.contains(orgButtonText).click();
            } else {
                cy.logTestProgress('Tenant selection screen not visible, continuing...');
            }
        });
    }

    options.postLoginSelector
        ? cy.get(options.postLoginSelector).should('be.visible')
        : cy.get('nav').should('be.visible');
};

Cypress.Commands.add('loginWithMicrosoftAccountAndSelectOrg', (options) => {
    if (Cypress._.isString(options.sessionName)) {
        cy.session(
            options.sessionName,
            () => {
                msLogin(options);
            },
            {
                validate: () => {
                    cy.wrap(localStorage.getItem('msal.idtoken'), { log: false }).should('exist');
                },
                cacheAcrossSpecs: true
            }
        );
    } else {
        msLogin(options);
    }
});

The UI flow is:

1. Click the 'Microsoft' button to login
2. Enter creds on MS AAD pages when prompted
3. Click the 'No' button to not remain signed in
4. Redirect back to our app and retrieve data from AWS S3 to be displayed

Most of the time this works fine, but sometimes, after step 3, Cypress hangs and doesn't redirect to our app. On debugging, we are not receiving a load event so that Cypress fires a pageLoadTimeout event, which should cause the test to fail. Unfortunately it does not, Cypress hangs at this point and continues running infinitely until stopped - by the user locally, or by ADO after a predefined amount of time. In the latter case, this fails the build, which obviously impacts our delivery efficiency as we have to rerun any builds which have failed in this way.

The last line executed was:

cy.get('input[value="No"]').should('be.visible').click();

and then there is no more activity in the test runner.

I have been able to reproduce this on my Mac 14.6.1 and the latest version of Cypress. Here is a screenshot of the test runner when failing locally:

Desired behavior

The desired behaviour is that Cypress should not hang but should allow the test to fail as expected.

Test code to reproduce

I have not had time to provide a failing test, but provide example code

Cypress Version

13.8.1

Node version

20.16.0

Operating System

macOS 14.6.1 and ubuntu 20.04

Debug Logs

Happy to provide logs if you tell me which ones would be helpful.

Other

No response

[jennifer-shehane]

@blackgrouse Does this code work in non-Chrome browsers? Like Electron or Firefox?

It reminds me of this issue, which we have a workaround for if that is the case.

[blackgrouse]

@jennifer-shehane Thanks for getting back to me.

Yes, I'm able to reproduce in Electron.

[blackgrouse]

@blackgrouse Does this code work in non-Chrome browsers? Like Electron or Firefox?

It reminds me of this issue, which we have a workaround for if that is the case.

Hi @jennifer-shehane any updates for me on this please?

You hint at a workaround - is that on my end or yours?

[shaswot77]

@jennifer-shehane is there any update on this please?
Also is there a workaround?

[blackgrouse]

@jennifer-shehane Thanks for getting back to me.

Yes, I'm able to reproduce in Electron.

Hi @jennifer-shehane I have provided the information you requested quite a while ago so I was wondering if there was any feedback on this?

Our app is deployed by our pipeline after successful E2E tests and this issue is causing big issues for us and impacts our ability to deploy our app first time, every time, handsfree. It also impacts the trust of the business of the effectiveness and quality of our testing, so any information you can provide would be gratefully received.

[jennifer-shehane]

@blackgrouse There are no errors at the time this occurs in the console?

We'd need a way provided to test this on our end to narrow down the issue. We have Microsoft AAD tests that we run continuously that are passing and not encountering this issue, so it has to be something specific to the rerouting behavior back to your site - especially since that's where it's hanging.

It's strange to see the amazon.aws URL as in the errored state - you said it 'retrieve data from AWS S3 to be displayed'. Is there something not working in that communication specifically?

[blackgrouse]

@jennifer-shehane

I have managed to get a consistent repro of the issue by forcing the load event to not fire. It looks like the problem is that Cypress hangs if logging in using a Microsoft account using cy.origin() and there are plugins installed that execute after a test fails.

If I remove the problem plugins the test fails as expected.

There is a repo that can recreate the issue: https://github.com/squaredup/cypress-issue-repro

[cypress-bot]

Released in 13.15.2.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v13.15.2, please open a new issue.