3.5.0 - it.only no longer works changing while test is open when you have a global after()

[lukeapage]

Current behavior:

When I change one test to it.only I end up with:

The error is here:

Cypress.runner.onRunnableRun = function (runnableRun, runnable, args) {
  debug('_onRunnableRun')

  const r = runnable
  const isHook = r.type === 'hook'
  const isAfterAllHook = isHook && r.hookName.match(/after all/)

where r.hookName is undefined.

Desired behavior:

Works as in 3.4.3

Steps to reproduce: (app code and test code)

1. create a spec with 2 tests
2. use skip and only ui plugin
3. open cypress
4. open the spec
5. edit the file to change one test to it.only

Note: workaround, closing the window and re-opening will work

It seems to be caused by having a global after: https://github.com/bahmutov/cypress-skip-and-only-ui/blob/master/src/support.tsx#L61

so I'm raising here as it seems like its more likely a global cypress issue, but of course it could be skip and only ui

Versions

windows, cypress 3.5.0, running in electron

[jennifer-shehane]

@lukeapage Your issue can be addressed more quickly if you can provide the exact test code for us to run to see the issue. Can you provide?

Working on following your steps to recreate currently.

[jennifer-shehane]

This is not reproducible in Cypress without the https://github.com/bahmutov/cypress-skip-and-only-ui plugin. Please open an issue in that repo. The below code works.

context('Connectors', () => {
  it.only('test 1', () => {
    cy.visit('https://example.cypress.io/commands/connectors')
    cy.get('.connectors-each-ul>li')
  })

  it('test 2', () => {
    cy.visit('https://example.cypress.io/commands/connectors')
    cy.get('.connectors-its-ul>li')
  })

  after(() => {
    cy.log('after')
  })
})

[lukeapage]

this reproduces it:

cypress-io/cypress-test-tiny#51

[lukeapage]

its not always straight away, but if you try changing it.only on the fly it happens after 1 or 2 tries

[lukeapage]

from that pr:

[lukeapage]

it actually happens at the end of executing the test where it has a it.only - so if I change to it.only, save, then remove it.only and save, I am left in a broken state

[lukeapage]

@jennifer-shehane please re-open this

[jennifer-shehane]

I'm not able to reproduce this with the example provided either. 😞

Did you isolate this to only happening in Electron? I am running on MacOS Mohave. What is the IDE you're using?

[jennifer-shehane]

I did get this console error once, the UI was still accurate though. It may be completely unrelated.

Can you open the console and print any errors printed there please?

[lukeapage]

I use webstorm, but I reproduced it also with notepad++ (see screenshot).
Did you see this comment:

it actually happens at the end of executing the test where it has a it.only - so if I change to it.only, save, then remove it.only and save, I am left in a broken state

Because your screenshot is after the it.only but not after going back to 2 tests.

It may be that this bug occurs when editing the test whilst it is running rather than after it has finished (so a wait may help?)

I am on windows, but by the looks of the js exception, I would be surprised if it was os specific.

I can reproduce in chrome and electron.

[lukeapage]

I get this in the chrome console

Uncaught Error: runnable must have an id
    at Object.onRunnableRun (cypress_runner.js:103453)
    at $Cypress.action (cypress_runner.js:97300)
    at Hook.Runnable.run (cypress_runner.js:102519)
    at next (cypress_runner.js:31274)
    at cypress_runner.js:31296
    at timeslice (cypress_runner.js:26364)

And after I get that error, on the next edit of my text editor and save - the spec window is broken forever

[lukeapage]

yes the runnable without an id is the global after function

[jennifer-shehane]

it actually happens at the end of executing the test where it has a it.only - so if I change to it.only, save, then remove it.only and save, I am left in a broken state

Yah, I am doing this and can't recreate.

Do you have manual saving on in your IDE? Some IDEs auto-save as you type after some delay, just wondering if that is an aspect. I only have manual saving on.

Thanks for sending the console error, that is helpful.

[lukeapage]

Do you have manual saving on in your IDE? Some IDEs auto-save as you type after some delay, just wondering if that is an aspect. I only have manual saving on.

Only manual saving and as i said, I tried 2 editors

[lukeapage]

I can try it on mac tomorrow to see if its windows specific

[bahmutov]

Hmm, could not recreate this on Mac. Tried adding skip ony ui plugin - but the plugin does not seem to work at all.

[lukeapage]

I can reproduce on mac too - but it was harder - I added wait commands and then switched between it and it.only 5 or 6 times whilst the command is waiting and then it breaks.

So it probably is not OS dependent but timing dependent. Maybe you're on faster macs than me or something along those lines.

This is again just using my PR to cypress-test-tiny.

[lukeapage]

Note that when it breaks the timer continues forever and No commands were issues during test is present under both tests. Editing the file again does not fix it.

This is relatively low priority because we can just stop using skip and only ui plugin, but if I were to need a global after function it would make development hell.

[lukeapage]

This is no longer reproducible in 4.2.0

[lukeapage]

Oops, this is reproducible, forgot to read all my steps - it doesn't happen straight away but only after a few times of editing the spec file.

[jennifer-shehane]

I'm still not able to reproduce this. I was suspecting that this PR in 4.2.0 should have fixed this. #6620

Honestly, there's nothing we can do without being able to reproduce this though.

[jennifer-shehane]

Right now there doesn't seem to be enough information to reproduce the problem on our end. We'll have to close this issue until we can reproduce it. This does not mean that your issue is not happening - it just means that we do not have a path to move forward.

Please comment in this issue with a reproducible example and we will consider reopening the issue. Here are some tips for providing a Short, Self Contained, Correct, Example and our own Troubleshooting Cypress guide.