You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If browser-level test starts after no-browser (protocol-level) tests, an error in browser.newContext() may occur.
It happens when browser-level test reuses VU, that was already used for protocol-level test (or simply test without BrowserContext creation).
All iterations of all tests have to pass without errors.
Actual behaviour
It seems it takes 30 seconds to free VUs used by scenario with constant-vus executor.
Before that moment browser-level test ramping completely new VUs with no problems for browser.newContext(). But at 31 second browser-level test takes VU that was used for protocol-level test. In each iteration of this reused VU occurs an error when trying to create BrowserContext (see logs below):
browser not found in registry. make sure to set browser type option in scenario definition in order to use the browser module
Output:
execution: local
script: browserVuErrorTest.js
output: -
scenarios: (100.00%) 2 scenarios, 3 max VUs, 1m4s max duration (incl. graceful stop):
* protocolLevel: 2 looping VUs for 1s (exec: protocolLevelTest, gracefulStop: 30s)
* browserLevel: Up to 2 looping VUs for 4s over 2 stages (gracefulRampDown: 30s, exec: browserLevelTest, startTime: 30s, gracefulStop: 30s)
INFO[0000] {"idInInstance":2,"idInTest":2,"iterationInInstance":0,"iterationInScenario":0,"tags":{"group":"","scenario":"protocolLevel"},"metrics":{"tags":{"group":"","scenario":"protocolLevel"},"metadata":{}}}: protocol-level test dummy source=console
INFO[0000] {"idInInstance":1,"idInTest":1,"iterationInInstance":0,"iterationInScenario":0,"tags":{"group":"","scenario":"protocolLevel"},"metrics":{"tags":{"group":"","scenario":"protocolLevel"},"metadata":{}}}: protocol-level test dummy source=console
INFO[0000] {"idInInstance":1,"idInTest":1,"iterationInInstance":1,"iterationInScenario":1,"tags":{"group":"","scenario":"protocolLevel"},"metrics":{"tags":{"group":"","scenario":"protocolLevel"},"metadata":{}}}: protocol-level test dummy source=console
INFO[0000] {"idInInstance":2,"idInTest":2,"iterationInInstance":1,"iterationInScenario":1,"tags":{"group":"","scenario":"protocolLevel"},"metrics":{"tags":{"group":"","scenario":"protocolLevel"},"metadata":{}}}: protocol-level test dummy source=console
INFO[0001] {"idInInstance":2,"idInTest":2,"iterationInInstance":2,"iterationInScenario":2,"tags":{"group":"","scenario":"protocolLevel"},"metrics":{"tags":{"group":"","scenario":"protocolLevel"},"metadata":{}}}: protocol-level test dummy source=console
INFO[0001] {"idInInstance":1,"idInTest":1,"iterationInInstance":2,"iterationInScenario":2,"tags":{"group":"","scenario":"protocolLevel"},"metrics":{"tags":{"group":"","scenario":"protocolLevel"},"metadata":{}}}: protocol-level test dummy source=console
INFO[0030] {"idInInstance":3,"idInTest":3,"iterationInInstance":0,"iterationInScenario":0,"tags":{"group":"","scenario":"browserLevel"},"metrics":{"tags":{"group":"","scenario":"browserLevel"},"metadata":{}}}: BrowserContext created source=console
INFO[0031] {"idInInstance":3,"idInTest":3,"iterationInInstance":1,"iterationInScenario":1,"tags":{"group":"","scenario":"browserLevel"},"metrics":{"tags":{"group":"","scenario":"browserLevel"},"metadata":{}}}: BrowserContext created source=console
ERRO[0032] {"idInInstance":1,"idInTest":1,"iterationInInstance":3,"iterationInScenario":0,"tags":{"group":"","scenario":"browserLevel"},"metrics":{"tags":{"group":"","scenario":"browserLevel"},"metadata":{}}}: GoError: browser not found in registry. make sure to set browser type option in scenario definition in order to use the browser module source=console
INFO[0032] {"idInInstance":3,"idInTest":3,"iterationInInstance":2,"iterationInScenario":2,"tags":{"group":"","scenario":"browserLevel"},"metrics":{"tags":{"group":"","scenario":"browserLevel"},"metadata":{}}}: BrowserContext created source=console
ERRO[0033] {"idInInstance":1,"idInTest":1,"iterationInInstance":4,"iterationInScenario":1,"tags":{"group":"","scenario":"browserLevel"},"metrics":{"tags":{"group":"","scenario":"browserLevel"},"metadata":{}}}: GoError: browser not found in registry. make sure to set browser type option in scenario definition in order to use the browser module source=console
INFO[0034] {"idInInstance":3,"idInTest":3,"iterationInInstance":3,"iterationInScenario":3,"tags":{"group":"","scenario":"browserLevel"},"metrics":{"tags":{"group":"","scenario":"browserLevel"},"metadata":{}}}: BrowserContext created source=console
data_received........: 0 B 0 B/s
data_sent............: 0 B 0 B/s
iteration_duration...: avg=705.92ms min=400.46ms med=709.23ms max=1.01s p(90)=1s p(95)=1s
iterations...........: 12 0.344638/s
vus..................: 2 min=0 max=2
vus_max..............: 3 min=3 max=3
running (0m34.8s), 0/3 VUs, 12 complete and 0 interrupted iterations
protocolLevel ✓ [======================================] 2 VUs 1s
browserLevel ✓ [======================================] 0/2 VUs 4s
As you can see in output there is vus_max = 3, while both protocol-level and browser-level tests configured to use 2 VUs.
Therefore, at the moment, browser-level tests cannot be combined with protocol-level tests due to this bug in most cases.
The text was updated successfully, but these errors were encountered:
The newBrowserRegistry function's called once for each VU and subscribes to their IterStart, IterEnd, Exit events.
If VU runs in a protocol-level scenario at first, then following code executes in the handleIterEvents func (IterStart event):
30 seconds after a protocol-level scenario finishes VU frees. A browser-level scenario launches, takes this VU, but there is no subscription to IterStart event anymore, so the browser initialization code is not executed, the errBrowserNotFoundInRegistry error occures in the getBrowser function.
I tried to fix this, but I don't see a simple solution to this error. It seems to be by design (there is the unsubscribe_on_non_browser_vu test), and to fix this behaviour working with VU's events and browser initialization needs to be reviewed.
Or am I wrong, and the problem is simpler?
Brief summary
If browser-level test starts after no-browser (protocol-level) tests, an error in
browser.newContext()
may occur.It happens when browser-level test reuses VU, that was already used for protocol-level test (or simply test without
BrowserContext
creation).Use case: hybrid load testing, many protocol-level tests and some browser-level ones (for testing ASPX pages with multiple internal requests for example), https://k6.io/docs/using-k6-browser/running-browser-tests/#run-both-browser-level-and-protocol-level-tests-in-a-single-script
k6 version
k6 v0.47.0 (commit/5ceb210056, go1.21.2, windows/amd64)
OS
Windows 10
Docker version and image (if applicable)
No response
Steps to reproduce the problem
Just run following script:
Expected behaviour
All iterations of all tests have to pass without errors.
Actual behaviour
It seems it takes 30 seconds to free VUs used by scenario with
constant-vus
executor.Before that moment browser-level test ramping completely new VUs with no problems for
browser.newContext()
. But at 31 second browser-level test takes VU that was used for protocol-level test. In each iteration of this reused VU occurs an error when trying to createBrowserContext
(see logs below):Output:
As you can see in output there is
vus_max = 3
, while both protocol-level and browser-level tests configured to use 2 VUs.Therefore, at the moment, browser-level tests cannot be combined with protocol-level tests due to this bug in most cases.
The text was updated successfully, but these errors were encountered: