sf apex run test not respecting wait flag

[alan-morey]

Summary

The command sf apex run test continues to run after the specified wait time (e.g. --wait=2) has ended and does not return the test run id response as expected.

Steps To Reproduce:

1. Run all local tests with a wait time specified as 2 minutes

   sf apex run test --test-level RunLocalTests --wait=2
   

   NOTE: Assumes running all local apex tests will take much longer than 2 minutes to execute, in my case it takes hours

Expected result

Since the tests have not finished by the end of the 2 minute wait time, the command should exit, displaying the message with test run ID.

e.g.

Run "sf apex get test -i 7078F00000oJGyP -o [email protected]" to retrieve test results

Actual result

Command continues waiting for results even after 2 minutes wait time has ended and eventually displays the test results after the test run has completed.

System Information

CLI Version:
@salesforce/cli/1.75.6

Architecture:
linux-x64

Node Version:
node-v18.16.0

Plugin Version:
@oclif/plugin-autocomplete 2.1.9 (core)
@oclif/plugin-commands 2.2.14 (core)
@oclif/plugin-help 5.2.9 (core)
@oclif/plugin-not-found 2.3.23 (core)
@oclif/plugin-plugins 2.4.6 (core)
@oclif/plugin-search 0.0.16 (core)
@oclif/plugin-update 3.1.12 (core)
@oclif/plugin-version 1.3.3 (core)
@oclif/plugin-warn-if-update-available 2.0.35 (core)
@oclif/plugin-which 2.2.19 (core)
@salesforce/cli 1.75.6 (core)
apex 2.2.13 (core)
auth 2.7.12 (core)
data 2.3.10 (core)
deploy-retrieve 1.8.14 (core)
info 2.6.7 (core)
limits 2.3.13 (core)
login 1.2.5 (core)
org 2.7.0 (core)
schema 2.3.8 (core)
settings 1.4.4 (core)
sobject 0.1.17 (core)
source 2.10.2 (core)
telemetry 2.1.4 (core)
templates 55.4.9 (core)
trust 2.4.9 (core)
user 2.3.8 (core)
gr-sfdx-plugin 0.0.0 (link) /home/alan.morey/repos/salesforce/gr-sfdx-plugin

OS and Version:
Linux 5.19.0-40-generic

Shell:
zsh

Root Path:
/home/alan.morey/.npm-packages/lib/node_modules/@salesforce/cli

Additional information

When running the same command without the wait flag, the command exits after the default 1 minute wait time.

$ sf apex run test -l RunLocalTests
Run "sf apex get test -i 7078F00000oJGyP -o [email protected]" to retrieve test results

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[alan-morey]

FYI, this bug is still present in v1.79.5

CLI Version:
        @salesforce/cli/1.79.5

 Architecture:
        linux-x64

 Node Version:
        node-v18.16.0

 Plugin Version:
        @oclif/plugin-autocomplete 2.2.0 (core)
        @oclif/plugin-commands 2.2.15 (core)
        @oclif/plugin-help 5.2.9 (core)
        @oclif/plugin-not-found 2.3.23 (core)
        @oclif/plugin-plugins 3.0.1 (core)
        @oclif/plugin-search 0.0.17 (core)
        @oclif/plugin-update 3.1.15 (core)
        @oclif/plugin-version 1.3.4 (core)
        @oclif/plugin-warn-if-update-available 2.0.37 (core)
        @oclif/plugin-which 2.2.20 (core)
        @salesforce/cli 1.79.5 (core)
        apex 2.2.19 (core)
        auth 2.7.15 (core)
        data 2.3.18 (core)
        deploy-retrieve 1.9.1 (core)
        info 2.6.13 (core)
        limits 2.3.15 (core)
        login 1.2.9 (core)
        org 2.9.0 (core)
        schema 2.3.10 (core)
        settings 1.4.8 (core)
        signups 1.4.14 (user)
        sobject 0.1.19 (core)
        source 2.10.9 (core)
        telemetry 2.2.0 (core)
        templates 55.4.15 (core)
        trust 2.4.18 (core)
        user 2.3.13 (core)
        gr-sfdx-plugin 0.0.0 (link) /home/alan.morey/repos/salesforce/gr-sfdx-plugin

 OS and Version:
        Linux 5.19.0-41-generic

 Shell:
        zsh

 Root Path:
        /home/alan.morey/.npm-packages/lib/node_modules/@salesforce/cli

[jocularchen]

same problem. I have to wait 1 min without -w -y flag

[cristiand391]

Hey all, sorry for the late reply.

After looking at the apex command & library I see --wait is only used to make the command return as soon as it gets the test run ID and doesn't respect any wait time in minutes:
https://github.com/salesforcecli/plugin-apex/blob/ddc74174df6b02671ab149219ccc00fc436e9195/src/commands/apex/run/test.ts#L222

if --wait is > 0, the command will wait until tests finish.
if --wait is = 0 or isn't specified, it will return the test id and sf command to get the results asynchronously.

--wait summaryf for apex run test:
Sets the streaming client socket timeout in minutes; specify a longer wait time if timeouts occur frequently.

I'm stil confused why it says that when the command and library seems to handle it that way on purpose (we've recently taken ownership of apex commands).

[git2gus]

This issue has been linked to a new work item: W-14364504

[cristiand391]

labeled as a bug to make --wait respect the for timing out.

[jshackell-sfdc]

This issue is fixed in version 2.39.6 of Salesforce CLI.