0.7.1

QIT CLI 0.7.1 Release Notes

What's New

Environment variables in Custom Tests

• Adds --env and --env_file to run:e2e, run:activation and env:up commands.
• This can be used to pass secrets or configuration that you don't want to hardcode in your tests.

Documentation

Check the documentation for Environment Variables.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Custom Tests: Allow to pass env vars by @Luc45 in #217

Full Changelog: 0.7.0...0.7.1

0.7.0

QIT CLI 0.7.0 Release Notes

We are excited to announce the release of QIT CLI version 0.7.0! This update introduces tunneling support, along with UX/developer experience improvements.

What's New

Tunneling Support

• Easy Configuration: Use our built-in tunneling methods for easy tunneling.
• Improved Testing: Tunneling support enables you to test your extensions with a live URL.
• Multiple Tunneling Methods: Choose the tunneling method that works best for you.

Documentation

You can find the documentation for Tunneling here.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Disable Mac test, add Validation workflow by @Luc45 in #207
• Sync 9.3.2 e2e by @zhongruige in #208
• 9.4.0-beta.2 snapshots by @Luc45 in #211
• Fix self-tests for the Custom Tests by @Luc45 in #212
• Allow to use WooCommerce as the SUT for Activation Test by @Luc45 in #213
• Docker pull only once a day by @Luc45 in #215
• Prioritize extension slugs over local directories and improve Docker volume handling by @Luc45 in #214
• Overwrite scaffolded tests with safety checks by @Luc45 in #216
• Add Tunnel support to enable sites to have a live URL and HTTPs by @Luc45 in #209

Full Changelog: 0.6.0...0.7.0

0.6.0

QIT CLI 0.6.0 Release Notes

We are excited to announce the release of QIT CLI version 0.6.0! This update introduces the Activation Test 2.0, our first smart generic test, along with several enhancements designed to improve your testing workflow.

This is the first test type built on top of our Custom Tests platform!

What's New

Activation Test 2.0

• Improved Coverage: Now visits pages added by your extensions, catching errors that previous tests might have missed.
• Smart Generic Tests: Automatically detects and tests essential functionalities without custom scripts.
• Fast Execution: Completes tests in 50 seconds to 3 minutes, depending on your setup.
• Local Testing: Run tests directly on your machine or CI environment for immediate feedback.
• Error Detection: Captures PHP and JavaScript notices, warnings, and errors, including those on custom plugin pages, as well as Database errors.
• Configurable Environment:
  • PHP Versions: Test with PHP 7.2 up to 8.4 RC 3.
  • WooCommerce & WordPress Versions: Choose any version, including development builds.
• Plugin Dependencies: Activate additional plugins by their slugs to simulate real-world scenarios.
• Notifications: Use --notify to receive immediate alerts when issues are detected.
• Centralized Results: All test results are collected in QIT for easy access and analysis.
• Built on Custom Tests: Leverages existing frameworks for efficiency and easier maintenance.

Backwards Compatibility

• Seamless Integration: Designed to work with existing CI workflows without disruptions.
• Minimal Adjustments: No significant changes required for extensions currently using QIT.

Documentation

You can find the documentation for the Activation Test type here.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Fix --source option by @Luc45 in #196
• Sync 9.2.2 by @zhongruige in #197
• Add self-tests for the Validation test by @Luc45 in #198
• Activation as a custom test by @Luc45 in #200
• 9.3.0-beta.1 sync snapshots by @Luc45 in #201
• Activation/custom test improvements by @Luc45 in #202
• Update snapshots by @Luc45 in #204
• Fix old doc link by @zhongruige in #205
• Update snapshots for validation tests by @Luc45 in #206
• Activation refactor backwards compatibility by @Luc45 in #203
• Support qm logs by @MrJnrman in #195

Full Changelog: 0.5.4...0.6.0

0.5.4

QIT CLI 0.5.4 Release Notes

• Adds support for a QIT_RESULTS_DIR env var which specifies where to store the e2e results when running run:e2e command.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Update snapshots for Security Tests by @Luc45 in #179
• 9.0.0 sync by @Luc45 in #181
• Snapshot updates for WC 9.0.1 by @zhongruige in #182
• Sync e2e snapshots for WC 9.1.0 and 9.0.2 tests by @zhongruige in #183
• Sync WC 9.1.0 and 9.1.1 by @zhongruige in #184
• Wait 10 seconds between each self-test test by @Luc45 in #185
• Configurable poll intervall to prevent 429 by @Luc45 in #186
• Temporary test environment tweaks by @Luc45 in #190
• Sync 9.2.0-beta.1 by @Luc45 in #191
• Fix typo in docker compose version by @Luc45 in #192
• Set Result Dir using ENV vat by @MrJnrman in #194

Full Changelog: 0.5.3...0.5.4

0.5.3

QIT CLI 0.5.3 Release Notes

• This is a follow-up to the 0.5.2 release. Fixes an issue with using "--woo" parameter that would cause a Custom E2E test files to not be found.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Use Input instead of argv, activate Woo instead of bootstrapping when using "--woo" parameter by @Luc45 in #178

Full Changelog: 0.5.2...0.5.3

0.5.2

QIT CLI 0.5.2 Release Notes

• Fix an error that can happen when using both "--woo" and "--plugin=woocommerce" at the same time when running a Custom E2E Test.
• Many fixes and improvements

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• 9.0.0-RC1 snapshots with skips by @Luc45 in #172
• Custom tests snapshots by @Luc45 in #173
• Increase mass test timeout by @Luc45 in #175
• Sync Woo E2E 8.9.2 snapshots by @Luc45 in #174
• Update snapshots by @Luc45 in #176
• Fix issue with "--plugins=woocommerce" and "--woo" parameter by @Luc45 in #177

Full Changelog: 0.5.1...0.5.2

0.5.1

QIT CLI 0.5.1 Release Notes

• Register a test run in QIT for local custom test runs, with an embedded Allure Report with support for screenshots and videos of failures in E2E. You can share a link to that test report with others.
• Many fixes and improvements

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Add woo:validate-zip <path> command by @rcstr in #163
• Upload test artifacts to debug self-tests flakiness by @Luc45 in #167
• Sync WC 8.9.0 by @zhongruige in #168
• 9.0.0-beta.2 sync by @Luc45 in #171
• Send custom tests results to Manager by @Luc45 in #170

New Contributors

• @rcstr made their first contribution in #163

Full Changelog: 0.5.0...0.5.1

0.5.0

QIT CLI 0.5.0 Release Notes

• New Feature: Local Test Environments: Added the ability to create Local Test Environments with a single command. These environments are disposable and fully customizable, making automated testing much simpler. (Read More: Test Environment Documentation)
• New Feature: Custom Tests: Added support for Custom E2E tests, allowing developers to write, share, and run personalized automated tests easily. (Read more: Custom Tests Documentation)
• Many fixes and improvements

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Use Composer CA Bundle instead of custom implementation by @Luc45 in #139
• When doing a retry, throw some randomness into the mix by @Luc45 in #140
• Hide deprecated options (except on CI) by @Luc45 in #141
• Xdebug on QIT tests by @Luc45 in #142
• PHPCS Develop by @Luc45 in #145
• Add self-tests for WordPress.DB in Security Tests by @Luc45 in #146
• Custom E2E Tests 1 - Renaming Environment to Backend by @Luc45 in #133
• Custom E2E Tests 2 - Local Test Environment by @Luc45 in #130
• Local test env tweaks by @Luc45 in #147
• Update malware tests by @MrJnrman in #150
• Update snapshots for 8.7.0 sync by @Luc45 in #149
• Add empty extension list exception by @zhongruige in #152
• 8.8.0-beta.1 sync by @Luc45 in #155
• Added nonJsonOutput property by @zhongruige in #153
• Port Fast PHPCS PR to CLI by @Luc45 in #156
• Update WC 8.8 rc test snapshots by @zhongruige in #158
• Readme suggestions by @Luc45 in #159
• Sync 8.8.2 by @Luc45 in #161
• Rename woo.com to woocommerce.com by @zhongruige in #157
• Sync 8.9.0 RC by @zhongruige in #164
• Custom Tests by @Luc45 in #148
• Add more badges to the readme by @Luc45 in #165
• Tweaks to release process by @Luc45 in #166

Full Changelog: 0.4.2...0.5.0

0.4.2

QIT CLI 0.4.2 Release Notes

• When encountering "429 - Too Many Requests" when starting tests, respect the "Retry-After" header if it's present, otherwise do an exponential backoff.
• When running on Windows without a Certificate Authority file, ask the user if they want to download a CA file to be used in QIT, related to php/php-src#13601.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Update snapshots 8.7.0-beta.2 by @Luc45 in #135
• Retry after 429 by @Luc45 in #137
• Windows SSL by @Luc45 in #138

Full Changelog: 0.4.1...0.4.2

0.4.1

QIT CLI 0.4.1 Release Notes

• Fix the config path on native Windows.
• Groundwork for upcoming local test environments and integrated custom E2E tests for wide-scale compatibility testing.

For any queries, issues, or suggestions, feel free to visit our Github page at https://github.com/woocommerce/qit-cli/issues. We appreciate your feedback as it helps us to continuously improve our tool.

What's Changed

• Update docs by @Luc45 in #119
• Update snapshots for Woo 8.6.0-beta.1 E2E test sync by @Luc45 in #120
• Remove analytics-data.spec.js by @Luc45 in #121
• Allow retry on get request by @Luc45 in #123
• Test snapshots for 8.6.0 rc.1 by @Luc45 in #124
• Use dev version of symfony/process by @Luc45 in #126
• Rename "api" to "woo-api" and "e2e" to "woo-e2e" by @Luc45 in #125
• Minor tweak by @Luc45 in #128
• Update E2E snapshots for Woo 8.6.1 by @Luc45 in #129
• Add self-tests for Activation Tests loopback by @Luc45 in #131
• Fix QIT config path in Windows by @Luc45 in #134

Full Changelog: 0.4.0...0.4.1