Unit Tests incompatible with PHPUnit 8

[lbajsarowicz]

Magento is slowly moving towards the latest dependencies in the PHP world. Part of this is to have (at least) supported version of PHPUnit ( https://phpunit.de/supported-versions.html )

Preconditions (*)

1. Current Unit, Integration and API functional tests are not compatible with PHPUnit 8

Steps to reproduce (*)

1. Install PHPUnit 8 on separate directory and run Unit Tests from Magento

Expected result (*)

1. Tests should pass.

Actual result (*)

1. Tests are failing due to multiple compatibility issues

Contribution

What needs to be done

Most of the cases, the scenario to make Unit Tests compatible with PHPUnit 8 is:

1. setUp and tearDown methods has to have : void return type hint
2. Replace PHPUnit_Framework_TestCase with PHPUnit\Framework\TestCase
3. Replace \PHPUnit_Framework_MockObject_MockObject with \PHPUnit\Framework\MockObject\MockObject
4. Replace @expectedException \Magento\Framework\Exception\ConfigurationMismatchException annotation with $this->expectException(\Magento\Framework\Exception\ConfigurationMismatchException::class)
5. Replace @expectedExceptionMessage with $this->expectExceptionMessage('The message');

If you find any other typical steps to be done - let me know in the comment, I'm going to update the issue.

PHPUnit 9+ compatibility

https://phpunit.de/announcements/phpunit-8.html

Review the use of deprecated methods: assertContains, assertNotContains etc.
... and annotations expectException, expectExceptionMessage etc.

Notice!

Make sure that you have linted your code after finish your work!
❗ ❗ ❗ If you see Tests that test nothing - rewrite or remove them. Let's avoid massive delusion of Tests coverage, based on the tests that test implementation instead of method / unit outcome!

Modules

Due to some compatibility issues and to make Code Review easier - please do your changes per module. You can find list of checkboxes underneath with list of modules that should be covered. When you start working on specific module, please check the chekbox and add the comment which module you start work. When you finish - edit your commend adding the link to PR.

Open Source

• app/code/Magento/AdminNotification
• app/code/Magento/AdminAnalytics
• app/code/Magento/AdvancedPricingImportExport
• app/code/Magento/AdvancedSearch
• app/code/Magento/Amqp
• app/code/Magento/Analytics
• app/code/Magento/AsynchronousOperations
• app/code/Magento/Authorization
• app/code/Magento/Authorizenet
• app/code/Magento/Backend
• app/code/Magento/Backup
• app/code/Magento/Braintree
• app/code/Magento/Bundle
• app/code/Magento/BundleImportExport
• app/code/Magento/CacheInvalidate
• app/code/Magento/Captcha
• app/code/Magento/CardinalCommerce
• app/code/Magento/Catalog
• app/code/Magento/CatalogImportExport
• app/code/Magento/CatalogInventory
• app/code/Magento/CatalogRule
• app/code/Magento/CatalogRuleConfigurable
• app/code/Magento/CatalogSearch
• app/code/Magento/CatalogUrlRewrite
• app/code/Magento/CatalogWidget
• app/code/Magento/Checkout
• app/code/Magento/CheckoutAgreements
• app/code/Magento/Cms
• app/code/Magento/CmsUrlRewrite
• app/code/Magento/Config
• app/code/Magento/ConfigurableImportExport
• app/code/Magento/ConfigurableProduct
• app/code/Magento/Contact
• app/code/Magento/Cookie
• app/code/Magento/Cron
• app/code/Magento/CurrencySymbol
• app/code/Magento/Customer
• app/code/Magento/CustomerImportExport
• app/code/Magento/Deploy
• app/code/Magento/Developer
• app/code/Magento/Directory
• app/code/Magento/Downloadable
• app/code/Magento/DownloadableImportExport
• app/code/Magento/Eav
• app/code/Magento/Elasticsearch
• app/code/Magento/Elasticsearch6
• app/code/Magento/Elasticsearch7
• app/code/Magento/Email
• app/code/Magento/EncryptionKey
• app/code/Magento/GiftMessage
• app/code/Magento/GoogleAdwords
• app/code/Magento/GoogleAnalytics
• app/code/Magento/GoogleOptimizer
• app/code/Magento/GroupedImportExport
• app/code/Magento/GroupedProduct
• app/code/Magento/ImportExport
• app/code/Magento/Indexer
• app/code/Magento/InstantPurchase
• app/code/Magento/Integration
• app/code/Magento/LayeredNavigation
• app/code/Magento/Marketplace
• app/code/Magento/MediaStorage
• app/code/Magento/MessageQueue
• app/code/Magento/Msrp
• app/code/Magento/Multishipping
• app/code/Magento/MysqlMq
• app/code/Magento/NewRelicReporting
• app/code/Magento/Newsletter
• app/code/Magento/OfflinePayments
• app/code/Magento/OfflineShipping
• app/code/Magento/PageCache
• app/code/Magento/Payment
• app/code/Magento/Paypal
• app/code/Magento/Persistent
• app/code/Magento/ProductAlert
• app/code/Magento/ProductVideo
• app/code/Magento/Quote
• app/code/Magento/ReleaseNotification
• app/code/Magento/Reports
• app/code/Magento/RequireJs
• app/code/Magento/Review
• app/code/Magento/Robots
• app/code/Magento/Rss
• app/code/Magento/Rule
• app/code/Magento/Sales
• app/code/Magento/SalesInventory
• app/code/Magento/SalesRule
• app/code/Magento/SalesSequence
• app/code/Magento/SampleData
• app/code/Magento/Search
• app/code/Magento/Security
• app/code/Magento/SendFriend
• app/code/Magento/Shipping
• app/code/Magento/Signifyd
• app/code/Magento/Sitemap
• app/code/Magento/Store
• app/code/Magento/Swagger
• app/code/Magento/SwaggerWebapi
• app/code/Magento/SwaggerWebapiAsync
• app/code/Magento/Swatches
• app/code/Magento/Tax
• app/code/Magento/TaxImportExport
• app/code/Magento/Theme
• app/code/Magento/Translation
• app/code/Magento/Ui
• app/code/Magento/UrlRewrite
• app/code/Magento/User
• app/code/Magento/Variable
• app/code/Magento/Vault
• app/code/Magento/Webapi
• app/code/Magento/WebapiAsync
• app/code/Magento/Weee

B2B

• .b2b/app/code/Magento/BundleNegotiableQuote
• .b2b/app/code/Magento/BundleSharedCatalog
• .b2b/app/code/Magento/Company
• .b2b/app/code/Magento/CompanyCredit
• .b2b/app/code/Magento/CompanyPayment
• .b2b/app/code/Magento/ConfigurableNegotiableQuote
• .b2b/app/code/Magento/ConfigurableSharedCatalog
• .b2b/app/code/Magento/GiftCardRequisitionList
• .b2b/app/code/Magento/GiftCardSharedCatalog
• .b2b/app/code/Magento/GroupedRequisitionList
• .b2b/app/code/Magento/GroupedSharedCatalog
• .b2b/app/code/Magento/NegotiableQuote
• .b2b/app/code/Magento/NegotiableQuoteSharedCatalog
• .b2b/app/code/Magento/OrderHistorySearch
• .b2b/app/code/Magento/QuickOrder
• .b2b/app/code/Magento/RequisitionList
• .b2b/app/code/Magento/SharedCatalog

EE

• .ee/app/code/Magento/AdminGws
• .ee/app/code/Magento/AdvancedCatalog
• .ee/app/code/Magento/AdvancedCheckout
• .ee/app/code/Magento/AdvancedRule
• .ee/app/code/Magento/AdvancedSalesRule
• .ee/app/code/Magento/Banner
• .ee/app/code/Magento/BannerCustomerSegment
• .ee/app/code/Magento/BundleImportExportStaging
• .ee/app/code/Magento/BundleStaging
• .ee/app/code/Magento/CatalogEvent
• .ee/app/code/Magento/CatalogImportExportStaging
• .ee/app/code/Magento/CatalogInventoryStaging
• .ee/app/code/Magento/CatalogPermissions
• .ee/app/code/Magento/CatalogRuleStaging
• .ee/app/code/Magento/CatalogStaging
• .ee/app/code/Magento/CheckoutStaging
• .ee/app/code/Magento/CmsStaging
• .ee/app/code/Magento/ConfigurableProductStaging
• .ee/app/code/Magento/CustomAttributeManagement
• .ee/app/code/Magento/CustomerBalance
• .ee/app/code/Magento/CustomerCustomAttributes
• .ee/app/code/Magento/CustomerFinance
• .ee/app/code/Magento/CustomerSegment
• .ee/app/code/Magento/Cybersource
• .ee/app/code/Magento/Enterprise
• .ee/app/code/Magento/Eway
• .ee/app/code/Magento/GiftCard
• .ee/app/code/Magento/GiftCardAccount
• .ee/app/code/Magento/GiftCardImportExport
• .ee/app/code/Magento/GiftRegistry
• .ee/app/code/Magento/GiftWrapping
• .ee/app/code/Magento/GoogleTagManager
• .ee/app/code/Magento/GroupedProductStaging
• .ee/app/code/Magento/Invitation
• .ee/app/code/Magento/Logging
• .ee/app/code/Magento/MultipleWishlist
• .ee/app/code/Magento/PaymentStaging
• .ee/app/code/Magento/PersistentHistory
• .ee/app/code/Magento/PricePermissions
• .ee/app/code/Magento/PromotionPermissions
• .ee/app/code/Magento/Reminder
• .ee/app/code/Magento/ResourceConnections
• .ee/app/code/Magento/ReviewStaging
• .ee/app/code/Magento/Reward
• .ee/app/code/Magento/Rma
• .ee/app/code/Magento/SalesArchive
• .ee/app/code/Magento/SalesRuleStaging
• .ee/app/code/Magento/ScalableCheckout
• .ee/app/code/Magento/ScalableInventory
• .ee/app/code/Magento/ScalableOms
• .ee/app/code/Magento/ScheduledImportExport
• .ee/app/code/Magento/Staging
• .ee/app/code/Magento/Support
• .ee/app/code/Magento/TargetRule
• .ee/app/code/Magento/VersionsCms
• .ee/app/code/Magento/VisualMerchandiser
• .ee/app/code/Magento/WebsiteRestriction
• .ee/app/code/Magento/Worldpay
• .ee/lib/internal/Magento/Framework/ForeignKey/Test/Unit

Inventory module

• ext/magento/inventory/Inventory/
• ext/magento/inventory/InventoryImportExport/
• ext/magento/inventory/InventoryMultiDimensionalIndexerApi/
• ext/magento/inventory/InventoryReservations/

[m2-assistant]

Hi @lbajsarowicz. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

@lbajsarowicz do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[dmytro-ch]

Hi @lbajsarowicz, I'll take care of app/code/Magento/AdvancedSearch. :)
Thank you!

[lbajsarowicz]

I'm going to take care of Magento/Cron module

[AndreyChorniy]

Hi @lbajsarowicz, I would like to care about app/code/Magento/Newsletter

[cristiano-pacheco]

Hi @lbajsarowicz, I would like to take care app/code/Magento/Payment

[AndreyChorniy]

Hey @lbajsarowicz,
I have added PR for compatibility test with PHPUnit8 for module Magento_Authorization

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27814 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• eac7b83606a271a0dc776f1ab4eb13da646204ff

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27815 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 1804ef51108849b01f803c204f781587dbb9e8e5
• 5829242bc1dafe8cf00452261c0dba36bcb5b7f9

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27816 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 8141fce164c5abfbb13a47cedf9e8da4f295fb14

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27817 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 99e517fc9cd1cb332adf1553e6f503e74a9c2ca6
• 1ad40ea947a40b2016de5d26d9804c3760238de4

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27818 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 0a5c603b870df0a7259241f9d50d28bc660cb8dd
• f3fcbb867c5ccad890f510e321b18b34d38e87e0
• 48561b9ec692892442e9ffd4e7b03280fc42506d

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27819 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• b7faa405fe88fe943b589d987932c16a1fed13e5
• 91fa4e1ccbfa1d1ddf14ea9544bdd9a893d1be54
• 2d31d6c451b11c1ac4f137fe3e22cd0af8018922

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27820 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 0c346ed82117f5493b3fcf9588eaa07df46e388a

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27821 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 3e84fafd06410dff5fbfe06ef3bf2105c09eb494

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27822 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• c29aadc0316b0ba6f48c00562768c45794c2ee82
• 6d922fe6f16cf71c39d5c5adef39e7300fe5994f
• 2e9a64323837d9cb61cffaaa5b548d11ef1e6f94
• a7561ed6fa025b17c7c5f15f7b74fdce3378baf1
• 45c87daeb7fe973da6a68c574636653b58732e7b
• f829ab0d4e5144b4c20ca85042fb513f450c132b
• d1f2f0300fb30563347aa0f868e60015301ccfc6
• 868d898efe47ba52ba7be9c3a851337899185074
• 9ff82b82c887a78278c557f8be22539ece5edd5f

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27823 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 5d2fd3d3db5b2bde01e49aef870ba0519b0fa814
• a09b56d7ae4545ab2c18400b1afc28e96ff24ce3
• bc8159bb04e281020d07848312a67f214ecc4bab
• dc9ff87d9970a1804c44f936efc895fb23653084
• 44151073b0cc485cc349bb2465447a4c88fdfe29
• 4f0d66130965fbb8d89ab7350ff1f65db1595e62
• 20df9c132f84c533cbb087c1a6aa69bc3c9997bf
• f8ba88a1ce90ea59587b683fb0d9b79be1754e99
• 7ff0bf84cb61a0b726488582aa25fe2ad741f8fa
• aff74ded835c7c243cf33de0efbabf063ec0e655
• 30ac195d75f056f43992d978add5c5cc20c3145a
• c21be03151c9ca99795251266d7db5d5e8ce111f
• c07b96d931a45d4a2178e68321a13e7587b6ba21
• 49a8b3453e559f2cd55286f39fcf5b4eb4df3cbe
• c7d6c2cf21a86bbd2568bce77183c40b3ff522dc
• efa81d78176c80c5f7b61b4740c1d99b312919b7

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27824 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 02aa0ad306d653625481982d2f9c95b8dee0d7c8
• 4e0f00e83e3ae4bdde365d84afdd7e99a4a341a4

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27825 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• d5743c7bd0397bd4f4107765f0a426d6547ebcaa
• a327ad91b845a46e6c762ef7a334bd7f3d2a3fcb

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27826 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• b1fae3c6f842da7af2218e0792466093163aba1a
• e51d8126a9bc968916d9a02a7d70383019b7252a
• a2cee496e2e89c23a9875fa1e7044735d22219c1

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27827 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 5eb9681307f126b5a90297aa8db25e00da591966
• 35664d517d3c33424978e7997680ba0cc23cbbca
• 2da330c53ad39db77c54caee20a7efb7a2b47ef4

The fix will be available with the upcoming release.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27885 by @lbajsarowicz in phpunit8 branch
Related commit(s):

• 9d9a59f4cde3a6fcf5921c8b84253e72ce05f8f3

The fix will be available with the upcoming release.

[zhartaunik]

cc @lbajsarowicz

magento@fpm:/var/www/magento$ vendor/bin/phpunit -c /var/www/magento/dev/tests/unit/phpunit.xml /var/www/magento/app/code/Magento/CatalogImportExport/Test/Unit/Model/Export/
PHPUnit 9.1.5 by Sebastian Bergmann and contributors.

  Warning - The configuration file did not pass validation!
  The following problems have been detected:

  Line 18:
  - Element 'testsuite': This element is not expected.

  Test results may not be as expected.


EEEEE                                                                                                                                                                                          5 / 5 (100%)

Time: 00:00.782, Memory: 16.00 MB

There were 5 errors:

1) Magento\CatalogImportExport\Test\Unit\Model\Export\ProductTest::testGetEntityTypeCode
ArgumentCountError: Too few arguments to function Magento\Catalog\Model\ResourceModel\ProductFactory::__construct(), 0 passed in /var/www/magento/vendor/phpunit/phpunit/src/Framework/MockObject/Generator.php on line 539 and at least 1 expected

/var/www/magento/generated/code/Magento/Catalog/Model/ResourceModel/ProductFactory.php:29
/var/www/magento/app/code/Magento/CatalogImportExport/Test/Unit/Model/Export/ProductTest.php:175

2) Magento\CatalogImportExport\Test\Unit\Model\Export\ProductTest::testUpdateDataWithCategoryColumnsNoCategoriesAssigned
ArgumentCountError: Too few arguments to function Magento\Catalog\Model\ResourceModel\ProductFactory::__construct(), 0 passed in /var/www/magento/vendor/phpunit/phpunit/src/Framework/MockObject/Generator.php on line 539 and at least 1 expected

/var/www/magento/generated/code/Magento/Catalog/Model/ResourceModel/ProductFactory.php:29
/var/www/magento/app/code/Magento/CatalogImportExport/Test/Unit/Model/Export/ProductTest.php:175

3) Magento\CatalogImportExport\Test\Unit\Model\Export\ProductTest::testGetHeaderColumns
ArgumentCountError: Too few arguments to function Magento\Catalog\Model\ResourceModel\ProductFactory::__construct(), 0 passed in /var/www/magento/vendor/phpunit/phpunit/src/Framework/MockObject/Generator.php on line 539 and at least 1 expected

/var/www/magento/generated/code/Magento/Catalog/Model/ResourceModel/ProductFactory.php:29
/var/www/magento/app/code/Magento/CatalogImportExport/Test/Unit/Model/Export/ProductTest.php:175

4) Magento\CatalogImportExport\Test\Unit\Model\Export\ProductTest::testExportCountZeroBreakInternalCalls
ArgumentCountError: Too few arguments to function Magento\Catalog\Model\ResourceModel\ProductFactory::__construct(), 0 passed in /var/www/magento/vendor/phpunit/phpunit/src/Framework/MockObject/Generator.php on line 539 and at least 1 expected

/var/www/magento/generated/code/Magento/Catalog/Model/ResourceModel/ProductFactory.php:29
/var/www/magento/app/code/Magento/CatalogImportExport/Test/Unit/Model/Export/ProductTest.php:175

5) Magento\CatalogImportExport\Test\Unit\Model\Export\ProductTest::testExportCurPageEqualToLastBreakInternalCalls
ArgumentCountError: Too few arguments to function Magento\Catalog\Model\ResourceModel\ProductFactory::__construct(), 0 passed in /var/www/magento/vendor/phpunit/phpunit/src/Framework/MockObject/Generator.php on line 539 and at least 1 expected

/var/www/magento/generated/code/Magento/Catalog/Model/ResourceModel/ProductFactory.php:29
/var/www/magento/app/code/Magento/CatalogImportExport/Test/Unit/Model/Export/ProductTest.php:175

ERRORS!
Tests: 5, Assertions: 0, Errors: 5.

Need to ->disableOriginalConstructor() for 3 Factories or am I running something wrong?

[ihor-sviziev]

@zhartaunik do you have this issue on 2.4-develop branch?

[zhartaunik]

@zhartaunik do you have this issue on 2.4-develop branch?

Yes

[ihor-sviziev]

@zhartaunik could you report separate issue for that? Thank you!

[zhartaunik]

Hi @ihor-sviziev
I noticed that I was mistaken with my branches and this issue referred to some old revision.
Tests work well now.

[magento-engcom-team]

Hi @lbajsarowicz. Thank you for your report.
The issue has been fixed in #27832 by @Krielkip in 2.4-develop branch
Related commit(s):

• 65c06eb13208f9f4fe851ae136e61fe5a06642e4
• 7f4b2766b8b068ca03595d0548e84ed94c47773b
• 21ad13f6880fa6be86b18a993d38dbb18752916b
• 8f1020af9f5cca737bb24696769f7eb81958556a
• cfb762d664a2da0933ee1a6fc46deb684fc6eecd
• c6a43f3920d29c046ca7a097e520f30f72f14ac7
• 84124f74b55a88b09947f1fc561a4ad30ecdd07b
• 43b4f648a1efd0a24c668b2f80f34f7d60b6f649
• 9fd54363ced22b9d627f0768cf1820d125468e80

The fix will be available with the upcoming 2.4.2 release.