-
Notifications
You must be signed in to change notification settings - Fork 312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate snapshot tests away from unmaintained testing frameworks #2791
Comments
Are any of these libraries shared across Mapbox SDKs? Wondering if there's an opportunity here to consolidate testing strategies across products. /cc: @knov |
|
The map SDK uses XCUITest directly for UI testing, but there’s a limit to how much we can share approaches between the two SDKs, because testing the internals of a view is different than testing a whole UI. |
iOSSnapshotTestCase is a popular successor to FBSnapshotTestCase, but it isn’t maintained either and doesn’t support SPM: uber/ios-snapshot-test-case#91. |
SnapshotTesting looks like a promising replacement for all three unsupported frameworks. |
I'm actively working on reviving the iOSSnapshotTestCase repo, in the meantime I'll be working on SPM support on my branch until I get ownership of the original repo again, feel free to use that: https://github.com/alanzeino/ios-snapshot-test-case I've merged in SPM support today. Since SPM uses git URIs you can use that for now and change the URI back to the original repo once it's back under development again (note, the URI for the original repo will likely change again to a separate organization at this point). |
I'm struggling with configuring alanzeino/ios-snapshot-test-case with SPM build system. The problem is that I don't see a way to specify reference images path. According to docs, there are currently three ways:
We immediately dismiss the first option because we use Swift. A fork with SPM support isn't active as well and will need more improvements to make it suitable for us.
I successfully migrated Also, I suggest using git lfs to store snapshot images for the following reason: Committing images to the git repository dramatically increases the repository size. As time goes, more and more snapshot images will be added to the repository, increasing our repository size and consuming unnecessary bandwidth during repository checkout and disk space on user/developer machines.
cc @mapbox/navigation-ios @1ec5 |
The motivation for migrating to new snapshot testing library is described here: #2791 (comment). Main consequence is that we test only on one device. The env for current snapshots are: iPhone 12 Pro Max, iOS 14.5. Xcode based project isn't updated yet because the work depends on #3031.
Follow up on:
Unfortunately without a workaround, using git-lfs breaks SPM support for the clients with the following error:
This is significant usability downgrade to my opinion. |
The motivation for migrating to new snapshot testing library is described here: #2791 (comment). Main consequence is that we test only on one device. The env for current snapshots are: iPhone 12 Pro Max, iOS 14.5. Xcode based project isn't updated yet because the work depends on #3031.
The motivation for migrating to new snapshot testing library is described here: #2791 (comment). The env for current snapshots are: - iPhone 12 Pro Max, iOS 14.4 - iPhone 11 Pro Max, iOS 13.7 Xcode based project isn't updated yet because the work depends on #3031. Fix regression in test due to global dependency on applied Theme in tests. Some tests change the global theme. To return it to the expected state, we apply the needed theme in our snapshot tests. Temporary fix to InstructionLabel clear cache functionality Check #3043 (comment) for more info.
Implemented in #3043 in the release-v2.0 branch. |
MapboxNavigationTests currently relies on a varied assortment of testing frameworks; most of them are redundantly used for snapshot tests:
Some of these testing frameworks are unmaintained and probably won’t survive the transition to SPM in #2629. We should migrate any tests using Cedar, FBSnapshotTestCase, or SnappyShrimp to an actively maintained fork or an alternative library. Ideally, we should end up with fewer testing dependencies, because each dependency adds a learning curve and an unnecessary increase in the time it takes to set up a development environment using Carthage or CocoaPods.
/cc @mapbox/navigation-ios
The text was updated successfully, but these errors were encountered: