[macOS] Generate universal gen_snapshots

[cbracken]

This reverts commit ac4c31a (#52913).
This relands commit 4e33c10 (#52885).

Previously, the gen_snapshot_arm64 and gen_snapshot_x64 binaries used by the tool were all built for x64 architecture. As such, developers building apps with Flutter rely on Rosetta translation with every build.

This refactors the gen_snapshot build rules on macOS hosts to consistently produce gen_snapshot_arm64 and gen_snapshot_x64 binaries with the target architecture of the build but with as universal binaries with both host architectures.

arm64 host build

Prior to this patch we emitted:

• gen_snapshot_arm64 (arch: x64, target_arch: simarm64)

After this patch, we emit:

• artifacts_x64/gen_snapshot_arm64 (arch: x64, target_arch: simarm64)
• artifacts_arm64/gen_snapshot_arm64 (arch: arm64, target_arch: arm64)
• gen_snapshot_arm64 (universal binary composed of both of the above)

x64 host build

Prior to this patch we emitted:

• gen_snapshot_x64 (arch: x64, target_arch: x64)

After this patch, we emit:

• artifacts_x64/gen_snapshot_x64 (arch: x64, target_arch: x64)
• artifacts_arm64/gen_snapshot_x64 (arch: arm64, target_arch: simx64)
• gen_snapshot_x64 (universal binary composed of both of the above)

Note that host builds on macOS currently default to a host architecture of x64 (can be overridden via --force-mac-arm64) regardless of host architecture and thus, the build itself relies on Rosetta translation when invoked on Apple Silicon arm64 hardware. This is to ensure a consistent build in CI regardless of bot architecture. See:

engine/tools/gn

Lines 502 to 505 in 6fa734d

	# TODO(cbracken):
	# https://github.com/flutter/flutter/issues/103386
	if get_host_os() == 'mac' and not args.force_mac_arm64:
	gn_args['host_cpu'] = 'x64'

Issue: flutter/flutter#101138
Issue: flutter/flutter#69157

Related issue: flutter/flutter#103386

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene wiki page, which explains my responsibilities.
• I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
• I listed at least one issue that this PR fixes in the description above.
• I added new tests to check the change I am making or feature I am adding, or the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
• I'm pretty sure something infra-related will go wrong with this reland, but I'm still not quite sure what it is.
• I updated/added relevant documentation (doc comments with ///).
• I signed the CLA.
• All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[flutter-dashboard]

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact "@test-exemption-reviewer" in the #hackers channel in Chat (don't just cc them here, they won't see it! Use Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

[stuartmorgan]

test-exempt: configuration change

[jmagman]

Passed in post-submit with longer timeout: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20mac_host_engine/11165/overview

[jfahrenkrug]

@cbracken Thanks for this fix! Do we have an idea which Flutter version this will be released in?

[jmagman]

This commit is at flutter/flutter@c9f72d5, check out
https://github.com/flutter/flutter/blob/master/docs/releases/Where's-my-commit.md#finding-the-versions-that-contain-framework-commit-x

[zanderso]

AFAICT, the zip archives produced by the GN rules for the gen_snapshot archive aren't actually being uploaded to storage and pulled by the tool.

See the build config file here where a different script is used to produce gen_snapshot.zip files. Those are the ones that get uploaded to storage.

I think the fix is to just upload the gen_snapshot.zip files produced by the individual builds instead of using a global generator after the fact. However, I suspect that GN rule that creates the gen_snpashot.zip file will have to also embed an entitlements.txt file like create_macos_gen_snapshot.py does here.

[jmagman]

AFAICT, the zip archives produced by the GN rules for the gen_snapshot archive aren't actually being uploaded to storage and pulled by the tool.

@cbracken, when you take a look at this can you also add an arch check to the framework so we can make sure the right ones are getting pulled in?
https://github.com/flutter/flutter/blob/32c0c39ea5d81f0c9b3e51a513c76c4ca3d5d111/packages/flutter_tools/test/general.shard/artifacts_test.dart#L45

Here's an arch check example
https://github.com/flutter/flutter/blob/32c0c39ea5d81f0c9b3e51a513c76c4ca3d5d111/packages/flutter_tools/test/host_cross_arch.shard/ios_content_validation_test.dart#L314-L318

[bryantwilliam]

Sorry if this is the wrong place to post. But when building with Circle CI using Fastlane for uploading an iOS app, I still get this error:

[00:36:14]: ▸ ❌  error: Target aot_assembly_release failed: Error: Flutter failed to run "/Users/distiller/usr/development/flutter/bin/cache/artifacts/engine/ios-release/gen_snapshot_arm64 --deterministic --snapshot_kind=app-aot-assembly --assembly=/Users/distiller/project/atomic_sdk/.dart_tool/flutter_build/1de751b82136897de44c980685a5c798/arm64/snapshot_assembly.S /Users/distiller/project/atomic_sdk/.dart_tool/flutter_build/1de751b82136897de44c980685a5c798/app.dill".
[00:36:14]: ▸ ** ARCHIVE FAILED **
[00:36:14]: ▸ The following build commands failed:
[00:36:14]: ▸   PhaseScriptExecution Run\ Script /Users/distiller/Library/Developer/Xcode/DerivedData/Runner-cjeecdxxvkifhlgusllkvvlhhchf/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/Runner.build/Release-iphoneos/Runner.build/Script-9740EEB61CF901F6004384FC.sh (in target 'Runner' from project 'Runner')
[00:36:14]: ▸ (1 failure)

Flutter version: 3.22.3

[jmagman]

Sorry if this is the wrong place to post. But when building with Circle CI using Fastlane for uploading an iOS app, I still get this error:

[00:36:14]: ▸ ❌  error: Target aot_assembly_release failed: Error: Flutter failed to run "/Users/distiller/usr/development/flutter/bin/cache/artifacts/engine/ios-release/gen_snapshot_arm64 --deterministic --snapshot_kind=app-aot-assembly --assembly=/Users/distiller/project/atomic_sdk/.dart_tool/flutter_build/1de751b82136897de44c980685a5c798/arm64/snapshot_assembly.S /Users/distiller/project/atomic_sdk/.dart_tool/flutter_build/1de751b82136897de44c980685a5c798/app.dill".
[00:36:14]: ▸ ** ARCHIVE FAILED **
[00:36:14]: ▸ The following build commands failed:
[00:36:14]: ▸   PhaseScriptExecution Run\ Script /Users/distiller/Library/Developer/Xcode/DerivedData/Runner-cjeecdxxvkifhlgusllkvvlhhchf/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/Runner.build/Release-iphoneos/Runner.build/Script-9740EEB61CF901F6004384FC.sh (in target 'Runner' from project 'Runner')
[00:36:14]: ▸ (1 failure)

Flutter version: 3.22.3

@bryantwilliam can you file a new issues? This PR is likely unrelated, and isn't in 3.22.3. Thanks!