Add Core BPF Migrations to Program-Test

[buffalojoec]

Problem

Currently, solana-program-test has no way to override a builtin in any Bank instance because Bank's finish_init and apply_builtin_program_feature_activations use the static BUILTINS list to add all of the required builtin programs.

When testing a should-be Core BPF program using its desired address (ie. Config1111111111111111111111111111111111111), any Bank used within program-test will have overwritten this program with its builtin version.

I thought this was made possible by solana-labs/solana#35145, but this only works if prefer_bpf is set to false.

Solution

There are a few ways we could do this. Keep in mind that as of solana-labs/solana#34407, it's no longer possible to just store the two accounts under BPF Loader Upgradeable, since the program cache will panic if the program's current state doesn't match the last cache entry.

One idea is to deploy the program on every test setup using the loader itself. This can be done in two ways: send the transaction(s) required to deploy or use solana_bpf_loader_program::direct_deploy_program.

Alternatively, since all of the steps involved in invoking solana_bpf_loader_program::direct_deploy_program are contained in the Core BPF migration path, it's likely simpler to just wire in the functionality to trigger a feature-activated Core BPF migration on the newly created Bank in setup_bank.

[buffalojoec]

cc @2501babe @joncinque unless there's some simple way to do this I'm missing, then I'm probably going to go ahead and implement the last paragraph's approach on the coattails of anza-xyz/agave#525

[joncinque]

This brings up a larger question, which is how we'll bundle the core BPF programs in dev tools like program-test and test-validator.

Once the core BPF program is released on all networks, we just need to always include the BPF program at the correct address, same as how spl-token and friends are included, so that's easy.

What do we do for 2.0? In that situation, we have the feature gates to migrate the programs, and people can choose to enable or disable the features. If someone chooses to enable (or just uses the default, which is all_enabled), then we'll definitely need to run a code path to migrate the program.

All that to say: yep, makes sense!

[buffalojoec]

What do we do for 2.0? In that situation, we have the feature gates to migrate the programs, and people can choose to enable or disable the features. If someone chooses to enable (or just uses the default, which is all_enabled), then we'll definitely need to run a code path to migrate the program.

Yeah that's a great point. For any feature gates we manage to squeeze into 2.0 we'll do exactly that.

However, for those programs that don't get prod feature gates in 2.0, we're going to have to piggy back on the mock (test-only) feature gates introduced in anza-xyz/agave#525.

This would really be just for us in program-test. Once their prod feature gates are added and included in a release, then they can join the crew!

[buffalojoec]

As of anza-xyz/agave#1418, this is ready to go. All we have to do is add the program to genesis, and it will never be replaced by a builtin.