[CT-661] test_builtin_set_function is flaky

[jtcohen6]

Flaky (flakey?) test added by #5107:

dbt-core/tests/functional/context_methods/test_builtin_functions.py

Lines 43 to 48 in 0d8e061

	def test_builtin_set_function(self, project):
	_, log_output = run_dbt_and_capture(["--debug", "run-operation", "validate_set"])
	expected_set = {False, 1, 2, 3, "foo"}
	assert f"set_result: {expected_set}" in log_output
	assert f"try_set_result: {expected_set}" in log_output

FAILED tests/functional/context_methods/test_builtin_functions.py::TestContextBuiltins::test_builtin_set_function

This is cropping up, unreliably, on a variety of systems and Python versions:

E assert "set_result: {False, 1, 2, 3, 'foo'}" in '\n\n============================== 2022-05-18 09:53:50.328276 | 8031f5a4-c01a-455b-9acf-4ebac14497f5 ==============================\n09:53:50.328283 [info ] [MainThread]: Running with dbt=1.2.0-a1\n09:53:50.329122 [debug] [MainThread]: running dbt with arguments {\'debug\': True, \'write_json\': True, \'use_colors\': True, \'printer_width\': 80, \'version_check\': True, \'partial_parse\': True, \'static_parser\': True, \'profiles_dir\': \'/tmp/pytest-of-runner/pytest-1/popen-gw1/profile6\', \'send_anonymous_usage_stats\': False, \'event_buffer_size\': 100000, \'quiet\': False, \'no_print\': False, \'macro\': \'validate_set\', \'args\': \'{}\', \'which\': \'run-operation\', \'rpc_method\': \'run-operation\', \'indirect_selection\': \'eager\'}\n09:53:50.329772 [debug] [MainThread]: Tracking: do not track\n09:53:50.374089 [info ] [MainThread]: Partial parse save file not found. Starting full parse.\n09:53:50.420050 [debug] [MainThread]: Parsing macros/validate_zip.sql\n09:53:50.430266 [debug] [MainThread]: Parsing macros/validate_set.sql\n09:53:50.443607 [debug] [MainThread]: Parsing macros/catalog.sql\n09:53:50.451913 [debug] [MainThread]: Parsing macros/adapters.sql\n09:53:50.593...t_custom_name/get_custom_alias.sql\n09:53:51.743135 [debug] [MainThread]: Parsing tests/generic/builtin.sql\n09:53:52.873105 [debug] [MainThread]: Acquiring new postgres connection "macro_validate_set"\n09:53:52.873855 [debug] [MainThread]: Using postgres connection "macro_validate_set"\n09:53:52.874491 [debug] [MainThread]: On macro_validate_set: BEGIN\n09:53:52.875207 [debug] [MainThread]: Opening a new connection, currently in state init\n09:53:52.903665 [debug] [MainThread]: SQL status: BEGIN in 0.03 seconds\n09:53:52.904474 [debug] [MainThread]: On macro_validate_set: COMMIT\n09:53:52.905145 [debug] [MainThread]: Using postgres connection "macro_validate_set"\n09:53:52.905778 [debug] [MainThread]: On macro_validate_set: COMMIT\n09:53:52.916436 [debug] [MainThread]: SQL status: COMMIT in 0.01 seconds\n09:53:52.926373 [debug] [MainThread]: set_result: {False, \'foo\', 1, 2, 3}\n09:53:52.927120 [debug] [MainThread]: try_set_result: {False, \'foo\', 1, 2, 3}\n09:53:52.935994 [debug] [MainThread]: On macro_validate_set: Close\n09:53:52.939436 [debug] [MainThread]: Flushing usage events\n09:53:52.940424 [debug] [MainThread]: Connection \'macro_validate_set\' was properly closed.\n'

I guess because it's looking for {False, 1, 2, 3, 'foo'} and finds {False, \'foo\', 1, 2, 3}?

[gshank]

Yes. The order of the set isn't guaranteed, so we'll have to figure out some other way of checking this.