[Bug]: Activate version message does not respect stream alias

[spacecowboy]

Singer SDK Version

0.11.1

Python Version

3.9

Bug scope

Targets (data type handling, batching, SQL object generation, etc.)

Operating System

Linux

Description

ActivateVersion messages crash the tap if a stream map alias is used.

This is the offending function in target_base.py

    def _process_activate_version_message(self, message_dict: dict) -> None:
        """Handle the optional ACTIVATE_VERSION message extension.

        Args:
            message_dict: TODO
        """
        stream_name = message_dict["stream"]
        sink = self.get_sink(stream_name)
        sink.activate_version(message_dict["version"])

Note this:

sink = self.get_sink(stream_name)

This is wrong. It should be this:

sink = self.get_sink(stream_map.stream_alias)

Code

meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.695027Z","message":"INFO:target-bigquery:Initializing 'target-bigquery' target sink...","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.695375Z","message":"INFO:target-bigquery:Initializing target sink for stream 'e_call_center_candidate_submission'...","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.742855Z","message":"Traceback (most recent call last):","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.743201Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/bin/target-bigquery\", line 8, in <module>","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.743459Z","message":"    sys.exit(TargetBigQuery.cli())","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.743688Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/click/core.py\", line 1130, in __call__","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.744026Z","message":"    return self.main(*args, **kwargs)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.744270Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/click/core.py\", line 1055, in main","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.744502Z","message":"    rv = self.invoke(ctx)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.744736Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/click/core.py\", line 1404, in invoke","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.744967Z","message":"    return ctx.invoke(self.callback, **ctx.params)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.745209Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/click/core.py\", line 760, in invoke","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.745439Z","message":"    return __callback(*args, **kwargs)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.745664Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/singer_sdk/target_base.py\", line 563, in cli","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.745894Z","message":"    target.listen(file_input)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.746119Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/singer_sdk/io_base.py\", line 34, in listen","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.746345Z","message":"    self._process_lines(file_input)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.746572Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/singer_sdk/target_base.py\", line 264, in _process_lines","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.746806Z","message":"    counter = super()._process_lines(file_input)","severity":"info"}
meltano_yobify_1            | Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
meltano_yobify_1            | join our friendly Slack community.
meltano_yobify_1            | 
meltano_yobify_1            | Run invocation could not be completed as block failed: Loader failed
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.747043Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/singer_sdk/io_base.py\", line 84, in _process_lines","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.747281Z","message":"    self._process_activate_version_message(line_dict)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.747537Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/singer_sdk/target_base.py\", line 403, in _process_activate_version_message","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.747779Z","message":"    sink = self.get_sink(stream_name)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.748016Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/singer_sdk/target_base.py\", line 153, in get_sink","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.748342Z","message":"    self._assert_sink_exists(stream_name)","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.748603Z","message":"  File \"/project/.meltano/loaders/target-bigquery/venv/lib/python3.9/site-packages/singer_sdk/target_base.py\", line 247, in _assert_sink_exists","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.748923Z","message":"    raise RecordsWithoutSchemaException(","severity":"info"}
meltano_yobify_1            | {"consumer":true,"producer":false,"string_id":"target-bigquery","cmd_type":"elb","stdio":"stderr","name":"target-bigquery","level":"info","timestamp":"2022-10-10T08:57:15.749170Z","message":"singer_sdk.exceptions.RecordsWithoutSchemaException: A record for stream 'public-e_call_center_candidate_submission' was encountered before a corresponding schema.","severity":"info"}
meltano_yobify_1            | {"exc_info":["<class 'meltano.core.runner.RunnerError'>","RunnerError('Loader failed')","<traceback object at 0x7f94e395b840>"],"timestamp":"2022-10-10T08:57:16.965835Z","level":"error","message":"Loader failed","severity":"error"}

[aaronsteers]

@spacecowboy - Thanks for raising this! A PR would be very welcome, but otherwise we will assign for a future iteration. 👍

[stale]

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

[edgarrmondragon]

Still relevant and open to contributions

[stale]

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.