[Bug]: StorageTarget.split_url() does not return head portion of tuple for Windows file paths

[BuzzCutNorman]

Singer SDK Version

0.11.1

Python Version

3.9

Bug scope

Targets (data type handling, batching, SQL object generation, etc.)

Operating System

Windows

Description

When running a tap and target combo with a batch_config present for the tap on Windows the run errors out when the target tries to process the batch files. I found the occurs because the function StorageTarget.split_url() which is used in SINK.process_batch_files() returns a blank head string when running on Windows. This is due to the functionality of fs.path.split() which does not split a given string when a \ is detected. Since Windows uses \ the given url string is returned as the tail. When with batch_fs.open(tail, mode="rb") as file: is called in SINK.process_batch_files() the run stops with the error contains invalid characters.

Code

2022-10-03T18:48:16.276605Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.292231Z [info     ]   File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.292231Z [info     ]     return _run_code(code, main_globals, None, cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.292231Z [info     ]   File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.292231Z [info     ]     exec(code, run_globals)    cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.292231Z [info     ]   File "C:\development\tap-stackoverflow-sampledata\.meltano\loaders\target-sqlbuzz\venv\Scripts\target-sqlbuzz.exe\__main__.py", line 7, in <module> cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.307856Z [info     ]   File "C:\development\tap-stackoverflow-sampledata\.meltano\loaders\target-sqlbuzz\venv\lib\site-packages\click\core.py", line 1130, in __call__ cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.307856Z [info     ]     return self.main(*args, **kwargs) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.307856Z [info     ]   File "C:\development\tap-stackoverflow-sampledata\.meltano\loaders\target-sqlbuzz\venv\lib\site-packages\click\core.py", line 1055, in main cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.307856Z [info     ]     rv = self.invoke(ctx)      cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.323478Z [info     ]   File "C:\development\tap-stackoverflow-sampledata\.meltano\loaders\target-sqlbuzz\venv\lib\site-packages\click\core.py", line 1404, in invoke cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.323478Z [info     ]     return ctx.invoke(self.callback, **ctx.params) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.323478Z [info     ]   File "C:\development\tap-stackoverflow-sampledata\.meltano\loaders\target-sqlbuzz\venv\lib\site-packages\click\core.py", line 760, in invoke cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.323478Z [info     ]     return __callback(*args, **kwargs) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.339104Z [info     ]   File "C:\development\sdk\singer_sdk\target_base.py", line 563, in cli cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.339104Z [info     ]     target.listen(file_input)  cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.339104Z [info     ]   File "C:\development\sdk\singer_sdk\io_base.py", line 34, in listen cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.339104Z [info     ]     self._process_lines(file_input) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.354728Z [info     ]   File "C:\development\sdk\singer_sdk\target_base.py", line 264, in _process_lines cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.354728Z [info     ]     counter = super()._process_lines(file_input) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.354728Z [info     ]   File "C:\development\sdk\singer_sdk\io_base.py", line 90, in _process_lines cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.354728Z [info     ]     self._process_batch_message(line_dict) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.354728Z [info     ]   File "C:\development\sdk\singer_sdk\target_base.py", line 415, in _process_batch_message cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.370353Z [info     ]     sink.process_batch_files(  cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.370353Z [info     ]   File "C:\development\sdk\singer_sdk\sinks\core.py", line 468, in process_batch_files cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.370353Z [info     ]     with batch_fs.open(tail, mode="rb") as file: cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.370353Z [info     ]   File "C:\development\tap-stackoverflow-sampledata\.meltano\loaders\target-sqlbuzz\venv\lib\site-packages\fs\osfs.py", line 639, in open cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.385976Z [info     ]     _path = self.validatepath(path) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.385976Z [info     ]   File "C:\development\tap-stackoverflow-sampledata\.meltano\loaders\target-sqlbuzz\venv\lib\site-packages\fs\osfs.py", line 686, in validatepath cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.385976Z [info     ]     return super(OSFS, self).validatepath(path) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.385976Z [info     ]   File "C:\development\tap-stackoverflow-sampledata\.meltano\loaders\target-sqlbuzz\venv\lib\site-packages\fs\base.py", line 1581, in validatepath cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.385976Z [info     ]     raise errors.InvalidCharsInPath(path) cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.401601Z [info     ] fs.errors.InvalidCharsInPath: path 'C:\development\tap-stackoverflow-sampledata\output\test-batch-tap-stackoverflow-sampledata--tags-59fd48e2-0040-46a0-8505-6587c5f53abe-1.json.gz' contains invalid characters cmd_type=elb consumer=True name=target-sqlbuzz producer=False stdio=stderr string_id=target-sqlbuzz
2022-10-03T18:48:16.823470Z [error    ] Loader failed