Diference between handling activate_version messages between hard_delete true and false

[vitoravancini]

Hello everyone,

It seems that with hard_delete = true when we receive an activate_version message we delete everything equal or lesser than the new version:

sdk/singer_sdk/sinks/sql.py

Line 407 in 7c77366

f"WHERE {self.version_column_name} < :version \n"

and when we set hard_delete = false we soft delete everything less than the new version:

sdk/singer_sdk/connectors/sql.py

Line 1214 in 7c77366

f"WHERE {version_column_name} <= {current_version}",

Is this how it supposed to be?
When using target_snowflake(based on singer_sdk) with tap mssql we drop everything but the last batch at the end of the execution when hard_delete = true.
The tap emits another activate_message at the end with the same version, which triggers a sql like

delete from {table} where _sdc_record_version <= {version}

that {version} beeing the same version of all the records inserted up to that point.

Should we make both logics(hard_delete true and false) the same?

thank you!

[edgarrmondragon]

Should we make both logics(hard_delete true and false) the same?

@vitoravancini I think we should :)

PRs welcome!

[vitoravancini]

Nice, will try to send it later today