Normalization: Snowflake doesn't identify 2021-04-04T22:54:29+0000 as valid timestamp #5439

subodh1810 · 2021-08-16T20:34:31Z

Slack : https://airbytehq.slack.com/archives/C01MFR03D5W/p1629094676136200

For some reason snowflake doesn't identify 2021-04-04T22:54:29+0000 as valid timestamp and because of it the normalisation fails. It does seem like a valid ISO-8601 format but for reasons we dont understand Snowflake doesnt accept it.
The funny thing is that if we add colon to the value i.e. change it to 2021-04-04T22:54:29+00:00 snowflake accepts it.
The exact same behaviour is observed in JAVA as well i.e. Instant.parse("2021-04-04T22:54:29+0000"); fails but Instant.parse("2021-04-04T22:54:29+00:00"); passes.
In order to fix this we could add an extra step in the Snowflake consumer and transform the such values before writing it to the warehouse, it would work.

We should also add a test case in acceptance tests to see which other destinations cant handle it

io.airbyte.config.StandardSyncOutput@1302a129[standardSyncSummary=io.airbyte.config.StandardSyncSummary@185a9763[status=completed,recordsSynced=1000,bytesSynced=255766,startTime=1629092070552,endTime=1629092085423],state=io.airbyte.config.State@302705e7[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@4cd6abcc[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@39072ab8[stream=io.airbyte.protocol.models.AirbyteStream@43290f4d[name=cc_info,jsonSchema={"type":"object","properties":{"CCV":{"type":["null","integer"]},"Name":{"type":["null","string"]},"Expiry_Date":{"type":["null","string"]},"Credit_Card_Type":{"type":["null","string"]},"Credit_Card_Number":{"type":["null","integer"]},"_ab_source_file_url":{"type":"string"},"_ab_additional_properties":{"type":"object"},"_ab_source_file_last_modified":{"type":"string","format":"date-time"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[_ab_source_file_last_modified],sourceDefinedPrimaryKey=[],namespace=<null>,additionalProperties={}],syncMode=full_refresh,cursorField=[_ab_source_file_last_modified],destinationSyncMode=overwrite,primaryKey=[],additionalProperties={}]],additionalProperties={}]]
2021-08-16 05:34:45 INFO () TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.29.7-alpha
2021-08-16 05:34:45 INFO () DefaultNormalizationWorker(run):61 - Running normalization.
2021-08-16 05:34:45 INFO () LineGobbler(voidCall):85 - Checking if airbyte/normalization:0.1.39 exists...
2021-08-16 05:34:45 INFO () LineGobbler(voidCall):85 - airbyte/normalization:0.1.39 was found locally.
2021-08-16 05:34:45 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/8/0/normalize --network host --log-driver none airbyte/normalization:0.1.39 run --integration-type snowflake --config destination_config.json --catalog destination_catalog.json
2021-08-16 05:34:45 INFO () LineGobbler(voidCall):85 - Running: transform-config --config destination_config.json --integration-type snowflake --out /data/8/0/normalize
2021-08-16 05:34:45 INFO () LineGobbler(voidCall):85 - Namespace(config='destination_config.json', integration_type=<DestinationType.snowflake: 'snowflake'>, out='/data/8/0/normalize')
2021-08-16 05:34:45 INFO () LineGobbler(voidCall):85 - transform_snowflake
2021-08-16 05:34:45 INFO () LineGobbler(voidCall):85 - Running: transform-catalog --integration-type snowflake --profile-config-dir /data/8/0/normalize --catalog destination_catalog.json --out /data/8/0/normalize/models/generated/ --json-column _airbyte_data
2021-08-16 05:34:46 INFO () LineGobbler(voidCall):85 - Processing destination_catalog.json...
2021-08-16 05:34:46 INFO () LineGobbler(voidCall):85 -   Generating airbyte_ctes/AIRBYTE_SCHEMA/CC_INFO_AB1.sql from cc_info
2021-08-16 05:34:46 INFO () LineGobbler(voidCall):85 -   Generating airbyte_ctes/AIRBYTE_SCHEMA/CC_INFO_AB2.sql from cc_info
2021-08-16 05:34:46 INFO () LineGobbler(voidCall):85 -   Generating airbyte_ctes/AIRBYTE_SCHEMA/CC_INFO_AB3.sql from cc_info
2021-08-16 05:34:46 INFO () LineGobbler(voidCall):85 -   Generating airbyte_tables/AIRBYTE_SCHEMA/CC_INFO.sql from cc_info
2021-08-16 05:34:46 INFO () LineGobbler(voidCall):85 - Running with dbt=0.19.0
2021-08-16 05:34:47 INFO () LineGobbler(voidCall):85 - [[33mWARNING[0m]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
2021-08-16 05:34:47 INFO () LineGobbler(voidCall):85 - There are 1 unused configuration paths:
2021-08-16 05:34:47 INFO () LineGobbler(voidCall):85 - - models.airbyte_utils.generated.airbyte_views
2021-08-16 05:34:47 INFO () LineGobbler(voidCall):85 - 
2021-08-16 05:34:47 INFO () LineGobbler(voidCall):85 - Found 4 models, 0 tests, 0 snapshots, 0 analyses, 392 macros, 0 operations, 0 seed files, 1 source, 0 exposures
2021-08-16 05:34:47 INFO () LineGobbler(voidCall):85 - 
2021-08-16 05:34:52 INFO () LineGobbler(voidCall):85 - 05:34:52 | Concurrency: 32 threads (target='prod')
2021-08-16 05:34:52 INFO () LineGobbler(voidCall):85 - 05:34:52 | 
2021-08-16 05:34:52 INFO () LineGobbler(voidCall):85 - 05:34:52 | 1 of 1 START table model AIRBYTE_SCHEMA.CC_INFO.............................................................. [RUN]
2021-08-16 05:34:55 INFO () LineGobbler(voidCall):85 - 05:34:55 | 1 of 1 ERROR creating table model AIRBYTE_SCHEMA.CC_INFO..................................................... [[31mERROR[0m in 2.53s]
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 - 05:34:56 | 
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 - 05:34:56 | Finished running 1 table model in 8.84s.
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 - 
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 - [31mCompleted with 1 error and 0 warnings:[0m
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 - 
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 - [33mDatabase Error in model CC_INFO (models/generated/airbyte_tables/AIRBYTE_SCHEMA/CC_INFO.sql)[0m
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 -   100035 (22007): Timestamp '2021-04-04T22:54:29+0000' is not recognized
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 -   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/AIRBYTE_SCHEMA/CC_INFO.sql
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 - 
2021-08-16 05:34:56 INFO () LineGobbler(voidCall):85 - Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
2021-08-16 05:34:57 INFO () TemporalAttemptExecution(lambda$getWorkerThread$2):149 - Completing future exceptionally...
io.airbyte.workers.WorkerException: Normalization Failed.

Acceptance Criteria

Implementation guide

See the individual tickets for guidance.

The text was updated successfully, but these errors were encountered:

sherifnada · 2021-08-27T05:15:01Z

After some investigation, it seems like Snowflake just doesn't accept this format. See this list: https://docs.snowflake.com/en/user-guide/date-time-input-output.html#timestamp-formats -- this format is not listed there. I also tried entering it by hand in the Snowflake UI -- got a failure.

SELECT to_timestamp_tz('2021-04-04T22:54:29+0000') AS blah; // this line fails
SELECT to_timestamp_tz('2021-04-04T22:54:29+00:00') AS blah; // this line succeeds

bazarnov · 2021-09-03T08:05:40Z

ISO8601 possible date & datetime formats

General Information

Based on the official documentation for JSONSchema about Date & Time formats:
Dates and times are represented in RFC 3339, section 5.6. This is a subset of the date format also commonly known as ISO8601 format. Which could be represented in the table down bellow:

VALID DATE-TIME formats

Table with `VALID Date-Time formats` acceptable for JsonSchema

Format	Data Type	JsonSchema Type	JsonSchema Format	Example
`YYYY-M-D`	`string`	`string`	`date`	`"2021-1-1"`
`YYYY-M-D hh:mm:ss`	`string`	`string`	`date-time`	`"2021-1-1 01:01:01"`
`YYYY-M-D hh:mm:ss Z`	`string`	`string`	`date-time`	`"2021-1-1 01:01:01 +1"`
`YYYY-M-DThh:mm:ss Z`	`string`	`string`	`date-time`	`"2021-1-1T01:01:01 +1:00"`
=======	=======	=======	=======	=======
`YYYY-MM-DD`	`string`	`string`	`date`	`"2021-01-01"`
`YYYY-MM-DD hh:mm:ss`	`string`	`string`	`date-time`	`"2021-01-01 01:01:01"`
`YYYY-MM-DD hh:mm:ss Z`	`string`	`string`	`date-time`	`"2021-01-01 01:01:01 +0000"`
`YYYY-MM-DDThh:mm:ssZ`	`string`	`string`	`date-time`	`"2021-01-01T01:01:01Z"` `"2021-01-01T01:01:01-01:00"`
`YYYY-MM-DDThh:mm:ss Z`	`string`	`string`	`date-time`	`"2021-01-01T01:01:01 +1:00"`

INVALID DATE-TIME formats

Table with `INVALID Date-Time formats` for JsonSchema

This examples will lead to validation error in json schema, as well as Timestamp 'your_timestamp' is not recognized inside Snowflake as a destination.

Format	Data Type	JsonSchema Type	JsonSchema Format	Example
`YYYY-MM-DD hh:mm:ss Z`	`string`	`string`	`date-time`	`"2021-01-01 01:01:01 UTC"`
`YYYY-MM-DDThh:mm:ss Z`	`string`	`string`	`date-time`	`"2021-01-01T01:01:01 UTC"` `"2021-01-01T01:01:01 +1"` `"2021-01-01T01:01:01 +0000"`
`YYYY-MM-DDThh:mm:ssZ`	`string`	`string`	`date-time`	`"2021-01-01T01:01:01+0000"` `"2021-01-01T01:01:01UTC"` `"2021-01-01T01:01:01+1"`

TESTS ON Snowflake Destination

The tests are made with the single conversion statement:
SELECT to_timestamp_tz('input_string_with_possible_datetime') and the output is the result of the test, indicated in the tables bellow.

Table with tests on `Snowflake` as a destination

VALID DATE-TIME formats

Input String	Format	Snowflake Output
`"2021-1-1"`	`YYYY-M-D`	2021-01-01 00:00:00.000 -0800
`"2021-1-1 01:01:01"`	`YYYY-M-D hh:mm:ss`	2021-01-01 01:01:01.000 -0800
`"2021-1-1 01:01:01 +1"`	`YYYY-M-D hh:mm:ss Z`	2021-01-01 01:01:01.000 +0100
`"2021-1-1T01:01:01 +1:00"`	`YYYY-M-DThh:mm:ss Z`	2021-01-01 01:01:01.000 +0100
`"2021-01-01"`	`YYYY-MM-DD`	2021-01-01 00:00:00.000 -0800
`"2021-01-01 01:01:01"`	`YYYY-MM-DD hh:mm:ss`	2021-01-01 01:01:01.000 -0800
`"2021-01-01 01:01:01 +0000"`	`YYYY-MM-DD hh:mm:ss Z`	2021-01-01 01:01:01.000 +0000
`"2021-01-01T01:01:01Z"`	`YYYY-MM-DDThh:mm:ssZ`	2021-01-01 01:01:01.000 +0000
`"2021-01-01T01:01:01-01:00"`	`YYYY-MM-DDThh:mm:ssZ`	2021-01-01 01:01:01.000 -0100
`"2021-01-01T01:01:01 +1:00"`	`YYYY-MM-DDThh:mm:ss Z`	2021-01-01 01:01:01.000 +0100

INVALID DATE-TIME formats

Input String	Format	Snowflake Output
`"2021-01-01 01:01:01 UTC"`	`YYYY-MM-DD hh:mm:ss Z`	Timestamp '2021-01-01 01:01:01 UTC' is not recognized
`"2021-01-01T01:01:01 UTC"`	`YYYY-MM-DDThh:mm:ss Z`	Timestamp '2021-01-01T01:01:01 UTC' is not recognized
`"2021-01-01T01:01:01 +1"`	`YYYY-MM-DDThh:mm:ss Z`	Timestamp '2021-01-01T01:01:01 +1' is not recognized
`"2021-01-01T01:01:01 +0000"`	`YYYY-MM-DDThh:mm:ss Z`	Timestamp '2021-01-01T01:01:01 +0000' is not recognized
`"2021-01-01T01:01:01+0000"`	`YYYY-MM-DDThh:mm:ssZ`	Timestamp '2021-01-01T01:01:01+0000' is not recognized
`"2021-01-01T01:01:01UTC"`	`YYYY-MM-DDThh:mm:ssZ`	Timestamp '2021-01-01T01:01:01UTC' is not recognized
`"2021-01-01T01:01:01+1"`	`YYYY-MM-DDThh:mm:ssZ`	Timestamp '2021-01-01T01:01:01+1' is not recognized

pdulapalli · 2021-11-19T21:19:49Z

I believe I've also encountered the same issue, when using a Marketo Source --> Snowflake Destination. I've added context below.

Environment

Airbyte version: 0.32.4-alpha
OS Version / Instance: Ubuntu 20.04 on AWS EC2
Deployment: Airbyte-provided docker image
Source Connector and version: Marketo 0.1.0
Destination Connector and version: Snowflake 0.3.17
Step where error happened: Sync job --> Normalization stage

Current Behavior

Sync job fails during Normalization stage and does not go further.

Logs

2021-11-08 08:50:16 INFO () DefaultNormalizationWorker(run):46 - Running normalization.
2021-11-08 08:50:16 INFO () DefaultNormalizationRunner(runProcess):114 - Running with normalization version: airbyte/normalization:0.1.58
2021-11-08 08:50:16 INFO () LineGobbler(voidCall):82 - Checking if airbyte/normalization:0.1.58 exists...
2021-11-08 08:50:16 INFO () DockerProcessFactory(create):127 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/18/2/normalize --network host --log-driver none airbyte/normalization:0.1.58 run --integration-type bigquery --config destination_config.json --catalog destination_catalog.json
2021-11-08 08:50:16 INFO () LineGobbler(voidCall):82 - airbyte/normalization:0.1.58 was found locally.
�[32mnormalization�[0m - 2021-11-08 08:50:16 INFO () LineGobbler(voidCall):82 - Running: transform-config --config destination_config.json --integration-type bigquery --out /data/18/2/normalize
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 - Namespace(config='destination_config.json', integration_type=<DestinationType.bigquery: 'bigquery'>, out='/data/18/2/normalize')
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 - transform_bigquery
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 - Running: transform-catalog --integration-type bigquery --profile-config-dir /data/18/2/normalize --catalog destination_catalog.json --out /data/18/2/normalize/models/generated/ --json-column _airbyte_data
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 - Processing destination_catalog.json...
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_ctes/raw_programs_marketo/programs_ab1.sql from programs
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_ctes/raw_programs_marketo/programs_ab2.sql from programs
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_views/raw_programs_marketo/programs_ab3.sql from programs
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_incremental/scd/raw_programs_marketo/programs_scd.sql from programs
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_incremental/raw_programs_marketo/programs.sql from programs
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_ctes/raw_programs_marketo/programs_folder_ab1.sql from programs/folder
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_ctes/raw_programs_marketo/programs_folder_ab2.sql from programs/folder
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_ctes/raw_programs_marketo/programs_folder_ab3.sql from programs/folder
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 -   Generating airbyte_tables/raw_programs_marketo/programs_folder.sql from programs/folder
�[32mnormalization�[0m - 2021-11-08 08:50:17 INFO () LineGobbler(voidCall):82 - detected no config file for ssh, assuming ssh is off.
�[32mnormalization�[0m - 2021-11-08 08:50:20 INFO () LineGobbler(voidCall):82 - Running with dbt=0.21.0
�[32mnormalization�[0m - 2021-11-08 08:50:20 INFO () LineGobbler(voidCall):82 - Unable to do partial parsing because ../build/partial_parse.msgpack not found
�[32mnormalization�[0m - 2021-11-08 08:50:23 INFO () LineGobbler(voidCall):82 - Found 9 models, 0 tests, 0 snapshots, 0 analyses, 500 macros, 0 operations, 0 seed files, 1 source, 0 exposures
�[32mnormalization�[0m - 2021-11-08 08:50:23 INFO () LineGobbler(voidCall):82 - 
�[32mnormalization�[0m - 2021-11-08 08:50:23 INFO () LineGobbler(voidCall):82 - 08:50:23 | Concurrency: 32 threads (target='prod')
�[32mnormalization�[0m - 2021-11-08 08:50:23 INFO () LineGobbler(voidCall):82 - 08:50:23 | 
�[32mnormalization�[0m - 2021-11-08 08:50:23 INFO () LineGobbler(voidCall):82 - 08:50:23 | 1 of 4 START view model _airbyte_raw_programs_marketo.programs_ab3........................................... [RUN]
�[32mnormalization�[0m - 2021-11-08 08:50:24 INFO () LineGobbler(voidCall):82 - 08:50:24 | 1 of 4 OK created view model _airbyte_raw_programs_marketo.programs_ab3...................................... [�[32mOK�[0m in 0.79s]
�[32mnormalization�[0m - 2021-11-08 08:50:24 INFO () LineGobbler(voidCall):82 - 08:50:24 | 2 of 4 START incremental model raw_programs_marketo.programs_scd............................................. [RUN]
�[32mnormalization�[0m - 2021-11-08 08:50:24 INFO () LineGobbler(voidCall):82 - 08:50:24 + `bigquery-313713`.raw_programs_marketo.`programs_scd`._airbyte_ab_id does not exist. The table needs to be rebuilt in full_refresh
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - 08:50:27 | 2 of 4 ERROR creating incremental model raw_programs_marketo.programs_scd.................................... [�[31mERROR�[0m in 2.55s]
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - 08:50:27 | 3 of 4 SKIP relation raw_programs_marketo.programs........................................................... [�[33mSKIP�[0m]
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - 08:50:27 | 4 of 4 SKIP relation raw_programs_marketo.programs_folder.................................................... [�[33mSKIP�[0m]
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - 08:50:27 | 
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - 08:50:27 | Finished running 1 view model, 2 incremental models, 1 table model in 4.05s.
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - 
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - �[31mCompleted with 1 error and 0 warnings:�[0m
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - 
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - �[33mDatabase Error in model programs_scd (models/generated/airbyte_incremental/scd/raw_programs_marketo/programs_scd.sql)�[0m
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 -   Invalid timestamp: '2018-09-07T17:37:18Z+0000'
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 -   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_incremental/scd/raw_programs_marketo/programs_scd.sql
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - 
�[32mnormalization�[0m - 2021-11-08 08:50:27 INFO () LineGobbler(voidCall):82 - Done. PASS=1 WARN=0 ERROR=1 SKIP=2 TOTAL=4
2021-11-08 08:50:27 INFO () TemporalAttemptExecution(lambda$getWorkerThread$2):170 - Completing future exceptionally...
io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.30.33-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.30.33-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:167) ~[io.airbyte-airbyte-workers-0.30.33-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]

lazebnyi · 2021-11-29T11:41:20Z

@pdulapalli Thanks for the info about the Marketo bug, created an #8296 related to your issue.

lazebnyi · 2021-12-03T12:02:31Z

Closed with #5712

leoebfolsom · 2022-07-20T13:00:20Z

@sherifnada @subodh1810 I'm guessing you found a solution already, but this is how I'm handling it, in case this helps anyone:

select
   ...
    concat(
      left(col, len(col)-2),
      ':',
      right(col,2)
    )::timestamp as date_col,
   ...
from your_table;

This Stack Overflow answer was helpful to me.

subodh1810 added the type/bug Something isn't working label Aug 16, 2021

marcosmarxm added the normalization label Aug 16, 2021

marcosmarxm changed the title ~~Snowflake doesn't identify 2021-04-04T22:54:29+0000 as valid timestamp~~ Normalization: Snowflake doesn't identify 2021-04-04T22:54:29+0000 as valid timestamp Aug 16, 2021

sherifnada added the area/connectors Connector related issues label Aug 16, 2021

sherifnada added lang/java normalization priority/medium Medium priority priority/high High priority and removed normalization priority/medium Medium priority lang/java labels Aug 27, 2021

sherifnada mentioned this issue Sep 2, 2021

🐛 📝 DAT: verify that a destination is able to write any ISO8601-compliant date string #5814

Merged

14 tasks

lazebnyi self-assigned this Nov 26, 2021

lazebnyi removed their assignment Nov 29, 2021

lazebnyi closed this as completed Dec 3, 2021

karinakuz added connectors/destinations-warehouse connectors/destination/snowflake labels Jan 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Normalization: Snowflake doesn't identify 2021-04-04T22:54:29+0000 as valid timestamp #5439

Normalization: Snowflake doesn't identify 2021-04-04T22:54:29+0000 as valid timestamp #5439

subodh1810 commented Aug 16, 2021 •

edited by sherifnada

Loading

sherifnada commented Aug 27, 2021

bazarnov commented Sep 3, 2021

pdulapalli commented Nov 19, 2021

lazebnyi commented Nov 29, 2021 •

edited

Loading

lazebnyi commented Dec 3, 2021

leoebfolsom commented Jul 20, 2022

Normalization: Snowflake doesn't identify 2021-04-04T22:54:29+0000 as valid timestamp #5439

Normalization: Snowflake doesn't identify 2021-04-04T22:54:29+0000 as valid timestamp #5439

Comments

subodh1810 commented Aug 16, 2021 • edited by sherifnada Loading

Acceptance Criteria

Implementation guide

sherifnada commented Aug 27, 2021

bazarnov commented Sep 3, 2021

ISO8601 possible date & datetime formats

General Information

VALID DATE-TIME formats

Table with VALID Date-Time formats acceptable for JsonSchema

INVALID DATE-TIME formats

Table with INVALID Date-Time formats for JsonSchema

TESTS ON Snowflake Destination

Table with tests on Snowflake as a destination

VALID DATE-TIME formats

INVALID DATE-TIME formats

pdulapalli commented Nov 19, 2021

Environment

Current Behavior

Logs

lazebnyi commented Nov 29, 2021 • edited Loading

lazebnyi commented Dec 3, 2021

leoebfolsom commented Jul 20, 2022

subodh1810 commented Aug 16, 2021 •

edited by sherifnada

Loading

Table with `VALID Date-Time formats` acceptable for JsonSchema

Table with `INVALID Date-Time formats` for JsonSchema

Table with tests on `Snowflake` as a destination

lazebnyi commented Nov 29, 2021 •

edited

Loading