SHOPIFY abandoned_checkouts_customer has a variable type error

[fortisoftware]

I'm trying to take tables from a SHOPIFY api to SNOWFLAKE and it fails to use basic normalization with the endpoint abandoned checkouts in the field customer.
The only problem is that the tables from abandoned checkouts customer are empty or don't exist, everything else is ok.
i think that the problem is in abandoned_checkouts.customer.tax_excemptions.
Here are the logs of the error:

2022-02-04 13:34:00 �[42mnormalization�[0m > �[31mCompleted with 1 error and 0 warnings:�[0m
2022-02-04 13:34:00 �[42mnormalization�[0m >
2022-02-04 13:34:00 �[42mnormalization�[0m > �[33mDatabase Error in model ABANDONED_CHECKOUTS_CUSTOMER (models/generated/airbyte_incremental/SHOPIFY/ABANDONED_CHECKOUTS_CUSTOMER.sql)�[0m
2022-02-04 13:34:00 �[42mnormalization�[0m > 100037 (22018): Boolean value '[]' is not recognized
2022-02-04 13:34:00 �[42mnormalization�[0m >
2022-02-04 13:34:00 �[42mnormalization�[0m > Done. PASS=15 WARN=0 ERROR=1 SKIP=2 TOTAL=18
2022-02-04 13:34:00 �[32mINFO�[m i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):177 - Completing future exceptionally...
io.airbyte.workers.WorkerException: Normalization Failed.
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
... 3 more
Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:159) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:45) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
2022-02-04 13:34:00 �[32mINFO�[m i.a.w.t.TemporalAttemptExecution(get):144 - Stopping cancellation check scheduling...
2022-02-04 13:34:00 �[33mWARN�[m i.t.i.s.POJOActivityTaskHandler(activityFailureToResult):363 - Activity failure. ActivityId=173b5e3e-3a57-31f7-9920-462b0862ac2c, activityType=Normalize, attempt=1
java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed.
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:142) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.normalize(NormalizationActivityImpl.java:106) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:286) ~[temporal-sdk-1.6.0.jar:?]
at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:252) ~[temporal-sdk-1.6.0.jar:?]
at io.temporal.internal.sync.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:209) ~[temporal-sdk-1.6.0.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:193) ~[temporal-sdk-1.6.0.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:151) ~[temporal-sdk-1.6.0.jar:?]
at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:73) ~[temporal-sdk-1.6.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
... 1 more
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
... 1 more
Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:159) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:45) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.2-alpha.jar:?]
at java.lang.Thread.run(Thread.java:833) [?:?]

[bazarnov]

This is potentially missing values inside of abandoned_checkouts.customer.default_address.default which was expecting boolean value by default. We will force that to be a string instead.

[fortisoftware]

Hi, I made a custom dbt and change ABANDONED_CHECKOUTS_CUSTOMER_AB2:

cast(TAX_EXEMPTIONS as {{ dbt_utils.type_string() }}) as TAX_EXEMPTIONS

the problem was solved. TAX_EXEMPTIONS was specting a boolean and it is an array.

[bazarnov]

Fixed in #10419