Source Google Ads: Normalization produces improper data since introduction of "segments.hour" in 0.2.3 #20283

tigitz · 2022-12-09T07:44:38Z

Environment

Airbyte version: 0.40.14
OS Version / Instance: airbyte docker-compose
Deployment: docker-compose in local
Source Connector and version: Google Ads 0.2.5
Destination Connector and version: Postgres 0.3.26
Step where error happened: Dbt Normalization

Current Behavior

While normalizing google ads extracted data with the Incremental | Deduped + history strategy, since the introduction of the segments.hour in 0.2.3 I believe, the row normalized in the final table campaign is now a single hour data of the day.

Meaning

SELECT _airbyte_data ->> 'segments.date' as segment_date
     , (_airbyte_data ->> 'segments.hour')::float as segment_hour
     , (_airbyte_data ->> 'metrics.impressions')::float as impressions
FROM public._airbyte_raw_campaigns
WHERE _airbyte_data ->> 'campaign.id' = '9605993431' AND _airbyte_data ->> 'segments.date' = '2022-11-30'

Gives

But

SELECT "segments.date",
       "segments.hour",
       "metrics.impressions"
FROM public.campaigns
WHERE "campaign.id"= '9605993431' AND "segments.date" = '2022-11-30';

Gives

Expected Behavior

I would expect it to keep the previous behavior by aggregating data by the date, meaning having this result basically in campaigns

SELECT _airbyte_data ->> 'segments.date' as segment_date
     , SUM((_airbyte_data ->> 'metrics.impressions')::float) as impressions
FROM public._airbyte_raw_campaigns
WHERE _airbyte_data ->> 'campaign.id' = '9605993431' AND _airbyte_data ->> 'segments.date' = '2022-11-30'
GROUP BY _airbyte_data ->> 'segments.date'

But I understand since the introduction of segments.hour it adds more granularity to the data so aggregating here would mean losing this new granularity, defeating the purpose of introducing it.

In this case I would expect the final campaigns table to properly show all the the rows for each hours.

Hints

I believe problem is in the generated campaign_scd.sql where it defines uniqueness by partitioning by segments.date only, missing the new segments.hour granularity introduced.

 -- line 101
    case when row_number() over (
        partition by "campaign.id", "segments.date"
        order by
            "segments.date" is null asc,
            "segments.date" desc,
            _airbyte_emitted_at desc
      ) = 1 then 1 else 0 end as _airbyte_active_row,

The text was updated successfully, but these errors were encountered:

baudouin-notilus · 2023-03-09T10:11:20Z

Having the exact same issue here!

A quick fix I did is working with campaigns_scd table instead of campaigns table, but it is a bit of a hack.

tigitz · 2023-03-09T10:52:39Z

Yeah that's unfortunate, issue seems quite problematic since it results in incorrect data and forces to downgrade AFAIU.

I'm wondering how users using the GADS source haven't hit this issue or realised they're affected. Seems quite a priority for airbyte maintainers to fix it. Tried to help as much as I can with clear explanations and solutions hints but despite that there doesn't seem to be any traction 🤷‍♂️

tigitz · 2023-03-13T20:32:43Z

Downgrading doesn't work anymore as the google ads python lib used by the connector seems now unsupported by the google ads api.

Had to apply this patch on campaigns_scd for anyone wondering

===================================================================
diff --git a/models/_sync/airbyte_incremental/scd/_sync_google_ads/campaigns_google_ads_scd.sql b/models/_sync/airbyte_incremental/scd/_sync_google_ads/campaigns_google_ads_scd.sql
--- a/models/_sync/airbyte_incremental/scd/_sync_google_ads/campaigns_google_ads_scd.sql	(revision 474c9895c148a79fc17adb8623e016ea20a541e1)
+++ b/models/_sync/airbyte_incremental/scd/_sync_google_ads/campaigns_google_ads_scd.sql	(date 1678738942665)
@@ -105,6 +105,7 @@
       {{ dbt_utils.surrogate_key([
       adapter.quote('campaign.id'),
       adapter.quote('segments.date'),
+      adapter.quote('segments.hour'),
       ]) }} as _airbyte_unique_key,
       {{ adapter.quote('campaign.id') }},
       {{ adapter.quote('metrics.ctr') }},
@@ -197,10 +198,12 @@
             _airbyte_emitted_at desc
       ) as _airbyte_end_at,
       case when row_number() over (
-        partition by {{ adapter.quote('campaign.id') }}, {{ adapter.quote('segments.date') }}
+        partition by {{ adapter.quote('campaign.id') }}, {{ adapter.quote('segments.date') }}, {{ adapter.quote('segments.hour') }}
         order by
             {{ adapter.quote('segments.date') }} is null asc,
             {{ adapter.quote('segments.date') }} desc,
+            {{ adapter.quote('segments.hour') }} is null asc,
+            {{ adapter.quote('segments.hour') }} desc,
             _airbyte_emitted_at desc
       ) = 1 then 1 else 0 end as _airbyte_active_row,
       _airbyte_ab_id,

⚠️ Beware it changes the granularity of the campaigns table, it's now 1 row = 1 hour of data, while previously it was 1 row = 1 day. You'll need to do introduce an intermediary model yourself to go back on a 1 row = 1 day model as the source of all your dependant models.

baudouin-notilus · 2023-03-14T11:06:46Z

Hi! Thanks a lot for working on this, I will do the updates soon! Have a great day, Baudouin

…

On Tue, Mar 14, 2023 at 12:00 PM Hans Lemm ***@***.***> wrote: Hi! Yesterday, this issue was fixed with PR #23999 <#23999>. Update your connector *and refresh the source schema* to update the incremental sync. — Reply to this email directly, view it on GitHub <#20283 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AXKYOM4J3QAZNZLTTNBZISTW4BFWJANCNFSM6AAAAAASZBFLQI> . You are receiving this because you commented.Message ID: ***@***.***>

hanslemm · 2023-03-14T11:09:32Z

This issue was fixed in yesterday's PR for Source Google Ads #23999.

Update the connector & refresh source schema to make use of the fixed incremental sync.

tigitz · 2023-03-15T11:08:43Z

@hanslemm @evantahler tried it but didn't fix anything.

campaigns model

campaigns_scd model

Attached Logs

2023-03-15 10:59:05 �[32mINFO�[m i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/28/0/logs.log
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.40.32
2023-03-15 10:59:05 �[32mINFO�[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to save workflow id for cancellation
2023-03-15 10:59:05 �[32mINFO�[m i.a.c.EnvConfigs(getEnvOrDefault):1177 - Using default value for environment variable METRIC_CLIENT: ''
2023-03-15 10:59:05 �[33mWARN�[m i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(run):152 - start sync worker. job id: 28 attempt id: 0
2023-03-15 10:59:05 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - 
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(run):167 - configured sync modes: {null.click_view=incremental - append_dedup, null.campaigns=incremental - append_dedup}
2023-03-15 10:59:05 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - ----- START REPLICATION -----
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.i.DefaultAirbyteDestination(start):84 - Running destination...
2023-03-15 10:59:05 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - 
2023-03-15 10:59:05 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-postgres:0.3.26 exists...
2023-03-15 10:59:05 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-postgres:0.3.26 was found locally.
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.p.DockerProcessFactory(create):125 - Creating docker container = destination-postgres-write-28-0-pfeud with resources io.airbyte.config.ResourceRequirements@58aaf41e[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] and allowedHosts null
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.p.DockerProcessFactory(create):170 - Preparing command: docker run --rm --init -i -w /data/28/0 --log-driver none --name destination-postgres-write-28-0-pfeud --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e FIELD_SELECTION_WORKSPACES= -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_CONNECTOR_IMAGE=airbyte/destination-postgres:0.3.26 -e WORKER_JOB_ATTEMPT=0 -e AUTO_DETECT_SCHEMA=true -e AIRBYTE_VERSION=0.40.32 -e WORKER_JOB_ID=28 airbyte/destination-postgres:0.3.26 write --config destination_config.json --catalog destination_catalog.json
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.i.VersionedAirbyteMessageBufferedWriterFactory(createWriter):38 - Writing messages to protocol version 0.2.0, messages will be downgraded from protocol version 1.0.0
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.i.VersionedAirbyteStreamFactory(create):107 - Reading messages from protocol version 0.2.0, messages will be upgraded to protocol version 1.0.0
2023-03-15 10:59:05 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/source-google-ads:0.2.11 exists...
2023-03-15 10:59:05 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - airbyte/source-google-ads:0.2.11 was found locally.
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.p.DockerProcessFactory(create):125 - Creating docker container = source-google-ads-read-28-0-wnkgj with resources io.airbyte.config.ResourceRequirements@23b40ea1[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] and allowedHosts null
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.p.DockerProcessFactory(create):170 - Preparing command: docker run --rm --init -i -w /data/28/0 --log-driver none --name source-google-ads-read-28-0-wnkgj --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e FIELD_SELECTION_WORKSPACES= -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_CONNECTOR_IMAGE=airbyte/source-google-ads:0.2.11 -e WORKER_JOB_ATTEMPT=0 -e AUTO_DETECT_SCHEMA=true -e AIRBYTE_VERSION=0.40.32 -e WORKER_JOB_ID=28 airbyte/source-google-ads:0.2.11 read --config source_config.json --catalog source_catalog.json
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.i.VersionedAirbyteStreamFactory(create):107 - Reading messages from protocol version 0.2.0, messages will be upgraded to protocol version 1.0.0
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromDstRunnable$4):275 - Destination output thread started.
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(replicate):247 - Waiting for source and destination threads to complete.
2023-03-15 10:59:05 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):337 - Replication thread started.
2023-03-15 10:59:06 �[43mdestination�[0m > starting destination: class io.airbyte.integrations.destination.postgres.PostgresDestination
2023-03-15 10:59:07 �[43mdestination�[0m > integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2023-03-15 10:59:07 �[43mdestination�[0m > Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2023-03-15 10:59:07 �[43mdestination�[0m > Command: WRITE
2023-03-15 10:59:07 �[43mdestination�[0m > Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2023-03-15 10:59:07 �[43mdestination�[0m > Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-03-15 10:59:07 �[43mdestination�[0m > Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2023-03-15 10:59:07 �[44msource�[0m > Starting syncing SourceGoogleAds
2023-03-15 10:59:07 �[43mdestination�[0m > Starting connection with method: NO_TUNNEL
2023-03-15 10:59:07 �[43mdestination�[0m > HikariPool-1 - Starting...
2023-03-15 10:59:07 �[43mdestination�[0m > HikariPool-1 - Start completed.
2023-03-15 10:59:07 �[43mdestination�[0m > Write config: WriteConfig{streamName=click_view, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_yvx_click_view, outputTableName=_airbyte_raw_click_view, syncMode=append_dedup}
2023-03-15 10:59:07 �[43mdestination�[0m > Write config: WriteConfig{streamName=campaigns, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_rfl_campaigns, outputTableName=_airbyte_raw_campaigns, syncMode=append_dedup}
2023-03-15 10:59:07 �[43mdestination�[0m > class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started.
2023-03-15 10:59:07 �[43mdestination�[0m > Preparing tmp tables in destination started for 2 streams
2023-03-15 10:59:07 �[43mdestination�[0m > Preparing tmp table in destination started for stream click_view. schema: public, tmp table name: _airbyte_tmp_yvx_click_view
2023-03-15 10:59:07 �[43mdestination�[0m > Preparing tmp table in destination started for stream campaigns. schema: public, tmp table name: _airbyte_tmp_rfl_campaigns
2023-03-15 10:59:07 �[43mdestination�[0m > Preparing tables in destination completed.
2023-03-15 10:59:08 �[44msource�[0m > Syncing stream: click_view 
2023-03-15 10:59:08 �[44msource�[0m > Generating slices for customer 6376226450. Start date is 2023-03-12, end date is None
2023-03-15 10:59:08 �[44msource�[0m > Next slice is {'start_date': '2023-02-27', 'end_date': '2023-02-28', 'customer_id': '6376226450'}
2023-03-15 10:59:11 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 1000 (829 KB)
2023-03-15 10:59:14 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 2000 (1 MB)
2023-03-15 10:59:17 �[44msource�[0m > Starting a while loop iteration
2023-03-15 10:59:17 �[44msource�[0m > Read records using g-ads client. Stream slice is {'start_date': '2023-03-08', 'end_date': '2023-03-09', 'customer_id': '6376226450'}
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Updated state for customer 6376226450. Full state is {'6376226450': {'segments.date': '2023-03-08'}}.
2023-03-15 10:59:18 �[44msource�[0m > Current slice has been read. Exiting read_records()
2023-03-15 10:59:18 �[44msource�[0m > Next slice is {'start_date': '2023-03-09', 'end_date': '2023-03-10', 'customer_id': '6376226450'}
2023-03-15 10:59:20 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 3000 (2 MB)
2023-03-15 10:59:27 �[44msource�[0m > Generating slices for customer 3946802903. Start date is 2023-03-12, end date is None
2023-03-15 10:59:27 �[44msource�[0m > Next slice is {'start_date': '2023-02-27', 'end_date': '2023-02-28', 'customer_id': '3946802903'}
2023-03-15 10:59:27 �[44msource�[0m > Starting a while loop iteration
2023-03-15 10:59:27 �[44msource�[0m > Read records using g-ads client. Stream slice is {'start_date': '2023-02-28', 'end_date': '2023-03-01', 'customer_id': '3946802903'}
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-02-28'}}.
2023-03-15 10:59:28 �[44msource�[0m > Current slice has been read. Exiting read_records()
2023-03-15 10:59:30 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 4000 (3 MB)
2023-03-15 10:59:32 �[44msource�[0m > Next slice is {'start_date': '2023-03-09', 'end_date': '2023-03-10', 'customer_id': '3946802903'}
2023-03-15 10:59:33 �[44msource�[0m > Starting a while loop iteration
2023-03-15 10:59:33 �[44msource�[0m > Read records using g-ads client. Stream slice is {'start_date': '2023-03-10', 'end_date': '2023-03-11', 'customer_id': '3946802903'}
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Updated state for customer 3946802903. Full state is {'6376226450': {'segments.date': '2023-03-15'}, '3946802903': {'segments.date': '2023-03-10'}}.
2023-03-15 10:59:33 �[44msource�[0m > Current slice has been read. Exiting read_records()
2023-03-15 10:59:36 �[44msource�[0m > Read 4508 records from click_view stream
2023-03-15 10:59:36 �[44msource�[0m > Finished syncing click_view
2023-03-15 10:59:36 �[44msource�[0m > SourceGoogleAds runtimes:
Syncing stream click_view 0:00:27.830990
2023-03-15 10:59:36 �[44msource�[0m > Syncing stream: campaigns 
2023-03-15 10:59:36 �[44msource�[0m > Generating slices for customer 6376226450. Start date is 2023-03-12, end date is None
2023-03-15 10:59:36 �[44msource�[0m > Next slice is {'start_date': '2023-02-27', 'end_date': '2023-03-13', 'customer_id': '6376226450'}
2023-03-15 10:59:42 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 5000 (5 MB)
2023-03-15 10:59:43 �[43mdestination�[0m > Flushing campaigns: 698 records (24 MB)
2023-03-15 10:59:44 �[43mdestination�[0m > Flushing click_view: 4508 records (24 MB)
2023-03-15 10:59:49 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 6000 (9 MB)
2023-03-15 10:59:54 �[43mdestination�[0m > Flushing campaigns: 1669 records (24 MB)
2023-03-15 10:59:54 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 7000 (12 MB)
2023-03-15 11:00:00 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 8000 (16 MB)
2023-03-15 11:00:03 �[44msource�[0m > Generating slices for customer 3946802903. Start date is 2023-03-12, end date is None
2023-03-15 11:00:03 �[44msource�[0m > Next slice is {'start_date': '2023-02-27', 'end_date': '2023-03-13', 'customer_id': '3946802903'}
2023-03-15 11:00:06 �[43mdestination�[0m > Flushing campaigns: 1672 records (24 MB)
2023-03-15 11:00:07 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):385 - Records read: 9000 (20 MB)
2023-03-15 11:00:12 �[44msource�[0m > Read 5199 records from campaigns stream
2023-03-15 11:00:12 �[44msource�[0m > Finished syncing campaigns
2023-03-15 11:00:12 �[44msource�[0m > SourceGoogleAds runtimes:
Syncing stream campaigns 0:00:36.453091
Syncing stream click_view 0:00:27.830990
2023-03-15 11:00:12 �[44msource�[0m > Finished syncing SourceGoogleAds
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):388 - Source has no more messages, closing connection.
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$7):397 - Total records read: 9747 (23 MB)
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(replicate):252 - One of source or destination thread complete. Waiting on the other.
2023-03-15 11:00:13 �[43mdestination�[0m > Airbyte message consumer: succeeded.
2023-03-15 11:00:13 �[43mdestination�[0m > executing on success close procedure.
2023-03-15 11:00:13 �[43mdestination�[0m > Flushing campaigns: 1160 records (17 MB)
2023-03-15 11:00:13 �[43mdestination�[0m > No onDestinationCloseOperations required for this destination.
2023-03-15 11:00:13 �[43mdestination�[0m > Finalizing tables in destination started for 2 streams
2023-03-15 11:00:13 �[43mdestination�[0m > Finalizing stream click_view. schema public, tmp table _airbyte_tmp_yvx_click_view, final table _airbyte_raw_click_view
2023-03-15 11:00:13 �[43mdestination�[0m > Finalizing stream campaigns. schema public, tmp table _airbyte_tmp_rfl_campaigns, final table _airbyte_raw_campaigns
2023-03-15 11:00:13 �[43mdestination�[0m > Executing finalization of tables.
2023-03-15 11:00:13 �[43mdestination�[0m > Finalizing tables in destination completed.
2023-03-15 11:00:13 �[43mdestination�[0m > Cleaning tmp tables in destination started for 2 streams
2023-03-15 11:00:13 �[43mdestination�[0m > Cleaning tmp table in destination started for stream click_view. schema public, tmp table name: _airbyte_tmp_yvx_click_view
2023-03-15 11:00:13 �[43mdestination�[0m > Cleaning tmp table in destination started for stream campaigns. schema public, tmp table name: _airbyte_tmp_rfl_campaigns
2023-03-15 11:00:13 �[43mdestination�[0m > Cleaning tmp tables in destination completed.
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromDstRunnable$4):286 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@4cd41a5c[type=STATE,log=<null>,spec=<null>,connectionStatus=<null>,catalog=<null>,record=<null>,state=io.airbyte.protocol.models.AirbyteStateMessage@4213fdb6[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@462c712c[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@32ef78d[name=campaigns,namespace=<null>,additionalProperties={}],streamState={"3946802903":{"segments.date":"2023-03-14"},"6376226450":{"segments.date":"2023-03-14"}},additionalProperties={}],global=<null>,data={"click_view":{"3946802903":{"segments.date":"2023-03-15"},"6376226450":{"segments.date":"2023-03-15"}},"campaigns":{"3946802903":{"segments.date":"2023-03-14"},"6376226450":{"segments.date":"2023-03-14"}}},additionalProperties={}],trace=<null>,control=<null>,additionalProperties={}]
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(lambda$readFromDstRunnable$4):286 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@6f440b4e[type=STATE,log=<null>,spec=<null>,connectionStatus=<null>,catalog=<null>,record=<null>,state=io.airbyte.protocol.models.AirbyteStateMessage@3267b55d[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@17e7172c[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@630108c2[name=click_view,namespace=<null>,additionalProperties={}],streamState={"3946802903":{"segments.date":"2023-03-15"},"6376226450":{"segments.date":"2023-03-15"}},additionalProperties={}],global=<null>,data={"click_view":{"3946802903":{"segments.date":"2023-03-15"},"6376226450":{"segments.date":"2023-03-15"}}},additionalProperties={}],trace=<null>,control=<null>,additionalProperties={}]
2023-03-15 11:00:13 �[33mWARN�[m i.a.w.i.b.AirbyteMessageTracker(handleDestinationEmittedState):226 - The message tracker encountered an issue that prevents committed record counts from being reliably computed.
2023-03-15 11:00:13 �[33mWARN�[m i.a.w.i.b.AirbyteMessageTracker(handleDestinationEmittedState):227 - This only impacts metadata and does not indicate a problem with actual sync data.
2023-03-15 11:00:13 �[33mWARN�[m i.a.w.i.b.AirbyteMessageTracker(handleDestinationEmittedState):228 - Delta was not stored for state hash -1372282103
io.airbyte.workers.internal.book_keeping.StateDeltaTracker$StateDeltaTrackerException: Delta was not stored for state hash -1372282103
	at io.airbyte.workers.internal.book_keeping.StateDeltaTracker.commitStateHash(StateDeltaTracker.java:126) ~[io.airbyte-airbyte-commons-worker-0.40.32.jar:?]
	at io.airbyte.workers.internal.book_keeping.AirbyteMessageTracker.handleDestinationEmittedState(AirbyteMessageTracker.java:223) ~[io.airbyte-airbyte-commons-worker-0.40.32.jar:?]
	at io.airbyte.workers.internal.book_keeping.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.java:146) ~[io.airbyte-airbyte-commons-worker-0.40.32.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.lambda$readFromDstRunnable$4(DefaultReplicationWorker.java:288) ~[io.airbyte-airbyte-commons-worker-0.40.32.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2023-03-15 11:00:13 �[43mdestination�[0m > Completed integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2023-03-15 11:00:13 �[43mdestination�[0m > completed destination: class io.airbyte.integrations.destination.postgres.PostgresDestination
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(replicate):254 - Source and destination threads complete.
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(prepStateForLaterSaving):561 - Source output at least one state message
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(prepStateForLaterSaving):567 - State capture: Updated state to: Optional[io.airbyte.config.State@1c3caf34[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"campaigns"},"stream_state":{"3946802903":{"segments.date":"2023-03-14"},"6376226450":{"segments.date":"2023-03-14"}}}},{"type":"STREAM","stream":{"stream_descriptor":{"name":"click_view"},"stream_state":{"3946802903":{"segments.date":"2023-03-15"},"6376226450":{"segments.date":"2023-03-15"}}}}]]]
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(getReplicationOutput):494 - sync summary: {
  "status" : "completed",
  "recordsSynced" : 9707,
  "bytesSynced" : 24540779,
  "startTime" : 1678877945351,
  "endTime" : 1678878013800,
  "totalStats" : {
    "bytesEmitted" : 24540779,
    "destinationStateMessagesEmitted" : 2,
    "destinationWriteEndTime" : 1678878013799,
    "destinationWriteStartTime" : 1678877945472,
    "meanSecondsBeforeSourceStateMessageEmitted" : 23,
    "maxSecondsBeforeSourceStateMessageEmitted" : 26,
    "maxSecondsBetweenStateMessageEmittedandCommitted" : 63,
    "meanSecondsBetweenStateMessageEmittedandCommitted" : 37,
    "recordsEmitted" : 9707,
    "recordsCommitted" : 9707,
    "replicationEndTime" : 1678878013800,
    "replicationStartTime" : 1678877945351,
    "sourceReadEndTime" : 1678878013182,
    "sourceReadStartTime" : 1678877945402,
    "sourceStateMessagesEmitted" : 40
  },
  "streamStats" : [ {
    "streamName" : "campaigns",
    "stats" : {
      "bytesEmitted" : 20700615,
      "recordsEmitted" : 5199,
      "recordsCommitted" : 5199
    }
  }, {
    "streamName" : "click_view",
    "stats" : {
      "bytesEmitted" : 3840164,
      "recordsEmitted" : 4508,
      "recordsCommitted" : 4508
    }
  } ]
}
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.g.DefaultReplicationWorker(getReplicationOutput):495 - failures: [ ]
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - 
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - ----- END REPLICATION -----
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - 
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):214 - sync summary: io.airbyte.config.StandardSyncOutput@56d14b9f[standardSyncSummary=io.airbyte.config.StandardSyncSummary@4048363e[status=completed,recordsSynced=9707,bytesSynced=24540779,startTime=1678877945351,endTime=1678878013800,totalStats=io.airbyte.config.SyncStats@a03d814[bytesEmitted=24540779,destinationStateMessagesEmitted=2,destinationWriteEndTime=1678878013799,destinationWriteStartTime=1678877945472,estimatedBytes=<null>,estimatedRecords=<null>,meanSecondsBeforeSourceStateMessageEmitted=23,maxSecondsBeforeSourceStateMessageEmitted=26,maxSecondsBetweenStateMessageEmittedandCommitted=63,meanSecondsBetweenStateMessageEmittedandCommitted=37,recordsEmitted=9707,recordsCommitted=9707,replicationEndTime=1678878013800,replicationStartTime=1678877945351,sourceReadEndTime=1678878013182,sourceReadStartTime=1678877945402,sourceStateMessagesEmitted=40,additionalProperties={}],streamStats=[io.airbyte.config.StreamSyncStats@549790e5[streamName=campaigns,streamNamespace=<null>,stats=io.airbyte.config.SyncStats@8c6a45b[bytesEmitted=20700615,destinationStateMessagesEmitted=<null>,destinationWriteEndTime=<null>,destinationWriteStartTime=<null>,estimatedBytes=<null>,estimatedRecords=<null>,meanSecondsBeforeSourceStateMessageEmitted=<null>,maxSecondsBeforeSourceStateMessageEmitted=<null>,maxSecondsBetweenStateMessageEmittedandCommitted=<null>,meanSecondsBetweenStateMessageEmittedandCommitted=<null>,recordsEmitted=5199,recordsCommitted=5199,replicationEndTime=<null>,replicationStartTime=<null>,sourceReadEndTime=<null>,sourceReadStartTime=<null>,sourceStateMessagesEmitted=<null>,additionalProperties={}]], io.airbyte.config.StreamSyncStats@9be8abb[streamName=click_view,streamNamespace=<null>,stats=io.airbyte.config.SyncStats@1d6930e0[bytesEmitted=3840164,destinationStateMessagesEmitted=<null>,destinationWriteEndTime=<null>,destinationWriteStartTime=<null>,estimatedBytes=<null>,estimatedRecords=<null>,meanSecondsBeforeSourceStateMessageEmitted=<null>,maxSecondsBeforeSourceStateMessageEmitted=<null>,maxSecondsBetweenStateMessageEmittedandCommitted=<null>,meanSecondsBetweenStateMessageEmittedandCommitted=<null>,recordsEmitted=4508,recordsCommitted=4508,replicationEndTime=<null>,replicationStartTime=<null>,sourceReadEndTime=<null>,sourceReadStartTime=<null>,sourceStateMessagesEmitted=<null>,additionalProperties={}]]]],normalizationSummary=<null>,webhookOperationSummary=<null>,state=io.airbyte.config.State@1c3caf34[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"campaigns"},"stream_state":{"3946802903":{"segments.date":"2023-03-14"},"6376226450":{"segments.date":"2023-03-14"}}}},{"type":"STREAM","stream":{"stream_descriptor":{"name":"click_view"},"stream_state":{"3946802903":{"segments.date":"2023-03-15"},"6376226450":{"segments.date":"2023-03-15"}}}}]],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@75919b10[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@48c81d85[stream=io.airbyte.protocol.models.AirbyteStream@35dead75[name=click_view,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"ad_group.id":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.id":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"customer.id":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"ad_group.name":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.name":{"$ref":"WellKnownTypes.json#/definitions/String"},"segments.date":{"$ref":"WellKnownTypes.json#/definitions/String"},"click_view.gclid":{"$ref":"WellKnownTypes.json#/definitions/String"},"click_view.keyword":{"$ref":"WellKnownTypes.json#/definitions/String"},"click_view.ad_group_ad":{"$ref":"WellKnownTypes.json#/definitions/String"},"segments.ad_network_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"click_view.keyword_info.text":{"$ref":"WellKnownTypes.json#/definitions/String"},"click_view.keyword_info.match_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.network_settings.target_google_search":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.network_settings.target_search_network":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.network_settings.target_content_network":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.network_settings.target_partner_search_network":{"$ref":"WellKnownTypes.json#/definitions/Boolean"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[segments.date],sourceDefinedPrimaryKey=[[click_view.gclid], [segments.date], [segments.ad_network_type]],namespace=<null>,additionalProperties={}],syncMode=incremental,cursorField=[segments.date],destinationSyncMode=append_dedup,primaryKey=[[click_view.gclid], [segments.date], [segments.ad_network_type]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@6164651e[stream=io.airbyte.protocol.models.AirbyteStream@18542b39[name=campaigns,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"campaign.id":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"metrics.ctr":{"$ref":"WellKnownTypes.json#/definitions/Number"},"campaign.name":{"$ref":"WellKnownTypes.json#/definitions/String"},"segments.date":{"$ref":"WellKnownTypes.json#/definitions/Date"},"segments.hour":{"$ref":"WellKnownTypes.json#/definitions/Number"},"metrics.clicks":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.labels":{"type":["null","array"],"items":{"$ref":"WellKnownTypes.json#/definitions/String"}},"campaign.status":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.end_date":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.manual_cpm":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.manual_cpv":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.start_date":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.target_cpm":{"$ref":"WellKnownTypes.json#/definitions/String"},"metrics.conversions":{"$ref":"WellKnownTypes.json#/definitions/Number"},"metrics.cost_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"metrics.impressions":{"$ref":"WellKnownTypes.json#/definitions/Number"},"metrics.video_views":{"$ref":"WellKnownTypes.json#/definitions/Number"},"campaign.payment_mode":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.base_campaign":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.resource_name":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.frequency_caps":{"type":["null","array"],"items":{"$ref":"WellKnownTypes.json#/definitions/String"}},"campaign.serving_status":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.campaign_budget":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.experiment_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.bidding_strategy":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.final_url_suffix":{"$ref":"WellKnownTypes.json#/definitions/String"},"metrics.conversions_value":{"$ref":"WellKnownTypes.json#/definitions/Number"},"campaign.optimization_score":{"$ref":"WellKnownTypes.json#/definitions/Number"},"campaign_budget.amount_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.bidding_strategy_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.tracking_url_template":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.url_custom_parameters":{"type":["null","array"],"items":{"$ref":"WellKnownTypes.json#/definitions/String"}},"campaign.target_roas.target_roas":{"$ref":"WellKnownTypes.json#/definitions/Number"},"metrics.video_quartile_p100_rate":{"$ref":"WellKnownTypes.json#/definitions/Number"},"campaign.advertising_channel_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.accessible_bidding_strategy":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.app_campaign_setting.app_id":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.advertising_channel_sub_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.shopping_setting.merchant_id":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.target_cpa.target_cpa_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.hotel_setting.hotel_center_id":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.shopping_setting.enable_local":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.tracking_setting.tracking_url":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.ad_serving_optimization_status":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.app_campaign_setting.app_store":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.shopping_setting.sales_country":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.video_brand_safety_suitability":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.manual_cpc.enhanced_cpc_enabled":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.target_cpa.cpc_bid_floor_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.dynamic_search_ads_setting.feeds":{"type":["null","array"],"items":{"$ref":"WellKnownTypes.json#/definitions/String"}},"campaign.percent_cpc.enhanced_cpc_enabled":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.real_time_bidding_setting.opt_in":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.target_impression_share.location":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.target_roas.cpc_bid_floor_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.target_spend.target_spend_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.vanity_pharma.vanity_pharma_text":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.commission.commission_rate_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.excluded_parent_asset_field_types":{"type":["null","array"],"items":{"$ref":"WellKnownTypes.json#/definitions/String"}},"campaign.target_cpa.cpc_bid_ceiling_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.percent_cpc.cpc_bid_ceiling_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.shopping_setting.campaign_priority":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.target_roas.cpc_bid_ceiling_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.target_spend.cpc_bid_ceiling_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.maximize_conversion_value.target_roas":{"$ref":"WellKnownTypes.json#/definitions/Number"},"campaign.network_settings.target_google_search":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.targeting_setting.target_restrictions":{"type":["null","array"],"items":{"$ref":"WellKnownTypes.json#/definitions/String"}},"campaign.dynamic_search_ads_setting.domain_name":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.maximize_conversions.target_cpa_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.network_settings.target_search_network":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.network_settings.target_content_network":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.dynamic_search_ads_setting.language_code":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.selective_optimization.conversion_actions":{"type":["null","array"],"items":{"$ref":"WellKnownTypes.json#/definitions/String"}},"campaign.local_campaign_setting.location_source_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.vanity_pharma.vanity_pharma_display_url_mode":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.network_settings.target_partner_search_network":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.target_impression_share.cpc_bid_ceiling_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.app_campaign_setting.bidding_strategy_goal_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.geo_target_type_setting.negative_geo_target_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.geo_target_type_setting.positive_geo_target_type":{"$ref":"WellKnownTypes.json#/definitions/String"},"campaign.target_impression_share.location_fraction_micros":{"$ref":"WellKnownTypes.json#/definitions/Integer"},"campaign.dynamic_search_ads_setting.use_supplied_urls_only":{"$ref":"WellKnownTypes.json#/definitions/Boolean"},"campaign.optimization_goal_setting.optimization_goal_types":{"type":["null","array"],"items":{"$ref":"WellKnownTypes.json#/definitions/String"}}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[segments.date],sourceDefinedPrimaryKey=[[campaign.id], [segments.date]],namespace=<null>,additionalProperties={}],syncMode=incremental,cursorField=[segments.date],destinationSyncMode=append_dedup,primaryKey=[[campaign.id], [segments.date]],additionalProperties={}]],additionalProperties={}],failures=[]]
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):219 - Sync summary length: 13005
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.t.TemporalUtils(withBackgroundHeartbeat):283 - Stopping temporal heartbeating...
2023-03-15 11:00:13 �[32mINFO�[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to get state
2023-03-15 11:00:13 �[32mINFO�[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to create or update state
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/28/0/logs.log
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.40.32
2023-03-15 11:00:13 �[32mINFO�[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to save workflow id for cancellation
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - 
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.n.DefaultNormalizationRunner(runProcess):125 - Running with normalization version: airbyte/normalization:0.3.0
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - ----- START DEFAULT NORMALIZATION -----
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - 
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/normalization:0.3.0 exists...
2023-03-15 11:00:13 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - airbyte/normalization:0.3.0 was found locally.
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.p.DockerProcessFactory(create):125 - Creating docker container = normalization-normalize-28-0-filnv with resources io.airbyte.config.ResourceRequirements@105fc983[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] and allowedHosts null
2023-03-15 11:00:13 �[32mINFO�[m i.a.w.p.DockerProcessFactory(create):170 - Preparing command: docker run --rm --init -i -w /data/28/0/normalize --log-driver none --name normalization-normalize-28-0-filnv --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_VERSION=0.40.32 airbyte/normalization:0.3.0 run --integration-type postgres --config destination_config.json --catalog destination_catalog.json
2023-03-15 11:00:14 �[42mnormalization�[0m > Running: transform-config --config destination_config.json --integration-type postgres --out /data/28/0/normalize
2023-03-15 11:00:14 �[42mnormalization�[0m > Namespace(config='destination_config.json', integration_type=<DestinationType.POSTGRES: 'postgres'>, out='/data/28/0/normalize')
2023-03-15 11:00:14 �[42mnormalization�[0m > transform_postgres
2023-03-15 11:00:14 �[42mnormalization�[0m > Running: transform-catalog --integration-type postgres --profile-config-dir /data/28/0/normalize --catalog destination_catalog.json --out /data/28/0/normalize/models/generated/ --json-column _airbyte_data
2023-03-15 11:00:15 �[42mnormalization�[0m > Processing destination_catalog.json...
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_google_search (#46) to campaign.network_set_.target_google_search (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_google_search (#46) to campaign.network_set_.target_google_search (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_search_network (#47) to campaign.network_set_target_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_search_network (#47) to campaign.network_set_target_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_content_network (#48) to campaign.network_set_arget_content_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_content_network (#48) to campaign.network_set_arget_content_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_partner_search_network (#55) to campaign.network_set_artner_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_partner_search_network (#55) to campaign.network_set_artner_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_spend.cpc_bid_ceiling_micros (#44) to campaign.target_spen_pc_bid_ceiling_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_spend.cpc_bid_ceiling_micros (#44) to campaign.target_spen_pc_bid_ceiling_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.maximize_conversion_value.target_roas (#46) to campaign.maximize_co_ion_value.target_roas (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.maximize_conversion_value.target_roas (#46) to campaign.maximize_co_ion_value.target_roas (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_google_search (#46) to campaign.network_set_.target_google_search (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_google_search (#46) to campaign.network_set_.target_google_search (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.targeting_setting.target_restrictions (#46) to campaign.targeting_s_g.target_restrictions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.targeting_setting.target_restrictions (#46) to campaign.targeting_s_g.target_restrictions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.domain_name (#47) to campaign.dynamic_sea_s_setting.domain_name (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.domain_name (#47) to campaign.dynamic_sea_s_setting.domain_name (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.maximize_conversions.target_cpa_micros (#47) to campaign.maximize_co_ons.target_cpa_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.maximize_conversions.target_cpa_micros (#47) to campaign.maximize_co_ons.target_cpa_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_search_network (#47) to campaign.network_set_target_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_search_network (#47) to campaign.network_set_target_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_content_network (#48) to campaign.network_set_arget_content_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_content_network (#48) to campaign.network_set_arget_content_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.language_code (#49) to campaign.dynamic_sea_setting.language_code (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.language_code (#49) to campaign.dynamic_sea_setting.language_code (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.selective_optimization.conversion_actions (#50) to campaign.selective_o_on.conversion_actions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.selective_optimization.conversion_actions (#50) to campaign.selective_o_on.conversion_actions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.local_campaign_setting.location_source_type (#52) to campaign.local_campa_.location_source_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.local_campaign_setting.location_source_type (#52) to campaign.local_campa_.location_source_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.vanity_pharma.vanity_pharma_display_url_mode (#53) to campaign.vanity_phar_arma_display_url_mode (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.vanity_pharma.vanity_pharma_display_url_mode (#53) to campaign.vanity_phar_arma_display_url_mode (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_partner_search_network (#55) to campaign.network_set_artner_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_partner_search_network (#55) to campaign.network_set_artner_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_impression_share.cpc_bid_ceiling_micros (#55) to campaign.target_impr_pc_bid_ceiling_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_impression_share.cpc_bid_ceiling_micros (#55) to campaign.target_impr_pc_bid_ceiling_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.app_campaign_setting.bidding_strategy_goal_type (#56) to campaign.app_campaig_ng_strategy_goal_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.app_campaign_setting.bidding_strategy_goal_type (#56) to campaign.app_campaig_ng_strategy_goal_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.geo_target_type_setting.negative_geo_target_type (#57) to campaign.geo_target__ative_geo_target_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.geo_target_type_setting.negative_geo_target_type (#57) to campaign.geo_target__ative_geo_target_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.geo_target_type_setting.positive_geo_target_type (#57) to campaign.geo_target__itive_geo_target_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.geo_target_type_setting.positive_geo_target_type (#57) to campaign.geo_target__itive_geo_target_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_impression_share.location_fraction_micros (#57) to campaign.target_impr_ation_fraction_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_impression_share.location_fraction_micros (#57) to campaign.target_impr_ation_fraction_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.use_supplied_urls_only (#58) to campaign.dynamic_sea_se_supplied_urls_only (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.use_supplied_urls_only (#58) to campaign.dynamic_sea_se_supplied_urls_only (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.optimization_goal_setting.optimization_goal_types (#58) to campaign.optimizatio_timization_goal_types (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.optimization_goal_setting.optimization_goal_types (#58) to campaign.optimizatio_timization_goal_types (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaigns_campaign_dynamic_search_ads_setting_feeds (#51) to campaigns_campaign_d_rch_ads_setting_feeds (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaigns_campaign_excluded_parent_asset_field_types (#52) to campaigns_campaign_e_ent_asset_field_types (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaigns_campaign_targeting_setting_target_restrictions (#56) to campaigns_campaign_t_g_target_restrictions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaigns_campaign_selective_optimization_conversion_actions (#60) to campaigns_campaign_s_on_conversion_actions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaigns_campaign_optimization_goal_setting_optimization_goal_types (#68) to campaigns_campaign_o_timization_goal_types (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_google_search (#46) to campaign.network_set_.target_google_search (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_google_search (#46) to campaign.network_set_.target_google_search (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_search_network (#47) to campaign.network_set_target_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_search_network (#47) to campaign.network_set_target_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_content_network (#48) to campaign.network_set_arget_content_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_content_network (#48) to campaign.network_set_arget_content_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_partner_search_network (#55) to campaign.network_set_artner_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_partner_search_network (#55) to campaign.network_set_artner_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_ctes/public/click_view_ab1.sql from click_view
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_ctes/public/click_view_ab2.sql from click_view
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_incremental/public/click_view_stg.sql from click_view
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_incremental/scd/public/click_view_scd.sql from click_view
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_incremental/public/click_view.sql from click_view
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_spend.cpc_bid_ceiling_micros (#44) to campaign.target_spen_pc_bid_ceiling_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_spend.cpc_bid_ceiling_micros (#44) to campaign.target_spen_pc_bid_ceiling_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.maximize_conversion_value.target_roas (#46) to campaign.maximize_co_ion_value.target_roas (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.maximize_conversion_value.target_roas (#46) to campaign.maximize_co_ion_value.target_roas (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_google_search (#46) to campaign.network_set_.target_google_search (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_google_search (#46) to campaign.network_set_.target_google_search (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.targeting_setting.target_restrictions (#46) to campaign.targeting_s_g.target_restrictions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.targeting_setting.target_restrictions (#46) to campaign.targeting_s_g.target_restrictions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.domain_name (#47) to campaign.dynamic_sea_s_setting.domain_name (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.domain_name (#47) to campaign.dynamic_sea_s_setting.domain_name (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.maximize_conversions.target_cpa_micros (#47) to campaign.maximize_co_ons.target_cpa_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.maximize_conversions.target_cpa_micros (#47) to campaign.maximize_co_ons.target_cpa_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_search_network (#47) to campaign.network_set_target_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_search_network (#47) to campaign.network_set_target_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_content_network (#48) to campaign.network_set_arget_content_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_content_network (#48) to campaign.network_set_arget_content_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.language_code (#49) to campaign.dynamic_sea_setting.language_code (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.language_code (#49) to campaign.dynamic_sea_setting.language_code (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.selective_optimization.conversion_actions (#50) to campaign.selective_o_on.conversion_actions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.selective_optimization.conversion_actions (#50) to campaign.selective_o_on.conversion_actions (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.local_campaign_setting.location_source_type (#52) to campaign.local_campa_.location_source_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.local_campaign_setting.location_source_type (#52) to campaign.local_campa_.location_source_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.vanity_pharma.vanity_pharma_display_url_mode (#53) to campaign.vanity_phar_arma_display_url_mode (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.vanity_pharma.vanity_pharma_display_url_mode (#53) to campaign.vanity_phar_arma_display_url_mode (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_partner_search_network (#55) to campaign.network_set_artner_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.network_settings.target_partner_search_network (#55) to campaign.network_set_artner_search_network (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_impression_share.cpc_bid_ceiling_micros (#55) to campaign.target_impr_pc_bid_ceiling_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_impression_share.cpc_bid_ceiling_micros (#55) to campaign.target_impr_pc_bid_ceiling_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.app_campaign_setting.bidding_strategy_goal_type (#56) to campaign.app_campaig_ng_strategy_goal_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.app_campaign_setting.bidding_strategy_goal_type (#56) to campaign.app_campaig_ng_strategy_goal_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.geo_target_type_setting.negative_geo_target_type (#57) to campaign.geo_target__ative_geo_target_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.geo_target_type_setting.negative_geo_target_type (#57) to campaign.geo_target__ative_geo_target_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.geo_target_type_setting.positive_geo_target_type (#57) to campaign.geo_target__itive_geo_target_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.geo_target_type_setting.positive_geo_target_type (#57) to campaign.geo_target__itive_geo_target_type (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_impression_share.location_fraction_micros (#57) to campaign.target_impr_ation_fraction_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.target_impression_share.location_fraction_micros (#57) to campaign.target_impr_ation_fraction_micros (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.use_supplied_urls_only (#58) to campaign.dynamic_sea_se_supplied_urls_only (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.dynamic_search_ads_setting.use_supplied_urls_only (#58) to campaign.dynamic_sea_se_supplied_urls_only (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.optimization_goal_setting.optimization_goal_types (#58) to campaign.optimizatio_timization_goal_types (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m > Truncating campaign.optimization_goal_setting.optimization_goal_types (#58) to campaign.optimizatio_timization_goal_types (#43)
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_ctes/public/campaigns_ab1.sql from campaigns
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_ctes/public/campaigns_ab2.sql from campaigns
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_incremental/public/campaigns_stg.sql from campaigns
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_incremental/scd/public/campaigns_scd.sql from campaigns
2023-03-15 11:00:15 �[42mnormalization�[0m >   Generating airbyte_incremental/public/campaigns.sql from campaigns
2023-03-15 11:00:15 �[42mnormalization�[0m >   Ignoring stream 'campaign.labels' from campaigns/campaign.labels because properties list is empty
2023-03-15 11:00:15 �[42mnormalization�[0m >   Ignoring stream 'campaign.frequency_caps' from campaigns/campaign.frequency_caps because properties list is empty
2023-03-15 11:00:15 �[42mnormalization�[0m >   Ignoring stream 'campaign.url_custom_parameters' from campaigns/campaign.url_custom_parameters because properties list is empty
2023-03-15 11:00:15 �[42mnormalization�[0m >   Ignoring stream 'campaign.dynamic_search_ads_setting.feeds' from campaigns/campaign.dynamic_search_ads_setting.feeds because properties list is empty
2023-03-15 11:00:15 �[42mnormalization�[0m >   Ignoring stream 'campaign.excluded_parent_asset_field_types' from campaigns/campaign.excluded_parent_asset_field_types because properties list is empty
2023-03-15 11:00:15 �[42mnormalization�[0m >   Ignoring stream 'campaign.targeting_setting.target_restrictions' from campaigns/campaign.targeting_setting.target_restrictions because properties list is empty
2023-03-15 11:00:15 �[42mnormalization�[0m >   Ignoring stream 'campaign.selective_optimization.conversion_actions' from campaigns/campaign.selective_optimization.conversion_actions because properties list is empty
2023-03-15 11:00:15 �[42mnormalization�[0m >   Ignoring stream 'campaign.optimization_goal_setting.optimization_goal_types' from campaigns/campaign.optimization_goal_setting.optimization_goal_types because properties list is empty
2023-03-15 11:00:15 �[42mnormalization�[0m > detected no config file for ssh, assuming ssh is off.
2023-03-15 11:00:18 �[42mnormalization�[0m >            [--event-buffer-size EVENT_BUFFER_SIZE]
2023-03-15 11:00:18 �[42mnormalization�[0m >   --event-buffer-size EVENT_BUFFER_SIZE
2023-03-15 11:00:18 �[32mINFO�[m i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 
2023-03-15 11:00:18 �[42mnormalization�[0m > DBT >=1.0.0 detected; using 10K event buffer size
2023-03-15 11:00:18 �[32mINFO�[m i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 
2023-03-15 11:00:22 �[42mnormalization�[0m > Running with dbt=1.0.0
2023-03-15 11:00:22 �[42mnormalization�[0m > Partial parse save file not found. Starting full parse.
2023-03-15 11:00:24 �[42mnormalization�[0m > [�[33mWARNING�[0m]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 2 unused configuration paths:
- models.airbyte_utils.generated.airbyte_views
- models.airbyte_utils.generated.airbyte_tables

2023-03-15 11:00:24 �[42mnormalization�[0m > Found 10 models, 0 tests, 0 snapshots, 0 analyses, 586 macros, 0 operations, 0 seed files, 2 sources, 0 exposures, 0 metrics
2023-03-15 11:00:25 �[42mnormalization�[0m > Concurrency: 8 threads (target='prod')
2023-03-15 11:00:25 �[42mnormalization�[0m > 1 of 6 START incremental model _airbyte_public.click_view_stg........................................................... [RUN]
2023-03-15 11:00:26 �[42mnormalization�[0m > 2 of 6 START incremental model _airbyte_public.campaigns_stg............................................................ [RUN]
2023-03-15 11:00:26 �[42mnormalization�[0m > 1 of 6 OK created incremental model _airbyte_public.click_view_stg...................................................... [�[32mINSERT 0 4508�[0m in 1.27s]
2023-03-15 11:00:26 �[42mnormalization�[0m > 3 of 6 START incremental model public.click_view_scd.................................................................... [RUN]
2023-03-15 11:00:27 �[42mnormalization�[0m > 11:00:27 + "postgres".public."click_view_scd"._airbyte_ab_id does not exist yet. The table will be created or rebuilt with dbt.full_refresh
2023-03-15 11:00:27 �[42mnormalization�[0m > 3 of 6 OK created incremental model public.click_view_scd............................................................... [�[32mSELECT 4509�[0m in 0.43s]
2023-03-15 11:00:27 �[42mnormalization�[0m > 4 of 6 START incremental model public.click_view........................................................................ [RUN]
2023-03-15 11:00:27 �[42mnormalization�[0m > 4 of 6 OK created incremental model public.click_view................................................................... [�[32mINSERT 0 4508�[0m in 0.18s]
2023-03-15 11:00:29 �[42mnormalization�[0m > 2 of 6 OK created incremental model _airbyte_public.campaigns_stg....................................................... [�[32mINSERT 0 5199�[0m in 3.98s]
2023-03-15 11:00:30 �[42mnormalization�[0m > 5 of 6 START incremental model public.campaigns_scd..................................................................... [RUN]
2023-03-15 11:00:30 �[42mnormalization�[0m > 11:00:30 + "postgres".public."campaigns_scd"._airbyte_ab_id does not exist yet. The table will be created or rebuilt with dbt.full_refresh
2023-03-15 11:00:30 �[42mnormalization�[0m > 5 of 6 OK created incremental model public.campaigns_scd................................................................ [�[32mSELECT 5200�[0m in 0.35s]
2023-03-15 11:00:30 �[42mnormalization�[0m > 6 of 6 START incremental model public.campaigns......................................................................... [RUN]
2023-03-15 11:00:30 �[42mnormalization�[0m > 6 of 6 OK created incremental model public.campaigns.................................................................... [�[32mINSERT 0 327�[0m in 0.25s]
2023-03-15 11:00:30 �[42mnormalization�[0m > Finished running 6 incremental models in 6.16s.
2023-03-15 11:00:30 �[42mnormalization�[0m > �[32mCompleted successfully�[0m
2023-03-15 11:00:30 �[42mnormalization�[0m > Done. PASS=6 WARN=0 ERROR=0 SKIP=0 TOTAL=6
2023-03-15 11:00:30 �[32mINFO�[m i.a.w.n.DefaultNormalizationRunner(close):190 - Terminating normalization process...
2023-03-15 11:00:30 �[32mINFO�[m i.a.w.n.DefaultNormalizationRunner(close):203 - Normalization process successfully terminated.
2023-03-15 11:00:30 �[32mINFO�[m i.a.w.g.DefaultNormalizationWorker(run):94 - Normalization executed in 17 seconds for job 28.
2023-03-15 11:00:30 �[32mINFO�[m i.a.w.g.DefaultNormalizationWorker(run):106 - Normalization summary: io.airbyte.config.NormalizationSummary@48d59611[startTime=1678878013936,endTime=1678878030942,failures=[]]
2023-03-15 11:00:30 �[32mINFO�[m i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-03-15 11:00:30 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - 
2023-03-15 11:00:30 �[32mINFO�[m i.a.c.t.TemporalUtils(withBackgroundHeartbeat):283 - Stopping temporal heartbeating...
2023-03-15 11:00:30 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 - ----- END DEFAULT NORMALIZATION -----
2023-03-15 11:00:30 �[32mINFO�[m i.a.c.i.LineGobbler(voidCall):114 -

hanslemm · 2023-03-15T11:52:46Z

It is important to refresh the source schema inside Airbyte UI after updating the connector. Otherwise, the SCD logic will remain the old one. Could you confirm that this step was done?

tigitz · 2023-03-15T11:57:56Z

Yes I did it

tigitz · 2023-03-15T12:03:09Z

@hanslemm Could you copy / paste here your generated dbt scd model please ? I don't see any change since your connector update and I'm convinced this is the file that needs the changes mentioned above

AFAIU airbyte generates a dbt project for each sync so there's no chance that the fact it didn't change could be related to a leftover of previous syncs when the connector was not in the latest version, right ?

hanslemm · 2023-03-15T12:03:28Z

Could you confirm me that the primary key of your replication setting is as follows for the campaigns:

tigitz · 2023-03-15T12:12:03Z

@hanslemm huh 🤦‍♂️ I misread and confused refreshing the schema and resetting the data

Now it makes sense and the generated scd model contains the appropriate changes.

Thanks, I wish there would be a notification or automatic process that says "Hey this connector upgrade requires a schema refresh, do it (or we did it automatically for you)"

evantahler · 2023-03-15T16:07:31Z

Glad you solved, and thanks for the assist @hanslemm!

Tagging @andyjih, who is looking into schema updates and notifying people about them

tigitz added needs-triage type/bug Something isn't working labels Dec 9, 2022

octavia-squidington-iv added team/triage autoteam community team/tse Technical Support Engineers and removed team/triage labels Dec 9, 2022

tigitz changed the title ~~Source Google Ads:~~ Source Google Ads: Normalization produce bad data since introduction of "segments.hour" in 0.2.3 Dec 9, 2022

tigitz changed the title ~~Source Google Ads: Normalization produce bad data since introduction of "segments.hour" in 0.2.3~~ Source Google Ads: Normalization produce improper data since introduction of "segments.hour" in 0.2.3 Dec 9, 2022

tigitz changed the title ~~Source Google Ads: Normalization produce improper data since introduction of "segments.hour" in 0.2.3~~ Source Google Ads: Normalization produces improper data since introduction of "segments.hour" in 0.2.3 Dec 9, 2022

tigitz mentioned this issue Dec 9, 2022

🎉 Source Google Ads: Add fields to the campaign stream 🚨 #18069

Merged

14 tasks

natalyjazzviolin added python Pull requests that update Python code connectors/source/google-ads team/connectors-python and removed needs-triage team/tse Technical Support Engineers autoteam labels Mar 10, 2023

tigitz closed this as completed Mar 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source Google Ads: Normalization produces improper data since introduction of "segments.hour" in 0.2.3 #20283

Source Google Ads: Normalization produces improper data since introduction of "segments.hour" in 0.2.3 #20283

tigitz commented Dec 9, 2022 •

edited

Loading

baudouin-notilus commented Mar 9, 2023

tigitz commented Mar 9, 2023

tigitz commented Mar 13, 2023

baudouin-notilus commented Mar 14, 2023 via email

hanslemm commented Mar 14, 2023

tigitz commented Mar 15, 2023 •

edited

Loading

hanslemm commented Mar 15, 2023 •

edited

Loading

tigitz commented Mar 15, 2023

tigitz commented Mar 15, 2023

hanslemm commented Mar 15, 2023

tigitz commented Mar 15, 2023

evantahler commented Mar 15, 2023

Source Google Ads: Normalization produces improper data since introduction of "segments.hour" in 0.2.3 #20283

Source Google Ads: Normalization produces improper data since introduction of "segments.hour" in 0.2.3 #20283

Comments

tigitz commented Dec 9, 2022 • edited Loading

Environment

Current Behavior

Expected Behavior

Hints

baudouin-notilus commented Mar 9, 2023

tigitz commented Mar 9, 2023

tigitz commented Mar 13, 2023

baudouin-notilus commented Mar 14, 2023 via email

hanslemm commented Mar 14, 2023

tigitz commented Mar 15, 2023 • edited Loading

hanslemm commented Mar 15, 2023 • edited Loading

tigitz commented Mar 15, 2023

tigitz commented Mar 15, 2023

hanslemm commented Mar 15, 2023

tigitz commented Mar 15, 2023

evantahler commented Mar 15, 2023

tigitz commented Dec 9, 2022 •

edited

Loading

tigitz commented Mar 15, 2023 •

edited

Loading

hanslemm commented Mar 15, 2023 •

edited

Loading